RE: Systems compromised with ShellBOT perl script - part 2

From: KEM Hosting (security_at_kemhosting.com)
Date: 10/20/04

  • Next message: Jim Halfpenny: "re: Systems compromised with ShellBOT perl script - part 2" 
    To: <incidents@securityfocus.com>
Date: Wed, 20 Oct 2004 14:13:06 -0500

    Thanks for the Perl/noexec explanation - makes a lot more sense now! And
    yes, looking in /var/tmp/ uncovered several hacker files, though none of
    them recent. Maybe I should mount that noexec as well?

    Going further...
    For the actual Perl script that was the culprit, plus another version I
    found in my /var/tmp (looks like the same thing, but in English vs.
    Portuguese) grab this:
    http://www.kemhosting.com/hacks/scripts.zip

    Looking through /var/log/messages, I see this:

    Oct 19 17:00:42 kemhosting httpd: httpd shutdown succeeded
    Oct 19 17:00:43 kemhosting httpd: (98)Address already in use: make_sock:
    could not bind to address 0.0.0.0:443
    Oct 19 17:00:43 kemhosting httpd: no listening sockets available, shutting
    down
    Oct 19 17:00:43 kemhosting httpd: Unable to open logs!
    Oct 19 17:00:43 kemhosting httpd: httpd startup failed
    Oct 19 17:05:12 kemhosting last message repeated 2 times
    Oct 19 17:05:40 kemhosting httpd: httpd shutdown failed
    Oct 19 17:05:41 kemhosting httpd: (98)Address already in use: make_sock:
    could not bind to address 0.0.0.0:443
    Oct 19 17:05:41 kemhosting httpd: no listening sockets available, shutting
    down
    Oct 19 17:05:41 kemhosting httpd: Unable to open logs!
    Oct 19 17:05:41 kemhosting httpd: httpd startup failed

    I have a service (SIM) that monitors httpd and restarts when needed.
    Because the hacker's script was apparently blocking the needed port, the log
    messages above repeat several more times until I logged onto the server and
    killed the process.

    As far as tracking where/how exactly it came in, I would assume through
    Apache since it was owned by user Apache, but I can't be sure because the
    process went through all my logs and (probably) edited them. You can see
    this in the output of the lsof I ran before I killed the process:

    /usr/sbin/lsof -p 5575
    COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
    perl 5575 apache cwd DIR 3,3 4096 2 /
    perl 5575 apache rtd DIR 3,3 4096 2 /
    perl 5575 apache txt REG 3,3 12700 1130763 /usr/bin/perl
    perl 5575 apache mem REG 3,3 63577 5308429
    /usr/lib/perl5/5.8.0/i386-linux-thread-multi/auto/Socket/Socket.so
    perl 5575 apache mem REG 3,3 56843 671758
    /usr/lib/perl5/5.8.0/i386-linux-thread-multi/auto/IO/IO.so
    perl 5575 apache mem REG 3,3 32148976 3276805
    /usr/lib/locale/locale-archive
    perl 5575 apache mem REG 3,3 89640 524374
    /lib/ld-2.3.2.so
    perl 5575 apache mem REG 3,3 2524941 5357601
    /usr/lib/perl5/5.8.0/i386-linux-thread-multi/CORE/libperl.so
    perl 5575 apache mem REG 3,3 82564 524308
    /lib/libnsl-2.3.2.so
    perl 5575 apache mem REG 3,3 14284 524304
    /lib/libdl-2.3.2.so
    perl 5575 apache mem REG 3,3 209204 524306
    /lib/libm-2.3.2.so
    perl 5575 apache mem REG 3,3 99468 524334
    /lib/libpthread-0.10.so
    perl 5575 apache mem REG 3,3 1463404 524300
    /lib/libc-2.3.2.so
    perl 5575 apache mem REG 3,3 22004 524302
    /lib/libcrypt-2.3.2.so
    perl 5575 apache mem REG 3,3 12020 524342
    /lib/libutil-2.3.2.so
    perl 5575 apache mem REG 3,3 47644 524324
    /lib/libnss_files-2.3.2.so
    perl 5575 apache mem REG 3,3 17796 524321
    /lib/libnss_dns-2.3.2.so
    perl 5575 apache mem REG 3,3 68220 524336
    /lib/libresolv-2.3.2.so
    perl 5575 apache 0r CHR 1,3 67054 /dev/null
    perl 5575 apache 1w REG 7,0 0 45 /tmp/cmdtemp
    (deleted)
    perl 5575 apache 2w REG 7,0 0 45 /tmp/cmdtemp
    (deleted)
    perl 5575 apache 3u IPv4 129962344 TCP *:http
    (LISTEN)
    perl 5575 apache 4u IPv4 129962346 TCP *:https
    (LISTEN)
    perl 5575 apache 5r FIFO 0,5 129962450 pipe
    perl 5575 apache 6w FIFO 0,5 129962450 pipe
    perl 5575 apache 7w REG 3,3 77778 7667915
    /var/log/httpd/error_log.2
    perl 5575 apache 8w REG 3,3 33070 3817764
    /home/httpd/vhosts/VIRTUALHOST/statistics/logs/error_log
    perl 5575 apache 9w REG 3,3 162 5374200
    /home/httpd/vhosts/VIRTUALHOST/statistics/logs/error_ssl_log
    ...
    (Continues to go through ea. Virtual host's logs in the vhosts directory)
    ....
    perl 5575 apache 43w REG 3,3 2268 7667913
    /var/log/httpd/ssl_error_log.2
    perl 5575 apache 44w REG 3,3 1618129 7667939
    /var/log/httpd/access_log.2
    perl 5575 apache 45w REG 3,3 22302 3817765
    /var/log/httpd/ssl_access_log
    perl 5575 apache 82w REG 3,3 0 7667929
    /var/log/httpd/ssl_request_log
    perl 5575 apache 83w REG 3,3 0 7667733
    /var/log/httpd/ssl_mutex.19103 (deleted)
    perl 5575 apache 84u sock 0,0 144348342 can't identify
    protocol
    perl 5575 apache 85r REG 3,3 4478 3358938
    /home/httpd/vhosts/VIRTUALHOST/httpdocs/index.php~ (deleted)
    perl 5575 apache 86u unix 0xe764d400 144348343 socket
    perl 5575 apache 87u IPv4 181431706 TCP

    ed

    -----Original Message-----
    From: Stephen J. Smoogen [mailto:smooge@gmail.com]
    Sent: Wednesday, October 20, 2004 12:02 PM
    To: security@kemhosting.com
    Cc: incidents@securityfocus.com
    Subject: Re: Systems compromised with ShellBOT perl script - part 2

    On Wed, 20 Oct 2004 00:04:36 -0500, security@kemhosting.com
    <security@kemhosting.com> wrote:
    > This thread is a couple months old, but I'm having issues with this hack,
    found
    > it in the archives and thought it'd be helpful if I 'resusitated' it. See
    > bottom of email for rest of thread.
    >
    > Today, hackers used the ShellBOT perl script to bring down Apache and
    start up
    > their IRC listener. They (somehow) copied it into /tmp and executed it.
    This
    > confuses me because I have my /tmp directory mounted rw,noexec,nosuid.
    Does
    > Perl somehow bypass this?
    >
    > While the script was running, I ran lsof and found that it had recursively
    > accessed all my (virtual host) httpd logs (probably in an attempt to
    delete
    > it's tracks = the reason I can't see how they copied the script into /tmp)
    > which are owned by root. this is also confusing since the process the
    script
    > spawned was owned by user apache.
    >
    > Some info on my box:
    > Redhat ES kernel 2.4.21-9.0.1.ELsmp
    > httpd-2.0.46-32.ent
    > php-4.3.2-11.ent
    >
    > Anyone have any ideas on how this can happen? Mainly the executing of a
    script
    > on a noexec mount! Obviously I'm not a guru, so it's probably something
    simple
    > - so please, share!
    >

    I think you may have other issues. As of 2004/10/18, the packages for
    Red Hat Enterprise are the following:

    kernel-2.4.21-20.EL
    php-4.3.2-14.ent
    httpd-2.0.46-40.ent
    perl-5.8.0-88.7

    There have been several security updates to php and httpd since the
    versions you have installed. The attackers may have been able to use
    this to say upload items into /var/tmp and execute it there. If you no
    longer have a Red Hat Enterprise License you should be able to get
    updates from WhiteBox and several other of the RHEL clones.

    /var/tmp gets overlooked a lot. I have also seen some webcoders use it
    as a default for their scripts because /tmp is too small/restricted
    for some reason. I would check to make sure that none of the
    PHP/perl/etc are defaulting to using /tmp as their "temp space" as
    that would avoid the noexec,nosuid.

    I would also check that some other listening service (ssh, etc) wasnt
    initially used to get the compromise and they havent installed a
    kernel root kit that silently ignores noexec,nosuid when it mounts
    disks. That would be what I would do to make sure I could come back in
    later :). 

    -- 
Stephen J Smoogen.
CSIRT/Linux System Administrator
  • Next message: Jim Halfpenny: "re: Systems compromised with ShellBOT perl script - part 2"