Re: Linux hacked

From: Miles Stevenson (miles_at_mstevenson.org)
Date: 10/21/04

  • Next message: Martin Mewes: "Re: breakout of citrix" 
    To: security-basics@securityfocus.com
Date: Thu, 21 Oct 2004 13:16:52 -0400

    Dale,

    First of all, I'd like to point out that you are asking all the right
    questions, and that I'm impressed by how far you've come without having any
    sysadmin experience.

    Contrary to the advice that you have been given thus far, I'm hoping that you
    have not interacted with the system at all so far, aside from unplugging it
    from the network and/or shutting it down. If this is the case, then don't.
    The first thing you want to do is take a forensically sound "image" of your
    system, from which you can work. This way, you can work from the image, and
    not the real system in trying to determine what happened and how you were
    attacked. I think the best approach, is to boot your system with a separate
    bootable CD, such as Knoppix STD, Phlak, or another forensics-focused
    bootable linux OS. After you boot up into the OS running from CD, you can
    connect the system back to your internal network. You can then use the dd and
    netcat utilities, to take a perfect forensic snapshot of your system, and
    send that snapshot to another system on the network.

    Instead of explaining how to do this, I will point you to another resource in
    order to save space:
    http://www.rajeevnet.com/hacks_hints/os_clone/os_cloning.html

    Once you have a forensic copy of your system, you can now safely continue your
    investigation of what went wrong and why. You can also choose to completely
    wipe and rebuild your system if that is the most appropriate course of action
    for you, and you decide to investigate later. But, the longer you wait to
    perform an investigation, the more difficult that investigation is going to
    be. Choose carefully.

    The most important thing for you to keep in mind here, is that once your
    system has been compromised, you can *no longer trust ANY of the data on your
    system*. Netstat might lie to you. Your kernel might lie to you. In essence,
    the attacker could have made any alterations to your systems to change the
    way it behaves or what it reports to you. You can't trust the logs, you can't
    even trust the output of the commands. This is why you have to run these
    tools from a separate, TRUSTED source, such as from a read-only forensic CD
    like Phlak. Don't trust the "ls" command on the hacked system, but DO trust
    the "ls" command on your forensics disk. This is VERY important.

    This process is going to get more and more complicated as you continue, and is
    best handled by someone with experience. If you can get to this point, and
    then hand things over to someone else, I recommend it. If you are unable to
    do that, then I am willing to help you as much as I can. But I think you
    should first get to this point of taking a forensic snapshot of your system,
    and obtaining a bootable forensic cd (I personally like Phlak, but there are
    many others) that you can use as a tool. Once you get to this point, let me
    know your situation, and we can continue. If I cover too much right now, not
    only will I run the risk of "information overload", but I also have to start
    making assumptions about your system in order to recommend how to proceed,
    and these assumptions can be disasterous, even when made by those of us that
    know what we are doing. You can contact me off-list if you prefer.

    Good luck.

    On Wednesday 20 October 2004 12:52 pm, Nicholson, Dale wrote:
    > First let me say I'm a security novice. Please bear with me.
    >
    > My home linux (gentoo) machine was hacked last Thursday. Installed active
    > on the box was ssh, apache, php 5, and a squirl mail. Iptables was set up
    > for a firewall. The box was set up as a web server with a number of
    > websites and about 35 email accounts (separate passwords for the mail than
    > the user accounts on the box).
    >
    > I'm guessing it was some sort of script kiddie if the names taking credit
    > for the hack in the hidden folders I found are any indication. I did some
    > research on the person taking credit and found all kinds of information on
    > him, he's an 18yr old kid in Germany. I doubt he is very knowledgeable or
    > he would not have alerted me to the intrusion by somehow locking out all
    > accounts from the machine.
    >
    > To get in I have to boot from cd and chroot in. Everything I've tried has
    > been unsuccessful in getting root back.
    >
    > I found a hidden directory /var/tmp/.tmp that has a bunch of directories
    > under it with names like +_01_+++++++HaXorEd by ... and
    > +_05_++++++++++Movies++++++....
    >
    > I unplugged the machine from the internet shortly after the hack and can
    > find no evidence of any uploads. I do see that the person somehow was able
    > to break root. I was only able to find the hidden directories because the
    > person forgot to clean up root's history file where I found the command
    > used to create the them. The box was set up to not allow remote login of
    > root via ssh but you could su in once logged in as one of three users.
    >
    > I'm a novice at security and had been depending on my system admin to keep
    > the box up to date. He tells me he's been doing an emerge world every week
    > but I don't know how to tell.
    >
    > Can someone help me with where to get a listing of everything I have
    > installed and the versions? I can't remember if the kernel is a 2.4 or 2.6
    > but I think it's 2.6. Plus I know there have been problems with ssh in the
    > past but I don't know which versions have problems and I'm not sure how to
    > find out what version I'm running. I'm kind of stuck as my sys-admin
    > normally handles these things but he cannot ssh in to the box without me
    > first fixing the problem since he lives 13 hours from me (the box is in my
    > basement).
    >
    > Also, I need something that can detect root kits etc. on linux. I've heard
    > knoppix mentioned as having good tools on this list for an example, but I
    > wouldn't know what tools to use for this particular case.
    >
    > This is what I tried so far:
    > I logged in using a boot CD, mounted the hard disks, chrooted in, blanked
    > out the root password in the /etc/shadow file, changed the root password,
    > rebooted and tried to log in normally. This did not work. I also checked
    > that the correct users were in both /etc/passwd and /etc/shadow.
    >
    > Note that both the email and websites were still working despite not being
    > able to log in, although not now of course since I unplugged the ethernet
    > cable.
    >
    > Any comments/assistance will be greatly appreciated. 

    -- 
Miles Stevenson
miles@mstevenson.org
PGP FP: 035F 7D40 44A9 28FA 7453 BDF4 329F 889D 767D 2F63

    • application/pgp-signature attachment: stored
  • Next message: Martin Mewes: "Re: breakout of citrix"

    Relevant Pages

    • Re: Rootkit
      ... You can trust the results if you reboot your system from a CD, ... forensics that way, but you won't see the hacked network stuff since the ... Unplug the suspect machine from your network, ...
      (Fedora)
    • Re: HELP! Ive been had! Someone hacked into my Linux box. What now?
      ... I have since closed my router's ssh virtual server's redirect to it. ... I cannot really do a private/public key since I need ... You can do your forensics on the hacked box later. ... public network. ...
      (comp.os.linux.security)
    • Re: NIS/NFS question
      ... it's trust is still only based on ip. ... >add some sort of certificate trust to nis or some other mechanism to ... How about not allowing anyone to login as root from ssh? ...
      (RedHat)