Am I Hacked? What should I do next?

From: kaliban (kaliban01_at_yahoo.com)
Date: 05/01/03


Date: 1 May 2003 01:54:37 -0700

I think my mail serve may have been hacked and I hope someone will
kindly look over the details and help me know if that is true or not,
and what steps I should take next.

Preface: Our organization has been steadily downsizing, which leaves
me as kind of an accidental SA. I have some basic skills, but
obviously have a long way to go. I hope some kind souls will help
point me in a forward direction. Please forgive me if inexperience
leads me to inadvertently overlook the obvious or supply some
irrelevant information; I'm not 100% sure what all applies to this
case.

Initial Symptoms: In testing our backup system, I found that
ufsrestore coredumped each time I tried to restore from our mail
server, whereas I could restore files from other servers off the same
tape with no problem. We have also been having some frequent problems
connecting to IMAP of late, seemingly at random, not sure if that's
related.

When I logged into the mail server to look around, I started receiving
a barrage of more or less blank messages from the syslogd, one every
minute or so. After searching the news groups, I determined that this
might have to do with an attempted buffer overflow hack. Apache had
quit running for some reason and I could not get it to start back up
and stay running. The syslogd seemed to be down also. I restarted it,
but the klogd seems to start ok, but immediately stops. I rebooted the
machine and the mysterious syslogd messages quit and apache stays
running, but the klogd still will not stay running. Additionally, I've
suddenly been getting quite a number of emails returned that some
spammer has sent out with one of our servers as the return address
(not sure if that is relevant or just coincidence)

Chkrootkit: I decided to install and run chkrootkit. Below are the
results. For brevity sake, I've left out of this post any normal
seeming results. I'm not sure how to proceed form here. BTW, I have
not installed any clean binaries yet for chkrootkit to use, these
results are using whatever binaries are on the machine. Also, the
machine is behind a firewall and was hardened prior to plugging it in.

Chkroot Suspicious Results:

Checking `pstree'... INFECTED
Searching for Ambient's rootkit (ark) default files and dirs...
Possible Ambient's rootkit (ark) installed
Searching for suspicious files and dirs, it may take a while...
/usr/lib/perl5/5.6.1/i386-linux/.packlist
/usr/lib/perl5/site_perl/5.6.1/i386-linux/auto/TimeDate/.packlist
/usr/lib/.ark /usr/lib/.ark?
Checking `lkm'... You have 1 process hidden for readdir command
You have 1 process hidden for ps command
Warning: Possible LKM Trojan installed

Questions:
Do the above results for sure mean I'm rootkit'd?
What should I do next?
If I am rootkit'd, is it possible to disable the hackers access while
I work on backing up the machine and planning a rebuild?

Any help would be greatly appreciated! Thanks in advance!

--ken