Re: use ipchains to block all ports > 60,000

From: Neil Sandow (rx_at_rxlist.com)
Date: 08/28/03


Date: Wed, 27 Aug 2003 23:17:32 GMT


Tim Haynes wrote:

> Neil Sandow <rx@rxlist.com> writes:
>
>
>>> Next question: does that persist on subsequent runs? Does the number
>>> vary, if so, by how much?
>>
>>
>> Pretty much the same every time, might vary by 1 or 2
>
>
>
> OK. So there's a consistent difference, quite a gross one. A
variation of 1
> or 2 is experimental error; in a false-positive report, I'd expect
> chkrootkit not to be able to discover between 0-2 pids, once every
> quite-a-few runs, if the box is heavily fork()ing processes left right &
> centre. A near-as-dammit-constant large-number like 27 going AWOL isn't
> cool.

Yes, but running ./chkrootkit -x lkm shows all the process id's and they
are no different than what ps shows. It's not like there are hidden
processes going on that shouldn't be running (as far as I can tell)

>
>
>>>> When I run ./chkrootkit -x lkm it shows me the processes and when
I check
>>>> the output of ps (ps -edf or ps aux) I see all the same processes
id's.
>>>> I've read that you can get false positives for short lived
processes that
>>>> may have finished while chkrootkit is running but these are all stable
>>>> running processes such as init, sshd, mingetty, etc. I'm not sure
why I'm
>>>> getting this error since ps and chkproc both show identical
process id's.
>>>
>>>
>>> How about `ls /proc/[0-9]* | wc -l' and compare that with `ps
auxww | wc
>>> -l'
>>> several times over?
>>>
>>
>> Hmmm very different.
>>
>> `ls /proc/[0-9]* | wc -l' returns 377 (every time)
>>
>>
>> `ps auxww | wc -l' returns 28 (every time)
>
>
>
> OK, I cocked-up on the first of those. Woops I should've requested:
>
> ls -1d /proc/[0-9]* | wc -l
>
> instead - a simple count of how many second-level directories comprised
> solely of pid#s there are, contrasted with how many processes are
running.
>
> Only 28 processes on the box? What is its role?

It's an LVS (director) for a web site (load balancer) As the LVS is
compiled into the kernel it doesn't even show as a process. Nothing
else going on here except sshd which allows me to log in and monitor the
load. That's all it does.

>
> Give us a `whereis ps` output as well.
>

[root@www chkrootkit]# whereis ps
ps: /bin/ps /usr/share/man/man1/ps.1.bz2

>
>>> Do both a netstat -plant | grep LISTEN
>>
>>
>> I get:
>> tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
480/sshd
>
>
>
> Just sshd? That's possibly a good thing. Now what version of ssh is that?
> (Telnet into port 22 and give us the banner string.)
>

Telnet not running but here's the ouput of ssh -V and sshd -V

OpenSSH_3.0.1p1, SSH protocols 1.5/2.0, OpenSSL 0x0090581f
sshd version OpenSSH_3.0.1p1

>
>>> and nmap your box remotely - all ports - and compare what it thinks are
>>> listening.
>>
>>
>> I'm not sure how to do that....
>
>
>
> On a remote box with nmap,
>
> nmap -sS -P0 -p1-65535 myboxip -O -o somelogfile.nmap
>
> (syn/fin scan, ignoring failure to respond to pings, all ports, your
box's
> externally-visible IP#, with OS-detection, outputs to file as well as
> stdout).
>
OK, ran that from an external box and it showed open ports 22, 80, plus
the masqueraded ports which have been set up to allow ssh into the
realservers behind the LVS. Should I be concerned that netstat -plant
  | grep listen only returned port 22?

>
>>> Also you need to monitor traffic going out - preferably by inserting a
>>> hub just adjacent to the machine with another box listening in
>>> promiscuous mode to what's going past. Also, snort would be an idea. If
>>> you can't place a sniffing box in the way, run snort on the box itself
>>> and configure it to look for port-scans and dodgy outgoing traffic.
>>
>>
>> Now I'm way over my head. Drowning......
>
>
>
>
> Your box -----------ethernet--------> outside world
>
> right?
>
> Now consider:
>
> Your box ----HUB------ethernet--------> outside world
> |
> |
> `-------ethernet---> sniffing host
>
> By temporarily breaking the network connection and inserting a hub there,
> you can plug another box into that hub that'll see all the traffic
coming &
> going with tcpdump, ethereal or snort, as long as its network device
is in
> promiscuous mode.
>
Would I have to get off my couch? :>

>
> One idea you might want to consider: get pen and paper, and list all the
> functions and services this box provides, all the critical data stored on
> it, all the tiresome configuration-tweaks present on it. List the
cron-jobs
> it runs. List the users on it and how it gets them.
> Now, in the background, you can start work on building a replacement box
> with clean, uptodate, trusted CDs offline, apply distribution's
errata, and
> be vaguely confident of being able to switch the two in the eventuality
> that you discover (a) a crack has been perpetrated or (b) you never get
> certain knowlege[0] but want a clean slate.
>

That certainly sounds reasonable!

> [0] Finding a running pid that shouldn't be, an LKM, a network socket
open
> & listening that netstat doesn't show, evidence of massive scans
emanating
> from that machine, evidence of users you know not of appearing on the
box,
> - these things are concrete. Without one or more of these, or something
> very similar, you're more or less clean. People, please provide more
clues
> to look for, by all means.
>
> ~Tim

My ISP looked for evidence of massive scans emanating from my ip address
and found none. No unknown users, no unknown pid's. I'm reluctant to
take heroic measures without, as you say, concrete evidence of a real
problem. -Neil


Quantcast