detecting network crowd surges

I'm curious to get some feedback on detecting zombie networks and
such by looking at common unique destination IP/port combinations
for control and "phone home" traffic.

The idea is to watch a large population of "good guys" like all
of the user IPs on an ISP's cable modem network or all of the IPs at
a university and detect when ~100 or more all go to IRC, an FTP
site, SSH, .etc all in the same time frame.

We've written some correlation rules for our log analysis products
to do this in realtime with firewall, network, ids, netflow, .etc
traffic, and are getting all sorts of results. I have a blog entry
on it (including some screen shots) at:

http://blog.tenablesecurity.com/2006/08/detecting_crowd.html

Sometimes the results are very conclusive, such as ~50 different IPs
all checking into IRC at a certain time or all SSHing into an IP
address for a second or so.

We've also been able to discriminate this sort of activity on web/ssl
traffic by changing some of the thresholds. Occasionally, you can see
false positives such as everyone hitting Google or MySpace in a short
amount of time. Also, some P2P apps, Skype and others do seem to behave
in this sort of 'surge' manner.

Most of the operational stuff I've run across for detecting botnets
is either looking at inbound/outbound IDS alerts or running a
honeypot. I think those approaches just skim the surface of all the
different ways to manage a botnet. A good paper on a broader approach
is:

http://www.eecs.umich.edu/~emcooke/pubs/botnets-sruti05.pdf

I'm curious operationally, what other people are detecting. We all
run NIDS, SIMS and NBAD products right? What happens to your logs
when someone fires up bittorrent, emule, skype, tor, .etc and what
happens when you have a real botnet?

Ron Gula, CTO
Tenable Network Security
http://www.nessus.org
http://www.tenablesecurity.com
http://blog.tenablesecurity.com


------------------------------------------------------------------------
Test Your IDS

Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 to learn more.
------------------------------------------------------------------------

Relevant Pages

  • RE: IDS and Spywares
    ... Network based detection is able to deal ... hIDS/hIPS ar much more effective in detecting and preventing these attacks. ... malware with a network based IDS or IPS. ... I think this is what Dhruv meant. ...
    (Focus-IDS)
  • RE: IDS and Spywares
    ... no 100% fool proof method for detecting anything. ... Subject: IDS and Spywares ... Spyware detection through any ... > detected by an antivirus system and not by a network ...
    (Focus-IDS)
  • Re: IDS and NMS
    ... Start by designing and installing a network. ... Next, a more detailed view of the network is required, so a NMS is ... the network administrator wants to see what ... This is where integrating the IDS console into the NMS makes sense. ...
    (Focus-IDS)
  • Re: "false positive" inanity
    ... So Mr. Snyder is asking for an IDS that does not need to be configured? ... maximum control of his/her network. ... attack. ... > assuming that it is not an intrusion. ...
    (Focus-IDS)
  • Re: Secure Network Design (DMZ, LAN, etc)
    ... I'd like one outside the firewall and one ... I assumed I could make the first IDS ... should I have the IDS listening on the 192.168.1.0/24 network as well (web ... >Since the whole world will need access to your web servers, ...
    (Security-Basics)