Re: Signature Counts between IDS's

From: Robert Graham (robert_david_graham@yahoo.com)
Date: 09/10/02


Date: Tue, 10 Sep 2002 13:34:45 -0700 (PDT)
From: Robert Graham <robert_david_graham@yahoo.com>
To: SEdwards@toplayer.com, focus-ids@securityfocus.com


--- SEdwards@toplayer.com wrote:
>
> Has anyone ever done a comparison between ISS & Snort on signature counts -
> so which product has the most sigs (and how many) - and which sigs match
> which in the two products

I have.

It depends upon how you count a signature. For example, ISS RealSecure has a
single check called "RPC_Statd_Format_String", whereas Snort has two "rules"
that check for the "statdx.c" exploit over either UDP or TCP. Is Snort twice as
good (in this one instance)?

The ISS RealSecure signature catches the statdx.c exploit over both UDP and
TCP, as well as three other well-known exploits for the same vulnerability, but
is only counted as a single "signature". Snort does not detect these other
exploits, nor would it detect a hacker who intentionally changes the statdx.c
exploit. If Snort were to attempt to detect the other exploits, the number of
"signatures" would grow. At mininum, Snort has twice as many signatures for
this one vulnerability as RealSecure, and in order to improve Snort detects to
match RealSecure's detection, the number of signatures will grow.

The underlying technology beneath Snort causes rule expansion. To detect the
same number of intrusions, Snort requires more rules than most other products.
This is neither good nor bad. In much the same the "instructions-per-clock" is
neither good nor bad for a microprocessor, but it means you can't directly use
"megahertz" when comparing and AMD Athlon vs. an Intel Pentium. You can't
directly compare the signature sets among products simply by counting them.

Somewhere on the ISS website is a whitepaper comparing the signature coverage
of just the RPC signatures between Cisco, Snort, and RealSecure. As you would
expect, RealSecure comes out on top (or the whitepaper wouldn't be on our
website), but it is educational looking at why it came out on top. In Snort's
"rpc.rules" file, it definately has more "rules" that either Cisco or
RealSecure, but falls far behind both Cisco and RealSecure in terms of security
coverage. In other words, if you are worried about RPC attacks against Unix
systems, measuring the number of RPC signatures hints at exactly the wrong
conclusion.

Actually, I wrote the paper originally comparing BlackICE vs. RealSecure vs.
Cisco vs. Snort, but RealSecure v7 is the combination fo BlackICE v3 and
RealSecure v6. The URL is:
documents.iss.net/whitepapers/RPC_Sig_Quality.pdf

Robert Graham
Chief Architect
Internet Security Systems

__________________________________________________
Yahoo! - We Remember
9-11: A tribute to the more than 3,000 lives lost
http://dir.remember.yahoo.com/tribute



Relevant Pages

  • Re: Intrusion Detection Evaluation Datasets
    ... I understand most IDS vendors do not actually use the Snort code ... SourceFire and some vendors who include Snort with hardware appliances ... of interest that one signature based IDS could detect that another ... I say attacks of interest because I am aware of some DoS ...
    (Focus-IDS)
  • RE: Signature Counts between IDSs
    ... Snort 1.8.7b121 with Analysis Console for Intrusion Detection 0.96b21 ... RealSecure top for detection ... Enterasys Networks Dragon Squire ... > Cisco Secure IDS 2.5 B ...
    (Focus-IDS)
  • Re: Snort + (OpenBSD or Linux)
    ... Snort + ... >> on the same packet. ... > 2.0 design calls for a much more streamlined detection engine, ... of your signature engine for the Prelude hybryde IDS ...
    (Focus-IDS)
  • Re: Snort + (OpenBSD or Linux)
    ... Snort + ... > Another problem that the Snort algorithm have is that it'll stop matching ... > the packet match another begnin signature (which have to be matched ... > *before* the one for the harmful attack). ...
    (Focus-IDS)
  • Re: signature based IDS/IPS effectiveness
    ... That depends greatly on the signature. ... For example, using snort it is possible to write a signature that checks first for the protocol, then the application, then the specific function and then the size of the data. ... can get superuser privileges or carry out DOS on database services. ... Mod_security is a good choice for apache, for example, and can stop db attacks before they even get to the web server ...
    (Focus-IDS)