RE: Signature Counts between IDS's

From: Graham, Robert (ISS Atlanta) (rgraham@iss.net)
Date: 09/13/02 

Date: Fri, 13 Sep 2002 14:02:37 -0400
From: "Graham, Robert (ISS Atlanta)" <rgraham@iss.net>
To: "Shripal Meghani" <meghani@nsecure.net>, <focus-ids@securityfocus.com>

>From: Shripal Meghani [mailto:meghani@nsecure.net]
>If someone says signature strength does not count,

Everyone agrees "strength" matters.
Everyone is saying that "count" doesn't measure "strength".

Take for example CVE-2000-0666 (rpc.statd format string vulnerbility). RealSecure has 1 "signature" that detects all exploits against this vulnerability, Snort has 2 "signatures" that detects only a single exploit of this vulnerability. The "strength" of the RealSecure signature is greater, but the "count" is less.

>Most IDS's _ARE_ signature based IDS's

No, most are are _NOT_ signature-based IDSs. Every vendor (including Snort) claims to be more than a simple signature-based IDS. Signature-evasion techniques like ADMmutate cannot evade more than 20% of "coverage" of any IDS.

>Will the signature count affect the performance of the IDS?

No. Everyone assumes that the more signatures a product has, the slower the product will be. This isn't true in the general sense. Historically, some IDSs have had scaling problems (slower with more signatures), but by this time next year, these IDSs will have either disappeared from the market or fixed the problem. (The core issue is grep vs. fgrep: grep scales poorly as you add more patterns, the fgrep utility allows searches of thousands of simultaneous patterns with no performance loss).

>Comparisons with CVE, etc would
>definitely be desirable. But again, coverage is important.

Comparison with CVE is valuable, but likewise limited. Just because two products have signatures that reference the same CVE entry doesn't mean that the "coverage" is the same. My example above shows two products that appear on the service to cover the same CVE entry, but clearly one product has more complete "coverage" than the other.

>2) the strength of the research team the vendor has...

This is exactly our argument. The ISS X-Force team invests more effort in "signature" creation than anybody else. We don't write signatures that just target the popular exploits in the script-kiddy community, we go the extra step and write "deeper" content. Of course, most reviewers only bother with script-kiddy programs, so this never really shows up in reviews. About the only way you can really see this extra effort is the fact that we have more extensive help for our signatures than anybody else.