Re: False positives, negatives and don't cares

• Previous message: Jason Haar: "Re: IDS is dead, etc"
• In reply to: Martin Roesch: "False positives, negatives and don't cares"
• Next in thread: Arian J. Evans: "Gartner is Dead, nCircle, Fusion, asset-correlation--was-->False positives, negatives and don't cares"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]

Date: Mon, 11 Aug 2003 20:57:30 -0500
To: focus-ids@securityfocus.com

--On Monday, August 11, 2003 00:12:46 -0400 Martin Roesch
<roesch@sourcefire.com> wrote:
>
> This is one of the things that frustrates me when (some) people say that
> Snort generates "tons of false positives". If you know how Snort's
> detection engine works, you know that Snort merely presents a framework
> run through a set of analysis nodes arranged in certain ways against a
> network traffic stream (or single packet). Snort doesn't have a
> hard-wired detection engine, it is defined and built at run-time based on
> the user-defined rules that are loaded into it. In this model it is very
> difficult to have false positives except in the case there are insertion
> attacks, Snort only detects what you tell it to.

Herein lies IDSes' true weakness. They're only as good as the rules that
are written for them. And BTW, it's my considered opinion that rules are
going to get harder and harder to maintain as the "body" of rules grows
ever larger. (For an analogous explanation, see my two articles - "Past
Its Prime: Is Anti-Virus Scanning Obsolete?[0] and "Life After AV: If
Anti-Virus is Obsolete, What Comes Next?[1])

I'll give you a real world example from snort:

sid 1002.
alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS (msg:"WEB-IIS
cmd.exe access"; flow:to_server,established; content:"cmd.exe"; nocase;
classtype:web-application-attack; sid:1002; rev:5;)

(I see that in rev 5 someone has finally limited the target to
$HTTP_SERVERS rather than any, which should help greatly in reducing the
FPs, but will result in FNs when someone installs an IIS server that you
don't know about. Previously the rule read $EXTERNAL_NET any -> any
$HTTP_PORTS.)

Previously this rule looked for "cmd.exe" in the content of any packet,
regardless of the purpose of the computer to which the packet was
traveling. The problem is that "cmd.exe" shows up in a *lot* of packets
that are coming from the windowsupdate site, because MS is polling for that
information. So, you get flooded with FPs and you begin to ignore the rule.

When I pointed this out on the rules list and suggested that the content
section should look for "cmd.exe?" instead, I was told that "this might
miss some legitimate infections". I'm not sure how, because I have yet to
see an exploit that doesn't supply the "?" immediately after the "cmd.exe".
It's pretty much necessary. (I suppose you could put padding between the
"cmd.exe" and the "?", just to bypass this rule, but no one has yet.)

Rather than swim upstream, I created a local rule that looks for *outgoing*
traffic only (because I only care about infections coming *from* my
network) and the content is "cmd.exe?", not "cmd.exe". I still get some
FPs, but not nearly as many. I've even asked if anyone on the list can
explain the FPs, because they appear to be just random crap stuck in
memory, but I've never gotten an answer.

I'm not complaining, mind you, I think snort is great. It's the IDS we
use. The idea that there are no FPs is simply not accurate, however.
There are.

Now, I think your idea of vulnerability correlation holds great promise to
make IDS much more useful than it already is. Pouring through 1000's of
alerts trying to figure out if the boxes they're hitting are vulnerable to
the attack detected is not my idea of fun, nor is a productive use of my
always limited time.

Paul Schmehl (pauls@utdallas.edu)
Adjunct Information Security Officer
The University of Texas at Dallas
AVIEN Founding Member
http://www.utdallas.edu

[0]<http://www.securityfocus.com/infocus/1562>
[1]<http://www.securityfocus.com/infocus/1604>

---------------------------------------------------------------------------
Captus Networks - Integrated Intrusion Prevention and Traffic Shaping
 - Instantly Stop DoS/DDoS Attacks, Worms & Port Scans
 - Automatically Control P2P, IM and Spam Traffic
 - Ensure Reliable Performance of Mission Critical Applications
Precisely Define and Implement Network Security and Performance Policies
**FREE Vulnerability Assessment Toolkit - WhitePapers - Live Demo
Visit us at: http://www.captusnetworks.com/ads/31.htm
---------------------------------------------------------------------------

• Previous message: Jason Haar: "Re: IDS is dead, etc"
• In reply to: Martin Roesch: "False positives, negatives and don't cares"
• Next in thread: Arian J. Evans: "Gartner is Dead, nCircle, Fusion, asset-correlation--was-->False positives, negatives and don't cares"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]