RE: Firewall Tester 0.6

From: Marcus J. Ranum (mjr@nfr.com)
Date: 04/11/02 

Date: Thu, 11 Apr 2002 11:48:40 -0400
To: "Steve" <steve@securesolutions.org>, "'Greg Shipley'" <gshipley@neohapsis.com>, "'Andrea Barisani'" <lcars@infis.univ.trieste.it>
From: "Marcus J. Ranum" <mjr@nfr.com>

Steve wrote:
>I agree with Greg on this point. In theory, if you replay 100 signature
>files the IDS should detect 100/100 of the "attacks". If you actually
>do the attacks themselves you are performing a true test.

It's actually worse! If you're replaying signature fragments what
you're doing is benchmarking the IDS' ability to generate false
positives!! That's _NOT_ a "feature" ;)

We had a case where a tester played snort-sig fragments at an NFR
and the NFR didn't generate a single false alarm. Why? Because
there was no actual hostile traffic. It did the right thing. A
lesser system would have generated a ton of false alarms and
irritated the heck out of the administrator, which would have been
the wrong thing to do because there was no actual attack or even
threat.

mjr. 

---
Marcus J. Ranum          Chief Technology Officer, NFR Security, Inc.
Work:                    http://www.nfr.com
Personal:                http://www.ranum.com

Relevant Pages

  • RE: Firewall Tester 0.6
    ... signature replaying attacks ... >>do the attacks themselves you are performing a true test. ... >and the NFR didn't generate a single false alarm. ...
    (Focus-IDS)
  • RE: False Positives with IntruVert
    ... Subject: False Positives with IntruVert ... a different statement than IPS is not functional or not worth time or money. ... prevent attacks, ... profiled the attacks (signature or anomaly or combination of both)) has ...
    (Focus-IDS)
  • RE: IDS detection approaches
    ... Signature based analysis on TCP and UDP payload is no longer sufficient. ... Protocol Decoding combined with signature analysis is required to detect ... many recent attacks - such as SQL injection, XSS injection, RFE, LFI, buffer ... Subject: IDS detection approaches ...
    (Focus-IDS)
  • Re: Neural Net based Host/Application Anomaly detection systems
    ... You might want to investigate NFR NID hw/sw turnkey device. ... Neural Net based Host/Application Anomaly detection systems ... > ...as that might not trigger a standard NIDS signature but seems likely to ... >> to detect previously unseen attacks. ...
    (Focus-IDS)
  • Re: Testing IDS with tcpreplay
    ... two different IPS' is to use replay tools. ... why is that harder to accomplish with Metasploit than with tcpreplay? ... Also what about attacks that Metasploit ... Just capture each of those 500 attacks and replay each of them against ...
    (Focus-IDS)