Re: Best Method(s) for signature verifcation.

From: Hervé Debar (
Date: 04/30/02

Date: Tue, 30 Apr 2002 14:20:20 +0200
From: Hervé Debar <>
To: Kurt Seifried <>

Kurt Seifried wrote:
> My preference would be to first of all fire up Nessus. If it triggers on
> Nessus that means you'll be getting a lot of false positives with it. Then I
> might sweep it with Nmap using various settings to see how well the port
> scanning can be tuned. Lastly I would hit it with the real tools like Blade
> IDS Inforner, Impact, Nexpose and of course a collection of goodies from
> packetstorm (the first three are not free, the last is).

I just cannot agree with Nessus usage. It seems to me that this is the
first (and maybe only) thing that vendors would use to test against,
to ensure that their customers are satisfied that their IDS screams
when they start Nessus.

> What you probably want to determine is:
> signature coverage (most are pretty exhasutive)
> false positives (pretty high in some products without tuning)
> how to tune it to prevent false positives
> how to tune it to flag things it doesn't explicitly know about (systems that
> use protocol analysis make this easier).

There is an additional, important thing: look at what your test tool
does in order to evaluate the response of the IDS. Does the test tool
test the existence of the vulnerability or does it perform an actual
attack. In both cases, does it succeed or fail ? Does it use well
known targets (/etc/passwd comes to mind) or less obvious attempts ?
This will give you plenty of information not only on false
positives/negatives, but also on how to interpret the alerts that your
IDS gives you.


Hervé Debar                <>
Tel: +33 (0)2 31 75 92 61               GSM: +33 (0)6 74 09 09 66
France Télécom R&D                      Fax: +33 (0)2 31 75 93 13
42 rue des Coutures  (-/-)  BP 6243  (-/-)   F-14066 Caen Cedex 4

Relevant Pages

  • Re: Rooting out false positives
    ... One of my main disappointement with the Nessus project in general is ... see how the remote server reacts. ... daemons, and our plugins found out many occurences of DoS, ... Of course, we also found some false positives that we fixed, ...
  • RE: Some Few Doubts on IIS Vuln
    ... Basically i was asking how to determine nessus results ... to be false positives or actual holes in network. ... false positives or not..but am failing to craft those ... cud actually uplod a file and delete it.So basically ...
  • Re: Vulnerability Scanning
    ... > After reviewing some scan results and finding a number of false positives from ... > This is in no way reflecting upon nessus's ability to find vulnerabilities and I ... Nessus' thousands of checks. ... many folks don't use scanners. ...
  • Re: Worm generating network attack traffic?
    ... You bring up a good point, but not all Nessus checks are ... with benign payloads and check for a known-vulnerable response. ... should be sufficient to generate an IDS alert. ... FWIW, I have found tools such as Core Impact, Metasploit, and Canvas ...
  • Re: IDS Evaluation
    ... >about the accuracy of the ids. ... Nessus has a lot of anti-ids features which still bypass some systems ... the NeWT scanner which does not have a cost for Class-C usage. ... However, when you run vuln scanners against an IDS, you only really ...