RE: Signature Counts between IDS's

From: Greg Shipley (gshipley@neohapsis.com)
Date: 09/13/02


Date: Thu, 12 Sep 2002 18:04:13 -0500 (CDT)
From: Greg Shipley <gshipley@neohapsis.com>
To: "Palmer, Paul (ISSAtlanta)" <PPalmer@iss.net>


On Tue, 10 Sep 2002, Palmer, Paul (ISSAtlanta) wrote:

> The problem with measuring IDS products by signature count is that the
> information has such a short half-life. That Network Computing article
> is a case in point. It is only a little more than a year old and the
> figures are now virtually useless. ISS RealSecure 5.5 is not even
> supported by ISS any longer.

Totally correct, IMHO.

> Furthermore, it was not even the current release when the article was
> written.

Totally wrong, IMNSHO.

This drives me nuts. Guys, articles are NOT written one day before the
magazine ships. Most articles are written, in fact, MONTHS before the
magazine date. Most of us writers need to turn things in 30 days in
advance - AT LEAST - and we need to finish testing before that. That
particular Network Computing article was SIX MONTHS in the making. So
when we were doing the testing, research, and writing, the products were
ABSOLUTELY updated to the "current release."

> There is also the problem that signature counts are very easy for
> vendors to manipulate. So if you are using signature counts as the basis
> for a decision you may be disappointed in the results.

Agreed - raw signature counts are not necessarily a good way to measure
thoroughness, particularly as some products have many signatures to
monitor a single attack type...and others don't.

Apples and oranges.

My .02,

-Greg