# Re: Crossover Error Rate (WAS "Intrusion Prevention")

From: Raistlin (raistlin@gioco.net)
Date: 12/11/02

• Next message: Chris Petersen: "RE: Intrusion Prevention"

From: "Raistlin" <raistlin@gioco.net>
To: <focus-ids@securityfocus.com>
Date: Wed, 11 Dec 2002 20:15:55 +0100

> Just as with an IDS, you can reduce
> one at the expense of increasing the other, but unlike IDS, there's a
> commonly-known standard called the CER, or "Crossover Error Rate,"

That's not indicative, really.

In evaluating a system with that metric, you are supposing that both kind of
errors are equally costly. They could not be (for example, in a biomedic
system it is FAR better to have a false alarm than a false negative !).

In addition it is not known, a priori, if the cost linearly scales. Having
10 false positives a day can be acceptable, 100 false positives may be a bit
more harassing (but not, necessarily, 10 times more), while of thousands of
false positives are completely unmanageable (they have an "infinite" cost:
we don't absolutely want to have that). At the same time, 1 false negative
may be bad, and 100 false negatives are probably in the scale of "better to
launch this crap out of the window".

Please note that all the figures are totally subjective, and here only for
the sake of an example, do not flame me on the figures :P

What you really want to build is an ROC, Receiver Operating Curve, which is
a diagram with a measure of the false positives on X axis, and a measure of
the detection rate on the other. They are in some kind of 1/x - like
relationship (the more false positives you accept, the better you find
attacks, and vice versa). A "higher" graph (A larger area under it) means a
"better" system, on the whole. But more accurately, you can match this
graph with your own "cost function" for false detections and misses, by
using really simple operational research techniques (you build the gradient
on the graph, and find the tangent with the ROC curve).

It's all theory with 40 years of background.

Stefano

## Relevant Pages

• RE: False Positives
... There isn't an IDS system that will not report "false positives" ... tools are not actually attacking but testing, and they report an attack, ... > IntruShield now offers unprecedented Intrusion IntelligenceTM ...
(Focus-IDS)
• Re: Snot/state
... but not eliminate false positives by enabling this feature. ... > maintaining what the IDS considers state, ... maybe the ultimate IDS is only going to alert me to things that I ... they handle quite a few attacks - attacks that they are well aware of. ...
(Focus-IDS)
• RE: Best Method(s) for signature verifcation.
... if the IDS is trying to be "smart" it may not listen on ports ... listening in order to get the IDS to see an attack. ... > Subject: Re: Best Methodfor signature verifcation. ... > false positives ...
(Focus-IDS)
• RE: Truth about False Positives
... Subject: Truth about False Positives ... When using any kind of IDS wether it is host or network based first thing to ... defining false positives & false alarms, and what steps we are taking to ... algorithms into having the most comprehensive set of IDS attack algorithms. ...
(Focus-IDS)
• RE: False Positives
... > when no actual exploited attack has ... > when attackers attempt to overload an IDS' alert processing ... > Subject: False Positives ... > IntruShield now offers unprecedented Intrusion IntelligenceTM ...
(Focus-IDS)