Re: Crossover Error Rate (WAS "Intrusion Prevention")

From: Raistlin (raistlin@gioco.net)
Date: 12/11/02

  • Next message: Chris Petersen: "RE: Intrusion Prevention"
    From: "Raistlin" <raistlin@gioco.net>
    To: <focus-ids@securityfocus.com>
    Date: Wed, 11 Dec 2002 20:15:55 +0100
    
    

    > Just as with an IDS, you can reduce
    > one at the expense of increasing the other, but unlike IDS, there's a
    > commonly-known standard called the CER, or "Crossover Error Rate,"

    That's not indicative, really.

    In evaluating a system with that metric, you are supposing that both kind of
    errors are equally costly. They could not be (for example, in a biomedic
    system it is FAR better to have a false alarm than a false negative !).

    In addition it is not known, a priori, if the cost linearly scales. Having
    10 false positives a day can be acceptable, 100 false positives may be a bit
    more harassing (but not, necessarily, 10 times more), while of thousands of
    false positives are completely unmanageable (they have an "infinite" cost:
    we don't absolutely want to have that). At the same time, 1 false negative
    may be bad, and 100 false negatives are probably in the scale of "better to
    launch this crap out of the window".

    Please note that all the figures are totally subjective, and here only for
    the sake of an example, do not flame me on the figures :P

    What you really want to build is an ROC, Receiver Operating Curve, which is
    a diagram with a measure of the false positives on X axis, and a measure of
    the detection rate on the other. They are in some kind of 1/x - like
    relationship (the more false positives you accept, the better you find
    attacks, and vice versa). A "higher" graph (A larger area under it) means a
    "better" system, on the whole. But more accurately, you can match this
    graph with your own "cost function" for false detections and misses, by
    using really simple operational research techniques (you build the gradient
    on the graph, and find the tangent with the ROC curve).

    It's all theory with 40 years of background.

    Stefano



    Relevant Pages

    • RE: False Positives
      ... There isn't an IDS system that will not report "false positives" ... tools are not actually attacking but testing, and they report an attack, ... > IntruShield now offers unprecedented Intrusion IntelligenceTM ...
      (Focus-IDS)
    • Re: Snot/state
      ... but not eliminate false positives by enabling this feature. ... > maintaining what the IDS considers state, ... maybe the ultimate IDS is only going to alert me to things that I ... they handle quite a few attacks - attacks that they are well aware of. ...
      (Focus-IDS)
    • RE: Best Method(s) for signature verifcation.
      ... if the IDS is trying to be "smart" it may not listen on ports ... listening in order to get the IDS to see an attack. ... > Subject: Re: Best Methodfor signature verifcation. ... > false positives ...
      (Focus-IDS)
    • RE: Truth about False Positives
      ... Subject: Truth about False Positives ... When using any kind of IDS wether it is host or network based first thing to ... defining false positives & false alarms, and what steps we are taking to ... algorithms into having the most comprehensive set of IDS attack algorithms. ...
      (Focus-IDS)
    • RE: False Positives
      ... > when no actual exploited attack has ... > when attackers attempt to overload an IDS' alert processing ... > Subject: False Positives ... > IntruShield now offers unprecedented Intrusion IntelligenceTM ...
      (Focus-IDS)