RE: How do behavioral/anomaly detection systems learn?

From: Teicher, Mark (Mark) (teicher_at_avaya.com)
Date: 02/08/04

  • Next message: Teicher, Mark (Mark): "RE: How do behavioral/anomaly detection systems learn?"
    Date: Sun, 8 Feb 2004 10:19:32 -0700
    To: "david maynor" <david.maynor@oit.gatech.edu>, "Sasha Romanosky" <sasha_romanosky@yahoo.com>
    
    

    Theoretically based on your comments, what products would apply to your
    theory ?/

    /m

    -----Original Message-----
    From: david maynor [mailto:david.maynor@oit.gatech.edu]
    Sent: Thursday, February 05, 2004 7:44 AM
    To: Sasha Romanosky
    Cc: focus-ids@securityfocus.com
    Subject: Re: How do behavioral/anomaly detection systems learn?

    Depending on the system it can widely vary. Most of these system create
    a baseline of network traffic and flag on behavior that widely varies
    from the baseline. This is not the only method, many systems include
    protocol analysis and rfc compliance. An example of protocol analysis is
    checking for encrypted tunnels over port 80 by the amount of traffic
    transfered with out valid HTTP traffic.

    Your question is more about how they learn. There are two answers to
    this and neither of them are pretty. One is manual. This means after a
    certain number of false positives (like a user running an application
    that was present during the baseline) you would add the traffic pattern
    to the profile by hand. As everyone knows this is not effective for
    anything larger than a class C. The second way is through an automated
    process where a threshold and a time value are set and after a certain
    amount of time the abnormal traffic behavior becomes part of the
    offending hosts profile. This means in the future similar traffic will
    not cause and alarm. There are some provisions in the systems to alert
    on know bad traffic patterns like fileswapping but you the effectiveness
    of the device is limited at this point.

    There are attacks you can do against such a system like a "low slow"
    attack where someone could do whatever they as long as it is rate
    limited. Another example is someone who spends the time to "teach" the
    system bad habits.

    Simple thing like this are why such systems should be used in
    conjunction with signature based systems. The ideal product would have a
    hybrid of both.

    On Thu, 2004-02-05 at 01:18, Sasha Romanosky wrote:
    > Greetings,
    >
    > In regards to "behavioral" or "anomaly" detection systems vs. pure
    > signature-based detection systems, I'm trying to understand how these
    > behavioral technologies differentiate "good" traffic from "bad"
    traffic.
    > I don't want to get into which is better, because they both have their
    > place, of course. What I'm trying to understand is how these
    behavioral
    > systems work, or "learn".
    >
    > I have seen that this technique is not unique to intrusion detection
    > systems, but also appears in application firewalls (e.g. Teros) and
    > email virus scanners (e.g. using bayesian filtering).
    >
    > With some products, I see that you configure them with specific rules,
    > tailored to your particular environment, and with other products, you
    > just point it to the network and it creates a profile all by itself.
    >
    > Does this simply amount to another form of signature system, just with
    > more intelligent signatures? Or is it more complex than this?.
    >
    > Any references (whitepapers, archives, sites, etc) explaining this
    > learning would be most appreciated.
    >
    >
    > Cheers,
    > Sasha Romanosky
    >
    >
    >
    ------------------------------------------------------------------------

    ---
    >
    ------------------------------------------------------------------------
    ---
    > 
    ------------------------------------------------------------------------
    ---
    ------------------------------------------------------------------------
    ---
    ---------------------------------------------------------------------------
    ---------------------------------------------------------------------------
    

  • Next message: Teicher, Mark (Mark): "RE: How do behavioral/anomaly detection systems learn?"

    Relevant Pages

    • Re: How do behavioral/anomaly detection systems learn?
      ... a baseline of network traffic and flag on behavior that widely varies ... checking for encrypted tunnels over port 80 by the amount of traffic ... offending hosts profile. ... conjunction with signature based systems. ...
      (Focus-IDS)
    • Re: How do behavioral/anomaly detection systems learn?
      ... Thus, the tuning process not only includes establishing a baseline, but also identifying existing problems on the network so that they can be dealt with and not "tuned-in". ... How do behavioral/anomaly detection systems learn? ... >Does this simply amount to another form of signature system, ... >Sasha Romanosky ...
      (Focus-IDS)