RE: Specification-based Anomaly Detection

From: Ofer Shezaf (Ofer.Shezaf_at_breach.com)
Date: 01/14/05

  • Next message: Kohlenberg, Toby: "RE: Specification-based Anomaly Detection"
    Date: Thu, 13 Jan 2005 22:19:54 -0500
    To: "Stefano Zanero" <zanero@elet.polimi.it>, "Kohlenberg, Toby" <toby.kohlenberg@intel.com>
    
    

    Hi Stefano & Toby,

    I'll refrain from in-lining as it becomes cluttered down there, and I
    feel that everybody is saying the same thing just with different
    emphasis and a lot of arguments about wording (got to lawyers level)....
    And I swear not to use "actionable" in non-commercial situations.

    So I'll try to roll the discussion forward:

    Stefano writes that a host and a port define a listening application and
    then you carry on about detecting an application automatically. While
    mail is an application, it is a static application in the sense that it
    behaves very much the same over time and between users and sessions (and
    so does FTP and so on).

    I feel that the mind set of the discussion was about such applications,
    and that an IDS system (signatures or anomaly based) for such protocols
    would not be much different than a network IDS.

    I would like to call the types of applications that my company's
    products handle "dynamic applications". I'm referring to interactive
    http based applications. Why are they different? For many reasons, only
    some of them directly related IDS, but all have security relevance:

    1. Only widely used system that allows a very large community to write
    client server applications (hence the tons of poor coding).
    2. Protocol elements are polymorphic, not just the content, and are
    changed by the above "programmers".
    3. Only widely used system where code is constantly downloaded by the
    user.

    And as a result, a lot more action...

    Does this make intrusion detection in web applications deferent? Based
    on our experience with out product I think so.

    Why?

    Probably because the balance between know how and mathematical analysis
    is different. When I think of it, our product includes a lot of implicit
    know-how about http, html and how different application environment use
    it. We don't have to apply abnormal behavior algorithms to a steam of
    information but to clearly identified attributes of transactions that we
    know quite a lot about.

    In some ways this is more similar to HIDS than to NIDS (And by the way,
    we also passively decrypt SSL - if we get the key - so even less
    difference than a host IDS).

    Another issue evolved around my assertion that the protocol is
    polymorphic. When I stated in a previous e-mail that we learn the
    application behavior and not the user behavior, I referred, in terms
    more commonly used in IDS that we learn the protocol. As the protocol is
    defined by the specific programmer at the organization building the web
    site, learning it and validating that users are in conformance provides
    a layer of security that I'm not sure should be called abnormal behavior
    detection in the common IDS terminology.

    ~ Ofer

    Ofer Shezaf
    CTO, Breach Security

    Tel: +972.9.956.0036 ext.212
    Cell: +972.54.443.1119
    ofers@breach.com
    http://www.breach.com

    > -----Original Message-----
    > From: Stefano Zanero [mailto:zanero@elet.polimi.it]
    > Sent: Tuesday, January 11, 2005 11:29 AM
    > To: Kohlenberg, Toby
    > Cc: Ofer Shezaf; focus-ids@lists.securityfocus.com
    > Subject: Re: Specification-based Anomaly Detection
    >
    > Kohlenberg, Toby wrote:
    >
    > > Stefano, could you expand on which part you agree with? I'm really
    > > confused to think that you would agree that anomaly detection would
    > > be new to IDS.
    >
    > I would agree that:
    > - anomaly detection is needed as a complementary approach to misuse
    > detection because of the inherent limits of the latter
    > - and that anomaly detection (in particular techniques which are not
    > rate-based) is a relative "newcomer" in the COMMERCIAL field of
    > intrusion detection, where most of the products are built on a misuse
    > detection approach.
    >
    >
    > >>>is zero day
    >
    > >>Or highly polimorph attacks, yes.
    >
    > > Or custom-written attacks
    >
    > Absolutely correct !
    >
    > > Really? What about apps that all tunnel over a single port?
    >
    > That would be a problem even if you work at application layer ;)
    >
    > Please note that Ofer was not advocating HOST-based intrusion
    detection
    > but NETWORK-based approaches working at layer 7
    >
    > > Are you getting the application that IANA says runs on that port or
    > > are you getting SAP using telnet on some random port or Cisco using
    > > HTTP on yet another random port?
    >
    > That's something that the algorithm we have developed can recognize ;)
    >
    > >>This is basic misuse detection, it does not mean you can deliver an
    > >>actionable anomaly detection result.
    > >
    > > No, but it does give you a much better chance of finding
    "actionable"
    > > (or ignorable)
    >
    > Yes, but since we are discussing wether or not ANOMALY detection is
    > "actionable" (I'm not a native speaker but this word sounds horrible
    to
    > me :) this objection is not relevant. Or better, it says exactly what
    > Tom and I were saying: anomaly detection is not, and this is a
    > disadvantage wrt misuse detection.
    >
    > Best,
    > Stefano
    >
    >
    ------------------------------------------------------------------------

    --
    > Test Your IDS
    > 
    > Is your IDS deployed correctly?
    > Find out quickly and easily by testing it with real-world attacks from
    > CORE IMPACT.
    > Go to
    http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
    > to learn more.
    >
    ------------------------------------------------------------------------
    --
    > 
    --------------------------------------------------------------------------
    Test Your IDS
    Is your IDS deployed correctly?
    Find out quickly and easily by testing it with real-world attacks from 
    CORE IMPACT.
    Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
    to learn more.
    --------------------------------------------------------------------------
    

  • Next message: Kohlenberg, Toby: "RE: Specification-based Anomaly Detection"

    Relevant Pages

    • RE: Changes in IDS Companies?
      ... It does intrusion detection with alerting and pattern matching ... IDS is down...but at least your network isn't, ... ::: mode being rolled into Snort) are both good technologies ...
      (Focus-IDS)
    • Re: Alarming (was protocol analysis)
      ... Obviously, there are different ways to "detect" attacks, but John uses the ... no one should ever "rely" on any IDS for our ... As for Johns Metaphor of the motion sensor vs the pressure sensor, ... toward Intrusion Prevention as opposed to just Intrusion Detection. ...
      (Focus-IDS)
    • IDS Assessment (was: Intrusion Prevention... probably something else at one point)
      ... scrutiny of all IDS features/technologies. ... Anomaly-type detection engines can ... weaknesses of each detection methodology (which is described in much ... attack d'jour with a cool sounding name and/or press ...
      (Focus-IDS)
    • RE: Hi, I want to study IPS
      ... >>of systems to pull everything together into an IDS solution. ... you are right that some IPS products use similar techniques as ... technologies in attack detection. ... capabilities, and so have less false positives, which is not true. ...
      (Focus-IDS)
    • Re: IPS comparison
      ... > rules to capture normal behavior, and I can use regexp ... "If there is a GIVEN SET of regexp there, it's not anomaly detection" ... Stefano Zanero ...
      (Focus-IDS)