RE: Specification-based Anomaly Detection

From: Ofer Shezaf (
Date: 01/14/05

  • Next message: Kohlenberg, Toby: "RE: Specification-based Anomaly Detection"
    Date: Thu, 13 Jan 2005 22:19:54 -0500
    To: "Stefano Zanero" <>, "Kohlenberg, Toby" <>

    Hi Stefano & Toby,

    I'll refrain from in-lining as it becomes cluttered down there, and I
    feel that everybody is saying the same thing just with different
    emphasis and a lot of arguments about wording (got to lawyers level)....
    And I swear not to use "actionable" in non-commercial situations.

    So I'll try to roll the discussion forward:

    Stefano writes that a host and a port define a listening application and
    then you carry on about detecting an application automatically. While
    mail is an application, it is a static application in the sense that it
    behaves very much the same over time and between users and sessions (and
    so does FTP and so on).

    I feel that the mind set of the discussion was about such applications,
    and that an IDS system (signatures or anomaly based) for such protocols
    would not be much different than a network IDS.

    I would like to call the types of applications that my company's
    products handle "dynamic applications". I'm referring to interactive
    http based applications. Why are they different? For many reasons, only
    some of them directly related IDS, but all have security relevance:

    1. Only widely used system that allows a very large community to write
    client server applications (hence the tons of poor coding).
    2. Protocol elements are polymorphic, not just the content, and are
    changed by the above "programmers".
    3. Only widely used system where code is constantly downloaded by the

    And as a result, a lot more action...

    Does this make intrusion detection in web applications deferent? Based
    on our experience with out product I think so.


    Probably because the balance between know how and mathematical analysis
    is different. When I think of it, our product includes a lot of implicit
    know-how about http, html and how different application environment use
    it. We don't have to apply abnormal behavior algorithms to a steam of
    information but to clearly identified attributes of transactions that we
    know quite a lot about.

    In some ways this is more similar to HIDS than to NIDS (And by the way,
    we also passively decrypt SSL - if we get the key - so even less
    difference than a host IDS).

    Another issue evolved around my assertion that the protocol is
    polymorphic. When I stated in a previous e-mail that we learn the
    application behavior and not the user behavior, I referred, in terms
    more commonly used in IDS that we learn the protocol. As the protocol is
    defined by the specific programmer at the organization building the web
    site, learning it and validating that users are in conformance provides
    a layer of security that I'm not sure should be called abnormal behavior
    detection in the common IDS terminology.

    ~ Ofer

    Ofer Shezaf
    CTO, Breach Security

    Tel: +972.9.956.0036 ext.212
    Cell: +972.54.443.1119

    > -----Original Message-----
    > From: Stefano Zanero []
    > Sent: Tuesday, January 11, 2005 11:29 AM
    > To: Kohlenberg, Toby
    > Cc: Ofer Shezaf;
    > Subject: Re: Specification-based Anomaly Detection
    > Kohlenberg, Toby wrote:
    > > Stefano, could you expand on which part you agree with? I'm really
    > > confused to think that you would agree that anomaly detection would
    > > be new to IDS.
    > I would agree that:
    > - anomaly detection is needed as a complementary approach to misuse
    > detection because of the inherent limits of the latter
    > - and that anomaly detection (in particular techniques which are not
    > rate-based) is a relative "newcomer" in the COMMERCIAL field of
    > intrusion detection, where most of the products are built on a misuse
    > detection approach.
    > >>>is zero day
    > >>Or highly polimorph attacks, yes.
    > > Or custom-written attacks
    > Absolutely correct !
    > > Really? What about apps that all tunnel over a single port?
    > That would be a problem even if you work at application layer ;)
    > Please note that Ofer was not advocating HOST-based intrusion
    > but NETWORK-based approaches working at layer 7
    > > Are you getting the application that IANA says runs on that port or
    > > are you getting SAP using telnet on some random port or Cisco using
    > > HTTP on yet another random port?
    > That's something that the algorithm we have developed can recognize ;)
    > >>This is basic misuse detection, it does not mean you can deliver an
    > >>actionable anomaly detection result.
    > >
    > > No, but it does give you a much better chance of finding
    > > (or ignorable)
    > Yes, but since we are discussing wether or not ANOMALY detection is
    > "actionable" (I'm not a native speaker but this word sounds horrible
    > me :) this objection is not relevant. Or better, it says exactly what
    > Tom and I were saying: anomaly detection is not, and this is a
    > disadvantage wrt misuse detection.
    > Best,
    > Stefano

    > Test Your IDS
    > Is your IDS deployed correctly?
    > Find out quickly and easily by testing it with real-world attacks from
    > Go to
    > to learn more.
    Test Your IDS
    Is your IDS deployed correctly?
    Find out quickly and easily by testing it with real-world attacks from 
    Go to 
    to learn more.

  • Next message: Kohlenberg, Toby: "RE: Specification-based Anomaly Detection"