RE: Neural Net based Host/Application Anomaly detection systems

From: Reidy, Patrick (Patrick.Reidy@veritect.com)
Date: 02/22/02


From: "Reidy, Patrick" <Patrick.Reidy@veritect.com>
To: 'Stephen & Laren Baker' <larenbaker@attbi.com>, Scott Nursten <scottn@s2s.ltd.uk>, husmousa <husmousa@aucegypt.edu>, focus-ids@securityfocus.com
Date: Fri, 22 Feb 2002 09:46:54 -0500

Based on my experience with enterprise wide anomaly and trend analysis I
think the big problem of more advanced anomaly intrusion detection systems
is the ability of the user to create a large enough data set on which to
base the anomaly detection. Meaning that most enterprises do not have the
time, money or inclination to launch hundreds of attacks to evaluate their
behavior on their given network. Simply providing the user with a "generic"
base data set is one way to solve the problem. The problem with that, of
course, is the more granular the anomaly detection becomes, the less generic
the anomaly detection "scripts" become (i.e. trying to mine out protocol
anomalies on one network would be completely different that another). Of
course, the more granular, the more effective the system. Basically it is a
big pain in the neck and other body parts of note. That is why I am
skeptical of any product or tool that claims to be "anomaly based" right out
of the box. If you can solve the data set problem, I think you would really
have something their.

-----Original Message-----
From: Stephen & Laren Baker [mailto:larenbaker@attbi.com]
Sent: Tuesday, February 19, 2002 4:10 AM
To: Scott Nursten; husmousa; focus-ids@securityfocus.com
Subject: Re: Neural Net based Host/Application Anomaly detection systems

You might want to investigate NFR NID hw/sw turnkey device. The NFR NID
provides anomaly detection capabilities (attack signature logic) added to
the ability to do *the standard NID string matching* in a single network
device/sensor. For technical data: http://www.nfr.com/products/NID/

----- Original Message -----
From: "Scott Nursten" <scottn@s2s.ltd.uk>
To: "husmousa" <husmousa@aucegypt.edu>; <focus-ids@securityfocus.com>
Sent: Monday, February 18, 2002 11:57 AM
Subject: Re: Neural Net based Host/Application Anomaly detection systems

> Working on security systems always involves using multi-layered
approaches.
> I can see this becoming a widely accepted product if it delivers what is
> promised. I can also see it helping in certain non-invasive attempts to
> exploit/overflow server-side code (ie. when a script kiddie does a
>
> GET /index.jsp?blah=aaaaaaaaaaaaaaaa x (50000)
> )
>
> ...as that might not trigger a standard NIDS signature but seems likely to
> show up on your system. It also helps to have some feedback AFTER the
> application has done the necessary translations (ie. Unicode etc) but I'm
> not sure whether or not your product does that.
>
> All in all, I'd say it's a good idea...! Where do we get the beta
(alpha?)?
> :)
>
> Regards,
>
> Scott Nursten
>
>
> On 17/2/02 7:41 pm, "husmousa" <husmousa@aucegypt.edu> wrote:
>
> > Hi All,
> >
> > as part of a 1 year graduation project we ( a team of 4 CSCI graduates)
> > developed an application based anomaly detection system. By application
based
> > I mean we focused on securing certain applications (in our case Apache
and pro
> > ftp). The reason we choose this approach was due to the analysis mode we
used.
> > Our analysis engine was based on a neural network which simply tries to
> > "categorize" access requests in clusters. These clusters have been
previoiusly
> > trained using lots and lots of log entries that have been grouped into
> > categories (for example, normal, buffer overflow, cgi abuse, ... ).
> >
> > So far nothing new. the promise of this system though is that it has
potential
> > to detect previously unseen attacks. In fact in our limited lab tests we
have
> > isolated some logs and used them later to test our system (without being
> > previously used in training). This can approximately mimic the real
world case
> > of new and modified attacks being used against IDS systems that can only
> > detect known threats. Our initial results were impressive.
> >
> > Of course our system will never ever ever replace traditional
signature/rule
> > based IDS systems (such as snort, realsecure, etc...) but it can
effectively
> > add a good 10-20% of coverage power specifically against brand new
attacks.
> >
> > I was deliberating on our next step. I wasn't so sure about how widely
would
> > such a product be accepted if offered as an open source standalone (or
> > possibly as a plugin to SNORT).
> > we still need to do some work on "commercializing" it basicly doing some
more
> > training and resolving some performance issues.
> >
> > I hope I can generate alot of feedback, thoughts, ideas, advice, ten
cents,
> > two cents about this project
> >
> > thanks
> >
>
> --
>
>
>



Relevant Pages

  • RE: Neural Net based Host/Application Anomaly detection systems
    ... Interesting enough however, anomaly detection is ... >> behavior on their given network. ... >> base data set is one way to solve the problem. ... >> anomalies on one network would be completely ...
    (Focus-IDS)
  • RE: IPS comparison
    ... >- maybe one day, the hard drive crashes, and all the network starts ... > doing DNS requests to the backup DNS server which looks like some ... >traffic, netflow, firewall logs, host logs, .etc, but anomaly detection ... That's why having a NADS to prioritize these anomalies could save you ...
    (Focus-IDS)
  • Re: Neural Net based Host/Application Anomaly detection systems
    ... > base data set is one way to solve the problem. ... > course, is the more granular the anomaly detection becomes, the less generic ... > anomalies on one network would be completely different that another). ...
    (Focus-IDS)
  • Re: Neural Net based Host/Application Anomaly detection systems
    ... You might want to investigate NFR NID hw/sw turnkey device. ... Neural Net based Host/Application Anomaly detection systems ... > ...as that might not trigger a standard NIDS signature but seems likely to ... >> to detect previously unseen attacks. ...
    (Focus-IDS)
  • Re: Specification-based Anomaly Detection
    ... > shortcomings of signatures, it has to be considered seriously. ... Or highly polimorph attacks, yes. ... > enables us to both correlated different anomalies to generate more ... there is no real difference than using an advanced network IDS ...
    (Focus-IDS)