RE: Neural Net based Host/Application Anomaly detection systems

From: Reidy, Patrick (
Date: 02/22/02

From: "Reidy, Patrick" <>
To: 'Stephen & Laren Baker' <>, Scott Nursten <>, husmousa <>,
Date: Fri, 22 Feb 2002 09:46:54 -0500

Based on my experience with enterprise wide anomaly and trend analysis I
think the big problem of more advanced anomaly intrusion detection systems
is the ability of the user to create a large enough data set on which to
base the anomaly detection. Meaning that most enterprises do not have the
time, money or inclination to launch hundreds of attacks to evaluate their
behavior on their given network. Simply providing the user with a "generic"
base data set is one way to solve the problem. The problem with that, of
course, is the more granular the anomaly detection becomes, the less generic
the anomaly detection "scripts" become (i.e. trying to mine out protocol
anomalies on one network would be completely different that another). Of
course, the more granular, the more effective the system. Basically it is a
big pain in the neck and other body parts of note. That is why I am
skeptical of any product or tool that claims to be "anomaly based" right out
of the box. If you can solve the data set problem, I think you would really
have something their.

-----Original Message-----
From: Stephen & Laren Baker []
Sent: Tuesday, February 19, 2002 4:10 AM
To: Scott Nursten; husmousa;
Subject: Re: Neural Net based Host/Application Anomaly detection systems

You might want to investigate NFR NID hw/sw turnkey device. The NFR NID
provides anomaly detection capabilities (attack signature logic) added to
the ability to do *the standard NID string matching* in a single network
device/sensor. For technical data:

----- Original Message -----
From: "Scott Nursten" <>
To: "husmousa" <>; <>
Sent: Monday, February 18, 2002 11:57 AM
Subject: Re: Neural Net based Host/Application Anomaly detection systems

> Working on security systems always involves using multi-layered
> I can see this becoming a widely accepted product if it delivers what is
> promised. I can also see it helping in certain non-invasive attempts to
> exploit/overflow server-side code (ie. when a script kiddie does a
> GET /index.jsp?blah=aaaaaaaaaaaaaaaa x (50000)
> )
> that might not trigger a standard NIDS signature but seems likely to
> show up on your system. It also helps to have some feedback AFTER the
> application has done the necessary translations (ie. Unicode etc) but I'm
> not sure whether or not your product does that.
> All in all, I'd say it's a good idea...! Where do we get the beta
> :)
> Regards,
> Scott Nursten
> On 17/2/02 7:41 pm, "husmousa" <> wrote:
> > Hi All,
> >
> > as part of a 1 year graduation project we ( a team of 4 CSCI graduates)
> > developed an application based anomaly detection system. By application
> > I mean we focused on securing certain applications (in our case Apache
and pro
> > ftp). The reason we choose this approach was due to the analysis mode we
> > Our analysis engine was based on a neural network which simply tries to
> > "categorize" access requests in clusters. These clusters have been
> > trained using lots and lots of log entries that have been grouped into
> > categories (for example, normal, buffer overflow, cgi abuse, ... ).
> >
> > So far nothing new. the promise of this system though is that it has
> > to detect previously unseen attacks. In fact in our limited lab tests we
> > isolated some logs and used them later to test our system (without being
> > previously used in training). This can approximately mimic the real
world case
> > of new and modified attacks being used against IDS systems that can only
> > detect known threats. Our initial results were impressive.
> >
> > Of course our system will never ever ever replace traditional
> > based IDS systems (such as snort, realsecure, etc...) but it can
> > add a good 10-20% of coverage power specifically against brand new
> >
> > I was deliberating on our next step. I wasn't so sure about how widely
> > such a product be accepted if offered as an open source standalone (or
> > possibly as a plugin to SNORT).
> > we still need to do some work on "commercializing" it basicly doing some
> > training and resolving some performance issues.
> >
> > I hope I can generate alot of feedback, thoughts, ideas, advice, ten
> > two cents about this project
> >
> > thanks
> >
> --