Re: Views and Correlation in Intrusion Detection

• Previous message: DAVID MARKLE: "Re: Views and Correlation in Intrusion Detection"
• In reply to: Blake Matheny: "Views and Correlation in Intrusion Detection"
• Next in thread: Stephen P. Berry: "Re: Views and Correlation in Intrusion Detection"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]

Date: Tue, 17 Jun 2003 13:54:31 -0400
To: Blake Matheny <bmatheny@mkfifo.net>

At 01:32 PM 17/06/2003, Blake Matheny wrote:

>Two areas that I have recently been doing research in, are views and their
>connection to correlation techniques.

> There are really several issues here. First of all, a tremendous amount of
>time is being spent, trying to correlate all the relevant information. This is
>something that _can_ be automated. Second, the applications logs may not be
>trustworthy. Third, and to me, most importantly, is the fact that this is such
>a 'basic' thing that people using ID systems have to do, and there is no piece
>of software yet that does this.

Totally agree with you. I've been working on the same problems myself
during the last 1-2 years.

> So something we have been working on, is a system to deal with this basic
>type of scenario. This will entail data transformations into an intermediary
>language, an event description language, offline state analysis and several
>other components (there is more information at http://www.nongnu.org/babe/).
>If you spend some time thinking about everything involved to do this in a
>scalable fashion, it's an enormous task (I said basic, not simple). What I am
>finding frustrating, is that much of the base research has not yet even been
>done. Much of the research that has been done, is either too primitive or too
>impractical to be implemented. Is this due to the infancy and immaturity of
>the field, do people not see this as being feasible and therefor aren't
>spending the research time, or is this simply too far down the line? In any
>case, feedback welcome. Thanks.

Believe me, I do see this as being feasible, but I think that this topic is
probably still in its infancy stages, which is why there is not much work
published around this. As I said, I worked on these problems myself, and
have released my tools (with documentation, which also covers the theory
nehind it) approx 3 weeks ago, so maybe you have missed it. My
implementation works on Win NT/2K/XP platforms, but could very well work
with logs coming from *NIX systems if you can get them to a UNC share on a
Windows box. You can download my tools at http://securit.iquebec.com/.

I looked at your doc and flow chart, and we have roughly the same approach
to the problem, although we have some fondamental development differences
(for example, my tools don't have DB support for log storage). To gather
application and Event Viewer log files to a central location, I have made
LogAgent (now version 4.0, available in Open Source and Pro versions). So
with it, you can monitor-and-centralize on the fly log files for antivirus,
personnal firewall, main firewall if applicable, NIDS like Snort,
etc... LogAgent 4.0 also comes with a HIDS program that checks for file
system integrity, and an Alternate Data Stream scanner (ADS are a way to
hide files on a Windows system). It also generates forensics-related data
like running services, startup conf, open shares, that is then matched
against a list of allowed ressources. I have also developped a command
prompt (cmd.exe) logger, ComLog (now version 1.05, OS and Pro), so it is
now possible to keep a log of hacking incidents where commands were passed
this way (with a cryptcat tunnel, for example).

Now that all these logs from various applications are gathered from all
over the network to a central place, the challenge is to analyse them in an
efficient way. You say you want to work on data output also, but you have
no screenshot or description of what you have in mind. Any interface
ideas? I tried to make something new. To monitor and analyse these logs,
I made the console program LogIDS 1.0 (av. in OS and Pro), which will
monitor these log files for you and apply rules in order to sift through
them and select what is worthy of attention. The interface is a
representation of your network map, where each node have its own monitoring
window, and icons can be specified to illustrate the event reported in the
log. LogIDS is very flexible, you get to define the fields of every log
file you include, and apply rules using these fileds definition. The flow
of data is somewhat similar to what you describe on your flow chart
http://www.nongnu.org/babe/papers/data_flow.png.

I have to admit that it is not perfect tough, as you mentionned this is
still a very new topic, and far from being as mainstream as, say, firewalls
technology. Feel free to look at it to get ideas/make suggestions, it's
nice to see that something similar is growing in the *NIX world. I already
have some plans for future releases, like distributed analysis, allow for
new modules to be added easily to LogAgent, some performance tweaks, an
enhanced ruleset, and a couple other nice features. Not to be expected
before at least a few months.

Hope this helps!

Adam Richard, aka Floydman
SécurIT Informatique Inc.

>Cheers,
>
>-Blake
>
>--
>Blake Matheny "... one of the main causes of the fall of the
>bmatheny@mkfifo.net Roman Empire was that, lacking zero, they had
>http://www.mkfifo.net no way to indicate successful termination of
>http://ovmj.org/GNUnet/ their C programs." --Robert Firth

-------------------------------------------------------------------------------
Attend the Black Hat Briefings & Training, July 28 - 31 in Las Vegas, the
world's premier technical IT security event! 10 tracks, 15 training sessions,
1,800 delegates from 30 nations including all of the top experts, from CSO's to
"underground" security specialists. See for yourself what the buzz is about!
Early-bird registration ends July 3. This event will sell out. www.blackhat.com
-------------------------------------------------------------------------------

• Previous message: DAVID MARKLE: "Re: Views and Correlation in Intrusion Detection"
• In reply to: Blake Matheny: "Views and Correlation in Intrusion Detection"
• Next in thread: Stephen P. Berry: "Re: Views and Correlation in Intrusion Detection"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]