Re: IDS Correlation

From: Marcus J. Ranum (mjr@nfr.com)
Date: 04/02/02 

Date: Mon, 01 Apr 2002 21:39:47 -0500
To: "Stephen P. Berry" <spb@meshuggeneh.net>
From: "Marcus J. Ranum" <mjr@nfr.com>

Stephen P. Berry wrote:
>The fact XML is seen as adequate for something like
>a standard for representing generic intrusions indicates that we (as a
>whole) are still hung up on an incident model that consists of events on
>the scale of single packets.

Actually, I beg to disagree with this point in your otherwise well-reasoned
observations. :)

Incident models are independent of the data representation used. The
fact that most incident models to date are trivial simply lends them
to trivial representation schemes. :) What's embarrassing is when overcomplex
representation schemes are used to store trivial incident models. ;) I have
painful experience in this area that I am not willing to share! :) So don't
ask.

That being said, consider that you can take a series of events that are
represented simply and coalesce them into meta-events that contain
references to their ancestors. It's not hard to do some pretty impressive
stuff with it once you've done that. That's some of the research I've been
working on with the fargo log analyzer; taking multiple events and
mapping them into coalesced meta-events. All you do then is browse
the references to whatever depth you need, if you want the full view
of how the IDS got to where it is.

I think we both agree that complex XML representations are a
good indicator that "does not get it" is taking place. Simple XML
representations, though, are a decent way of leveraging existing
stuff that relies on XML, or making it easy for post-processors
to act quickly and reliably on the data. Frankly, I don't give a fig
for XML document type declarations, but there's no real
difference between:

delimiter=,
event, priority, text string, event-id\n

and

<rec>
<event>event data</event>
<priority>priority ##</priority>
<message>message text</message text>
<id>event id</id>
</rec>

and if you use even a basic compression algorithm it'll compress
them down to pretty much the same thing. Takes about the same
amount of code to pack and unpack them, sanity check them,
and process them for escape codes.

There's basically only a few ways to skin this cat;*the details are
all different enough to fool the casual observer.

mjr.
(I and my cats do not approve of skinning cats)

Relevant Pages

  • Re: Interplatform (interprocess, interlanguage) communication
    ... I think the issue is that one particular technology, XML, is used in significantly different ways by different people and for different reasons. ... many people use XML for data-binding, and many other people who use it could care less about data-binding. ... many people neither use schemas nor any sort of schema validation. ... admittedly, I also partly dislike traditional ways of using data-binding as it often exposes things which are theoretically internal to the app, namely structural data representation, with things which should theoretically be isolated from the internal data representation: ...
    (comp.lang.java.programmer)
  • Re: Document order (was: MV Keys)
    ... it is that order has no meaning. ... Historically (XML coming from a ... the XML representation of each node occurs in the XML representation of ...
    (comp.databases.theory)
  • Re: XQuery (and XML) vs LISP
    ... In terms of representation, XML therefore does ... The fact that we NEED to traverse edges of the tree, ... can be represented more easily using XQuery on XML data. ... persistently using both XML and simple binary relation. ...
    (comp.databases.theory)
  • Re: Howto read line from a stream
    ... or else it is non-trivial and then a representation of it as a tree does ... Ada program as a configuration for some processor. ... XML uses graphs BTW. ... Every proper XML grammar is documented. ...
    (comp.lang.ada)