Re: [fw-wiz] RE: IDS (was: FW appliance comparison)

At 12:24 AM 25/01/2006, Marcus J. Ranum wrote:

Cat Okita wrote:
>> Would you care to elaborate on the way that you handle the vast
>> amounts of data that you collect, then? Sorting the gold from the
>> dross is a monumental challenge on a good day.

Like he says.

<lights cigarette, leans on post while Marcus paces and gestures>

> Use an artificial ignorance to weed out the majority of it, then
> revector stuff that should be counted and quantified into a
.d.
> For truly huge amounts of log data, you can use hardcoded
> tools and get amazing data rates out of them; for example,
> building a parse-tree out of nested calls to sscanf using the
> magic %n operator to offset directly into the last match.

Computers are fast and people are smart. When you break down the logical structure of the problem you find that there are not significant hurdles that can't be knocked down with the usual brow sweat and frayed nerves of any technical endeavor.

Once you accept the idea that your operational goal is to monitor the living bejesus out of everything, the model changes. *Since* you can see everything (and if you can't you'll fix it), you can focus on dealing with what is happening, make more intelligent forecasts for planning, look back at what happened for analysis and reporting and basically take a more strategic role in making a network secure.

> System log processing remains a backwater in spite of the
> recent interest in the topic thanks to HIPAA and whatnot.

It's the calm before the storm. More people need to (and will) contribute to the effort before it's really mature.

> www.loganalysis.org has some resources on some of
> this stuff. But it remains the land of do-it-yourselfers
> because log data is very site-specific. On the other hand
> it's not freakin' rocket science; if you just sit down and
> start eyeballing the stuff you'll get an idea what you
> need for your site within an hour or 2.

Agreed. It has gotten to the point that when I see each new network it feels like 1995 ("well - since you've done *nothing*, and I can telnet directly into the middle of your network from home, and you make parts for manned spacecraft - any firewall might be a good start"). Except now it's: "Well, since you have no way to see even the *slightest* bit of what the hell is happening on your network while we're standing here drinking bottled water, even a crappy [within limits] SIM solution is probably a good idea."

It is worth the effort to find a way to Manage the Information about the Security of your network (a SIM by any name would smell so sweet...). The SIM vendorsphere is completely fubared (venodrs are easily recognizable: they're the ones saying, "well, we're not a *SIM*, we're a ....") , but there are workable bits of technology out there. Applying some of the products and processes available is a good start.

As always with fundamental shifts, it will take time for the solution providers to make the solutions fully respectable, but it will take time for the consumers to work through adoption pains as well so IMHO current solutions are fit for early adoption in volume. By the time a company today adopts and deploys a solution to the point of being sore about solution shortcoming, they will have benefited directly from the effort, they will be better positioned to ask intelligent questions of the providers, and the solution choices will be richer.

-cheers!

-chris

[So, Paul. Obviously I'm still sending html, eh?]


_______________________________________________
firewall-wizards mailing list
firewall-wizards@xxxxxxxxxxxxxxxxxx
http://honor.icsalabs.com/mailman/listinfo/firewall-wizards

Relevant Pages

  • Re: Crazy samba problem
    ... parameter is not specified is zero. ... HUGE amounts of log data, most of which is extremely cryptic. ... Note that specifying this parameter here will override the log ...
    (comp.os.linux.setup)
  • Re: sorting of text file
    ... I have learned in school several alghorithms for sorting big ... amounts of data. ...
    (microsoft.public.vc.mfc)
  • Re: Extract select group of numbers
    ... AFTER sorting on the absolute value, I want to to be able to extract just ... these amounts from my example. ... Offset = -1,000.00 ...
    (microsoft.public.excel.misc)
  • Re: OT:The World at War
    ... Command could have put up would have decimated any attempt at daylight ... bombing, and also possibly that such an attempt would have used up vast ... amounts of fuel, something they were rather short of. ...
    (uk.rec.motorcycles)
  • Re: XML and its uses
    ... The use of XML for data-exchange is largely a done deal, with vast ... amounts of software available, plentiful skills, relatively simple data ...
    (comp.text.xml)