Re: Truth about False Positives

From: Aigars Grins (
Date: 09/11/01

Message-ID: <017001c13ae4$12fe6160$>
From: "Aigars Grins" <>
To: <>
Subject: Re: Truth about False Positives
Date: Tue, 11 Sep 2001 18:06:18 +0100

Hi all,

This all might sound as marketing b**. If so, then please disregard this. I
work for a service provider which might make be biased. And sorry for my bad
English as well..

----- Original Message -----
From: "Jackie Chan" <>

> [..] Perhaps a better term could be "meaningless drivel",
> or "Junk alerts" Things that do nothing but clutter up the entire system,
> from the sensor, to the network, to the console, to the database.

I agree that a better then "false positives" would be good, but as Graham
says we're probably stuck with what we have (cmp to the definition of

> My first blame, is to blame the lack of education of the user when it
> comes to deriving policies that are specific to their networks.

I strongly agree. Way too many organizations don't have well defined
security policies and even more don't enforce them throughout the entire
organization. How are you supposed to deploy/configure things on your
network if you don't know what they're supposed to achieve?

> Followed
> by a close second blame to the developers who decide that any url with
> 'phf' in the string is suspicious.

:) Yes, saying that an IDS doesn't produce "false positives" is looking at
things from the wrong perspective. While it may be true that an IDS detects
things that _correlate_ to attacks/intrusions the user is most often not
that interested in such details as such. From _there_ perspective the IDS is
there to detect attacks/intrusions. The rest are implementation specifics
that, while unfortunately being necessary to take into account, aren't
interesting in themselves. From that perspective there are "false positives"
produced from most probably all IDSs.

> So how do we solve this problem? First off you cant hand an erector set
> to a child, and expect him to build something of his own creation without
> spending time with him. Secondly, you can't build it for the child, and
> expect him to learn from that... you have to teach him how to build, and
> some of the laws of structure. Translated into business terms, this means
> mandatory services, hopefully good services. Translated into tech terms,
> this means that we have to figure out what it is we do when we ourselves
> go through this process and automate that as much as possible...

Sure, the entire computer community has to be educated in a lot of matters
(to make things better). Alas, security is but one of them..

> (question: how many IDS developers have actually ever been responsible for
> network security somewhere in the last 3 years? Now lets see how many
> CTO's fit this same definition? Shockingly low, isn't it?)

Yes and no. There are some firms that have a better situation. I'm thinking
of the ones that offer remote surveillance of IDSs etc. as well. Eg. ISS
both offers IDS solutions and also 24/7 monitoring of deployments. Within
these type of organizations there is a _potential_ to make use of the
knowledge of both worlds and come up with something good. (Disclaimer: I'm
not pushing any specific vendors and have no affiliation with ISS)

> There are at least 5 different ways of tackling this problem of the
> worthless information generated by our technologies, from origins of
> development, to the level of effort of services, to a general business
> model of sorts.
> Regardless of how we attack the problem, lets stop quibbling over terms,
> and mitigate the need for such a term.

:) I think that the IDS technology of today sucks. It most definitely does.
There are things that suck less then others of course. And all is relative.
The mission task is far from trivial. So I'm not saying that the vendors are
doing a bad job. Just that they're far from achieving the goals of maturity
yet. But they'll get there eventually.

As you say, one of the problems is that to make the noise ratio drop the
configuration of the IDS becomes increasingly complex and site specific.
This makes the IDS much less workable out-of-the-box. That of course makes
the product look bad in many tests and at different deployment sites
(because of bad configurations).

While the requirements on education increase, so does the maintenance level
of the IDS deployment as a whole. I was once at a customers site and asked
them for a list of their internal DNSs. They just laughed at me - they (the
central administration techs) had no idea. Imagine _closely_ configuring an
IDS to fit into the network at their place. Impossible..

That said a company can efficiently buy competence by outsourcing the
configuration/tweaking/monitoring. In the end the company has to learn
themselves as well. After all, they haven't outsourced the workings of the
security policies etc. so they have to understand the situation to make
utopia exist. But it's a start which is available today. To think that it's
going to be easy to go from ignorance to someone all-knowing is naive to say
the least.

Sorry about the long rambling. I have problems with expressing myself
quickly. The marketing blurb is of course about promoting MSS. But the point
is that to decrease noise I don't see a quick breakthrough without being
able to configure things a _lot_ more then most things can be today. That
comes at a price. What costs most: more configuration or more logs to read?

Aigars Grins


Internet communications are not secure and therefore Defcom does not accept legal responsibility for the contents or accuracy of this message. The views and opinions contained in the message are solely those of the author and do not necessarily represent those of Defcom unless otherwise specifically stated.