RE: Comparing the performance of two IDS products with different architectures

From: RJ H (huberrj@hotmail.com)
Date: 10/25/01 

From: "RJ H" <huberrj@hotmail.com>
To: focus-ids@securityfocus.com
Subject: RE: Comparing the performance of two IDS products with different architectures
Date: Thu, 25 Oct 2001 00:33:19 +0000
Message-ID: <F73fUdG2b1niYs6QuWi00013759@hotmail.com>

So if you are using protocol analysis, and the attack/misuse subverts the
protocol itself then what? You're back to pattern matching. And if you are
not pattern matching the whole string, how do you know that they did not
subvert the protocol, and go right past your protocol analysis engine?

Using the telnet example, if you are looking for the login information in
only the login field, and someone figures a way out to login, not following
the protocol then you missed it, ......unless you did your pattern search
through the entire content of the session.

Are there not exploits out there that are successful because they find a
flaw in the protocol and exploit that?

I certainly would agree though, that the protocol analysis MAY be able to
reduce the number of false postitives, but I would not place that much faith
in protocol analysis alone. A combination of protocol analysis and pattern
matching is the defense in depth that I am looking for as a consumer.

.....maybe I just missed the meaning of Gary's message...I'll defer to the
experts on the list...

>From: "robert_david_graham" <robert_david_graham@yahoo.com>
>To: "'Gary Golomb'" <gee_two@yahoo.com>, "'robert_david_graham'"
><robert_david_graham@yahoo.com>, <iheagwarac@aol.com>,
><focus-ids@securityfocus.com>
>Subject: RE: Comparing the performance of two IDS products with different
>architectures
>Date: Wed, 24 Oct 2001 15:11:23 -0400
>
> > From: Gary Golomb [mailto:gee_two@yahoo.com]
> > No one claimed that it is the "best" way to do ID, so I
> > am not attacking that point.
>
>The original post was about contrasting different theoretical approaches.
>
>This is much like the discussion of some of the theoretical basis of RISC
>vs. CISC CPU performance. Unfortunately, for some reasons, engineers can't
>keep value judgements like "best" out of such discussions. You can describe
>some really interesting things that RISC processors use to increase
>performance. However, it doesn't mean the RISC technique is "best": Intel
>CISC processors have kept pace with RISC over the past 10 years. Whether we
>are discussing CPUs or IDS performance, there is no "best" approach, but
>there are certainly interesting theoretical differences.
>
>
> > it needs to 1) recognize the
> > protocol and extract the fields it will be doing an analysis
> > on, and 2) ummmm... do pattern
> > matching? Well, unless the device has some sort of ESP, it
> > has do some kind of comparison, while
> > seems like looking for patterns, which I in turn call pattern
> > matching - even though that word was
> > avoided like the plague in the above email when discussing
> > his own IDSs.
>
>This is an inaccurate characterization.
>
>The first problem is that "my own IDS" uses "pattern-matching" for only
>about half the signatures. The other half tests against other criteria that
>have no relationship with pattern-matching. For example, the Telnet decode
>tests the length of the login name field and triggers an "overflow" event
>if
>it exceeds a certain threshold. Similarly, a Telnet login failure event
>triggers when it has seen 4 login failures.
>
>The second problem is the difference between "searching" and
>"exact-matching". In my example, I described how we "match" a login-pattern
>only against the login-field, whereas other systems "search" for the
>login-pattern anywhere in the content. There is a difference between the
>two
>techniques, whatever terms you might choose for them.
>
>
> > You bet it is which is why all of these marketing terms
> > people inject into these discussions are
> > not only frustrating, they are EXTREMELY misleading.
>
>It isn't "marketing" terms. Any engineer can tell you that there are some
>fundamental differences between the functions "strcmp" and "strstr": one
>does an exact match, the other does a search.
>
>If you look at Snort or Dragon, most of the signatures are done through
>pattern "searches". Of course, it isn't as simple as doing a "strstr" for
>all patterns on all ports; efficient pattern search is an evolved science
>(e.g. Boyer-Moore, etc.) and has gotten quite fast. Snort is constantly
>evolving how it does this, it gets better every release, and there is
>extensive documentation on how they optimize this.
>
>On the other hand, if you look at BlackICE (my IDS), 99% of the signatures
>do no "searching" whatsoever. Half the signatures don't even contain
>"patterns", and the other half do "strcmp" style exact matches (actually,
>memcmp() style matches).
>
>Certainly, engineers often squabble about the best terminology to describe
>things here, but this is an engineering-term disagreement, not a vacuous
>marketing-term disagreement.
>
>
> > To be cut and dry about it, if I am looking for a telnet
> > login attack of some sort, then I want to
> > look at ALL telnet traffic going to a server
>
>This is a separate discussion than theoretical performance.
>
>One rootkit allows a username of "friday" to bypass security. I'm sure you
>could look for "friday" thoughout the Telnet content, but the average
>trigger would probably be a false-positive. However, if you trigger only
>when "friday" is used as a username, then you will still potentially have
>false-positives when it is used as a legitimate username, but most triggers
>for the average customer will be for a true attack.
>
>This is one of the key things we are doing in RealSecure 7 -- getting rid
>of
>most of the "searches" RealSecure 6.x does and changing them to
>BlackICE-style "exact-matches". We estimate that we'll eliminate 99% of the
>false-positives customers are experiencing right now by doing this.
>
>There are specialized cases where you DO want to search all content (e.g.
>90
>90 90 90) -- again, this is a different set of tradeoffs, different
>theoretical approaches, different customers might want different things.
>
>
> > As an end user, why should I care what
> > algorithm or type of hash table a
> > prospective IDS uses.
>
>...because it has enormous impact on how the IDS runs in different
>environments.
>
>Right now, IDSs aren't plug-and-play. They get closer every year, but if
>you
>drop an IDS into your environment, you will get vastly different
>performance
>characteristics depending on how you tune it. Luckily, most customers need
>only 10-Mbps, so most customers don't need advanced performance tuning.
>
>For example, customers often ask me how to disable signatures in order to
>tune for performance. For my IDS (BlackICE), you don't -- because it's
>protocol-analysis based. On the other hand, there is a range of OTHER
>settings you DO need to configure to tune for performance that is related
>to
>the protocol-analysis engine. For example, you might want to consider
>changing the size of certain hash-tables.
>
>
>
>_________________________________________________________
>Do You Yahoo!?
>Get your free @yahoo.com address at http://mail.yahoo.com
>

_________________________________________________________________
Get your FREE download of MSN Explorer at http://explorer.msn.com/intl.asp

Relevant Pages

  • RE: Value of "richer" signatures?
    ... Is it that much faster to do "protocol parsing" than ... > Here's an example of how the newer IDS signatures help ... > Let's say you are using a simple packet grepping IDS ...
    (Focus-IDS)
  • Re: IDS detection approaches
    ... it is a technology limitation: pattern matching. ... This is an easy target that recently got updated in the _signature_ ... Do not underestimate the value of signatures in protocol modeling. ... I would completely go with a signature based IDS. ...
    (Focus-IDS)
  • RE: ids inquisition
    ... Well, I also fully believe that BOTH protocol analysis, AND pattern ... Dozens of IDS companies out there are merketing millions of dollars ... One signature for ANY buffer overflow, ... > we just knew that this would likely be a vulnerability. ...
    (Focus-IDS)
  • Re: ids inquisition
    ... Subject: ids inquisition ... Whenever I buy an IDS I life it ... Well, I also fully believe that BOTH protocol analysis, AND pattern ... > we just knew that this would likely be a vulnerability. ...
    (Focus-IDS)
  • RE: Comparing the performance of two IDS products with different architectures
    ... Comparing the performance of two IDS products with different architectures ... An interesting point, “a packet is only tested for a signature when needed, and not when it isn't ... and only tests signatures that apply to those contents. ... could argue all day long about the strengths and weaknesses of “pattern matching” vs “protocol ...
    (Focus-IDS)