RE: Comparing the performance of two IDS products with different architectures
From: robert_david_graham (robert_david_graham@yahoo.com)Date: 10/24/01
- Previous message: Gary Golomb: "RE: Comparing the performance of two IDS products with different architectures"
- In reply to: Gary Golomb: "RE: Comparing the performance of two IDS products with different architectures"
- Next in thread: RJ H: "RE: Comparing the performance of two IDS products with different architectures"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]
From: "robert_david_graham" <robert_david_graham@yahoo.com> To: "'Gary Golomb'" <gee_two@yahoo.com>, "'robert_david_graham'" <robert_david_graham@yahoo.com>, <iheagwarac@aol.com>, <focus-ids@securityfocus.com> Subject: RE: Comparing the performance of two IDS products with different architectures Date: Wed, 24 Oct 2001 15:11:23 -0400 Message-ID: <002401c15cbf$acef2ff0$89a486d1@computer11111111111111111111111111111111>
> From: Gary Golomb [mailto:gee_two@yahoo.com]
> No one claimed that it is the "best" way to do ID, so I
> am not attacking that point.
The original post was about contrasting different theoretical approaches.
This is much like the discussion of some of the theoretical basis of RISC
vs. CISC CPU performance. Unfortunately, for some reasons, engineers can't
keep value judgements like "best" out of such discussions. You can describe
some really interesting things that RISC processors use to increase
performance. However, it doesn't mean the RISC technique is "best": Intel
CISC processors have kept pace with RISC over the past 10 years. Whether we
are discussing CPUs or IDS performance, there is no "best" approach, but
there are certainly interesting theoretical differences.
> it needs to 1) recognize the
> protocol and extract the fields it will be doing an analysis
> on, and 2) ummmm... do pattern
> matching? Well, unless the device has some sort of ESP, it
> has do some kind of comparison, while
> seems like looking for patterns, which I in turn call pattern
> matching - even though that word was
> avoided like the plague in the above email when discussing
> his own IDSs.
This is an inaccurate characterization.
The first problem is that "my own IDS" uses "pattern-matching" for only
about half the signatures. The other half tests against other criteria that
have no relationship with pattern-matching. For example, the Telnet decode
tests the length of the login name field and triggers an "overflow" event if
it exceeds a certain threshold. Similarly, a Telnet login failure event
triggers when it has seen 4 login failures.
The second problem is the difference between "searching" and
"exact-matching". In my example, I described how we "match" a login-pattern
only against the login-field, whereas other systems "search" for the
login-pattern anywhere in the content. There is a difference between the two
techniques, whatever terms you might choose for them.
> You bet it is which is why all of these marketing terms
> people inject into these discussions are
> not only frustrating, they are EXTREMELY misleading.
It isn't "marketing" terms. Any engineer can tell you that there are some
fundamental differences between the functions "strcmp" and "strstr": one
does an exact match, the other does a search.
If you look at Snort or Dragon, most of the signatures are done through
pattern "searches". Of course, it isn't as simple as doing a "strstr" for
all patterns on all ports; efficient pattern search is an evolved science
(e.g. Boyer-Moore, etc.) and has gotten quite fast. Snort is constantly
evolving how it does this, it gets better every release, and there is
extensive documentation on how they optimize this.
On the other hand, if you look at BlackICE (my IDS), 99% of the signatures
do no "searching" whatsoever. Half the signatures don't even contain
"patterns", and the other half do "strcmp" style exact matches (actually,
memcmp() style matches).
Certainly, engineers often squabble about the best terminology to describe
things here, but this is an engineering-term disagreement, not a vacuous
marketing-term disagreement.
> To be cut and dry about it, if I am looking for a telnet
> login attack of some sort, then I want to
> look at ALL telnet traffic going to a server
This is a separate discussion than theoretical performance.
One rootkit allows a username of "friday" to bypass security. I'm sure you
could look for "friday" thoughout the Telnet content, but the average
trigger would probably be a false-positive. However, if you trigger only
when "friday" is used as a username, then you will still potentially have
false-positives when it is used as a legitimate username, but most triggers
for the average customer will be for a true attack.
This is one of the key things we are doing in RealSecure 7 -- getting rid of
most of the "searches" RealSecure 6.x does and changing them to
BlackICE-style "exact-matches". We estimate that we'll eliminate 99% of the
false-positives customers are experiencing right now by doing this.
There are specialized cases where you DO want to search all content (e.g. 90
90 90 90) -- again, this is a different set of tradeoffs, different
theoretical approaches, different customers might want different things.
> As an end user, why should I care what
> algorithm or type of hash table a
> prospective IDS uses.
...because it has enormous impact on how the IDS runs in different
environments.
Right now, IDSs aren't plug-and-play. They get closer every year, but if you
drop an IDS into your environment, you will get vastly different performance
characteristics depending on how you tune it. Luckily, most customers need
only 10-Mbps, so most customers don't need advanced performance tuning.
For example, customers often ask me how to disable signatures in order to
tune for performance. For my IDS (BlackICE), you don't -- because it's
protocol-analysis based. On the other hand, there is a range of OTHER
settings you DO need to configure to tune for performance that is related to
the protocol-analysis engine. For example, you might want to consider
changing the size of certain hash-tables.
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com
- Previous message: Gary Golomb: "RE: Comparing the performance of two IDS products with different architectures"
- In reply to: Gary Golomb: "RE: Comparing the performance of two IDS products with different architectures"
- Next in thread: RJ H: "RE: Comparing the performance of two IDS products with different architectures"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]
Relevant Pages
|
