RE: Belaboring the point of FPs (haha!)

• Previous message: Evans, Arian: "RE: Network IDS, or IPS, or Proxy?"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]

Date: Tue, 19 Aug 2003 11:45:17 -0400
To: "Paul Schmehl" <pauls@utdallas.edu>, <focus-ids@securityfocus.com>

Yes, this is the perfect platform to spin a sales pitch. I'm surprised
it took this long for someone else to jump in the fray with said
intentions... However there's a bigger issue here, and not something new
either. I remember spouting-off a bit about this a couple of years ago
on this list...

That is, the QUALITY and INTEGRITY of the signatures being written. No
matter who the IDS vendor is (ISS, Sourcefire, Enterasys, or otherwise)
and no matter what kind of "restaurant you order at," if the signatures
are poorly written, you're going to *increase* your chance of false
positives ***AND*** false negatives.

In the case below, we see an example of a signature increasing the
chance of false positives. The signature is a good idea and a great way
to generically detect a compromised system via looking for a Windows
shell, but it should definitely be tuned. Banners like this appear at
the beginning of a session and payload - not in the middle. Maybe a
"Banner Field" protocol decoder can be developed so this problem doesn't
affect trons. I could show you just as many examples of bad signatures
increasing your chance of getting false negatives because of the same
simple mistakes. And these examples are MUCH more interesting!

On the flip side, traffic changes, so signature tuning is an evolving
process. In addition to that, what might be suspicious in one
environment, might not be in another. That's why the ability to fully
edit **ALL** signatures from your vendor is imperative. Detection
mechanisms are developed around the most common forms of an attack, but
depending on the protocols and custom applications in use on your
network, there could be an annoying overlap (read: false positive)
between a common attack profile, and normal traffic for you. That's why
not being able to fully view and edit signatures from your vendor can
quickly become a pain. This takes us down a whole new path, but I'm
trying to keep this email under 15 pages...

Anyways, this is the one thing that cracks me up about industry analysis
and testing. There's tons of focus on performance, stability, look and
feel, protocol decoding, etc. in IDS tests being published. But what
about the reason you buy an IDS in the first place??? Who's *really*
testing how well an IDS can detect the attacks it says it will. Not the
decoders and the normalizers, I mean the signatures. Surprisingly VERY
few, and everyone else just blindly accepts that. My hat's off to Blade
Software for raising this issue in the first place.

At any rate, whether it's Bob, or Marty, or whoever... If you make
mistakes writing signatures, then it's not the IDSs fault - it's the
signature's fault. Protocol decoding (like in Bob's example below) and
all these other sales pitches we've been getting on this list recently
are band-aids for this problem. And think about what it means to
higher-layer analysis (ie: correlation) methodologies that need to make
decisions based on data that was flagged by bad signatures in the first
place... Nice, huh?

Just some off-the-cuff thoughts...

-gary

> -----Original Message-----
> From: Graham, Robert (ISS Atlanta) [mailto:rgraham@iss.net]
> Sent: Friday, August 15, 2003 2:25 PM
> To: Paul Schmehl; focus-ids@securityfocus.com
> Subject: RE: Belaboring the point of FPs
>
> There is a lot of good points Marty makes about how Snort gives you
the
> exact results that you ask for.
>
> I don't mean to take this analogy too far, but I'd like to point out
the
> stories of genies-in-bottles who give you exactly what you ask for,
but
> not quite what you want. We all know the story of king Midas who
wished
> that everything he touched turned to gold -- but then realized the
folly
> of his ways when he hugged his daughter, which turned her to gold.
>
> I'm not saying that this is a BIG problem for Snort, please don't read
> too much into the analogy. It is indeed a good thing that Snort gives
> you what you ask for -- I'm just trying to point that it isn't 100%
> good.
>
> The basic problem is that the Snort rules language is not expressive
> enough to give you what you want. It's like going to a French
restaurant
> and ordering only those things on the menu that you can pronounce. I
> have friends that order bagels at the morning because they are
> embarrassed by their poor pronunciation of the French word
"croissant".
> It's true that the coffee shop is not at fault for delivering what the
> customer asked for, but it doesn't mean the customer is completely
happy
> with his bagel.
>
> I write lots of signatures for my IDS (RealSecure 7). I have written a
> clone of Snort (Trons). Most of the signatures that I write cannot be
> expressed in the Snort rules language. For example, I put an IMAP
> protocol-decode on port 143 that explicitly recognizes what an e-mail
> message is, and therefore won't match any patterns inside it (unless,
of
> course, those patterns are supposed to be for e-mail messages).
>
> You could certainly extend the Snort rules languages with plugins. The
> 'uricontent' keyword is a good example of a limitation with
> pattern-matching that had to be resolved. You could certainly add a
> plugin for IMAP that resolves the false-positive discussed below, but
> the issue is that nobody has. Such problems can easily be solved
within
> the Snort architecture, it's just that when you get Snort today, such
> problems are not solved.
>
> Again, I'd like to point out that when you use my IDS, you'll get a
set
> of signatures that I wanted to give you. You can certainly add your
own
> with the Trons feature and other "protocol-field" capabilities we give
> you, and you can sometimes adjust the signatures, but you DON'T have
the
> complete ability (like Snort) to arbitrarily change the signatures
that
> I wrote for you. As you can expect, since I wrote the signatures in a
> specific way, I believe that you'll get what you 'want' better out of
my
> IDS than Snort, but it's certainly true that you have less ability to
> 'ask' my IDS to do something slightly different.
>
>
> -----Original Message-----
> From: Paul Schmehl [mailto:pauls@utdallas.edu]
> Sent: Monday, August 11, 2003 10:29 PM
> To: focus-ids@securityfocus.com
> Subject: Belaboring the point of FPs
>
>
> Marty, I'm not picking on you, honest I'm not. I'm sitting here at
> home,
> monitoring our DMZ snort, waiting for the RPC worm to hit, and sure
> enough,
> I get a hit on sid 2123 - successful admin, cmd.exe. So I think, yep,
> there's the first box to get infected.
>
> Here's rule 2123:
> alert tcp $HOME_NET !21:23 -> $EXTERNAL_NET any (msg:"ATTACK-RESPONSES
> Microsoft cmd.exe banner"; flow:from_server,established;
> content:"Microsoft
> Windows"; content:"(C) Copyright 1985-"; distance:0;
content:"Microsoft
> Corp."; distance:0; reference:nessus,11633;
classtype:successful-admin;
> sid:2123; rev:1;)
>
> Looks good, but....analysis of the three packets shows very quickly
that
>
> it's an FP. The traffic is *from* our imap server on port 143 *to* an
> off
> campus site. Right direction, wrong alert. The payload? A bugtraq
> post
> someone was reading about the worm. I recognized it right away,
because
> I
> had just read the same post myself. (No, the off campus address was
not
>
> me.)
>
> An anomaly? Not really. I see these *every* time some new exploit
> shows
> up. List traffic triggers alerts, because the attack ports are either
> not
> specified or by default include mail ports (POP3, IMAP and SMTP). Now
> surely you will admit *those* are false positives?
>
> Here's the payload (yeah, I know, more alerts :( ):
>
> 000 : 2A 20 36 39 31 36 20 46 45 54 43 48 20 28 46 4C * 6916 FETCH
(FL
> 010 : 41 47 53 20 28 5C 53 65 65 6E 29 20 42 4F 44 59 AGS (\Seen)
BODY
> 020 : 5B 31 5D 20 7B 33 32 32 38 7D 0D 0A 0D 0A 6D 75 [1]
{3228}....mu
> 030 : 6C 74 69 74 68 72 65 61 64 69 6E 67 20 26 6F 73 ltithreading
&os
> 040 : 20 64 65 74 65 63 74 69 6F 6E 20 26 26 20 6D 61 detection &&
ma
> 050 : 63 72 6F 73 20 73 75 70 70 6F 72 74 2E 2E 2E 0D cros
support....
> 060 : 0A 0D 0A 65 78 70 6C 6F 69 74 20 63 61 6E 20 62 ...exploit can
b
> 070 : 65 20 66 6F 75 6E 64 20 68 65 72 65 3A 20 20 77 e found here:
w
> 080 : 77 77 2E 63 72 6F 75 6C 64 65 72 2E 63 6F 6D 2F
ww.croulder.com/
> 090 : 68 61 78 6F 72 63 69 74 6F 73 2F 6B 61 68 74 32
haxorcitos/kaht2
> 0a0 : 2E 7A 69 70 0D 0A 0D 0A 0D 0A 65 78 61 6D 70 6C
.zip......exampl
> 0b0 : 65 3A 20 4B 61 48 54 2E 65 78 65 20 31 30 2E 31 e: KaHT.exe
10.1
> 0c0 : 30 2E 34 30 2E 30 20 31 30 2E 31 30 2E 32 35 35 0.40.0
10.10.255
> 0d0 : 2E 32 35 35 20 33 30 30 0D 0A 5F 5F 5F 5F 5F 5F .255
300..______
> 0e0 : 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F
________________
> 0f0 : 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F
________________
> 100 : 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 0D 0A 20 20 20 ___________..
> 110 : 20 20 20 20 20 20 20 20 4B 41 48 54 20 49 49 20 KAHT
II
> 120 : 2D 20 4D 41 53 53 49 56 45 20 52 50 43 20 45 58 - MASSIVE RPC
EX
> 130 : 50 4C 4F 49 54 0D 0A 20 20 44 43 4F 4D 20 52 50 PLOIT.. DCOM
RP
> 140 : 43 20 65 78 70 6C 6F 69 74 2E 20 4D 6F 64 69 66 C exploit.
Modif
> 150 : 69 65 64 20 62 79 20 61 54 34 72 40 33 77 64 65 ied by
aT4r@3wde
> 160 : 73 69 67 6E 2E 65 73 0D 0A 20 20 23 68 61 78 6F sign.es..
#haxo
> 170 : 72 63 69 74 6F 73 20 26 26 20 23 6C 6F 63 61 6C rcitos &&
#local
> 180 : 68 6F 73 74 20 20 40 45 66 6E 65 74 20 4F 77 6E host @Efnet
Own
> 190 : 7A 20 79 6F 75 21 21 21 0D 0A 5F 5F 5F 5F 5F 5F z
you!!!..______
> 1a0 : 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F
________________
> 1b0 : 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F
________________
> 1c0 : 5F 5F 5F 5F 5F 5F 5F 5F 5F 5F 0D 0A 0D 0A 20 5B __________....
[
> 1d0 : 2B 5D 20 54 61 72 67 65 74 73 3A 20 31 30 2E 31 +] Targets:
10.1
> 1e0 : 30 2E 34 30 2E 30 2D 31 30 2E 31 30 2E 32 35 35
0.40.0-10.10.255
> 1f0 : 2E 32 35 35 20 77 69 74 68 20 33 30 30 20 54 68 .255 with 300
Th
> 200 : 72 65 61 64 73 0D 0A 20 5B 2B 5D 20 53 63 61 6E reads.. [+]
Scan
> 210 : 20 49 6E 20 50 72 6F 67 72 65 73 73 2E 2E 2E 0D In
Progress....
> 220 : 0A 2D 20 43 6F 6E 6E 65 63 74 69 6E 67 20 74 6F .- Connecting
to
> 230 : 20 31 30 2E 31 30 2E 34 30 2E 34 0D 0A 20 20 20 10.10.40.4..
> 240 : 53 65 6E 64 69 6E 67 20 45 78 70 6C 6F 69 74 20 Sending
Exploit
> 250 : 74 6F 20 61 20 5B 57 69 6E 32 6B 5D 20 53 65 72 to a [Win2k]
Ser
> 260 : 76 65 72 2E 2E 2E 2E 20 46 41 49 4C 45 44 0D 0A ver....
FAILED..
> 270 : 20 20 2D 20 43 6F 6E 6E 65 63 74 69 6E 67 20 74 - Connecting
t
> 280 : 6F 20 31 30 2E 31 30 2E 34 30 2E 39 0D 0A 20 20 o 10.10.40.9..
> 290 : 20 53 65 6E 64 69 6E 67 20 45 78 70 6C 6F 69 74 Sending
Exploit
> 2a0 : 20 74 6F 20 61 20 5B 57 69 6E 58 50 5D 20 53 65 to a [WinXP]
Se
> 2b0 : 72 76 65 72 2E 2E 2E 2E 20 46 41 49 4C 45 44 0D rver....
FAILED.
> 2c0 : 0A 20 20 2D 20 43 6F 6E 6E 65 63 74 69 6E 67 20 . -
Connecting
> 2d0 : 74 6F 20 31 30 2E 31 30 2E 34 30 2E 31 32 0D 0A to
10.10.40.12..
> 2e0 : 20 20 20 53 65 6E 64 69 6E 67 20 45 78 70 6C 6F Sending
Explo
> 2f0 : 69 74 20 74 6F 20 61 20 5B 57 69 6E 58 50 5D 20 it to a
[WinXP]
> 300 : 53 65 72 76 65 72 2E 2E 2E 2E 20 46 41 49 4C 45 Server....
FAILE
> 310 : 44 0D 0A 20 20 2D 20 43 6F 6E 6E 65 63 74 69 6E D.. -
Connectin
> 320 : 67 20 74 6F 20 31 30 2E 31 30 2E 34 30 2E 32 31 g to
10.10.40.21
> 330 : 0D 0A 20 20 20 53 65 6E 64 69 6E 67 20 45 78 70 .. Sending
Exp
> 340 : 6C 6F 69 74 20 74 6F 20 61 20 5B 57 69 6E 58 50 loit to a
[WinXP
> 350 : 5D 20 53 65 72 76 65 72 2E 2E 2E 0D 0A 20 2D 20 ] Server.....
-
> 360 : 43 6F 6E 65 63 74 61 6E 64 6F 20 63 6F 6E 20 6C Conectando con
l
> 370 : 61 20 53 68 65 6C 6C 20 52 65 6D 6F 74 61 2E 2E a Shell
Remota..
> 380 : 2E 0D 0A 0D 0A 4D 69 63 72 6F 73 6F 66 74 20 57 .....Microsoft
W
> 390 : 69 6E 64 6F 77 73 20 58 50 20 5B 56 65 72 73 69 indows XP
[Versi
> 3a0 : 3D 46 33 6E 20 35 2E 31 2E 32 36 30 30 5D 0D 0A =F3n
5.1.2600]..
> 3b0 : 28 43 29 20 43 6F 70 79 72 69 67 68 74 20 31 39 (C) Copyright
19
> 3c0 : 38 35 2D 32 30 30 31 20 4D 69 63 72 6F 73 6F 66 85-2001
Microsof
> 3d0 : 74 20 43 6F 72 70 2E 0D 0A 0D 0A 43 3A 5C 57 49 t
Corp.....C:\WI
> 3e0 : 4E 44 4F 57 53 5C 73 79 73 74 65 6D 33 32 3E 2E
NDOWS\system32>.
> 3f0 : 0D 0A 20 2D 20 43 6F 6E 6E 65 63 74 69 6F 6E 20 .. -
Connection
> 400 : 43 6C 6F 73 65 64 0D 0A 20 2D 20 43 6F 6E 6E 65 Closed.. -
Conne
> 410 : 63 74 69 6E 67 20 74 6F 20 31 30 2E 31 30 2E 34 cting to
10.10.4
> 420 : 30 2E 35 32 0D 0A 20 20 20 53 65 6E 64 69 6E 67 0.52..
Sending
> 430 : 20 45 78 70 6C 6F 69 74 20 74 6F 20 61 20 5B 57 Exploit to a
[W
> 440 : 69 6E 58 50 5D 20 53 65 72 76 65 72 2E 2E 2E 20 inXP]
Server...
> 450 : 46 41 49 4C 45 44 0D 0A 20 2E 20 2D 20 43 6F 6E FAILED.. . -
Con
> 460 : 6E 65 63 74 69 6E 67 20 74 6F 20 31 30 2E 31 30 necting to
10.10
> 470 : 2E 34 30 2E 35 30 0D 0A 20 20 20 53 65 6E 64 69 .40.50..
Sendi
> 480 : 6E 67 20 45 78 70 6C 6F 69 74 20 74 6F 20 61 20 ng Exploit to
a
> 490 : 5B 57 69 6E 32 6B 5D 20 53 65 72 76 65 72 2E 2E [Win2k]
Server..
> 4a0 : 2E 0D 0A 20 2D 20 43 6F 6E 65 63 74 61 6E 64 6F ... -
Conectando
> 4b0 : 20 63 6F 6E 20 6C 61 20 53 68 65 6C 6C 20 52 65 con la Shell
Re
> 4c0 : 6D 6F 74 61 2E 2E 2E 0D 0A 0D 0A 4D 69 63 72 6F
mota.......Micro
> 4d0 : 73 6F 66 74 20 57 69 6E 64 6F 77 73 20 32 30 30 soft Windows
200
> 4e0 : 30 20 5B 56 65 72 73 69 3D 46 33 6E 20 35 2E 30 0 [Versi=F3n
5.0
> 4f0 : 30 2E 32 31 39 35 5D 0D 0A 28 43 29 20 43 6F 70 0.2195]..(C)
Cop
> 500 : 79 72 69 67 68 74 20 31 39 38 35 2D 32 30 30 30 yright
1985-2000
> 510 : 20 4D 69 63 72 6F 73 6F 66 74 20 43 6F 72 70 2E Microsoft
Corp.
> 520 : 0D 0A 0D 0A 43 3A 5C 57 49 4E 4E 54 5C 73 79 73
....C:\WINNT\sys
> 530 : 74 65 6D 33 32 3E 65 78 69 74 0D 0A 0D 0A 20 2D tem32>exit....
-
> 540 : 20 43 6F 6E 6E 65 63 74 69 6F 6E 20 43 6C 6F 73 Connection
Clos
> 550 : 65 64 0D 0A 20 2D 20 43 ed.. - C
>
> Paul Schmehl (pauls@utdallas.edu)
> Adjunct Information Security Officer
> The University of Texas at Dallas
> AVIEN Founding Member
> http://www.utdallas.edu
>

---------------------------------------------------------------------------
Attend Black Hat Briefings & Training Federal, September 29-30 (Training), October 1-2 (Briefings) in Tysons Corner, VA; the world’s premier
technical IT security event. Modeled after the famous Black Hat event in
Las Vegas! 6 tracks, 12 training sessions, top speakers and sponsors.
Symanetc is the Diamond sponsor. Early-bird registration ends September 6 Visit: www.blackhat.com
---------------------------------------------------------------------------

• Previous message: Evans, Arian: "RE: Network IDS, or IPS, or Proxy?"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]