RE: False Positives with IntruVert

From: Bill Boyle (bboyle@intruvert.com)
Date: 04/09/03

  • Next message: Peteris Krumins: "Developing IDS"
    From: Bill Boyle <bboyle@intruvert.com>
    To: "'pauls@utdallas.edu'" <pauls@utdallas.edu>, "'focus-ids@securityfocus.com'" <focus-ids@securityfocus.com>
    Date: Tue, 8 Apr 2003 17:35:33 -0700 
    
    

    To make it easier to follow, I have extracted your points and responded in
    line. Once again, these are my views and may/may not be those of my
    company.

    --bill

    -----Original Message-----
    From: Paul Schmehl [mailto:pauls@utdallas.edu]
    Sent: Monday, March 31, 2003 5:30 PM
    To: Alan Shimel
    Cc: Bill Boyle; focus-ids@securityfocus.com
    Subject: RE: False Positives with IntruVert

    On Sat, 2003-03-29 at 10:52, Alan Shimel wrote:
    Since we're now into the congratulatory mode, why don't I address some
    of Bill's points?

    If we *knew* there would never be a false positive, then the answer
    would obviously be we should. Of course we should. The problem is we
    don't *know* that, and a vendor saying it's true isn't exactly a ringing
    endorsement. (No offense, but salesman aren't exactly a good source for
    technical information like, "show me why you can say there are 0 false
    positives.)

    BOYLE--------------------\
    You are right. There has to be trust because the average admin will have
    neither the time nor the skill to assess properly the entire signature
    database. If they did, they could write their own IDS/IPS. :)

    Have you reversed engineered your firewall code? Have you analyzed your
    antivirus signatures? When you get a warning that an attachment is infected
    with Nimda, how do you respond? The fact that the trust has to be earned is
    a different statement than IPS is not functional or not worth time or money.
    I am sure there are plenty of readers who would have saved a large chunk of
    change and time if they blocked code red or Nimda.
    BOYLE--------------------/

    No matter how important security becomes to an institution, it will
    never trump availability. After all, the purpose of a network isn't to
    prevent attacks, it's to get work done. If preventing attacks means
    preventing work from being done, then we're into the risk analysis
    arena, weighing the odds of attack against the need for the service.

    For example, we elected four years ago to block email that had
    attachments with certain "dangerous" extensions (such as .exe). However
    we did not block attachments with .doc or .xls, because even though they
    can also carry viruses, the business need for availability was greater
    (in our judgment) than the need to protect us at the gateway.

    BOYLE----------------------\
    Great statement! The purpose of security is to ensure availability. I
    agree with the risk analysis statement--that is exactly my point. However,
    when I read your post I am confused. You state that risk analysis is
    valuable but I still see a blanket statement from you that IPS should not be
    used because it might block something valid.

    Using your example applied to IPS, you are saying allow ALL attachments with
    .doc and .exe because denying them MIGHT hinder productivity. Yet, you have
    done risk analysis and determined that you are better off blocking .exe.
    Risk Analysis. SELECTIVELY blocking attacks with an IPS will not only
    minimize your risk of a false positive denying a service, it will IMPROVE
    uptime by keeping the malicious traffic off of the wire. In fact, the
    multiple signatures of IPS permits a GREATER level of accuracy than simply
    =.exe. For example: Code Red could wreak havoc on your network. With risk
    analysis, I would feel pretty safe dropping packets with ida in the URI Path
    and
    \x90\x90\x68\x58\xcb\xd3\x78\x01\x90\x90\x68\x58\xcb\xd3\x78\x01\x90\x90\x68
    \x58\xcb\xd3\x78\x01\x90\x90\x90\x90\x81\x90\xc3\x03\x8b\x00\x53\x1b\x53\xff
    \x78\x00\x25\x75\x30\x30 in the request.
    BOYLE------------------/

    "More accurately" does not mean 100% however.

    BOYLE-------------------\
    Correct. Nothing in totality is 100% but subsets of IPS can be, such as
    individual attacks. Reiterating my previous post--you would not set EVERY
    attack to block. You would choose attacks that are less likely to false
    positive. Regarding a good point from Brian Lang's email: How does the
    average admin know which signatures will false positive and which will not?
    Hopefully, the IPS vendor (who DOES know the attack inside out and has
    profiled the attacks (signature or anomaly or combination of both)) has
    rated the degree which the attack may be a false positive. It helps the
    admin make the decision.
    BOYLE-------------------/

    Why? An IPS that does no blocking is an IDS by another name.

    BOYLE---------------------\
    I would agree that before you can block, you must accurately detect. What
    might clarify my statement is that waiting for a product with 0 false
    positives is the wrong approach. There are other factors that I feel are
    more critical and, judging by your post, you agree with at least the
    availability portion. It is the accurate subset of attacks that makes IPS a
    useful solution today.
    BOYLE---------------------/

    And that is a *huge* consideration for most networks. A single point of
    failure for your *entire* Internet pipe is a *big* concern. (Yes, you
    can use load balancing and redundancy to overcome that problem, but at a
    much higher cost.)

    BOYLE----------------------\
    Risk Analysis. A single router or switch or firewall or... is a single
    point of failure as well. I guess what I am saying is that you are not
    adding a SPOF to an environment that does not have it already. Most
    companies already have several single points of failure. IPS can be run in
    High Availability mode for those environments that necessitate that level of
    redundancy. If you are calling stability into question, there are IPS
    solutions out there that are just as stable as the gear listed above.
    BOYLE-----------------------/
     
    A NIDS sniffing the wire poses no threat to connectivity.

    BOYLE------------------------\
    Yes. On the contrary, a NIDS cannot prevent all the other things on that
    wire that do pose a threat to connectivity. Your line of thinking would
    eliminate value-adding ACLs, firewalls, antivirus, etc.
    BOYLE------------------------/

    For IDS, but not for blocking - unless you are willing to accept the
    consequences of degraded availability. Again, we're in to risk
    analysis.

    BOYLE--------------------------\
    "Degraded availability"...I would make the argument that the protection an
    IPS provides would IMPROVE availability. I do agree with the statement
    regarding the more generic signatures are more appropriate for IDS than IPS.
    It is not all or nothing. I would configure those generic attacks to alert
    only because they will require grey-matter to be certain.
    BOYLE--------------------------/

    Out of curiosity, what alerts *would* you block and why?

    BOYLE---------------------------\
    That is a tough question to answer because one could not say "buffer
    overflows" or "worms" or "backdoors". For the same reasons, I could not
    even say individual attacks like "code red" or "back orifice". Each of
    these might have multiple signatures to detect on the request, response,
    probe, propagation, etc. It might use an anomaly algorithm instead of a
    signature. It may vary by vendor.

    My approach would be as follows:
    Run the device in detection-only to tune the box to eliminate any glaring
    false positives.

    After it has been in my network for an appropriate amount of time (mileage
    may vary based upon the dynamics of traffic), I would put it in-line not
    blocking anything.

    I would let it run in this fashion for a while to avoid any finger pointing
    for every little thing that goes wrong on a daily basis.

    Meanwhile, I would begin to choose what attacks I would begin to block.
    Severity will be an important factor for the first round. In a Microsoft
    environment, I would probably start with more recent very well known attacks
    such as code red, nimda, slammer, fbi top 20 attacks, etc. I would pay
    attention to vendor notes--hopefully there is a rating on the chance the
    attack signature will be a false positive. Those deemed good candidates, I
    would read up on some of them using CVE references, bugtraq, Microsoft Tech
    Bulletins, etc. to determine my exposure.

    Once I have determined high severity attacks that are applicable to my
    environment, I would assess how I would apply policy.
            Which devices does this particular signature affect? If only
    Microsoft then only block if destined to/from a MS box. Ignore or alert
    only for Unix boxes.
            Do I care about attacks originating inside my environment? Maybe it
    is too political to block traffic outbound so I will only drop on inbound
    malicious traffic.
            Do I only want to block traffic to my key web server and mail
    server? Personally protect devices I am responsible for without causing
    political issues with the development and engineering departments.
            Are there key segments/IPs that I need to ignore? Risk Analysis.
            
    I would apply the blocking rules slowly to aid in troubleshooting should an
    application stop working. It may not even be because of the IPS but it is
    much easier to determine that the telnet problem that arose last night is
    not your issue if all you added was a mssql rule.

    I would watch the console and look at the captured packet data of the
    blocked traffic to help determine whether the traffic is a false positive or
    maybe to prove that no action was taken by the IPS and the above telnet
    issue was related to something other than IPS.
            
    Some people may see value in just permitting everything until the next
    worm/vulnerability and will add the signature after the attack to help
    contain their exposure.

    BOYLE------------------------------/

    -- 
    Paul Schmehl (pauls@utdallas.edu)
    Adjunct Information Security Officer
    The University of Texas at Dallas
    http://www.utdallas.edu/~pauls/
    AVIEN Founding Member
    -----Original Message-----
    From: Paul Schmehl [mailto:pauls@utdallas.edu] 
    Sent: Monday, March 31, 2003 5:30 PM
    To: Alan Shimel
    Cc: Bill Boyle; focus-ids@securityfocus.com
    Subject: RE: False Positives with IntruVert
    On Sat, 2003-03-29 at 10:52, Alan Shimel wrote:
    > Bill
    > 
    > Its not often that I agree with the competition, but your email was one
    > of the most constructive, right on defense for IPS that I have seen.
    > Nice points!
    > 
    Since we're now into the congratulatory mode, why don't I address some
    of Bill's points?
    > 
    > -----Original Message-----
    > From: Bill Boyle [mailto:bboyle@intruvert.com] 
    > Sent: Friday, March 28, 2003 1:18 PM
    > To: 'focus-ids@securityfocus.com'
    > Subject: RE: False Positives with IntruVert 
    > 
    > There are clearly attacks that will 100% not false positive.  My
    > question to
    > the community is: Why would you NOT block this traffic on your network?
    If we *knew* there would never be a false positive, then the answer
    would obviously be we should.  Of course we should.  The problem is we
    don't *know* that, and a vendor saying it's true isn't exactly a ringing
    endorsement.  (No offense, but salesman aren't exactly a good source for
    technical information like, "show me why you can say there are 0 false
    positives.)
    No matter how important security becomes to an institution, it will
    never trump availability.  After all, the purpose of a network isn't to
    prevent attacks, it's to get work done.  If preventing attacks means
    preventing work from being done, then we're into the risk analysis
    arena, weighing the odds of attack against the need for the service.
    For example, we elected four years ago to block email that had
    attachments with certain "dangerous" extensions (such as .exe).  However
    we did not block attachments with .doc or .xls, because even though they
    can also carry viruses, the business need for availability was greater
    (in our judgment) than the need to protect us at the gateway.
    > We
    > have all been burned with false positives and I am sure that is where
    > the
    > mistrust originates.  But, a lot of technology has been put forth to
    > more
    > accurately identify attacks since IDS's inception.  
    > 
    "More accurately" does not mean 100% however.
    > What I see as necessary for IPS to be useful TODAY is not zero false
    > positives.  It is performance, accuracy, and granularity of policy.
    > 
    Why?  An IPS that does no blocking is an IDS by another name.
    > PERFORMANCE:  First and foremost, the box sitting on the wire must
    > process
    > the packets at wire speed and not drop ANY packets. It must also have
    > little
    > downtime (same expectations as my switch and router or firewall
    > infrastructure).  
    > 
    And that is a *huge* consideration for most networks.  A single point of
    failure for your *entire* Internet pipe is a *big* concern.  (Yes, you
    can use load balancing and redundancy to overcome that problem, but at a
    much higher cost.)
    A NIDS sniffing the wire poses no threat to connectivity.
    > ACCURACY:  Exemplified in Paul Schmehl's email, a signature alone is not
    > always accurate enough.  Correlation must be used to enhance the
    > accuracy.
    > For example, Intruvert uses Boolean logic in its signature set to
    > provide
    > more accurate detection.  Example: sig#1 in URI request AND (sig2 OR
    > sig3 in
    > body).
    > 
    > Not every signature is 100% accurate but it does not mean it is not of
    > use.
    For IDS, but not for blocking - unless you are willing to accept the
    consequences of degraded availability.  Again, we're in to risk
    analysis.
    > Example: a TCP connection on port 5432.  It may be backoriface or may be
    > a
    > random source port that happened to be a well know backoriface port.  It
    > will be a false positive but one that I would want to review.  The fact
    > that
    > I got this alert does not mean my IPS is worthless, it just means that I
    > will not choose to automatically block this event.  Other backoriface
    > attacks that have a lower false positive rate will (at MY choosing-not
    > the
    > vendor's).  
    > 
    Out of curiosity, what alerts *would* you block and why?
    -- 
    Paul Schmehl (pauls@utdallas.edu)
    Adjunct Information Security Officer
    The University of Texas at Dallas
    http://www.utdallas.edu/~pauls/
    AVIEN Founding Member
    -----------------------------------------------------------
    ALERT: Exploiting Web Applications- A Step-by-Step Attack Analysis
    Learn why 70% of today's successful hacks involve Web Application
    attacks such as: SQL Injection, XSS, Cookie Manipulation and Parameter 
    Manipulation.
    http://www.spidynamics.com/mktg/webappsecurity71
    

  • Next message: Peteris Krumins: "Developing IDS"

    Relevant Pages

    • RE: False Positives with IntruVert
      ... right on defense for IPS that I have seen. ... Subject: False Positives with IntruVert ... There are clearly attacks that will 100% not false positive. ...
      (Focus-IDS)
    • RE: False Positives with IntruVert
      ... product but to put into perspective the CURRENT value of IPS. ... There are clearly attacks that will 100% not false positive. ... have all been burned with false positives and I am sure that is where the ... Subject: False Positives with IntruVert ...
      (Focus-IDS)
    • Re: ForeScout ActiveScout
      ... things are even (no bug or weird network issues), ... false positives COULD rarely occur with weird ... It is not a regular IDS or IPS and in no way comes to ... More complicated benefit is, it will catch attacks, new worms, etc. ...
      (Focus-IDS)
    • Re: False Positives with IntruVert
      ... > Looking for some feedback on IntruVert. ... > IntruVert in the lab and has been getting a lot of false positives on their ... They are afraid to put IntruVert into the IPS mode, ... > confidently use it and start blocking all attacks. ...
      (Focus-IDS)
    • Re: IPS/IDS behavior with ISIC/UDPSIC/TCPSIC/ICMPSIC traffic
      ... considered as an attack that need to be protected by IPS devices? ... ISIC generates many packets with different IP protocols. ... If you still see 100% CPU problem, you may like to check you log settings. ... with real-world attacks from CORE IMPACT. ...
      (Focus-IDS)