On the definition of false positive - was: Re: location of an IPS

From: Evil Adam Smith (eviladamsmith_at_yahoo.com)
Date: 10/28/05

  • Next message: Dave Aitel: "Re: RPC Evasion techniques" 
    Date: Thu, 27 Oct 2005 22:34:41 -0700 (PDT)
To: focus-ids@securityfocus.com

    Hey List,

    Since Kurt obviously isn’t afraid to correct
    others...and I know at least one person on the list
    might also benefit from this comment...

    From Kurt's post below:
    "One the one hand good, that would have been a false
    positive technically speaking, otoh that's bad, it
    probably should have alerted on that (even if it is a
    false positive)."
     
    Actually, I believe it would be either a true or false
    negative - depending on how you defined the terms. In
    this example choose to use true.
    For example in the model I'm thinking of:
    A false positive is when an attack is detected
    (positive), but it wasn't a real attack (false) -
    whatever the reason the signature triggered falsely or
    some such.
    A true positive is when it was detected (positive) and
    it was a real attack (true).
    A false negative is when it wasn't detected (negative)
    and it wasn't a real attack (false) - you could test
    for false positives with false negatives (things the
    IPS shouldn't ever detect as malicious(valid
    traffic)).
    Thus, a true negative is a real attack(true) that goes
    undetected (negative).

    I guess Kurt was thinking intent of the attacker
    matters a la an alternative definition of "attack" but
    such a definition would be I believe untestable - how
    would IDSes, etc. ever be able to establish the intent
    of a packet? If I scan myself my ids either detected
    it or it did not.

    Semantic quibbles aside I don't see a more useful way
    to think about this problem area using only two sets
    of two terms and use them in a meaningful practical
    way.

    Cheers
    eviladamsmith

    >
    > "Kurt Seifried" <bt@seifried.org>
    > 10/19/2005 09:13 PM
    > Please respond to
    > "Kurt Seifried" <bt@seifried.org>
    >
    >
    > To
    > "Doug Fox" <dfox168@hotmail.com>,
    focus-ids@securityfocus.com
    > cc
    >
    > Subject
    > Re: location of an IPS
    >
    >
    >
    >
    >
    >
    > > I'm sorry for this dumb question, which may have
    been answered many
    > times.
    > >
    > > Where should one place an TippingPoint Unity 50
    IPS device? Behind or
    > in
    > > front of a firewall?
    >
    > Depends what you want to measure. Broadly speaking
    in front of the
    > firewall
    > means you're measuring attempts, behind the firewall
    they are penetrations
    >
    > (or do both and then compare them, that way you can
    actually tell
    > management
    > "look we're stoping 90% of detected attacks, now
    would you please let me
    > tighten the firewall rules so that's 100%?" or
    something). One thing to
    > remember is to look for outgoing attacks as well,
    that's a good indication
    >
    > of a compromised host or a hostile user.
    >
    > > I have a/the TippingPoint behind a Check Point
    firewall. Even though we
    > > externally and internally port-scanned the
    firewall and the IPS many
    > > times, the activity log did not contain any record
    of the "attacks".
    >
    > One the one hand good, that would have been a false
    positive technically
    > speaking, otoh that's bad, it probably should have
    alerted on that (even
    > if
    > it is a false positive). Sounds like you need to sit
    down and do the
    > setup/configuration/alerting/whatnot (aka the hard
    parts of IDS/IPS).
    > Broadly speaking you're saying "it's broken" to
    which I can only say
    > "bummer. try fixing it."
    >
    > > What am I missing here? Any pointers are
    appreciated.
    > >
    > > Thanks,
    >
    > The dreaded C word comes to mind (consultant), if
    your company lacks the
    > expertise to set this up buy someones time who does.
    >
    > -Kurt
    >

                    
    __________________________________
    Yahoo! FareChase: Search multiple travel sites in one click.
    http://farechase.yahoo.com

    ------------------------------------------------------------------------
    Test Your IDS

    Is your IDS deployed correctly?
    Find out quickly and easily by testing it
    with real-world attacks from CORE IMPACT.
    Go to http://www.securityfocus.com/sponsor/CoreSecurity_focus-ids_040708
    to learn more.
    ------------------------------------------------------------------------

  • Next message: Dave Aitel: "Re: RPC Evasion techniques"

    Relevant Pages

    • RE: [Full-Disclosure] Sidewinder G2
      ... Secure Computing Sidewinder G2 Firewall Stops New High-Profile Sendmail ... Technology Prevents Sendmail Attack Warned About in CERT Advisory ...
      (Full-Disclosure)
    • RE: Thinking about Security rules...
      ... > Subject: Re: Thinking about Security rules... ... >>rules for the IDS. ... by which you attack. ... firewalls in series isn't nearly as nice as a stateful firewall coupled ...
      (Vuln-Dev)
    • Re: Can I protect myself against network attacks?
      ... > I guess that was one purpose of the attack. ... > had happened if you just used the SP2 firewall which does not warn you ... back, I've seen the firewall crash before my eyes, without warning. ... network attacks, or trojans. ...
      (comp.security.firewalls)
    • Re: I was hacked
      ... Only me noticing that the requests seemed to come from a LAN? ... To secure IIS somewhat, remove all the virtual directories even if they are ... > Do you have some kind of application level firewall on this machine? ... a series of attempts to attack IIS that the IIS log claimed were coming ...
      (microsoft.public.inetserver.iis.security)
    • Re: Firewall security: Re: Problems with simple Samba file share
      ... >>million doesn't change my action of deploying a firewall ONCE. ... They keys can be obtained ... > What I suspect is that you think a special attack will be developed ... the firewall helps protect us. ...
      (comp.os.linux.misc)