RE: ICSA [WAS: Re: Intrusion Prevention]

Date: 12/30/02

  • Next message: Greg Shipley: "RE: ICSA [WAS: Re: Intrusion Prevention]"
    Date: Mon, 30 Dec 2002 16:29:39 -0500

    >On 12/29/02 Greg Shipley wrote:
    >Over the past six years Neohapsis Labs has been testing products in the
    >security space, with the vast majority of our results appearing in Network
    >Computing magazine. Year after year we learn from our successes, and
    >mistakes, and roll that knowledge into our ever evolving testing
    >methodologies. We tend to be leaders in this regard. For example. the
    >careful reader will note that our documented testing methods in 1999
    >weren't mirrored by others until around 2001, and that our present-day
    >methods are quite a bit beyond what anyone else has done, to-date.

    All - I have remained silent on this list for years. I am interested in
    helping mature an Industry. That is what ICSA Labs does and IDS has been one
    of my responsibilities since early 1999. After cutting through the stinging
    criticism and saber rattling, I have chosen to respond only to the paragraph
    above. Any vendor that knows the ICSA Labs testing methodology knows that
    for over ten years we have perfected pass/fail certification testing with
    evolving test methodology and criteria. We did this when everyone else
    argued that it was the wrong approach. This is the standard, and it is in
    fact the ICSA Labs approach that has been mirrored by other test labs.

    >On 1/18/01 Greg Shipley wrote:
    >[edit] Don't get me wrong, I think there is a huge need for 3rd-party
    involvement, and dare I say it, "certification."
    >IMHO, there are some fronts to this that are REALLY important on. For
    example, I've heard that the ICSA team is working on >IPSEC *compliance* and
    interoperability testing. Ok, that's huge, as anyone who has worked with
    multi-vendor VPN
    >deployments knows that the VPN space is a mess on that front.

    >The problem is, I question whether or not people are being mislead, and how
    much good some of these certifications (like the >firewall one) really do.
    Ultimately, does this type of "branding" help provide for a false sense of
    security? [end]

    The problem, clearly stated by Greg, is whether people are being misled. The
    answer is emphatically NO. The ICSA Labs NIDS test is geared toward three
    different network types. ICSA Labs has never mirrored the 1999 Neohapsis
    test, nor will we - it was flawed. We have built a real network to test
    NIDS. We have always used working exploits that are targeting a victim
    machine that is vulnerable to each specific attack. We have also included
    the first false positive test...ever. You may be a bit beyond, however, your
    F-1 vs. Garbage Truck analogy reminds me of the tortoise and the hare. You
    may have gone farther in terms of performance but you yourself have admitted
    errors caused by the pace. This is where people have been misled. They read
    a magazine article that states vendor x has the best NIDS. End-users do not
    need to know who has the best product in a snap-shot-in-time lab test, they
    need to know the best product for their real live environment. That is where
    ICSA Labs NIDS testing and certification has excelled and IMNSHO will never
    be caught.

    Greg - I sincerely ask you to contact me off-line and discuss a possible
    visit to the ICSA labs. It is evident by your post that you do not have a
    complete knowledge of what we do. This thread has also included reference to
    the ICSA Labs Firewall program. I have asked one of our most vocal critics
    in the past to give you his opinion on the current state of the ICSA Labs
    Firewall program. Look for a post in the near future on that subject.

    Scott Markle
    IDS Program Manager
    ICSA Labs

    This message is intended only for the use of the intended recipient and
    may contain information that is PRIVILEGED and/or CONFIDENTIAL. If you
    are not the intended recipient, you are hereby notified that any use,
    dissemination, disclosure or copying of this communication is strictly
    prohibited. If you have received this communication in error, please
    destroy all copies of this message and its attachments and notify us

    Relevant Pages

    • Re: Sandboxing
      ... Linux (already working in the labs) and Solaris are on deck for an upcoming release. ... > I was wanting to get the advice and feedback from the community on ... > case of a compromise, ... > practical in large network environments. ...
    • Re: which ip scheme i have to prefer?
      ... The simplest way to isolate the the two labs from the rest of the network ... Internet and other machines on your network. ... LAN machines ...
    • network connectivity icon switch available in Sysprep?
      ... I enable the "Show icon in notification area when connected" in Network ... properties in my labs as an immediate and simple troubleshooting tool (eg. ... Sysprep is first run as part of our image. ...
    • Computers talking in their sleep
      ... I have two labs full of systems that when powered off (power going to ... system but the system is not booted) are sending tons of network ... and can't find anything in there pertaining to the network adapter ...