[fw-wiz] Re: Flawed Surveys [was: VPN endpoints] (Paul D. Robertson)

From: Abe Singer (abe_at_sdsc.edu)
Date: 09/01/04

  • Next message: Stailey, Mike: "RE: [fw-wiz] Re: Flawed Surveys [was: VPN endpoints]" 
    To: firewall-wizards@honor.icsalabs.com
Date: Wed, 1 Sep 2004 13:52:39 -0700

    > Date: Wed, 1 Sep 2004 16:16:38 -0400 (EDT)
    > From: "Paul D. Robertson" <paul@compuwar.net>
    >
    > I'm not saying "Let's base everything we can on surveys!" I'm saying that
    > survey data can be useful, and you can improve the usefulness of that data
    > by throwing out the obviously bad data (ooutliers) and by checking against
    > the data you do have.

    How about instead of continuing the "my idea is less f*ck3d than
    *your* idea, there be a more productive discussion of what some good
    methodologies would be for identifying, collecting, and analysing data
    to produce metrics.

    * If you are going to do a survey, how do you target/vet respondents?
    What questions do you ask. What controls do you have in place?

    * If you collect incident data, financial data, etc., what data, and how
    do you validate it?

    * What do you do with all this data once you collect it? What sort of
    analysis? How do you calculate amount of error? How do you account
    for missing data? How do you interpret the results of your analysis?

    Maybe you'll never get the data you need, or it will cost to much to get it,
    but you won't really know that until you can say what it actually is.

    It *would* be really useful to have some truly meaningful measurements.
    It could do a lot to reduce the amount of snake-oil and magic security dust
    beings sold.
    _______________________________________________
    firewall-wizards mailing list
    firewall-wizards@honor.icsalabs.com
    http://honor.icsalabs.com/mailman/listinfo/firewall-wizards

  • Next message: Stailey, Mike: "RE: [fw-wiz] Re: Flawed Surveys [was: VPN endpoints]"