Re: [fw-wiz] VPN endpoints

From: Paul D. Robertson (
Date: 09/01/04

  • Next message: Al Cooper: "[fw-wiz] Cisco VPN Client Behind a Cisco PIX or Router"
    To: "Marcus J. Ranum" <>
    Date: Wed, 1 Sep 2004 07:41:40 -0400 (EDT)

    On Mon, 30 Aug 2004, Marcus J. Ranum wrote:

    > or the CIO magazine survey on security) - a lot of these surveys are
    > fundamentally flawed. They yield results but it's hard to say what the
    > results actually _measured_.

    So long as they're flawed approximately the same way from survey to
    survey, they're often both "better than nothing[1]" and a good relative
    metric. We often don't need absolute metrics, relative metrics will do
    just fine. I know what my $foo risk was last year, and I know what it was
    the year before, and I can compare to the survey and see the relative
    differences and the relative change- therefore, I can figure out my
    approximate relative change for this year.

    > Specifically, many security surveys are based on self-selected
    > samples (e.g: "polls"). When you do a poll, what you're doing is
    > asking "Please fill this out." But there are a lot of assumptions
    > that get dropped on the floor. :( What you're really measuring is:
    > - How much the person cared about the topic (motive to respond)
    > - How honest the respondent is (hard to verify)
    > - Other factors (hard to predict)

    You can also (a) drop outliers, (b) have cross-conflicting questions, and
    (c) answer the questions on behalf of a known quantity and still be able
    to validate polls pretty well. You obviously don't get people who don't
    care to respond, but if the number of people who do respond is
    significant, that's ok.

    > I'm sure nobody on this list has ever filled out one of those surveys
    > from a magazine in which they asked you your job position, whether
    > you were a decision-maker, company size, etc... And I'm sure you
    > all fill them out EXACTLY right. I used to enjoy periodically asserting
    > that I was the CEO of a 1 person company with a $4,000,000 IT
    > budget (well, a guy can dream, huh?) Unfortunately, sometimes

    You're out of the range of the mean by orders of magnitude, anyone doing
    it even half-way should be throwing that response away (assuming they
    *want* correct data,) which in that case is only half-right- better
    qualified leads should be worth more, but either fudging is built into the
    pricing model, you got sold cheaper, they didn't care, or someone got
    ripped off.

    [1] That doesn't mean they aren't often worse than nothing, just that they
    can be useful. Just like assessing risk actually- same rules apply.
    Paul D. Robertson "My statements in this message are personal opinions which may have no basis whatsoever in fact." Director of Risk Assessment TruSecure Corporation
    firewall-wizards mailing list

  • Next message: Al Cooper: "[fw-wiz] Cisco VPN Client Behind a Cisco PIX or Router"