Re: [fw-wiz] PCI DSS & Firewalls

Chris Blask wrote:
having more Pen Testing done in the world is itself a move in a positive direction, so that's a good thing by any metric.

I disagree.

What does pen testing show?? Pen testing can show one of two things:
- your security sucks
- your security is better than your pen tester

Neither of those two determinations are equal to "your security is

Ultimately, any kind of "security proofs" attempt to prove a negative:
i.e: "there are no security holes" and simple logic tells us that you
can't prove a negative.

The reason pen testing is popular - in spite of the fact that it
is a flawed idea - is because "your security sucks" is still a
useful answer for a lot of organizations. I'd go a step further
and suggest that if the answer is "your security sucks" there's
a root cause and it's that "your managers are stupid" or "your
executive management is clueless" or both. Those are not especially
popular results but we both know of infinite numbers of stories of
executives who didn't take security seriously until some pen
test rubbed their nose in it. Pen testing may be a short-term
cure for stupid, but it's a fairly expensive way of doing
it and I doubt that it works particularly well in the long-term.

If we were to ever move security past the "your security sucks"
stage, it would have to result from systems being designed with
security built in from the ground up, rather than bolted on
(or, more likely, as the case is, stuck on with bubble gum
and duct tape) after it's safely too late. Don't worry about
that happening any time soon, though - Web2.0 and cloud
computing are in the process of blowing a gigantic smoking
hole through any notion of trust in computing. How do you
make a statement about assurance and critical data in an
environment where, by design, you aren't to know anything beyond
"it's in our cloud; trust us" ?? I am guessing that the
pen testers are already drooling at the feast to come.

As they used to say, "you can't make a silk purse out of
a sow's ear" - implying that there's no amount of improvement
that you can make to something that just isn't capable of
meeting your expectations. The same applies to pen testing:
it is impossible to badness-test your security into being good.
If you try, all you'll find is that it's expensive. It's
only a coincidence, I'm sure, that the badness-testers are
standing by. There are also duct tape and bubblegum sellers
standing by. Its all coincidence.

So, generally I disagree with you, Chris. I think pen testing
serves as an indicator of stupid more than anything else.
Don't be confused by the fact that the indicator is in the
red zone; it doesn't mean what you think it does.

Marcus J. Ranum CSO, Tenable Network Security, Inc.
firewall-wizards mailing list

Relevant Pages

  • Re: Evaluating Pen Testers
    ... into the cyber security realm because it was a way to prove problems ... pen testing is still needed but much less so these days as people ARE ... only until the next pen tester or next 0day comes along. ... necessarily also a member of OWASP. ...
  • Re: Need Some Guidance Please
    ... without employees knowing it, he did it with the permission of the CEO ... I started taking Information Security Courses at the ... EVERYONE in my classes work in some sort of Computer Security Field ... I have done some research on Pen Testing and this seems to be ...
  • RE: Government Compliance
    ... Pen testing is indeed just that a sanity check to verify that the overall ... Subject: Government Compliance ... I currently work as an IT Security professional ... Department was adopting for penetration testing. ...
  • Re: Windows Mobile 5.0
    ... Please tell me how to approach pen testing these kind of devices ... Concerned about Web Application Security? ... As attacks through web applications continue to rise, ... Download FREE whitepaper on how a managed service can ...
  • Re: Kernel 2.6.11
    ... >>testers so don't shy away from upstream sources because of some rumor ... > 2.6 is shipping with known security holes. ... > developer to stand up and disagree but not one of them did. ...