Re: Thou shalt have no other gods before the ANSI C standard

From: David Wagner (daw_at_taverner.cs.berkeley.edu)
Date: 02/23/05 

Date: Wed, 23 Feb 2005 18:49:09 +0000 (UTC)

Hank Oredson wrote:
>For adversarial testing the first thing you do is hand out
>your source code and your design specs and then you
>say "now break it in some significant manner please."

Agreed! This is a much better formulation of the problem. I like this
much better. It makes clear what the goal is, and leaves it up to the
security evaluator to use whatever tools are most effective at the tool
(rather than artificially constraining them to use testing, or any other
one method).

Other typical goals include "evaluate its security and assess the likely
risks", or "tell me what are the most important things I could be doing
to improve its security". These are more common than "break it, please",
in my experience, but all three are important.

Now you can ask, what is typically going to be the most effective method?
How much of the time is the reviewer going to spend on tasks like:
reading design specs? questioning developers? forming hypotheses about
possible failure modes? analyzing the threat model and security goals?
examining the test plan, quality processes, and prior security defenses
that have already been put in place? inspecting code? running automated
code-scanning tools? developing test cases and executing the program on
these inputs?

My guess is that, in a limited-time review done by a competent security
professional, the latter (writing and running test cases) is likely to
account for only a small fraction of the overall effort and overall value.
That's because often testing doesn't rank as high in terms of metrics
like "bugs (or surprises) found per hour spent" or "progress towards
evaluating the security of the system per hour spent".

To give a concrete example, here is an example of one security review I
have participated in:
  http://www.combex.com/papers/darpa-review/
Two of us spent about a week (ok, 4 days, but those were long days...)
evaluating the security of a system called E. We spent maybe 10-20% of
our time on writing and executing test cases. I think we were pretty
productive. If we had spent 80-90% of our time focused solely on testing,
I don't think we could have made nearly as much progress in that time.

To give another example, here is another security review I participated
in recently:
  http://www.servesecurityreport.org/
We spent about a week on evaluating the security of a voting system.
I doubt that we spent more than a couple of hours on testing or running
the system. We didn't need to.

Part of the point I'm making is that generally in a security review,
you should be starting with a more holistic, systems viewpoint. You can
make a lot more progress that way than by just diving into the code at
some random point. And testing is so low-level that it is almost never
where you should start.

Another part of the point I'm making is that the power of testing at
uncovering new security holes (that you didn't already know about) is
very limited. I have found it more common that I identify potential
security defects in some other way, and then I use testing to confirm my
hypothesis. Testing just has very limited power at inducing worst-case
behavior from the system; you need to figure out what the worst cases are
some other way, and that is exactly the hard part.

Relevant Pages

  • Re: Warning - Bullguard causes problems!
    ... any other computer supplier that bundles Bullguard Security). ... I spent quite a bit of time on the phone to her ISP, ... The problem wasn't the security package the problem was that you didn't ... manual would have helped but they seldom come with manuals nowadays ...
    (uk.telecom.broadband)
  • Re: Thou shalt have no other gods before the ANSI C standard
    ... > security evaluator to use whatever tools are most effective at the tool ... The "what might someone (attacker, ... > evaluating the security of the system per hour spent". ... here is another security review I participated ...
    (sci.crypt)
  • Re: Warning - Bullguard causes problems!
    ... any other computer supplier that bundles Bullguard Security). ... I spent quite a bit of time on the phone to her ISP, ... The problem wasn't the security package the problem was that you didn't ... Firewalls with the newest norton, mcafee and AVG all seem to subsribe to ...
    (uk.telecom.broadband)
  • Re: Warning - Bullguard causes problems!
    ... any other computer supplier that bundles Bullguard Security). ... I spent quite a bit of time on the phone to her ISP, ... The problem wasn't the security package the problem was that you didn't ... Firewalls with the newest norton, mcafee and AVG all seem to subsribe to ...
    (uk.telecom.broadband)
  • Re: Warning - Bullguard causes problems!
    ... any other computer supplier that bundles Bullguard Security). ... I spent quite a bit of time on the phone to her ISP, ... The problem wasn't the security package the problem was that you didn't ... Firewalls with the newest norton, mcafee and AVG all seem to subsribe to ...
    (uk.telecom.broadband)