Re: Towards a responsible vulnerability process

From: David LeBlanc (dleblanc@MINDSPRING.COM)
Date: 11/04/01

Message-ID:  <001401c16576$9fc86020$0100a8c0@davenet.local>
Date:         Sun, 4 Nov 2001 13:19:33 -0800
From: David LeBlanc <dleblanc@MINDSPRING.COM>
Subject:      Re: Towards a responsible vulnerability process

> From: Kirk Corey

> In his article, Scott Culp makes the following statement:
> "Providing a
> recipe for exploiting a vulnerability doesn't aid administrators in
> protecting their networks." This statement, and some of the
> conclusions
> based upon it, are incorrect.

I should probably let Scott defend his own statements, but I think your
disagreement is more based on fine points of interpretation than a
substantial difference (overall). I will also point out that Scott and I
don't agree on everything (so please don't expect me to defend everything
Scott says - I have enough to do defending what I say) and we often learn
from one another by discussing differences.

> When a vulnerability is announced, a system administrator may ask the
> following questions:

> 1. Is my system vulnerable? (If it ain't broke....)

Does it have the patch? If not, it is vulnerable. I don't need a script
kiddie exploit to test for this. I do need admin-level access, which can
present a problem in some cases.

> 2. If so, should I attempt a workaround?

You don't need a script kiddie exploit to know about work-arounds. I should
also back up here and state that the situation is different when a
vulnerability is announced prior to either a patch or any information from
the vendor.

> 3. If so, and I configure a workaround, has the workaround now indeed
> protected my system against the vulnerability? Or did I make
> a mistake
> somewhere, or misunderstand some part of the advisory, or (fill in the
> blank)?

You have a good point here.

> 4. Or should I apply the patch?

I think this depends on the quality of the information given in the
work-around. I know that when a work-around is cited in a Microsoft security
bulletin, it's been tested and ought to work (yes, we are humans and
occasionally make mistakes).

> 5. If so, and I do, has the application of the patch
> protected the system
> against the vulnerability? Or is there perhaps something unique to my
> configuration that causes the application of the patch to fail without
> warning, or to succeed in application, but still fail to
> protect me against
> the vulnerability? (Perhaps I need the workaround after all.)

Conversely, many of the script kiddie exploits do not correctly find all
vulnerable systems. For example, my own check for the .printer overflow
(which, BTW, uses the behavior differences cited by Chris St. Crois in
BUGTRAQ) yields a much larger number of vulnerable systems than by simply
running the widely available exploit. The exploit is buggy, and fails under
a variety of conditions. If you're using it to test with, you're going to
miss a lot of systems. If someone corrects the bugs and keeps the better
exploit to themselves, they are going to hack you if you depend on a script
kiddie exploit to evaluate your vulnerability status. In order to be really
sure your systems are protected, you need a well-written vulnerability
scanner that is of production quality. I refuse to debate the merits of the
available commercial scanners here. You also need a way to test for whether
hotfixes are applied, e.g., hfnetcheck.

> Questions 1, 3, and 5 will often be answered to a large extent by the
> vendor's and/or researcher's initial announcement. However,
> they can only
> be answered definitively and conclusively by actually launching the
> exploit, in all known variants, against the system in
> question.* If I am
> unable to do this, because the exploit is not released, I may
> be deprived
> of a tool that I need in order to assure the protection of my system.
> QED.

This conclusion is based on the assumption that you have all known variants.
You typically won't as the real black hats closely hold their exploits. It
is also based on the assumption that a script kiddie exploit is most
effective at determining vulnerability, and I can tell you with certainty
that it is often not the most effective way to check.

> If what's wanted is to improve the current situation, by all
> means, let's
> try-- but let's start with correct and accurate premises.

Sure - we can only learn from one another by engaging in a dialogue.

[snip .sig]

> * Yes, this is sometimes a risky manner of testing, but that judgment
> belongs to the sysadmin, not the vendor, and not the security
> community as
> a whole, and the risk varies not only according to the
> exploit itself, but
> also to the role of the system under attack (e.g., low risk for
> non-production systems).

Agreed, but we also have to weigh the usefulness of having an exploit
available against the risk of having it available. Are the benefits you get
from testing more valuable than the drawback of having a bazillion script
kiddiez also testing your systems?