Re: Thou shalt have no other gods before the ANSI C standard

From: Trevor L. Jackson, III (tlj3_at_comcast.net)
Date: 02/27/05 

Date: Sun, 27 Feb 2005 17:37:22 -0500

Brian Inglis wrote:

> On Sun, 27 Feb 2005 14:33:06 -0500 in alt.folklore.computers, "Trevor
> L. Jackson, III" <tlj3@comcast.net> wrote:
>
>>David Wagner wrote:
>>
>>>Trevor L. Jackson, III wrote:
>>>
>>>>Of course in the impersonal sense you are correct. But I'd like to hear
>>>>your description of what "good enough" would be. Can you articulate a
>>>>criteria than can be applied at the project level to determine whether
>>>>hat project is good enough or not?
>>>
>>>I'm not sure we know what would be "good enough". How could we tell?
>>>The only way I know to tell is to do a controlled experiment and measure.
>>>Until we can articulate some process, try that process out, and measure
>>>its effectiveness, I don't think we will be able to tell for sure
>>>whether that process will be adequate -- i.e., whether we will be able to
>>>(justifiably) count on that process to eliminate all buffer overruns.
>>>I have yet to see such evidence provided for any process currently known.
>>
>>There is room for further discussion here. The topic would be absolute
>>value versus relaive value. The phrase "good enough" suggest that there
>>is an absolute value or critera that should be applied. That path leads
>>to the hopless goal of perfect software.
>>
>>The alternative or relative or incremental value, such as disciplined
>>methodolgy or automated error checks is easier to envision, but does not
>>guarantee "enough" improvement to materially reduce the icidenc the of
>>exploits. After all, any given progam needs only one serious
>>vulnerability, so finding all but the last one does not increase the
>>effective security of the system. Also, along this path lies hype
>>because we still lack a metric for (in)security.
>
>
> There are a number of predictive measures for estimating the remaining
> number of defects from the number already found.
> OTOH the Lockheed-Martin US space shuttle software team has metrics
> that allow them to predict how many defects to expect in a new
> release, so if their validation processes come up short on the count,
> they institute a bug hunt to find the balance. But the process is
> expensive at ~$35M/year for 100 people supporting ~420KLOC IIRC.

... And I suspect it is not useful in single digits. It may not be
useful at all in predicting exploitable bugs as opposed to spec and
design "excursions". Do they distinguish the defects on any of those bases?

/tj3