Re: Towards a responsible vulnerability process

From: David LeBlanc (dleblanc@MINDSPRING.COM)
Date: 11/04/01 

Message-ID:  <001601c16585$0e60df90$0100a8c0@davenet.local>
Date:         Sun, 4 Nov 2001 14:54:39 -0800
From: David LeBlanc <dleblanc@MINDSPRING.COM>
Subject:      Re: Towards a responsible vulnerability process
To: NTBUGTRAQ@LISTSERV.NTBUGTRAQ.COM

> From: Russ

> David LeBlanc wrote;
> To believe that behaviors don't change
> over time is to dismiss people's ability to learn from both their own
> mistakes and others.

> If seatbelts had just been invented,

Actually, it was more of a technology problem. The seatbelts were fixed, not
retractable, and were difficult to use and still be able to drive the
vehicle. But, yes, people and organizations do react to unpleasant events,
which was the point. Our environment changes, our experiences change, and
our behaviors change as a result.

> PoisonBox + Code Red + Nimda + Gartner report = Microsoft Strategic
> Technology Protection Program.

> Doesn't matter whether that equation is true or not, it's a
> valid perception
> for the public to have. And it's a motivation for some
> malcode writers.

OK, so I should pry open your door with a crowbar in order to demonstrate
that your deadbolt is a mere toy? You'd certainly learn from the experience.
Or maybe we should have riots to show that our police don't have good enough
crowd control?

> The Secure
> Windows Initiative resulted in Windows XP being shipped with
> a Critical
> Update being available on the day of its launch.

That's a completely cheap shot. SWI has resulted in a very large number of
bugs that never got shipped in the first place. Their efforts have resulted
in tremendous improvements in developer education. It has also resulted in a
very sophisticated code scanning tool. I don't think anyone is claiming that
these efforts are going to result in perfect code. I know for a fact it has
resulted in greatly improved code.

> To use your car analogy, had XP been a car it would have been
> recalled, fixed, and shipped anew. No such mechanism exists for
> Microsoft CDs

Actually, you might note that when you start XP install, it prompts you to
download new, improved setup bits.

> The hard part is getting the fix applied before its widely exploited.

Agreed.

> The best way to effect that is to not produce software that
> requires such fixes in the first place.

Indeed. But the best example we have of code that is nearly bug-free is the
software that runs the space shuttle. If you'd like to apply those same
development methods to a commercial operating system, expect to shell out on
the order of a million dollars a copy - probably more, as the number of
customers would tend to drop rapidly.

> Regardless, its as factual that no
> software is
> bug-free as it is that software must be bug-free.

This is nonsense. I honestly can't find a way to say it nicely. It makes
about as much sense as insisting that you must never make inflammatory
statements unintentionally. You just do that, sometimes you mean to, other
times you don't. It's quite obvious that you're not an experienced
developer. The very best developers create few bugs, and giving developers
better tools to detect bugs leads to still better code, and improved testing
practices helps reduce bugs further. But even the code that runs the space
shuttle has had a small number of bugs. The key is to find ways to
cost-effectively reduce the number of shipping bugs and still produce
software you can afford.

> Fact is that very few software vendors have demonstrated any
> capacity to do
> better at this. In general, software vendors produce software
> that contains
> security vulnerabilities

This is a result of inadequate tools, lack of developer education, a need
for improved testing, and human error. I'm aware of much active work done on
the first 3 problems, but when you reach nirvana, you can then explain to
the rest of us how we might also achieve perfection.

> So as you can see, everyone repeats the same mistakes over
> and fails to learn.

I have learned from a number of mistakes, both programming and otherwise. I
don't make all of the same mistakes over and over. I am part of everyone.
Thus your statement is false.

Some people do not learn. Some groups do not learn. Some people and groups
learn after some number of unpleasant experiences.

> The insistence that there cannot be bug-free software,

You go write a piece of bug-free software that's on the order of 10,000 LOC
and I'll listen to you on this. You go ship a piece of commercial software
that's bug-free and you'll have some credibility. To the best of my
knowledge, there has never been a significantly large commercial application
that is completely bug-free.

> "Prefix" yielded the launch-date release of a Critical Update for XP.

Absolute nonsense. Prefix has yielded a large improvement in code quality.
You also really don't know what you're talking about. Prefix very
effectively finds certain coding errors. It won't find logic errors, and
that's what caused this bug. It is quite possible to write code that is
technically perfectly correct from the code standpoint, but has unexpected
behaviors.

Furthermore, if you were an experienced developer, you'd have worked with
tools like lint, BoundsChecker and Purify. From these experiences you'd know
that these tools don't find all possible errors, and flag false positives on
other types of errors. It's a very, very complex problem and expecting any
code review tool to eliminate all errors is simply unrealistic. The best you
can possibly hope for is to continually make the tool more effective.

> David LeBlanc and
> Michael Howard write a book about "writing secure code"
> despite not seeing a
> significant tangible benefit from their talent through their
> employers.

You truly have no idea what you're talking about. I have seen numerous
benefits from our efforts, but then we don't tend to drop you a mail every
time we correct some design flaw or programming error.

> Not
> to mention the fact that books have been written since the
> cows came home
> decrying the perils of coding insecurely without noticeable effect on
> Microsoft programmers.

Actually, there is a lack of books on secure coding practices, which is why
our publisher thinks it might be a viable book. There was another book very
recently released on the topic, and from what I've seen it is a good one.
Aside from that, there are a few papers on isolated web sites, and some
short sections in books like "Practical UNIX and Internet Security". Many of
the papers on web sites are directed at UNIX-specific security flaws, and
are of limited value to Windows developers.

Likewise, developers are typically not trained in school to write secure
code. I don't know of many CS programs that have courses on secure
development practices.

I personally refuse to adopt such a defeatist attitude. I want to help
people avoid making the mistakes that I've seen lead to security flaws.
Hopefully, we've written a book that will help. I don't expect it to
completely eradicate security bugs, but if it stops at least some of them,
I'll be happy. Certainly not content - I'll keep working to get rid of bugs.

[snip]

> 2. Can we slip-stream NT 4.0 service packs and Hotfixes yet?
> Nope,

Actually, I believe you can with RIS. I also think there's a KB article on
some of this.

> So, we should accept that there are only two choices.

Only if you choose to contruct a box containing only 2 choices and decide to
contrain your thinking to that box. I see a lot more than 2 choices.

[snip]

David LeBlanc
dleblanc@mindspring.com

Relevant Pages

  • Re: Is Visual Studio .NET ideal for ASP.NET + HTML site?
    ... I am a developer just like you. ... > 02/03 or are you just lumping all your FP experiences together? ... >> excuses for why it does what it does but they are still bugs. ... >> because they chose to use some code for IE for the designer and it ...
    (microsoft.public.dotnet.framework.aspnet)
  • Re: HEADSUP: ibcs2 and svr4 compat headed for history
    ... That includes kernel interfaces and security bugs. ... The svr4 code was apparently not written ... If we opt to have it off by default, mainstream functionality ... the hard thing to tell from a developer perspective is whether ...
    (freebsd-current)
  • Re: Why has the Metrowerks sign been taken down?
    ... > decisions that need to be made - if every developer wrote up a bug on IB ... time you figure out you can't import menus with IB, ... to wait for Apple to add that feature in response to a feature request. ... reproducible bugs that never should have even been shipped. ...
    (comp.sys.mac.programmer.codewarrior)
  • Re: TDD: Test-Driven Design or Test-Driven Development?
    ... You are allowed to cause friction on USENET - that's what it's for. ... The TDD cycle is very good at preventing bugs, and making the few that slip ... > also be mandated that every developer should document ...
    (comp.object)
  • Re: TRICK: methods in ASPX pages with <%%> code blocks
    ... > developer, adopt best practices. ... They are called "best practices" because ... > experiences, and by observing the experiences of others, over a long period ... >> I didn't come from classic ASP so I can't speak for someone's been used to ...
    (microsoft.public.dotnet.framework.aspnet)