Re: Problems with MS01-052 - Microsoft responds
From: Russ (Russ.Cooper@RC.ON.CA)Date: 11/08/01
- Previous message: http-equiv@excite.com: "Re: MS Passport exploit"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]
Message-ID: <E9A01F52DC939448BBDE44ED2E1C468F23CCB1@muskie.rc.on.ca> Date: Thu, 8 Nov 2001 08:48:25 -0500 From: Russ <Russ.Cooper@RC.ON.CA> Subject: Re: Problems with MS01-052 - Microsoft responds To: NTBUGTRAQ@LISTSERV.NTBUGTRAQ.COM
As you may or may not know, I asked Microsoft to provide a detailed response
to the situation that culminated in the Terminal Server hotfix, MS01-052. In
testimony to the normally close relationship between Microsoft and NTBugtraq
posters, I received the following yesterday from Steve Lipner. Steve Lipner
is Microsoft's Directory of Security Assurance, and oversees the Microsoft
Security Response Center and the Secure Windows Initiative.
Cheers,
Russ - NTBugtraq Editor
///
-----Original Message-----
From: Steve Lipner [mailto:slipner@windows.microsoft.com]
Sent: Wednesday, November 07, 2001 12:35 PM
To: Russ
Subject: RE: Problems with MS01-052
Russ,
I wanted to get back to you and respond to your comments on the problems
with MS01-052.
First, I'd like to say that you raise an entirely fair point: why should
customers trust Microsoft to autoinstall patches to their systems when an
error like this can occur?
Let me first summarize what happened, and then I'll tell you what we're
doing about it.
The problem with the patch for MS01-052 resulted from an error in the test
group. The terminal server patch was initially developed for Windows 2000,
and then its release was put on hold while the development and testing of
the NT4 patch was completed.
While the NT4 fix was being developed and tested, two private hotfixes
(hotfixes developed to respond to individual customers' problems unrelated
to the security bulletin) affecting the same Windows 2000 Terminal Server
modules as were included in MS01-052 were developed, tested, and released.
When it came time to release the security bulletin, because of a clerical
error, the test team (perhaps confused by the fact that the two private
Windows 2000 hotfixes had completed testing) believed that both the patches
for NT4 and Windows 2000 had completed testing and signed off. Obviously,
only the NT4 patch had done so.
The specific problem with the Windows 2000 patch was that digital signatures
were missing from two of the terminal server DLLs in the patch. These are
signatures that are required for terminal server DLLs and they're distinct
from the signature on the package that's downloaded; as you say, there's no
way that the Windows 2000 patch could have worked.
As soon as reports of problems with the patch started to come in, we removed
the patch from the Download Center. We did this within four hours of
releasing the patch, and (contrary to your comment) we posted a message to
the bulletin mailing list at once explaining the problem and what customers
should do as best we understood at the time.
We also began at once to build a functional patch. Since the patch would
have to overwrite the flawed version on any systems where it had been
installed, it had to have new version numbers. And because any general
distribution patch must include all prior fixes to the DLLs it includes, we
now had to incorporate the changes made by the two private hotfixes I
mentioned above. The presence of these additional changes meant that the
patch had to undergo full general distribution testing (rather than just
testing to ensure that the DLLs were properly signed). This repetition of
the test process is what accounts for the four-day delay between initial and
corrected releases. When the updated patch was available, we again sent a
message to the bulletin mailing list telling customers of the availability
of the corrected patch and what they should do to restore their systems to
functional status.
Let me talk next about what we are doing to prevent this problem from
recurring. Effective immediately, every patch that's released with a
security bulletin will undergo testing by two testers. In addition, we have
revised and increased the formality of the procedures for signoff on
security patches by test managers to ensure that all test steps have in fact
been completed satisfactorily before a patch is released. We're continuing
to look at options that will allow us to increase the level of review of
security patches without introducing unacceptable delay in patch release.
We believe that this change will eliminate the chance of a repetition of the
problem of MS01-052.
We're also planning on bringing in an outside consultant to audit and
evaluate our security patch development process and make recommendations as
to how we can make it more reliable and efficient. We'll be providing you
with an update on the results of this audit as it proceeds.
We apologize again to the customers who were affected by our error. We hope
that the changes we're making will allow us to regain your confidence.
Steve
///
I then responded to Steve's message with some additional questions, and he
sent me back the following response;
///
-----Original Message-----
From: Steve Lipner [mailto:slipner@windows.microsoft.com]
Sent: Wednesday, November 07, 2001 6:44 PM
To: Russ
Subject: RE: Problems with MS01-052
Hi Russ,
I find myself answering point by point, so I'll answer inline - less
confusing.
Steve
>-----Original Message-----
>From: Russ [mailto:Russ.Cooper@rc.on.ca]
>Sent: Wednesday, November 07, 2001 10:39 AM
>To: Steve Lipner
>Subject: RE: Problems with MS01-052
>
>Steve,
>
>Thanks for the message.
>
>Can this be shared with the NTBugtraq subscribers?
>
Yes.
>The stated procedure clearly fell apart in many places. Not only would
>there have been problems during the fix test phase (to verify the
>problem had been resolved), but also during the packaging phase, and
>the test of the packaging phase.
The problem was in essence that the package that had passed testing was not
the one that was released. This was a result of the confusion I outlined
below.
>If the .dlls were missing signatures, how come WFP didn't overwrite
>them with the previous versions on customer's systems who applied the
>broken patch? Is there a problem in WFP that allows unsigned .dlls to
>be written over signed ones?
These are not the WFP signatures (nor the signatures on the package) but a
separate signature that is applied to the terminal server executables. The
DLLs were properly WFP signed.
>How did a signed hotfix package get produced for unsigned .dlls? When
>the signature was placed on the hotfix, and the files it contained,
>isn't there a manifest recorded? Shouldn't there have been some sort
>of comparison between the files being put into the hotfix package,
>and those that had been approved as needing to be in there? Hashes
>compared, that sort of thing?
See above. These packages were all properly WFP signed, and were the proper
files.
>Doesn't the download process get checked each time? Shouldn't it be
>part of the process that someone within MS Test actually downloads the
>bits according to the web process, installs the patch, and verifies
>that it has been deployed correctly and completely?
That is part of the test process that the test team thought had been
completed and had not.
>Microsoft's message about the problems with the patch took 30 hours to
>come out after MSRC received initial notification that I can verify.
>Considering how easily reproducible the problem was, the time-of-day
>MSRC was original told about the problems with the patch, don't you
>agree that's a very long time for such a warning? That the bits were
>not available is one thing, but many people had it downloaded and
>deployed on numerous servers in the short time it was available.
You're right. The message was delayed because the MSRC team thought that
we'd be able to repost the patch within the day, and then send one message
that said "it's been replaced". This was a mistake, and I've directed that,
in such incident in the future (I hope we don't have any) we send a message
to the listserv within an hour after pulling the patch.
///
- Previous message: http-equiv@excite.com: "Re: MS Passport exploit"
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]
Relevant Pages
|
|
