Russ Cooper's AusCERT Presentation on MS Security Bulletins

From: Russ (Russ.Cooper_at_RC.ON.CA)
Date: 06/02/04

  • Next message: Andrew Aronoff: "Submerged Subkeys in W2K" 
    Date:         Wed, 2 Jun 2004 13:43:22 -0400
To: NTBUGTRAQ@LISTSERV.NTBUGTRAQ.COM

    Howdy,

    As you may have heard, I did a presentation last week at the 2004
    AusCERT Conference in Gold Coast, Australia.

    My presentation was the culmination of analysis I performed on all
    Microsoft Security Bulletins published by Microsoft from January 1, 2000
    to date. I analyzed the *vulnerabilities*, dissecting each bulletin into
    their respective vulnerabilities. As we all know, each bulletin MS
    produces may involve numerous vulnerabilities. In addition,
    vulnerabilities addressed by a bulletin may affect some versions but not
    others. I tabulated all of this information based on the facts in the
    bulletins.

    The purpose of this was to address a common problem I see in the media,
    namely, attempting to use the count of bulletins in comparisons with
    other OS'. Patch count comparisons serve no purpose, but counting
    patched vulnerabilities, IMO, is a more accurate comparison. In
    addition, I grouped vulnerability counts according to whether they'd
    likely affect desktop, server, and IIS servers. Again, comparing raw
    numbers for "Windows" against other OS' isn't a correct comparison
    either, so using numbers based on a role made more sense to me.

    My presentation made no attempt to compare Windows to any other OS. I
    compared MS to MS, NT 4.0 Workstation versus W2K Professional versus
    Windows XP, NT 4.0 Server versus W2K Server versus W2K3. I compared IE
    5.0x versus IE 5.5 versus IE 6.0. For all, I made no distinction wrt
    Service Packs (and therefore if security vulnerabilities were addressed
    by Service Packs alone, they were not considered.)

    Again, I'd like to stress, my analysis did not consider many things
    which are important in determining the security of a system. For
    example, I did not address severity, either as designated by MS or by me
    nor did I address exploitability. Although my numbers did not include
    vulnerabilities which were not installed by default, they did include
    vulnerabilities which could have been avoided by configuration or by
    removing components. I did exclude all vulnerabilities for things like
    SQL (including MSDE), Exchange, IIS add-ons (like Media Server) and
    other server products.

    I analyzed 452 vulnerabilities in 298 Microsoft Security Bulletins. I
    found very little difference in the number of vulnerabilities affecting
    different versions of a given product. The vast majority of
    vulnerabilities affected all of the versions of a product I looked at,
    meaning to me that the vulnerabilities were in legacy code carried
    forward into new versions. That's the sort of stuff the Security Push
    was supposed to find, IMO, yet there was nothing in my analysis to show
    the Push had an impact on this problem.

    In the period I covered, here are some of the stats I discussed;

    1. Comparing Desktop OS' without browsers

    NT 4.0 Workstation = 68 vulnerabilities
    Windows 2000 Professional = 86 vulnerabilities
    Windows XP = 55 vulnerabilities

    NT 4.0 Server = 76 vulnerabilities
    Windows 2000 Server = 98 vulnerabilities
    Windows Server 2003 = 24 vulnerabilities

    2. Comparing the first 300 days of Windows 2000 Server to Windows Server
    2003

    Windows 2000 Server = 27 vulnerabilities
    Windows Server 2003 = 24 vulnerabilities

    3. Comparing Windows NT 4.0 Server to Windows Server 2003 during the
    first 300 days of Windows Server 2003

    Windows NT 4.0 Server = 22 vulnerabilities
    Windows Server 2003 = 24 vulnerabilities

    This is where I made some comments about NT and Microsoft shareholders.
    What I said was this. Some people think they should wait for an SP1
    before they deploy a product. Considering that NT 4.0 Server was at SP6
    prior to the beginning of my 52 month analysis, if you waited for SP6 of
    W2K3 before deploying you may only experience an 8.333% reduction in the
    number of vulnerabilities which affect it. IOWs, if you think that
    waiting for a product to mature is going to make it less vulnerable,
    you're probably waiting forever.

    I also made the point that since newer versions aren't significantly
    less vulnerable, people aren't going to upgrade to them to be less
    vulnerable. That may be something MS shareholders might be concerned
    about, if they believe the desire to be less vulnerable is a purchase
    motivator. The actual bulletin point in the presentation was;

    "If Microsoft made money from upgrades, you'd think these numbers might
    bother some people"

    4. I then did a comparison of desktop OS' where the browsers were kept
    up-to-date, revised to new versions as they were released. It showed, to
    me, that this tactic didn't significantly reduce vulnerabilities either.
    Again, largely because vulnerabilities affected all versions.

    5. I then did a comparison of NT 4.0 desktops, one which stuck with IE
    5.01 throughout the time, another which upgraded and stuck with IE 5.5,
    and another which upgraded to each new IE version. Overall, the one
    which stuck with IE 5.01 had less vulnerabilities, but even so, it was
    only 7% less vulnerable than the one which upgraded to each new version.

    6. I made the same comparison using a Windows 2000 Professional and
    found almost identical differences (albeit there were more
    vulnerabilities applicable to W2K than NT 4.0).

    I then had a slide that simply said;

    "Older is clearly better!"

    Throughout my presentation the lowest number of vulnerabilities always
    applied to NT 4.0. My point here was that new features meant new
    vulnerabilities, either because new features were introduced which had
    new vulnerabilities, or, new versions meant a new examination by
    vulnerability discoverers. In retrospect, chances are the reason new
    versions means new vulnerabilities is because of new examination, since
    by and large vulnerabilities apply to all versions.

    In the context of Microsoft's security efforts, I made comments about
    the fact that the newer versions had more vulnerabilities than the older
    versions. Despite the fact the difference is extremely small, fact is
    that its not getting better, something which cannot be ignored by
    analysis of the facts.

    7. I then compared IIS versions. Given the timeframe of the products,
    the numbers are very different;

    IIS 4.0 = 231 vulnerabilities
    IIS 5.0 = 282 vulnerabilities
    IIS 6.0 = 60 vulnerabilities

    I went on to say that in the period since W2K3's release, IIS 6.0 boxes
    were 11% less vulnerable than W2K IIS 5.0 servers. This, however, IMO
    was largely due to configuration and not a lack of vulnerable code. I
    said that if you configured any IIS box the way W2K3 IIS 6.0 was
    configured and you'd get roughly the same security. IOWs, where were the
    results of the Security Push? Surely the results weren't only a new
    configuration.

    8. My last graph depicted the number of vulnerabilities versus the
    number of patches. IOWs, does the fact that there have been fewer
    patches of late mean there have been fewer vulnerabilities. The graph
    indicated that the answer is no, not really. At various points in time
    over the past 52 months there have been ups and downs, and the last 6
    months is no different.

    9. My concluding slides were all titled "Patch-O-Mania". I started by
    stating that I believed patching was not a reason to upgrade to a new
    product. IOWs, if you thought you'd upgrade to a new product because it
    would mean you wouldn't have to patch so often, you're wrong. I
    questioned whether or not an 11% reduction in the vulnerability of W2K3
    represented a start to Microsoft's Security Push. Personally, I don't
    believe that's a significant enough figure to say it really is.

    I then said the Security Push is about consumers, not corporate users. I
    believe the MS Security Bulletins are designed for home users, not for
    corporate users. The whole idea of patching is primarily intended, again
    IMO, for home users. I mentioned how I believed that XP SP2 was
    excellent, for home users, and that it would enable consumers to be less
    aware of "Patch-O-Mania" because automatic updates would be enabled.
    Since I was in Australia, and since there people pay for bandwidth above
    an ~50MB cap, I did point out however that XPSP2 was 276MB and its
    adoption is required for us to reap its benefits.

    I concluded with Corporations. TruSecure's Sasser survey showed that
    unless you achieved 100.0% patching you were in worse shape than if you
    didn't try patching at all. I pointed out that none of Microsoft's patch
    deployment products could guarantee 100.0% effective patch deployment,
    nor could any 3rd party product. Ergo, Corporations have no way of
    knowing whether they've achieved the 100.0% patch deployment. I
    questioned how the Security Push could think that patching was an
    effective part of its efforts given these facts.

    So, in the end, I wasn't recommending that people stick with NT 4.0, nor
    was I saying that newer products are more exploitable or less secure
    than older ones. In the numerous media interviews I gave throughout the
    region over the days following the event I repeatedly told reporters
    this. I do believe that we have received insufficient benefits from the
    Microsoft Security Initiative, but stressed that XP SP2 represented very
    significant changes in the way MS approaches security. I stressed that
    it remained to be seen whether or not MS will be able to apply the new
    principles contained within XP SP2 to Corporate users, but if they can
    it would be great.

    I called for a new SP for Windows NT 4.0, considering the fact its still
    widely in use and has been more than 4 years without one yet has had at
    least 68 vulnerabilities patched.

    I stressed that, IMO, far too much effort is being placed on patching IE
    vulnerabilities. To the best of my knowledge, only 2 wide-spread attacks
    have occurred involving IE vulnerabilities, yet there have been at least
    83 vulnerabilities patched for IE. Clearly a lot of effort is being
    spent patching vulnerabilities which have not resulted in exploits, IMO,
    a large waste of Corporate resources.

    Finally, without making a sales pitch, I tried to stress that there are
    many ways to mitigate against vulnerabilities. TruSecure Corporation, my
    employer, and I, firmly believe that patching is amongst the least
    effective methods. For example, through our knowledge transfer annual
    subscription Risk Management Programs, we helped our customers stay
    secure despite only recommending 3 Microsoft patches be applied urgently
    last year. For more information about those services, see;

    http://www.trusecure.com/solutions/programs/risk_reduction.shtml

    The bottom line is that we, TruSecure Corporation, and I, are firm
    believers in Microsoft Products. Anyone who's been a subscriber of
    NTBugtraq for any length of time should realize this. We and Microsoft
    may differ on the best approaches to stay secure, or on where Microsoft
    should put its efforts or priorities, we remain a committed Microsoft
    Partner. Neither TruSecure nor Microsoft have asked me to say this, I
    just feel that with so many media reports making me out to be a vehement
    Microsoft detractor, clarification was needed.

    Cheers,
    Russ - TruSecure Corporation Senior Scientist/NTBugtraq Editor
    (note: I've wanted to drop the "Surgeon General" moniker for 2 years
    now. I dislike "Senior Scientist" almost as much considering I never
    finished High School. Suggestions for a new corporate title for me are
    welcome!!)

    -----
    Patch Automation v6.0 by Mobile Automation, Inc. allows you to quickly
    identify and fix all PC's that are exposed to the Sasser worm! Our
    solution provides quick and seamless discovery and deployment of all your
    PC computer's Microsoft security patching needs. Regardless of where
    you're PC's reside (inside the LAN, at home or on the road), Patch
    Automation gets the job done. Contact us to learn about our free 30-day
    trial version at 800-344-1150 or visit our website at
    <http://www.patchautomation.com>
    -----

  • Next message: Andrew Aronoff: "Submerged Subkeys in W2K"

    Relevant Pages

    • SecurityFocus Microsoft Newsletter #179
      ... pinpointing the exact location of security vulnerabilities that are the ... ArGoSoft FTP Server Multiple Vulnerabilities ... Windows file move restriction ... Relevant URL: http://www.securityfocus.com/bid/9761 ...
      (Focus-Microsoft)
    • Black Hat Windows Security 2002 Speakers Announced
      ... With the recent Windows vulnerabilities announced by eEye and @stake as ... BLACK HAT WINDOWS SECURITY BRIEFINGS & TRAINING 2002 IN NEW ORLEANS ADDRESS ...
      (Vuln-Dev)
    • Re: FTP Tagging anyone?
      ... > secured against various different kinds of vulnerabilities, ... formatting the system is probably not necessary. ... baseline server looks like, so they can't tell what is and isn't suspicious ... this depends on your security needs. ...
      (microsoft.public.inetserver.iis.security)
    • Multiple Vulnerabilities Sybase Anywhere 9
      ... NGSSoftware Insight Security Research Advisory ... Multiple Vulnerabilities in Adaptive Server Anywhere Network Server ... attack allowing an authenticated user to escalate privileges to 'dba' within ...
      (NT-Bugtraq)
    • Worm hole found in Windows 2000
      ... A serious flaw has been discovered in a core component of Windows 2000, ... Maiffret, chief hacking officer at eEye Digital Security, said on ... The vulnerabilities affect Windows 2000, ...
      (comp.sys.mac.advocacy)