Re: Vulnerability Auditing Checklist

From: Steven M. Christey (coley_at_mitre.org)
Date: 05/04/04

  • Next message: jet: "Re: Vulnerability Auditing Checklist"
    Date: Tue, 4 May 2004 01:49:41 -0400 (EDT)
    To: mads@opencs.com.br
    
    

    >Maybe Steven have a newer version available?

    Unfortunately, I haven't been able to do a major overhaul, or to
    refine the categories.

    >Would be nice with some illustrations to each vulnerability listed.

    Below is an updated version. There are some more sub-categories, and
    now most categories have some example vulnerabilities. It still needs
    a lot of work, but maybe it will be useful.

    I haven't had a chance to read it closely, but McGraw and Hoglund's
    new book "Exploiting Software" defines a number of specific attack
    patterns. There's also the OWASP WebApp Pen Test Checklist. Both of
    these are a good move forward in formalizing some aspects of auditing
    and, by extension, vulnerability research.

    - Steve

    ================================================================
    Vulnerability Auditing Checklist
    ================================================================
    Version: 0.0000004
    Modified: 2004/05/03 (Change Log at end)
    Created: 2002/12/04

    Disclaimer: This is a DRAFT document. The list of categories is
    incomplete. In addition, some categories overlap, and some terms are
    wholly invented or ill-defined. It has not been compared with other
    sources. This document has been publicly posted to facilitate
    discussion of code review/testing procedures, and to spur progress in
    vulnerability classification.

    Many of these categories include CVE identifiers (CVE-yyyy-nnnn or
    CAN-yyyy-nnnn) for specific vulnerabilities that demonstrate the given
    category. The identifiers can be accessed from the search form at
    http://cve.mitre.org/cve

    Notes on the Checklist
    ----------------------

    The categories, as presented, sometimes mix the underlying programming
    error ("the vulnerability") with the attack vector and/or the
    consequence/impact of the issue. This is a reflection of the current
    state of affairs in vulnerability/attack research. Significant work
    is required to clearly delineate the boundaries between these closely
    related concepts.

    Some bug types are probably "under-studied" by researchers. That is,
    there may be many latent issues that remain undiscovered because
    researchers have not been paying close attention to them. These types
    are appropriately noted, based on the author's experience.

    General Unexpected or Malformed Input Problems
    ----------------------------------------------

    1. Buffer Overflows

       Terminology Note: many issues that are now called "buffer
       overflows" are substantively different than the "classic" overflow,
       including entirely different bug types that rely on overflow
       exploit techniques, such as integer signedness errors and format
       string bugs. This imprecise terminology can make it difficult to
       determine which variant is being reported.

       1a. Boundary end violation ("classic overflow")

           Note: from the standpoint of programming errors, stack-based
           and heap-based overflows may not differ significantly (closer
           study is needed).

           Examples: many, such as CVE-2000-1094

       1b. Boundary beginning violation ("buffer underflow" ?)

           Note: term may have been used for multiple issue types; concept
                 seems understudied

           Ref: VULN-DEV:20040110 Buffer UNDERFLOWS: What do you know about it?

           Examples:
               BUGTRAQ:20020911 Buffer over/underflows in ssldump prior to 0.9b3

       1c. Array index modification (also called "index overflow" or
           "out-of-bounds array index" or "index-out-of-range")

           Note: involves accessing memory outside an allocated buffer,
           and exploit uses buffer overflow techniques, but this can be
           exploited without "large inputs"

           Note: can overlap integer signedness errors, parameter/arg
           tampering

           Examples: CAN-2003-0072, CAN-2004-0093, CAN-2001-1009,
                     CAN-2001-1036, CAN-2002-1066, CAN-1999-0798,
                     CAN-2003-0721, CAN-2004-0093

       1d. Length parameter manipulation ("length tampering")

           Definition: the attacker can manipulate the length parameter
           associated with an input, and provide an input of a different
           length.

           Note: probably overlaps other categories

           Examples: CAN-2001-1186, CVE-2001-0191, CAN-2003-0429,
                     CVE-2000-0655, others

       1e. Off-by-one error

           Note: not always a buffer overflow

           Examples: CAN-2001-1391, CVE-2002-0083, CVE-2002-0653,
                     CAN-2002-0844, CAN-2004-0342 (this is an interesting
                     example that might not be an off-by-one)

       1f. Other length calculation error

           Note: this is a broad category. Some examples include: (1)
           simple math errors, (2) incorrectly updating parallel counters,
           (3) not accounting for size differences when "transforming" one
           input to another format (e.g. URL canonicalization or other
           transformation that can generate a result that's larger than
           the original input, i.e. "expansion")

           Examples: likely many, with variants such as those described
           above. However, this level of detail is rarely available.

    2. Format strings

       Note: format string issues seem to be heavily under-studied for
       languages other than C. Memory or disk consumption, control flow
       or variable alteration, and data corruption may result from format
       string exploitation in applications written in other languages such
       as Perl, PHP, Python, etc.

       Ref: Format String Vulnerabilities (Tim Newsham)

       Examples: many, including CVE-2000-0573, CAN-2000-0574,
       CAN-2004-0354, CAN-2004-0232, CAN-2004-0159, CAN-2004-0104

    3. Syntax/grammar violation

      Note: the general problem of "malformed input" is under-studied from
      the standpoint of underlying programming errors. Most efforts have
      been in developing attack methods, which rarely suggest the nature
      of the underlying error. Attack-related research in this type of
      "malformed input" is scattered but ongoing, e.g. see fuzzers,
      suite-based testing (PROTOS style), and fault injection. The effect
      of these is often a denial of service, although other impacts may be
      under-studied. However, the specific underlying cause is rarely
      diagnosed by the researcher. This is especially problematic when
      researchers report that "a number of random inputs were provided,
      which led to a crash."

      3a. "Empty" or blank input

          Note: some of "crash by port scan" bugs are probably due to
          this, but lack of diagnosis makes it difficult.

          Examples: CAN-2002-0422 (overlaps information leak)

      3b. Missing argument

          Examples: CAN-2004-0276, CAN-2002-1488, CAN-2002-1169,
                    CVE-2000-0521, CVE-2001-0590, CAN-2002-1236,
                    CAN-2003-0239, CAN-2003-0477, CAN-2003-0422

      3c. Extra argument

          Examples: N/A

      3d. Repeated argument

          Examples: N/A

      3e. Missing/repeated/extra separator or delimiter

          Examples: CAN-2002-1362, CVE-2000-0116, CAN-2001-1157

      3f. Wrong data type

          Definition: the application does not properly handle input data
          that is of the wrong type, e.g. it expects a digit (0-9) but is
          provided with a letter (A-Z).

          Examples: CVE-1999-1156

      3g. Incomplete input

          Note: overlaps incomplete resource release

          Examples: CAN-2002-1532, CAN-2003-0195

      3h. Missing/misplaced special characters (delimiters/etc.)

          Examples: N/A

      3i. Unknown/unrecognizable argument/command/whatever

          Examples: CVE-2001-0650

    4. Special character mismanagement

      Note: some bug types overlap this category, e.g. SQL injection and
      LDAP injection.

      Note: some of these types of special chars have been observed at one
      point or another, but it's difficult to construct the proper query
      to find suitable examples. In an attempt to be complete about what
      kinds of "special characters" exist, some types may have been added
      to this list without any publicly reported vulnerability for those
      types.

      Note: some buffer overflow vulnerabilities rely on mismanagement of
      special characters

      4a. Shell metacharacters

          Note: same as item 11

          Examples: see item 11

      4b. Delimiter between fields

          Examples: CAN-2003-0307

      4c. Delimiter between values

          Examples: CVE-2000-0076, CAN-2000-0293

      4d. Delimiter between records

          Examples: N/A

      4e. CRLF injection (line delimiter)

           Ref: CRLF Injection (Ulf Harnhammer)

           Note: probably under-studied.

           Examples: CAN-2002-1575, CAN-2003-0336, CVE-2000-0610, CAN-2001-0902

      4f. Section delimiter

          Note: overlaps CRLF injection (e.g. CRLF between MIME headers
          and content)

      4g. End-of-input delimiter (e.g. "." in mail message data)

          Examples: CVE-2000-0319, CVE-2000-0320, CAN-2001-0996

      4h. Input terminator

          Note: overlaps null character injection

          Examples: N/A

      4i. Quoting character

          Examples: N/A

      4j. Escape/meta/control character

          Examples: CAN-2000-0476, CAN-2003-0083 (overlaps interaction
                    error), CAN-2002-0985 (overlaps argument injection),
                    CVE-2002-0542, CAN-2003-0021, CAN-2003-0022,
                    CAN-2003-0023, CAN-2000-0476, CAN-2001-0523

      4k. General separator char

          Examples: N/A

      4l. Comment char

          Examples: CAN-2002-0001

      4m. Macro symbol or other char for substitution

          Examples: CAN-2002-0770

      4n. Variable name leader/terminator (e.g. "$" for env. variable)

          Examples: N/A

      4o. Wildcard or "completion" character

          Examples: CAN-2002-0433 (overlaps equivalent filename),
                    CAN-2002-1010, CVE-2000-0587

      4p. Disallowed character

          Definition: does not properly handle a character that is
          specifically disallowed by specification or other convention.

          Note: overlaps wrong data type, probably others

          Examples: CAN-1999-1265

    5. Dependent Field/Value Inconsistency (e.g.: a "length" field for a
       buffer does not reflect the actual length of the buffer; or, two
       fields have values that do not make sense when combined)

       Note: partially overlaps length parameter manipulation (1d)

       Examples: N/A (difficult to search for examples of this type)

    6. Null dereference

       Note: typically this is the result of another vuln (e.g. failed
             error handling, malformed inputs); however, most
             vulnerability reports only list the null dereference and not
             the underlying trigger.

       Note: some attacks that use long input values can trigger null
             dereferences, which are then mis-diagnosed by the researcher
             as buffer overflows.

       Examples: CAN-2004-0365, CAN-2003-1013, CAN-2003-1000,
                 CAN-2004-0389 (overlaps malformed inputs), CAN-2004-0119

    File/Directory Processing
    -------------------------

    7. Directory traversal and filename equivalents

       Terminology note: some people use "directory traversal" only to
       refer to the injection of ".." and equivalent sequences whose
       specific meaning is to traverse directories. Other variants like
       "absolute pathname" and "drive letter" have the *effect* of
       directory traversal, but some people may not call it such, since it
       doesn't involve ".." or equivalent.

       Note: filename equivalents can be regarded as a type of
       canonicalization error.

       Note: this item should be split into multiple sub-categories, but
       for the sake of consistency with the numbering convention in
       earlier versions of this checklist, they were combined.

       Note: Incomplete diagnosis or reporting of vulnerabilities can make
       it difficult to know which variant is affected. For example, a
       researcher might say that "..\" is vulnerable, but not test "../"
       which may also be vulnerable.

       Note: any combination of the items below can provide its own
       variant, e.g. "//../" is not listed (CAN-2004-0325).

       Note: most of these issues are probably under-studied.

      7a. ../filename

          Examples: many

      7b. /../filename

          Examples: many

      7c. /absolute/pathname/here

          Examples: CAN-2000-0614, CVE-2001-0265, CVE-2000-0778,
                    CAN-2000-0845, CVE-2000-0915, CVE-2000-1196,
                    CAN-2001-1269, CVE-2001-0677, CAN-2003-0753

      7d. /directory/../filename

          Examples: CAN-2001-1152, many others

      7e. directory/../../filename

          Examples: many, including CAN-2002-0298

      7f. ..\filename ("dot dot backslash")

          Examples: many, including CAN-2002-0661, CVE-2002-0946,
                    CAN-2002-1042, CAN-2002-1209

      7g. \..\filename

          Examples: N/A

      7h. \absolute\pathname\here ("absolute path")

          Examples: many, including CVE-1999-1263, CAN-2003-0753,
                    CAN-2002-1344, CAN-2002-1525, CAN-2000-0614,

      7i. \directory\..\filename

          Examples: N/A

      7j. directory\..\..\filename

          Examples: CVE-2002-0160

      7k. C:dirname or C: ("drive letter")

          Examples: CAN-2001-0687, CAN-2001-0933, CAN-2002-0466, CAN-2002-1483

      7l. ... (triple dot)

          Examples: CVE-2001-0963, CVE-2001-0615, CAN-2001-0419,
                    CAN-2003-0425, CVE-2001-0467, CAN-2002-0325,
                    CAN-2003-0313, CAN-2002-0288, CAN-2002-0261,
                    CAN-2001-1335, CVE-2001-1193, CAN-2001-1131,
                    CAN-2002-0784, CAN-2002-0708

      7m. .... (or other multiple-dots)

          Examples: CVE-2000-0240, CVE-2001-0615, CAN-1999-1082,
                    CAN-2001-0491

      7n. \\UNC\share\name\

          Examples: CAN-2001-0687

      7o. filename.ext. (trailing dot)

          Examples: CAN-2001-1386

      7p. filename.ext[SPACE] (trailing space)

          Examples: N/A

      7q. dirname. (trailing dot)

          Examples: CAN-2004-0061

      7r. dirname[SPACE] (trailing space)

          Examples: CVE-2001-0054

      7s. dirname/ (trailing slash)

          Examples: CAN-2002-0253 (overlaps infoleak), CAN-2001-0446

      7t. dirname/filename.ext/ (trailing slash)

          Examples: CAN-2001-0893, CAN-2001-0892

      7u. //multiple/leading/slash ("multiple leading slash")

          Examples: CVE-2000-1050, CAN-2002-1483, CVE-1999-1456

      7v. /multiple//internal/slash ("multiple internal slash")

          Examples: N/A

      7w. /multiple/trailing/slash// ("multiple trailing slash")

          Examples: N/A

      7x. \double\\windows\\separator

          Examples: N/A

      7y. /./

          Examples: possibly CAN-1999-1083 (could be a cleansing error)

      7z. dirname/fakechild/../realchild/filename

          Examples: CAN-2001-1152

      7aa. dirname....

          Example: BUGTRAQ:20040205 Apache + Resin Reveals JSP Source Code ...

      7ab. filename...

          Example: BUGTRAQ:20040205 Apache + Resin Reveals JSP Source Code ...

    8. Link Following

      Note: symlink following bugs are the combination of multiple
            elements: file or directory permissions, filename
            predictability, etc.

      Note: UNIX hard links and Windows hard/soft links are probably
            vastly under-studied.

      Note: Windows soft links have been exploited remotely.

      Note: these can overlap filename equivalents.

      8a. UNIX symbolic link (symlink) following

          Examples: many, including CVE-1999-1386, CVE-2000-0972,
                    CVE-2000-1178, CAN-2004-0217

      8b. UNIX hard link

          Examples: CAN-2002-0793, CAN-2003-0578, CVE-1999-0783

      8c. Windows .LNK

          Examples: CVE-2000-0342, CAN-2001-1042, CVE-2001-1043

      8d. Windows hard link

          Examples: CAN-2002-0725, CAN-2003-0844

    9. Windows 8.3 filenames

       Note: these are probably under-studied

       Examples: CAN-2001-0795, but there are others

    10. "Virtual" files

       10a. Windows MS-DOS device names

            Examples: CAN-2002-0106, CAN-2002-0200, CAN-2002-1052,
                      CVE-2001-0493, CVE-2001-0558, CVE-2000-0168, CAN-2001-0492

       10b. Windows ::DATA alternate data stream

            Examples: CVE-1999-0278, CVE-2000-0927 (note: there may be
                      others with different attack vectors and impacts)

       10c. Apple ".DS_Store"

            Examples:

             BUGTRAQ:20010910 More security problems in Apache on Mac OS X

    Process/Command Execution
    -------------------------

    11. Shell metacharacters

        Examples: many, including CVE-1999-0066, CVE-2001-1246,
                  CVE-2002-0061, CAN-2003-0041

    12. Malicious search path execution (search path can be modified by
        untrusted user to point to malicious program, e.g. UNIX PATH
        environment variable)

        Note: this doesn't just apply to Unix.

        Examples: CAN-2002-1576, CAN-2000-1128, CAN-1999-1461,
                  CVE-1999-1318

    13. Program/command argument modification or argument injection

        Examples: CAN-2004-0121, CAN-2003-0907, CVE-1999-0113

    Canonicalization Errors
    -----------------------

    14. Encodings

      Note: partially overlaps directory traversal/filename equivalents

      Note: many other types of encodings should be listed here

      14a. URL encoding (hex encoding)

        - %2e (encoded dot "." - overlap directory traversal)
          - Examples: CAN-2004-0072
        - %5c (encoded backslash "\" - overlap directory traversal)
          - Examples: CAN-2004-0072
        - %0a (overlap CRLF)
          - Examples: CAN-2002-1575
        - %20 (encoded space " " - overlap filename equivalent)
          - Examples: CAN-2004-0280, CAN-2003-0424, CAN-2001-0693
        - %00 (encoded null - overlap filename equivalent, factor in
               directory traversal)
          - Examples: CVE-2000-0671, CAN-2004-0189, CAN-2002-1291,
                      CVE-2002-1031, CAN-2001-1140
        - + (may be converted to a space when decoding the query string)
          - Examples: CAN-2002-1451

      14b. Unicode

        Examples: CVE-2000-0884, CAN-2001-0709, CAN-2001-0669 (overlaps
                  interaction error)

    15. Multiple separators or other characters

        Note: overlaps the current items in the directory traversal /
        filename equivalents section.

        Examples: N/A

    16. Case sensitivity

        Note: these are probably under-studied in Windows environments,
        e.g. case sensitivity in filenames (see filename equivalents).

        Examples: CVE-2000-0497, CVE-2000-0498, CAN-2001-0766,
                  CAN-2001-0795, CAN-2001-1238, CAN-2003-0411

    17. Validate-Before-Canonicalize

        Definition: a program "validates" data before it is canonicalized.
        This can allow attackers to "insert" invalid data by causing it to
        be produced by canonicalization.

        Note: this is probably a general category, and we may see more of
        it as programmers begin to take defensive steps, but take those
        steps in the wrong order.

        Examples: CAN-2002-0433, CAN-2003-0332, CVE-2002-0802

    18. Validate-Before-Cleanse

        Definition: a program "validates" data before it has been
        cleansed. This can allow attackers to make data look "clean" at
        first, but the cleansing step produces "dirty" output.

        Note: this is probably a general category, and we may see more of
        it as programmers begin to take defensive steps, but take those
        steps in the wrong order.

        Note: currently seen mostly in directory traversal

        Note: this category is probably under-studied.

        Examples: CAN-2002-0934, CAN-2003-0282, possibly CAN-2003-0417

    Leaks
    -----

    19. Information Leak

      Note: there seem to be many different types of problems that lead to
      information leaks, whose severity can range widely depending on the
      type of information that is leaked. In addition, information leaks
      are often the result of other bugs.

      19a. Sensitive information not cleared after use

           Examples: CAN-2001-0984

      19b. Sensitive memory not cleared due to compiler removal

           Examples:

            BUGTRAQ:20021105 When scrubbing secrets in memory doesn't work

      19c. Command-line arguments visible to other processes

           Examples: N/A

      19d. Environment variables visible to other processes

           Examples: N/A

      19e. State information leak due to inconsistent responses (e.g. user
           name enumeration: valid username/wrong pass generates
           "incorrect password," but invalid username generates "incorrect
           user")

           Note: can overlap errors related to escalated privileges

           Examples: CAN-2004-0294, CAN-2004-0243, CAN-2002-0514,
                     CAN-2002-0515, CAN-2001-1387

      19f. State information leak due to timing discrepancies (e.g. a
           "successful" operation takes more time than an unsuccessful
           one)

           Examples: CAN-2003-0078, CAN-2000-1117, CAN-2003-0637

      19g. Incomplete removal of temporary resources (e.g. files)

           Examples: CVE-2001-0465

      19h. Application-controlled diagnostic or error messages

           Examples: N/A

      19i. Uncontrolled, external diagnostic or error messages (e.g. the
           programming language leaks information on an error that happens
           in the application)

           Note: PHP applications are often targeted for having this issue
           when the PHP interpreter generates the error outside of the
           application's control.

           Examples: N/A

      19j. Design-intended or configuration-intended leak (information is
           intended for publication, but sensitive)

           Examples: CAN-2003-1038, CAN-2004-0033 (and many other PHP apps
           that support phpinfo)

      19k. Sensitive information not cleared before reuse or
           initialization

           Examples: CAN-2003-0001, CAN-2003-0291, CAN-2004-0177,
           CVE-2001-1074, CAN-2002-0339

      19l. Sensitive information leaked via "stale" pointer

           Examples: CAN-2004-0080

    20. Resource leaks

       20a. UNIX file descriptor leak

           Examples: CVE-1999-0083, CAN-2003-0740, others

    Multiple Operation/Action Errors
    --------------------------------

    21. Duplicate operation

        21a. Double-free

             Examples: CVE-2002-0004, CVE-2000-0550, CAN-2002-0847,
                       CVE-2002-0059

        21b. Double-encoding / double-decoding

             Note: overlaps canonicalization. Probably under-studied.

             Examples: CVE-2001-0333

    22. Improper handler deployment (dispatch error)

        Note: may be the same as item 28.

    23. Inability to handle out-of-order actions (state machine
        violations).

        Examples: possibly CAN-2000-0647 and/or CAN-2000-0648 (diagnosis
                  incomplete; could be null dereferences or similar
                  issues)

    24. Race Condition (non-file link)

        24a. Signal handler race condition

             Note: probably under-studied.

             Examples: CVE-2001-1349

        24b. Other TOCTOU (time-of-change time-of-use)

             Examples: N/A

    25. Deadlock

        Note: this is under-studied relative to vulnerability research.
        This item is more of an effect that is triggere by an underlying
        bug.

        Examples: CAN-2001-1400

    Configuration Errors
    --------------------

    Note: configuration errors are vastly under-studied, especially in
    terms of classification. They can be more complex than
    vulnerabilities (which are often discrete and easily separable). In
    addition, configuration overlaps with the general area of "policy"
    which can have elements that are not always considered to be relevant
    to security.

    26. Permissions, ACLs, and ownership

       26a. Bad default or inherited permissions (read, write, execute)

             Examples: many

       26b. Bad program-assigned permissions (read, write, execute)

             Examples: many, such as CVE-2002-0265, CAN-2003-0876

       26c. Ownership of critical resource not verified

             Examples: many

       26d. Does not properly manage failures due to insufficient
            permissions

            Note: probably under-studied.

            Examples: CAN-2004-0148

    27. Default configuration enables insecure feature

        27a. Default password

             Note: default passwords should probably be distinguished from
             hard-coded passwords.

             Examples: many

        27b. Default, non-essential service or component

             Examples: many

        27c. Network-based admin capability accessible to arbitrary hosts
             by default

             Examples: many

    Error Condition Identification/Management Errors
    ------------------------------------------------

    28. Handler dispatch error

        Note: may be the same as item 22.

        Note: may be the result of another vulnerability

        28a. Improper handler deployment (the wrong "handler" is assigned
             to process an input, e.g. calling a servlet to reveal source
             code of a .JSP file, or automatically "determines" type even
             if contradictory to an explicitly specified type)

             Examples: CVE-2002-0025, CVE-2000-0682, CVE-2000-0778,
                       CAN-2000-1052, CVE-2001-0004, CVE-2001-0126

        28b. Missing handler (handler not available or implemented)

             Examples: CAN-2002-0614, CVE-2001-0330

        28c. Dangerous handler not cleared/disabled during sensitive
             operations

    29. Insufficient logging of security-critical events

        Note: should be distinguished from cases when logging does not
        coccur as the result of other vulnerabilities (e.g. those that
        cause a crash)

        Examples: CAN-2000-0118, CVE-2000-0542, CVE-2000-0937,
                  CVE-2001-0056, CAN-2001-0471, CAN-2001-0978

    30. Incomplete error detection (product does not properly detect or
        check for security-critical error conditions)

    GUI Errors
    ----------

    31. Insufficient user warning of "unsafe" actions

        Examples: CVE-2001-0072, CAN-2001-0919, CAN-2000-1138 (overlap
                  error handling?)

    32. Interface inconsistency

        Definition: an API, GUI, or other interface behaves inconsistently
        with what operations are actually performed on the system,
        e.g. checking a security option does nothing, or user tells
        interface "restrict ALL" and it says "restrict SOME")

        Note: can be the result of another issue

        Examples: CVE-2000-0127, CAN-2001-1387, CVE-2000-0225,
                  CVE-2000-1169, CVE-2001-0003, CVE-2001-0081,
                  CAN-2001-0617, CVE-2001-0863, CVE-2001-0864,
                  CVE-2001-0865, CVE-2001-0866, CVE-2001-0867,
                  CVE-2001-0969, CAN-2001-1387

    Product Management Errors
    -------------------------

    Note: many of these categories cover specific phases of the software
    life cycle, which is often independent of the category of bug that is
    involved. However, some types of bugs seem to be inherent to certain
    phases of the life cycle, so at this early stage, these categories may
    be useful as a "catch-all" for certain issues.

    33. Design limitations

       Note: this seems under-studied, especialy with respect to
       classification of design flaws.

       33a. Incomplete specification

       33b. Vague specification

       33c. Support (or lack of support) for security-relevant options

    34. Distribution Error

        34a. Debugging code not omitted from production version

             Note: exposes vuln's in the debugging code itself; perhaps
             this is not a category

             Examples: CVE-1999-0095, CVE-2001-0528

        34b. Product is shipped with insecure configuration that was used
             to facilitate internal testing.

             Examples: CAN-2003-0983

    35. Patch Error

        35a. Regression error - an old vulnerability is introduced into
             new code

        35b. Incomplete vulnerability fix

    36. Documentation Error

       36a. Omission of security-critical information

       36b. Error/typo causes user to introduce a vulnerability or risk

    37. Developer-introduced back door / Trojan Horse

        Examples: CVE-1999-0145

    38. Port Error

        Definition: A product is ported to a different environment
        (e.g. OS) and does not consider differences with the original
        environment - sometimes introducing vulnerabilities specific to
        the new environment.

        Examples: CAN-2002-0661 (overlaps directory traversal)

    39. Interaction Error

        Two independent products work correctly and according to
        specification, but interact in ways that cause problems.

        Note: this category needs refinement.

        39a. Interpretation conflict ("Conflicting Interpretation
              Error"): A interprets inputs differently than B.

          Note: this is generally found in proxies, firewalls, anti-virus
          software, and other "mid-stream" devices that allow or deny
          traffic based on how the client or server is expected to behave.

          Examples: classic Ptacek/Newsham IDS design flaws, CAN-2002-1121
          (anti-virus), poison NULL byte between Perl and C

        39b. Incomplete awareness: A is not aware of all types of input
             that B can take.

    Technology-Specific Problems
    ----------------------------

    This is probably missing a number of issues in web technologies.

    40. Cross-site scripting (XSS)

        Terminology Note: some people distinguish between XSS and "HTML
        injection." Terminology is imprecise for this category, which has
        a number of variants.

        Note: overlaps special char management

        Note: this category needs work

        Examples: many, such as CVE-2002-0074, CVE-2002-0938,
                  CVE-2002-1053, CAN-2002-0530, CAN-2002-0840

    41. Form field / parameter tampering

        Note: overlaps other categories; not just present in web apps.

        Examples: many, including CVE-2000-0758, CAN-2000-0554 (overlap
                  informtion leak), CVE-2000-0926, CAN-2002-1352,
                  CVE-2000-0253, CVE-2000-0254, CAN-2000-0101, CAN-2000-0102

    42. SQL injection

        Note: overlaps special character mismanagement.

        Examples: many, including CAN-2004-0366, CAN-2004-0343,
                  CAN-2003-0779, CAN-2003-0500, CAN-2003-0377

    43. PHP-specific issues

        Note: these may overlap other categories, especially design

        Ref: A Study in Scarlet (Clowes)

        43a. PHP remote code injection

             Examples: many, including CVE-2002-0953, CAN-2004-0285,
                       CAN-2004-0030, CAN-2004-0068, CAN-2004-0127,
                       CAN-2004-0128

        43b. PHP untrusted external initialization of critical variables

             Note: can overlap PHP remote code injection, other types

             Examples: CVE-2000-0860, CAN-2002-0764, CAN-2001-0854,
                       CAN-2001-1025, CAN-2003-0754 (overlaps
                       authentication)

    44. Perl null character injection (technically an interaction
        vulnerability, but important to mention specifically)

        Note: overlaps directory traversal / filename equivalents

    Other Errors
    ------------

    45. Initialization Error

        45a. Insecure default initialization (e.g. variables or
             permissions)

             Note: overlaps other categories

        45b. Untrusted/externally controlled initialization of trusted
             variables or values

             Note: overlaps other categories

             Examples: CVE-2000-0959, CVE-2001-0033, CAN-2001-0084,
                       CVE-2001-0872

        45c. Non-exit on failed initialization affecting security-critical
             resource (e.g. configuration file format error)

    46. Resource exhaustion (memory, application-specific objects, general
        objects)

        Note: exhaustion is often the result of another bug.

        46a. Memory leak

             Definition: software does not sufficiently track and release
             allocated memory when it is finished, which slowly consumes
             remaining memory. This is often triggered by improper
             handling of malformed data or unexpectedly interrupted
             sessions.

             Note: memory leaks are probably more often the result of an
             underlying bug, than an inherent bug themselves.

             Terminology Note: "memory leak" has sometimes been used to
             describe other kinds of issues, e.g. for information leaks in
             which the contents of memory are inadvertently leaked
             (CAN-2003-0400 is one such example of this terminology
             conflict).

             Examples: CAN-2004-0221 (overlaps other bug types),
                       CAN-2004-0222, CVE-2001-0136, CAN-2001-0337,
                       CVE-2002-0574, CAN-2004-0427

        46b. Other incomplete resource release

             Definition: resource is not "released" for re-use or
             deletion, often as a result of an unusual error

             Note: overlaps memory leaks, asymmetric resource consumption,
             malformed input errors

             Examples: CVE-1999-1127, CVE-2001-0830, CVE-2002-0741,
                       CAN-1999-1569, CAN-2002-1372

        46c. Asymmetric resource consumption (aka "amplification")

             Definition: an "untrusted" entity can make a "trusted" entity
             consume more resources than necessary. Sometimes a factor in
             "flood" attacks, but other types of amplification exist.

             Note: There are probably several sub-types.

             Ref: Algorithmic Complexity Attacks (Crosby, Wallach)

             Examples:

               network amplification: CAN-2001-1244, CVE-2000-0041,
                                      CVE-1999-1379, CVE-1999-0513

               algorithmic complexity: CAN-2003-0364, CAN-2003-0244,
                                       CAN-2002-1203, CAN-1999-1537,
                                       CVE-2000-1184; similar issues exist
                                       in cryptography

               CAN-2004-0002 identifies both subtypes.

        46d. Insufficient resource pool

             Definition: the software's resource pool is not large enough
             to handle peak demand, which allows an attacker to prevent
             others from accessing the resource by using a (relatively)
             large number of requests for resources. Frequently the
             resource is a "flood" of connection or sessions.

             Note: "large" is relative to the size of the resource pool.
             See examples.

             Note: floods often cause a crash or other problem besides
             denial of the resource itself; these are likely examples of
             *other* vulnerabilities, not an insufficient resource pool.

             Note: overlaps other categories.

             Examples: CAN-2002-1063, CAN-2002-0782, CVE-2002-0406, CAN-2002-0234

    47. Numeric conversion errors

        47a. Integer Signedness Error (aka "signed integer" error)

             Note: overlaps array out-of-bounds

             Examples: CVE-2001-1279, CAN-2003-0619, CAN-2003-0972,
             CAN-2002-1420, CAN-2003-0166, CAN-2003-0721, CAN-2004-0094

        47b. Integer overflow / underflow (value "resets" to maximum or
             minimum, often through incrementing values)

             Note: exploit sometimes uses buffer overflow attack

             Note: terminology may be inconsistent.

             Examples: overflows: CVE-2002-0391, CVE-2002-0639,
                       CVE-2001-0144; possible underflows: CAN-2003-0761,
                       CAN-2004-0219, CAN-2004-0184

        47c. Network byte ordering mismanagement

             Examples: N/A

        47d. Truncation error (happens when bits are truncated, e.g. due
             to casting or conversion between numeric types)

             Examples: N/A (but Halvar Flake mentioned some of these in
                       his talk at CanSecWest 2004)

    48. Authentication Error

        Note: this is a broad category, as there can be multiple errors
        that lead to authentication problems. More study is needed.

        Note: often, an authentication error is the result of a
        vulnerability.

        48a. Authentication bypass by alternate path

             Definition: multiple "execution paths" exist for accessing
             resources that require authentication, but the authentication
             step is not placed in front of all paths. Seen in web
             applications that assume access to a particular CGI program
             can only be obtained through a "front" screen. But this
             problem is not just in web apps.

             Examples: CVE-2000-1179, CAN-1999-1454, CVE-2000-0944, many
                       web applications

        48b. Authentication bypass by alternate name

             Note: overlaps equivalent encodings, canonicalization

             Examples: CAN-2003-0317

        48c. Dependency on user-controlled key data: user controls key
             data elements that are used for authentication, e.g. if a web
             application relies on a cookie "Authenticated=1"

             Examples: CVE-2002-0367 (DebPloit)

        48d. Logic error

             Note: overlaps other categories; in these cases, an
             authentication bug is the *result* of a logic error.

             Examples: CAN-2003-0750 (conditional should have been an 'or'
             not an 'and')

        48e. Partial password comparison

             Definition: the application uses the length of an input
             password to compare against the target password, which can
             allow an attacker to easily conduct brute force guessing.

             Examples: CAN-2002-1374, CVE-2000-0979, CAN-2001-0088

    49. Unnecesarily large privilege window (app runs at higher privileges
        longer than it "has to")

        Note: difficult to distinguish this from failures to drop
        privileges.

        Examples: N/A

    50. Capability operating at higher privilege than necessary without
        authentication

        Note: probably overlaps authentication

        Examples: N/A

    51. Infinite loop

        Note: this is more the result of a programming error. Multiple
        sub-categories likely. More study is needed.

        Note: can overlap amplification or flooding

        Examples: CVE-2000-0620, CAN-2000-1203, CVE-2000-0738,
                  CAN-2002-1355 (overlaps integer signedness)

    52. Incomplete/missing security check for standardized
        algorithm/technique.

        Note: this is an implementation error, in which the
        algorithm/technique requires certain security-related behaviors
        that are not implemented, thus causing a vulnerability

        Note: overlaps cryptographic errors

        Examples: "Basic Constraints" browser cert issues (CAN-2002-0862,
                  CAN-2002-0970, CAN-2002-1407), other issues (CVE-2000-0974

    53. Cryptographic error

        Note: this category is incomplete and needs refinement, as there
        is some good documentation of cryptographic flaws.

        53a. Stores sensitive data in plaintext (passwords, credit cards,
             etc.)

             Note: often overlaps bad permissions problems

             Examples: CVE-1999-0982, CVE-2000-0402, CAN-1999-1256,
                       CAN-2002-0344, many others

        53b. Does not use peer-reviewed cryptographic algorithms

             Note: this is a "best practice" more than a vulnerability

             Examples: N/A

        53c. Does not perform all required cryptographic steps

             Note: overlaps incomplete/missing security check

             Examples: N/A

    54. Insufficient Randomness

        54a. Predictable from system or network state (time, process ID, etc.)

             Note: overlaps symlink attacks, cryptographic errors,
             information leaks

             Examples: CVE-2002-0389, CVE-2001-1141

        54b. Insufficiently large space of random values (susceptible to
             brute force)

             Examples: CAN-2002-0583, CAN-2002-0903, CAN-2002-1107,
                       CAN-2004-0230 (overlaps design limitation)

        54c. Use of "known weak" randomness algorithms

             Note: probably a number of sub-types here

             Examples: CAN-2000-0916

        54d. Does not initialize PRNG

             Note: overlaps predictable system state

             Examples: CVE-2002-0872

        54e. Predictable from previous "random" values

             Examples: CAN-2002-1463

    55. Miscellaneous remote code injection (inputs are fed directly into
        an interpreted language which is dynamically evaluated; other
        "classes" such as SQL injection are covered elsewhere)

        Examples: CVE-2000-0353, CAN-2002-0317, CAN-2001-1192

    56. LDAP injection

        Note: overlaps SQL injection, special character mismanagement

        Note: probably under-studied

        Ref: Web Applications and LDAP Injection (SPI Dynamics)

        Examples: no public examples known yet, but probably frequently
                  found by pen testing companies in custom apps

    Gaps in the Checklist
    ---------------------

    These items are known "gaps" relative to the above checklist, but they
    have not been studied well enough to place them into categories. This
    list is not complete relative to the known issues, but time
    constraints prevented a more complete list being provided.

    - Shatter style attacks

      Note: it is suspected that a number of publicized vulnerabilities
      that involve local privilege escalation on Windows systems may be
      related to Shatter attacks, but are not labeled as such.

      BUGTRAQ:20020806 White paper: Exploiting the Win32 API

      Examples: CAN-2002-0971, CAN-2002-1230, CAN-2003-0350, CAN-2003-0908

    - web browser vulnerabilities, especially cross-domain and similar
      categories related to "sandboxing"

      Note: some browser bugs have a high complexity that makes them
      difficult to classify. These might be regarded as
      "technology-specific" except that they typically involve crossing
      security boundaries, and thus may have equivalents in other
      technologies that involve security models for multiple "domains."
      These bugs may stem from more basic programming errors, and thus may
      be an *effect* of those errors.

      Examples: many, such as CAN-2004-0380, CAN-2003-1026, CAN-2002-0284

    - Windows named pipes

      Note: probably under-studied.

      Note: these sometimes play an important factor as an "alternate
      path" into a system, or an "equivalent filename."

      Examples: CVE-2002-0066, CAN-2003-0230, CAN-2003-0496

    - physical access

      Note: problems that require physical access to exploit are probably
      under-studied, and may have certain categories that are unique to
      physical access.

    - Application-specific logic errors

      Note: this covers logic errors that are specific to a particular
      application, which can only be viewed as vulnerabilities within the
      context of that application's implicit or explicit security model.
      "Price tampering" in CGI applications has some overlap here, though
      that is not a solid example (see "form field" item 41)

    - "partial password comparison" is an example of a much larger
      category which involves the comparison or management of "smaller"
      items as if they were "bigger" items, or vice versa. Examples:
      CAN-2002-0499, CAN-2003-1025, CAN-2003-0412

    - authorization errors

    - privilege errors

    - susceptibility to certain attacks (brute force, spoofing)

    - dotless IP addresses

      Examples: CVE-2001-0664, CAN-2001-0724

    ==========
    Change Log
    ==========

    0.0000001 - initial version, posted to secprog on 2002/12/05

    0.0000002 - lost in the mists of time

    0.0000003 - some dir. trans variants added, posted to Vuln-Dev on 2003/06/13

    0.0000004 - extended some categories
              - added CVE examples
              - added notes for individual categories
              - added "Notes on the Checklist"
              - added "Gaps in the Checklist"


  • Next message: jet: "Re: Vulnerability Auditing Checklist"

    Relevant Pages

    • RE: Six Step IE Remote Compromise Cache Attack
      ... >> vulnerabilities and attack vectors to help secure users and ... be to "put security first". ... Because Liu Die Yu has worked with Microsoft in the past, ... Anyone who is capable, come, find bugs. ...
      (Bugtraq)
    • [NT] Multiple .NET NULL Byte Injection Vulnerabilities (MS07-040)
      ... Get your security news from a reliable source. ... Multiple .NET NULL Byte Injection Vulnerabilities ... through String Termination vulnerabilities. ...
      (Securiteam)
    • Re: RFC: Starting a stable kernel series off the 2.6 kernel
      ... >> You mentioned security issues in your initial post. ... > I've fixed bugs which turned out to be security vulnerabilities. ... > pose is normally /much/ harder than to fix the bugs. ...
      (Linux-Kernel)
    • Re: DEFCON 16 and Hacking OpenVMS
      ... credible because you spoke "chinese" with terms that are foreign to VMS.. ... it is a common used description in SECURITY ... vulnerabilities and write PoC code to proof that the vulnerabilities ... we can't fix bugs that are found ...
      (comp.os.vms)
    • [UNIX] Mailman Multiple Security Issues
      ... Get your security news from a reliable source. ... Mailman is "a mailing list server. ... Mailman is subject to multiple security vulnerabilities, ... cross site scripting to log file injection. ...
      (Securiteam)