Re: [Lit.] Buffer overruns

From: Anne & Lynn Wheeler (lynn_at_garlic.com)
Date: 12/12/04 

Date: Sun, 12 Dec 2004 09:29:22 -0700

Bryan Olson <fakeaddress@nowhere.org> writes:
> By the standards of safer programming languages, memcpy is *not*
> perfectly well-behaved. It has undefined behavior if the programmer
> passes a size_t that is too large.

my experience is that programs written in C, except for buffer length
failures ... tend to have failures & mistake rates similar to other
environments; for instance storage cancers seem to crop up about as
frequently in C environments as in many other environments.

one might then conclude that storage cancers are a relatively common
problem across a large number of different environments (except for
the apl/lisp/etc environments were storage allocation/deallocation
responsibilities have been totally removed from the implementor's
responsibility). given that programmers are going to be given the
allocation/deallocation responsibility ... then it is probably going
to be require some development methodology to address storage cancer
type issues.

the buffer length scenario goes to a completely different level
... since the buffer length failure & mistake rate in significantly
larger in c than lots of other environments (possibly by two orders of
magnitude) ... there is some statement somewhare that given large enuf
quantitative difference ... it can become a qualitative difference.

during much of the 90s ... i was told that it was simply just another
programming development issue and better tools were going to eliminate
the C language environment tendency towards buffer length mistakes.
However, it seems like that the better tools still have had little
effect on the buffer length mistakes ... new C application seems to
have just as many buffer length mistakes as the code from the 80s.

i looked at the structural differences and observed

a) null termination convention appeared to encourage programmers
to believe that length was an attribute of the data pattern

b) default buffer allocation/deallocation in C ... was having buffer
construct being simply default to a C address/pointer construct
... with the responsibility being placed on the programmer for
maintanence of the buffer length attribute. however, one could claim
that the result of the null-convention encouraging programmers to
think of length as an attribute of the data pattern rather than and
attribute of the structure containing the data ... programmers would
frequently forgot to enforce lengths about the length attributes of
buffers (which was their responsibility by the convention of mapping
buffer constructs to simply pointers and leaving programmers to manage
the length attribute).

so, i was aware of numerous infrastructures during the 60s which took
the approach that length was an attribute of the structures ...
rather than the data. a simple example is buffer structures which had
used a pointer convention to the start of the data portion of the
buffer ... but if you backed up two bytes, it had the length of the
data in the buffer ... and if you backed up two more bytes, it had the
(max) length of the buffer (buffers explicitly carried their length
attribute, it was not the responsibility of the programmer to carry it
around in their heads). Further more, the length attribute of data
was carriend by the structure that contained that data ... rather than
an attribute implicit in the pattern of the data.

long ago and far away ... when i asked why the null data pattern
methodology was chosen and not the length header convention (common in
the 60s) ... i was told that they specifically chose the C approach
because it saved a couple bytes of storage per structure, an
instruction or two in loops ... and possibly a register.

so i claim that there is an intersection of characteristics ...
somewhat unique to C (as compared to many other environments that have
radically lower frequency of buffer length problems); the null pattern
convention for length ... which encourages programmers to think of
length as an attribute of the pattern of the data (and something that
don't need to explicitly manager) and buffer allocation/deallocation
construct is mapped to simple pointer with implicity requirement that
the programmer is (now manually) responsible for the length attribute
of the buffer constructs. Buffers are real constructs in C
environments with both an pointer attribute and a length attribute ...
however only the pointer attribute is mapped to explicit C language
construct, a pointer ... and the length attribute is carried
implicitly ... forcing programmers to manage it ... and the null
terminating convention is encouraging programmers to think of length
as an attribute of the data pattern ... as opposed to an attribute of
the structures that contain the data.

so switching roles ... i'm an attacker ... i'm looking at typical
environment where C programmers frequently are assuming that length is
an attribute of some pattern in well-behaved data. so it seems that
besides just old-fashion, run-of-the-mill program failures because the
programmer made a mistake ... the bad guys can frequently and
succesfully attack C-language environments by sending ill-behaved
data.

that makes common c-language implemented environments ... not only
subject to failures because of large increase in buffer related
programmer mistakes (compared to other environments) .... but these
failure characteristics also represent explicit attack
vulnerabilities. 

-- 
Anne & Lynn Wheeler | http://www.garlic.com/~lynn/

Relevant Pages

  • Re: Secure C library
    ... >> make buffer size decisions more visible, ... >> data about the commonest sources of bugs and security gaffes. ... any number of professional programmers over the past several decades. ... the proposer will come to meetings for a few years to shepherd ...
    (comp.std.c)
  • Re: [Lit.] Buffer overruns
    ... >> correctness of defective code. ... Consider a buffer you provide for DMA. ... >> there are address ranges that may be validly read and others that may ... > programmers for generating bad software quickly. ...
    (sci.crypt)
  • Re: Calling unauthorized code from an authorized address space
    ... Prior to passing the buffer to a work queue for the SRB, ... programmers supply assembler language code to do ... In theory you could interpret compiled assembler code, ... For IBM-MAIN subscribe / signoff / archive access instructions, ...
    (bit.listserv.ibm-main)
  • Re: C++ Bounds Checking
    ... Giles wrote: ... It seems that some C programmers ... a pointer to the beginning of a buffer without also passing the length, ... no deficiencies and the other way is to make it so complicated ...
    (comp.lang.fortran)
  • Re: Why is recursion hard?
    ... >>Programmers understand scoping and most are also familiar with stack ... I think that having seen Scheme environment diagrams a year ... I learned about environments from reading ...
    (comp.lang.scheme)