[Full-disclosure] Major gcc 4.1.1 and up security issue



So, in my gnupg diff, I used code like this:

assert(a+100 > a);

with a being an int. Here, have this example code:

#include <assert.h>
#include <stdio.h>

int foo(int a) {
assert((int)(a+100) > 0);
printf("%d %d\n",a+100,a);
return a;
}

int main() {
foo(100);
foo(0x7fffffff);
}

(Also available as http://ptrace.fefe.de/int.c)

Now, if you compile this on a system where int is 32-bit (i.e. almost
anywhere these days), you might expect the assert to trigger in the
second call to foo. Not so:

200 100
-2147483549 2147483647

I opened a gcc bug for this. They told me that the C standard says
integer overflow for signed integers in undefined and therefore gcc is
right in doing this.

Now you might think that it's just assert, we use if and we are safe.
No, assert() is just a macro that turns into if. The whole assert code
gets removed here, you won't see a trace of the whole overflow check in
the disassembly.

I found the same issue with gcc regarding pointers. I found it with gcc
4.1, they told me the same story and fixed it for gcc 4.1.1. At least
with pointers there is a workaround: you could cast the pointer to a
ptrint_t, add the 100, and then check if it became smaller. But with
signed it, the portable workaround would be a monstrosity like:

assert((((unsigned int)a)<<1)+100 > (((unsigned int)a)<<1));

or am I overlooking something?

I'm not saying that the gcc people are wrong with their legalese answer.
I'm saying this will break tons of security checks in existing
applications and will get people to get 0wned. Please help make the gcc
people fix this!

Felix

PS: I think this is the same base problem and thus starts with gcc 4.1,
but I can only say for sure it happens with gcc 4.1.1.

_______________________________________________
Full-Disclosure - We believe in it.
Charter: http://lists.grok.org.uk/full-disclosure-charter.html
Hosted and sponsored by Secunia - http://secunia.com/



Relevant Pages

  • Re: data types
    ... Usually short is smaller than an int. ... Your compiler is either *really* old or broken or both. ... recommend either a version of gcc,, or Visual Studio ... Microsoft's Visual Studio Express has an onerous EULA). ...
    (comp.lang.c)
  • Re: A simple memset(..) question
    ... Forgot gcc and OS: ... DISTRIB_DESCRIPTION="Ubuntu (The Breezy Badger Release)" ... > that contains pointers to user's data. ... > int i; ...
    (comp.os.linux.development.apps)
  • Re: stream io in c
    ... zero to 255, putchar'ing in the body of the loop, then redirecting output ... int main ... // gcc -o chars mkchars1.c ... This compiles but gcc warns of incompatible pointer types. ...
    (comp.lang.c)
  • Re: [RFC] Generic population count function
    ... - gcc builtin - cpufunc inline ... int ffs_builtin ... Note clang generates very inefficient code for ffslfrom cpufunc.h. ... bsfq() returns a result ...
    (freebsd-arch)
  • Re: gcc pointer equality bug?
    ... With both gcc and clang, it prints "inconsistent behavior", ... Two pointers compare equal if and only if both are null pointers, ... pointers to one past the last element of the same array object, ... int main{ ...
    (comp.lang.c)