Re: [fw-wiz] Efficiently detecting obfuscated shell code

From: Don Parker (
Date: 02/04/04

  • Next message: Marcus J. Ranum: "Re: [fw-wiz] Botnets, IRC servers and firewalls?"
    To: Paul Robertson <>
    Date: Wed, 4 Feb 2004 12:25:52 -0500 (EST)

    Well the thing of it is if you have someone who knows how to swap out 0x90 for some
    other 1 byte function that renders most of the hex matching useless. What remains?
    Possible content matching ie: MEOW in the recent RPC DCOM exploits. That too however can
    be removed. All of this is only good anyways for that period of time where the exploit
    is made public and the time it takes for the companies to patch. As I know from personal
    experience that can sometimes be a lengthy procedure. The experimentation I have done in
    my home lab has only reinforced the fact that it can be done, and relatively simple. As
    you say however there is no sure fire cure. I do have some idea's on how to improve
    detectioin but they of course involve time, and having the proper in-house talent to do
    so :-)



    Don Parker, GCIA
    Intrusion Detection Specialist
    Rigel Kent Security & Advisory Services Inc
    ph :613.249.8340

    On Feb 4 , Paul Robertson <> wrote:

    On Wed, 4 Feb 2004, Don Parker wrote:

    > Hey guys/gals, I have been sending this question around some of the lists, and have had
    > little real discussion on it. Question being; is it possible to reliably detect an
    > obfuscated egg? Many of the ids signatures I have seen are a little loose, and always
    > for the nop sled with some port matching.

    I think it depends on the obfuscation. Just like with polymorphic
    viruses, or indeed packed viruses, it's possible to grab the thing and
    sometimes flag on the packer or the extraction engine, and sometimes on
    payload characteristics, but sometimes you have to go to the exectution point.

    The packer/extractor will be detectable, it's just a question of if it's
    detecable without too high a false positive rate (I've been meaning to
    look at those things, but real work keeps getting in the way...)

    > The problem though is that it is a relatively trivial matter to sub the nop with an
    > ascii character. Or someone who has a little more skill can insert another 1 byte
    > function that won't affect the egg itself. These ids evasion attempts are becoming more
    > widely known. With the prevalence of such programs as ADMutate and phiral.c simplifying
    > the task as it were this will probably become more prevalent.

    In detecting malcode, you have to go after the indcator, not the
    instantiation if you want good generic detection. "This unwraps itself"
    is always a good leading indicator.

    > Its not every company which has layered defences which includes application level
    > firewalls, and a properly tuned ids with good signatures. This is not even taking into
    > account an analyst who will recognize what they are seeing. Snort's fnord does a good
    > job of detecting shell code actually, and known obfuscated variants too. Any thoughts
    > this?

    To really do things right, you want to be at the point of execution. I
    wrote an interesting proof-of-concept kernel module a couple of years ago
    that trapped execve calls to about 20 things, including /bin/sh for
    processes who had a listening socket (or who's parents had a listening
    socket) -- you could opt out a process with a signal, but kill() was also
    wrappered- it was pretty good for "this stops 99% of "tunnel in/tunnel
    out" trojans and shellcode in the wild (for the primary shellcode vectors
    such as Apache, Sendmail...)- but there's nothing that's going to be 100%.

    Obviously MAC in the kernel is the "right way" to do this, but my module
    was a drop-in- I just never got the time to take it to where it really
    needed to be.

    I don't think you can do good detection on the wire for obfuscation
    without significant slowdown and either a sandboxed execution or emulation
    engine when you're talking actual executable content, but the indicators
    of it probably have enough similarity that you could get pretty darned
    close, but that many depend heavily on what it is you're trying to

    If you run 1000 samples through an obfuscator, and can't get a reliable
    detector for more than 40% of them, it's probably a lost cause. The good
    news is that most of the bad guys use the same tools, so if you can get
    "today's obfuscator," it's probably good enough for prime time, even
    though you might not get generic detection.

    I have been meaning to go through the common shellcode libs and see if
    they code match with any distribution binaries- that would give a good
    starting point for on-machine protection.

    Paul D. Robertson "My statements in this message are personal opinions which may have no basis whatsoever in fact." Director of Risk Assessment TruSecure Corporation
    firewall-wizards mailing list
    <a href='

    firewall-wizards mailing list

  • Next message: Marcus J. Ranum: "Re: [fw-wiz] Botnets, IRC servers and firewalls?"