Re: [fw-wiz] Efficiently detecting obfuscated shell code

From: Don Parker (dparker_at_rigelksecurity.com)
Date: 02/04/04

  • Next message: Marcus J. Ranum: "Re: [fw-wiz] Botnets, IRC servers and firewalls?"
    To: Paul Robertson <proberts@patriot.net>
    Date: Wed, 4 Feb 2004 12:25:52 -0500 (EST)
    
    

    Well the thing of it is if you have someone who knows how to swap out 0x90 for some
    other 1 byte function that renders most of the hex matching useless. What remains?
    Possible content matching ie: MEOW in the recent RPC DCOM exploits. That too however can
    be removed. All of this is only good anyways for that period of time where the exploit
    is made public and the time it takes for the companies to patch. As I know from personal
    experience that can sometimes be a lengthy procedure. The experimentation I have done in
    my home lab has only reinforced the fact that it can be done, and relatively simple. As
    you say however there is no sure fire cure. I do have some idea's on how to improve
    detectioin but they of course involve time, and having the proper in-house talent to do
    so :-)

    Cheers

    Don

    -------------------------------------------
    Don Parker, GCIA
    Intrusion Detection Specialist
    Rigel Kent Security & Advisory Services Inc
    www.rigelksecurity.com
    ph :613.249.8340
    fax:613.249.8319
    --------------------------------------------

    On Feb 4 , Paul Robertson <proberts@patriot.net> wrote:

    On Wed, 4 Feb 2004, Don Parker wrote:

    > Hey guys/gals, I have been sending this question around some of the lists, and have had
    > little real discussion on it. Question being; is it possible to reliably detect an
    > obfuscated egg? Many of the ids signatures I have seen are a little loose, and always
    go
    > for the nop sled with some port matching.

    I think it depends on the obfuscation. Just like with polymorphic
    viruses, or indeed packed viruses, it's possible to grab the thing and
    sometimes flag on the packer or the extraction engine, and sometimes on
    payload characteristics, but sometimes you have to go to the exectution point.

    The packer/extractor will be detectable, it's just a question of if it's
    detecable without too high a false positive rate (I've been meaning to
    look at those things, but real work keeps getting in the way...)

    > The problem though is that it is a relatively trivial matter to sub the nop with an
    > ascii character. Or someone who has a little more skill can insert another 1 byte
    > function that won't affect the egg itself. These ids evasion attempts are becoming more
    > widely known. With the prevalence of such programs as ADMutate and phiral.c simplifying
    > the task as it were this will probably become more prevalent.

    In detecting malcode, you have to go after the indcator, not the
    instantiation if you want good generic detection. "This unwraps itself"
    is always a good leading indicator.

    > Its not every company which has layered defences which includes application level
    > firewalls, and a properly tuned ids with good signatures. This is not even taking into
    > account an analyst who will recognize what they are seeing. Snort's fnord does a good
    > job of detecting shell code actually, and known obfuscated variants too. Any thoughts
    on
    > this?

    To really do things right, you want to be at the point of execution. I
    wrote an interesting proof-of-concept kernel module a couple of years ago
    that trapped execve calls to about 20 things, including /bin/sh for
    processes who had a listening socket (or who's parents had a listening
    socket) -- you could opt out a process with a signal, but kill() was also
    wrappered- it was pretty good for "this stops 99% of "tunnel in/tunnel
    out" trojans and shellcode in the wild (for the primary shellcode vectors
    such as Apache, Sendmail...)- but there's nothing that's going to be 100%.

    Obviously MAC in the kernel is the "right way" to do this, but my module
    was a drop-in- I just never got the time to take it to where it really
    needed to be.

    I don't think you can do good detection on the wire for obfuscation
    without significant slowdown and either a sandboxed execution or emulation
    engine when you're talking actual executable content, but the indicators
    of it probably have enough similarity that you could get pretty darned
    close, but that many depend heavily on what it is you're trying to
    protect.

    If you run 1000 samples through an obfuscator, and can't get a reliable
    detector for more than 40% of them, it's probably a lost cause. The good
    news is that most of the bad guys use the same tools, so if you can get
    "today's obfuscator," it's probably good enough for prime time, even
    though you might not get generic detection.

    I have been meaning to go through the common shellcode libs and see if
    they code match with any distribution binaries- that would give a good
    starting point for on-machine protection.

    Paul
    -----------------------------------------------------------------------------
    Paul D. Robertson "My statements in this message are personal opinions
    proberts@patriot.net which may have no basis whatsoever in fact."
    probertson@trusecure.com Director of Risk Assessment TruSecure Corporation
    _______________________________________________
    firewall-wizards mailing list
    firewall-wizards@honor.icsalabs.com
    <a href='http://honor.icsalabs.com/mailman/listinfo/firewall-
    wizards'>http://honor.icsalabs.com/mailman/listinfo/firewall-wizards>

    _______________________________________________
    firewall-wizards mailing list
    firewall-wizards@honor.icsalabs.com
    http://honor.icsalabs.com/mailman/listinfo/firewall-wizards


  • Next message: Marcus J. Ranum: "Re: [fw-wiz] Botnets, IRC servers and firewalls?"

    Relevant Pages

    • Re: [fw-wiz] Efficiently detecting obfuscated shell code
      ... I think it depends on the obfuscation. ... out" trojans and shellcode in the wild (for the primary shellcode vectors ... I don't think you can do good detection on the wire for obfuscation ...
      (Firewall-Wizards)
    • Re: getting around Ken Thompsons compiler Trojan
      ... > the names of the source code files. ... The obfuscation program will have to do a lot more than that unloess ... the detection algorithm is really simplistic. ... provably inaccurate for some inputs. ...
      (comp.security.unix)
    • [Full-disclosure] Alphanumeric Shellcode Encoding and Detection
      ... I'd like to share with you some of the work I've done researching alphanumeric shellcode detection. ... Although alphanumeric shellcode encoding isn't new, I believe this post will present you with some value. ... UINT offset1, ... // push eax push eax push eax push ecx push edx ...
      (Full-Disclosure)