Re: [Full-Disclosure] Multiple AV Vendors ignoring tar.gz archives

From: Nick FitzGerald (nick_at_virus-l.demon.co.uk)
Date: 02/06/05

  • Next message: Barrie Dempster: "Re: [Full-Disclosure] Multiple AV Vendors ignoring tar.gz archives"
    Date: Sun, 06 Feb 2005 17:51:16 +1300
    To: full-disclosure@lists.netsys.com
    
    

    James Eaton-Lee wrote:

    > In the majority of businesses, firewalling and virus protection are done
    > at the border of the network for a reason; when you eliminate as many
    > viruses and malicious network traffic in one place at the edge of your
    > network first, you effectively segregate 'good' and 'bad' information,
    > and provide business accountability into the bargain.
    >
    > It is for precisely this reason that 99% of businesses have some sort of
    > firewall (even if it's only a NAT gateway) on the *EDGE* of their
    > network, at the internet point of presence, rather than having client
    > firewall software installed on every client PC. Although you'd be
    > hard-pressed to find a business without client antivirus software on
    > every machine (if only for the simple reason that you can't protect
    > floppy disks with a firewall), you can get away with not scanning
    > e-mails on a client machine.

    Did you miss the part of my message where I wrote:

       Well, OK -- in a gateway scanner it is likely to be a terrible flaw.
       Any vaguely competent gateway scanner should have basic knowledge of
       all archive formats and should have an option to quarantine all
       messages with archives in the formats it cannot unpack and inspect.
       Sadly, most gateway scanners are not designed this way. It is the
       job of a gateway scanner to not let anything "dangerous" in and if
       you cannot tell what something is, prudence says you keep it out, or
       at least set it aside for more expert inspection.

    Didn't that make you think I may have had an idea or two about the
    border/inside distinction?

    > Add to this the simple fact that it is difficult to manage and account
    > for client anti-virus software - client machines are to an extent
    > unmanaged and an 'unknown quantity' ...

    Actually, that's only true in a badly run outfit, but as that is most
    outfits, I'll let it pass as a pragmatic reality...

    > ... whereas servers can be *very*
    > tightly locked down, assuring a far higher degree of reliability from
    > server-based virus-scanning tools, firewalling, and intrusion detection
    > than client machines which are vulnerable to a wide range of attacks and
    > problems. Server anti-virus software starts to become more and more
    > attractive at this point..

    Even if it ignores known archive formats that it cannot scan inside and
    simply passes such attachments on? (See above...)

    <<snip>>

    Yes, yes, we all agree that scanning Email before it gets to the
    clueless users is a really spiffing idea. That I never said otherwise
    makes me wonder why you think you are disagreeing with me on this, so
    I'll just put that down to your mis-reading (or not actually botrhering
    to read at all??) what I wrote.

    > I also disagree with you that it's a non sequitur to say that anti-virus
    > writers would be slow to react to viruses transmitted inside archives
    > they could not scan; ...

    That is because you clearly did not actually understand what I wrote.
    I suggest you go read my message again...

    > ... all that an antivirus package is - essentially - is
    > a daemon doing pattern matching on all data coming into and out of
    > machines; ...

    At the very most abstract, that is quite true of today's known virus
    scanners (it ignores all manner of much more complex stuff that
    happens, but as you cannot understadn what I write, I'll not bother
    trying to explain further).

    > ... anti-virus definitions (patterns) are very easy to write and

    No.

    Perhaps if you have looked at ClamAV you would think that, but then
    Clam is not exactly the paragon of good AV practice...

    > require little work to push to customers of anti-virus writers; archive
    > scanning, however, is a *functional* difference in the way in which
    > antivirus software works, which carries numerous implications; updating
    > an antivirus scanner's scanning engine is quite different to updating
    > the definitions.

    I _know_ that, as you would understand had you actually tried to
    understand my message.

    HOWEVER, what you have still missed is that it is entirely unnecessary
    IN A DESKTOP SCANNER to be able to scan inside most archive formats
    because any code deleivered in archives must be unpacked, AT WHICH
    POINT THE AV WILL SEE IT, to be able to do anything. (Of course, sub-
    systems and components that handle certain "archive" formats purely in
    memory, are the exceptions -- .CHM and .JAR spring to mind as likely
    examples, and there are probably a few others, but this does not extend
    to the whole gamut of file archiving/compressing formats).

    > Add to this the fact that implementing archive support in an antivirus
    > package isn't as simple as it might seem; although bz2 is released under
    > a BSD license, gzip isn't - it's GPL, and therefore any antivirus vendor
    > would have to write their gzip code totally from scratch. There could
    > certainly be enough of a curfuffle surrounding a virus using one of
    > these archive formats to cause a delay of a few hours or even days in
    > releasing updates for antivirus software which addressed the issue - and
    > as we all know, the major damage in any virus epidemic is done in a very
    > short space of time.

    In general, archive and compression handlers are written to slot into
    the recursive pseudo-filesystem harness of virus detection engines.
    Getting the algorithm right is seldom the most time consuming or
    complex piece of doing such an addition...

    BUT you have still missed the flaming obvious -- a desktop scanner does
    not have to detect malware inside an archive. As such, the malware is
    neutered. _IFF_ the user has suitable archive handling utilities to
    unpack the archive, _then_ the on-access virus scanner will be able to
    detect the malware file and block further access/warn the user/etc.
    So, while it will take several days to several weeks (depending on the
    amount of format reverse engineering that is needed, developer
    avaialability and amount and quality of QA typically done for such
    things) for a vendor to add handling of a new archive format once they
    decide they should add such handling, adding detection of the "normal"
    binary form of the malware to the desktop scanner can progress
    unhindered by the fact that some new malware uses an archive format
    that is not supported by the AV engine. (And, just in case you've
    forgotten, that is not desirable in gateway scanners, but as most of
    them simply pass "unhandled" archive formats now, you're not really
    that much worse off there either)

    > To be honest, though, that isn't really the point. antivirus vendors
    > necessarily have to write antivirus definitions reactively - but there's
    > nothing other than sheer laziness which is preventing them from
    > *pro*actively incorporating support for these types of archives into
    > their software.

    One thing that "prevents" them from adding such support is the scanning
    overhead in the on demand scanner when the user sets a scan of their
    whole hard drive going. In general, most vendors seem to prefer to
    spare your CPU cycles rather than watsing them unpacking all manner of
    compressed, archived and "compound" file formats that are currently not
    known to "naturally" carry malware.

    -- 
    Nick FitzGerald
    Computer Virus Consulting Ltd.
    Ph/FAX: +64 3 3267092
    _______________________________________________
    Full-Disclosure - We believe in it.
    Charter: http://lists.netsys.com/full-disclosure-charter.html
    

  • Next message: Barrie Dempster: "Re: [Full-Disclosure] Multiple AV Vendors ignoring tar.gz archives"

    Relevant Pages

    • Re: [Full-Disclosure] Multiple AV Vendors ignoring tar.gz archives
      ... > If virus writers start using these formats AV companies could be slow to ... If a virus was released that depended on, say tar.gz archives, and some ... regardless of whether the malware in question self-packs in .RAR, .ZIP, ... OK -- in a gateway scanner it is likely to be a terrible flaw. ...
      (Full-Disclosure)
    • [Full-Disclosure] Multiple AV Vendors ignoring tar.gz archives
      ... some AV companies ignore tar.gz's and possibly other archive formats ... that aren't very common on windows systems (but supported by the common ... products that doesn't currently exist (support for scanning inside said ... archives) this could delay signature updates. ...
      (Full-Disclosure)
    • Re: RAR bits and peices
      ... >> to handle many archive formats, so your archive may be created in some ... real RAR archives always begin with 'Rar!' ... WinRar will attempt the ...
      (comp.os.os2.utilities)
    • Re: [Full-Disclosure] Multiple AV Vendors ignoring tar.gz archives
      ... firewalling and virus protection are done ... antivirus software at all, and yet in the case of .tar.gz and .tar.bz2 ... writers would be slow to react to viruses transmitted inside archives ... > some modestly common archive type is a terrible flaw in a scanner. ...
      (Full-Disclosure)
    • [Full-Disclosure] Re: Antivirus/Trojan/Spyware scanners DoS [summary]
      ... to any AV scanner DoS attack. ... believe many people have confusion with my advisory ... condition when a scanner attempts to extract the ... Usually files within archives ...
      (Full-Disclosure)