Re: Let's have fun with EICAR test file

• Previous message: Russ: "Administrivia: That's it for the weekend..."
• In reply to: keepitsecret_at_HUSH.COM: "Let's have fun with EICAR test file"
• Next in thread: kinky.logic_at_HUSHMAIL.COM: "Re: Let's have fun with EICAR test file"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]

Date:         Fri, 4 Jul 2003 20:19:35 +0000
To: NTBUGTRAQ@LISTSERV.NTBUGTRAQ.COM

--On Friday, 27 June, 2003 10:35 -0700 keepitsecret@HUSH.COM wrote:

> Let's have fun with EICAR test file

What a piece of incompetent crap. You clearly have no clue how the
anti-virus programs work. You clearly have no clue what the EICAR Test File
*is*. You have wasted your time writing this paper, you have wasted our
time reading it, and you're wasting my time writing a reply to every single
point that is wrong in it. All I can hope for is that you had some "fun"
doing so and that others (maybe even you, although doubtful) will learn
something from my reply. Otherwise the whole exercise what be just one
monumental waste of time.

OK, let's dissect the thing...

> download it. Here's its contents:
>
> X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*
>
> Figure 1. ESATF ASCII form.

You have missed some important parts of its definition - parts, which are
quite clear from the text posted on the EICAR Web site:

        <http://www.eicar.org/anti_virus_test_file.htm>

Namely:

(1) The above 68 characters *MUST* be at the beginning of the file. If they
aren't there, it's not the ESATF - it's that simple. Any anti-virus product
that detects as "the ESATF" something which is not it is wrong. For
instance, any product that detects it in this message is wrong. This
message, despite that it contains these 68 characters, is not the ESATF,
since they are not at its very beginning. Keep that in mind when examining
the various examples you gave. Had you paid attention to this requirement
in the first place, you wouldn't have bothered writing half of your paper.

(2) The only characters that can follow the above 68 characters are SPACE,
TAB, CR, LF, and EOF (Ctrl-Z). The total size of the file MUST NOT exceed
128 bytes. Any file that does not match this condition simply isn't "the
ESATF".

> Every AV should react when facing ESATF. It's a now well known industry-
> standard test file and all credible running AV must "detect" it. Actually,
> it should behave "as if" ESATF was a virus: appropriate warning message
> (some display something like "File infected with EICAR-Test-File" but
> they ought to be less stressful; ESATF isn't a virus and AVs shouldn't
> frighten novices) locking access to the file, putting in quarantine,

Don't know about the other products, but ours (F-PROT) even *disinfects* it
as a virus. It treats it as a simple overwriting COM infector. Keep that in
mind - it is important when addressing some other of your points.

> Okay, some will say "Hey dude, ESATF is not designed to test and stress
> AVs algorithms, but to check if AVs are working...". I know that, but

Precisely. It's not even designed to test their virus detection abilities
and MUST NOT be used for such purposes. The ability of an anti-virus
product to detect the ESATF is completely unrelated to its ability to
detect other viruses. Just because a product detects the ESATF does not
necessarily mean that it also detects viruses and how well. It only tells
us that the product is active and working.

This is another important point, because in your experiments you have used
the ESATF (and various modifications of it) for such purposes as to test
the abilities of the heuristics to detect new variants, to discover
(unsuccessfully) what detection techniques are used, etc. This is WRONG and
MISLEADING. The ESATF is simply NOT SUITABLE for such purposes. And test
results obtained in this aspect are wrong, misleading, incompetent.

> I partially agree with them; after all, ESATF is used to "check", so,
> in order to get a "full" checking, I think it should be treated like
> a true virus, even if it's not supposed to "evolve" like the real ones
> do.

No. The "real ones" don't "evolve". Only a very small class of them (the
so-called polymorphic viruses) modify themselves as they replicate - and
only in a relatively restricted way. The computer viruses don't "evolve"
like the biological ones. They are only modified by humans (and
occasionally by the programming environment).

> To check or not to check... Then, using ESATF is a cool and legal
> way to learn how AVs do their job: we won't disassemble any AV or break
> any licence agreements in our tests!

But it is not a *VALID* way to achieve such a task. And, unsurprisingly, by
using an invalid way you have failed to learn how the anti-virus programs
works - all you have reached is false conclusions.

> FAV EICAR_Test_File (exact).

Note the word "(exact)". When F-PROT says that, it *means* it. It means,
ideally, that every single non-variable bit of the virus has been
identified exactly. (Due to various technical reasons, this isn't strictly
true in all cases - but it certainly is in this one.)

> Fun stuff - Part I
>
> Now, let's play with the character string in ESATF. Instead of "EICAR-
> STANDARD-ANTIVIRUS-TEST-FILE!" we use "EICAR-STANDING-ANTIVIRUS-TEST-
> FILE!"; only the last three letters of the word "STANDARD" have been
> modified ("ARD" replaced by "ING"), the size of the virus is still the
> same, instructions too. The test file name is eicar2.zip. Here are the

The result is no longer "the ESATF". Pure and simple. Since it is no longer
that, it can no longer be used for the purposes for which the ESATF can be.

> AAV EICAR_Test ( modified ).
> BAV N/D.
> CAV N/D.
> FAV EICAR_Test_File.unknown?
> KAV N/D.
> NAV N/D.
> PAV EICAR-AV-TEST-FILE.
> RAV N/D.
> VAV N/D.
>
> Figure 8. String altered test.

PAV is wrong, because this is no longer the ESATF. The others are right,
although AAV and FAV provide more information than is strictly necessary.

> Only three AVs are aware of the alteration!

Wrong. Only one of them (PAV) isn't aware of it. The others are aware of it
and correctly no longer report the thing as "the ESATF".

> Are others using the original
> ESATF string as signature? If so, it's not very clever (should they learn
> about wildcard string?

Dunno about the other products, but F-PROT does not use signatures to
detect this particular "virus". It uses checkpoints and a map of the
non-variable areas (there is only one such area, covering the whole EICAR
Test String). Even if somebody would use a "signature" ("scan string" is a
more appropriate term), there is absolutely no need to use a wildcard scan
string for a virus that contains no variable areas.

> Some can say that ESATF altered... is not ESATF! Blind AVs are right
> then?

Exactly.

> I don't think so.

You are wrong.

> Tons of viruses variants are simply alterations
> of some bytes of the genuine virus.

Of course. So what? You are mixing two very important things here:

1) The ability of the scanners to detect known viruses.

2) The ability of the scanners to detect unknown viruses.

Two very different sets of methods are used to achieve these two goals.
Known viruses are detected using known-virus techniques. Unknown viruses
are detected using heuristics. Since heuristics are likely to cause false
positives, they are usually used only to detect code that looks very
virus-like. The ESATF code does not, so few producers waste heuristics to
detect its possible modifications - not to mention that it's completely
unnecessary.

> Therefore, it's not a good method
> to detect viruses using fixed bytes that are never executed: most of

Wrong. The difference between a virus that formats your hard disk and one
that doesn't can be only a single bit. And that bit doesn't have to be part
of the executable code, either. It could be some kind of data flag - or
even a character in a text message.

> the time, it leads to absurdities (recently, a guy showed that NAV
> detected any file containing the string "Eddie lives...somewhere in
> time!" as being infected by the Dark Avenger virus...). This kind of

This is not absurd at all. In your ignorance you probably don't know it,
but besides infecting files, the Dark Avenger virus can also damage them.
The damage is performed by overwriting a random sector with the first 512
bytes of the virus body. Believe it or not, these 512 bytes begin with this
text message. It is perfectly possible that some anti-virus products are
trying to detect the files damaged by this virus - not just the ones
infected by it. Looking only for the message (if this is indeed what they
are doing - I am not willing to take your word on this subject on faith,
given the general ignorance about how anti-virus products work that you
have demonstrated) might not be a very good idea (when I was writing my own
anti-virus programs, I was looking for the whole 512 bytes before declaring
a file as damaged) but it is by far not as absurd as you think.

> I guess AVs ought to "see" a variant.

Your guess is wrong. The anti-virus producers have to make tradeoff
decisions all the time. They might very well decide not to try to detect
new variants of something that is not likely to produce new variants,
especially if it doesn't pose a threat and/or if doing so is likely to
cause false positives.

And it's different for the different products, too - since they use
different techniques. Without a deep understanding of how a particular
product works, your judgment on whether the decision of its author is
correct or not is likely only to reveal your own incompetence.

> At last, a strange idea comes to my mind: do some AVs bypass stages if
> a "known" virus doesn't look like what it's supposed to be? I mean, if
> a virus isn't, for example, supposed to be encrypted, is this possibility
> purely bypassed during scanning?

It depends on the virus and on the product. You are making the capital
mistake of assuming that all products work the same way and treat all
viruses equally.

> If so, detection
> of known viruses variants is confronted with a strong limitation...

Yep, detection of known virus variants has one very strong limitation - it
can detect only the known virus variants! (But, hey, isn't that its job by
definition?!)

To detect unknown virus variants, you need something else. There are
various "something elses". There are heuristics, which try to detect code
that seems to be performing virus-like behavior. There are integrity
checkers, which try to detect the modifications the virus introduces in the
infected system. There are behavior blockers, which try to detect the
functions invoked by a virus during its replication. One common problem
among all these different means is that they are not exact. They cause both
false positives and false negatives. They also require a somewhat higher
level of knowledge and intelligence from their users - which is why many of
them aren't widely used (given that more than 97% of the human consists of
idiots, as I have demonstrated in a paper of mine) and when they are used
they are usually toned down.

> Let's go one step further, what about emulation? In the first test, we
> just add a NOP (90h) instruction at the beginning of ESATF. Of course,

The result is no longer "the ESATF". Therefore, it is no longer suitable
for the purposes that the ESATF is. End of story.

> AV Message
>
> AAV N/D.
> BAV N/D.
> CAV N/D.
> FAV New or modified variant of Trivial.
> KAV N/D.
> NAV N/D.
> PAV N/D.
> RAV N/D.
> VAV N/D.
>
> Figure 9. NOP test.
>
> One AV show some interest for our test...

Wrong. All of the anti-virus products are correct, but FAV provides more
information than strictly necessary. Since the file is no longer the ESATF,
none of the products reports it as the ESATF. Therefore, they are all
correct.

> but F-Prot is on the bad path;
> Trivial is a family of short in size COM infectors damaging one single
> file at a time. Nothing to do with ESATF!

Wrong again. Trivial is a very special virus family. Normally, viruses are
grouped into families according to the similarity of their replication
code. The Trivial family, however, is one of the relatively few exceptions.
By definition, it contains simple (less than 100 bytes of code) overwriting
COM infectors. As I mentioned in the beginning, our product treats the
ESATF as an overwriting COM infector (and will even disinfect it as such).
Is it any wonder, then, that modifications of it are reported as a new
variant of such a virus?

But that's just us. We're perfectionists, you see - we try to provide even
more information than strictly necessary. Just not reporting the file as
the ESATF is correct enough.

> In the second test, we bypass ESATF: we simply jump to the end of the
> code where an int 20h instruction ends the process. ESATF is never

IT IS NO LONGER THE ESATF!

> AAV N/D.
> BAV N/D.
> CAV N/D.
> FAV N/D.
> KAV N/D.
> NAV N/D.
> PAV N/D.
> RAV N/D.
> VAV N/D.
>
> Figure 11. JMP test.
>
> No one cares!

And rightly so! It is no longer the ESATF, even the code often specific for
Trivial-related viruses is no longer executed, so everybody correctly
doesn't report the resulting file as anything worth reporting. The file
simply does nothing! Is it any wonder, then, that it is reported as nothing?

> Does it mean that emulation and heuristics are a de luxe tool?

If anything, it means that the emulation correctly detects that the code
does nothing and just exits - that's why the products report nothing. But
you can't make even such a conclusion - because you don't really know which
one of the products use emulation and in what way exactly. Their report is
correct even if they didn't use emulation at all. So, you cannot conclude
that they use it. End result of your test: you have wasted your time (and
ours), have learned nothing, and have reached a wrong conclusion.

> Now, let's give ESATF a true suspicious viral behavior: a decryption
> loop. Here is the code:

IT IS NO LONGER THE ESATF!!!

> AV Message
>
> AAV N/D.
> BAV N/D.
> CAV N/D.
> FAV New or modified variant of Trivial.
> KAV EICAR-Test-File.
> NAV N/D.
> PAV N/D.
> RAV EICAR_TEST_FILE.
> VAV N/D.
>
> Figure 14. Encryption test.
>
> Actually, just three AVs do their job (but F-Prot is still wrong)! As

WRONG! Two of the products (KAV and RAV) are wrong - they report as the
ESATF something which isn't it. One product (FAV) is right but reports more
information than strictly necessary. The remaining products are right.

> Figure 13 source code, KAV didn't react. But someone send it to
> Kaspersky's and a few hours later the detection worked...

I can't speak for them and I don't know why they have added detection of
it. I wouldn't have done it. The thing is not a virus and it is not the
ESATF - therefore, it shouldn't be detected.

> I find this a little bit funny. For sure, we must note the speed of
> reaction and the importance to warn AV editors, but, however, what if we
> use thousand of different "modified" ESATF? Do they add signatures
> indefinitely?

Well, we've been doing it for something like 16 years already. :-)

> Let's try with a 67h XORed version: KAV finds ESATF again. Okay! what
> with an ORed crypted version then? This time, KAV is defeated. Did they
> take care of XOR encryption only? Strange strategy...

XOR with a constant key is a very commonly used encryption method in
viruses. KAV's author has a background in cryptography. He even has a
little-known paper (full of mathematics) which shows a method for automatic
recovery of the key when this class of encryption ("encoding", more
exactly) algorithms are used.

> Well, now... the great show: file search and replication parts are
> included! The final code is similar to what we found in the past inside
> true lamer COM overwriter viruses:

This is now a virus (while again it is no longer the ESATF). I don't think
that the moderator of this forum should have permitted the posting of virus
source here.

> You may think "well, overwriters viruses are known for ages, heuristics
> will defeat this kind of code".

Again, depends on the product and on its heuristics. The different products
work in different ways and use different methods.

> AV Message
>
> AAV N/D.
> BAV N/D.
> CAV N/D.
> FAV New or modified variant of Trivial.
> KAV N/D.
> NAV N/D.
> PAV N/D.
> RAV EICAR_TEST_FILE.
> VAV A virus was detected.

FAV and VAV show stellar performance - they have detected the new virus.
RAV is wrong, because this is not the ESATF. The other products are just
right - this is not a known virus, so they don't detect any known viruses
in the file.

> Note that McAfee detects a virus but is unable to name it.

And rightfully so - because it isn't a known virus.

> RAV still recognizes ESATF... but it goofed up!

Yes. RAV is very clearly wrong here.

> user. Even not really accurate, McAfee approch is more "safe". F-Prot
> finds a Trivial variant; this time, it's a good answer.

The previous time it was a good answer too - better than necessary, in fact.

> This time, let's replace "goat.com" by "*.com" at step 19 in Figure 15
> source code:
>
>
> AV Message
>
> AAV Found unknown virus .EXE.COM.
> BAV Is infected with Trivial.136.Gen.
> CAV N/D.
> FAV New or modified variant of Trivial.
> KAV Type_Trivial.
> NAV Bloodhound.DirActCOM.
> PAV N/D.
> RAV Trivial-based.136.
> VAV Univ.ow/e.

All of them are right, although I'm not quite sure what the VAV report
means.

> Note that NAV uses a very strange naming.

It's a very logical one, though. The "Bloodhound." prefix means that this
report is produced by the heuristics - not by the known-virus scanner. The
"DirAct" part means that this is a direct-action (i.e., non-resident)
virus. The "COM" part means that it is a COM-only infector. Lots of info
stuffed in a short report - and all of it correct. Bravo, NAV!

> Once
> connected to their "security response" page, I only get a "No additional
> information."... not very explicit!

But very correct. Since this is a new virus, by definition it is not a
known one. Since it is not known, there can be no additional information
about it! Simple, eh?

> Our file was named eicar6_a.com. Just let's rename it eicar6_b.abc. You
> know what? NAV and McAfee see nothing now! Detection based on file
> extension? AVs are amazingly fascinating. Definitely.

No more amazing than DOS (the command interpreter, to be precise). The file
named eicar6_b.abc cannot be executed. If it cannot be executed, it cannot
run. If it cannot run, it cannot infect anything. If it cannot infect
anything, it is not a virus.

This is beyond the scope of a single message, but generally, there is no
such thing as "virus" per se. They always go in pairs - the viral program
and the environment in which it is viral. For every sequence of symbols, it
is possible to find an environment in which it is a virus and one in which
it isn't. You are strongly advised to read Dr. Fred Cohen's ground-breaking
papers on this subject. Assuming, of course, that you'll be able to digest
all the math in them. Which I don't think you will be.

Anyway. The point is, many anti-virus products choose to detect something
as a virus only if it is in an environment in which it is likely to be
viral. That's not always the case - for instance, we detect macro viruses
in Word documents even if there is no Word installed on the system we're
scanning - despite the fact that they aren't viruses in such an
environment. Some products also may choose to detect viruses in files
regardless of their extension - despite that they are viruses only for some
file extensions. There is no real need for that but testers of anti-virus
products often rename the samples in their virus collections (so that they
don't run one by mistake - the testers of anti-virus products are,
generally speaking, a rather incompetent lot, at least when compared to the
genuine anti-virus researchers), so we're forced to detect such idiocies,
or risk to score unfavorably in the tests.

> A last one? Still at step 19, we now write ").com" instead of "*.com".
>
> AV Message
>
> AAV Found unknown virus .EXE.COM.
> BAV N/D.
> CAV N/D.
> FAV New or modified variant of Trivial.
> KAV Type_Trivial.
> NAV Bloodhound.DirActCOM.
> PAV N/D.
> RAV Trivial-based.140.
> VAV Univ.ow/e.

Again, correct responses from all the products.

> - - Detection of known viruses variants using only signatures has its
> limits.

Its only limit is that it, uh, detects only known viruses!

> - - Obviously, there are as many algorithms as there are AVs. But
> no one can claim the absolute truth.

Obviously. And your experiments haven't approached you one iota to it. In
fact, you have reached all the wrong conclusions.

> - - Emulation isn't always used or inneficient.

You don't know that. Your tests did not reveal when emulation was used and
when not. They didn't reveal how efficient it was, either.

> - - Even with known viruses, AVs aren't absolutely reliable; just modify
> a few bytes and they are blind.

Modify a few bytes and IT IS NO LONGER A KNOWN VIRUS!

> - - In case of true harmful code, heuristics are aware.

Wrong. The purpose of the heuristics is usually to detect *viral* code -
not necessarily "harmful" code. It is just a coincidence that the virus you
created was an overwriter. The heuristics detected the ability to replicate
- not the ability to destroy. But since you have no clue how the heuristics
work, you couldn't have figured out that on your own...

> - - Signatures aren't always optimal.

Nonsense. First, most contemporary anti-virus products don't use
"signatures" (i.e., scan strings) nowadays - at least not for primary virus
detection. In our product we use them only to trigger our much more
powerful (but slower) virus detection and identification algorithms. We
might not report anything in a file, even if we detect all the scan strings
we use in it. We'll just slow down a bit (unnoticeably to a human when
scanning just a single file), will invoke our more powerful methods, they
will tell us that no known virus was really found, and we won't report
anything. And many viruses we'll detect and identify without even a single
scan string. (That's how we detect most macro viruses.)

Second, you experiments do not allow you to reach any kind of meaningful
conclusion about the "optimality" of the scan strings used. I doubt you are
even capable of defining the term!

> - - AVs have weird behaviors: often it's all or nothing, a good
> identification or... the void.

Exactly. Believe it or not, this is what our users want. They want a yes/no
answer to the question "Is this object infected?". From their point of
view, it either is or isn't. "Maybe", "Could be something new", "I don't
really know" and "You'll have to decide" isn't good enough for them, so we
try to keep this kind of answers to the minimum. Of course, we also have to
detect new, unknown viruses, so it's a kind of trade-off.

> Above all, why not a common naming for viruses?

Believe it or not, this is a very complicated question. Basically, because
the different products work in different ways and because detection of new
viruses has to be added *fast*, we don't always have the luxury of having
the time to coordinate virus names in a committee.

> - - Viruses research is a hard topic, whether it is for known or unknown
> viruses.

It sure is. Much harder than you seem to think. What you probably consider
a "scientific paper" is actually full of elementary nonsense and doesn't
even begin to scratch the surface of the field... Your understanding of how
anti-virus products are supposed to work is more than a decade out-of-date.

> - - Is RAV a good choice for Microsoft (don't kick my head!)?

Yes, otherwise they wouldn't have bought it. The difficult question is
whether it is a good choice for their users.

Regards,
Vesselin

--
Vesselin Vladimirov Bontchev, not speaking for FRISK Software International,
Postholf 7180, IS-127, Reykjavik, Iceland              producers of F-PROT.
e-mail: bontchev@complex.is, tel.: +354-540-7422, fax: +354-561-7274
PGP 2.6.2i key fingerprint: E5 FB 30 0C D4 AA AB 44  E5 F7 C3 18 EA 2B AE 4E
oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
Are You "Certifiable"? Summer's Hottest Certification Just Got HOTTER!
With a growth rate exceeding 110%, the TICSA security practitioner
certification is one of the hottest IT credentials available.  And now, for
a limited time, you can save 33% off of the TICSA certification exam! To
learn more about the TICSA certification, and to register as a TICSA
candidate online, just go to
http://www.trusecure.com/offer/s0100/
oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo

• Previous message: Russ: "Administrivia: That's it for the weekend..."
• In reply to: keepitsecret_at_HUSH.COM: "Let's have fun with EICAR test file"
• Next in thread: kinky.logic_at_HUSHMAIL.COM: "Re: Let's have fun with EICAR test file"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]