Re: Windows EAL4 Evaluation

From: S. Pidgorny [MVP] (slavickp@yahoo.com)
Date: 12/07/02 

From: "S. Pidgorny [MVP]" <slavickp@yahoo.com>
Date: Sat, 7 Dec 2002 10:51:18 +1100

This is an interesting opinion. What was said about the Common Criteria
certification is true: it's as useful as ISO900x. What was said about
Windows 2000 is applicable to any other system certified. I particularly
liked the bit about EROS. yes, quite possibly, that will be more securethan
Windows :) 

--
Svyatoslav Pidgorny, MS MVP, MCSE
-= F1 is the key =-
"Benn Wolff" <Benn_Wolff@REMOVEhotmail.com> wrote in message
news:uC$VnhLnCHA.212@TK2MSFTNGP09...
> ----------------------------------------------------------------------
>
> Date: Wed, 27 Nov 2002 7:11:48 PST
> From: "Peter G. Neumann" <neumann@csl.sri.com>
> Subject: Understanding the Windows 2000 EAL4 Evaluation, Jonathan S.
Shapiro
>
> Understanding the Windows EAL4 Evaluation
> Jonathan S. Shapiro, Johns Hopkins University Information Security
Institute
>   "Jonathan S. Shapiro" <shap@eros-os.org>
>   http://eros.cs.jhu.edu/~shap/NT-EAL4.html
>     [Via Bruce Schneier's Crypto-Gram (courtesy of Paul Walczak)]
>
> By now, you may have heard that Microsoft has received a Common Criteria
> certification for Windows 2000 (with service pack 3) at Evaluation
Assurance
> Level (EAL) 4. Since a bunch of people know that I work on operating
system
> security and on security assurance, I've received lots of notes asking
"What
> does this mean?" On this page I will try to answer the question. For the
> impatient the answer is:
>
> Security experts have been saying for years that the security of the
Windows
> family of products is hopelessly inadequate. Now there is a rigorous
> government certification confirming this.
>
> Since that's a pretty strong statement, bear with me while I try to
explain
> it in plain English.
>
> How a Security Purchase Should Work (In Abstract)
> At the risk of telling you something you already know, here is how a
> purchaser ought to proceed when buying a security product:
>
>   * Assess your needs. Determine what your requirements are.
>   * Decide which product you are most confident will meet those needs.
>   * Buy and deploy it.
>
> Each of these is potentially an involved process, and most customers don't
> have the expertise to do them effectively. Even if you did, Microsoft (or
> any other vendor) isn't likely to let you examine their code and design
> documents in order to evaluate their product.
>
> The purpose of the Common Criteria process is to develop standard packages
> of commonly found requirements (called Protection Profiles) and have a
> standard process of independent evaluation by which an expert evaluation
> team arrives at a level of confidence for some particular software
product.
>
> As a customer, this makes your life simpler, because you can compare your
> needs against existing requirements constructed by experts and then see
how
> well the software you are buying meets those requirements. Security
> requirements are fairly hard to write down correctly, but if the resulting
> document is annotated properly they aren't all that hard to understand.
>
> Obviously, if you don't know your needs (requirements) you don't stand
much
> of a chance of getting them met. Likewise, if you don't know what
> requirements a software product was evaluated against, the evaluation
result
> isn't terribly useful to you in practical terms.
>
> How Common Criteria Works
>
> >From the customer perspective, a Common Criteria evaluation has two
parts:
>
> A standardized requirements specification called a Protection Profile that
> says what the system is supposed to do. Sometimes there will be more than
> one of these -- usually a general baseline protection profile and then
some
> others describing additional, specialized requirements.
>
> An evaluation rating. This is basically an investigation by well-trained
> experts to determine whether the system actually meets the requirements
> specified in the protection profile(s). The result of the evaluation is an
> "Evaluation Assurance Level" which can be between 1 and 7. This number
> expresses the degree of confidence that you can place in the system.
>
> In order to understand the result of an evaluation, you need to know both
> the evaluation result, which will be a level between EAL1 and EAL7, and
the
> protection profile (the requirements that were tested). Given two systems
> evaluated against the same protection profile, a higher EAL rating is a
> better rating provided the requirements meet your needs.
>
> Knowing that a product has met an EAL4 evaluation -- or even an EAL7
> evaluation -- tells you absolutely nothing useful. It means that you can
> have some amount of confidence that the product meets an unknown set of
> requirements. To give a contrived example, you might need a piece of
> software that always paints the screen black. I might build a piece of
> software that paints the screen red with very high reliability, and get it
> evaluated at EAL4. Obviously my software isn't going to solve your
problem.
>
> The Windows 2000 Evaluation
> Microsoft sponsored an evaluation of Windows 2000 (with Service Pack 3 and
> one patch) against the Controlled Access Protection Profile (plus some
> enhancements) and obtained an EAL4 evaluation rating. This is most
> accurately written as "CAPP/EAL4".
>
> Problem 1: The Protection Profile
>
> The Controlled Access Protection Profile (CAPP) standard document can be
> found at the Common Criteria website. Here is a description of the CAPP
> requirements taken from the document itself (from page 9):
>
> The CAPP provides for a level of protection which is appropriate for an
> assumed non-hostile and well-managed user community requiring protection
> against threats of inadvertent or casual attempts to breach the system
> security. The profile is not intended to be applicable to circumstances in
> which protection is required against determined attempts by hostile and
well
> funded attackers to breach system security. The CAPP does not fully
address
> the threats posed by malicious system development or administrative
> personnel.
>
> Translating that into colloquial English:
>
> Don't hook this to the Internet, don't run e-mail, don't install software
> unless you can 100% trust the developer, and if anybody who works for you
> turns out to be out to get you you are toast.
>
> In fairness to Microsoft, CAPP is the most complete operating system
> protection profile that is presently standardized. This may be the best
that
> Microsoft can do, but it is very important for you as a user to understand
> that These requirements are not good enough to make the system secure. It
> also needs to be acknowledged that commercial UNIX-based systems like
Linux
> aren't any better (though they are more resistant to penetration).
>
> Note that the "Don't install software" part means that you probably
> shouldn't install a word processor. On several occasions Microsoft has
> unintentionally shipped CD's with viruses on them. A CD with a virus
> qualified as "malicious system development."
>
> Problem 2: The Evaluation Assurance Level
>
> Having described the requirements problem, I now need to describe the
> problem of the EAL4 evaluation assurance level that Windows 2000 received.
>
> As I mentioned before, EAL levels run from 1 to 7. EAL1 basically means
that
> the vendor showed up for the meeting. EAL7 means that key parts of the
> system have been rigorously verified in a mathematical way. EAL4 means
that
> the design documents were reviewed using non-challenging criteria. This is
> sort of like having an accounting audit where the auditor checks that all
of
> your paperwork is there and your business practice standards are
> appropriate, but never actually checks that any of your numbers are
correct.
> An EAL4 evaluation is not required to examine the software at all.
>
> An EAL4 rating means that you did a lot of paperwork related to the
software
> process, but says absolutely nothing about the quality of the software
> itself. There are no quantifiable measurements made of the software, and
> essentially none of the code is inspected. Buying software with an EAL4
> rating is kind of like buying a home without a home inspection, only more
> risky.
>
> The Bottom Line for Windows 2000
>
> In the case of the CAPP protection profile, there actually isn't much
point
> to doing anything better than a low-confidence evaluation, because the
> requirements set itself is very weak. In effect, you would be saying "My
> results are inadequate, but the good news is that I've done a lot of work
so
> that I can be really sure that the results are inadequate.
>
> In the case of CAPP, an EAL4 evaluation tells you everything you need to
> know. It tells you that Microsoft spent millions of dollars producing
> documentation that shows that Windows 2000 meets an inadequate set of
> requirements, and that you can have reasonably strong confidence that this
> is the case.
>
> Conclusion
>
> Security isn't something that a large group can do well. It is something
> achieved by small groups of experts. Adding more programmers and more
> features makes things worse rather than better. Microsoft has been adding
> features demanded by their customers for a very long time.
>
> It is possible to do much better. EROS, a research operating system that
we
> are working on here in the Systems Research Laboratory at Johns Hopkins
> University, should eventually achieve an EAL7 evaluation rating, and is
> expected to provide total defense against viruses and malicious code. It
> won't be compatible, because the most important security problems in
Windows
> and UNIX are design problems rather than implementation problems. In fact,
> none of the viable research efforts toward secure operating systems are
> compatible with existing systems.
>
> It remains to be seen whether EROS or one of the other attempts to build
> secure operating systems will prevail, but better solutions are coming.
>
>   [We somehow keep coming back to electronic voting machines in RISKS.
The
>   2002 FEC "Voting System Standards" document says that COTS software does
>   not have to be inspected if it is used in the construction of a voting
>   system.  So any voting machine using Win2K can claim Common Criteria
>   compliance, even though it may be riddled with security flaws!  PGN]
>
> ------------------------------
>
>

Relevant Pages

  • Re: EAL5 Certification for z10 Enterprise Class Server
    ... have gotten an EAL5 evaluation, but obviously I don't have all the details. ... The Common Criteria allows some small intended ... That's not to say it's a bad or improper protection profile. ...
    (bit.listserv.ibm-main)
  • Re: EAL5 Certification for z10 Enterprise Class Server
    ... have gotten an EAL5 evaluation, but obviously I don't have all the details. ... The Common Criteria allows some small intended ... That's not to say it's a bad or improper protection profile. ...
    (bit.listserv.ibm-main)
  • CC and Windows evaluation
    ... I am interested in evaluation of impact on the security ... of USB and FireWire devices with respect to the ... Common Criteria in Windows 2000, ... Windows 2000 passed the standardization proces, ...
    (Focus-Microsoft)
  • Re: Common Criteria Certification
    ... evaluation method with NSA, we have no plans now to abandon Common Criteria, ... and Steve Lipner has never made such a claim. ... We kicked off our Common Criteria evaluation of Windows Vista and Windows ... CC certification both here and abroad. ...
    (microsoft.public.windowsxp.security_admin)
  • Re: Using OEM to activate eval Server 2008
    ... "This software is for evaluation and testing purposes. ... Any edition of Windows Server 2008 may be installed without ... If you need more time to evaluate Windows Server 2008, ... You can't use an OEM key with a non-OEM copy of Windows, ...
    (microsoft.public.windows.server.setup)
Loading