Re: Certifications: Not worth the paper they are printed on?


(opening can of worms)

My previous experience of recruiters is that they will happily market anyone and its usually previous job experience that gets you through the door along with recommendations. I know because that is the case with me.

In terms with joining an pentest company we have have all CV's from recruiters (or HR) sent to the team leader, who then decides on who they interview, then once in the interview they are accessed on technical abilities and if they will be suitable to the team, then if we like them during said interview we put them on a Vmware based assult course and ask them to demonstrate said abilities. Sorts the men from the boys.

We dont look for people with CISSP (its nice if you have it, but your more of a security consultant (sorry!!) than a pen-test consultant; we dont actively push people to CISSP or CCNA - we want people with CREST or CHECK or in a position to be able to easily pass it.


R. DuFresne wrote:
Hash: SHA1

Perhaps once you get to the interview, but when attempting to get to the interview, with recruiters, HR and various contract agencies, the paper means the most.


Ron DuFresne

On Tue, 7 Oct 2008, Ray Chow wrote:

If you don't have the experience or the urge to understand how things work. That piece of paper (cert) will only help you to walk so far.

At the end of the day, you will only get some of the top infosec jobs by networking. People will know whether you know your stuff or bluff.
From: listbounce@xxxxxxxxxxxxxxxxx [listbounce@xxxxxxxxxxxxxxxxx] On Behalf Of R. DuFresne [dufresne@xxxxxxxxxxx]
Sent: Tuesday, October 07, 2008 6:23 AM
To: Jon Kibler
Cc: pen-test@xxxxxxxxxxxxxxxxx; pen-test-return-1078487202@xxxxxxxxxxxxxxxxx
Subject: Re: Certifications: Not worth the paper they are printed on?

Hash: SHA1

The main key here is though, it's an "industry". Money changes


Ron DuFresne

On Sun, 5 Oct 2008, Jon Kibler wrote:


Yesterday I was reading a blog where someone with no security experience
whatsoever was grousing that they flunked the Security+ exam. The
blogger also claimed to have over 100 certifications. In my opinion,
that many certifications undoubtedly qualifies this blogger to be the
Poster Boy for everything that is wrong with the certification process.

I do not know of anyone who has the real world experience to pass 100+
certification exams based only upon their experience. The fact that
someone can pass a certification exam WITHOUT ANY EXPERIENCE clearly
illustrates something is critically wrong with our industry's
certification process. (MCSE: Must Call Someone Experienced!)

The certification process today is utterly and completely broken. The
single biggest problem that I see with the certification industry is the
scarcity of "real world" certifications -- those certifications that
cannot be passed by book knowledge alone -- certifications that require
hands-on real-world experience to pass, such as the RHCE, CCIE, or any
of the GIAC Gold certifications. All certifications should be as
rigorous as these and similar certifications that reflect one's ability
to do real work in the area in which they are certified.

In my humble opinion, most certifications today are not worth the paper
they are printed on. Certifications were originally conceived as a means
to help weed out fictitious resumes, or to verify that someone claiming
to have "10 years of experience" is not someone who really has "the
equivalent of one year of experience, times ten."

However, the fact that so many certifications are so lame that anyone
can buy a book, memorize it, and take and pass an exam, shows how
critically broken is the certifications process. Most certifications
today do not show that you are capable of DOING anything except
memorizing mostly useless and dated facts.

Certifications have gone from something potentially useful and
meaningful to being the equivalent of Country Club Dues. It has become
the price of admission to join a certain group of people in the
workplace. Just like your ability to pay your country club dues does not
say anything about your ability to play golf, certifications say nothing
about your ability to do the work associated with the certification. We
need to change certifications from being country club dues to being more
like PGA tour qualifications.

The entire certification process needs to change. Certifications must
once again reflect an individual's ability to DO something, verses their
ability to memorize. When someone presents a certification, an employer
needs to have some confidence that the prospective employee can actually
do the job in the real world. What needs to change? At least four things
immediately come to mind:

1) Before taking a certification exam, you must be able to
demonstrate an auditable degree of associated work experience. For
example, the new Security+ certification calls for a minimum of 2 years
of day-to-day security experience as a recommended prerequisite. Well,
it should be made a REQUIREMENT that you MUST HAVE at least 2 years of
experience doing day-to-day security work before you are allowed to sit
for the exam.

2) Exams must be changed from being fact-based to become
experience-based. It should not be possible to simply read books and
pass an exam. For example, the Security+ exam should include questions
that only a security practitioner would be able to answer. It should
include packet captures and ask for an interpretation. It should require
you to be able to verify a digital signature. It should present log
files and ask you to identify how the system was compromised. Etc. Real
world experience-based questions should be an integral part of each
exam's questions. It should not be possible to pass the exam without the
required hands-on experience.

3) Certifications must have an expiration date. Knowledge in every
area of technology is transient in nature. Certifications must reflect
that they are based on the qualifications to do a job at a particular
point in time, and that those qualifications will change over time. As I
stated previously, the initial certification should require auditable
work experience. Recertification should require not only demonstrated
continued work experience, it should also require CEUs/CPEs to maintain
the certification. In fact, continuing education should be made an
annual requirement to maintain certifications between recertifications.

4) Instructors teaching certification courses *MUST* have
demonstrable real world work experience before being deemed qualified to
teach the certification course. Probably the two certifications with the
greatest "Instructor Qualification Laugh Factor" are the EC-Council's
CEH and CHFI courses. The majority of instructors that I have met that
teach either of these two courses have NEVER done ANY real work in
either associated profession.
-- How can an instructor properly convey to students the real thought
processes of a hacker, if they themselves have not performed dozens of
successful real world penetration tests?
-- How can an instructor properly convey to students all that they
need to know about forensics, if they themselves have never performed a
real world forensics examination, and prepared and presented evidence in
-- It is simply not possible to study, get a certification, and teach
these (and similar) courses without the instructor and ed center doing
an extreme disservice to their students. Instructors should be required
to not only have the certification, but they must have real world work
experience actually doing what they are teaching.
-- Instructors should also be required to maintain additional
CEUs/CPEs beyond those required to maintain certification. Attending two
relevant conferences a year should be mandatory. (I would bet that most
CEH instructors have never even been to Defcon! How many CHFI
instructors have ever attended TechnoForensics? I bet almost none have!)
Similar qualifications and continuing education needs to be mandated of
all instructors teaching in any area of technology.

Perhaps another analogy would help clarify my concerns. Would you hire a
pilot for your corporate jet that only has a certificate saying that
they had passed flight school ground training? Someone that had no
actual experience as a pilot? Would you want this same person teaching
other wannabe pilots? I would hope not!

However, that is the situation we find ourselves in with technology
certifications. We are getting hordes of people that simply "pass ground
school" and now claim to be "capable of flying a 747." Still worse, the
majority of our instructors for technology certifications have only
"passed ground school", but are using that as the basis to hang out
their shingle claiming that they can teach others to fly, when they
themselves have never even seen the inside of the cockpit of an
airplane, not less ever actually having piloted a real aircraft.

Until certifications can become a meaningful means of verifying a
claimed level of experience and expertise, they shall remain not worth
the paper they are printed on.

In the meantime, we in the industry need to educate our managers, and
our training and HR departments as to what certifications are meaningful
and which ones are not. At the same time, we need to be teaching them
what certifications are appropriate for a given job skill. For example,
I see CISSP mandated for numerous jobs (such as penetration tester)
where other more appropriate certifications should be used instead. But,
because CISSP is thought to be the ultimate certification in security,
they think that "one size fits all" security positions. We need help
change that thought process!

Jon Kibler
Jon R. Kibler
Chief Technical Officer
Advanced Systems Engineering Technology, Inc.
Charleston, SC USA
o: 843-849-8214
c: 843-224-2494
s: 843-564-4224

My PGP Fingerprint is:
BAA2 1F2C 5543 5D25 4636 A392 515C 5045 CF39 4253

gpg: Signature made Sun 05 Oct 2008 02:15:07 PM EDT using DSA key ID CF394253
gpg: Good signature from "Jon Kibler <Jon.Kibler@xxxxxxxx>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: BAA2 1F2C 5543 5D25 4636 A392 515C 5045 CF39 4253

- --
admin & senior security consultant:
Key fingerprint = 9401 4B13 B918 164C 647A E838 B2DF AFCC 94B0 6629

...We waste time looking for the perfect lover
instead of creating the perfect love.

-Tom Robbins <Still Life With Woodpecker>
Version: GnuPG v1.4.5 (GNU/Linux)


This list is sponsored by: Cenzic

Top 5 Common Mistakes in Securing Web Applications
Get 45 Min Video and PPT Slides

Relevant Pages

  • Re: Security+ Certification - Is the SANS materials enough?
    ... I have passed the CompTIA Security+ exam twice. ... The first edition of the exam(2003) and the most up-to-date and I can tell you that 2008's exam was alot *LESS* technical than the first edition. ... As far as the other certifications you have mentionned, i am not any familiar w/ the ACSP and have thoughed about becoming MCP but i remember reading an article when the world economy began to "colapse" that organisations not beeing able to pay back for their M$ licences would see their local MCPs handle the charges. ...
  • Certifications: Not worth the paper they are printed on?
    ... Yesterday I was reading a blog where someone with no security experience ... whatsoever was grousing that they flunked the Security+ exam. ... blogger also claimed to have over 100 certifications. ... Instructors teaching certification courses *MUST* have ...
  • Certifications: Not worth the paper they are printed on?
    ... Yesterday I was reading a blog where someone with no security experience ... whatsoever was grousing that they flunked the Security+ exam. ... blogger also claimed to have over 100 certifications. ... Instructors teaching certification courses *MUST* have ...
  • Re: To go to University - For the CISSP etc. - Good idea/Bad idea???
    ... I'd be majoring in Security of course... ... I studied IT - Networking at Technical College, ... But yes, I do have all those certifications, the 3 Cisco ones I did at ... Prove to peers and potential employers without a doubt that you can actually do a proper penetration test. ...
  • Re: IT Manager to CISO
    ... there's no comparing the two certifications. ... The Security+ exam is an entry level exam suitable for most people who are just entering the field. ... The CISSP is a well respected exam for people who are experienced and involved in designing and managing all forms of security at a high level. ... Sent from my BlackBerry® Smartphone ...