Re: [fw-wiz] VA vs PT tool

• Previous message: Devdas Bhagat: "[fw-wiz] HTTPS, proxies, and remote developers."
• Maybe in reply to: SimonChan_at_lifeisgreat.com.sg: "[fw-wiz] VA vs PT tool"
• Next in thread: Cat Okita: "Re: [fw-wiz] VA vs PT tool"
• Reply: Cat Okita: "Re: [fw-wiz] VA vs PT tool"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]

To: SimonChan@lifeisgreat.com.sg, firewall-wizards@honor.icsalabs.com
Date: Fri, 13 Jun 2003 18:39:13 -0500

>Hi fw-wiz,
>
>i posted some time on the list a couple of months back for some
>recommendations on a good VA tool.
>
>The bulk of the responses pointed to ISS, NetRecon and Vigilante.
>
>However, a VA tool is limited, in that it only stops at the vulnerability.
>
>I'm looking at a Pen Test tool that not only does the VA functionality but
>also exploit the vulnerability thus defining it as a real THREAT and not
> just a vulnerability.
>
>Is there a widely accepted tool on the market right now ?
>
>
>Rgds,
>Simon Chan, MCP/MCSA/CCNA/CCSA/WCSP
>Senior Security Engineer

Simon,

      I've been doing miscellaneous VA/Audit work for more than a few years
now, and I've worked with most of the popular VA products on the market on
and off. Moreover, I've been developing a VA product for the last year as
well, in part because I wanted to be able to have VA tool that did a few
things differently. The tools I've used most (based on a combination of
what I use myself and what my customers have purchased) are Nessus, ISS,
Netrecon, Retina, and up until they killed it, Cybercop. Beyond the
reasoning Ben gave in his response, I have to say the idea of someone
marketing the kind of tool you're asking about scares the heck out of me
for another reason as well:

      That old saying about statistics could easily be used to describe VA
tool output--"Lies, damn lies, and VA tool reports".

      The scary thing about a tool that purported to be "sure" about its
results is that people without the technical skills to analyze those
results might actually believe what it said. Plenty of exploits don't work
100% of the time anyway, and the worst thing in the world would be a "sure"
tool that coughed up a false-negative to someone who trusted it. The only
way this could work at all is if the tool only added something like
"verified" to its report for the items it could exploit--but in practice I
bet that would end up working just the same. The people I'm talking about
(and there are plenty of them, trust me) would just start to ignore the
unverified entries. Besides, a tool like this would surely be more likely
to cause problems on your network.

      Ben's response contained what I firmly believe--that the key factor
in successful technical vulnerability assessment is human. The person
interpreting the results, and following up on them, is the most important
piece of the puzzle. The products all (even the good ones) suck in some
way or another. The only way to get anything like accurate results is to
have something with opposable thumbs taking automated tool reports as one
necessary part of an information gathering process that includes more
intelligence than the tools can possess.

      Of course Ben's response also included what I think is an unjust shot
at Nessus. In my experience *all* of the tools are capable of screwing up
something on a production network, not just Nessus. Configured correctly
Nessus is no worse than most and better than some. IMNSHO Nessus is the
only product in this class that is worth as much or more than what you paid
for it. I'm often in the position of testing with both Nessus and another
(commercial) vulnerability assessment tool, and I've found that the biggest
difference between them is fairly small--their results mostly overlap, with
each one finding something useful the other didn't. Of course the other
not so minor difference is the $20,000 gap between the two when it comes to
testing a large environment. There are legitimate places to pick on Nessus
(occasional instability and weak data manipulation/reporting are a couple
that jump to mind) but I think suggesting it will burn down your network is
a bit silly. I've used it on plenty of production networks, and many of my
customers run it regularly on their production networks--with no unusual
amount of pain and suffering.

      Finally, if you have somebody performing a pen-test that's worth
their salt they won't need a tool that takes one step farther for them
anyway. Pen-testing can't be performed by software, not now and without
some sort of high-functioning AI probably not ever. No program is ever
going to be able to do even some of the simplest things that I've done when
testing (and I am a self-confessed moron). If pen-testing was just running
some v-scanners and then firing up a library of exploits (adding up to what
your "PT scanner" would amount to) I'd probably be out of a job. Consider
this, a human with a tiny bit of gray matter can often break into a company
that has every system patched 100% up to date. Real life example from a
few years ago for you: I was the check-up tester following a big-5
assessment and related round of fixes for Company X. Company X had
twenty-odd servers that could be contacted on one or more ports from the
Internet, all carefully DMZ'd behind clustered top-shelf
firewalls. V-scanners found *no* flaws on any of those systems. One of
the systems was a Citrix server that could be contacted on port 1494 from
the Internet. Firing up the Citrix client proved that the server allowed
the creation of custom connections. The same company's website had an
employee directory. Simple password guessing using the supplied list of
names to generate passwords gained access after just an hour or two of
work. Access that lead to the compromise of key internal systems. Your
hypothetical PT scanner would have just spit out "Everything's locked
down--good job!" in that instance--and it would have been dead wrong.

      Just my retarded opinion, certainly not my company's,

Greg

P.S. Other than giving Nessus a kick, I pretty much agree with everything
Ben had to say, except for this: Ben, you're wrong about nobody getting
it. Love the tick. :)

==============================
Greg is, among other things, a moron.
Anything he has said above is solely his
own opinion, not that of his employer.
==============================

_______________________________________________
firewall-wizards mailing list
firewall-wizards@honor.icsalabs.com
http://honor.icsalabs.com/mailman/listinfo/firewall-wizards

• Previous message: Devdas Bhagat: "[fw-wiz] HTTPS, proxies, and remote developers."
• Maybe in reply to: SimonChan_at_lifeisgreat.com.sg: "[fw-wiz] VA vs PT tool"
• Next in thread: Cat Okita: "Re: [fw-wiz] VA vs PT tool"
• Reply: Cat Okita: "Re: [fw-wiz] VA vs PT tool"
• Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]