Re: Content Filtering Statistics. What to expect?

From: Mike (michael.owen@hushmail.com)
Date: 12/31/02 

From: michael.owen@hushmail.com (Mike)
Date: 31 Dec 2002 11:07:16 -0800

brian_anon@hotmail.com (Brian E) wrote in message news:<737943bc.0212301724.193d7281@posting.google.com>...
> We currently use SurfControl to filter inbound and outbound email. I
> do not expect 100% accuracy with this technology. False positives
> (incorrectly blocking legitimate email) and false negatives (not
> blocking Spam or inappropriate email) will always be an issue.
>
> What is a reasonable expectation for false positive and false negative
> rates (in percentages of total mail processed)?
>
> Does anyone know if SurfControl publishes or makes claim to accuracy
> rates (I assume it would be based on false positive and false
> negative)?
>
> Does anyone know of third party reports comparing these rates amongst
> vendors or even in general what an organization can expect?
>
> Ultimately, I want to quantify the performance of the content
> filtering and be able to dismiss unrealistic expectations.

I worked for a fairly major player in the market for several years,
and know of no studies or reports, third party or otherwise, which
ever measured this. Any such study would only be a vague set of
numbers anyway - you can't pretend that you're going to have a data
set which accurately reflects the spam a company will be sent in the
next month.

Besides, if someone did do a study, as long as the systems were
configured with similar rulesets, the results would likely be the
same. Content filtering for spam is entirely about word matching,
stemming and fuzzy word matching, and most products are at about the
same level for this sort of work. If they all have the same wordsets
for matching for spam, they should all catch the same amount of spam.
The trick is getting a good wordset.

This all said, I'm not quite sure what Surfcontrol's "Anti-Spam Agent"
does - it sounds like a glorified wordlist to me. I've never played
with it, so can't really comment on it. 

--
Mike

Relevant Pages

  • Re: PLUG: PMAS
    ... The state of the art that can be obtained with DNSbsl is> 80% with out a DHCP list, and well into the 90% in spam detection. ... more good mail probably gets lost for other reasons beyond the mail server operator or network administrators control. ... So what DNSbls are you using that generate these higher rates of false positives? ... And most of the mail servers that I have seen allow local customization of the bounce message they send to their internal network users. ...
    (comp.os.vms)
  • Re: IMF and UceArchive folder
    ... could not get it to do what I needed, and decided to give the IMF a try. ... > 138 got a score of 1-3, making them almost certainly not spam. ... > potential false positives in the SCL range of 4-7." ... > It says you're a "Software Development Consultant" in your sig. ...
    (microsoft.public.exchange.admin)
  • Re: IMF and UceArchive folder
    ... The IMF has been running over the weekend at a level of 5. ... checking through every piece of archived email, and have no false positives. ... Some of the spam has been given a rating of 1 or 2, ... These guys are quoting MCP magazine. ...
    (microsoft.public.exchange.admin)
  • Re: IMF and UceArchive folder
    ... a lot of the stuff that waltzes right through IMF). ... 138 got a score of 1-3, making them almost certainly not spam. ... potential false positives in the SCL range of 4-7." ... MCSE+I, Exchange MVP ...
    (microsoft.public.exchange.admin)
  • Re: Open letter to Trend Micro
    ... I differ in that I want aggressive spam filtering, and I accept that this is ... Trend C/S/M is a pretty good product in this light, it does allow us to make ... >> If you don't get any false positives you either a) don't update your ... > Sure, it's logical to check the spam blocker on the recipient's end, ...
    (microsoft.public.windows.server.sbs)