8 ways to fight spam filter frustration

Recurrent pattern detection

This proprietary technique relies on the fact that a spam outbreak, by definition, involves widespread distribution of e-mail. The RPD system, developed and maintained by security vendor Commtouch, monitors the Internet for such outbreaks and determines the patterns they contain, then updates a central database of spam patterns. (Commtouch both sells its own antispam products and licenses the RPD technology to other antispam vendors.) Company e-mail systems using RPD query the database, and e-mail identified as spam is discarded or quarantined.

Tips for combating false positives

On both the sending and the receiving end, minimizing false positives is critical for your organization. The real challenge comes from the fact that any or all of the spam-filtering techniques listed above may be employed on your own systems and on the systems of your recipients. Here are some steps you can take.

1. Do use a spam filter

The occurrence of false positives can leave you wondering if you should simply toss your spam filter -- don't.

False positives can occur even without using a filter, such as when a user, seeing multiple spam subjects in an in-box, manually hits "delete" multiple times, not realizing that buried within that list is a "good" e-mail. A state-of-the-art spam filter, on the other hand, will catch 97 per cent to 99 per cent of spam, according to Ferris Research's Jennings, thus preventing the indiscriminate manual deleting scenario. And although spam filters can incur false positives, their rate of doing so is far lower (as low as.01 per cent) than is incurred through pure human action, says Jennings.

2. Locate your filter at the network DMZ

A demilitarized zone (DMZ) in the context of a computer network refers to a portion of that network that buffers the private internal network from the public Internet. The systems in the DMZ are vulnerable to attacks from the outside, but their presence protects the internal network from outside attacks.

Putting your spam filter at the DMZ, according to Jennings, allows it to monitor the characteristics of the connection and acquire more information about incoming e-mail messages, which can be critical to determining whether the message is spam or not. "If the sender is a Windows ME box," he says, "why would it be sending me e-mail directly, rather than going through a legitimate e-mail server? In that case, it's almost certainly a zombie, so the message is going to be spam."

3. Move away from older filtering technologies

Michael Briggs, director of information technology at The George Washington University Law School, recommends moving away from old-fashioned keyword technologies in favor of newer techniques such as graylisting. (See "How CAPTCHA got trashed.")

In the same way, Jennings has strong concerns about challenge response systems, saying they're "simply a terrible idea." He points out that a legitimate sender might never see the challenge message, because that message itself could be flagged as spam, and because spammers often disguise spam as such a message.