[OT] Spam filtering (was Re: [Imap-uw] Outlook deadlock)

[Per Foreby]

On Wed, 19 Sep 2007, Joel Reicher wrote:

>> Unfortunately, the problem with server-side spam filtering (we do it too!)
>> is not the false negatives; it's the false positives!  I've lost important
>> mail due to server-side spam filtering...
>
> FWIW I am religiously opposed to content-based filtering for exactly
> this reason.
>
> I "filter" only with blacklisting and greylisting. Although they can only
> be done at a boundary mail exchanger, I find it invaluable that there's
> really no such thing as a false positive; if legitimate mail bounces then
> it, well... bounces.

If the mail really bounces (is returned to the real sender) then the 
sender can take action. But if you accept the email, filter it, and 
then try to do some sort of semi bounce by looking at the header 
addresses, mail will eventually be lost.

We do all filtering at our incoming mail server while the smtp 
connection still is active (using sendmail milters) and thus can return 
an error message to the connecting mailserver for a true bounce.

The milters used are milter-greylist, spamass-milter and clamav-milter 
in that order. Greylisting stops most of the junk mail, but enough get 
through to need a dedicated server to run spamassassin and clamav.

We do have false positives, but the ones I'm aware of are 1-2 each year 
for over 10,000 users.  Of course, not having English as your native 
language really helps to avoid false positives. And of course the number 
depends on your line of work. If I was at the faculty of medicine, doing 
research on Viagra, I would probably hate spam filters. We whitelist 
people on request if they have these kind of problems. (I have 2 
whitelisted users at the moment, and one whitelisted server).

Tagging vs. rejecting (bouncing) junk mail in an interesting 
discussion. On thing often forgotten in these discussions is the number 
of false positives created when people browse through tagged email, 
quickly deleting everything based on the Subject header (or not 
looking at all).

/Per