Artificial Intelligence Aids the War on Spam

Paul Graham, is an interesting writer and a fresh thinker. On his website, he publishes on a frequent basis on topics such as Lisp, programming languages, and artificial intelligence. His latest interest is in fighting spam.

The war on spam actually raises some interesting points. For one, there seems to be developing, an arms race, in a sense. Technology of the spammers versus technology of the anti-spammers.

The idea of spam is very intriguing. As am owner of a business, I feel quite good at the prospect of my message reaching thousands and thousands of potential viewers quickly and in a personalized way. In a sense, it must be a marketers dream.

The fact that it is a marketers dream is what kills the spammers though. Emails that have been sent out by a marketers mind, will have a particular style to them. They will quite things like 'instant access' or include pricess like $19.95.

The Bayesian filter that Graham is proposing would be customized to one users set of email. The engine would parse through two sets of emails, one designated spam, and the other is desired mail. Indexes will be built cataloging how frequently various terms or tokens can be spotten. The effectiveness of this plan is comes from applying artificial intelligence to each individual members email.

The greatest concern in any sort of anti-spamming system is a 'false positive'. This is when a message is marked as being spam when in fact it is not! Even though the system that Graham has developed finds false positives less than 1% of the time, it is still greatly hampered. Any system that incorporates this spam filtering could never be fully trusted.

What is a good analogy here? It depends on the nature of the email that you get. What are the consequences of missing one email? It certainly depends on the email. But it might have been your boss, or a new client, or even a new friend, who doesn't know that sometimes I don't reply to every single email.

http://paulgraham.com/better.html

related stories, by category: