Viewable With Any Browser Your vote? Vote NO Vote YES


Bayesian Message Filtering for Python
Integrating Python with DSPAM

by Stuart D. Gathman
This web page is written by Stuart D. Gathman
sponsored by Business Management Systems, Inc.
Last updated Oct 04, 2011

Downloads,Bugs, Header Triage

This project provides Python support for fast sophisticated bayesian message filtering. It is based on the excellent DSPAM project provided by Jonathan A. Zdziarski. I have moved RPMS for dspam to a separate project. Neither BMS or Stuart Gathman are affiliated with Jonathan Zdziarski or Network Dweebs, except as enthusiastic users of their free product. Dspam was chosen because it provides a library with a C API in addition to a complete MDA based spam filtering application. Python applications use the C API through an extension module. Using a C library is faster than a pure Python bayesian filter.

What is DSPAM? Here is an excerpt from the DSPAM project README:

DSPAM is an open-source, freely available anti-spam solution designed to combat unsolicited commercial email using Baye's theorem of combined probabilities. The result is an administratively maintenance free system capable of learning each user's email behaviors with very few false positives.

DSPAM can be implemented in one of two ways:

  1. The DSPAM mailer-agent provides server-side spam filtering, quarantine box, and a mechanism for forwarding spams into the system to be automatically analyzed.
  2. Developers may link their projects to the dspam core engine (libdspam) in accordance with the GPL license agreement. This enables developers to incorporate libdspam as a "drop-in" for instant spam filtering within their applications - such as mail clients, other anti-spam tools, and so on.
Many of the ideas incorporated into this agent were contributed by Paul Graham's excellent white paper on combatting SPAM. Many new approaches have also been implemented by DSPAM.

DSPAM RPM support for Python

A Python In dspam-2.6 RPMS, the pydspam project is included as the dspam-python sub-package which is built from the pydspam source for pydspam version prior to 1.1.5. If you don't wish to build the python package, set the build_python macro to 0 at the top of the RPM spec file in the source RPM.

Beginning with pydspam-1.1.5, pydspam is its own RPM which obsoletes dspam-python. The dspam-python or pydspam binary RPM provides a Python module which wraps the dspam core engine (libdspam). Some of the dspam command line tools are reimplemented in Python to illustrate use of the library. (Installed as documentation by the RPM.) A new tool,, shows the contribution each token of a message makes to the total DSPAM score.

In dspam-2.8, pydspam has its own RPM.

Header Triage with Dspam and Python Milter

For a really powerful mail filtering system, combine the DSPAM Python module with sendmail and Python Milter. For instance, here is a simple change to milter-0.5.5 I am testing: Patch to from milter-0.5.5.

The dictionary is the one maintained by the dspam delivery agent installed with the dspam package. Scanning the headers in the milter allows us to REJECT common spams without a lot of processing.

To show just how bad the spam problem is, here are statistics for our domain with just 6 users. Two users (including me) are published on the web with HTML encoding. I also use my real email when posting to newsgroups. Because my email is acessible, I receive welcome email from fellow techies all over the world.

Statistics for Jul 15
1139 Messages from known spamming domains refused by sendmail.
160 Messages REJECTED by milter because of banned keywords like 'viagra'.
169 Messages REJECTED by milter because of high Dspam scores for headers.
261 Messages quarrantined by Dspam mail delivery agent.
40 Actual email received for 6 users.

We do not use a black hole list for known spamming IPs / domains. This is because some of our customers use blacklisted ISPs because they are the only broadband available in their area. Black hole lists like to blacklist entire ISPs, including innocent customers who have no other choice (other than dialup) for connectivity. With a little python programming to collect data, DSPAM will allow us to automate building the list of banned IPs / domains.

The header triage feature will be in milter-0.5.6. I envision a complete milter based implementation of dspam which appoints selected email destinations as 'moderators'. The MDA approach currently used by dspam requires all users to diligently classify their email to train the filter. In the new approach, moderators will do this work, and the resulting dspam dictionary used to filter mail for other users in their group.


Source and binaries have been moved to the Pymilter project on Sourceforge. Older binaries can still be found below.


RPMs for dspam have been moved to the libdspam project. I am working on a pydspam RPM for dspam-2.10. Currently, pydspam RPMS require dspam-

Activating the DSPAM CGI script

The RPM installs the CGI interface in the /var/www/cgi-bin/dspam directory. A wrapper script is installed as /var/www/cgi-bin/pydspam.cgi. The wrapper script runs the DSPAM CGI interface as the dspam user - which is also a member of the mail group.

To enable the CGI interface, you need to add an authorization entry to /etc/httpd/conf/httpd.conf. For example,

    ScriptAlias /cgi-bin/ "/var/www/cgi-bin/"

    # "/var/www/cgi-bin" should be changed to whatever your ScriptAliased
    # CGI directory exists, if you have that configured.
    <Directory "/var/www/cgi-bin">
	AuthName Dspam
	AuthType Basic
	AuthUserFile /etc/httpd/conf/passwd
	AuthGroupFile /etc/httpd/conf/group
	Require group dspam
        AllowOverride None
        Options None FollowSymLinks
        Order allow,deny
        Allow from all
You must also modify the script at /var/www/cgi-bin/dspam/ to change the DOMAIN configuration to your domain at a minimum.

pydspam Binary RPM

Binary RPMs are compiled for python2.3. Goto the Sourceforge site for pydspam-1.1.8 compiled for python2.4.
  • pydspam-1.1.7-1.i386.rpm Binary RPM for RH7.2.
  • pydspam-1.1.7-1.i386.rpm Binary RPM for RH7.3.
  • pydspam-1.1.6-1.i386.rpm Binary RPM for RH7.2.
  • pydspam-1.1.6-1.i386.rpm Binary RPM for RH7.3.
  • pydspam-1.1.5-1.i386.rpm Binary RPM for RH7.2.
  • pydspam Source RPM

  • pydspam-1.1.7-1.src.rpm Source RPM for RH7.x.
  • pydspam-1.1.6-1.src.rpm Source RPM for RH7.x.
  • pydspam-1.1.5-1.src.rpm Source RPM for RH7.x.
  • Sources

  • pydspam-1.1.7.tar.gz Python interface to libdspam and some dspam utilities in python.
  • pydspam-1.1.6.tar.gz Python interface to libdspam and some dspam utilities in python.
  • pydspam-1.1.5.tar.gz Python interface to libdspam and some dspam utilities in python.
  • pydspam-1.1.4.tar.gz Python interface to libdspam and some dspam utilities in python.
  • pydspam-1.1.3.tar.gz Python interface to libdspam and some dspam utilities in python.
  • pydspam-1.1.2.tar.gz Python interface to libdspam and some dspam utilities in python.
  • pydspam-1.1.1.tar.gz Python interface to libdspam and some dspam utilities in python.
  • pydspam-1.0.tar.gz Python interface to libdspam and some dspam utilities in python.

  •  [ Valid HTML 3.2! ]  [ Powered By Red Hat Linux ]

    Send Spam