Revision: See additional information at the end of this HOWTO regarding configuration to exclude internal IP's from being scanned against RBL lists.

Your sme server receives a lot of spam email and you want to reject it before it enters your servers mail system.

Current methods typically use Spamassassin or similar spam detection software. These methods are processor & memory intensive and still require user intervention to administer when reviewing spam email in the junk folder or Inbox prior to deletion.

sme server v6.0 (final release or better) has a feature in smtpfront-qmail which allows incoming messages to be rejected if the sender is on nominated Real Time Blacklist or Blocklist (RBL) lists. Note that this function was not available in earlier versions of sme server by default.

Email messages are rejected if the sender is identified on nominated RBL list(s), and as a result there is no further processing or manual checking required. In practice a large number of spam messages will be rejected, perhaps 50 - 75 % depending on which lists you use and the type of spam your system (email addresses) are exposed to.

A by product is that there will also be a significant reduction in virus infected email messages entering the server, probably due to the fact that virus infected messages come from similar sources as spam messages.

This method works OK for servers configured as Server & Gateway or Server Only as long as the mail server components are enabled (smtpfront-qmail & qmail) and the server has access to the Internet via another sme server or firewall.

This feature should be used in conjunction with spam filtering software and virus scanning software, although these programs will have a lot less work to do. It has been tested on a server with Spamassassin and Clamavis-ng installed and works very effectively.

The RBL blocking feature and ASSP are not compatible with each other. You need to uninstall ASSP before using RBL blocking feature. Effectively ASSP is obseleted by RBL blocking.

RBL blocking should be compatible with other brands of spam & virus software based programs. They generally scan or filter the message after it has been accepted by the servers mail system.

RBL blocking occurs before the message is accepted, and if a detection occurs the message is rejected so it would never be scanned by secondary software based systems. Incompatibilities are therefore unlikely except with other programs that also use this method.

An additional feature which may be worth implementing is "Pattern matching blocking" using smtpfront-qmail, to reject messages that have executable content in attachments, which effectively includes a large number of currently known viruses. Additional patterns can be created to cover "new" viruses as they are "developed". Pattern matching acts as a "gross filter" to reject many known virus types, but a regularly updated virus scanner is still required to catch new viruses. See separate HOWTO for details on this.

This how to is based on forum posts and my own investigations, thanks particularly to Greg Zartman and thanks to Charlie Brady for implementing this feature in sme v6.0. Thanks also to Gordon Rowell for reviewing the HOWTO and suggesting some improvements.

You will first need to decide which RBL list (or lists) to use.

See separate section below for information on RBL lists.

WARNING: Choose lists carefully and verify that the list characteristics match your requirements. DO NOT just add all the lists to the RBLList property as you may not receive email that you wish to receive. A safer approach is to add lists one at a time and verify that your system is functioning as required, then add further lists as needed (also one at a time).

All the lists shown below as "conservative" appear safe to use, and I have them all enabled on my system.

The lists shown below as "aggressive" block too many sites (also applies to some other lists not shown), and I DO NOT recommend their use.

Your requirements may be different to mine, so please assess the suitability of the lists for your own purposes.

The RBLList feature only works if the correct version of e-smith-mailfront is installed.

It should be installed by default in a properly installed or upgraded system running sme server v6.0 or v6.0.1. Note that v6beta has an older version of the rpm & needs to be upgraded.

You MUST have this rpm installed (or a more recent version)

e-smith-mailfront-1.3.0-11.noarch.rpm

Note that if you have updated your system to support Pattern Matching (see seperate HOWTO) you may already have a more recent version

ie

e-smith-mailfront-1.5.0-12

To check if the correct version is installed do

rpm -qi e-smith-mailfront

It should show (or similar)

Name : e-smith-mailfront
Version : 1.3.0 Release : 11
Build Date: Thu 11 Sep 2003 06:06:30 AM WST
Install date: Sat 10 Jan 2004 07:51:05 AM WST
Build Host: sme60build.nssg.mitel.com

If you have an older version of rpm listed

eg

e-smith-mailfront-1.2.0-01.noarch.rpm
you should upgrade your system to the latest version of the sme operating system.

/sbin/e-smith/config setprop smtpfront-qmail RBLList sbl-xbl.spamhaus.org

/sbin/e-smith/expand-template /var/service/smtpfront-qmail/runenv

svc -t /service/smtpfront-qmail

To add multiple RBLs to the RBLList property, delimit (separate) them with a colon

Note that using more lists will result in more queries being sent & received over your Internet connection. Some lists are included on other lists so be careful not to include "double listings" as these only result in extra unnecessary queries.

/sbin/e-smith/config setprop smtpfront-qmail RBLList sbl-xbl.spamhaus.org:whois.rfc-ignorant.org:dnsbl.njabl.org:relays.ordb.org

(the above should all be on one line)

/sbin/e-smith/expand-template /var/service/smtpfront-qmail/runenv

svc -t /service/smtpfront-qmail

/sbin/e-smith/config setprop smtpfront-qmail RBLList sbl-xbl.spamhaus.org:dsn.rfc-

ignorant.org:postmaster.rfc-ignorant.org:abuse.rfc-ignorant.org:whois.rfc-ignorant.org:bogusmx.rfc-

ignorant.org:dnsbl.njabl.org:relays.ordb.org:dnsbl.sorbs.net:list.dsbl.org

(the above should all be on one line)

/sbin/e-smith/expand-template /var/service/smtpfront-qmail/runenv

svc -t /service/smtpfront-qmail

/sbin/e-smith/config delprop smtpfront-qmail RBLList

/sbin/e-smith/expand-template /var/service/smtpfront-qmail/runenv

svc -t /service/smtpfront-qmail

As mentioned above, using more lists will result in more queries being sent & received over your Internet connection but should result in more spam being rejected.

Some lists are included on other lists so be careful not to include "double listings" as these only result in extra unnecessary queries, potentially slowing down the list servers response times.

Choose RBL lists carefully to ensure they meet your needs.

Some lists are very aggressive in the implementation of their "inclusion" policy, and while using those lists may block more spam they will also block legitimate messages.

You can read the "criteria for inclusion policies" on each list at the list owners web site. The web site addresses are readily discernible from the list names. See Web sites section below.

For example using the bl.spamcop.net list will result in email messages from yahoo, hotmail & earthlink accounts being rejected. If you have legitimate users sending messages from those types of accounts, then do not use the bl.spamcop.net list. This also applies to some other lists.

Inclusion on a list can happen for many reasons, including being a known spammer or having a dynamic dial up IP number or sending via open relay servers or having incorrect address information or being listed by a system admin after receiving a spate of unsolicited email. Inclusion on "conservative" lists usually requires a positive identification of spamming or similar type activity. It is possible for legitimate users to be listed as part of a "block listing" of an IP number range such as has happened with Telstra Bigpond, AOL & other "large" ISP's etc. These listings are generally temporary until the specific spam culprit is identified and has their account cancelled by the ISP.

Here is a list of what appear to be "conservative & safe" lists ie there is justifiable or provable reason for being included on these lists. This is by no means an exhaustive list but is the result of my own investigations and conclusions.

Note that all the lists except spamhaus.org include open relays, so using these lists will block email sent via open relays.

sbl-xbl.spamhaus.org - (a combination of the two spamhaus lists)

dsn.rfc-ignorant.org

postmaster.rfc-ignorant.org

abuse.rfc-ignorant.org

whois.rfc-ignorant.org

bogusmx.rfc-ignorant.org

dnsbl.njabl.org

relays.ordb.org

dnsbl.sorbs.net

list.dsbl.org

blackholes.mail-abuse.org

relays.mail-abuse.org

dialups.mail-abuse.org

cbl.abuseat.org - (included in xbl.spamhaus.org)

opm.blitzed.org - (included in xbl.spamhaus.org)

dynablock.njabl.org - (was dynablock.easynet.nl)

bl.spamcop.net

(Note that too many legitimate ISPs do not conform to this lists requirements. The use of this list will cause too many legitimate messages to be blocked so its use is not recommended

ipwhois.rfc-ignorant.org

contacts.abuse.net

http://www.spamhaus.org/

http://www.abuse.net/

http://dsbl.org/main

http://mail-abuse.org/

http://www.sorbs.net/

http://www.spews.org/

For a brief overview of the rblsmtpd program

http://cr.yp.to/ucspi-tcp/rblsmtpd.html

For a brief overview of the tcpserver program

http://cr.yp.to/ucspi-tcp/tcpserver.html

After you have enabled the RBLList property you can check your settings as follows

/sbin/e-smith/db configuration getprop smtpfront-qmail RBLList

which will give an output something like the following

(Note that your servers output may be different depending on your configuration)

sbl-xbl.spamhaus.org:whois.rfc-ignorant.org:dnsbl.njabl.org:relays.ordb.org

If you want to check the complete entry for smtpfront-qmail do this

/sbin/e-smith/db configuration show smtpfront-qmail

which will give an output something like the following

(Note that your servers output may be different depending on your configuration)

smtpfront-qmail=service

ExternalInterfacesFilter=/usr/bin/qmail-queue.amavis

Instances=40

InternalInterfacesFilter=/usr/bin/qmail-queue.amavis

MaxMessageSize=10500000

Patterns=enabled

PatternsFile=/var/qmail/control/patterns.default

Proxy=enabled

RBLList=sbl-xbl.spamhaus.org:whois.rfc-ignorant.org:dnsbl.njabl.org:relays.ordb.org
access=public

status=enabled

For the benefit of users unfamiliar with some of the other settings:

MaxMessageSize controls the maximum size of smtp email messages including attachments

Patterns entries relate to the use of Pattern Matching blocking (see separate HOWTO)

By reviewing /var/log/smtpfront-qmail/current and var/log/smtpfront-qmail/* you can see the entries for rejected messages and generally enough information as to why the rejection occurred, and therefore see the effectiveness of RBL list blocking. Note that you will only see these type of entries after blocking has been enabled and messages have been rejected.

If you do not see all of the types of entries shown below, it would most likely be due to NOT having the particular RBL list enabled.

Enable more lists, as per the "conservative" lists above, then you will get a higher rejection rate and see more types of entries in the logs.

You can view date formatted logs using the Server Manager View log files panel

To see ALL the log entries do

grep "" /var/log/smtpfront-qmail/current | tai64nlocal

To see only the rejected message entries and the reason for rejection do

grep rblsmtpd /var/log/smtpfront-qmail/current | tai64nlocal

Here is an example of some typical entries

2004-04-15 13:19:17.256098500 rblsmtpd: 68.71.192.205 pid 24955: 451 68.71.192.205 has inaccurate or missing WHOIS data at the RIR

2004-04-15 13:19:53.769569500 rblsmtpd: 68.248.47.5 pid 24961: 451 Dynamic IP Address See: http://www.dnsbl.sorbs.net/cgi-bin/lookup?IP=68.248.47.5

2004-04-15 14:20:47.275359500 rblsmtpd: 68.184.114.157 pid 26931: 451 http://www.spamhaus.org/query/bl?ip=68.184.114.157

2004-04-15 15:46:25.334425500 rblsmtpd: 144.135.24.156 pid 29623: 451 Spam Received See: http://www.dnsbl.sorbs.net/cgi-bin/lookup?IP=144.135.24.156

2004-04-15 15:49:07.352709500 rblsmtpd: 66.18.69.3 pid 29728: 451 Inaccurate or missing WHOIS data

2004-04-15 15:51:26.127937500 rblsmtpd: 202.105.138.34 pid 29768: 451 http://dsbl.org/listing?ip=202.105.138.34

The following information is from a forum post at

This information has not been tested by the author of this HOWTO, but is provided for the benefit of readers and is included here "as is".

rbl exclusion lists

Author Message davidbray

--------------------------------------------------------------------------------

I'm using the spamfilter_install.sh package (http://sme.swerts-knudsen.dk/)

It's great but;

Is there provision there to put an ip exclusion range into the rbl list

I'm using the mail server for a virtual ISP, he has dial up clients and they are using smtp authentication. But rbl checks before smtp auth kicks in.

They have managed to get the ip's onto lists at dsbl.org - I've deleted dsbl.org that from my lists of rbl servers, but really want to put in the ip as an exclusion range.

I'm thinking I may have to look at the http://cr.yp.to/djbdns/rbldns.html program and the rblsmtpd with the -a option

just checking before engaging the learning curve part of the brain (hurts)

_________________

David Bray...

Author Message davidbray

--------------------------------------------------------------------------------

Howto use the rblsmtpd's -a switch to bypass RBL

================================================

When you use an RBL ala http://sme.swerts-knudsen.dk/ every internal ip is queried in the external rbl engine

to test this tail -f /var/log/dnscache/current and telnet to you server on port 25

this is how i fixed it (sorry bout the wrapping).

Howto use the rblsmtpd's -a switch to bypass RBL

================================================

Code:

mkdir -p /etc/e-smith/templates-custom/var/service/tinydns/root/data

mc -e /etc/e-smith/templates-custom/var/service/tinydns/root/data/non-rbl

this goes in non-rbl

{

#----------------(taken from

# /usr/lib/perl5/site_perl/esmith/util.pm - computeLocalNetworkReversed

sub computeReverse ($$)

{

my ($ipaddr, $netmask) = @_;

my @addressBytes = split(/\./, $ipaddr);

my @maskBytes = split(/\./, $netmask);

my @result;

foreach ( @maskBytes )

{

last unless ($_ eq "255");

unshift(@result, shift(@addressBytes));

}

return join('.', @result);

}

#------------(end of

# taken from /usr/lib/perl5/site_perl/esmith/util.pm - computeLocalNetworkReversed

#----------------(taken from functions)----------------

# Compute local IP address, netmask and network values.

#------------------------------------------------------

my $ipaddrBits = esmith::util::IPquadToAddr ($LocalIP);

my $netmaskBits = esmith::util::IPquadToAddr ($LocalNetmask);

my $networkBits = $ipaddrBits & $netmaskBits;

my $maxHostid = ((~ $netmaskBits) & 0xffffff) - 1;

$maxHostid = ($maxHostid <= 65534) ? $maxHostid : 65534;

#--------------(end taken from functions)--------------

$OUT .= "# Reverse Lookups for RBL\n";

for ($i = 1; $i <= $maxHostid; $i++)

{

my $ip = esmith::util::IPaddrToQuad ($networkBits | $i);

my $reverse = computeReverse ($ip, $LocalNetmask);

# $reverse =~ s/\.$//;

$OUT .= "+" . $i . "." . $reverse . "." . get_local_domainname() . ":127.0.0.2\n";

}

}

/sbin/e-smith/signal-event host-modify

mkdir -p /etc/e-smith/templates-custom/var/service/smtpfront-qmail/runenv

cp /etc/e-smith/templates/var/service/smtpfront-qmail/runenv/10RBLLookup \

/etc/e-smith/templates-custom/var/service/smtpfront-qmail/runenv/

mc -e /etc/e-smith/templates-custom/var/service/smtpfront-qmail/runenv/10RBLLookup

edit 10RBLLookup to look like

{

my @rbllist = split /:/, ${'smtpfront-qmail'}{RBLList} || '';

if (scalar @rbllist)

{

$OUT = 'RBLSMTPD="/usr/local/bin/rblsmtpd -a your.domainname ';

$OUT .= join " ", map { "-r $_" } @rbllist;

$OUT .= '"';

}

else

{

$OUT = "# No RBLs are defined";

}

}

/sbin/e-smith/signal-event email-update

svc -t /service/smtpfront-qmail

_________________

David Bray...

Author Message sqlerror

--------------------------------------------------------------------------------

Hello David,

I was searching for this! Our ISP's DNS systems are now and then unreachable and that interferes with sending out e-mails from within the LAN beceause no RBL servers can be reached.

If this DNS failure occurs, some 60 users are stuck as their application freezes when outgoing mail can not be accepted by the SME server in a timely matter. The only workaround at this moment is to run a script that disables the RBL lookups to enable e-mail delivery to the SME. But that is not desired as the spam flows in as soon as the DNS of the ISP is up again....

Any way, as far as I can tell, the scripts in your HOWTO assume a netmask of 255.255.255.0. We run a non standard netmask of 255.255.252.0. Will changing the part ($_ eq "255") into ($_ eq "252") enable passing the rbl for local sent mail with our netmask i.e. :

Code:

foreach ( @maskBytes )

{

last unless ($_ eq "255");

unshift(@result, shift(@addressBytes));

}

Into:

Code:

foreach ( @maskBytes )

{

last unless ($_ eq "252");

unshift(@result, shift(@addressBytes));

}

Greetz,

Geert Jansen