Mine setup is quite hardcore, but efficient. It is inteneded for use not by providers, but for companies that do not like spam and value their time and traffic. This article provides not a "complete solution", but describes separately all filtering techniques. Here we assume that our mail server has external IP address 1.2.3.4.
Here are essential configuration options in master.cf:
disable_vrfy_command = yes
smtpd_helo_required = yes
smtpd_sender_restrictions = reject_non_fqdn_sender
reject_unknown_sender_domain
permit_mynetworks,
smtpd_recipient_restrictions = reject_non_fqdn_recipient,
reject_unknown_recipient_domain,
permit_mynetworks,
reject_unauth_destination
These configuration options are obvious and self-explaining. But the spammers are smart enough
to make their mail "look good" for these restrictions.
smtpd_client_restrictions = ...
reject_rbl_client bl.spamcop.net,
reject_rbl_client list.dsbl.org
reject_rbl_client dul.ru,
...
This option is much desired.
Now if the IP address of the client is in one of these lists then it is rejected. These lists are quite
accurate, and use of them reduces spam considerably. Of course it can be that some provider's mail server
can get in the list, but it happens rarely and because they are really lame.
There are other DNS black lists, you may find many of them on
http://www.robtex.com/rbls.html.
mime_header_checks = regexp:/usr/local/etc/postfix/mime_header_checksmime_header_checks looks like:
Teach your friends not to send you attachments with "bad" extensions. However most of the bad things come now inside ZIP archives, but from time to time worms with "Update-KB3374-x86.exe" are noticed, so having this is desired.
smtpd_helo_restrictions = ...
regexp:/usr/local/etc/postfix/dsl_stoplist.txt
...
smtpd_client_restrictions = ...
regexp:/usr/local/etc/postfix/dsl_stoplist.txt
...
dsl_stoplist.txt looks like:
/^dsl.*\..*\..*/i 553 AUTO_DSL You have been identified as a spammer. Go Away. /[ax]dsl.*\..*\..*/i 553 AUTO_XDSL You have been identified as a spammer. Go Away. /client.*\..*\..*/i 553 AUTO_CLIENT You have been identified as a spammer. Go Away. /cable.*\..*\..*/i 553 AUTO_CABLE You have been identified as a spammer. Go Away. /pool.*\..*\..*/i 553 AUTO_POOL You have been identified as a spammer. Go Away. /dial.*\..*\..*/i 553 AUTO_DIAL You have been identified as a spammer. Go Away. /ppp.*\..*\..*/i 553 AUTO_PPP You have been identified as a spammer. Go Away. /dslam.*\..*\..*/i 553 AUTO_DSLAM You have been identified as a spammer. Go Away. /node.*\..*\..*/i 553 AUTO_NODE You have been identified as a spammer. Go Away.Now if the reverse DNS name or HELO or contains "bad word", it is rejected, because it is surely an infected zombie personal computer. Sometimes Postfix fails to resolve reverse name, but spammers are smart enough to provide the right for HELO. Blessed are providers that care about their reverse DNS names. This option is much desired.
smtpd_helo_restrictions = ...
check_helo_access hash:/usr/local/etc/postfix/spammer-networks
...
smtpd_client_restrictions = ...
check_client_access hash:/usr/local/etc/postfix/spammer-networks
...
And now comes my spammer-networks, as up to date, available for all of you here:With this all mail coming from AMontpellier-156-1-91-54.w83-205.abo.wanadoo.fr[83.205.210.54] or any.shit.you.imagine.speedy.com.ar is rejected. This option reduces spam significantly, and I consider it much desired. The only drawback is that some companies never care about their mail server reverse DNS hostname, and they may happen to be your important clients, so you'll have to setup a "white list".
Warning! There are some not very smart providers that use addresses like adsl-80-241-246-98.sanet.ge for reverse names. After some consideration I decided to put the whole domain of sanet.ge into list. Dear providers! Consider naming your networks like 1-2-3-4.pool.sanet.ge, so I put only pool.sanet.ge domain to blacklist, not the whole your domain with your addresses too. But that's your problem, not mine. Think carefully before you make your business providing Ineternet to the masses.
smtpd_helo_restrictions = ...
permit_mynetworks
...
check_helo_access hash:/usr/local/etc/postfix/smtpd_helo_restrictions
...
Mine smtpd_helo_restrictions contains only one line:
1.2.3.4 REJECT You are not meSince it comes after permit_mynetworks, it's mail definitely not from our server, and mail from naughty guys should be rejected.
smtpd_client_restrictions = ...
check_client_access cidr:/usr/local/etc/postfix/chinese-spammer-networks
...
The IP networks of evil asian zombies I get from
http://okean.com/sinokoreacidr.txt.
This file I run with the script sinokoreacidr.perl:
#!/usr/bin/perl
$mess = $ARGV[0];
while (<STDIN>) {
if (/^(\d+\.\d+\.\d+\.\d+\/\d+)/) {
print "$1 REJECT $mess\n";
}
}
/usr/local/scripts/sinokoreacidr.perl "Asian Networks" < sinokoreacidr.txt > chinese-spammer-networks
Up to date it looks like:
No more spam from the "other side of the moon". I consider this option much desired.
address_verify_map = btree:/var/spool/postfix/verify/verify
smtpd_sender_restrictions = ...
reject_unverified_sender
...
Now if we get mail from bill@microsoft.com, then before we accept mail, our Postfix makes a try to send email
from postmaster@your.domain to bill@microsoft.com and checks the user's SMTP server response. If it is not possible to send email to the sender,
then the mail is rejected with "try again later" error code (in case we or they have problems with DNS or user's mailbox is full).
The results of the check are cached for some time, everything is configurable. With this option turned on we reject
significant amount of spam from forged email addresses (which in turn can be real but blocked by providers because of abuse).
This option is much desired.
smtpd_recipient_restrictions = ...
check_policy_service unix:private/spf
...
master.cf includes:
spf unix - n n - - spawn
user=nobody argv=/usr/bin/perl /usr/local/scripts/postfix-policyd-spf.pl
Script postfix-policyd-spf.pl can be found in Postfix distribution. For it to work you will
need to install /usr/ports/mail/p5-Mail-SPF-Query perl module.
To publish your SPF record in DNS you'll need to add a record like this for your domain:
IN TXT "v=spf1 a mx -all"
This one means that with SPF version 1 you allow mail with your domain name to be received from your domain's
A address and your domains MX addresses, and mail from all other addresses is rejected.
Notice! For SPF to work you will need to setup your MX records so your mail servers receive email directly, do not put provider's SMTP server into your MX! Or make sure you accept all mail from your backup MX. Anyway having something in your MX that you do not trust and that is lame by default in case of provider's mail server is a repugnant idea.
Notice! There's some features around SPF that are not a part of oficial SPF specification.
Script postfix-policyd-spf.pl uses Mail::SPF::Query perl module, and it can be modified to use this features.
These "guess" flag tells to use rule v=spf1 a/24 mx/24 ptr for those domains that do not publish
SPF records in DNS, which means to allow mail if it comes from domain's whole /24 networks derived from A and MX records,
and if any of client's IP reverse hostname's A records match the client IP address. The "trusted" flag makes use of
spf.trusted-forwarder.org
white list. "The trusted-forwarder.org domain provides a global whitelist (the T-FWL) for users of the SPF system.
It provides early adopters of SPF a way of allowing legitimate email that is sent through known,
trusted email forwarders from being blocked by SPF checks simply because the forwarders do not use some sort
of envelope-from rewriting system."
smtpd_recipient_restrictions = ...
check_policy_service inet:127.0.0.1:2525
...
The technique is described here.
I do it with the help of gld, which can be found
in /usr/ports/mail/gld/, which talks postfix policy protocol on 127.0.0.1:2525.
File /usr/local/etc/gld.conf looks like:
PORT=2525 LOOPBACKONLY=1 CLIENTS=127.0.0.1/32 USER=nobody GROUP=nobody MAXCON=100 MINTIME=60 LIGHTGREY=0 MXGREY=1 WHITELIST=0 ERRACCEPT=1 SYSLOG=1 FACILITY=mail MESSAGE=Greylisting in action, please try later TRAINING=0 KEEPDBOPEN=0 SQLHOST=localhost SQLUSER=greylistd SQLPASSWD=mysupersecret SQLDB=greylistIt makes use of SQL database to remember the past. This option is very useful and much desired. However some may find it inconvenient when the first email to you from some person comes 15 minutes later.
smtpd_helo_restrictions = ...
check_helo_access hash:/usr/local/etc/postfix/smtpd_helo_restrictions
...
smtpd_client_restrictions = ...
check_client_access hash:/usr/local/etc/postfix/allowed-spammers
...
smtpd_sender_restrictions = ...
hash:/usr/local/etc/postfix/smtpd_sender_restrictions
...
smtpd_recipient_restrictions = ...
check_recipient_access hash:/usr/local/etc/postfix/access-recipients
...
You'll have to make these files by your own hand for your own needs.
Another useful technique of whitelisting, which I have not yet setup, but find very useful is making a separate list of those addresses you have sent email to, so they are surely not spammers, and put a check for this white list before other checks. When I'll have time, I'll set it up and show you the scripts.
We need SASL setup:
broken_sasl_auth_clients = yes smtpd_sasl_auth_enable = yes smtpd_sasl_local_domain =Then we need SSL/TLS setup:
smtp_use_tls = yes smtpd_use_tls = yes smtp_tls_note_starttls_offer = yes smtpd_tls_key_file = /usr/local/etc/postfix/ssl/smtpd.pem smtpd_tls_cert_file = /usr/local/etc/postfix/ssl/smtpd.pem smtpd_tls_CAfile = /usr/local/etc/postfix/ssl/smtpd.pem smtpd_tls_loglevel = 1 smtpd_tls_received_header = yes smtpd_tls_session_cache_timeout = 3600s tls_random_source = dev:/dev/urandomFor information how to make your certificates read documentation for OpenSSL tools.
And now we need to pass email from our users:
smtpd_client_restrictions = ...
permit_sasl_authenticated
...
smtpd_recipient_restrictions = ...
permit_sasl_authenticated
...
master.cf should contatin:
smtps inet n - n - - smtpd
...
-o smtpd_tls_wrappermode=yes
You may want to make give your users your CA certificate so they import it as trusted,
but that's not necessary. Making users authenticate themselves is important if you have external users and there's
no like 10.0.0.0/8 in your "mynetworks" option.
clamsmtpd.conf is like:
OutAddress: 127.0.0.1:10026 MaxConnections: 64 TimeOut: 180 KeepAlives: 0 XClient: off Listen: 127.0.0.1:10025 ClamAddress: /var/run/clamav/clamd Header: X-Virus-Scanned: ClamAV using ClamSMTP TempDirectory: /tmp Bounce: off Quarantine: off TransparentProxy: off User: clamav VirusAction: /usr/local/scripts/email-virus-notice.shSo it listens on 127.0.0.1:10025 and then sends line-by-line what it gets to 127.0.0.1:10026, where we setup another Postfix SMTP service in master.cf:
smtp inet n - n - - smtpd
-o content_filter=smtp:127.0.0.1:10025
-o receive_override_options=no_address_mappings
smtps inet n - n - - smtpd
-o content_filter=smtp:127.0.0.1:10025
-o receive_override_options=no_address_mappings
...
127.0.0.1:10026 inet n - n - 32 smtpd
-o content_filter=
-o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
-o smtpd_data_restrictions=
-o smtpd_helo_restrictions=
-o smtpd_client_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o mynetworks_style=host
-o smtpd_authorized_xforward_hosts=127.0.0.0/8
Notice we make content_filter=smtp:127.0.0.1:10025 for our "external" SMTP service, not like described above, because
this example does not use dkfilter SMTP proxy, and here all external mail passes directly to clamsmtp.
The receive_override_options=no_address_mappings tells not to expand aliases etc. The options for the "internal" SMTP service tell not to reject anyting. Option smtpd_authorized_xforward_hosts=127.0.0.0/8 makes Postfix to hide "Received by: .. from 127.0.0.1" ugly header.
With this setup mail goes this way:
ports 25/465(postfix, antispam/policy checks) --> port 10025(clamsmtpd) --> port 10026(promiscuous postfix) --> mailboxes
dkfilter works as a two SMTP proxies, one that "signs" outgoing email, another one that "checks" incoming email. /etc/rc.conf has the following
dkfilter_in_enable="YES"
dkfilter_in_flags="127.0.0.1:10027 127.0.0.1:10025"
dkfilter_out_enable="YES"
dkfilter_out_flags=" --header --keyfile=/usr/local/etc/postfix/ssl/dk-private.key \
--selector=selector1 --domain=your.domain --method=nofws \
127.0.0.1:10028 127.0.0.1:10025"
"Checker" listens on 127.0.0.1:10027 and sends mail to 127.0.0.1:10025 (where the clamsmtpd accepts your viruses).
"Signer" listens on 127.0.0.1:10028 and sends mail to 127.0.0.1:10025 too (we check "internal" mail for viruses too).
"selector1" is just a word, that distinguishes one of the key pairs that you may use. The dkfilter home page describes
how to generate the keys, and how to publish them in DNS.
Now we stumble upon a question that we never met before: how do we know which email is internal(outgoing) and we sign it, and which email is external(incoming), and we check it? Clever guys that designed SMTP protocol specifically implemented a "submission" service and reserved a TCP port number for the reception of internal email.
We make arrangements for our users to accept and sign outgoing email in master.cf:
smtp inet n - n - - smtpd
-o content_filter=smtp:127.0.0.1:10027
-o receive_override_options=no_address_mappings
smtps inet n - n - - smtpd
-o content_filter=smtp:127.0.0.1:10027
-o receive_override_options=no_address_mappings
-o smtpd_tls_wrappermode=yes
submission inet n - n - - smtpd
-o smtpd_etrn_restrictions=reject
-o smtpd_tls_wrappermode=yes
-o content_filter=dksign:[127.0.0.1]:10028
-o receive_override_options=no_address_mappings
-o smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticated,reject
127.0.0.1:9025 inet n - n - - smtpd
-o smtpd_etrn_restrictions=reject
-o content_filter=dksign:[127.0.0.1]:10028
-o receive_override_options=no_address_mappings
-o smtpd_recipient_restrictions=permit_mynetworks,reject
pickup fifo n - n 60 1 pickup
-o content_filter=dksign:[127.0.0.1]:10028
dksign unix - - n - 10 smtp
-o smtp_send_xforward_command=yes
-o smtp_discard_ehlo_keywords=8bitmime
Our "smtp" and "smtpd" services (that from now should get only "external" mail)
now pass all mail to 127.0.0.1:10027 "checker" service.
"submission" service is only for our external users, we make them authenticate, and pass all mail to "signer".
127.0.0.1:9025 service is for local tools like SquirrelMail, so it's email messages are passed to "signer" too
(of course you'll have to change their configs). "pickup" service makes go to "signer" mail received by local
/usr/sbin/sendmail. "dksign" service is a stub, that tells that mail sent to "signer" (and subsequently to
promiscuous local postfix SMTP service) should not get an ugly "Received by: ... from 127.0.0.1" header.
With this setup mail goes this way:
ports 25/465(postfix) --> port 10027(checker)--> port 10025(clamsmtpd) --> port 10026(promiscious postfix) --> mailboxes
ports 9025/587(postfix, submission) --> port 10028(signer)--> port 10025(clamsmtpd) --> port 10026(promiscious postfix) --> away
This technique is quite new, and yet is far from wide use, but it's good to be prepared for it's wide acceptance. So far yahoo and dkfilter are compatible with each other, so you may try to run it in "drop bad mail" mode.
It occurred to me that a 90% of spam comes describing the 10% of spammer clients, like the famous National Business Center. And the spam describing their services contains all the same phone numbers for contact. So checking email for a certain phone numbers in them gives a good chance of detecting their spam.
But here we have some problems. First, the spammers are smart enough to change numbers to letters, like "0" (zero) becomes "o" or "O", english or cyrillic, and 6 (six) becomes "b" english etc. Second problem is that the numbers may contain spaces, dashes and HTML tags between them. And another one is that body of the text can be encoded in base64, or the phone number can be in image, or the phone number is not on a single line (postfix body_checks check content line-by-line). Yet the message can be in one of several cyrillic encodings. Some of the problems we can fight, some we can not.
My answer to them is using Postfix body_checks regular expressions. I maintain a list of phone numbers to check (a bit of it):
Now come the script that converts it to real body_checks rules.
Regular expressions we generate try all number substitutions I've met with several charsets in use.
Now it looks like:
All we need is to make it happen:
body_checks = pcre:/usr/local/etc/postfix/body_checks
The only disadvantage is that if someone forwards such message with good intenttions it will be rejected too. And so far only some logs from "webalizer" produced false hits ;-)
#!/bin/sh cd /usr/local/etc/postfix/ /usr/bin/newaliases ./make_body_checks.perl /usr/local/sbin/postmap smtpd_sender_restrictions /usr/local/sbin/postmap virtual /usr/local/sbin/postmap body_checks /usr/local/sbin/postmap spammer-networks /usr/local/sbin/postmap transport /usr/local/sbin/postmap mailbox_transport /usr/local/sbin/postmap access-recipients /usr/local/sbin/postmap smtpd_helo_restrictions /usr/local/scripts/sinokoreacidr.perl "Asian Networks" < sinokoreacidr.txt >chinese-spammer-networks /usr/local/sbin/postmap chinese-spammer-networks /usr/local/sbin/postmap allowed-spammers /usr/local/sbin/postmap mime_header_checks /usr/local/sbin/postfix reload
I am personally strongly dislike the idea of content filtering tools like SpamAassin that must be constantly "trained", yet spammers are smart enough to bypass these filters. My goal is not to receive spam at all saving incoming traffic, not to "precisely classify it's content". It's a pure stupidity having 1000 users make filtering rules for messages "tagged as spam" or having them look through "spam" folder which in it's turn becomes more popular than "incoming" folder. Yet if you can not send email because of your IP in black list, you can try to send it through your provider's mail server, but if the recepient's server has lame content filter, only phone call to support will make your message come through.
In a battle against it we could be much stronger if the providers and postmasters were more organized and conscious. The proper DNS reverse hostnames and the use of SPF and DomainKyes would solve the problem of spam up to 99.999%.