Setting a Postfix+ClamAV+DSPAM Mailgateway infront of Exchange
This document explains how to setup an mail-gateway that's infront of Exchange. We're using an OLD version of Exchange 5.5 (I know, I know). So, a tie into Active Directory won't work. What I need is a fairly semi-dumb mail-gateway to relay mail, but to the right people.
First Install the following base packages:
PostgreSQL 8.1.x
Postfix 2.2.x(Built against PostgreSQL)
ClamAV (Plus any Archive extracting utilities plus GMP)
Amavisd-new
Dspam 3.6.6
Apache 2.0.x (For the DSPAM webui plus GNU GD and whatever else you want for the Apache environment)
I built my own Sun Solaris SysV packages for the above. There is acommunity software distribution based on Debians 'apt-get' at http://www.blastewave.org - NOTE like every 'community distribution' packages your milage will vary with enabled options/features etc.
Compiling Instructions for the following can be found DSPAMBuild Building DSPAM on Solaris (this doesn't cover packaging you can get that at other locations)
NOTE: Software prefixes may change based on where your package put's stuff. The packages I build install along side the Companion CD ie /opt/sfw
NOTE: Always review your settings for YOUR setup! Do not take this as 100% will work scenario everyones setup is different!
I also choose my DB as PostgreSQL. Why you may ask? Well few reasons:
It's going to be fully supported by Sun in the near future
Sun's new DB in the works will be compatible
It's brain-dead compatable for Oracle (PLSQL baby!)
Solid and mature RDBMS
It's a dream to work with compared to MySQL
BSD License ROCKs over GPL!
Because I can
I also included Solaris 10 SMF (Service Manifest Files) - Distribution Specific/DSPAMSolaris/SMF DSPAM Specific SMF files
STEP ONE - Setup PgSQL database correctly
Create DB user & DB
/opt/sfw/pgsql8/bin/createuser -DREP -U pgsql dspamusr /opt/sfw/pgsql8/bin/createdb -O dspamusr -U pgsql dspam
Setup permissions (user authentication rights) in /var/pgdata/pg_hba.conf
# TYPE DATABASE USER CIDR-ADDRESS METHOD # "local" is for Unix domain socket connections only #local all all md5 local all pgsql md5 local dspam dspamusr md5 # IPv4 local connections: #host all all 127.0.0.1/32 md5 host all pgsql 127.0.0.1/32 md5 host dspam dspamusr 127.0.0.1/32 md5
Start / Restart PostgreSQL
/usr/sbin/svcadm -v start|restart pgsql
Setup Dspam database with provided dspam sql files
NOTE:
The pgsql_objects.sql file has a broken comment on line 58-62 remove this BEFORE you install the objects file! It will also complain that the following tables don't exist this is OK:
dspam_neural_data dspam_neural_decisions
# pwd /opt/sfw/doc/dspam # /opt/sfw/pgsql8/bin/createlang -d dspam plpgsql -U pgsql /opt/sfw/pgsql8/bin/psql -U dspamusr dspam < ./pgsql_objects.sql /opt/sfw/pgsql8/bin/psql -U dspamusr dspam < ./virtual_users.sql
STEP TWO - Setup ClamAV correctly
Edit /etc/postfix/clamd.conf with the following:
LogFile /var/clamav/clamd.log LogFileMaxSize 600M LogTime PidFile /var/clamav/clamd.pid TemporaryDirectory /var/tmp DatabaseDirectory /var/clamav/virusdb LocalSocket /var/clamav/clamd.sock FixStaleSocket StreamMaxLength 50M MaxThreads 30 ReadTimeout 120 MaxDirectoryRecursion 20 SelfCheck 900 User clamav AllowSupplementaryGroups ScanPE DetectBrokenExecutables ScanOLE2 ScanMail ScanHTML ScanArchive ArchiveMaxFileSize 50M ArchiveMaxRecursion 9 ArchiveMaxFiles 3000 ArchiveMaxCompressionRatio 300 ArchiveLimitMemoryUsage ArchiveBlockEncrypted ArchiveBlockMax ClamukoMaxFileSize 65M
Edit /etc/postfix/freshclam.conf with the following:
DatabaseDirectory /var/clamav/virusdb UpdateLogFile /var/clamav/freshclam.log PidFile /var/clamav/freshclam.pid DatabaseOwner clamav AllowSupplementaryGroups DNSDatabaseInfo current.cvd.clamav.net DatabaseMirror db.US.clamav.net DatabaseMirror database.clamav.net Checks 24
Start / Restart ClamAV
/usr/sbin/svcadm -v start|restart clamav
STEP THREE - Setup Amavisd-new correctly
I use Amavisd-new to control ClamAV after reading up I think it does a better job then allowing DSPAM 3.6.6 the functionality to do it for me.
There are a slew of options in the amavisd.conf you can use. Instead of giving cuts from the file, I'll provide the file below. However, there are a few things you WANT to keep alert to they are:
Make sure the sockets are the right naming for ClamAV in Amavisd
Make sure you run as the same user as ClamAV
Make sure your ports for notify_method & forward_method match what's in Postfix's master.cf
STEP THREE - Setup DSPAM correctly
Edit the /etc/postfix/dspam/dspam.conf
Home /var/dspam TrustedDeliveryAgent "/usr/sbin/sendmail" OnFail error Trust root Trust mail Trust postfix Trust clamav Trust Filter Trust dspamusr Trust www TrainingMode teft TestConditionalTraining on #Feature sbph #Feature noise Feature chained Feature whitelist Feature tb=5 Algorithm graham burton PValue graham ImprobabilityDrive on Preference "spamAction=quarantine" Preference "signatureLocation=message" # 'message' or 'headers' Preference "showFactors=on" Preference "spamAction=tag" Preference "spamSubject=***DETECTED SPAM***" AllowOverride trainingMode AllowOverride spamAction spamSubject AllowOverride statisticalSedation AllowOverride enableBNR AllowOverride enableWhitelist AllowOverride signatureLocation AllowOverride showFactors AllowOverride optIn optOut AllowOverride whitelistThreshold # # --- PostgreSQL --- # # Socket Method # PgSQLServer /tmp PgSQLUser dspamusr PgSQLPass changeme PgSQLDb dspam PgSQLConnectionCache 30 PgSQLUIDInSignature on HashRecMax 1572869 HashAutoExtend on HashMaxExtents 0 HashExtentSize 49157 HashMaxSeek 100 HashConnectionCache 10 Notifications off # # Purge configuration for SQL-based installations using purge.sql # PurgeSignature off # Specified in purge.sql PurgeNeutral 90 PurgeUnused off # Specified in purge.sql PurgeHapaxes off # Specified in purge.sql PurgeHits1S off # Specified in purge.sql PurgeHits1I off # Specified in purge.sql LocalMX 127.0.0.1 SystemLog on UserLog on TrainPristine off Opt out ParseToHeaders on ChangeModeOnParse on ChangeUserOnParse off Broken case Broken lineStripping ServerQueueSize 32 ServerPID /var/dspam/var/run/dspam.pid ServerMode standard ServerIdent "newmx.example.com" ServerDomainSocketPath "/tmp/dspam.sock" ProcessorBias on
Make sure the dspam home permissions and anything under it is okay in this setup it is: /var/dspam
chown -R dspam:www /var/dspam chmod -R 775 /var/dspam # ls -al total 24 drwxrwxr-x 7 dspam www 512 May 26 09:52 . drwxr-xr-x 30 root sys 512 May 25 20:41 .. drwxrwxr-x 5 dspam postdrop 512 May 26 11:28 data drwxrwxr-x 2 dspam www 512 May 26 09:52 log -rw-r--r-- 1 dspam www 537 May 4 2005 purge.sql drwxrwxr-x 2 dspam dspam 512 May 23 08:28 txt drwxrwxr-x 3 dspam www 512 May 22 11:50 var #
Make sure the /opt/sfw/bin/dspam binary has the right permissions
chown dspam:postdrop /opt/sfw/bin/dspam chmod 4555 /opt/sfw/bin/dspam
STEP FOUR - Setup Postfix
There are a some files specific to this setup and Postfix they are:
/etc/postfix/master.cf
/etc/postfix/main.cf
/etc/postfix/relay_domains
/etc/postfix/relay_recipients
/etc/postfix/transport
/etc/postfix/dspam_filter_access
/opt/scripts/dspam-retrain
Other Postfix Specific Files
/etc/postfix/master.cf
Make sure the following are in your master.cf
# amavisd-new + clamav : virus checker
# dspam : spam checker
smtp inet n - n - - smtpd
-o content_filter=smtp-amavis:[127.0.0.1]:10024
smtp-amavis unix - - n - 2 lmtp
-o smtp_send_xforward_command=yes
127.0.0.1:10025 inet n - n - - smtpd
-o content_filter=dspam:dummy
-o local_recipient_maps=
-o relay_recipient_maps=
-o smtpd_restriction_classes=
-o smtpd_client_restrictions=
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o mynetworks=127.0.0.0/8
-o strict_rfc821_envelopes=yes
-o smtpd_error_sleep_time=0
-o smtpd_soft_error_limit=1001
-o smtpd_hard_error_limit=1000
127.0.0.1:10026 inet n - n - - smtpd
-o local_recipient_maps=
-o relay_recipient_maps=
-o smtpd_restriction_classes=
-o smtpd_client_restrictions= permit_mynetworks, check_client_access dbm:/etc/postfix/access, check_client_access pcre:/etc/postfix/dspam_filter_access
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o mynetworks=127.0.0.0/8
-o strict_rfc821_envelopes=yes
-o smtpd_error_sleep_time=0
-o smtpd_soft_error_limit=1001
-o smtpd_hard_error_limit=1000
dspam unix - n n - 10 pipe
flags=Rhqu user=dspam argv=/opt/sfw/bin/dspam --deliver=innocent --user ${recipient} -i -f ${sender} -- ${recipient}
dspam-retrain unix - n n - 10 pipe
flags=Ru user=dspam argv=/opt/scripts/dspam-retrain $nexthop $sender $recipient
/etc/postfix/main.cf - This will not have the comments in this. Please look at a default main.cf file and add accordingly
The main.cf have a few variables that are manditory for dspam to function properly
*transport_maps = dbm:/etc/postfix/transport *virtual_transport = lmtp:unix:/tmp/dspam.sock *dspam_destination_recipient_limit = 1 *dspam-retrain_destination_recipient_limit = 1 *smtpd_client_restrictions = permit_mynetworks, check_client_access dbm:/etc/postfix/access, check_client_access pcre:/etc/postfix/dspam_filter_access
queue_directory = /var/spool/postfix command_directory = /usr/sbin daemon_directory = /usr/lib/postfix mail_owner = postfix myhostname = newmx.carsdirect.com myorigin = $mydomain inet_interfaces = all mydestination = local_recipient_maps = proxy:unix:passwd.byname $alias_maps $transport_maps unknown_local_recipient_reject_code = 550 mynetworks = 127.0.0.0/8, xxx.yyy.zzz.xy/24 relay_domains = /etc/postfix/relay_domains relay_recipient_maps = dbm:/etc/postfix/relay_recipients alias_maps = dbm:/etc/postfix/aliases alias_database = dbm:/etc/postfix/aliases mail_spool_directory = /var/mail fast_flush_domains = $relay_domains smtpd_banner = $myhostname ESMTP ServerName sendmail_path = /usr/sbin/sendmail newaliases_path = /usr/bin/newaliases mailq_path = /usr/bin/mailq setgid_group = postdrop html_directory = /usr/share/doc/postfix/html manpage_directory = /usr/share/man sample_directory = /etc/postfix readme_directory = /usr/share/doc/postfix ### WPOOL ADDED ### # To verify the purpose and default value for the below check: # http://www.postfix.org/postconf.5.html dspam_destination_recipient_limit = 1 dspam-retrain_destination_recipient_limit = 1 parent_domain_matches_subdomains = debug_peer_list smtpd_access_maps # We had set to no should be yes - Exchange 5.5 smtpd_helo_required = no smtpd_disable_vrfy_command = yes biff = no empty_address_recipient = MAILER-DAEMON queue_minfree = 78643200 local_transport = local virtual_maps = dbm:/etc/postfix/virtual # JUNK MAIL CONTROLS # # The controls listed here are only a very small subset. The file # SMTPD_ACCESS_README provides an overview. # The header_checks parameter specifies an optional table with patterns # that each logical message header is matched against, including # headers that span multiple physical lines. # # By default, these patterns also apply to MIME headers and to the # headers of attached messages. With older Postfix versions, MIME and # attached message headers were treated as body text. # # For details, see "man header_checks". # header_checks = pcre:/etc/postfix/header_checks mime_header_checks = pcre:/etc/postfix/mime_header_checks #nested_header_checks = pcre:/etc/postfix/nested_header_checks body_checks = pcre:/etc/postfix/body_checks mail_name = InternetBrands inet_protocols = ipv4 default_database_type = dbm # Default value for (maximal|bounce)_queue_lifetime is 5days bounce_queue_lifetime = 5d maximal_queue_lifetime = 5d maximal_queue_lifetime = 5d syslog_facility = mail syslog_name = postfix # Set Message size limit to 50mb message_size_limit = 52428800 ##### SMTPD Rules ##### # Error Codes unknown_address_reject_code = 450 # # Default defer_code 450 changed to 455 to trace issue # defer_code = 455 # # unknown_client_reject_code default 450 changed to 460 to trace issues # unknown_client_reject_code = 460 # # unknown_hostname_reject_code default 450 changed to 470 to trace issues # unknown_hostname_reject_code = 470 non_fqdn_reject_code = 504 invalid_hostname_reject_code = 501 unknown_local_recipient_reject_code = 550 access_map_reject_code = 554 # # unknown_relay_recipient_reject_code default 550 changed to 555 to trace issues # unknown_relay_recipient_reject_code = 555 # relay_domains_reject_code default 554 changed to 560 to trace issues relay_domains_reject_code = 560 smtpd_helo_restrictions = permit_mynetworks, check_helo_access dbm:/etc/postfix/access smtpd_client_restrictions = permit_mynetworks, check_client_access dbm:/etc/postfix/access smtpd_sender_restrictions = permit_mynetworks, check_sender_access dbm:/etc/postfix/access, reject_non_fqdn_sender smtpd_recipient_restrictions = permit_mynetworks, check_recipient_access dbm:/etc/postfix/access, permit_auth_destination, reject_unauth_destination, reject_unauth_destination, reject_non_fqdn_recipient
/etc/postfix/relay_domains - Domains that postfix will relay for *
example.com OK .example.com OK example-foo.com OK .example-foo.com OK
/etc/postfix/relay_recipients - This is the real magic. This contains domains that will be allowed TO the gateway. If youdon't have a complete user list this is a bit closer to a controlledlist (I have other ideas to will post when I get a chance). Thisalso controls what spam/ham/corpus will be allowed:
#### Recipient Domains added here #### @newmx.example.com dummy @example.com dummy @example-foo.net dummy @domain-c.com dummy
/etc/postfix/transport - This is your inbound transport. It will tell what inbound SMTP server to relay domain mail to. This also has the dspam-retrain' transport information.
#### SPAM /HAM Aliases here #### spam@example.com dspam-retrain:spam spam@newmx.example.com dspam-retrain:spam spam@example-foo.com dspam-retrain:spam ham@example.com dspam-retrain:innocent ham@newmx.example.com dspam-retrain:innocent ham@example-foo.com dspam-retrain:innocent notspam@example.com dspam-retrain:innocent nospam@example.com dspam-retrain:innocent notspam@newmx.example.com dspam-retrain:innocent notspam@example-foo.com dspam-retrain:innocent nospam@examle-foo.com dspam-retrain:innocent inoculation-spam@example.com dspam-retrain:inoculation-spam inoculation-notspam@example.com dspam-retrain:inoculation-innocent corpus-spam@example.com dspam-retrain:corpus-spam corpus-notspam@example.com dspam-retrain:corpus-innocent corpus-spam@newmx.example.com dspam-retrain:corpus-spam corpus-notspam@newmx.example.com dspam-retrain:corpus-innocent # Transport Domains example.com smtp:[real-inside-smtp.example.com] .example.com smtp:[real-inside-smtp.example.com] example-foo.com smtp:[real-inside-smtp.example.com] .example-foo.com smtp:[real-inside-smtp.example.com]
/etc/postfix/dspam_filter_access
This script is placed in the /etc/postfix/master.cf file in the smtpd_client_restrictions (amavsid 1026)
If it's NOT from our network scan it with DSPAM if it's on our "trusted" network it's okay. This also allows free outbound mail.
# more dspam_filter_access /./ FILTER dspam: #
/opt/scripts/dspam-retrain - This is a script that's availble on this site. However, it's broken, and I'm beta testing another script currently. This can be whatever you want to be for training. I'll update more later.
Other Postfix Specific Files - I didn't cover the following files because they're standard and you should have a slight idea about how to use them:
/etc/postfix/access /etc/postfix/aliases /etc/postfix/body_checks /etc/postfix/header_checks /etc/postfix/mime_header_checks /etc/postfix/virtual
STEP FIVE STARTUP the GATEWAY
/usr/sbin/svcadm -v enable -r postfix
Or start it up individualy
/usr/sbin/svcadm -v enable pgsql /usr/sbin/svcadm -v enable clamav /usr/sbin/svcadm -v enable postfix
STEP SIX WEB UI
This will come soon, but not that difficult to figure out RTFM
NOTES:
Here are some 'specific' notes I had because I was a 'newbie' with DSPAM and the docs weren't that clear.
Signatures are created by FRESH NEW mail that come into the gateway. At that same time the e-mail address from the TO: field will be recorded into the dspam_virtual_uids table into the database.
Forwarding NEW SPAM messages that have NOT been identifiedby a signature yet won't be trained. - I had this problem whereI'd send mail through from various e-mail addresses and signatureswould be created. However, when I sent e-mail to spam@newmx.example.com it'd complain it couldn't find a signatureand just pass the mail along. At this point you have to use:
inoculation-spam@example.com dspam-retrain:inoculation-spam inoculation-notspam@example.com dspam-retrain:inoculation-innocent corpus-spam@example.com dspam-retrain:corpus-spam corpus-notspam@example.com dspam-retrain:corpus-innocent corpus-spam@newmx.example.com dspam-retrain:corpus-spam corpus-notspam@newmx.example.com dspam-retrain:corpus-innocent
Where forwarded SPAM would use the corpus e-mail addresses for training.
Pristine SPAM would use the inoculation e-mail addresses for training
References:
http://dspamwiki.expass.de/Installation/Postfix/NealesSetup
Search for the title: "Make Postfix filter incoming mail only"
This will lead you here: http://dspam.nuclearelephant.com/dspam-users/1842.html