INSTALL INSTRUCTIONS for using MIMEDefang w/SpamAssassin on RedHat by Kevin A. McGrail (kmcgrail@pccc.com) Maintained at: http://www.pccc.com/downloads/ This is basically my HOW-TO install SpamAssassin on a RedHat system using sendmail and check all the mail using a Milter. The only reason to run this install is if you want to run a sendmail that then simply forwards the mail to another mail exchange. The milter in this example is the very fine MIMEDefang from your pal and mine, Roaring Penguin Software. NOTE: This milter was an absolute joy to work with compared to most of the milters I have done. It wasn't easy but it was the most configurable, stable and powerful one I've used ever. IMPORTANT NOTE: This documents assume you have already installed Sendmail and SpamAssassin per the current instructions at http://www.pccc.com/download/sendmail/ & http://www.pccc.com/download/SpamAssassin/. #REALLY IMPORTANT NOTE: SENDMAIL NEEDS TO BE COMPILED WITH LIBMILTER #ASSUMING YOU USE THE SOURCE CODE THE FOLLOWING SHOULD HELP (see http://www.pccc.com/downloads/sendmail/) #WGET, UNTAR & INSTALL THE VARIOUS SUPPORT DISTRIBUTIONS # # NOTE: Yes, I do know that CPAN can do this automatically but I am a bit of a freak who likes to confirm # the exact version I am getting. # # Do you have SpamAssassin, Mail-Tools & Digest-SHA1? You should have gotten these because you already have SpamAssassin Installed! #TEST IF PRESENT AND WHAT VERSION perl -e 'if (require Mail::SpamAssassin) { print "SpamAssassin Version is: $Mail::SpamAssassin::VERSION\n"; exit 0;} else {exit 1;}' || echo 'SpamAssassin Not Present!' perl -e 'if (require Mail::Internet) { print "Mail-Tools Version is: $Mail::Internet::VERSION\n"; exit 0;} else {exit 1;}' || echo 'Mail-Tools Not Present!' perl -e 'if (require Digest::SHA1) { print "Digest::SHA1 Version is: $Digest::SHA1::VERSION\n"; exit 0;} else {exit 1;}' || echo 'Digest-SHA1 Not Present!' # Install Compress-Zlib # PREREQ for Archive::Zip perl -e 'if (require Compress::Zlib) { print "Compress-Zlib Version is: $Compress::Zlib::VERSION\n"; exit 0;} else {exit 1;}' || echo 'Compress-Zlib Not Present!' mkdir /tmp/20061112MIME cd /tmp/20061112MIME/ wget http://www.peregrinehw.com/downloads/MIMEDefang/Compress-Zlib-1.42.tar.gz cd /usr/src rm -rf /usr/src/Compress-Zlib-1.42 tar zxvf /tmp/20061112MIME/Compress-Zlib-1.42.tar.gz cd Compress-Zlib-1.42/ perl Makefile.PL make test && make install # Install Archive::Zip # OPTIONAL for MIMEDefang perl -e 'if (require Archive::Zip) { print "Archive-Zip Version is: $Archive::Zip::VERSION\n"; exit 0;} else {exit 1;}' || echo 'Archive-Zip Not Present!' mkdir /tmp/20061112MIME cd /tmp/20061112MIME/ wget http://www.peregrinehw.com/downloads/MIMEDefang/Archive-Zip-1.23.tar.gz cd /usr/src rm -rf Archive-Zip-1.23 tar zxvf /tmp/20061112MIME/Archive-Zip-1.23.tar.gz cd Archive-Zip-1.23 perl Makefile.PL make test && make install # Install IO-Stringy # PREREQ for MIMEDefang perl -e 'if (require IO::Stringy) { print "IO-Stringy Version is: $IO::Stringy::VERSION\n"; exit 0;} else {exit 1;}' || echo 'IO-Stringy Not Present!' mkdir /tmp/20061112MIME cd /tmp/20061112MIME/ wget http://www.peregrinehw.com/downloads/MIMEDefang/IO-stringy-2.110.tar.gz cd /usr/src rm -rf /usr/src/IO-stringy-2.110 tar zxvf /tmp/20061112MIME/IO-stringy-2.110.tar.gz cd IO-stringy-2.110/ perl Makefile.PL make test && make install # Install MIME-Base64 v3.08 - NOTE: Perl 5.8.6 includes v3.0.5. # PREREQ for MIMEDefang perl -e 'if (require MIME::Base64) { print "MIME-Base64 Version is: $MIME::Base64::VERSION\n"; exit 0;} else {exit 1;}' || echo 'MIME::Base64 Not Present!' mkdir /tmp/20061112MIME cd /tmp/20061112MIME/ wget http://www.peregrinehw.com/downloads/MIMEDefang/MIME-Base64-3.08.tar.gz cd /usr/src rm -rf /usr/src/MIME-Base64-3.08 tar zxvf /tmp/20061112MIME/MIME-Base64-3.08.tar.gz cd MIME-Base64-3.08 perl Makefile.PL make test && make install # Install IO 1.25 for IO::File 1.14 # PREREQ FOR MIME::Tools perl -e 'if (require IO::File) { print "IO::File Version is: $IO::File::VERSION\n"; exit 0;} else {exit 1;}' || echo 'IO::File Not Present!' mkdir /tmp/20061112MIME cd /tmp/20061112MIME/ wget http://www.peregrinehw.com/downloads/MIMEDefang/IO-1.25.tar.gz cd /usr/src rm -rf /usr/src/IO-1.25 tar zxvf /tmp/20061112MIME/IO-1.25.tar.gz cd IO-1.25/ perl Makefile.PL make test && make install # Install File Temp 0.22 - Need at least 0.18 # PREREQ FOR MIME::Tools perl -e 'if (require File::Temp) { print "File::Temp Version is: $File::Temp::VERSION\n"; exit 0;} else {exit 1;}' || echo 'File::Temp Not Present!' mkdir /tmp/20061112MIME cd /tmp/20061112MIME/ wget http://www.peregrinehw.com/downloads/MIMEDefang/File-Temp-0.22.tar.gz cd /usr/src rm -rf /usr/src/File-Temp-0.22 tar zxvf /tmp/20061112MIME/File-Temp-0.22.tar.gz cd File-Temp-0.22/ perl Makefile.PL make test && make install # Install MIME-Tools 5.427 -- This includes the patches and is now maintained by Roaring Penguin # PREREQ for MIMEDefang perl -e 'if (require MIME::Tools) { print "MIME-Tools Version is: $MIME::Tools::VERSION\n"; exit 0;} else {exit 1;}' || echo 'MIME::Tools Not Present!' mkdir /tmp/20061112MIME cd /tmp/20061112MIME/ wget http://www.peregrinehw.com/downloads/MIMEDefang/MIME-tools-5.427.tar.gz cd /usr/src rm -rf /usr/src/MIME-tools-5.427 tar zxvf /tmp/20061112MIME/MIME-tools-5.427.tar.gz cd MIME-tools-5.427/ perl Makefile.PL make test && make install #WORKING ON ISSUES FOR WHY MAKE TEST FAILS - IT APPEARS 5.8.0 JUST DOESN'T WORK. TRY A NEWER VERSION! # Install Unix Syslog v0.100 # OPTIONAL for MIMEDefang perl -e 'if (require Unix::Syslog) { print "Unix-Syslog Version is: $Unix::Syslog::VERSION\n"; exit 0;} else {exit 1;}' || echo 'Unix::Syslog Not Present!' mkdir /tmp/20061112MIME cd /tmp/20061112MIME/ wget http://www.peregrinehw.com/downloads/MIMEDefang/Unix-Syslog-0.100.tar.gz cd /usr/src rm -rf /usr/src/Unix-Syslog-0.100/ tar zxvf /tmp/20061112MIME/Unix-Syslog-0.100.tar.gz cd Unix-Syslog-0.100/ perl Makefile.PL make test && make install # Add a MIMEDefang User /usr/sbin/adduser -c "MIMEDefang" -s /bin/false defang -u 107 mkdir /home/defang/.spamassassin chown defang.defang /home/defang/.spamassassin # Install MIMEDefang v2.67 mkdir /tmp/20061112MIME cd /tmp/20061112MIME/ wget http://www.peregrinehw.com/downloads/MIMEDefang/mimedefang-2.67.tar.gz cd /usr/src rm -rf /usr/src/mimedefang-2.67/ tar zxvf /tmp/20061112MIME/mimedefang-2.67.tar.gz cd mimedefang-2.67 # This configuration will place config files in /etc/mail with the spool dirs under /var/spool/ running with # a user called Defang ./configure --sysconfdir=/etc --with-confsubdir=mail --with-spooldir=/var/spool/MIMEDefang --with-quarantinedir=/var/spool/MD-Quarantine --with-user=defang make make install # Create the spool dirs & give them the proper permissions mkdir /var/spool/MIMEDefang /var/spool/MD-Quarantine chown defang.defang /var/spool/MIMEDefang /var/spool/MD-Quarantine chmod 700 /var/spool/MIMEDefang /var/spool/MD-Quarantine # Create the log dir & give them the proper permissions mkdir /var/log/mimedefang/ chown defang.defang /var/log/mimedefang chmod 700 /var/log/mimedefang #Add MIMEDefang to sendmail.mc and recreate sendmail.cf #NOTE: This line has higher timeouts then the default recommendation. See http://www.pccc.com/downloads/MIMEDefang/MILTER-PARAMETERS for more information. echo "INPUT_MAIL_FILTER(\`mimedefang', \`S=unix:/var/spool/MIMEDefang/mimedefang.sock, F=T, T=C:15m;S:4m;R:4m;E:10m')" >> /etc/mail/sendmail.mc m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf #Configure MIMEDefang to start with init script cd redhat cp mimedefang-sysconfig /etc/sysconfig/mimedefang cp mimedefang-init /etc/rc.d/init.d/mimedefang chkconfig --add mimedefang #BUG FIX FOR REDHAT INIT SCRIPT: # The configtest procedure in the redhat init script is needs $PROGDIR # prepended in case mimedefang.pl isn't in the path: # e.g., line 163 should read: $PROGDIR/mimedefang.pl $([ -n "$SUBFILTER" ] && echo "-f $SUBFILTER") -test > /var/spool/MIMEDefang/configtest.out 2>&1 #Configure a script to clean out the MD-Quarantine mkdir /tmp/20061112MIME cd /tmp/20061112MIME/ wget http://www.peregrinehw.com/downloads/MIMEDefang/contrib/SA-MD-cleanup.sh mv /tmp/20061112MIME/SA-MD-cleanup.sh /etc/cron.daily chmod +x /etc/cron.daily/SA-MD-cleanup.sh #START MIMEDEFANG MILTER AND MULTIPLEXOR # # The multiplexor is similar to the spamd/spamc solution to stop Perl from spawning all the time on busy mail servers. /etc/rc.d/init.d/mimedefang start /etc/rc.d/init.d/sendmail restart #REMOVE THE TEMPORARY FILES rm -rf /tmp/20030708SPAM/ ### END INSTALLATION -- BEGIN OPTIMIZATION/CONFIGURATION #CONFIGURATION/OPTIMIZATION TIPS # 0 -- If you are upgrading, read the MIMEDefang Changelog!!! Seriously, they mean it: "WARNING: Before upgrading MIMEDefang, please search this file for *** NOTE INCOMPATIBILITY ** to see if anything has changed that might affect your filter." # 1 -- Lower the Queue time for your sendmail submission queue Because MIMEDefang uses the client submission queue, we recommend changing your queue time to 5 minutes. Typically, you can edit /etc/sysconfig/sendmail and perhaps add a line such as: CLIENTQUEUE=5m Then, edit your /etc/rc.d/init.d/sendmail and look for the client queue line and update it to use this new variable. If I get feedback on this one, I'll make clearer instructions. # 2 -- Don't forget to disable spamassmilter and spamd if you were using them chkconfig spamassmilter off chkconfig spamassassin off # 3 -- We have found that most milters work better and more stably with the following lines in your sendmail.mc define(`confSEPARATE_PROC', `False')dnl define(`confMAX_DAEMON_CHILDREN', `12')dnl # 4 -- In /etc/sysconfig/mimedefang I make the following changes: enable LOG_FILTER_TIME enable MX_STATS set MX_MINIMUM to the 1/4 or 1/5th (i.e. 3 or 4) of "define(`confMAX_DAEMON_CHILDREN', `12')" above set MX_MAXIMUM to the one or two above "define(`confMAX_DAEMON_CHILDREN', `12')" above enable SUBFILTER=/etc/mail/mimedefang-filter enable MX_EMBED_PERL #5 -- If you change mimedefang-filter, the multiplexor will need to reread it: /etc/rc.d/init.d/mimedefang reread #6 -- setting up your mimedefang-filter: - the mimedefang-filter that I use is available at http://www.peregrinehw.com/downloads/MIMEDefang/mimedefang-filter-KAM - The main changes are: it is setup to use AWL, it doesn't strip multi-part HTML emails, and it adds both the MIMEDefang default headers and the headers which SpamAssassin would have normally added by default. Also, as of 10-29-03, it also mimics the Report_Safe functionality of SpamAssassin - IMPORTANT: change the Admin and DaemonAddress info at the top - Test a new filter before you put it into production with this command, thanks to Justin: mimedefang.pl -f test-filter -test #7 -- Turn on Bayesian & AWL Site-Wide Filters - I use and recommend (though there is MUCH debate) AWL and Bayesian for site-wide installs. To enable this, edit /etc/mail/spamassassin/sa-mimedefang.cf and add the lines below. Additionally, you will need to make sure your mimedefang-filter uses AWL which the example above does. #Enable bayes auto_learn 1 use_bayes 1 bayes_path /var/spool/MIMEDefang/bayes bayes_file_mode 0666 #Enable AWL auto_whitelist_path /var/spool/MIMEDefang/auto_whitelist auto_whitelist_file_mode 0666 #8 -- Using a temporary filesystem to speed up mail processing and lower disk I/O - If you have tmpfs in your kernel and you have enough RAM, then processing the emails on a ramdisk is the only way to go. Here's a simple way to achieve that thanks to Alan Premselaar & Steffen Kaiser for the mount hint: #Stop sendmail & mimedefang /etc/rc.d/init.d/sendmail stop /etc/rc.d/init.d/mimedefang stop mv /var/spool/MIMEDefang /var/spool/MIMEDefang.old mkdir /var/spool/MIMEDefang #add this line to /etc/fstab none /var/spool/MIMEDefang tmpfs uid=107,gid=107,mode=700,size=384M 0 0 mount /var/spool/MIMEDefang NOTE: You'll want to place your bayesian database and auto-whitelist somewhere else as noted by Bill Curtis! NOT NEEDED ON VERSION 2.63 (AND ABOVE. NOT SURE WHEN EXACTLY IT WAS CHANGED) BUT GOOD INFO ANYWAY - #9 -- Save some memory by changing the ulimit call in the init script - Edit the init script and change the gt 100 to gt 1 as seen below: "if test "$MX_MAXIMUM" -gt 1 ; then ulimit -s 2048 fi" Why? Reason from David F. Skoll: Here's some free advice: On RHEL3, type "ulimit -s": $ ulimit -s 10240 So each thread wants 10MB of stack space (ed: or 8192 on RH7.3). That can chew up your RAM pretty quickly. I recommend editing the MIMEDefang startup script and putting: ulimit -s 2048 just before mimedefang (not the multiplexor!) is invoked. Right now, the sample red hat script does it only if you have more than 100 slaves, but it should really do it unconditionally. #10 - Upgrading from one version to the next? Here's a good way to look at the sysconfig file: grep -v -r ^# /etc/sysconfig/mimedefang |grep -v -r ^$ #NOTES ABOUT HOW TO USE THIS MILTER TO FILTER YOUR MAIL This is more of a sendmail question but very related so I am including some real world examples. First solution: DNS based solution Pros: Works very well for maintaining many mail servers / firewalls simply through DNS Cons: Can cause mail delays and spikes Assuming you have installed the milter, you are just trying to make this installation a temporary hop on the way to the mailbox. So if the mail is getting to /var/spool/mail then it's not working correctly. Let's assume your domain is peregrinehw.com. Let's assume your Spam Milter box is IP 200.200.200.1. Let's assume your final destination MX is an internal IP firewalled by your SPAM Milter Box on IP 192.168.1.1. This will also work for a public IP for the final MX and will allow mail to work even if the milter server is down (See Note Below). Set your DNS like this: peregrinehw.com IN MX 5 internalmx.peregrinehw.com <--Primary MX is internal peregrinehw.com IN MX 10 mx.peregrinehw.com <--Secondary MX is external ...More Backup MX Records... internalmx.peregrinehw.com IN A 200.200.200.1 mx.peregrinehw.com IN A 192.168.1.1 On the Spam Milter box, make certain your local-host-names does NOT include peregrinehw.com. Make sure your relay-domains DOES include peregrinehw.com. Then what will happen is as follows: A person emails a user @ peregrinehw.com. EMail is transmitted to their SMTP server. Their server looks up the MX records and gets internalmx.peregrinehw.com It can't connect. It looks up the next MX priority and gets mx.peregrinehw.com It can connect and the mail is transferred to your milter box. Your milter box now filters for spam and looks to see where it has to go next. It's NOT the local host so it looks up the MX records. It gets internalmx.peregrinehw.com It CAN connect It transfers the mail all nicely filtered. VOILA! NOTE: If you use a secondary and primary MX that are both public, many spammers know this and try and bypass the Anti-Spam/Anti-Viral system by using the secondary MX. Therefore, your best solution is to put the milter box on the net as the primary MX record. Then DO NOT list the real primary MX as a DNS record. You should probably still have a queuing-only secondary MX. Use Solution 2 or 3 below to then direct the mail to its final destination! The only reason this solution isn't ideal is that if your milter box goes down, your mail flow stops because there is nothing in the MX records to hop that box. Unfortunately, that situation is unavoidable due to spammers. NOTE: As discussed by the writers of MIMEDefang/CanIT, we have seen spammers and viruses that are simply targetting A records and ignoring MX records. Second solution: Use a SmartHost Pros: All email being relay'd through the milter server will be forwarded to internal mail server. Cons: All email will be forwarded through to the internal server. If the server name of the final destination changes then the Sendmail configuration must be updated Won't work for ISPs or people with multiple mail destinations Requires Sendmail.cf file rebuild to implement changes. Using the same three assumptions above, add the following line to /etc/mail/sendmail.mc and rebuild the cf: define(`SMART_HOST', `internalmx.peregrinehw.com')dnl This will result in ALL mail being relayed by the Milter box to be sent to internalmx.peregrinehw.com after it has been filtered. Requirements: 1. internalmx.peregrinehw.com should not be listed as a formal MX record in DNS but rather just an A record 2. The milter box shold be the MX record with the highest priority (hence the lowest MX record #) 3. The final destination server should NOT use the Milter box as an outbound relay server as this will case loops! 4. Your local-host-names file does NOT include the domain to be relayed (for the example it is peregrinehw.com) 2. Your relay-domains file DOES include the domain to be relayed for (for the example, it is peregrinehw.com) Third solution: Use a Mailertable on the Milter machine Pros: Using the mailertable allows you to designate the final destination for a particular domain. Only mail for those domains in the mailer table are relay'd to the internal mail server. Outbound mail is relay'd normally. Con: If the server name of the final destination changes then the Sendmail configuration has to be updated. Again using the same three assumptions from the first solution and the requirements for the second solution, add the following line to /etc/mail/mailertable and rebuild the files with a make all in /etc/mail: peregrinehw.com smtp:[internalmx.peregrinehw.com] NOTE: The square brackets turn off MX records for this host only to prevent mail loops! Hope this helps and thanks to Bob Falkenberg for his excellent help with the second and third solutions!