.\" $Id: mimedefang-multiplexor.8.in,v 1.31 2005/10/14 16:16:53 dfs Exp $ .\"" .TH MIMEDEFANG-MULTIPLEXOR 8 "8 February 2005" .UC 4 .SH NAME mimedefang-multiplexor \- Process pool controller for mail filters. .SH SYNOPSIS .B mimedefang-multiplexor \fR[\fIoptions\fR] .SH DESCRIPTION \fBmimedefang-multiplexor\fR manages a pool of Perl processes for scanning e-mail. It is designed to work in conjunction with \fBmimedefang\fR(8) and \fBmimedefang.pl\fR(8). \fBmimedefang-multiplexor\fR opens a UNIX-domain socket and listens for requests for work from \fBmimedefang\fR. As requests come in, \fBmimedefang-multiplexor\fR creates Perl processes as needed to scan mail. The Perl processes are not killed when scanning is completed, but continue to run in a loop. Perl processes are re-used for subsequent e-mail messages. This eliminates the large overhead of starting a new Perl process for each incoming message. To avoid memory leaks, the Perl processes are killed after they have handled some number of scans. .SH OPTIONS .TP .B \-U \fIuser\fR Runs the multiplexor as \fIuser\fR rather than \fIroot\fR. This option is mandatory, and must match the \fB\-U\fR option supplied to \fBmimedefang\fR. .TP .B \-m \fIminSlaves\fR The minimum number of Perl processes to keep running at all times. The default is zero. .TP .B \-x \fImaxSlaves\fR The maximum number of Perl processes to run simultaneously. If a request comes in and all processes are busy, a temporary failure is signalled to the SMTP peer. The default is 2. .TP .B \-r \fImaxRequests\fR The maximum number of requests a given process handles before it is killed and a replacement started. The default is 500. .TP .B \-i \fIidleTime\fR The idle time in seconds after which to kill of excess Perl processes. That is, if the process is idle for longer than this time, and there are more than \fIminSlaves\fR running, the process is killed. Note that this is implemented as a timer which ticks every \fIidleTime\fR seconds; therefore, processes may be idle for up to twice this time before they are killed. The default for \fIidleTime\fR is 300 seconds. .TP .B \-b \fIbusyTime\fR The longest a Perl process is allowed to spend scanning an e-mail before it is declared hung up and killed. The default is 120 seconds. .TP .B \-Z This option specifies that the multiplexor should accept and process "status updates" from busy slaves. Note that this consumes one extra file descriptor per slave, plus a small amount of CPU time per status update. .TP .B \-c \fIcmdTime\fR The timeout for communication between \fBmimedefang-multiplexor\fR and \fBmimedefang\fR, or between \fBmimedefang-multiplexor\fR and a Perl scanning process. The default is 10 seconds. This timeout should be kept quite short. .TP .B \-w \fIwaitTime\fR When \fBmimedefang-multiplexor\fR starts the initial slaves, or needs to bring the number of running slaves up to the number defined by the \fB\-m\fR option, it does not start all the slaves at once, because this could overload your server. Instead, it starts one slave every \fIwaitTime\fR seconds. The default value for \fIwaitTime\fR is 3. .TP .B \-W \fIwaitTime\fR If you use this option, \fBmimedefang-multiplexor\fR will \fInever\fR activate a slave until \fIwaitTime\fR seconds have elapsed since the last slave activation. This could result in mail being tempfailed if slave activations do not keep pace with incoming mail. However, it may be preferable to tempfail mail rather than allow the load on your server to spike up too quickly. The default value for this option is 0, meaning that \fBmimedefang-multiplexor\fR will start slaves as quickly as necessary to keep up with incoming mail. .TP .B \-z \fIspooldir\fR Set the spool directory to \fIspooldir\fR. If this option is omitted, the spool directory defaults to @SPOOLDIR@. .TP .B \-s \fIpathName\fR The UNIX-domain socket on which \fBmimedefang-multiplexor\fR listens for requests. This should be specified as an absolute pathname. If this option is not supplied, it defaults to mimedefang-multiplexor.sock under the spool directory. .TP .B \-a \fIsocket\fR A socket for listening for requests. This is similar to the \fB\-s\fR socket, except that a restricted set of requests are processed. On this socket, the multiplexor will only process requests asking for status; it will not accept any commands to do scanning or that would consume a slave. See the SOCKET SPECIFICATION section for the format of \fIsocket\fR. .TP .B \-p \fIfileName\fR Causes \fBmimedefang-multiplexor\fR to write its process-ID (after becoming a daemon) to the specified file. .TP .B \-f \fIfilter_path\fR Normally, \fBmimedefang-multiplexor\fR executes a Perl filter script called \fBmimedefang.pl\fR to scan the e-mail. However, you can have it execute any program you like by specifying the full path to the program with the \fB\-f\fR option. This program must obey the protocol documented in \fBmimedefang-protocol\fR(7); see that manual page for details. Note that the \fB-f\fR option does \fInot\fR specify the "filter" to use with \fBmimedefang.pl\fR; instead, it specifies the program for \fBmimedefang-multiplexor\fR to execute. You almost certainly should \fInot\fR use this option unless you wish to replace \fBmimedefang.pl\fR with your own program. .TP .B \-F \fIrules_path\fR Specifies the path to the filter rules. By default, \fB@CONFDIR_EVAL@/mimedefang-filter\fR is used. If you use the \fB\-F\fR option, its value is passed to the underlying Perl filter program using \fB\-f\fR. .TP .B \-l Log certain events, including the output of the Perl slaves' standard-error, using syslog. Normally, the multiplexor does not log much information. .TP .B \-d Write debugging information about event-handling code in /var/log/mimedefang-event-debug.log. This is only of use to people debugging \fBmimedefang-multiplexor\fR. .TP .B \-R \fIkbytes\fR Limits the resident-set size of the slave filter processes to \fIkbytes\fR kilobytes. This limit is not supported on all operating systems; it is known to work on Linux. .TP .B \-M \fIkbytes\fR Limits the total memory space of slave filter processes to \fIkbytes\fR kilobytes. This limit is supported on all operating systems which support the setrlimit(2) system call. This should include most modern UNIX systems. We recommend that you monitor your slave filter processes and get a feel for how much memory they use. You should then limit the memory to two or three times the worst-case that you have observed. This can help mitigate denial-of-service attacks which use complicated MIME messages to force \fBmimedefang.pl\fR to consume lots of memory. .TP .B \-h Print usage information and exit. .TP .B \-t \fIfilename\fR Log statistical information to \fIfilename\fR. See the section STATISTICS for more information. .TP .B \-T Log statistical information using \fBsyslog\fR(2). You may use any \fB\-t\fR and \fB\-T\fR together, in which case statistical information is logged in a file and using \fBsyslog\fR. .TP .B \-u Flush the statistics file after every write. Normally, \fBmimedefang-multiplexor\fR does not flush the file; this is the best choice for minimizing disk I/O on a busy mail server. However, if you wish to watch statistics entries in real-time, you should enable flushing. .TP .B \-D Do not fork into the background and become a daemon. Instead, stay in the foreground. Useful mainly for debugging or if you have a supervisory process managing \fBmimedefang-multiplexor\fR. .TP .B \-q \fIqueue_size\fR Normally, if all slaves are busy and \fBmimedefang-multiplexor\fR receives another request, it fails it with the error "No free slaves." However, if you use the \fB\-q\fR option, then up to \fIqueue_size\fR requests will be queued. As soon as a slave becomes free, the queued requests will be handed off in FIFO order. If the queue is full and another request comes in, then the request is failed with "No free slaves". .TP .B \-Q \fIqueue_timeout\fR Queued requests should not stay on the queue indefinitely. If a queued request cannot be processed within \fIqueue_timeout\fR (default 30) seconds of being placed on the queue, it is failed with a "Queued request timed out" message. See the section "QUEUEING REQUESTS" for more discussion. .TP .B \-O \fIsock\fR Listen on a \fInotification socket\fR for connections from \fIlisteners\fR. \fBmimedefang-multiplexor\fR can inform external programs of state changes by sending messages over a notification socket. The external programs connect to this socket and then listen for notifications. See the section SOCKET SPECIFICATION for the format of \fIsock\fR. See the \fBmimedefang-notify\fR(7) man page for details of the notification protocol. .TP .B \-N \fImap_sock\fR Listen on a \fImap socket\fR for Sendmail SOCKETMAP connections. As of Sendmail 8.13, you can define a Sendmail map type that talks to a daemon over a socket. \fBmimedefang-multiplexor\fR implements that protocol; consult the \fBmimedefang-filter\fR(5) man page for detils (see the SOCKET MAPS section). See the section SOCKET SPECIFICATION for the format of \fImap_sock\fR. .TP .B \-I \fIbacklog\fR When \fBmimedefang-multiplexor\fR creates a listening socket, it calculates the "backlog" argument to \fBlisten\fR(2) based on the maximum number of slaves. However, you can explicitly set this backlog with the \fB\-I\fR option. Setting the backlog to a high value (around 30-50) may help on a very busy server. If you see mail log messages saying "MXCommand: socket: Connection refused" during busy periods, then that's an indication you need a higher listen backlog. .TP .B \-L \fIinterval\fR Log the slave status every \fIinterval\fR seconds. This logs a line using syslog; the line looks like this: .TP .nf Slave status: Stopped=s Idle=i Busy=b Killed=k Queued=q Msgs=m Activations=a .fi Here, "Stopped" is the number of non-running slaves, "Idle" is the number of idle slaves, "Busy" is the number of busy slaves, "Killed" is the number of killed slaves yet to be reaped, "Queued" is the number of queued requests, "Msgs" is the total number of messages processed since the multiplexor began running, and "Activations" is the number of times a Perl process has been started since the multiplexor began running. If you supply an \fIinterval\fR of 0 (which is the default), no periodic status logging is performed. If you supply an \fIinterval\fR of less than 5 seconds, it is silently reset to 5 seconds. .TP \fB\-S \fIfacility\fR Specifies the syslog facility for log messages. The default is \fImail\fR. See \fBopenlog\fR(3) for a list of valid facilities. You can use either the short name ("mail") or long name ("LOG_MAIL") for the facility name. .TP \fB\-E\fR Specifies that the multiplexor should create an embedded Perl interpreter. This can improve performance dramatically. But see the section "EMBEDDING PERL" for more information. .TP \fB\-X\fR \fIn\fR Specifies that the multiplexor should initiate a "tick" request every \fIn\fR seconds. This causes your \fIfilter_tick\fR function (if defined) to be called. Note that you have no control over which slave executes \fIfilter_tick\fR. If all slaves are busy when a tick occurs, that tick request is skipped and a warning message is logged. .TP \fB\-P\fR \fIn\fR Specifies that the multiplexor should run \fIn\fR tick requests in parallel. Each tick is run as often as specified with the \fB\-X\fR argument. (If you omit the \fB\-P\fR option, then the multiplexor behaves as if \fB\-P 1\fR had been specified.) If you run parallel ticks, each tick is assigned an integer identifying its "type". The type ranges from 0 to \fIn\fR-1. While there may be as many as \fIn\fR tick requests running at a time, only one tick of each type will be active at any time. .TP \fB\-Y\fR \fIlabel\fR Sets the tag used in the multiplexor's syslog messages to \fIlabel\fR instead of \fBmimedefang-multiplexor\fR. .SH SOCKET SPECIFICATION The \fB\-a\fR, \fB\-N\fR and \fB\-O\fR options take a socket as an argument. This socket can be specified as: .TP .B /path/to/socket A UNIX-domain socket .TP .B inet:portnum A TCP socket bound to port \fIportnum\fR, but which accepts connections only from the IPv4 loopback address (127.0.0.1). .TP .B inet_any:portnum A TCP socket bound to port \fIportnum\fR which will accept connections from any address. \fIUse inet_any with caution!\fR .SH QUEUEING REQUESTS Normally, if all slaves are busy, any additional requests are failed immediately. However, the \fB\-q\fR and \fB\-Q\fR options allow you to queue requests for a short amount of time. This facility is intended to gracefully handle a temporary overload; most of the time, your queue should be empty. Because \fBmimedefang\fR checks the number of free slaves when a connection is opened and fails the connection if there are no free slaves, the intent of the queue is to allow SMTP transactions that are already underway to continue if there is a slight overload. Any new connections will be failed if all slaves are busy, but existing connections are allowed to continue. Queuing requests may improve throughput on extremely busy servers. Note that if you supply the \fB\-q\fR option to \fBmimedefang\fR, then even new connections are allowed to queue. This may improve throughput by keeping the slave utilization higher. .SH EMBEDDING PERL Normally, when \fBmimedefang-multiplexor\fR activates a slave, it forks and execs \fBmimedefang.pl\fR. However, if the multiplexor was compiled with embedded Perl support, and you supply the \fB\-E\fR command-line option, the multiplexor works like this: .TP 1 It creates an embedded Perl interpreter, and sources \fBmimedefang.pl\fR with a special command-line argument telling it to read the filter, but not to enter the main loop. .TP 2 Each time a slave is activated, the multiplexor calls fork() and runs the \fBmimedefang.pl\fR main loop. This invokes \fBfilter_initialize\fR and then runs the main loop. .PP On some platforms (for example, Red Hat Linux 7.3 with Perl 5.6.1), it is not safe to destroy and recreate a Perl interpreter without causing a memory leak. On those platforms, if you attempt to reread the filter file (by sending the multiplexor a HUP signal or reread command), the filter will \fInot\fR be re-read, and a message will be logged to syslog. On those platforms, you must kill and restart MIMEDefang if you change the filter file. .PP On most platforms, however, a filter reread is accomplished by destroying and re-creating the embedded interpreter, re-sourcing \fBmimedefang.pl\fR and killing slaves as soon as they are idle. .PP \fBWARNING:\fR If you use the embedded Perl interpreter, the Perl variable "$$" will not be updated with the process ID of each slave. Instead, it will reflect the process ID of the master multiplexor. This is a limitation of embedded Perl; we have no plans to fix it. .SH STATISTICS With the \fB\-t\fR option, \fBmimedefang-multiplexor\fR logs certain events to a file. This file can be post-processed to gather statistics about the multiplexor. You can use it to tune the number of slaves you run, adjust timeouts, and so on. .PP Each line of the file looks like this: .nf YYYY/MM/DD:HH:MM:SS timestamp event key=val key=val... .fi Here, YYYY/MM/DD:HH:MM:SS is the local time of day. Timestamp is the number of seconds since January 1, 1970. Event is the name of an event. The valid events are: .TP .B StartSlave A slave process has been started. .TP .B KillSlave A slave process has been killed. .TP .B ReapSlave A dead slave process has been reaped. It is possible to have a ReapSlave event without a previous KillSlave event if the slave process terminated abnormally. .TP .B StartFilter A slave process has begun filtering an e-mail message. .TP .B EndFilter A slave process has finished filtering an e-mail message. .PP The possible keys in the key=value pairs are: .TP .B slave=\fIn\fR The slave involved in the event. Every slave is identified by a small integer. .TP .B nslaves=\fIn\fR The total number of running slaves immediately after the event happened. .TP .B nbusy=\fIn\fR The number of busy slaves (slaves which are processing an e-mail message) immediately after the event happened. .TP .B reason="\fIstring\fB" The reason for a StartSlave or KillSlave event. (Present only for these events.) .TP .B numRequests=\fIn\fR The number of e-mails processed by the slave. Present only for an EndFilter event. .PP If you send the \fBmimedefang-multiplexor\fR process a SIGHUP signal (kill -1 \fIpid\fR), it closes and reopens the statistics file. This is useful during log file rotation. .PP If you send the \fBmimedefang-multiplexor\fR process a SIGINT signal (kill -INT \fIpid\fR), it terminates all active-but-idle slaves. Also, any active-and-busy slaves terminate as soon as they finish filtering the current message. This is useful to force a reread of the filter rules file without stopping and restarting Sendmail. .PP If you send the \fBmimedefang-multiplexor\fR process a SIGTERM signal (kill \fIpid\fR), it terminates all slaves and exits immediately. .SH AUTHOR \fBmimedefang-mulitplexor\fR was written by David F. Skoll . The \fBmimedefang\fR home page is \fIhttp://www.mimedefang.org/\fR. .SH SEE ALSO mimedefang.pl(8), mimedefang-filter(5), mimedefang(8), mimedefang-protocol(7)