*** TERMS TO KNOW *** ****** MDA - Mail Delivery Agent Mail Delivery Agent is computer software that transfers the responsibility for the management of e-mail messages from the message transfer agent (MTA) to a recipient's environment, commonly transferring them into a mailbox. Within the Internet mail architecture, message delivery agents consist of two components, the message handling service side that accepts messages from the message transfer agent, and a component in the recipient's environment that affects message storage in a mailbox or other customized mechanisms. An example of an MDA is procmail. Procmail's handling of emails can be heavily influenced through the use of procmailrc scripts. Many products bundle multiple message delivery agents with the message transfer agent component, providing for site customization of the specifics of mail delivery to a user. List of message delivery agent software for Unix-like platforms binmail, the MDA part of Sendmail Delivermail fdm maildirproc maildrop postdrop procmail Courier-pop Courier-imap dovecot http://en.wikipedia.org/wiki/Mail_delivery_agent ****** MSA - Mail Submission Agent MSA is a software agent that receives electronic mail messages from a mail user agent (MUA) and cooperates with a mail transfer agent (MTA) for delivery of the mail. It uses a variant of the Simple Mail Transfer Protocol (SMTP), as specified in RFC 4409 RFC 4409 requires that clients are authorized and authenticated to use the mail submission service, e.g., as described in SMTP-AUTH, by other means such as RADIUS, or POP before SMTP. The MSA must check that the submitted mail is syntactically valid and conforms to the relevant site policies. * RFC 4409 contains some optional features: Enforce submission rights guarantees that the envelope sender address is valid and authorized with the used authentication. This is in essence the SPF model specified in RFC 4408. May add sender permits to add a Sender address header field if the envelope sender address does not match any author address in the "From" header field. This is roughly the Sender ID model specified in RFC 4406 - ignoring the tricky case of Resent-From header fields not covered in RFC 4409. * Many Internet service providers and enterprise or institutional networks restrict the ability to connect to remote MTAs on port 25 (the usual SMTP port). Availability of a Mail Submission Agent on port 587 enables nomadic users to continue to send mail via their preferred submission servers even from within others' networks. Using a specific submission server is a requirement when sender policies are enforced. Most of the benefits mentioned above may also apply to authenticated MTA, that is port 25 after the user logs in. In fact, the relevant server software is often the same for both services. However, the very fact that the service can be made available on a different port can be considered a further benefit, as it allows for circumvention of restrictions on port 25. http://en.wikipedia.org/wiki/Mail_submission_agent ****** Milters - Midstream Filters See also MIMEDefang for an example of "a milter". At each phase of the SMTP session, the filter is given data about the arriving message and then has an opportunity to make decisions concerning the message. For very large messages, this can have an enormous impact such as when a decision to reject can be made as early as possible. Moreover, unlike the former model, a milter-capable MTA can connect to multiple filters in parallel that serve specific purposes such as anti-virus, anti-spam, message authentication, flow regulation, etc. Finally, such filters can take special action on the message: add or remove recipients in the envelope; alter the body prior to delivery; add, change or remove header fields in the message, etc. For example, a very cursory look at our Raptor system shows that: 1) During the SMTP conversation, the mail is filtered by MIMEDefang. If the mail is being forwarded, MD calls SpamAssassin and the mail is scanned for Spam. 2) AFTER the SMTP Conversation concludes (by replying to the final "." with a 2xx SMTP reply code), the Milter has one more pass at the email and if the mail is delivered locally, the mail gets handed over to the delivery agent. In our case, that's procmail. 3) For local users, spamassassin is called by the global /etc/procmailrc script as part of the MDAs work. When the spamc command returns (possibly having altered the mail), procmail drops it into your INBOX. 4) For non-local users, MD can be configured to spam test the email prior to delivery http://en.wikipedia.org/wiki/Milter ****** MUA - Mail User Agent or email reader or email client Any agent acting as a client toward an email server, regardless of it being a mail user agent, a relaying server, or a human typing on a terminal. In addition, a web application providing message management, composition, and reception functionality is sometimes considered an email client. An MUA is only active when a user runs it. Messages arrive on the Mail Transfer Agent (MTA) server. Unless the MUA has access to the server's disk, messages are stored on a remote server and the MUA has to request them on behalf of the user. As a basic function, an MUA is able to introduce new messages in the transport system. Typically, it does so by connecting to either an MSA or an MTA, two variations of the SMTP protocol. The client needs to put a message quickly without worrying about where the message eventually will be delivered: that's why a transport system exists. Thus it always connects to the same preferred server, however, how does that server know that it should accept and relay submissions from that client? There are a number of ways. One way is to recognizes the client's IP address, e.g. because the client is on the same machine and uses internal address 127.0.0.1, or because the client's IP address is controlled by the same internet service provider that provides both internet access and mail services. Another way is to authenticate since the SMTP protocol has an authentication extension. The latter method eases modularity and nomadic computing but can cause email passwords to be sent using a clear-text protocol. Finally, there is a technique called POP Before SMTP that requires a user to check mail before sending email that then records the IP address and allows relay for a temporary period of time. This technique also uses IMAP and is used by PCCC. Client settings require the name or IP address of the preferred outgoing mail server, the port number (25 for MTA, 587 for MSA), and the user name and password for the authentication, if any. It is also typically possible to use port 465 for SSL encrypted SMTP sessions (SMTPS) that many clients and servers support. Transport Layer Security (TLS) encryption can be configured for the standard ports, if both the client and the server support it. PCCC supports TLS on ports 25 & 2025 and SSL on port 465. http://en.wikipedia.org/wiki/Mail_User_Agent ****** MTA - Mail Transfer Agent or Mail Relay A computer process or software agent that transfers electronic mail messages from one computer to another, in single hop application-level transactions. An MTA implements both the client (sending) and server (receiving) portions of the Simple Mail Transfer Protocol (SMTP). http://en.wikipedia.org/wiki/Mail_transfer_agent ****** POP3 testing: telnet intel1.peregrinehw.com 110 - THIS ESTABLISHES A CONNECTION TO intel1.peregrinehw.com 2004.89 POP3 SERVER user ebowman - THIS LOGS IN EBOWMAN pass - THIS AUTHENTICATES EBOWMAN WITH THE MAIL SERVER YOU WILL RECIEVE NOTIFICATION THAT "+OK mailbox open, Messages (where XXX is the number of messages in your inbox) retr (WHERE XXX IS THE NUMBER OF THE MESSAGE YOU WOULD LIKE TO RETRIEVE, 1 BEING THE FIRST MESSAGE IN THE BOX) - THIS RETRIEVES AND DISPLAYS ON-SCREE IN PLAIN TEXT 'AND' HTML THE SPECIFIED MAIL MESSAGE quit - SIGNS OUT OF THE CURRENT TELNET SESSION WITH THE POP3 SERVER ****** IMAP4 testing: per: IMAP: http://aplawrence.com/SCOFAQ/FAQ_scotec4testimap.html telnet intel1.peregrinehw.com 143 - THIS ESTABLISHES A CONNECTION TO THE DOVECOT IMAP SERVER AT abc1 login ebowman - THIS LOGS IN EBOWMAN USING MY PASSWORD abc1 select inbox - THIS CHECKS THAT THE INBOX CAN BE FOUND #IF THERE IS A MESSAGE IN THE INBOX YOU COULD REVIEW IT abc1 fetch 1 body[text] - THIS SHOWS A TEST MESSAGE IN "INBOX" FOR EBOWMAN abc1 logout - closes the connection to dovecotIMAP on intel1.peregrinehw.com:143 ****** SMTP testing telnet intel1.peregrinehw.com 25(OR 2025) - THIS ESTABLISHES A CONNECTION TO INTEL1.PEREGRINEHW.COM ESMTP SENDMAIL SERVER helo intel1.peregrinehw.com - THIS IS THE "HELO"(aka "Hello") GREETING, INITIATING DIALOG WITH THE MAIL SERVER BY IDENTIFYING THE MACHINE YOU'RE USING. SEE NOTE 39425 FOR MORE INFORMATION AS TO HOW "HELO" PERTAINS TO ANTI-SPAM. ;) mail from: rcpt to: subject: <-- THIS MUST END IN A <.> . <-- INPUT A PERIOD, THIS "SENDS" THE MESSAGE AT WHICH POINT THE SERVER WILL PROVIDE A DSN REGARDING YOUR SENT MESSAGE quit - THIS ENDS THE SESSION WITH THE SMTP SERVER ****** SMTP Culpability - As email moves from one server to the next(a "hop") on the way to its destination, SMTP adds a "header" marking the mail as having been passed along successfully, failed, or permanently failed. These headers help determine "SMTP Culpability" by showing which server along its way to its destination was culpable for a temporary or permanent failure. From RFC2821 3.8.2 Received Lines in Gatewaying - When forwarding a message into or out of the Internet environment, a gateway MUST prepend a Received: line, but it MUST NOT alter in any way a Received: line that is already in the header. "Received:" fields of messages originating from other environments may not conform exactly to this specification. However, the most important use of Received: lines is for debugging mail faults, and this debugging can be severely hampered by well-meaning gateways that try to "fix" a Received: line. As another consequence of trace fields arising in non-SMTP environments, receiving systems MUST NOT reject mail based on the format of a trace field and SHOULD be extremely robust in the light of unexpected information or formats in those fields. The gateway SHOULD indicate the environment and protocol in the "via" clauses of Received field(s) that it supplies. ****** Domain Keys - DomainKeys is an e-mail authentication system designed to verify the DNS domain of an e-mail sender and the message integrity. A domain owner generates a private/public key-pairs that will be used to sign messages originating from that domain. The public-key is placed in DNS as a TXT record. The private-key is kept on the mail server which sends email for the domain. When an email is submitted by an authorized user of that domain, dk-milter uses the private-key to digitally sign the email associated with the sending domain. The DomainKey-Signature header and signature is added to the email and the message is sent. When a message is received with a DomainKey-Signature header, dk-milter extracts the signature and the sender's domain from the email. It does a DNS lookup on the TXT record to fetch the public-key for the sender's domain. Using the public-key, dk-milter verifies whether the signature of the email is valid. An Authentication-Results: domainkeys=pass header is inserted in the email if the verification is successful. http://en.wikipedia.org/wiki/Domainkeys http://www.elandsys.com/resources/sendmail/domainkeys.html ****** ADSP - Author Domain Signing Practices (ADSP) is an optional extension to the E-mail authentication scheme DKIM, whereby a domain can publish the signing practices it adopts when relaying mail on behalf of associated authors. ADSP was adopted as a standards track RFC 5617 in August 2009. http://en.wikipedia.org/wiki/Author_Domain_Signing_Practices ****** RFC 2119 - specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. Specifies 'language and grammar rules' for writing/reading Request for Comments. Define's "RFC-ese" example: 1. MUST This word, or the terms "REQUIRED" or "SHALL", mean that the definition is an absolute requirement of the specification. 2. MUST NOT This phrase, or the phrase "SHALL NOT", mean that the definition is an absolute prohibition of the specification. 3. SHOULD This word, or the adjective "RECOMMENDED", mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course. 4. SHOULD NOT This phrase, or the phrase "NOT RECOMMENDED" mean that there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful, but the full implications should be understood and the case carefully weighed before implementing any behavior described with this label. http://tools.ietf.org/html/rfc2119 ****** MX Records should never be CNAMES - Though the practice of pointing MX records to CNAME (alias) records is not that uncommon, it certainly isn't in keeping with internet standards. The domain name used as the value of an NS resource record, or part of the value of a MX resource record must not be an alias. Not only is the specification clear on this point, but using an alias in either of these positions neither works as well as might be hoped, nor well fulfills the ambition that may have led to this approach. This domain name must have as its value one or more address records. Currently those will be A records, however in the future other record types giving addressing information may be acceptable. It can also have other RRs, but never a CNAME RR. Additional section processing does not include CNAME records, let alone the address records that may be associated with the canonical name derived from the alias. Thus, if an alias is used as the value of an NS or MX record, no address will be returned with the NS or MX value. This can cause extra queries, and extra network burden, on every query. ****** DSNs(DELIVERY STATUS NOTIFICATIONS)/SMTP STATUS CODES DSNs/SMTP STATUS CODES ARE RETURNED BY MAIL SERVERS TO INDICATE THE STATUS OF A SENDING/RECEIVING MAIL. THE STATUS CODE DEPENDS ON THE MAIL SERVER AND PREPROCESSOR USED TO ANALYZE THE LOG FILE. HERE ARE VALUES REPORTED BY MOST MAIL SERVERS SMTP ERRORS ARE CLASSIFIED IN 3 CATEGORIES * 2xx/3xx - successful * 4xx - failure, asking sender to try later * 5xx - permanent failure ********************************************************************* 2xx/3xx class - Success - These are SMTP protocols successful answers ********************************************************************* 200 Non standard success response 211 System status, or system help reply 214 Help message 220 Service ready 221 Service closing transmission channel 250 Requested mail action taken and completed - Your ISP mail server has successfully executed a command and the DNS is reporting a positive delivery. 251 User not local: will forward to: - Your message to a specified email address is not local to the mail server, but it will accept and forward the message to a different recipient email address. 252 Recipient cannot be verified - Recipient cannot be verified but mail server accepts the message and attempts delivery 354 Start mail input and end with . - Indicates mail server is ready to accept the message or instruct your mail client to send the message body after the mail server has received the message headers. ********************************************************************* 4xx class - Temporary Errors ********************************************************************* Those codes are temporary error message. They are used to tell the sender that an error occured but he can try to solve it but trying again, so in most cases, clients that receive such codes will keep the mail in their queue and will try again later. 421 Service not available, closing transmission channel - This may be a reply to any command if the service knows it must shut down. 450 Requested mail action not taken: mailbox busy or access denied - Your ISP mail server indicates that an email address does not exist or the mailbox is busy. It could be the network connection went down while sending, or it could also happen if the remote mail server does not want to accept mail from you for some reason i.e. (IP address, From address, Recipient, etc.) 451 Requested mail action aborted: error in processing - Your ISP mail server indicates that the mailing has been interrupted, usually due to overloading from too many messages or transient failure is one in which the message sent is valid, but some temporary event prevents the successful sending of the message. Sending in the future may be successful. 452 Requested mail action not taken: insufficient system storage - Your ISP mail server indicates, probable overloading from too many messages and sending in the future may be successful. 453 Too many messages - Some mail servers have the option to reduce the number of concurrent connection and also the number of messages sent per connection. If you have a lot of messages queued up it could go over the max number of messages per connection. To see if this is the case you can try submitting only a few messages to that domain at a time and then keep increasing the number until you find the maximum number accepted by the server. ********************************************************************** 5xx class - Permanent Errors ********************************************************************** These are permanent error codes. Mail transfer is definitly a failure. No other try will be done. 500 Syntax error, command unrecognized or command line too long 501 Syntax error in parameters or arguments 502 Command not implemented 503 Server encountered bad sequence of commands 504 Command parameter not implemented 521 does not accept mail or closing transmission channel - You must be pop-authenticated before you can use this SMTP server and you must use your mail address for the Sender/From field. 530 Access denied - A sendmailism ? 550 Requested mail action not taken (Relaying not allowed, Unknown recipient user, ...) - Sending an email to recipients outside of your domain are not allowed or your mail server does not know that you have access to use it for relaying messages and authentication is required. Or to prevent the sending of SPAM some mail servers will not allow (relay) send mail to any e-mail using another company’s network and computer resources. 551 User not local: please try or Invalid Address: Relay request denied 552 Requested mail action aborted: exceeded storage allocation - ISP mail server indicates, probable overloading from too many messages. 553 Requested mail action not taken: mailbox name not allowed - Some mail servers have the option to reduce the number of concurrent connection and also the number of messages sent per connection. If you have a lot of messages queued up (being sent) for a domain, it could go over the maximum number of messages per connection and/or some change to the message and/or destination must be made for successful delivery. 554 Requested mail action rejected: access denied 557 Too many duplicate messages - Resource temporarily unavailable Indicates (probable) that there is some kind of anti-spam system on the mail server. ************************ DNFTEC This acronym goes all the way back to 1996. Its the original "don't feed the trolls." DNFTEC stands for 'don't feed the energy creatures' and a great description of what/who these are can be found here: http://www.cryonet.org/cgi-bin/dsp.cgi?msg=6284 "There is a certain type of being that's all too common in the online world. I call them "Energy Creatures," a term I first heard on one of the commercial services. Energy Creatures are a bizarre lifeform which grow and feed off of the negative energy generated by others. Energy Creatures' favorite feeding tactic is to try to hurt people's feelings or get them angry. Then they can feed off the pain and anger they've generated. Their second favorite tactic is to hurt one person or group's feelings while gathering the sympathy of others. That way, when the injured party lashes back, others will jump to the Energy Creature's defense. Then the Energy Creature need do nothing except feed off the attention and the negative energy generated by the people fighting." While energy creatures are normally thought to frequent the various forums around the internet, the same term can be applied to spam mailers. Its more common today to find that spammers are actually members of organized crime around the world or are hackers with large robotic networks of hijacked computers called botnets. These fall under the DNFTEC acronym because they are trying to steal your money, personal information, contact information, of all of the above. PCCC highly recommends that users just mark the spam mail as such and move on. Do not hit the 'click here to be removed from our list' link since most of the time those just confirm to the spammers which emails addresses are actually live. ************************ Why Port 25 is Blocked and Alternate Ports that Are available Port 25 is often blocked by ISPs to help inhibit the distribution of spam. "But I'm not a spammer!" Many people say that and many people aren't spammers, but times are changing. As noted above in the DNFTEC section, more and more spammers are actually members of organized crime or just hackers in general. Many pieces of malware that are out in the wild don't do anything except make your computer a member of a larger network of hijacked computers called a botnet. Botnets can be very useful to people with bad intentions. Spammers use botnets to send out their millions of spam messages to the internet and as a result, eat bandwidth. The reason ISPs block port 25 is just in case your computer gets hijacked and becomes part of a botnet, it can't automatically send out email. http://www.pccc.com/base.cgim?template=port_25_blocked Ports like 2025, 2525 are non-standard ports that some ISPs provide. There is no standard to these port #'s. 587 is a standard submission port. It requires authentication and more ISPs are supporting (or even requiring) you to use this port. 465 is a standard SMTPS or SMTP over SSL port. ****** Last External Relay - ****** Deep Header Parsing aka Deep Header Inspection - Some anti-spam tools will actually look at the reputation of IP addresses in each of the received headers of an email. Reputation mechanisms usually involve multiple sources, e.g., DNSBLs and Honeypot-driven reputation services. Here’s a sample header: Delivered-To: address@gmail.com Received: by [[1.1.1.1]] with SMTP id e5cs33412ibd; Fri, 16 Apr 2010 08:38:08 -0700 (PDT) Received: by [[1.2.2.2]] with SMTP id e9mr1978437rvi.51.1271432287560; Fri, 16 Apr 2010 08:38:07 -0700 (PDT) Return-Path: me@mydomain.com Received: from SERVER.somedomain.com (mail.somedomain.com [[3.3.3.3]]) by mx.google.com with ESMTP id 11si4715430qyk.0.2010.04.16.08.38.06; Fri, 16 Apr 2010 08:38:07 -0700 (PDT) Received: from myserver.mydomain.com (unverified [[4.4.4.4]]) by SERVER.somedomain.com (XYZ MTA) with ESMTP id for ; Fri, 16 Apr 2010 11:38:08 -0400 [BRACKETED]: IP addresses that will be verified. http://www.emailsecuritymatters.com/site/blog/best-practices/deep-header-inspection-use-with-caution/ http://www.spamhaus.org/pbl/