** Note the following notes are slightly out of date, see: ./email.HTML/index for the lecture slides
Important terms:
This part of the lecture will be covering only MTAs and sendmail in particular. The sendmail program is probably the oldest and most widespread MTA in use on the internet.
Sendmail is a huge program and its main configuration file sendmail.cf is daunting for veteran administrators, let alone sendmail newbies! The main reason for the complexity has to do with the plethora of addressing styles in use on the internet. Here are some examples:
Stephen Dowdy <dowdy@cs.colorado.edu>
"Todd C. Miller" <Todd.Miller@cs.colorado.edu>
Hans.Mohling@cs.colorado.edu
tor@cs.colorado.edu
The above examples are relatively straight forward, but many variations
exist and many other address styles also; quite a few are UUCP style
addresses that are hold-overs from the seventies when Unix machines
were connected by dedicated point-to-point phone lines. Addresses
in the UUCP style have a format like:
<host1>!<host2>!<host3>!<hostN>!tor
What that specifies is the chain of machines that are linked together.
In order to have effectively sent mail back in the UUCP days, you had
to know exactly what machines would lie in between you and your
intended destination.
What is worse in terms of the complexity of sendmail's config file is that mixing of the two formats is allowed!
<hostN>!tor@coatl.colorado.edu
Would mean that hostN was connected to the machine coatl with
a UUCP connection; email would first go to coatl where it would
be queued until it could be sent via UUCP to hostN.
Here is a snippet of address resolution rules from the config file:
# lookup localpart (user@)
R<$+> $* < @ $+ > $* $: <USER $(access $2@ $: ? $) > <$1> $2 < @ $3 > $4
# no match, try full address (user@domain rest)
R<USER ?> <$+> $* < @ $* > $*
$: <USER $(access $2@$3$4 $: ? $) > <$1> $2 < @ $3 > $4
# no match, try address (user@domain)
R<USER ?> <$+> $+ < @ $+ > $*
$: <USER $(access $2@$3 $: ? $) > <$1> $2 < @ $3 > $4
# no match, try (sub)domain (domain)
R<USER ?> <$+> $* < @ $+ > $*
$: $>LookUpDomain <$3> <$1> <>
OUCH! It looks like goobledy gook at first glance, and even on the
tenth glance. Fortunately,
much of the sendmail config file details don't change from
site to site so a lot of the work creating the file is automated.
If you do want to learn the nuts-n-bolts of sendmail, there are
two places to start:
By the way, sendmail was originally written by Eric Allman when he was a student at Berkeley. He continues to maintain the source through his company Sendmail, inc. See: http://www.sendmail.com/ for more info.
Let's start by simply looking at an email message:
mroe % cat ~/Mail/inbox/3414
Return-Path: tor@cs.colorado.edu
Received: from mroe.cs.colorado.edu (mroe.cs.colorado.edu [128.138.243.151])
by suod.cs.colorado.edu (8.9.3/8.9.2) with ESMTP id OAA28688
for <tor@suod.cs.Colorado.EDU>; Tue, 14 Mar 2000 14:32:19 -0700 (MST)
Received: from mroe.cs.colorado.edu (localhost.cs.colorado.edu [127.0.0.1])
by mroe.cs.colorado.edu (8.9.3/8.9.2) with ESMTP id OAA13799
for <tor@cs.colorado.edu>; Tue, 14 Mar 2000 14:32:18 -0700 (MST)
Message-Id: <200003142132.OAA13799@mroe.cs.colorado.edu>
To: tor@cs.colorado.edu
Subject: exciting example email message
Date: Tue, 14 Mar 2000 14:32:18 -0700
From: Tor Mohling <tor@cs.colorado.edu>
hollow world
The first part of an email message is called the header part.
The remainder of the message is called the body.
Each line in the header has format of:
<field>: <value>
The header is seperated from the body of the mail message
by a single blank line.
The first header line above, Return-Path, is added by the MUA (mail user agent) I use - MH.
The next two lines are Received stamps. Every machine that processes your email message adds a Received line to your email's header. In this case, you can see that the machine mroe received the message from localhost (this is from the second of the two Received lines) and then sent it to the machine suod. (We'll explore why the email address tor@cs.colorado.edu was delivered instead to tor@suod.cs.Colorado.EDU later.)
The next line is of course the Date when the message was sent. Followed by the From address of the sender.
The Message-Id is added by the originating sendmail daemon and is unique globally.
Finally, the Content-Type is also added by the originating sendmail daemon to comply with MIME document attachment standards.
You will see lots of other header fields if you start looking at them in all the email you get. Really Mandatory header fields are From, To and Date. Others are useful for debugging (particularly the Received fields.) Most MTA servers, like sendmail, will propogate header fields that are not recognized. This just means that if the particular MTA doesn't use or understand a given header field present in a particular email message, then it just ignores it, but leaves it intact in the header. You can create your own header fields to use for your own purposes. For example, a trouble ticketing system we use at CSops called TQS makes use of a bunch of special header fields to keep track of trouble queue requests and responses. Whenever you do roll up your own header fields, the convention is to always start their names with X- to avoid any namespace collisions with real header fields used by the various mail agents. For example, a co-worker at CSops always includes the following header fields in his email messages:
X-url: http://www.cs.colorado.edu/~garnett
X-wotd: steatopygia (stee-at-uh-pij-ee-uh) noun: An extreme accumulation of
X-wotd: fat on the buttocks. [Steato- fat + Greek puge, rump + -ia.]
(The X-wotd (WordOfTheDay) header gets automatically rotated every week or so.)
Sendmail will also make use of header fields when it is reporting error messages, like about un-deliverable mail for example. Here is what I got back trying to send mail to an unknown user:
Return-Path: MAILER-DAEMON@suod.cs.colorado.edu
Received: from coatl.cs.colorado.edu (IDENT:root@coatl.cs.colorado.edu [128.138.242.212])
by suod.cs.colorado.edu (8.9.3/8.9.2) with ESMTP id PAA29651
for <tor@suod.cs.Colorado.EDU>; Tue, 14 Mar 2000 15:36:36 -0700 (MST)
Received: from localhost (localhost)
by coatl.cs.colorado.edu (8.9.3/8.9.2) with internal id PAA09434;
Tue, 14 Mar 2000 15:36:21 -0700 (MST)
Date: Tue, 14 Mar 2000 15:36:21 -0700 (MST)
From: Mail Delivery Subsystem <MAILER-DAEMON@coatl.cs.colorado.edu>
Message-Id: <200003142236.PAA09434@coatl.cs.colorado.edu>
To: <tor@coatl.cs.colorado.edu>
MIME-Version: 1.0
Content-Type: multipart/report; report-type=delivery-status;
boundary="PAA09434.953073381/coatl.cs.colorado.edu"
Subject: Returned mail: User unknown
Auto-Submitted: auto-generated (failure)
This is a MIME-encapsulated message
--PAA09434.953073381/coatl.cs.colorado.edu
The original message was received at Tue, 14 Mar 2000 15:36:20 -0700 (MST)
from IDENT:tor@localhost.cs.colorado.edu [127.0.0.1]
----- The following addresses had permanent fatal errors -----
<bazooka_joe@cs.colorado.edu>
----- Transcript of session follows -----
... while talking to cs.colorado.edu.:
>>> RCPT To:<bazooka_joe@cs.colorado.edu>
<<< 550 <bazooka_joe@cs.colorado.edu>... User unknown
550 <bazooka_joe@cs.colorado.edu>... User unknown
--PAA09434.953073381/coatl.cs.colorado.edu
Content-Type: message/delivery-status
Reporting-MTA: dns; coatl.cs.colorado.edu
Received-From-MTA: DNS; localhost.cs.colorado.edu
Arrival-Date: Tue, 14 Mar 2000 15:36:20 -0700 (MST)
Final-Recipient: RFC822; bazooka_joe@cs.colorado.edu
Action: failed
Status: 5.1.1
Remote-MTA: DNS; cs.colorado.edu
Diagnostic-Code: SMTP; 550 <bazooka_joe@cs.colorado.edu>... User unknown
Last-Attempt-Date: Tue, 14 Mar 2000 15:36:21 -0700 (MST)
--PAA09434.953073381/coatl.cs.colorado.edu
Content-Type: message/rfc822
Return-Path: <tor>
Received: from coatl.cs.colorado.edu (IDENT:tor@localhost.cs.colorado.edu [127.0.0.1])
by coatl.cs.colorado.edu (8.9.3/8.9.2) with ESMTP id PAA23827
for <bazooka_joe@cs.colorado.edu>; Tue, 14 Mar 2000 15:36:20 -0700 (MST)
Message-Id: <200003142236.PAA23827@coatl.cs.colorado.edu>
To: bazooka_joe@cs.colorado.edu
Subject: nuthin
Date: Tue, 14 Mar 2000 15:36:20 -0700
From: Tor Mohling <tor>
MIME-Version: 1.0
hell o whirled
--PAA09434.953073381/coatl.cs.colorado.edu--
The sendmail binary itself lives in /usr/sbin/sendmail on both OpenBSD and RedHat Linux. There are also a couple of related programs that are actually just links to the sendmail binary itself. Here is a list of some of them:
/usr/sbin/sendmail The Daemon itself - sends and receives email
/usr/bin/newaliases Rebuilds the 'aliases' database
(This is just: sendmail -bi)
/usr/bin/mailq Lists out the current contents of the
(outgoing) mail queue. (just: sendmail -bp)
There are quite a few others, but these are the most commonly
referenced. Sometimes you will find sendmail itself living
in /usr/lib instead of /usr/sbin.
The run-time configuration files for sendmail are often found in the directory /etc/mail (e.g. on RedHat Linux) but you will also find them simply living in /etc (e.g. on OpenBSD).
The main configuration file is called sendmail.cf and can include pointers to the following other files:
access This is a list of domains and user@domain from
whom mail is ALWAYS rejected.
aliases This is a list of email 'aliases' that translate
addresses to other address or even lists (see below).
relay-domains A list of domains for whom 'relaying' is allowed
(see below).
sendmail.cf The main config file
sendmail.cw A list of machines which are "MX'd" to this machine
(see sendmail and DNS below).
sendmail.hf This file contains error messages and help text
used by sendmail (you can define what sendmail will
report back for certain errors, for example).
sendmail.pid Contains the Process ID of the currently running
sendmail daemon.
You may not see some of the above files depending on how sendmail
on the machine in question has been configured.
Also, some of the files above (notably aliases and access have
binary hashed databases associated with them, called, for
example aliases.db. The hashed databases are used to speed up
access into large amounts of data (like a big aliases file).
A command to rebuild a database might look like this:
makemap hash /etc/mail/access < /etc/mail/access
Another method that is used for VERY large database files (i.e. on major
mailing list servers for example) is to use LDAP to store and retrieve
the data.
Finally, the last important infrastrure items are the spooling directories.
Incoming email to be delivered locally will often be placed in a user's mailspool which typically resides in /var/spool/mail/<user> or sometimes /var/mail/<user>. You can change how mail will be delivered for yourself locally by creating a .forward file in your home directory (see below for details and cautions !)
Outgoing mail that cannot (for whatever reason) be immediately sent on to its destination will be placed in another directory, typically /var/spool/mqueue, to await future delivery.
DNS MX (Mail eXchange) Records are used to tell Mail Transport Agents where email for a particular host should really be delivered. The MX record consists of a list of hostnames and associated preference levels. The @ symbol is used as a short-hand meaning: "the primary MX machine for the current domain." The lower the preference value, the more preferential the associated host. Delivery of email is attempted to each host in order of preference from lowest to highest until the delivery succeeds. For the CS department, the @ means mroe.cs.colorado.edu.
For example, the MX record for my desktop host might be denoted as follows:
10 coatl 99 @
This indicates that email addressed to the host coatl should
first be delivered directly to that machine. Only if delivery to coatl
is not possible will the email be sent to mroe instead.
You can view the MX record for an arbitrary host by using the nslookup command. Here is an example:
serl % nslookup
Default Server: suod.cs.colorado.edu
Address: 128.138.192.205
> set type=MX
> coatl
Server: suod.cs.colorado.edu
Address: 128.138.192.205
coatl.cs.colorado.edu preference = 99, mail exchanger = cs.colorado.edu
coatl.cs.colorado.edu preference = 10, mail exchanger = coatl.cs.colorado.edu
...
You can actually use sendmail directly to send email - in fact, this is one way that evil hackers use to forge email. It is also a good way to test a sendmail configuration remotely. If you do, you need to actually "speak" SMTP (The Simple Mail Transfer Protocol).
Here is a list of the basic SMTP commands:
HELO <hostname> identifies the connecting host (this is *your* hostname,
not the hostname the sendmail daemon you are connecting too).
MAIL From: <fromaddr> identifies the sender.
RCPT To: <toaddr> identifies the recipient, there may be more than one of these
VRFY <addr> verifies an email address. many sites disable this one to stop
the nasty, completely evil spammers.
EXPN <addr> expand the specified address (i.e. if it is an alias) This one
is also frequently disable to stop spammers.
DATA following the DATA command you may enter the text of the email
message. you can enter more header fields also, but they must
be first and have no leading blank lines and must be followed by
a blank line. Terminate the DATA portion with a '.' by itself.
QUIT
To access sendmail directly, you can telnet to the SMTP port (i.e. TCP port 25)
on a specific host. Here is an example:
serl % telnet mroe.cs.colorado.edu 25
Trying 128.138.243.151...
Connected to mroe.cs.colorado.edu.
Escape character is '^]'.
220 cs.colorado.edu ESMTP Sendmail 8.9.3/8.9.2/CSOps ready at Sat, 8 Apr 2000 12:11:04 -0600 (MDT)
EXPN tor
502 Sorry, we do not allow this operation
HELO serl.cs.colorado.edu
250 mroe.cs.colorado.edu Hello serl.cs.colorado.edu [128.138.243.145], pleased to meet you
MAIL From: SomeoneSpecial@myhouse.net
250 SomeoneSpecial@myhouse.net... Sender ok
RCPT To: tor@cs.colorado.edu
250 tor@cs.colorado.edu... Recipient ok
DATA
354 Enter mail, end with "." on a line by itself
X-my-personal-header: just a stupid header to illustrate..
hell o' whirled..
.
250 MAA16013 Message accepted for delivery
QUIT
221 mroe.cs.colorado.edu closing connection
Connection closed by foreign host.
And then, here is the actual mail message I received:
Return-Path: SomeoneSpecial@myhouse.net
Received: from mroe.cs.colorado.edu (mroe.cs.colorado.edu [128.138.243.151])
by suod.cs.colorado.edu (8.9.3/8.9.2) with ESMTP id MAA04643
for <tor@suod.cs.Colorado.EDU>; Sat, 8 Apr 2000 12:14:34 -0600 (MDT)
From: SomeoneSpecial@myhouse.net
Received: from serl.cs.colorado.edu (serl.cs.colorado.edu [128.138.243.145])
by mroe.cs.colorado.edu (8.9.3/8.9.2) with SMTP id MAA16013
for tor@cs.colorado.edu; Sat, 8 Apr 2000 12:12:15 -0600 (MDT)
Date: Sat, 8 Apr 2000 12:12:15 -0600 (MDT)
Message-Id: <200004081812.MAA16013@mroe.cs.colorado.edu>
X-my-personal-header: just a stupid header to illustrate..
hell o' whirled..
Notice that the first (bottom) Received line tells us the real sender
of the email - fortunately...
Aliases are used to re-route email from one destination to an arbitrary number of other destinations. The simplest alias just redirects email from one username to another. For example,
postmaster: tor
The above alias would deliver all email addressed to postmaster to the user tor.
The postmaster alias is required (i.e. email to postmaster must be deliverable)!
Instead of one user, you can specify a comma-separated list. That list can include
other aliases, too. For example:
graders: tor,clements,andersor
This would cause email sent to graders to be delivered to all three specified
users. This is the simplest way to implement a mailing list; it is also
probably the most cumbersome. You may also specify a file from which a list
addresses will be used for an alias:
graders: :include:/home/tor/saclass/graders
This would cause the file /home/tor/saclass/graders to be referenced each time
the alias 'graders' is used. (that means that addresses can be added to the
file /home/tor/saclass/graders without requiring a rebuild of the aliases database.)
You may specify instead that mail to a given alias be appended to a file. The file must be an absolute pathname and enclosed in double-quotes " to protect the /'s. For example,
webmaster: "/dev/null"
This would cause all email addressed to 'webmaster' to be appended to the file /dev/null
(hee hee). Using this feature is considered to be a security risk, especially if the
sendmail daemon is (stupidly) running as root!
Finally, you may cause mail to a given alias to be redirected to a program. The syntax for this is pretty obvious after any amount of shell experience:
trouble: "|/usr/local/bin/auto-responder"
This alias would cause email to 'trouble' to be sent instead as the STDIN stream
to the program /usr/local/bin/auto-responder (which is some fictional program
that might automatically respond to the email to 'trouble' with a ticket number
or something). Again, this feature is extremely dangerous and must be used
with care. Any program you use must be well-tested for security exploits before
it is actually deployed.
The .forward file allows an ordinary user to use some of the features of the system-wide alias file to manipulate his or her own email address. For example, you may specify a different email address to which all mail should be forwarded like so:
tor@niftymail.com
The .forward file must be in your homedirectory and conform to
restrictive file ownership and permissions (i.e. you
must own it and it must not be group or world writable).
Be careful you don't accidentally create a mail-loop with
a .forward file. The University of Colorado maintains a campus
wide database called mailhome that keeps track of the host
where each user receives their mail. So, if you were to set up
a .forward file that sent your mail from ucsub to nago
for example, email would go to ucsub where it would be forwarded to
nago (because of the .forward file);
nago would get the mail again and it would be sent back to ucsub
(because of the campus mailhome system); and so on.
You can specify a comma-separated list of addresses in the .forward file as well. Put a \ (back-slash) in front of your own username to force lacal delivery of email in addition to other addresses in the list. For example:
\tor, "/home/tor/mail-archive", tor@niftymail.com
This would deliver mail to my local mailspool (e.g. /var/spool/mail/tor),
append the mail to an archive-file, and forward a copy on to the address tor@niftymail.com.
You may also send mail to a program, as in the system-wide aliases file. We'll look at that feature in more detail when we get to MH, a Mail User Agent.
Fortunately, configuring Sendmail is miles easier than it used to be! Because many of the rules need for the interpretation of addresses are going to be the same irregardless of site, as well as other common assumptions, high level macros may be referenced in a .mc file which may be subsequently translated into a real sendmail .cf file.
The source distribution of Sendmail contains within it a series of subdirectories under the parent directory of cf. This the m4 configuration tree and the top level looks like so:
coatl % ls
README domain/ hack/ mailer/ sh/
cf/ feature/ m4/ ostype/ siteconfig/
(You'll notice that there is another cf directory within the cf parent dir -
don't be confused by this.) M4 is a macro preprocessing language that
was originally intended as a front-end for other programming languages, with
the idea that it would help to enable more readable code.. its value
for that is debatable, but it works wonders for Sendmail configuration.
The m4 macros have a form as follows:
name(arg1, arg2, ... argN)
Here are some of the commonly used default m4 macros;
define(arg1, arg2) define a new macro named arg1, with value arg2
undefine(arg1) undefine a previously defined macro named arg1
include(arg1) include another file into the input stream at this point
dnl Delete to NewLine - this is the M4 comment indicator
divert(N) used to manipulate output streams
Another important document that ships with the Sendmail source is called op.me (the .me indicates that it is using the 'e' macro-package for the troff text formatter, the original unix text formatting language). The source distribution also include a postscript version of this document. I also recommend that you read through it. It is intended to be the reference for configuring Sendmail.
The basic process of configuring Sendmail using M4 is as follows:
It probably comes as no great surprise that the place to start when you want to get Sendmail going under OpenBSD is the afterboot(8) manpage! Here is the relevant excerpt:
Sendmail
OpenBSD ships with a default /etc/sendmail.cf file that will work for
simple installations; it was generated from openbsd-proto.mc in
/usr/share/sendmail/cf. Please see /usr/share/sendmail/README and
/usr/share/doc/smm/08.sendmailop/op.me for information on generating your
own sendmail configuration files.
As you can see from the above excerpt, OpenBSD ships with the cf configuration
tree already resident in /usr/share/sendmail. It also ships with the op.me
document as /usr/share/doc/smm/08.sendmailop/op.me. There is a Makefile
living with the op.me document which you can use to convert it into postscript
for online viewing or printing. The all important Sendmail config README file
is also present in /usr/share/sendmail/README
The sendmail.cf file that Sendmail itself uses when it starts up lives in /etc/sendmail.cf, as do (by default) the other Sendmail database files like /etc/aliases and /etc/access.
To have Sendmail started at boot-time you need to (of course!) edit the /etc/rc.conf file. Make sure there is a line that looks like the following:
sendmail_flags="-bd -q30m"
The -bd argument tells Sendmail to Be a Daemon while the -q30m
argument directs Sendmail to check and process its (outgoing) /var/spool/mqueue
directory every 30 minutes.
To fire up sendmail from the command-line if your system is already running simply,
sudo /usr/sbin/sendmail -bd -q30m
and you will be good to go. It is important to invoke Sendmail by using its
absolute pathname in order that you may tell it to re-read the /etc/sendmail.cf
file by issuing a kill -1 <pid> (i.e. send the sendmail process a hangup
signal, aka HUP).
In order to configure Sendmail using the m4 configuration process you need to install a few more RPM files. Notably,
Like OpenBSD, Sendmail's main config file is placed in /etc/sendmail.cf. Unlike 'BSD, RedHat places the m4 cf tree in /usr/lib/sendmail-cf. Note also that the sendmail binary itself lives in /usr/lib/sendmail. And finally, the oft-referenced README is at /usr/lib/sendmail-cf/README.
The startup file for sendmail under RedHat is of course /etc/rc.d/init.d/sendmail and you must make sure that the proper links have been made in the runlevel directories. For example:
xibalba % ls -l /etc/rc.d/rc3.d/*sendmail
lrwxrwxrwx 1 root root 18 Jan 10 22:18 /etc/rc.d/rc3.d/S80sendmail -> ../init.d/sendmail*
Now let's look at a bunch of examples.. The first is for coatlicue, my gateway machine. Here is the file in its entirety; I copied the file /usr/share/sendmail/cf/openbsd-proto.mc to start with.
divert(-1)
#
# Copyright (c) 1998 Sendmail, Inc. All rights reserved.
# Copyright (c) 1983 Eric P. Allman. All rights reserved.
# Copyright (c) 1988, 1993
# The Regents of the University of California. All rights reserved.
#
# By using this file, you agree to the terms and conditions set
# forth in the LICENSE file which can be found at the top level of
# the sendmail distribution.
#
#
#
# This is the prototype file for a configuration that supports nothing
# but basic SMTP connections via TCP.
#
divert(0)dnl
include(`../m4/cf.m4')
VERSIONID(`@(#)openbsd-proto.mc $Revision: 1.4 $')
OSTYPE(openbsd)
dnl MASQUERADE_AS(colorado.edu)
MASQUERADE_AS(coatlicue.colorado.edu)
FEATURE(`masquerade_envelope')
define(`confPRIVACY_FLAGS', `needmailhelo, noexpn, novrfy, authwarnings')
FEATURE(`always_add_domain')
FEATURE(`access_db', `hash -o /etc/mail/access')
define(`ALIAS_FILE',`/etc/mail/aliases,/etc/mail/majordomo')
define(`confTRUSTED_USERS', `mjrdomo')
FEATURE(rbl)
FEATURE(nouucp)
MAILER(local)
MAILER(smtp)
dnl
dnl Enforce valid Message-Id to help stop spammers
dnl
LOCAL_RULESETS
HMessage-Id: $>CheckMessageId
SCheckMessageId
R< $+ @ $+ > $@ OK
R$* $#error $: 553 Header Error
The first chunk of the file is the header comments. Notice how the comment
block is enclosed by divert() macros. This is essential or the # comments will
become sytax errors. Immediately following the closing divert() macro is
an important include() that must be in the file; without it the sendmail.cf
file can not be built.
The VERSIONID macro can contain an RCS or SCCS version ID.
The OSTYPE must also be specified.
the special m4 directive dnl is used as a comment; it stands for delete to newline.
The MASQUERADE_AS macro is an important one for private networks. The cf README file has this to say about it:
MASQUERADE_AS(host.domain)
This causes mail being sent to be labeled as coming from the
indicated host.domain, rather than $j. One normally masquerades as
one of one's own subdomains (for example, it's unlikely that I would
choose to masquerade as an MIT site). This behaviour is modified by
a plethora of FEATUREs; in particular, see masquerade_envelope,
allmasquerade, limited_masquerade, and masquerade_entire_domain.
The symbol $j is set by Sendmail to be th canonical hostname of
the local machine. If you are setting up sendmail on a home machine
and you send email out through a server provided by your ISP, you
might, for example, masquerade as that host.
The next line:
define(`confPRIVACY_FLAGS', `needmailhelo, noexpn, novrfy, authwarnings')
This enables some important security/anti-spam features. The needmailhelo
forces the sending MTA to correctly use the HELO command to initiate
an SMTP session. The noexpn and novrfy turn off the corresponding
SMTP commands; this is very important for stopping the evil spammers
as it makes it so a spammer can't figure out usable addresses at your site.
Finally, the authwarnings causes sendmail to set X-Authentication-Warning
headers when it thinks there is reason to suspect the authenticity of
an email message.
The next line turns on another feature called always_add_domain. This feature causes the domain (or masqueraded domain) to always be added on locally delivered mail. This is followed by another feature that turns on the use of the relay access control file /etc/mail/access; it also specifies that the database should be converted into a hashed form. The next line changes the default alias file to be /etc/mail/aliases; it also defines a second alias file, /etc/mail/majordomo. The line following this (as well as the majordomo alias file are for enabling the majordomo mailing list management software.
The rbl feature enables the Realtime Blackhole List that is a centrally maintained database of known spammer addresses. The system utilizes DNS to share rbl information with local sendmail daemons. The nouucp feature turns off uucp-style addressing.
The two MAILER macros are generally mandatory. The first, local enables the delivery of mail to local user mailspools, the second, smtp enables the delivery of mail across the internet to other MTAs.
Finally, the LOCAL_RULESETS section is part of the default OpenBSD .mc file that I copied in the first place. This particular ruleset is defining a check for valid MessageId header fields.
coatlicue% sudo make coatlicue.cf
rm -f coatlicue.cf
(cd /usr/share/sendmail/cf && m4 coatlicue.mc > /usr/share/sendmail/cf/coatlicue.cf)
chmod 444 coatlicue.cf
coatlicue% sudo cp coatlicue.cf /etc
The .mc file for my RedHat PC xibalba sitting inside on my private network
is very simple:
VERSIONID(`@(#)clientproto.mc 8.12 (Berkeley) 5/19/1998')
OSTYPE(`linux')
FEATURE(nullclient, coatlicue)
The OSTYPE needs to be defined and then a special feature nullclient
is all we need to specify. The nullclient feature takes an additional
argument: the hostname of the local email server (in this case, of course,
the server is coatlicue, my gateway box.
The next example is the .mc for nago, the CSEL mail server:
divert(-1)
#
# NOTE: This file is under RCS in /src/cs/Config-files/sendmail/cf,
# use rcs to make your changes or they *WILL* get overwritten.
#
# $Id: class11.txt,v 1.4 2001/04/05 16:25:39 tor Exp tor $
#
# Sendmail 8 configuration file for nago.cs.colorado.edu
# For many of the default values check out ../domain/csops-generic.m4
#
include(`../m4/cf.m4')
VERSIONID(`$Id: class11.txt,v 1.4 2001/04/05 16:25:39 tor Exp tor $')
OSTYPE(solaris2)
DOMAIN(csops-generic)
dnl
dnl don't deliver to multiple users at once since we use quotas
define(`LOCAL_MAILER_FLAGS', `fSn9')
dnl
dnl get mailhomes via hesiod but look in real aliases file first
define(`ALIAS_FILE', ALIAS_FILE`,hesiod:maildrop')
dnl
dnl local address rewrites
FEATURE(mailnametable, `hesiod:mailname')
dnl
dnl mailers we support
MAILER(local)
MAILER(smtp)
There are two home-rolled macros in the nago.mc file. The first
is the DOMAIN csops-generic which is another .mc file that is kept
in the domain subdir of the m4 cf tree. The csops-generic domain
includes a bunch of rules that are used in common with a number of different
config files. Here are some of the important bits from the domain file:
define(`confQUEUE_LA', `15')
define(`confREFUSE_LA', `30')
define(`confMESSAGE_TIMEOUT', `7d/6h')dnl
define(`confDONT_BLAME_SENDMAIL', `GroupWritableDirPathSafe GroupWritableIncludeFileSafe \
ForwardFileInGroupWritableDirPath FileDeliveryToHardLink FileDeliveryToSymLink')
The first define sets the value for the local machine's load average
at which sendmail will queue messages instead of delivering them immediately.
The second define similarly sets the load average at which senmail will simply
refuse connections completely. Both of these measures are necessary when the
mail server gets swamped and can not keep up with the traffic: either you
need a bigger/faster machine, or, more likely, there is a misbehaving program
running on the server, or you just got spammed and there are 50,000 emails in your
outgoing /var/spool/mqueue.
The MESSAGE_TIMEOUT setting has two values. The first, 7d, says that email which can not be delivered will be deleted from the outgoing queue after seven days. The second value, 6h, says that an informational message should be sent back to the original sender explaining that their email is not yet delivered.
The last define is an important one to mention in terms of sendmail security. The DONT_BLAME_SENDMAIL definition turns off sendmail's built in paranoia about file and directory ownership and permission. It is strongly recommended that you do not turn this on!
The client .mc file for the CSEL lab is another trivial one:
include(`../m4/cf.m4')
VERSIONID(`$Id: class11.txt,v 1.4 2001/04/05 16:25:39 tor Exp tor $')
OSTYPE(unknown)
FEATURE(nullclient, csel.cs.colorado.edu)
define(`confSMTP_LOGIN_MSG', `$j Sendmail $v/$Z/nullclient ready at $b')
The OSTYPE of course needs to be filled in to actually use this .mc file.
The SMTP_LOGIN_MSG simply (re)defines the message which sendmail outputs when
a connection is made to its TCP port (usually tcp port 25).
Otherwise, this file is very similar to the one used on my private host, xibalba.
The nullclient feature is used and points to the local email server.
(Note that csel is just a DNS 'CNAME' for nago - which you can verify with
the 'nslookup' command.)
Now, let's go through the process of setting up the saclass-2 machine
as a sendmail server. The first thing to do is make sure sendmail will
be started properly at boot. To do this, I simply edit the /etc/rc.conf
file and verify that the following line is present in the file:
saclass-2 % grep sendmail /etc/rc.conf
sendmail_flags="-bd -q30m" # for normal use: "-bd -q30m"
Now I change to the m4 cf/cf directory and copy the 'BSD prototype file:
saclass-2 % cd /usr/share/sendmail/cf/
saclass-2 % sudo cp openbsd-proto.mc saclass-2.mc
saclass-2 % cat saclass-2.mc
divert(-1)
#
# Copyright (c) 1998 Sendmail, Inc. All rights reserved.
# Copyright (c) 1983 Eric P. Allman. All rights reserved.
# Copyright (c) 1988, 1993
# The Regents of the University of California. All rights reserved.
#
# By using this file, you agree to the terms and conditions set
# forth in the LICENSE file which can be found at the top level of
# the sendmail distribution.
#
#
#
# This is the prototype file for a configuration that supports nothing
# but basic SMTP connections via TCP.
#
divert(0)dnl
include(`../m4/cf.m4')
VERSIONID(`@(#)openbsd-proto.mc $Revision: 1.4 $')
OSTYPE(openbsd)
FEATURE(nouucp)
MAILER(local)
MAILER(smtp)
dnl
dnl Enforce valid Message-Id to help stop spammers
dnl
LOCAL_RULESETS
HMessage-Id: $>CheckMessageId
SCheckMessageId
R< $+ @ $+ > $@ OK
R$* $#error $: 553 Header Error
Then we need to edit the .mc file and add some necessary features and
definitions. Here are the differences between the changed saclass-2.mc
file and the original openbsd-proto.mc file:
saclass-2 % diff openbsd-proto.mc saclass-2.mc
23a24,31
> MASQUERADE_AS(saclass-2.cs.colorado.edu)
> FEATURE(`masquerade_envelope')
> define(`confPRIVACY_FLAGS', `needmailhelo, noexpn, novrfy, authwarnings')
> FEATURE(`always_add_domain')
> FEATURE(`access_db', `hash -o /etc/mail/access')
> define(`ALIAS_FILE',`/etc/mail/aliases,/etc/mail/majordomo')
> define(`confTRUSTED_USERS', `mjrdomo')
> FEATURE(rbl)
After adding in the macros we want, we need to make a .cf file from
the .mc file and copy it into place in /etc:
saclass-2 % sudo make saclass-2.cf
rm -f saclass-2.cf
(cd /usr/share/sendmail/cf && m4 saclass-2.mc > /usr/share/sendmail/cf/saclass-2.cf)
chmod 444 saclass-2.cf
saclass-2 % sudo cp saclass-2.cf /etc/sendmail.cf
Finally, we either restart sendmail if it was running already, or start it fresh.
In this case we will be restarting it:
saclass-2 % ps auxw | grep sendmail
tor 10823 0.0 0.0 540 0 p0 RV 5:24AM 0:00.00 grep sendmail (tcsh)
root 26535 0.0 0.5 548 468 ?? Ss Fri07AM 0:02.71 sendmail: accepting connections
on port 25 (sendmail)
saclass-2 % sudo kill -1 26535
To be sure sendmail got restarted, you can look at the logfile /var/log/maillog
(look in the /etc/syslog.conf file to know when sendmail logs to on your system):
saclass-2 % tail -f /var/log/maillog
Apr 10 05:24:28 saclass-2 sendmail[26535]: restarting /usr/sbin/sendmail on signal
Apr 10 05:24:28 saclass-2 sendmail[22443]: starting daemon (8.9.3): SMTP+queueing@00:30:00
Now, to verify that all is well, I will try and send and then receive some email
on saclass-2. The verification is the actual receiving of the email, and you
can tell what machine have touch the mail by looking at its Received header
fields. You can also look at the maillog for informational messages about
sendmail's delivery of your mail. Here is the log for the delivery:
Apr 10 05:29:17 saclass-2 sendmail[16210]: FAA16210: from=tor, size=82, class=0, pri=30082, \
nrcpts=1, msgid=<200004101229.FAA16210@saclass-2.cs.colorado.edu>, relay=tor@localhost
Apr 10 05:29:48 saclass-2 sendmail[28868]: FAA16210: to=tor@cs.colorado.edu, ctladdr=tor \
(1000/1000), delay=00:00:31, xdelay=00:00:31, mailer=esmtp, relay=cs.colorado.edu. \
[128.138.243.151], stat=Sent (KAA17679 Message accepted for delivery)
And here is the received mail:
Return-Path: tor@saclass-2.cs.colorado.edu
Received: from mroe.cs.colorado.edu (mroe.cs.colorado.edu [128.138.243.151])
by suod.cs.colorado.edu (8.9.3/8.9.2) with ESMTP id KAA20453
for <tor@suod.cs.Colorado.EDU>; Mon, 10 Apr 2000 10:51:53 -0600 (MDT)
Received: from saclass-2.cs.colorado.edu (saclass-2.cs.colorado.edu [128.138.192.21])
by mroe.cs.colorado.edu (8.9.3/8.9.2) with ESMTP id KAA17679
for <tor@cs.colorado.edu>; Mon, 10 Apr 2000 10:51:53 -0600 (MDT)
Received: (from tor@localhost)
by saclass-2.cs.colorado.edu (8.9.3/8.9.3) id FAA16210
for tor@cs.colorado.edu; Mon, 10 Apr 2000 05:29:17 -0700 (MST)
Date: Mon, 10 Apr 2000 05:29:17 -0700 (MST)
From: Torleif Mohling <tor@saclass-2.cs.colorado.edu>
Message-Id: <200004101229.FAA16210@saclass-2.cs.colorado.edu>
To: tor@cs.colorado.edu
Subject: test message from saclass-2
cross your fingers!
Success! Now I need to make sure that the reverse works (i.e. that saclass-2 will
also accept mail for local delivery). To do this, I'll simply reply to the first
message... Doh! That didn't work. The problem is that SMTP is being blocked on
the department firewall, except to specific email servers. So instead what I'll
do to test the setup is simply send mail from another saclass machine.
Here is the received email:
From tor@saclass-1.cs.colorado.edu Mon Apr 10 05:48:42 2000
Received: from saclass-1.cs.colorado.edu (IDENT:tor@saclass-1.cs.colorado.edu [128.138.192.20])
by saclass-2.cs.colorado.edu (8.9.3/8.9.3) with ESMTP id FAA10464
for <tor@saclass-2.cs.colorado.edu>; Mon, 10 Apr 2000 05:48:41 -0700 (MST)
Received: (from tor@localhost)
by saclass-1.cs.colorado.edu (8.9.3/8.9.3) id LAA11217
for tor@saclass-2; Mon, 10 Apr 2000 11:00:09 -0600 (MDT)
Date: Mon, 10 Apr 2000 11:00:09 -0600 (MDT)
From: Torleif Mohling <tor@saclass-1.cs.colorado.edu>
Message-Id: <200004101700.LAA11217@saclass-1.cs.colorado.edu>
To: tor@saclass-2.cs.colorado.edu
Subject: test msg from sa-1 to sa-2
sigh
And here is the excerpt from the logfile:
Apr 10 05:48:41 saclass-2 sendmail[10464]: FAA10464: from=<tor@saclass-1.cs.colorado.edu>, \
size=381, class=0, pri=30381, nrcpts=1, msgid=<200004101700.LAA11217@saclass-1.cs.colorado.edu>,\
proto=ESMTP, relay=IDENT:tor@saclass-1.cs.colorado.edu [128.138.192.20]
Apr 10 05:48:42 saclass-2 sendmail[10147]: FAA10464: to=<tor@saclass-2.cs.colorado.edu>, \
delay=00:00:01, xdelay=00:00:00, mailer=local, stat=Sent
MH is another mail reader, and one I recommend. In the labs it is accessible by adding /tools/cs/mh/bin/ to your $path. The concept behind MH is different from most mail-readers you may be familiar with in that it consists of a bunch a different programs that are all used from the Shell command-line; versus the traditional mail-readers which are monolithic programs. MH was orginally written at the Rand Corporation. A modern replacement is available called nmh.
On an OpenBSD system you can install the nmh package off your CD media or you can install via the ports collection. I will install it on saclass-2 out of the ports collection:
saclass-2 % cd /usr/ports/mail/nmh
saclass-2 % sudo make
>> nmh-1.0.tar.gz doesn't seem to exist on this system.
>> Attempting to fetch from ftp://ftp.math.gatech.edu/pub/nmh/.
...
saclass-2 % sudo make install
...
Each mail message in MH is stored in its own unix file. Messages are kept in unix directories called folders and the filename for each message is a monotonically increasing sequence number in the folder. Traditionally, the mail folders are stored in a directory called Mail that lives just below your homedir (i.e. ~/Mail) although this is configurable (as is just about every aspect of MH) through the use of the ~/.mh_profile config file. Here are some of the features I set in my own .mh_profile:
Aliasfile: my-aliases
Path: Mail
Editor: nvi
Msg-Protect: 600
Folder-Protect: 700
Draft-Folder: +drafts
To change the directory where your folders are kept you would change
the Path 'component' to be the directory you wanted (relative to your
homedir). Other files referenced in the .mh_profile are relative
to the Path component (e.g. the Aliasfile component designates
a file of personal email aliases - in this case my-aliases that
lives in the Path directory).
The Editor component specifies what editor you wish to use when you
compose, reply to or forward email. If it is not set, the environment
variable EDITOR is consulted.
The two -Protect components specify the file permissions that folders
and email files receive when they are created.
And the Draft-Folder folder specifies a folder where temporary
messages are kept if you quit an editor session with sending a message.
The first thing I like to set up is the automatic filing of various mail into specific folders. This requires two things, the first of which is to create a .forward file. My .forward file looks like this:
suod % cat .forward
"| /tools/cs/mh/lib/slocal -user tor"
suod %
The file consists of a single line that pipes each incoming email
to the MH program called slocal (short for "special local mail
delivery"). The second thing you need to create is a .maildelivery
file to specify how email should be placed into your folders.
The basic syntax of each line in the .maildelivery file is
as follows:
header pattern action result string
The first item is the email header field that will be used to match on.
The second item is the pattern that triggers a successful match.
The third item is the action to be taken if a match is successful.
Possible actions are:
destroy this obviously gets rid of the message
file or '>' causes the message to be appended to a file
(the *string* is the filename)
mbox similar to 'file' but uses traditional 'mbox' format
between each message.
pipe or '|' cause the message to be sent as STDIN to a program
(specified as the *string)*
The forth item, result, specified how the action should be
performed. Here are the possible values:
A Perform the action. If the action succeeds,
then the message is considered delivered.
R Perform the action. Regardless of the out-
come of the action, the message is not con-
sidered delivered.
? Perform the action only if the message has
not been delivered. If the action succeeds,
then the message is considered delivered.
N Perform the action only if the message has
not been delivered and the previous action
succeeded. If this action succeeds, then the
message is considered delivered.
Finally, here are some of the lines in my own .maildelivery file.
These rules cause email from certain mailing lists to get placed
into the appropriate folders:
Subject S-x86 | ? "/tools/cs/mh/lib/rcvstore +sun-x86"
To solaris-x86 | ? "/tools/cs/mh/lib/rcvstore +sun-x86"
cc solaris-x86 | ? "/tools/cs/mh/lib/rcvstore +sun-x86"
To ses_systemadmin | ? "/tools/cs/mh/lib/rcvstore +sunskl"
cc ses_systemadmin | ? "/tools/cs/mh/lib/rcvstore +sunskl"
There are lots more rules that look very similar to the one just above.
Each of them for a different mailing list. The very last rule in
the file is the only one that differs, and it simply acts
as a default catch-all to place everything else into my inbox:
default - | ? "/tools/cs/mh/lib/rcvstore +inbox"
The scan command is used to list messages in a folder.
Folders are always specified with a + (plus sign) in front of their name.
You can also give qualitative message number specifiers and ranges.
For example:
suod % scan +inbox last:5
2100 04/10 "Todd C. Miller" downtime for nag, nagina, and taussky<<The follow
2101 04/10 "Todd C. Miller" wizzy graphics card<<I got it to work after all b
2102+-04/10 "Todd C. Miller" fyi: coatl's only nameserver in resolv.conf is mr
2103 04/10 "Louie The Liquid HITECHCAFE.COM INSIDER - HEWLETT PACKARD SYSTEMS
2104 04/11 "Ryan Lynch" who-do-what for saclass-8<<Hi Tor, Here's a list
suod % scan 2000-2005
2000 -02/27 Ted Chiang VI<<HI Tor, I remember you said something about a
2001 -02/28 Walker Jason G Re: ssh questions<<Hi Tor- I'm able to login to t
2002 02/28 CS Operators <<CONFIG = cs matkat device dsk/c0t1d0s2 does not
2003 02/28 To:Walker Jason G Re: ssh questions<<> Hi Tor- > I'm able to login
2004 02/28 Therese Hofheins- Re: pub-night: pub night at the OLI<<Therese Hofh
2005 02/28 Tor Mohling csci4113 - NOTE important TYPO<<I am enjoying thi
MH maintains context about what your current folder and message numbers are
in a file called ~/Mail/context. The effect of this is that subsequent
MH commands operate on the most recently specified folder and message (this
can zap you sometimes if you are using multiple windows!)
To read messages you use the show command, which will use the 'pager' of your choice. You set what pager will be used in the .mh_profile:
showproc: less
(I use the command less for my pager - it is like more - more or less.)
To send email you use the comp command (for 'compose'). You can specify exactly what email header fields will be included in your email by default in the file ~/Mail/components. My 'components' file looks like this:
suod % cat ~/Mail/components
To:
cc:
Subject:
Fcc: +outbox
X-url: http://www.cs.colorado.edu/~tor
--------
suod %
When you run the comp command you are thrown into the editor of
your choosing with the contents of the components file present
as a default template. You are free to add email headers as you desire
just keep them all snug together at the top of the file and separated
from the body of the email message by a line of hyphens: --------
MH has a bazillion features; definately take a look throught some of the manpages! There are hooks to facilitate automatic viewing of most kinds of MIME attachments as well.
The mh(1) manpage is a great place to start. There are also pages for each individual command as well as for some of the files like .mh_profile.