When a message is stored in the queue, it is split into pieces.
Each of those pieces is stored as a separate file in the
queue directory. That is, the header and other information about
the message are stored in one file, while the body (the data)
is stored in another.
All told, six different types of files may appear in the queue directory.
The type of each is denoted by the first two
letters of the filenames. Each filename begins with a single letter
followed by an f
character. The complete list is shown in
Table 23.1.
File | Description | |
---|---|---|
df | Section 23.2.2, "The Data (Message Body) File: df" | Data (message body) |
lf | Section 23.2.3, "The Lock File (obsolete as of V5.62): lf" | Lock file (obsolete as of V5.62) |
nf | Section 23.2.4, "The ID Creation File (obsolete as of V5.62): nf" | ID creation file (obsolete as of V5.62) |
tf | Section 23.2.6, "The Temporary qf Rewrite Image: tf" | Temporary qf rewrite image |
xf | Section 23.2.7, "The Transcript File: xf" | Transcript file |
qf | Section 23.2.5, "The Queue Control File: qf" | Queue control file (and headers) |
The complete form for each filename is
Xfident
The X
is one of the leading letters shown in
Table 23.1.
The f
is the constant letter f
.
The ident
is a unique queue
identifier associated with each mail message.
In the following sections we first describe the identifier
that is common to all the queue file parts, then describe each file type in alphabetical order.
The internal details of the qf
file can vary depending on the
version of sendmail, so it is discussed separately
at the end of this chapter.
To ensure that new filenames are not the same as the names of
files that may already be in the queue, sendmail uses the
following pattern for each new ident
:
AApid
prior to V8.6hour
AApid
beginning with V8.6
Here, pid
is the process identification number of the incarnation
of sendmail that is trying to create the file. Because sendmail
often fork(2)'s to process the queue, that pid
is likely to be unique, resulting in a unique ident
.
For V8 sendmail an extra letter prefixes the AA
. Shown
as hour
, it is an uppercase letter that corresponds to
the hour (in a 24-hour clock) that the identifier was created.
For example, a file created in hour three of the day will
have a D
prefixed (the
hour begins at midnight with A
).
[2]
[2] Programs should not depend on the lead letter actually encoding the hour. It is intended only to ensure that all identifiers be unique within any 24-hour period and as an aid to scripts that need to extract information from log files.
If sendmail cannot create an exclusive filename
(because a file with that identifier already exists), it clocks the
second A
of the AA
to a B
and tries again.
It continues this process, clocking the right-hand letter
from A
to Z
and the left-hand letter
from A
to ~
until it succeeds:
AA start AB second try AC third try ... and so on ~W ~X ~Y last try ~Z failure
If it
never succeeds, the ident
ultimately looks like the following
and sendmail has failed:
hour
~Zpid
This ident
is unlikely to ever appear, because the clocking provides
for over 1600 possibilities.
All the files associated with a given mail message share the same
ident
as a part of their filenames. The individual files
associated with a single mail message differ only in the first
letter of their names.
All mail messages are composed of a header and a body. When queued, the body is stored in the df file.
Traditionally, the message body could contain only characters that
had the high (most significant) bit turned off (cleared,
set to 0).
But under V8 sendmail, with a version 2 or higher configuration
file (see Section 27.5, "The V Configuration Command"),
the high bit is left as is until delivery (whereupon the F=7
delivery-agent
flag, see Section 30.8.4, F=7,
determines whether or not that bit will be stripped during delivery).
Because the message body can contain sensitive or personal information,
the df
file should be protected from reading by ordinary
users. If the queue directory is world readable, then the
TempFileMode
(F
) option (see Section 34.8.68, TempFileMode (F))
should specify minimum permissions (such as 0600) for queued files.
But if the queue directory is protected
by both narrow permissions and a secure machine, the
TempFileMode
(F
) option may be relaxed for easier administration.
There is currently no plan to provide for encryption of df
files. If you are concerned about the privacy of your message,
you should use an end-to-end encryption package (not discussed in
this book).
When old versions of sendmail process a queued message (attempt
to redeliver it)
they create an empty lock file. That lock file is necessary to
signal to other running sendmail processes that the mail message
is busy so that they won't try to deliver the message too.
Current versions simply flock(2) or fcntl(2) the qf
file.
The method that sendmail uses to initially create an exclusive lock when first queueing a file is twofold. First it attempts to creat(2) the file with the argument
O_CREAT|O_WRONLY|O_EXCL
If that succeeds,
it then attempts to lock the file. If HASFLOCK (see Section 18.8.9, HAS...)
is defined
when sendmail is compiled, the file is locked with flock(2).
Otherwise, it is locked with a fcntl(2)
F_SETLK
argument.
When mailq is run (or the -bp
command-line switch is
given to sendmail), the contents of the queue are listed.
In that listing, an asterisk that appears to the right of an identifier
indicates that a lock exists on the message:
Mail Queue (1 request) -Q-ID- -Size- --Q-Time--- ------Sender/Recipient------ MAA17445* 126 Fri Apr 17 10:17 <gw@wash.dc.gov> <ben@franklin.edu> note
Occasionally, a file will become locked and remain that way for a long time. One indication of a stuck lock is a series of syslog messages about a given identifier:
Apr 12 00:33:38 ourhost sendmail[641]: AA00614: locked Apr 12 01:22:14 ourhost sendmail[976]: AA00614: locked Apr 12 02:49:23 ourhost sendmail[3251]: AA00976: locked Apr 12 02:49:51 ourhost sendmail[5977]: AA00614: locked Apr 12 03:53:05 ourhost sendmail[9839]: AA00614: locked
An occasional lock message, such as AA00976
in the third
line above, is normal. But when an identifier is continually reporting
as locked (like the AA00614
lines), an orphaned lock
may exist and should be investigated.
Use ps(1) to look for lines that list queue file identifiers:
root 5338 160 -AA00614 To wash.dc.gov (sendmail)
This shows that the queued mail message, whose identifier is
AA00614
, is currently being processed. It the lock
on that file is stuck, consider killing the sendmail that
is processing it.
Old versions of sendmail used an nf
file when creating
a message identifier to avoid race conditions.
[3]
But contemporary
versions of sendmail create the queue identifier when first
creating the qf
file. The nf
file is obsolete.
[3] Historical footnote: This stems from the days when the only atomic file-system call was link(2).
A queued mail message is composed of two primary parts. The df
file contains the message body. The qf
file contains the message
header.
In addition to the header, the qf
file also contains all the
information necessary to:
Deliver the message. It contains the sender's address and a list of recipient addresses.
Order message processing. It contains a priority that determines the current message's position in a queue run of many messages.
Expire the message. It contains the date that the message was originally queued. That date is used to time out a message.
Explain the message. It contains the reason that the message is in the queue and possibly the error that caused it to be queued.
The qf
file is line-oriented, with one item of information per
line. Each line begins with a single uppercase character (the
code letter), which
specifies the contents of the line. Each code letter is then followed
by the information appropriate to the letter.
The code letters and their meanings are shown in
Table 23.2 of Section 23.9, "The qf File Internals".
Here is an example of a version 1 (for V8.8 sendmail)
qf
file:
V1 T826845694 K0 N0 P30016 I7/4/20 $_you@localhost MDeferred: Host wash.dc.gov is down Syou@your.domain RPFD:george@wash.dc.gov RPFD:jefferson H?P?Return-Path: you@your.domain HReceived: (from you@your.domain) by your.domain (8.8.4/8.8.4) id QAA06571 for george@wash.dc.gov; Thu, 14 Mar 1996 16:21:34 -0700 (MST) H?D?Date: Thu, 14 Mar 1996 16:21:34 -0700 (MST) H?F?From: Your Name <you@your.domain> H?x?Full-Name: Your Name H?M?Message-Id: <199603142321.QAA06571@your.domain> HSubject: foo .
This fictional qf
file shows the information that will be
used to send a mail message from you@your.domain
(the S
line)
to two recipients: george@wash.dc.gov
and jefferson
(the R
lines).
It also shows the various headers that appear in that message (the
H
lines).
We discuss the individual lines of the qf
file in
at the end of this chapter.
When processing a queued message, it is often necessary for
sendmail to modify the contents of the qf
file.
This usually occurs if delivery has failed or if
delivery for only a part of the recipient
list succeeded. In either event, at least the message
priority needs to be incremented.
To prevent damage to the original qf
file, sendmail makes
changes to a temporary copy of that file. The temporary copy has
the same queue identifier as the original, but its name begins
with a t
.
After the tf
file has been successfully written and closed,
sendmail calls rename(2) to replace the original with
the copy. If the renaming fails, sendmail
syslog(3)'s at LOG_CRIT a message like the following:
cannot rename(tfAA00000, qfAA00000), df=dfAA00000
Failure to rename is an unusual, but serious, problem:
A queued message
has been processed, but its qf
file contains old and incorrect
information. This failure may, for example,
indicate a hardware error, a corrupted queue directory, or that the
system administrator accidentally removed the queue directory.
A given mail message may be destined for many recipients, requiring different delivery agents. During the process of delivery, error messages (such as "User unknown" and "Permission denied") can be printed back to sendmail by each delivery agent.
While calling the necessary delivery agents, sendmail
saves all the error messages it receives in a temporary file.
The name of that temporary file begins with the letters xf
.
After all delivery agents have been called, sendmail returns any collected
error messages to the sender and deletes the temporary xf
file.
If there are no errors, the empty xf
file is silently deleted.
The -d51.104
debugging switch (see Section 37.5.171, -d51.104)
can be used to prevent deletion of the xf
file.