21.2.1 Determine the Local Canonical Name

All versions of sendmail use more or less the same logical process to obtain the canonical name of the local host. As illustrated in the sample program below, sendmail first calls gethostname(3) to obtain the local host's name. That name may either be a short name or a fully qualified one depending on which comes first in the /etc/hosts file. If the call to gethostname(3) fails, the name of the local host is set to localhost:

#include <sys/types.h>
#include <sys/socket.h>
#include <sys/param.h>
#include <netdb.h>
#include <stdio.h>

main()
{
        char hostbuf[MAXHOSTNAMELEN];
        struct hostent *hp;

        /* Get the local hostname */
        if (gethostname(hostbuf, sizeof(hostbuf)) < 0)
                strcpy(hostbuf, "localhost");
        printf("hostname = \"%s\"\n", hostbuf);

        /* canonicalize it and get aliases */
        if((hp = gethostbyname(hostbuf)) == NULL)
                perror("gethostbyname"), exit(2);
        printf("canonical = \"%s\"\n", hp->h_name);
        while (*hp->h_aliases != NULL)
        {
                printf("alias: \"%s\"\n", *hp->h_aliases);
                ++hp->h_aliases;
        }
}

The local hostname is then given to the gethostbyname routine (see Section 37.5.186, -d61.10) to obtain the canonical name for the local host. That same routine also returns any aliases (other names for the local host).

On some Sun and Ultrix machines that are set up to use NIS services, the canonical name is the short name, and a fully qualified name that should have been the canonical name appears as an alias. For such systems you must link with the BIND library (libresolv.a) when compiling this program or compiling sendmail. That library gets its information from DNS rather than from NIS. But note that V8.7 and above versions of sendmail do the intelligent thing and use the canonical name that was found as the aliases if it exists.

If a good BIND library is not available, or if it is not convenient to compile and install a new version of sendmail, you can circumvent the short name assigned to $j by defining $j like this:

Dmyour domain here
Dj$w.$m

The canonical name is your site's hostname with a dot and your domain name appended. These two lines cause $j to have your host's fully qualified (and canonical) name assigned to it.

The canonical name found by gethostbyname(3) is assigned as the value of the $w macro. The short name and any aliases are added to the class $=w.

The result of all these lookups can be viewed by running sendmail with a -d0.4 debugging switch (see Section 37.5.2, -d0.4). The actual DNS lookups can be watched with the -d8.8 debugging switch (see Section 37.5.35, -d8.8).

21.2.2 Look Up a Remote Host's Name

When sendmail begins to run as a daemon, it creates a socket, binds to that socket, and listens for incoming SMTP connections. When a remote host connects to the local host, sendmail uses the accept(2) library routine to accept the connection. The accept(2) routine provides the IP address of the remote machine to sendmail. The sendmail program then calls gethostbyaddr(2) to convert that IP address to a canonical (official) hostname.

The sendmail program needs the canonical hostname for four reasons:

• The remote hostname is compared to the local hostname to prevent sendmail from connecting to itself.

• The remote hostname claimed in the HELO SMTP line is compared to the canonical name. If they differ, sendmail complains.

• The macro $s is assigned the canonical hostname as its value.

• The canonical name is included in many log messages produced by the setting of the LogLevel (L) option (see Section 34.8.33, LogLevel (L)) and is available for inclusion in Received: header (see Section 35.10.25, Received:) lines.

If the Timeout.ident (r) option (see Section 34.8.70.10, "Timeout.ident") is greater than zero, the local host also connects to the identd(8) daemon at the sending host to discover who opened the connection. If available, that user and host information is assigned to the $_ macro (see Section 31.10.1, $-).

21.2.3 Look Up Addresses for Delivery

When sendmail prepares to connect to a remote host for transfer of mail, it first performs a series of checks that vary from version to version. All versions accept an IP address surrounded with square brackets as a literal address and use it as is.

Beginning with V8.1, sendmail checks to see whether the host part of the address is surrounded with square brackets. If so, it skips looking up MX records. (We'll elaborate on MX records soon.)

Beginning with V8.8, sendmail first checks to see whether the F=0 flag (see Section 30.8.1, F=0) is set for the selected delivery agent. If it is set, sendmail skips looking up MX records.

If sendmail is allowed to look up MX records, it calls the res_search(3) BIND library routine [9] to find all the MX records for the host. If it finds any MX records, it sorts them in order of cost, selecting the least cost first. If V8 sendmail finds two costs that are the same, it randomizes the selection between the two when sorting. [10] After all MX records are found, or if no MX records were found, sendmail adds the host specified by the FallbackMXhost (V) option (see Section 34.8.25, FallbackMXhost (V)), if there was one, to that list.

[9] If the ServiceSwitchFile option (see Section 34.8.61, ServiceSwitchFile) lists a file that defines hosts as being looked up with NIS, all DNS lookups are skipped.

[10] Note that this is broken in many older versions of sendmail. Also note that when the MX record points to the local host, all MX records with a cost greater than the local host are tossed. (See Section 31.10.40 for a description of this process.)

The sendmail program then tries to deliver the message to each host in the list of MX hosts, one at a time, until one of them succeeds or until they all fail. Beginning with V8.8 sendmail, any host in the list that returns a 5xy SMTP code (permanent failure) causes all subsequent MX hosts to be ignored (but temporary and connect failures continue to the next MX host as usual).

If no MX records are found, sendmail tries to deliver the message to the single original host. If all else fails, sendmail attempts to deliver to the host listed with the FallbackMXhost (V) option.

Whether sendmail tries to connect to the original host or to a list of MX hosts, it calls gethostbyname(2) to get the network address for each. It then opens a network connection to that address and attempts to send SMTP mail.

21.2.4 The $[ and $] Operators

The $[ and $] operators (see Section 28.6.6, "Canonicalize Hostname: $[ and $]") are used to canonicalize a hostname. Here is a simplified description of the process.

Each lookup is actually composed of many lookups that occur in the form of a loop within a loop. In the outermost loop, the following logic is used:

• If the address has at least one dot somewhere in it, sendmail looks up that address unmodified first.

• If the unmodified address is not found and the RES_DNSRCH bit is set (see the ResolverOptions (I) option, Section 34.8.55), sendmail looks up variations on the domain part of the address. The default domain is tried first (for a host in the sub subdomain at dc.gov, that would be sub.dc.gov, thus looking up host.sub.dc.gov). If that fails, BIND 4.8 then throws away the lowest part of the domain and tries again (looks up host.dc.gov). BIND 4.9 uses the search attribute, if given, and tries that list of possible domains.

• If the address has no dots and the RES_DEFNAMES bit is set (see the ResolverOptions (I) option, Section 34.8.55), sendmail tries the single default domain (looks up host.sub.dc.gov). This is for compatibility with older versions of DNS.

Each lookup described above is performed by using the following steps:

• Try the hostname with a T_ANY query that requests all the cached DNS records for that host. If it succeeds, A records and/or MX records may be among those returned. However, success is not guaranteed, because sometimes only NS (name server) records are returned. In that instance the following steps are also taken.

• Try the hostname with a T_A query that requests the A record for that host.

• Try the hostname with a T_MX query that requests MX records for the host.

Each query searches the data returned as follows:

• Search for a CNAME (alias) record. If one is found, replace the initial hostname (the alias) with the canonical name returned and start over.

• Search for an A record (the IP address). If one is found, the hostname that was just used to query is considered the canonical address.

• Search for an MX record. If one is found and a default domain has not been added, treat the MX record like an A record. For example, if the input hostname is sub.dc.gov and an MX record is found the MX record is considered official. If, on the other hand, the input hostname has no domain added (is sub) and the query happens to stumble across sub.dc.gov as the MX record, the following searches are also tried.

• If an MX record is found and no MX record has been previously found, the looked-up hostname is saved for future use. For example, if the query was for sub.dc.gov and two MX records were returned (hostA.sub.dc.gov and hostB.sub.dc.gov), sub.dc.gov is saved for future use.

• If no MX record is found, but one was found previously, the previous one is used. This assumes that the search is normally from most to least complex (sub.sub.dc.gov, sub.dc.gov, dc.gov).

All this apparent complexity is necessary to deal with wildcard MX records (see Section 21.3.4, "Wildcard MX Records") in a reasonable and successful way.