LDAP and email authentication

Background

These notes were inspired by one department's experiences with using LDAP for email authentication at the University of Arizona, and is very much a work in progress, revised as we update and revise a particular configuration. It might be worth asking the people involved with the central University email system how they've been approaching things, because there will be some general similarities, but different particulars (I think they're entirely based on Solaris and Postfix, but the details here are specific to Linux, OpenLDAP, and Sendmail).

The Tree-Ring Lab uses its own servers for both incoming and outgoing email. We've always had strict limitations on relaying mail through the outgoing servers, requiring some kind of authentication if the connection didn't seem to originate from three specific subnets. Both Sendmail and the IMAP server use the same freely available version of SASL for authentication, Cyrus SASL from CMU; this can store usernames and passwords in its own database, but this is just one of a whole series of possible configurations, which include at least two ways it can use an LDAP directory. We've been using LDAP for all the user information for our Samba servers since before TNG split off, so the idea of having a single LDAP directory for everything was attractive.

Our IMAP server was initially getting its own replica of the LDAP directory & using it for part of the user & group information (a typical nss-ldap + pam-ldap configuration). The Cyrus IMAP software worked immediately, with no special configuration for LDAP, but I didn't like the results. Most of the email clients people were using could only handle plain-text passwords, and didn't use SSL or TLS to protect their connection to the server, and for years we'd been going through elaborate contortions to avoid this kind of thing (one time passwords for all Telnet & FTP). So initially we ended up with an email-specific password database on the server (the passwords distinct from those used for any kind of shell account). Later, when configuring Sendmail for authentication had become a more pressing issue, we added the email-specific passwords to the LDAP directory, but under a distinct subtree.

SMTP authentication

Spammers hunt the Internet for open relays, email systems that will re-transmit their spam to arbitrary addresses. If you don't restrict your email system, it will be easily discovered and soon used to spew forth large volumes of commercial or other offensive email. You will receive many complaints, and your address will find its way onto blacklists, so your legitimate users no longer receive their mail.

The restriction can be to only pass outgoing email that originates from the email system itself, or only from a few subnets (the address ranges of your departmental LAN), but this is inconvenient for people connecting from home over dynamically-addressed dial-up connections, or for people who are traveling. Remote users can log on to an interactive shell on the email system, or use webmail based there, but many people prefer to use the same email user agent (Eudora, Outlook, etc.) from home or on the road that they also use on the University campus.

Authentication is the most obvious general mechanism for allowing people restricted access to an email relay, exactly as in the case of other restricted network services. There is a recognized standard for how this gets done, described in RFC2554. The RFC gives several examples of how the authentication exchanges should proceed, and you can check to see if a particular mail server offers authentication by examining what it advertises when you start a connection by telnetting to port 25 (although this may not reveal the full range of potential authentication mechanisms). For example it's clear that the central University email system will handle authentication.

$ telnet smtpgate.email.arizona.edu 25
Trying 128.196.133.161...
Connected to smtpgate.email.arizona.edu.
Escape character is '^]'.
220 deimos.email.Arizona.EDU ESMTP Service (6.7.016) ready
EHLO quen.ltrr.arizona.edu
250-deimos.email.Arizona.EDU
250-DSN
250-8BITMIME
250-PIPELINING
250-HELP
250-AUTH=LOGIN
250-AUTH LOGIN DIGEST-MD5
250-STARTTLS
250-DELIVERBY 300
250 SIZE 52428800
QUIT
221 deimos.email.Arizona.EDU QUIT
Connection closed by foreign host.

SASL

The RFC2554 is quite short, because most of the details depend on SASL, a generalized framework for adding authentication to communications protocols. Rather than having to design one set of authentication mechanisms for SMTP, another for IMAP, and so on, the hope is that protocol designers can avoid re-inventing the wheel by using a common framework for many different protocols; significantly from our point of view, SASL authentication is a required part of the standard for version 3 of the LDAP protocol. Moreover it's intended to be modular, so that the authentication can be based on mechanisms that already work: password files, Kerberos, one-time passwords, and so on. This flexibility can be a source of confusion, however, particularly on a Unix system where it is combined with several other flexible, modular components. Some systems will have multiple nsswitch methods for looking up names; Solaris and Linux support multiple pluggable authentication modules (PAM), and there are equivalents for other operating systems. The interactions between these can become quite complex: so a SASL-aware network protocol might invoke a mechanism that exercised a PAM module, which in turn (via a particular nsswitch lookup configuration) caused a search in a LDAP directory. SASL is a
framework linking multiple protocols with multiple authentication mechanisms.

SASL is itself standardized, by not one but a whole family of RFCs. RFC2222 is the main SASL specification (now under revision as draft-ietf-sasl-rfc2222bis, then specific mechanisms are covered in additional RFCs, such as RFC2831 (the DIGEST-MD5 mechanism, currently under revision as draft-ietf-sasl-rfc2831bis), RFC2444 (one-time passwords). The standards don't specify implementation details, and in particular the specification of an API for a standard SASL library is still only an early draft, draft-newman-sasl-c-api. But there is one important free implementation, Cyrus SASL, from the same project at Carnegie-Mellon University that developed the Cyrus IMAP server. The author of many of the RFCs, Chris Newman, used to be associated with the group, then worked for a company that was assimilated into iPlanet (or whatever that division of Sun is calling itself this week). People keep complaining about the quality of the documentation, but the distribution contains three or four short HTML manuals, and pointers to additional sources of information, such as the articles on SASL in general and the Cyrus SASL implementation by Marshall Rose (of The Simple Book fame).

In theory there's considerable overlap between SASL and transport layer security mechanisms such as IPsec and TLS, since the "SL" in SASL stands for Security Layer, and the standard mentions that it can help negotiate this (e.g., encrypting all subsequent network traffic). This isn't used much in practice, and (particularly in the case of email communications) TLS tends to complement SASL, setting up a secure channel within which a possibly insecure SASL negotiation can take place. Even in cases where a secure SASL authentication mechanism is used (such as one of the digest challenge-response exchanges), the subsequent network traffic will usually be unprotected plaintext.

OpenLDAP and Cyrus SASL

The Cyrus SASL library can store authentication secrets in its own private database, and this configuration is enabled by default. However, if we're trying to use LDAP to integrate as many as possible of our system administration data stores this would be a retrogressive step. A freestanding daemon, saslauthd, offers a simple service that allows the SASL library to use other authentication mechanisms built into the system. The example shown here illustrates the hypothetical case mentioned earlier of SASL using Pluggable Authentication Modules, which in turn (via a nsswitch configuration) consult an LDAP directory. SASL using saslauthd to
get an OK or NO authentication response from LDAP via PAM.

There are several drawbacks to saslauthd. The scheme illustrated here is too complex, with a whole cascade of modules each of which provides an opportunity for misconfiguration or failure. The complexity makes it easy to leak sensitive information, for example coupling an unprotected plaintext mechanism with something much more secure, such as GSSAPI. The secrets that never left the Kerberized servers and workstations within a local area network might be exposed to sniffers in the world at large every time someone checked their email. Some improvements are possible: a contributer to the OpenLDAP and Cyrus SASL projects called Igor Brezac has written part of saslauthd that lets it query an LDAP directory directly. Brezac's code lets saslauthd query LDAP directly.

However there is a further serious problem with saslauthd. The daemon can only give two responses, OK if authentication succeeds or NO if it fails. Several of the better SASL mechanisms need access to the password directly: for most of the challenge-response mechanisms, secret information such as passwords or password equivalent hashes is known only at the end points of the authentication negotiation. The same desirable features that prevent a man-in-the-middle attack also prevent the SASL library from working with these mechanisms, since it isn't at the endpoint of the negotiation, but just a proxy to whatever saslauthd is doing. Nothing gives it a plaintext password or equivalent that it can pass on to saslauthd, and even if it did, the binary ok/no response it receives would not be adequate for the middle of a challenge-response dialog. An auxprop plugin lets the SASL library fetch passwords from LDAP.

Cyrus SASL solves this problem by what it calls Auxiliary Property (auxprop) Plugins. The documents are deliberately vague about excatly what an auxiliary property looks like, since this is another area of extensibility, but they're supposed to include plaintext passwords, or plaintext-equivalent password hashes. The SASL library's private password database is implemented through an auxprop plugin, and there are other plugins for everything from relational databases such as MySQL to LDAP. An early attempt at a fairly generic LDAP auxprop plugin was written as a patch by someone called Simon Loader. This binds to the LDAP directory, using a DN that has the privileges needed to retrieve the interesting attributes (such as userPassword). One of the OpenLDAP developers, Howard Chu, has written an experimental plug-in called ldapdb, which takes advantage of the support for SASL that's built into recent (2.1.x) versions of OpenLDAP. Because SASL is a mandatory part of LDAP version 3, there was an incentive to improve the handling of SASL authentication in OpenLDAP, and the developers have defined a mechanism for mapping SASL names onto the LDAP directory contents that's considered to be part of the server configuration.

Ultimately these plug-ins may be stopgap solutions, and some of the Cyrus SASL developers would like to see a free-standing daemon (rather like saslauthd) that would handle the fetching of the attributes, be intelligent about caching them and so on, but no one has yet offered to write it.

Caveats on SASL library usage

Unfortunately the current C API for Cyrus SASL is relatively new. Although recent programs such as the Cyrus IMAP server, current releases of Sendmail, and recent releases of OpenLDAP 2.1 all use this API (referred to as version 2), much older software continues to use the obsolete version 1 API. The improvements between the versions are not trivial, including not only much better support for memory management, but also the whole concept of auxprop plugins. Libraries implementing the old and new versions can coexist on the same system, so the obvious solution is to use the version 2 libraries with new software, and keep the version 1 libraries for old things that require them. Unfortunately dynamic linking to shared libraries makes this very difficult, particularly on a system that's using an older version of OpenLDAP. It's all too easy to end up with an application that's linked against libraries implementing both APIs simultaneously, and which will be at best highly unstable. The dependencies can run deep into fundamental system libraries and can't be controlled just by what is searched when an application is built: the Debian packager of some of the affected software ended up pleading for versioned symbols (as in Linux kernel modules) after laboring long over such a mess. Here's an example of what can happen to Sendmail.

$ ldd /usr/sbin/sendmail
        libssl.so.2 => /lib/libssl.so.2 (0x40023000)
        libcrypto.so.2 => /lib/libcrypto.so.2 (0x40051000)
        libdb-3.3.so => /lib/libdb-3.3.so (0x40115000)
        libresolv.so.2 => /lib/libresolv.so.2 (0x401a4000)
        libcrypt.so.1 => /lib/libcrypt.so.1 (0x401b5000)
        libnsl.so.1 => /lib/libnsl.so.1 (0x401e2000)
        libdl.so.2 => /lib/libdl.so.2 (0x401f7000)
        libldap.so.2 => /usr/lib/libldap.so.2 (0x401fb000)
        libsasl2.so.2 => /usr/local/lib/libsasl2.so.2 (0x40224000)
        libc.so.6 => /lib/i686/libc.so.6 (0x42000000)
        libsasl.so.7 => /usr/lib/libsasl.so.7 (0x40235000)
        liblber.so.2 => /usr/lib/liblber.so.2 (0x40240000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000)
        libgdbm.so.2 => /usr/lib/libgdbm.so.2 (0x4024a000)
        libpam.so.0 => /lib/libpam.so.0 (0x40251000)

Practicalities

We didn't want to expose valuable passwords in a way that would allow someone to harvest them from network traffic with a sniffer, and had no way of restricting our users to email clients that wouldn't use plaintext passwords over insecure connections, and hence had decided to use less valuable passwords just for email and similar applications (not so much single sign-on as dual sign-on). The accounts used by nsswitch-ldap, pam-ldap, and Samba were all defined in the usual LDAP subtree, ou=People directly under our root DN. Rather than adding an extra attribute to the account entries, we decided to put them in a completely different subtree, ou=exo. This makes it more difficult to keep all the information associated with people synchronized, but allows us to selectively replicate the email-specific information to machines that may be relatively exposed to intrusion, and don't have any reason to hold the more sensitive authentication information (the shell and Samba password hashes). The email passwords are stored in plaintext in a userPassword attribute, making it easy for some of the challenge-response mechanisms to use them. We also include attributes for email routing information, with the intention of replacing all the traditional aliases files. A typical entry (in LDIF format) would look like this...

dn: uid=hamish,ou=exo,dc=ltrr,dc=arizona,dc=edu
objectClass: top
objectClass: account
objectClass: simpleSecurityObject
objectClass: inetLocalMailRecipient
uid: hamish
userPassword: jim.v.seamus
mailLocalAddress: hamish@ltrr.arizona.edu
mailHost: schulman.ltrr.arizona.edu
mailRoutingAddress: hamish@schulman.ltrr.arizona.edu

When building Sendmail, we specified support for both SASL and the LDAP routing information in site.config.m4.

define(`confMAPDEF', `-DNEWDB -DMAP_REGEX -DLDAPMAP')
APPENDDEF(`confLIBS', `-lldap -lsasl2')
APPENDDEF(`confENVDEF', `-DSASL=2')
APPENDDEF(`conf_sendmail_ENVDEF', `-DSTARTTLS')
APPENDDEF(`conf_sendmail_LIBS', `-lssl -lcrypto')

We then added a few lines to the .mc file used to build the Sendmail configuration file (not shown here are the lines specifying where to find the SSL/TLS certificates). The local LDAP server had the hostname douglass.ltrr.arizona.edu...

define(`confLDAP_DEFAULT_SPEC',`-h douglass.ltrr.arizona.edu -d "ou=exo,dc=ltrr,dc=arizona,dc=edu"')dnl
FEATURE(`ldap_routing')dnl
LDAPROUTE_DOMAIN(`ltrr.arizona.edu')dnl

Mostly this is just a standard setup for SMTP AUTH and STARTTLS, as described by Claus Aßmann's web pages and in the documents in the Sendmail source distribution.