4. Services and Standards

The Internet Mail architecture comprises six basic types of functionality, which are arranged to support a store-and-forward service. As shown in Figure 5, each type can have multiple instances, some of which represent specialized roles. This section considers the activities and relationships among these components, and the Internet Mail standards that apply to them.

• Message

• Message User Agent (MUA)

  • Author MUA (aMUA)

  • Recipient MUA (rMUA)

• Message Submission Agent (MSA)

  • Author-focused MSA functions (aMSA)

  • MHS-focused MSA functions (hMSA)

• Message Transfer Agent (MTA)

• Message Delivery Agent (MDA)

  • Recipient-focused MDA functions (rMDA)

  • MHS-focused MDA functions (hMDA)

• Message Store (MS)

  • Author MS (aMS)

  • Recipient MS (rMS)

This figure shows function modules and the standardized protocols used between them.

                     ++========++
                     ||        ||                             +-------+
          ...........++  aMUA  ||<............................+ Disp  |
          .          ||        ||                             +-------+
          .          ++=+==+===++                                 ^
          .  local,imap}|  |{smtp,submission                      .
          .  +-----+    |  |                          +--------+  .
          .  | aMS |<---+  | ........................>| Return |  .
          .  +-----+       | .                        +--------+  .
          .                | .    *****************       ^       .
          .          +-----V-.----*------------+  *       .       .
          .      MSA | +-------+  *   +------+ |  *       .       .
          .          | | aMSA  +-(S)->| hMSA | |  *       .       .
          .          | +-------+  *   +--+---+ |  *       .       .
          V          +------------*------+-----+  *       .       .
    //==========\\                *      V {smtp  *       .       .
    || MESSAGE  ||                *   +------+    *  //===+===\\  .
    ||----------||            MHS *   | MTA  |    *  ||  dsn  ||  .
    || ENVELOPE ||                *   +--+---+    *  \\=======//  .
    ||  smtp    ||                *      V {smtp  *     ^   ^     .
    || CONTENT  ||                *   +------+    *     .   . //==+==\\
    ||  imf     ||                *   | MTA  +....*......   . || mdn ||
    ||  mime    ||                *   +--+---+    *         . \\=====//
    \\==========//                * smtp}| {local *         .     ^
          .           MDA         *      | {lmtp  *         .     .
          .      +----------------+------V-----+  *         .     .
          .      | +----------+   *   +------+ |  *         .     .
          .      | |          |   *   |      | +..*..........     .
          .      | |   rMDA   |<-(D)--+ hMDA | |  *               .
          .      | |          |   *   |      | |<.*........       .
          .      | +-+------+-+   *   +------+ |  *       .       .
          .      +------+---------*------------+  *       .       .
          .  smtp,local}|         *****************       .       .
          .             V                                 .       .
          .          +-----+                         //===+===\\  .
          .          | rMS |                         || sieve ||  .
          .          +--+--+                         \\=======//  .
          .             |{imap,pop,local                  ^       .
          .             V                                 .       .
          .       ++==========++                          .       .
          .       ||          ||                          .       .
          .......>||   rMUA   ++...........................       .
                  ||          ++...................................
                  ++==========++

    Legend: --- lines indicate primary (possibly indirect)
                transfers or roles
            === boxes indicate data objects
            ... lines indicate supporting transfers or roles
            *** lines indicate aggregated service

                     Figure 5: Protocols and Services

4.1. Message Data

The purpose of the Message Handling System (MHS) is to exchange an IMF message object among participants [RFC5322]. All of its underlying mechanisms serve to deliver that message from its Author to its Recipients. A message can be explicitly labeled as to its nature [RFC3458].

A message comprises a transit-handling envelope and the message content. The envelope contains information used by the MHS. The content is divided into a structured header and the body. The header comprises transit-handling trace information and structured fields that are part of the Author's message content. The body can be unstructured lines of text or a tree of multimedia subordinate objects, called "body-parts" or, popularly, "attachments". [RFC2045], [RFC2046], [RFC2047], [RFC4288], [RFC4289], [RFC2049].

In addition, Internet Mail has a few conventions for special control data, notably:

Delivery Status Notification (DSN):

A Delivery Status Notification (DSN) is a message that can be generated by the MHS (MSA, MTA, or MDA) and sent to the RFC5321.MailFrom address. MDA and MTA are shown as sources of DSNs in Figure 5, and the destination is shown as Returns. DSNs provide information about message transit, such as transfer errors or successful delivery [RFC3461].

Message Disposition Notification (MDN):

A Message Disposition Notification (MDN) is a message that provides information about post-delivery processing, such as indicating that the message has been displayed [RFC3798] or the form of content that can be supported [RFC3297]. It can be generated by an rMUA and is sent to the Disposition-Notification-To addresses. The mailbox for this is shown as Disp in Figure 5.

Message Filtering (SIEVE):

Sieve is a scripting language used to specify conditions for differential handling of mail, typically at the time of delivery [RFC5228]. Scripts can be conveyed in a variety of ways, such as a MIME part in a message. Figure 5 shows a Sieve script going from the rMUA to the MDA. However, filtering can be done at many different points along the transit path, and any one or more of them might be subject to Sieve directives, especially within a single ADMD. Figure 5 shows only one relationship, for (relative) simplicity.

4.1.1. Envelope

Internet Mail has a fragmented framework for transit-related handling information. Information that is used directly by the MHS is called the "envelope". It directs handling activities by the transfer service and is carried in transfer-service commands. That is, the envelope exists in the transfer protocol SMTP [RFC5321].

Trace information, such as RFC5322.Received, is recorded in the message header and is not subsequently altered [RFC5322].

4.1.2. Header Fields

Header fields are attribute name/value pairs that cover an extensible range of email-service parameters, structured user content, and user transaction meta-information. The core set of header fields is defined in [RFC5322]. It is common practice to extend this set for different applications. Procedures for registering header fields are defined in [RFC3864]. An extensive set of existing header field registrations is provided in [RFC4021].

One danger of placing additional information in header fields is that Gateways often alter or delete them.

4.1.3. Body

The body of a message might be lines of ASCII text or a hierarchically structured composition of multimedia body part attachments using MIME ([RFC2045], [RFC2046], [RFC2047], [RFC4288], and [RFC2049]).

4.1.4. Identity References in a Message

Table 1 lists the core identifiers present in a message during transit.

Layer	Field	Set By
Message Body	MIME Header	Author
Message header fields	From:	Author
	Sender:	Originator
	Reply-To:	Author
	To:, CC:, BCC:	Author
	Message-ID:	Originator
	Received:	Originator, Relay, Receiver
	Return-Path:	MDA, from MailFrom
	Resent-*:	Mediator
	List-Id:	Mediator
	List-*:	Mediator
SMTP	HELO/EHLO	Latest Relay Client
	ENVID	Originator
	MailFrom	Originator
	RcptTo	Author
	ORCPT	Originator
IP	Source Address	Latest Relay Client

Legend:

• Layer - The part of the email architecture that uses the identifier.

• Field - The protocol construct that contains the identifier.

• Set By - The Actor role responsible for specifying the identifier value (and this can be different from the Actor that performs the fill-in function for the protocol construct).

Table 1: Layered Identities

These are the most common address-related fields:

RFC5322.From: Set by - Author

Names and addresses for Authors of the message content are listed in the From: field.

RFC5322.Reply-To: Set by - Author

If a Recipient sends a reply message that would otherwise use the RFC5322.From field addresses in the original message, the addresses in the RFC5322.Reply-To field are used instead. In other words, this field overrides the From: field for responses from Recipients.

RFC5322.Sender: Set by - Originator

This field specifies the address responsible for submitting the message to the transfer service. This field can be omitted if it contains the same address as RFC5322.From. However, omitting this field does not mean that no Sender is specified; it means that that header field is virtual and that the address in the From: field is to be used.

Specification of the notifications Return Addresses, which are contained in RFC5321.MailFrom, is made by the RFC5322.Sender. Typically, the Return address is the same as the Sender address. However, some usage scenarios require it to be different.

RFC5322.To/.CC: Set by - Author

These fields specify MUA Recipient addresses. However, some or all of the addresses in these fields might not be present in the RFC5321.RcptTo commands.

The distinction between To and CC is subjective. Generally, a To addressee is considered primary and is expected to take action on the message. A CC addressee typically receives a copy as a courtesy.

RFC5322.BCC: Set by - Author

A copy of the message might be sent to an addressee whose participation is not to be disclosed to the RFC5322.To or RFC5322.CC Recipients and, usually, not to the other BCC Recipients. The BCC: header field indicates a message copy to such a Recipient. Use of this field is discussed in [RFC5322].

RFC5321.HELO/.EHLO: Set by - Originator, MSA, MTA

Any SMTP client -- including Originator, MSA, or MTA -- can specify its hosting domain identity for the SMTP HELO or EHLO command operation.

RFC3461.ENVID: Set by - Originator

The MSA can specify an opaque string, to be included in a DSN, as a means of assisting the Return Address Recipient in identifying the message that produced a DSN or message tracking.

RFC5321.MailFrom: Set by - Originator

This field is an end-to-end string that specifies an email address for receiving return control information, such as returned messages. The name of this field is misleading, because it is not required to specify either the Author or the Actor responsible for submitting the message. Rather, the Actor responsible for submission specifies the RFC5321.MailFrom address. Ultimately, the simple basis for deciding which address needs to be in the RFC5321.MailFrom field is to determine which address is to be informed about transfer-level problems (and possibly successes).

RFC5321.RcptTo: Set by - Author, Final MTA, MDA

This field specifies the MUA mailbox address of a Recipient. The string might not be visible in the message content header. For example, the IMF destination address header fields, such as RFC5322.To, might specify a Mailing List mailbox, while the RFC5321.RcptTo address specifies a member of that list.

RFC5321.ORCPT: Set by - Originator.

This is an optional parameter to the RCPT command, indicating the original address to which the current RCPT TO address corresponds, after a mapping was performed during transit. An ORCPT is the only reliable way to correlate a DSN from a multi-Recipient message transfer with the intended Recipient.

RFC5321.Received: Set by - Originator, Relay, Mediator, Dest

This field contains trace information, including originating host, Relays, Mediators, and MSA host domain names and/or IP Addresses.

RFC5321.Return-Path: Set by - Originator

The MDA records the RFC5321.MailFrom address into the RFC5321.Return-Path field.

RFC2919.List-Id: Set by - Mediator, Author

This field provides a globally unique Mailing List naming framework that is independent of particular hosts [RFC2919].

The identifier is in the form of a domain name; however, the string usually is constructed by combining the two parts of an email address. The result is rarely a true domain name, listed in the domain name service, although it can be.

RFC2369.List-*: Set by - Mediator, Author

[RFC2369] defines a collection of message header fields for use by Mailing Lists. In effect, they supply list-specific parameters for common Mailing-List user operations. The identifiers for these operations are for the list itself and the user-as-subscriber [RFC2369].

RFC0791.SourceAddr: Set by - The Client SMTP sending host immediately preceding the current receiving SMTP server

[RFC0791] defines the basic unit of data transfer for the Internet: the IP datagram. It contains a Source Address field that specifies the IP Address for the host (interface) from which the datagram was sent. This information is set and provided by the IP layer, which makes it independent of mail-level mechanisms. As such, it is often taken to be authoritative, although it is possible to provide false addresses.

4.2. User-Level Services

Interactions at the user level entail protocol exchanges, distinct from those that occur at lower layers of the Internet Mail MHS architecture that is, in turn, above the Internet Transport layer. Because the motivation for email, and much of its use, is for interaction among people, the nature and details of these protocol exchanges often are determined by the needs of interpersonal and group communication. To accommodate the idiosyncratic behavior inherent in such communication, only subjective guidelines, rather than strict rules, can be offered for some aspects of system behavior. Mailing Lists provide particularly salient examples.

4.2.1. Message User Agent (MUA)

A Message User Agent (MUA) works on behalf of User Actors and User applications. It is their representative within the email service.

The Author MUA (aMUA) creates a message and performs initial submission into the transfer infrastructure via a Mail Submission Agent (MSA). It can also perform any creation- and posting-time archiving in its Message Store (aMS). An MUA aMS can organize messages in many different ways. A common model uses aggregations, called "folders"; in IMAP they are called "mailboxes". This model allows a folder for messages under development (Drafts), a folder for messages waiting to be sent (Queued or Unsent), and a folder for messages that have been successfully posted for transfer (Sent). But none of these folders is required. For example, IMAP allows drafts to be stored in any folder, so no Drafts folder needs to be present.

The Recipient MUA (rMUA) works on behalf of the Recipient to process received mail. This processing includes generating user-level disposition control messages, displaying and disposing of the received message, and closing or expanding the user-communication loop by initiating replies and forwarding new messages.

NOTE: Although not shown in Figure 5, an MUA itself can have a distributed implementation, such as a "thin" user-interface module on a constrained device such as a smartphone, with most of the MUA functionality running remotely on a more capable server. An example of such an architecture might use IMAP [RFC3501] for most of the interactions between an MUA client and an MUA server. An approach for such scenarios is defined by [RFC4550].

A Mediator is a special class of MUA. It performs message re-posting, as discussed in Section 2.1.

An MUA can be automated, on behalf of a User who is not present at the time the MUA is active. One example is a bulk sending service that has a timed-initiation feature. These services are not to be confused with a Mailing List Mediator, since there is no incoming message triggering the activity of the automated service.

A popular and problematic MUA is an automatic responder, such as one that sends out-of-office notices. This behavior might be confused with that of a Mediator, but this MUA is generating a new message. Automatic responders can annoy Users of Mailing Lists unless they follow [RFC3834].

The identity fields are relevant to a typical MUA:

• RFC5322.From

• RFC5322.Reply-To

• RFC5322.Sender

• RFC5322.To, RFC5322.CC

• RFC5322.BCC

4.2.2. Message Store (MS)

An MUA can employ a long-term Message Store (MS). Figure 5 depicts an Author's MS (aMS) and a Recipient's MS (rMS). An MS can be located on a remote server or on the same machine as the MUA.

An MS acquires messages from an MDA either proactively by a local mechanism or even by a standardized mechanism such as SMTP(!), or reactively by using POP or IMAP. The MUA accesses the MS either by a local mechanism or by using POP or IMAP. Using POP for individual message accesses, rather than for bulk transfer, is relatively rare and inefficient.

4.3. MHS-Level Services

4.3.1. Mail Submission Agent (MSA)

A Mail Submission Agent (MSA) accepts the message submitted by the aMUA and enforces the policies of the hosting ADMD and the requirements of Internet standards. An MSA represents an unusual functional dichotomy. It represents the interests of the Author (aMUA) during message posting, to facilitate posting success; it also represents the interests of the MHS. In the architecture, these responsibilities are modeled, as shown in Figure 5, by dividing the MSA into two sub-components, aMSA and hMSA, respectively. Transfer of responsibility for a single message, from an Author's environment to the MHS, is called "posting". In Figure 5, it is marked as the (S) transition, within the MSA.

The hMSA takes transit responsibility for a message that conforms to the relevant Internet standards and to local site policies. It rejects messages that are not in conformance. The MSA performs final message preparation for submission and effects the transfer of responsibility to the MHS, via the hMSA. The amount of preparation depends upon the local implementations. Examples of aMSA tasks include adding header fields, such as Date: and Message-ID:, and modifying portions of the message from local notations to Internet standards, such as expanding an address to its formal IMF representation.

Historically, standards-based MUA/MSA message postings have used SMTP [RFC5321]. The standard currently preferred is SUBMISSION [RFC4409]. Although SUBMISSION derives from SMTP, it uses a separate TCP port and imposes distinct requirements, such as access authorization.

These identities are relevant to the MSA:

• RFC5321.HELO/.EHLO

• RFC3461.ENVID

• RFC5321.MailFrom

• RFC5321.RcptTo

• RFC5321.Received

• RFC0791.SourceAddr

4.3.2. Message Transfer Agent (MTA)

A Message Transfer Agent (MTA) relays mail for one application-level "hop". It is like a packet switch or IP router in that its job is to make routing assessments and to move the message closer to the Recipients. Of course, email objects are typically much larger than the payload of a packet or datagram, and the end-to-end latencies are typically much higher. Relaying is performed by a sequence of MTAs until the message reaches a destination MDA. Hence, an MTA implements both client and server MTA functionality; it does not change addresses in the envelope or reformulate the editorial content. A change in data form, such as to MIME Content-Transfer-Encoding, is within the purview of an MTA, but removal or replacement of body content is not. An MTA also adds trace information [RFC2505].

NOTE: Within a destination ADMD, email-relaying modules can make a variety of changes to the message, prior to delivery. In such cases, these modules are acting as Gateways, rather than MTAs.

Internet Mail uses SMTP ([RFC5321], [RFC2821], [RFC0821]) primarily to effect point-to-point transfers between peer MTAs. Other transfer mechanisms include Batch SMTP [RFC2442] and On-Demand Mail Relay (ODMR) SMTP [RFC2645]. As with most network-layer mechanisms, the Internet Mail SMTP supports a basic level of reliability, by virtue of providing for retransmission after a temporary transfer failure. Unlike typical packet switches (and Instant Messaging services), Internet Mail MTAs are expected to store messages in a manner that allows recovery across service interruptions, such as host-system shutdown. The degree of such robustness and persistence by an MTA can vary. The base SMTP specification provides a framework for protocol response codes. An extensible enhancement to this framework is defined in [RFC5248].

Although quite basic, the dominant routing mechanism for Internet Mail is the DNS MX record [RFC1035], which specifies an MTA through which the queried domain can be reached. This mechanism presumes a public, or at least a common, backbone that permits any attached MTA to connect to any other.

MTAs can perform any of these well-established roles:

Boundary MTA:

An MTA that is part of an ADMD and interacts with MTAs in other ADMDs. This is also called a Border MTA. There can be different Boundary MTAs, according to the direction of mail-flow.

Outbound MTA: An MTA that relays messages to other ADMDs.

Inbound MTA: An MTA that receives inbound SMTP messages from MTA Relays in other ADMDs, for example, an MTA running on the host listed as the target of an MX record.

Final MTA:

The MTA that transfers a message to the MDA.

These identities are relevant to the MTA:

• RFC5321.HELO/.EHLO

• RFC3461.ENVID

• RFC5321.MailFrom

• RFC5321.RcptTo

• RFC5322.Received: Set by - Relay Server

• RFC0791.SourceAddr

4.3.3. Mail Delivery Agent (MDA)

A transfer of responsibility from the MHS to a Recipient's environment (mailbox) is called "delivery". In the architecture, as depicted in Figure 5, delivery takes place within a Mail Delivery Agent (MDA) and is shown as the (D) transition from the MHS-oriented MDA component (hMDA) to the Recipient-oriented MDA component (rMDA).

An MDA can provide distinctive, address-based functionality, made possible by its detailed information about the properties of the destination address. This information might also be present elsewhere in the Recipient's ADMD, such as at an organizational border (Boundary) Relay. However, it is required for the MDA, if only because the MDA is required to know where to deliver the message.

Like an MSA, an MDA serves two roles, as depicted in Figure 5. Formal transfer of responsibility, called "delivery", is effected between the two components that embody these roles and is shown as "(D)" in Figure 5. The MHS portion (hMDA) primarily functions as a server SMTP engine. A common additional role is to redirect the message to an alternative address, as specified by the Recipient addressee's preferences. The job of the Recipient portion of the MDA (rMDA) is to perform any delivery actions that the Recipient specifies.

Transfer into the MDA is accomplished by a normal MTA transfer mechanism. Transfer from an MDA to an MS uses an access protocol, such as POP or IMAP.

NOTE: The term "delivery" can refer to the formal, MHS function specified here or to the first time a message is displayed to a Recipient. A simple, practical test for whether the MHS-based definition applies is whether a DSN can be generated.

These identities are relevant to the MDA:

RFC5321.Return-Path: Set by - Author Originator or Mediator Originator

The MDA records the RFC5321.MailFrom address into the RFC5321.Return-Path field.

RFC5322.Received: Set by - MDA server

An MDA can record a Received: header field to indicate trace information, including source host and receiving host domain names and/or IP Addresses.

4.4. Transition Modes

From the origination site to the point of delivery, Internet Mail usually follows a "push" model. That is, the Actor that holds the message initiates transfer to the next venue, typically with SMTP [RFC5321] or the Local Mail Transfer Protocol (LMTP) [RFC2033]. With a "pull" model, the Actor that holds the message waits for the Actor in the next venue to initiate a request for transfer. Standardized mechanisms for pull-based MHS transfer are ETRN [RFC1985] and ODMR [RFC2645].

After delivery, the Recipient's MUA (or MS) can gain access by having the message pushed to it or by having the receiver of access pull the message, such as by using POP [RFC1939] and IMAP [RFC3501].

4.5. Implementation and Operation

A discussion of any interesting system architecture often bogs down when architecture and implementation are confused. An architecture defines the conceptual functions of a service, divided into discrete conceptual modules. An implementation of that architecture can combine or separate architectural components, as needed for a particular operational environment. For example, a software system that primarily performs message relaying is an MTA, yet it might also include MDA functionality. That same MTA system might be able to interface with non-Internet email services and thus perform both as an MTA and as a Gateway.

Similarly, implemented modules might be configured to form elaborations of the architecture. An interesting example is a distributed MS. One portion might be a remote server and another might be local to the MUA. As discussed in [RFC1733], there are three operational relationships among such MSs:

Online:

The MS is remote, and messages are accessible only when the MUA is attached to the MS so that the MUA will re-fetch all or part of a message from one session to the next.

Offline:

The MS is local to the User, and messages are completely moved from any remote store, rather than (also) being retained there.

Disconnected:

An rMS and a uMS are kept synchronized, for all or part of their contents, while they are connected. When they are disconnected, mail can arrive at the rMS and the User can make changes to the uMS. The two stores are re-synchronized when they are reconnected.