Skip to main content

7. Transfer Codings

Transfer coding names are used to indicate an encoding transformation that has been, can be, or might need to be applied to a message's content in order to ensure "safe transport" through the network. This differs from a content coding in that the transfer coding is a property of the message rather than a property of the representation that is being transferred.

All transfer-coding names are case-insensitive and ought to be registered within the HTTP Transfer Coding registry, as defined in Section 7.3. They are used in the Transfer-Encoding (Section 6.1) and TE (Section 10.1.4 of [HTTP]) header fields (the latter also defining the "transfer-coding" grammar).

7.1. Chunked Transfer Coding​

The chunked transfer coding wraps content in order to transfer it as a series of chunks, each with its own size indicator, followed by an OPTIONAL trailer section containing trailer fields. Chunked enables content streams of unknown size to be transferred as a sequence of length-delimited buffers, which enables the sender to retain connection persistence and the recipient to know when it has received the entire message.

chunked-body   = *chunk
                 last-chunk
                 trailer-section
                 CRLF

chunk          = chunk-size [ chunk-ext ] CRLF
                 chunk-data CRLF
chunk-size     = 1*HEXDIG
last-chunk     = 1*("0") [ chunk-ext ] CRLF

chunk-data     = 1*OCTET ; a sequence of chunk-size octets

The chunk-size field is a string of hex digits indicating the size of the chunk-data in octets. The chunked transfer coding is complete when a chunk with a chunk-size of zero is received, possibly followed by a trailer section, and finally terminated by an empty line.

A recipient MUST be able to parse and decode the chunked transfer coding.

HTTP/1.1 does not define any means to limit the size of a chunked response such that an intermediary can be assured of buffering the entire response. Additionally, very large chunk sizes may cause overflows or loss of precision if their values are not represented accurately in a receiving implementation. Therefore, recipients MUST anticipate potentially large hexadecimal numerals and prevent parsing errors due to integer conversion overflows or precision loss due to integer representation.

The chunked coding does not define any parameters. Their presence SHOULD be treated as an error.

7.1.1. Chunk Extensions​

The chunked coding allows each chunk to include zero or more chunk extensions, immediately following the chunk-size, for the sake of supplying per-chunk metadata (such as a signature or hash), mid-message control information, or randomization of message body size.

chunk-ext      = *( BWS ";" BWS chunk-ext-name
                    [ BWS "=" BWS chunk-ext-val ] )

chunk-ext-name = token
chunk-ext-val  = token / quoted-string

The chunked coding is specific to each connection and is likely to be removed or recoded by each recipient (including intermediaries) before any higher-level application would have a chance to inspect the extensions. Hence, the use of chunk extensions is generally limited to specialized HTTP services such as "long polling" (where client and server can have shared expectations regarding the use of chunk extensions) or for padding within an end-to-end secured connection.

A recipient MUST ignore unrecognized chunk extensions. A server ought to limit the total length of chunk extensions received in a request to an amount reasonable for the services provided, in the same way that it applies length limitations and timeouts for other parts of a message, and generate an appropriate 4xx (Client Error) response if that amount is exceeded.

7.1.2. Chunked Trailer Section​

A trailer section allows the sender to include additional fields at the end of a chunked message in order to supply metadata that might be dynamically generated while the content is sent, such as a message integrity check, digital signature, or post-processing status. The proper use and limitations of trailer fields are defined in Section 6.5 of [HTTP].

trailer-section   = *( field-line CRLF )

A recipient that removes the chunked coding from a message MAY selectively retain or discard the received trailer fields. A recipient that retains a received trailer field MUST either store/forward the trailer field separately from the received header fields or merge the received trailer field into the header section. A recipient MUST NOT merge a received trailer field into the header section unless its corresponding header field definition explicitly permits and instructs how the trailer field value can be safely merged.

7.1.3. Decoding Chunked​

A process for decoding the chunked transfer coding can be represented in pseudo-code as:

length := 0
read chunk-size, chunk-ext (if any), and CRLF
while (chunk-size > 0) {
   read chunk-data and CRLF
   append chunk-data to content
   length := length + chunk-size
   read chunk-size, chunk-ext (if any), and CRLF
}
read trailer field
while (trailer field is not empty) {
   if (trailer fields are stored/forwarded separately) {
       append trailer field to existing trailer fields
   }
   else if (trailer field is understood and defined as mergeable) {
       merge trailer field with existing header fields
   }
   else {
       discard trailer field
   }
   read trailer field
}
Content-Length := length
Remove "chunked" from Transfer-Encoding

7.2. Transfer Codings for Compression​

The following transfer coding names for compression are defined by the same algorithm as their corresponding content coding:

compress (and x-compress) : See Section 8.4.1.1 of [HTTP].

deflate : See Section 8.4.1.2 of [HTTP].

gzip (and x-gzip) : See Section 8.4.1.3 of [HTTP].

The compression codings do not define any parameters. The presence of parameters with any of these compression codings SHOULD be treated as an error.

7.3. Transfer Coding Registry​

The "HTTP Transfer Coding Registry" defines the namespace for transfer coding names. It is maintained at \https://www.iana.org/assignments/http-parameters\``.

Registrations MUST include the following fields:

  • Name
  • Description
  • Pointer to specification text

Names of transfer codings MUST NOT overlap with names of content codings (Section 8.4.1 of [HTTP]) unless the encoding transformation is identical, as is the case for the compression codings defined in Section 7.2.

7.4. Negotiating Transfer Codings​

The TE header field (Section 10.1.4 of [HTTP]) is used in requests to indicate what transfer codings, besides chunked, the client is willing to accept in response, and whether or not the client is willing to preserve trailer fields in a chunked transfer coding.

A client MUST NOT send the chunked transfer coding name in TE; chunked is always acceptable for HTTP/1.1 recipients.

Last updated on