RFC 9112
HTTP/1.1
HTTP/1.1 🎀
Okay bestie, this is the big one 💅 RFC 9112 is the spec for HTTP/1.1, the protocol that moves the web around ✨ It's part 1 of 10, so we're starting with the front matter, the intro, and the message basics.
Quick facts first. It's from the IETF (Internet Engineering Task Force) 🦋 and it's a Standards Track document, so it's an actual Internet Standard and it's labelled STD 99 👑 The editors are R. Fielding (Adobe), M. Nottingham (Fastly), and J. Reschke (greenbytes). It came out in June 2022, and its ISSN is 2070-1721. It obsoletes RFC 7230 💀 (only the messaging and connection stuff, more on that in the intro), and the header doesn't list any Updates.
Abstract ✨
HTTP is a stateless application-level protocol for distributed, collaborative, hypertext information systems 🌈 This doc covers the HTTP/1.1 message syntax, how messages get parsed, how connections get managed, and the related security concerns 🔒 It also obsoletes parts of RFC 7230.
Status of This Memo 🌸
This is an Internet Standards Track document, so it's official. 🎀
It's a product of the IETF, and it represents the consensus of the IETF community. It went through public review and was approved for publication by the Internet Engineering Steering Group (IESG). If you want more on Internet Standards, check Section 2 of RFC 7841 📚
For the current status, any errata, and how to send feedback, go to https://www.rfc-editor.org/info/rfc9112 🦋
Copyright Notice 💜
Copyright (c) 2022 IETF Trust and the persons identified as the document authors. All rights reserved. 🧚
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they stand on the publication date. Please actually read these, because they explain your rights and restrictions for this document 🙈 Any Code Components pulled out of this document have to include Revised BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come with no warranty, same as the Revised BSD License says.
Some of this document may contain material from IETF Documents or IETF Contributions that were published or made publicly available before November 10, 2008. The people who control the copyright in some of that material may not have given the IETF Trust permission to allow modifications outside the IETF Standards Process 😭 Unless you get an adequate license from those copyright holders, this document can't be modified outside the IETF Standards Process, and derivative works can't be made outside it either. The only exceptions are formatting it for publication as an RFC or translating it into languages other than English 🌈
Table of Contents 🍓
1. Introduction
1.1. Requirements Notation
1.2. Syntax Notation
2. Message
2.1. Message Format
2.2. Message Parsing
2.3. HTTP Version
3. Request Line
3.1. Method
3.2. Request Target
3.2.1. origin-form
3.2.2. absolute-form
3.2.3. authority-form
3.2.4. asterisk-form
3.3. Reconstructing the Target URI
4. Status Line
5. Field Syntax
5.1. Field Line Parsing
5.2. Obsolete Line Folding
6. Message Body
6.1. Transfer-Encoding
6.2. Content-Length
6.3. Message Body Length
7. Transfer Codings
7.1. Chunked Transfer Coding
7.1.1. Chunk Extensions
7.1.2. Chunked Trailer Section
7.1.3. Decoding Chunked
7.2. Transfer Codings for Compression
7.3. Transfer Coding Registry
7.4. Negotiating Transfer Codings
8. Handling Incomplete Messages
9. Connection Management
9.1. Establishment
9.2. Associating a Response to a Request
9.3. Persistence
9.3.1. Retrying Requests
9.3.2. Pipelining
9.4. Concurrency
9.5. Failures and Timeouts
9.6. Tear-down
9.7. TLS Connection Initiation
9.8. TLS Connection Closure
10. Enclosing Messages as Data
10.1. Media Type message/http
10.2. Media Type application/http
11. Security Considerations
11.1. Response Splitting
11.2. Request Smuggling
11.3. Message Integrity
11.4. Message Confidentiality
12. IANA Considerations
12.1. Field Name Registration
12.2. Media Type Registration
12.3. Transfer Coding Registration
12.4. ALPN Protocol ID Registration
13. References
13.1. Normative References
13.2. Informative References
Appendix A. Collected ABNF
Appendix B. Differences between HTTP and MIME
B.1. MIME-Version
B.2. Conversion to Canonical Form
B.3. Conversion of Date Formats
B.4. Conversion of Content-Encoding
B.5. Conversion of Content-Transfer-Encoding
B.6. MHTML and Line Length Limitations
Appendix C. Changes from Previous RFCs
C.1. Changes from HTTP/0.9
C.2. Changes from HTTP/1.0
C.2.1. Multihomed Web Servers
C.2.2. Keep-Alive Connections
C.2.3. Introduction of Transfer-Encoding
C.3. Changes from RFC 7230
Acknowledgements
Index
Authors' Addresses1. Introduction ✨
HTTP is a stateless application-level request/response protocol 🦋 It uses extensible semantics and self-descriptive messages so you can interact with network-based hypertext information systems in a flexible way. HTTP/1.1 is defined by three things:
- This document
- "HTTP Semantics" [HTTP]
- "HTTP Caching" [CACHING]
This document says how HTTP semantics get carried over the HTTP/1.1 message syntax, framing, and connection management mechanisms 💖 Its goal is to define the complete set of requirements for HTTP/1.1 message parsers and message-forwarding intermediaries. Basically, it's the full rulebook for anyone building parsers or forwarders 📚
It obsoletes the parts of RFC 7230 that cover HTTP/1.1 messaging and connection management, and the changes are summarised in Appendix C.3 🌸 The other parts of RFC 7230 are obsoleted by "HTTP Semantics" [HTTP].
1.1. Requirements Notation
Okay, this part is important, so pay attention 👑 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174], but only when they appear in all capitals, as shown here. Lowercase versions are just regular words, no cap 🙈
Conformance criteria and considerations about error handling are defined in Section 2 of [HTTP] 💅
1.2. Syntax Notation
This spec uses the Augmented Backus-Naur Form (ABNF) notation of [RFC5234], extended with the notation for case-sensitivity in strings defined in [RFC7405] 🧚
It also uses a list extension from Section 5.6.1 of [HTTP]. It lets you define comma-separated lists compactly with a "#" operator, which works kind of like how "*" shows repetition 🍓 Appendix A has the collected grammar with all the list operators expanded into standard ABNF.
By convention, ABNF rule names that start with "obs-" are obsolete grammar rules that only exist for historical reasons 🕰️
The following core rules are included by reference, as defined in [RFC5234], Appendix B.1: ALPHA (letters), CR (carriage return), CRLF (CR LF), CTL (controls), DIGIT (decimal 0-9), DQUOTE (double quote), HEXDIG (hexadecimal 0-9/A-F/a-f), HTAB (horizontal tab), LF (line feed), OCTET (any 8-bit sequence of data), SP (space), and VCHAR (any visible [USASCII] character) ✨
These rules are defined in [HTTP]:
BWS = <BWS, see [HTTP], Section 5.6.3>
OWS = <OWS, see [HTTP], Section 5.6.3>
RWS = <RWS, see [HTTP], Section 5.6.3>
absolute-path = <absolute-path, see [HTTP], Section 4.1>
field-name = <field-name, see [HTTP], Section 5.1>
field-value = <field-value, see [HTTP], Section 5.5>
obs-text = <obs-text, see [HTTP], Section 5.6.4>
quoted-string = <quoted-string, see [HTTP], Section 5.6.4>
token = <token, see [HTTP], Section 5.6.2>
transfer-coding =
<transfer-coding, see [HTTP], Section 10.1.4>These rules are defined in [URI]:
absolute-URI = <absolute-URI, see [URI], Section 4.3>
authority = <authority, see [URI], Section 3.2>
uri-host = <host, see [URI], Section 3.2.2>
port = <port, see [URI], Section 3.2.3>
query = <query, see [URI], Section 3.4>2. Message 💌
HTTP/1.1 clients and servers talk to each other by sending messages 🫶 See Section 3 of [HTTP] for the general terminology and core concepts of HTTP.
2.1. Message Format
An HTTP/1.1 message is a start-line, then a CRLF, then a sequence of octets that looks a lot like the Internet Message Format [RFC5322] ✨ That means zero or more header field lines (collectively called the "headers" or the "header section"), an empty line that marks the end of the header section, and an optional message body.
HTTP-message = start-line CRLF
*( field-line CRLF )
CRLF
[ message-body ]A message can be a request from client to server or a response from server to client. Syntactically, the two types only differ in the start-line, which is either a request-line (for requests) or a status-line (for responses). They also differ in how the length of the message body is worked out (Section 6) 🔥
start-line = request-line / status-lineIn theory, a client could receive requests and a server could receive responses, telling them apart by their different start-line formats 🤔 In practice though, servers are built to only expect requests (a response is treated as an unknown or invalid request method), and clients are built to only expect responses 💀
HTTP borrows some protocol elements from the Multipurpose Internet Mail Extensions (MIME) [RFC2045] 🌈 See Appendix B for the differences between HTTP and MIME messages.
Okay, that's the foundation sorted 🎀 Next up is 2.2 Message Parsing, and that's where the real chaos starts 🦋
2.2. Message Parsing 🎀
okay so the normal way to parse an HTTP message is actually pretty chill 💅. you read the start-line into a structure, then read each header field line into a hash table by field name until you hit the empty line. after that you use the parsed data to figure out whether a message body is expected. if one is, you read it as a stream until you've gotten as many octets as the message body length, or the connection gets closed 🦋
A recipient MUST parse an HTTP message as a sequence of octets in an encoding that is a superset of US-ASCII [USASCII]. parsing it as a stream of Unicode characters without caring about the specific encoding is lowkey dangerous 😭. string processing libraries handle invalid multibyte character sequences that contain the octet LF (%x0A) in different ways, and that opens up security vulnerabilities. string-based parsers can only be used safely on protocol elements after the element has been pulled out of the message, like a header field line value after message parsing has split up the individual field lines 🔥
the line terminator for the start-line and fields is the sequence CRLF, but a recipient MAY recognize a single LF as a line terminator and ignore any CR right before it ✨
A sender MUST NOT generate a bare CR (a CR character not immediately followed by LF) within any protocol elements other than the content. if a recipient gets a bare CR like that, it MUST consider that element to be invalid, or replace each bare CR with SP before processing the element or forwarding the message 💀
older HTTP/1.0 user agent implementations might send an extra CRLF after a POST request. they did this as a workaround for some early server applications that couldn't read message body content that wasn't ended by a line-ending 🙈. an HTTP/1.1 user agent MUST NOT preface or follow a request with an extra CRLF. if you do want to end the request message body with a line-ending, the user agent MUST count the terminating CRLF octets as part of the message body length 🍓
for robustness, a server that is expecting to receive and parse a request-line SHOULD ignore at least one empty line (CRLF) received before the request-line 🌸
A sender MUST NOT send whitespace between the start-line and the first header field. 🙅♀️
A recipient that receives whitespace between the start-line and the first header field MUST either reject the message as invalid, or consume each whitespace-preceded line without processing it any further. that means ignoring the entire line, along with any later lines preceded by whitespace, until a properly formed header field shows up or the header section ends 🧚. rejecting or removing invalid whitespace-preceded lines is necessary so downstream recipients don't misinterpret them, since those recipients might be vulnerable to request smuggling (Section 11.2) or response splitting (Section 11.1) attacks. this part is serious, bestie 😤
when a server that only listens for HTTP request messages (or that is processing something whose start-line looks like an HTTP request message) gets a sequence of octets that doesn't match the HTTP-message grammar, the robustness exceptions above aside, the server SHOULD respond with a 400 (Bad Request) response and close the connection 🚫
2.3. HTTP Version 🦋
HTTP uses a "<major>.<minor>" numbering scheme to show versions of the protocol. this specification defines version "1.1". Section 2.5 of [HTTP] is where the semantics of HTTP version numbers are spelled out 👑
the version of an HTTP/1.x message is shown by an HTTP-version field in the start-line. HTTP-version is case-sensitive, so no sneaky lowercase 🙃
HTTP-version = HTTP-name "/" DIGIT "." DIGIT
HTTP-name = %s"HTTP"when an HTTP/1.1 message gets sent to an HTTP/1.0 recipient [HTTP/1.0], or to a recipient whose version is unknown, the HTTP/1.1 message is built so it can be read as a valid HTTP/1.0 message if all the newer features get ignored 🌈. this specification puts recipient-version requirements on some of the new features, so a conformant sender will only use features that are compatible until it has figured out, through configuration or by receiving a message, that the recipient supports HTTP/1.1 ✨
Intermediaries that process HTTP messages (so, all intermediaries other than the ones acting as tunnels) MUST send their own HTTP-version in forwarded messages, unless it's purposefully downgraded as a workaround for an upstream issue. basically an intermediary is not allowed to blindly forward the start-line without making sure the protocol version in that message matches a version it is conformant with for both receiving and sending messages 😭. forwarding a message without rewriting the HTTP-version might cause communication errors, because downstream recipients use the message sender's version to decide which features are safe to use for later communication with that sender 💅
a server MAY send an HTTP/1.0 response to an HTTP/1.1 request if it is known or suspected that the client implements the HTTP specification incorrectly and can't process later version responses correctly. examples are when a client fails to parse the version number correctly, or when an intermediary is known to blindly forward the HTTP-version even though it doesn't conform to the given minor version of the protocol 🍓. protocol downgrades like that SHOULD NOT be performed unless they're triggered by specific client attributes, such as when one or more of the request header fields (e.g., User-Agent) uniquely match the values sent by a client that's known to be in error 🎀
3. Request Line 🌸
A request-line begins with a method token, followed by a single space (SP), the request-target, another single space (SP), and ends with the protocol version 💖
request-line = method SP request-target SP HTTP-versionAlthough the request-line grammar rule requires that each component element be separated by a single SP octet, recipients MAY instead parse on whitespace-delimited word boundaries. aside from the CRLF terminator, they can treat any form of whitespace as the SP separator, and ignore preceding or trailing whitespace. that whitespace includes one or more of these octets: SP, HTAB, VT (%x0B), FF (%x0C), or bare CR 🧚. but heads up, lenient parsing can lead to request smuggling security vulnerabilities if a message has multiple recipients and each one has its own unique interpretation of robustness (see Section 11.2) 💀
HTTP doesn't put a predefined limit on request-line length, as described in Section 2.3 of [HTTP]. a server that receives a method longer than any method it implements SHOULD respond with a 501 (Not Implemented) status code. a server that receives a request-target longer than any URI it wishes to parse MUST respond with a 414 (URI Too Long) status code (see Section 15.5.15 of [HTTP]) 😤
there are various ad hoc limits on request-line length out there in the wild. it is RECOMMENDED that all HTTP senders and recipients support, at a minimum, request-line lengths of 8000 octets 🌈
3.1. Method 💅
The method token indicates the request method to be performed on the target resource. the request method is case-sensitive 🦋
method = tokenThe request methods defined by this specification can be found in Section 9 of [HTTP], along with info on the HTTP method registry and things to consider when defining new methods 📚
3.2. Request Target ✨
The request-target identifies the target resource that the request should be applied to. the client derives a request-target from its desired target URI. there are four distinct formats for the request-target, depending on both the method being requested and whether the request is going to a proxy 👑
request-target = origin-form
/ absolute-form
/ authority-form
/ asterisk-formNo whitespace is allowed in the request-target. unfortunately some user agents don't properly encode or exclude whitespace found in hypertext references, so those disallowed characters end up getting sent as the request-target in a malformed request-line 🙈
recipients of an invalid request-line SHOULD respond with either a 400 (Bad Request) error or a 301 (Moved Permanently) redirect with the request-target properly encoded. a recipient SHOULD NOT try to autocorrect the request and then process it without a redirect, since the invalid request-line might be deliberately crafted to get past security filters along the request chain 🚨
A client MUST send a Host header field (Section 7.2 of [HTTP]) in all HTTP/1.1 request messages. if the target URI includes an authority component, then a client MUST send a Host field value that is identical to that authority component, excluding any userinfo subcomponent and its "@" delimiter (Section 4.2 of [HTTP]). if the authority component is missing or undefined for the target URI, then a client MUST send a Host header field with an empty field value 🫶
A server MUST respond with a 400 (Bad Request) status code to any HTTP/1.1 request message that lacks a Host header field, and to any request message that contains more than one Host header field line or a Host header field with an invalid field value 💀
3.2.1. origin-form 🍓
The most common form of request-target is the "origin-form" 🌟
origin-form = absolute-path [ "?" query ]When making a request directly to an origin server, other than a CONNECT or server-wide OPTIONS request (as detailed below), a client MUST send only the absolute path and query components of the target URI as the request-target. if the target URI's path component is empty, the client MUST send "/" as the path within the origin-form of request-target. a Host header field is also sent, as defined in Section 7.2 of [HTTP] 💖
for example, say a client wants to retrieve a representation of the resource identified as http://www.example.org/where?q=now directly from the origin server. it would open (or reuse) a TCP connection to port 80 of the host "www.example.org" and send these lines:
GET /where?q=now HTTP/1.1
Host: www.example.orgfollowed by the remainder of the request message 🧚
3.2.2. absolute-form 💖
okay bestie, when a client is sending a request to a proxy, a client MUST send the target URI in "absolute-form" as the request-target 🎀 the only exceptions are CONNECT and server-wide OPTIONS requests, which have their own forms (more on those below) ✨
absolute-form = absolute-URIso the proxy is asked to either serve it from a valid cache if it can 🧚, or make the same request for the client, either to the next inbound proxy or straight to the origin server named in the request-target 🦋 the rules for that "forwarding" are in Section 7.6 of [HTTP] 💅
here's an example absolute-form request-line:
GET http://www.example.org/pub/WWW/TheProject.html HTTP/1.1now the Host header thing 🫶 a client MUST send a Host header field in an HTTP/1.1 request even when the request-target is in absolute-form, because that lets the Host info survive being forwarded through ancient HTTP/1.0 proxies that might not have implemented Host 😭 it's giving "just in case" energy
when a proxy gets a request with an absolute-form request-target, the proxy MUST ignore whatever Host header field came in (if there is one) and replace it with the host info from the request-target 🌸 and a proxy that forwards that request MUST generate a new Host field value from the received request-target, rather than just forwarding the Host value it got 🔥
when an origin server gets an absolute-form request-target, the origin server MUST ignore the received Host header field (if any) and use the host info from the request-target instead 👑 fun fact (not really fun, but important 🙈): if the request-target doesn't have an authority component, an empty Host header field will be sent in this case
and last but not least, a server MUST accept the absolute-form in requests even though most HTTP/1.1 clients will only ever send absolute-form to a proxy 🌈 accepting it is non-negotiable, periodt 💜
3.2.3. authority-form 🦋
the "authority-form" of request-target is only used for CONNECT requests (Section 9.3.6 of [HTTP]) 🍓 it's just the uri-host and port number of the tunnel destination, separated by a colon (":") 💖
authority-form = uri-host ":" portwhen a client is making a CONNECT request to set up a tunnel through one or more proxies, a client MUST send only the host and port of the tunnel destination as the request-target ✨ the client gets the host and port from the target URI's authority component, except it uses the scheme's default port if the target URI leaves the port out 🎀 so a CONNECT request to http://www.example.com looks like this:
CONNECT www.example.com:80 HTTP/1.1
Host: www.example.com3.2.4. asterisk-form 🌸
the "asterisk-form" of request-target is only used for a server-wide OPTIONS request (Section 9.3.7 of [HTTP]) 🧚
asterisk-form = "*"when a client wants OPTIONS for the server as a whole (instead of one specific named resource on that server), the client MUST send only "*" (%x2A) as the request-target 🫶 for example:
OPTIONS * HTTP/1.1now, if a proxy gets an OPTIONS request with an absolute-form request-target where the URI has an empty path and no query component, then the last proxy on the request chain MUST send a request-target of "*" when it forwards the request to the indicated origin server 🔥
for example, this request:
OPTIONS http://www.example.org:8001 HTTP/1.1would be forwarded by the final proxy as:
OPTIONS * HTTP/1.1
Host: www.example.org:8001after connecting to port 8001 of host "www.example.org" 💅
3.3. Reconstructing the Target URI 🔮
okay so the target URI is just the request-target when the request-target is in absolute-form 💖 in that case, a server will parse the URI into its generic components so it can evaluate it further ✨
otherwise, the server has to rebuild the target URI from the connection context and different parts of the request message, so it can figure out the target resource (Section 7.1 of [HTTP]) 🦋 here's how that goes:
- if the server's configuration provides a fixed URI scheme, or a trusted outbound gateway provides one, that scheme gets used for the target URI 🎀 this is super common in big deployments, because a gateway server gets the client's connection context and swaps it for its own connection to the inbound server 👑 otherwise, if the request came in over a secured connection, the target URI's scheme is "https"; if not, the scheme is "http" 🔥
- if the request-target is in authority-form, the target URI's authority component IS the request-target 💜 otherwise, the target URI's authority component is the field value of the Host header field 🌈 if there's no Host header field, or its field value is empty or invalid, the target URI's authority component is empty 😭
- if the request-target is in authority-form or asterisk-form, the target URI's combined path and query component is empty 🙈 otherwise, the target URI's combined path and query component is the request-target 🌸
- once they've been figured out as above, the components of a reconstructed target URI can be put back together into absolute-URI form by concatenating the scheme, "://", the authority, and the combined path and query component 🧚
Example 1: the following message received over a secure connection
GET /pub/WWW/TheProject.html HTTP/1.1
Host: www.example.orghas a target URI of
https://www.example.org/pub/WWW/TheProject.htmlExample 2: the following message received over an insecure connection
OPTIONS * HTTP/1.1
Host: www.example.org:8080has a target URI of
http://www.example.org:8080⚠️ if the target URI's authority component is empty and its URI scheme requires a non-empty authority (which is the case for "http" and "https"), the server can reject the request, or it can work out whether a configured default applies that matches the incoming connection's context 😭 that context might include things like address and port, what security has been applied, and locally defined info specific to that server's setup 💀 an empty authority gets swapped for the configured default before any more processing of the request happens 🫶
supplying a default name for the authority within the context of a secured connection is inherently unsafe if there's any chance the user agent's intended authority might be different from the default 💀 a server that can uniquely identify an authority from the request context MAY use that identity as a default without this risk 🔥 alternatively, it might be better to redirect the request to a safe resource that explains how to get a new client 🌈
and remember, rebuilding the client's target URI is only half of identifying a target resource 🦋 the other half is figuring out whether that target URI identifies a resource the server is willing and able to send a response for, as defined in Section 7.4 of [HTTP] 💖
4. Status Line ✨
the first line of a response message is the status-line 🎀 it's made of the protocol version, a space (SP), the status code, another space, and then an OPTIONAL textual phrase describing the status code 💅
status-line = HTTP-version SP status-code SP [ reason-phrase ]so the status-line grammar rule says each component has to be separated by a single SP octet, but recipients MAY instead parse on whitespace-delimited word boundaries 🧚 and, apart from the line terminator, they can treat any form of whitespace as the SP separator, while ignoring whitespace before or after 🦋 that whitespace includes one or more of these octets: SP, HTAB, VT (%x0B), FF (%x0C), or bare CR 🌸 BUT (and this is a big but 😭) lenient parsing can cause response splitting security vulnerabilities if there are multiple recipients of the message and each one has its own unique take on robustness (see Section 11.1) 💀
the status-code element is a 3-digit integer code that describes the result of the server's attempt to understand and satisfy the client's matching request 🔮 a recipient parses and interprets the rest of the response message in light of the semantics defined for that status code, if the recipient recognizes the status code 🫶 if it doesn't recognize the specific code, it goes by the class of that status code instead 🌈
status-code = 3DIGITHTTP's core status codes are defined in Section 15 of [HTTP], along with the classes of status codes, considerations for defining new status codes, and the IANA registry for collecting those definitions 👑
the reason-phrase element exists for one reason only: giving a textual description that goes with the numeric status code 🍓 it's mostly out of respect for older Internet application protocols that were used more often with interactive text clients 🥹
reason-phrase = 1*( HTAB / SP / VCHAR / obs-text )a client SHOULD ignore the reason-phrase content, because it's not a reliable channel for info 🙈 it might get translated for a given locale, overwritten by intermediaries, or discarded when the message is forwarded via other versions of HTTP 💅 also, a server MUST send the space that separates the status-code from the reason-phrase even when the reason-phrase is absent (so the status-line would end with the space) 🔥
5. Field Syntax 🎀
each field line is made of a case-insensitive field name, followed by a colon (":"), optional leading whitespace, the field line value, and optional trailing whitespace 💖
field-line = field-name ":" OWS field-value OWSthe rules for parsing inside field values are defined in Section 5.5 of [HTTP] 🦋 this section covers the generic syntax for including header fields in HTTP/1.1 messages, and pulling them back out 🧚
5.1. Field Line Parsing 🌸
messages get parsed with a generic algorithm that doesn't care about individual field names 💜 the contents of a given field line value aren't parsed until a later stage of message interpretation (usually after the whole field section of the message has been processed) ✨
no whitespace is allowed between the field name and colon 🙈 in the past, differences in how that whitespace got handled have led to security vulnerabilities in request routing and response handling 😭 so a server MUST reject any received request message that has whitespace between a header field name and colon, with a response status code of 400 (Bad Request) 🔥 and a proxy MUST remove any such whitespace from a response message before forwarding it downstream 🫶
a field line value might be preceded and/or followed by optional whitespace (OWS); a single SP before the field line value is preferred for consistent readability by humans 🎀 the field line value doesn't include that leading or trailing whitespace: OWS before the first non-whitespace octet of the field line value, or after the last non-whitespace octet, gets excluded by parsers when they pull the field line value out of a field line 👑
5.2. Obsolete Line Folding 🌸
Okay so, back in the HTTP/1.x days, you could make a field value stretch over multiple lines by starting each extra line with at least one space or horizontal tab. That's called obs-fold, and this spec is basically waving bye to it 👋. The only place it's still allowed is inside the message/http media type (Section 10.1). Here's the grammar, bestie:
obs-fold = OWS CRLF RWS
; obsolete line foldingA sender MUST NOT generate a message that includes line folding (i.e., that has any field line value that contains a match to the obs-fold rule) unless the message is intended for packaging within the message/http media type. Lowkey the most important rule in this whole section 💅.
If a server gets an obs-fold in a request message that is not inside a message/http container, it MUST either reject the message by sending a 400 (Bad Gateway, wait no, Bad Request) 🙈, preferably with a representation explaining that obsolete line folding is unacceptable, or replace each received obs-fold with one or more SP octets prior to interpreting the field value or forwarding the message downstream. Either way, it can't just let the fold slide through.
A proxy or gateway that gets an obs-fold in a response message that is not inside a message/http container MUST either discard the message and replace it with a 502 (Bad Gateway) response, preferably with a representation explaining that unacceptable line folding was received, or replace each received obs-fold with one or more SP octets prior to interpreting the field value or forwarding the message downstream. Same energy, different vibes 💀.
A user agent that receives an obs-fold in a response message that is not within a message/http container MUST replace each received obs-fold with one or more SP octets prior to interpreting the field value. That's the whole move for user agents ✨.
6. Message Body ✨
The message body (if any) of an HTTP/1.1 message is used to carry content (Section 6.4 of [HTTP]) for the request or response. The message body is identical to the content unless a transfer coding has been applied, as described in Section 6.1. So if there's no transfer coding, body and content are literally the same thing 🦋.
message-body = *OCTETThe rules for figuring out when a message body is present in an HTTP/1.1 message are different for requests and responses, so let's split them up 🎀.
For requests, the presence of a message body is signaled by a Content-Length or Transfer-Encoding header field. Request message framing is independent of method semantics, which means the method name doesn't decide whether a body shows up. Main character energy for the headers 👑.
For responses, it's a bit more involved. The presence of a message body in a response, as detailed in Section 6.3, depends on both the request method to which it is responding and the response status code. This corresponds to when response content is allowed by HTTP semantics (Section 6.4.1 of [HTTP]). So you have to look at both the method and the status code together 🧚.
6.1. Transfer-Encoding 🎀
The Transfer-Encoding header field lists the transfer coding names corresponding to the sequence of transfer codings that have been (or will be) applied to the content in order to form the message body. Transfer codings are defined in Section 7. Think of it as a little receipt listing every wrapper that got put on the content, in order 🍓.
Transfer-Encoding = #transfer-coding
; defined in [HTTP], Section 10.1.4Transfer-Encoding is analogous to the Content-Transfer-Encoding field of MIME, which was designed to enable safe transport of binary data over a 7-bit transport service ([RFC2045], Section 6). However, safe transport has a different focus for an 8bit-clean transfer protocol. In HTTP's case, Transfer-Encoding is primarily intended to accurately delimit dynamically generated content. It also serves to distinguish encodings that are only applied in transit from the encodings that are a characteristic of the selected representation. So it's the same vibe as MIME's version, but HTTP has its own priorities 💖.
A recipient MUST be able to parse the chunked transfer coding (Section 7.1) because it plays a crucial role in framing messages when the content size is not known in advance. Chunked is the MVP here 🏆. A sender MUST NOT apply the chunked transfer coding more than once to a message body (i.e., chunking an already chunked message is not allowed). No double chunking, bestie 🙅♀️. If any transfer coding other than chunked is applied to a request's content, the sender MUST apply chunked as the final transfer coding to ensure that the message is properly framed. If any transfer coding other than chunked is applied to a response's content, the sender MUST either apply chunked as the final transfer coding or terminate the message by closing the connection.
For example:
Transfer-Encoding: gzip, chunkedThis indicates that the content has been compressed using the gzip coding and then chunked using the chunked coding while forming the message body. Two layers, one outfit, very intentional 🧁.
Unlike Content-Encoding (Section 8.4.1 of [HTTP]), Transfer-Encoding is a property of the message, not of the representation. Any recipient along the request/response chain MAY decode the received transfer coding(s) or apply additional transfer coding(s) to the message body, assuming that corresponding changes are made to the Transfer-Encoding field value. Additional information about the encoding parameters can be provided by other header fields not defined by this specification. So anyone in the middle can peel layers off or add new ones, as long as they update the field 🦋.
Transfer-Encoding MAY be sent in a response to a HEAD request or in a 304 (Not Modified) response (Section 15.4.5 of [HTTP]) to a GET request, neither of which includes a message body. It's used to indicate that the origin server would have applied a transfer coding to the message body if the request had been an unconditional GET. This indication is not required, however, because any recipient on the response chain (including the origin server) can remove transfer codings when they are not needed. Optional hint, not a must-have ✨.
A server MUST NOT send a Transfer-Encoding header field in any response with a status code of 1xx (Informational) or 204 (No Content). A server MUST NOT send a Transfer-Encoding header field in any 2xx (Successful) response to a CONNECT request (Section 9.3.6 of [HTTP]). Those responses are a hard no 🚫.
A server that receives a request message with a transfer coding it does not understand SHOULD respond with 501 (Not Implemented). That's the recommended move when the coding is a mystery 🤷♀️.
Transfer-Encoding was added in HTTP/1.1. It is generally assumed that implementations advertising only HTTP/1.0 support will not understand how to process transfer-encoded content, and that an HTTP/1.0 message received with a Transfer-Encoding is likely to have been forwarded without proper handling of the chunked transfer coding in transit. Honestly, a 1.0 message with this header is kinda sus 🙈.
A client MUST NOT send a request containing Transfer-Encoding unless it knows the server will handle HTTP/1.1 requests (or later minor revisions). Such knowledge might be in the form of specific user configuration or by remembering the version of a prior received response. A server MUST NOT send a response containing Transfer-Encoding unless the corresponding request indicates HTTP/1.1 (or later minor revisions). Basically, check the version before you flex 💅.
Early implementations of Transfer-Encoding would occasionally send both a chunked transfer coding for message framing and an estimated Content-Length header field for use by progress bars. This is why Transfer-Encoding is defined as overriding Content-Length, as opposed to them being mutually incompatible. Unfortunately, forwarding such a message can lead to vulnerabilities regarding request smuggling (Section 11.2) or response splitting (Section 11.1) attacks if any downstream recipient fails to parse the message according to this specification, particularly when a downstream recipient only implements HTTP/1.0. This part is seriously dramatic, so read the security sections too 😭.
A server MAY reject a request that contains both Content-Length and Transfer-Encoding or process such a request in accordance with the Transfer-Encoding alone. Regardless, the server MUST close the connection after responding to such a request to avoid the potential attacks. Either it rejects the request or it trusts Transfer-Encoding, but it always hangs up after 📞.
A server or client that receives an HTTP/1.0 message containing a Transfer-Encoding header field MUST treat the message as if the framing is faulty, even if a Content-Length is present, and close the connection after processing the message. The message sender might have retained a portion of the message, in buffer, that could be misinterpreted by further use of the connection. So it's faulty, it gets treated as faulty, and the connection gets closed. No drama, just boundaries 🫶.
6.2. Content-Length 💅
When a message does not have a Transfer-Encoding header field, a Content-Length header field (Section 8.6 of [HTTP]) can provide the anticipated size, as a decimal number of octets, for potential content. For messages that do include content, the Content-Length field value provides the framing information necessary for determining where the data (and message) ends. For messages that do not include content, the Content-Length indicates the size of the selected representation (Section 8.6 of [HTTP]). Same header, different jobs depending on whether there's a body 🌈.
A sender MUST NOT send a Content-Length header field in any message that contains a Transfer-Encoding header field. Pick one framing method, babe, not both 🙅♀️.
Note: HTTP's use ofContent-Lengthfor message framing differs significantly from the same field's use in MIME, where it is an optional field used only within themessage/external-bodymedia-type. Don't let the MIME version fool you, it's a whole different beast in HTTP ✨.
6.3. Message Body Length 💅✨
Okay so bestie, this is where we figure out how long a message body actually is, and there's a strict order of precedence, so read it top to bottom 🎀. The length of a message body is determined by one of the following (in order of precedence):
- Any response to a HEAD request and any response with a 1xx (Informational), 204 (No Content), or 304 (Not Modified) status code is always terminated by the first empty line after the header fields, regardless of the header fields present in the message, and thus cannot contain a message body or trailer section. 🦋
- Any 2xx (Successful) response to a CONNECT request implies that the connection will become a tunnel immediately after the empty line that concludes the header fields. A client MUST ignore any Content-Length or Transfer-Encoding header fields received in such a message. 🌸
- If a message is received with both a Transfer-Encoding and a Content-Length header field, the Transfer-Encoding overrides the Content-Length. Such a message might indicate an attempt to perform request smuggling (Section 11.2) or response splitting (Section 11.1) and ought to be handled as an error. An intermediary that chooses to forward the message MUST first remove the received Content-Length field and process the Transfer-Encoding (as described below) prior to forwarding the message downstream. 🚨💜
- If a Transfer-Encoding header field is present and the chunked transfer coding (Section 7.1) is the final encoding, the message body length is determined by reading and decoding the chunked data until the transfer coding indicates the data is complete. 🧚
If a Transfer-Encoding header field is present in a response and the chunked transfer coding is not the final encoding, the message body length is determined by reading the connection until it is closed by the server.
If a Transfer-Encoding header field is present in a request and the chunked transfer coding is not the final encoding, the message body length cannot be determined reliably; the server MUST respond with the 400 (Bad Request) status code and then close the connection. 🙈
- If a message is received without Transfer-Encoding and with an invalid Content-Length header field, then the message framing is invalid and the recipient MUST treat it as an unrecoverable error, unless the field value can be successfully parsed as a comma-separated list (Section 5.6.1 of [HTTP]), all values in the list are valid, and all values in the list are the same (in which case, the message is processed with that single value used as the Content-Length field value). If the unrecoverable error is in a request message, the server MUST respond with a 400 (Bad Request) status code and then close the connection. If it is in a response message received by a proxy, the proxy MUST close the connection to the server, discard the received response, and send a 502 (Bad Gateway) response to the client. If it is in a response message received by a user agent, the user agent MUST close the connection to the server and discard the received response. 💀
- If a valid Content-Length header field is present without Transfer-Encoding, its decimal value defines the expected message body length in octets. If the sender closes the connection or the recipient times out before the indicated number of octets are received, the recipient MUST consider the message to be incomplete and close the connection. 🍓
- If this is a request message and none of the above are true, then the message body length is zero (no message body is present). 🫶
- Otherwise, this is a response message without a declared message body length, so the message body length is determined by the number of octets received prior to the server closing the connection. 👑
Since there is no way to distinguish a successfully completed, close-delimited response message from a partially received message interrupted by network failure, a server SHOULD generate encoding or length-delimited messages whenever possible. The close-delimiting feature exists primarily for backwards compatibility with HTTP/1.0. 🌈
Note: Request messages are never close-delimited because they are always explicitly framed by length or transfer coding, with the absence of both implying the request ends immediately after the header section. 📦
A server MAY reject a request that contains a message body but not a Content-Length by responding with 411 (Length Required). 🎈
Unless a transfer coding other than chunked has been applied, a client that sends a request containing a message body SHOULD use a valid Content-Length header field if the message body length is known in advance, rather than the chunked transfer coding, since some existing services respond to chunked with a 411 (Length Required) status code even though they understand the chunked transfer coding. This is typically because such services are implemented via a gateway that requires a content length in advance of being called, and the server is unable or unwilling to buffer the entire request before processing. 💖
A user agent that sends a request that contains a message body MUST send either a valid Content-Length header field or use the chunked transfer coding. A client MUST NOT use the chunked transfer coding unless it knows the server will handle HTTP/1.1 (or later) requests; such knowledge can be in the form of specific user configuration or by remembering the version of a prior received response. ✨
If the final response to the last request on a connection has been completely received and there remains additional data to read, a user agent MAY discard the remaining data or attempt to determine if that data belongs as part of the prior message body, which might be the case if the prior message's Content-Length value is incorrect. A client MUST NOT process, cache, or forward such extra data as a separate response, since such behavior would be vulnerable to cache poisoning. 🔥
7. Transfer Codings 🎀
Transfer coding names are used to indicate an encoding transformation that has been, can be, or might need to be applied to a message's content in order to ensure "safe transport" through the network. This differs from a content coding in that the transfer coding is a property of the message rather than a property of the representation that is being transferred. 🦋
All transfer-coding names are case-insensitive and ought to be registered within the HTTP Transfer Coding registry, as defined in Section 7.3. They are used in the Transfer-Encoding (Section 6.1) and TE (Section 10.1.4 of [HTTP]) header fields (the latter also defining the "transfer-coding" grammar). 💅
7.1. Chunked Transfer Coding 🧁
The chunked transfer coding wraps content in order to transfer it as a series of chunks, each with its own size indicator, followed by an OPTIONAL trailer section containing trailer fields. Chunked enables content streams of unknown size to be transferred as a sequence of length-delimited buffers, which enables the sender to retain connection persistence and the recipient to know when it has received the entire message. 🌸
chunked-body = *chunk
last-chunk
trailer-section
CRLF
chunk = chunk-size [ chunk-ext ] CRLF
chunk-data CRLF
chunk-size = 1*HEXDIG
last-chunk = 1*("0") [ chunk-ext ] CRLF
chunk-data = 1*OCTET ; a sequence of chunk-size octetsThe chunk-size field is a string of hex digits indicating the size of the chunk-data in octets. The chunked transfer coding is complete when a chunk with a chunk-size of zero is received, possibly followed by a trailer section, and finally terminated by an empty line. 🍭
A recipient MUST be able to parse and decode the chunked transfer coding. 🫶
HTTP/1.1 does not define any means to limit the size of a chunked response such that an intermediary can be assured of buffering the entire response. Additionally, very large chunk sizes may cause overflows or loss of precision if their values are not represented accurately in a receiving implementation. Therefore, recipients MUST anticipate potentially large hexadecimal numerals and prevent parsing errors due to integer conversion overflows or precision loss due to integer representation. 🚨
The chunked coding does not define any parameters. Their presence SHOULD be treated as an error. 🙅♀️
7.1.1. Chunk Extensions 🧚♀️
The chunked coding allows each chunk to include zero or more chunk extensions, immediately following the chunk-size, for the sake of supplying per-chunk metadata (such as a signature or hash), mid-message control information, or randomization of message body size. 🌈
chunk-ext = *( BWS ";" BWS chunk-ext-name
[ BWS "=" BWS chunk-ext-val ] )
chunk-ext-name = token
chunk-ext-val = token / quoted-stringThe chunked coding is specific to each connection and is likely to be removed or recoded by each recipient (including intermediaries) before any higher-level application would have a chance to inspect the extensions. Hence, the use of chunk extensions is generally limited to specialized HTTP services such as "long polling" (where client and server can have shared expectations regarding the use of chunk extensions) or for padding within an end-to-end secured connection. 🤫
A recipient MUST ignore unrecognized chunk extensions. A server ought to limit the total length of chunk extensions received in a request to an amount reasonable for the services provided, in the same way that it applies length limitations and timeouts for other parts of a message, and generate an appropriate 4xx (Client Error) response if that amount is exceeded. 👀
7.1.2. Chunked Trailer Section 💌
A trailer section allows the sender to include additional fields at the end of a chunked message in order to supply metadata that might be dynamically generated while the content is sent, such as a message integrity check, digital signature, or post-processing status. The proper use and limitations of trailer fields are defined in Section 6.5 of [HTTP]. 🔐
trailer-section = *( field-line CRLF )A recipient that removes the chunked coding from a message MAY selectively retain or discard the received trailer fields. A recipient that retains a received trailer field MUST either store/forward the trailer field separately from the received header fields or merge the received trailer field into the header section. A recipient MUST NOT merge a received trailer field into the header section unless its corresponding header field definition explicitly permits and instructs how the trailer field value can be safely merged. 🎀
7.1.3. Decoding Chunked 🧩
Okay so this part is the decoding recipe for chunked transfer coding, and it is so satisfying bestie 😭✨ It's written as pseudo-code, which means it's basically a little step-by-step cooking recipe:
length := 0
read chunk-size, chunk-ext (if any), and CRLF
while (chunk-size > 0) {
read chunk-data and CRLF
append chunk-data to content
length := length + chunk-size
read chunk-size, chunk-ext (if any), and CRLF
}
read trailer field
while (trailer field is not empty) {
if (trailer fields are stored/forwarded separately) {
append trailer field to existing trailer fields
}
else if (trailer field is understood and defined as mergeable) {
merge trailer field with existing header fields
}
else {
discard trailer field
}
read trailer field
}
Content-Length := length
Remove "chunked" from Transfer-EncodingSo the vibe is: start length at 0, read the chunk size (plus any chunk extension and the CRLF), and keep looping while the chunk size is bigger than 0 🔁. Each time, you read the chunk data, tack it onto the content, add its size to length, then grab the next size line. Once you hit a zero-sized chunk, the loop is done and you go through the trailer fields 🎀. Each trailer gets handled one of three ways: if they're stored or forwarded separately, it's appended to the existing trailers; if it's understood and defined as mergeable, it gets merged into the existing header fields; and if neither of those applies, it's discarded 🗑️. Then the very last steps are to set Content-Length to the final length and remove "chunked" from Transfer-Encoding 💅. Not gonna lie, that last bit is iconic 🦋.
7.2. Transfer Codings for Compression 🗜️
Compression codings are giving déjà vu, because these transfer coding names are defined by the exact same algorithm as their matching content coding 💜. So you've got:
compress (and x-compress): see Section 8.4.1.1 of [HTTP] 🌸
deflate: see Section 8.4.1.2 of [HTTP] 🌸
gzip (and x-gzip): see Section 8.4.1.3 of [HTTP] 🌸
The compression codings don't define any parameters, so if you see a parameter attached to one of them, that is a big red flag 🚩. It SHOULD be treated as an error. I'm putting that SHOULD in caps because it's an actual requirement word, not me being dramatic 🙈.
7.3. Transfer Coding Registry 📚
There's a registry for transfer coding names, called the "HTTP Transfer Coding Registry", and it lives at <https://www.iana.org/assignments/http-parameters>. Think of it as the official guest list for names 🎟️.
Every registration MUST include these three fields:
- Name
- Description
- Pointer to specification text
Names of transfer codings MUST NOT overlap with names of content codings (Section 8.4.1 of [HTTP]) unless the encoding transformation is identical. That's exactly the case for the compression codings in Section 7.2, so they're the allowed exception 🤝.
The TE header field (Section 10.1.4 of [HTTP]) uses a pseudo-parameter named "q" as the rank value when more than one transfer coding is acceptable. Future registrations SHOULD NOT define parameters called "q" (case-insensitively), so nobody gets confused 😩.
Values added to this namespace require IETF Review (see Section 4.8 of [RFC8126]) and MUST conform to the purpose of transfer coding defined in this specification 👑.
And a little tea ☕: using program names to identify encoding formats is not desirable and is discouraged for future encodings. We want names that describe the format, not the app that happens to make it 💅.
7.4. Negotiating Transfer Codings 🤝
The TE field (Section 10.1.4 of [HTTP]) is how HTTP/1.1 lets a client say which transfer codings, besides chunked, it's willing to accept in the response. It also says whether the client is willing to keep trailer fields in a chunked transfer coding 🎀.
A client MUST NOT send the chunked transfer coding name in TE. Chunked is always acceptable for HTTP/1.1 recipients, so it's already invited and doesn't need to RSVP 💖.
Here are three examples of TE in action:
TE: deflate
TE:
TE: trailers, deflate;q=0.5When multiple transfer codings are acceptable, the client MAY rank them by preference using a case-insensitive "q" parameter. It works kind of like the qvalues in content negotiation (see Section 12.4.2 of [HTTP]) 🍓. The rank value is a real number from 0 through 1. 0.001 is the least preferred, 1 is the most preferred, and a value of 0 means "not acceptable" 🙅♀️.
If the TE field value is empty, or if there's no TE field at all, chunked is the only acceptable transfer coding. A message with no transfer coding is always acceptable ✨.
The keyword "trailers" means the sender will not discard trailer fields, as described in Section 6.5 of [HTTP] 🧚.
Here's a fun detail: the TE header field only applies to the immediate connection. So a sender of TE MUST also send a "TE" connection option within the Connection header field (Section 7.6.1 of [HTTP]). That stops intermediaries that don't understand TE from forwarding it along 🛑. Main character energy for the Connection header, honestly 👑.
8. Handling Incomplete Messages 🥺
A server that receives an incomplete request message, usually because of a canceled request or a triggered timeout, MAY send an error response before closing the connection 💌.
A client that receives an incomplete response message MUST record the message as incomplete. That can happen when the connection gets closed early, or when decoding a supposedly chunked transfer coding fails 😭. Cache requirements for incomplete responses are defined in Section 3.3 of [CACHING] 📦.
If a response stops in the middle of the header section (before the empty line arrives), and the status code might rely on header fields to convey the full meaning, then the client can't assume that meaning got through. It might have to repeat the request to figure out what to do next 🔁. Ugh, so relatable.
A message body that uses chunked transfer coding is incomplete if the zero-sized chunk that ends the encoding never shows up 🫥. A message with a valid Content-Length is incomplete if the body received (in octets) is smaller than the value in Content-Length 📏.
A response with neither chunked transfer coding nor Content-Length ends when the connection closes. If the header section arrived intact, it counts as complete, UNLESS the underlying connection reported an error. For example, an "incomplete close" in TLS would leave the response incomplete, as described in Section 9.8 🚨.
9. Connection Management 🔌
HTTP messaging doesn't care what transport- or session-layer connection protocol is underneath it 🫶. HTTP only assumes a reliable transport with in-order delivery of requests, and in-order delivery of the matching responses. How HTTP request and response structures get mapped onto the data units of an underlying transport protocol is outside the scope of this spec 🙃.
As described in Section 7.3 of [HTTP], the specific connection protocols used for an HTTP interaction are decided by client configuration and the target URI. For example, the "http" URI scheme (Section 4.2.1 of [HTTP]) implies a default connection of TCP over IP with a default TCP port of 80. But the client might be configured to use a proxy over some other connection, port, or protocol 🌈.
HTTP implementations are expected to do connection management, which covers keeping track of current connections, establishing new connections or reusing existing ones, processing messages received on a connection, detecting connection failures, and closing each connection 🧹. Most clients keep multiple connections open in parallel, including more than one per server endpoint. Most servers are built to handle thousands of concurrent connections while controlling request queues for fair use and to detect denial-of-service attacks 🛡️. Protecting the queue is giving bodyguard energy 💪.
9.1. Establishment 🏗️
This spec doesn't describe how connections get established through the various transport- or session-layer protocols. That's beyond scope 🙅. What matters is that each HTTP connection maps to one underlying transport connection 🦋.
9.2. Associating a Response to a Request 🎯
HTTP/1.1 doesn't include a request identifier to link a request message with its corresponding response message(s). So it relies on responses arriving in the same order the requests were made on that connection 📬. Having more than one response per request only happens when one or more informational responses (1xx; see Section 15.2 of [HTTP]) come before a final response to the same request ✨.
A client with more than one outstanding request on a connection MUST keep a list of outstanding requests in the order they were sent. It MUST also link each response received on that connection to the first outstanding request that hasn't gotten a final (non-1xx) response yet 🧠.
If a client gets data on a connection with no outstanding requests, that data MUST NOT be treated as a valid response. The client SHOULD close the connection, because message delimitation is now ambiguous 😵. The exception is if the data is only one or more CRLF, which can be discarded per Section 2.2 🫧.
9.3. Persistence 🔁
HTTP/1.1 defaults to "persistent connections", which let multiple requests and responses ride over a single connection 🚂. HTTP implementations SHOULD support persistent connections 💖.
A recipient decides whether a connection is persistent based on the protocol version and the Connection header field (Section 7.6.1 of [HTTP]) in the most recently received message, if there is one 🕵️♀️:
- If the "close" connection option is present (Section 9.6), the connection will not persist after the current response. Otherwise,
- If the received protocol is HTTP/1.1 (or later), the connection will persist after the current response. Otherwise,
- If the received protocol is HTTP/1.0, the "keep-alive" connection option is present, either the recipient is not a proxy or the message is a response, and the recipient wishes to honor the HTTP/1.0 "keep-alive" mechanism, the connection will persist after the current response. Otherwise,
- The connection will close after the current response.
A client that doesn't support persistent connections MUST send the "close" connection option in every request message 🚪.
A server that doesn't support persistent connections MUST send the "close" connection option in every response message that does not have a 1xx (Informational) status code 🚪.
A client MAY send additional requests on a persistent connection until it sends or receives a "close" connection option, or receives an HTTP/1.0 response without a "keep-alive" connection option 🫠.
To stay persistent, every message on a connection needs a self-defined message length (one that isn't defined by the connection closing), as described in Section 6 📏. A server MUST read the entire request message body, or close the connection after sending its response. Otherwise the leftover data on a persistent connection would get misread as the next request 😱. Likewise, a client MUST read the entire response message body if it wants to reuse the same connection for a later request 🔄.
A proxy server MUST NOT keep a persistent connection with an HTTP/1.0 client. See Appendix C.2.2 for info and discussion on the problems with the Keep-Alive header field that lots of HTTP/1.0 clients implement 🙈.
See Appendix C.2.2 for more on backwards compatibility with HTTP/1.0 clients 📖.
9.3.1. Retrying Requests 🔄
Connections can close at any time, on purpose or not 💀. Implementations ought to plan for recovering from asynchronous close events 🛟. The conditions under which a client can automatically retry a sequence of outstanding requests are defined in Section 9.2.2 of [HTTP] 🦋.
Girl, the next part is about to get spicy with connection termination 🌈💜
9.3.2. Pipelining
okay so pipelining is giving main character energy 💅✨ A client that supports persistent connections MAY "pipeline" its requests (i.e., send multiple requests without waiting for each response). 🦋 A server MAY process a sequence of pipelined requests in parallel if they all have safe methods (Section 9.2.1 of [HTTP]), but it MUST send the corresponding responses in the same order that the requests were received. Responses always go in the order requests came in, bestie, no exceptions 🫶
A client that pipelines requests SHOULD retry unanswered requests if the connection closes before it receives all of the corresponding responses. 🔁 When retrying pipelined requests after a failed connection (a connection not explicitly closed by the server in its last complete response), a client MUST NOT pipeline immediately after connection establishment, since the first remaining request in the prior pipeline might have caused an error response that can be lost again if multiple requests are sent on a prematurely closed connection (see the TCP reset problem described in Section 9.6). 😭 Translation: after a dropped connection, don't go full speed right away. Wait for the first retried request to go through before you pile on more.
Idempotent methods (Section 9.2.2 of [HTTP]) are significant to pipelining because they can be automatically retried after a connection failure. 🌸 A user agent SHOULD NOT pipeline requests after a non-idempotent method, until the final response status code for that method has been received, unless the user agent has a means to detect and recover from partial failure conditions involving the pipelined sequence. 🙈 Non-idempotent requests are the drama queen of the request world, so don't stack stuff behind them unless you've got a recovery plan.
An intermediary that receives pipelined requests MAY pipeline those requests when forwarding them inbound, since it can rely on the outbound user agent(s) to determine what requests can be safely pipelined. 🧚 If the inbound connection fails before receiving a response, the pipelining intermediary MAY attempt to retry a sequence of requests that have yet to receive a response if the requests all have idempotent methods; otherwise, the pipelining intermediary SHOULD forward any received responses and then close the corresponding outbound connection(s) so that the outbound user agent(s) can recover accordingly. 🍓 Middle-person energy, but make it responsible 👑
9.4. Concurrency
A client ought to limit the number of simultaneous open connections that it maintains to a given server. 💜
Previous revisions of HTTP gave a specific number of connections as a ceiling, but this was found to be impractical for many applications. ✨ As a result, this specification does not mandate a particular maximum number of connections but, instead, encourages clients to be conservative when opening multiple connections. 🎀 No hard cap, just please don't be greedy, okay?
Multiple connections are typically used to avoid the "head-of-line blocking" problem, wherein a request that takes significant server-side processing and/or transfers very large content would block subsequent requests on the same connection. 🌈 However, each connection consumes server resources. 💀
Furthermore, using multiple connections can cause undesirable side effects in congested networks. 😭 Using larger numbers of connections can also cause side effects in otherwise uncongested networks, because their aggregate and initially synchronized sending behavior can cause congestion that would not have been present if fewer parallel connections had been used. It's giving "everyone rushing the same door at once" 🚪💥
Note that a server might reject traffic that it deems abusive or characteristic of a denial-of-service attack, such as an excessive number of open connections from a single client. 🛡️
9.5. Failures and Timeouts
Servers will usually have some timeout value beyond which they will no longer maintain an inactive connection. ⏰ Proxy servers might make this a higher value since it is likely that the client will be making more connections through the same proxy server. The use of persistent connections places no requirements on the length (or existence) of this timeout for either the client or the server. 🦋
A client or server that wishes to time out SHOULD issue a graceful close on the connection. Implementations SHOULD constantly monitor open connections for a received closure signal and respond to it as appropriate, since prompt closure of both sides of a connection enables allocated system resources to be reclaimed. 🧹✨ Clean exits only, babe.
A client, server, or proxy MAY close the transport connection at any time. 🚪 For example, a client might have started to send a new request at the same time that the server has decided to close the "idle" connection. From the server's point of view, the connection is being closed while it was idle, but from the client's point of view, a request is in progress. 😬 Classic mixed signals, both sides thought they were on a break.
A server SHOULD sustain persistent connections, when possible, and allow the underlying transport's flow-control mechanisms to resolve temporary overloads rather than terminate connections with the expectation that clients will retry. 🫶 The latter technique can exacerbate network congestion or server load. Don't kick people off just because it's busy, let flow control do its thing 💅
A client sending a message body SHOULD monitor the network connection for an error response while it is transmitting the request. 👀 If the client sees a response that indicates the server does not wish to receive the message body and is closing the connection, the client SHOULD immediately cease transmitting the body and close its side of the connection. 🛑 Read the room, stop uploading, and bounce 😭
9.6. Tear-down
The "close" connection option is defined as a signal that the sender will close this connection after completion of the response. 👋 A sender SHOULD send a Connection header field (Section 7.6.1 of [HTTP]) containing the "close" connection option when it intends to close a connection. For example,
Connection: closeas a request header field indicates that this is the last request that the client will send on this connection, while in a response, the same field indicates that the server is going to close this connection after the response message is complete. 🎀 Same header, two vibes: "last request from me" or "I'm dipping after this response."
Note that the field name "Close" is reserved, since using that name as a header field might conflict with the "close" connection option. 🙅♀️ Don't name a header "Close," it's a whole mess.
A client that sends a "close" connection option MUST NOT send further requests on that connection (after the one containing the "close") and MUST close the connection after reading the final response message corresponding to this request. 🔒 No more requests after the close, and close it once the final response is in, no exceptions.
A server that receives a "close" connection option MUST initiate closure of the connection (see below) after it sends the final response to the request that contained the "close" connection option. 🌙 The server SHOULD send a "close" connection option in its final response on that connection. The server MUST NOT process any further requests received on that connection. 🛑 Once the close comes in, the door is shut, babe.
A server that sends a "close" connection option MUST initiate closure of the connection (see below) after it sends the response containing the "close" connection option. The server MUST NOT process any further requests received on that connection. 👑 Same energy from the other side: send the close, then the server starts the closing process and ignores anything else that shows up.
A client that receives a "close" connection option MUST cease sending requests on that connection and close the connection after reading the response message containing the "close" connection option; if additional pipelined requests had been sent on the connection, the client SHOULD NOT assume that they will be processed by the server. 🥺 Stop sending, close after reading, and don't count on those extra pipelined requests getting answered, ngl.
If a server performs an immediate close of a TCP connection, there is a significant risk that the client will not be able to read the last HTTP response. 😱 If the server receives additional data from the client on a fully closed connection, such as another request sent by the client before receiving the server's response, the server's TCP stack will send a reset packet to the client; unfortunately, the reset packet might erase the client's unacknowledged input buffers before they can be read and interpreted by the client's HTTP parser. 💔 A reset can wipe out data the client hasn't read yet, which is so unfair.
To avoid the TCP reset problem, servers typically close a connection in stages. 🪜 First, the server performs a half-close by closing only the write side of the read/write connection. The server then continues to read from the connection until it receives a corresponding close by the client, or until the server is reasonably certain that its own TCP stack has received the client's acknowledgement of the packet(s) containing the server's last response. Finally, the server fully closes the connection. 🌷 Half-close, keep listening, then fully close. Slow and steady, bestie.
It is unknown whether the reset problem is exclusive to TCP or might also be found in other transport connection protocols. 🤔 Nobody's sure yet, so that's an open question, babe.
Note that a TCP connection that is half-closed by the client does not delimit a request message, nor does it imply that the client is no longer interested in a response. 🙋♀️ In general, transport signals cannot be relied upon to signal edge cases, since HTTP/1.1 is independent of transport. Transport signals are not the boss of HTTP, period 💅
9.7. TLS Connection Initiation
Conceptually, HTTP/TLS is simply sending HTTP messages over a connection secured via TLS [TLS13]. 🔐 Super simple idea, honestly iconic.
The HTTP client also acts as the TLS client. It initiates a connection to the server on the appropriate port and sends the TLS ClientHello to begin the TLS handshake. When the TLS handshake has finished, the client may then initiate the first HTTP request. All HTTP data MUST be sent as TLS "application data" but is otherwise treated like a normal connection for HTTP (including potential reuse as a persistent connection). 🤝 Handshake first, then the HTTP party starts, and all the HTTP data has to ride as TLS "application data" 🎀
9.8. TLS Connection Closure
TLS uses an exchange of closure alerts prior to (non-error) connection closure to provide secure connection closure; see Section 6.1 of [TLS13]. 🔒 When a valid closure alert is received, an implementation can be assured that no further data will be received on that connection. ✅ Closure alert means "okay, nothing else is coming."
When an implementation knows that it has sent or received all the message data that it cares about, typically by detecting HTTP message boundaries, it might generate an "incomplete close" by sending a closure alert and then closing the connection without waiting to receive the corresponding closure alert from its peer. 🫠 That's an "incomplete close," and it's a bit of a shortcut.
An incomplete close does not call into question the security of the data already received, but it could indicate that subsequent data might have been truncated. 🧐 As TLS is not directly aware of HTTP message framing, it is necessary to examine the HTTP data itself to determine whether messages are complete. Handling of incomplete messages is defined in Section 8. 📖 Check the HTTP data itself, because TLS doesn't know what a message is.
When encountering an incomplete close, a client SHOULD treat as completed all requests for which it has received either
- as much data as specified in the Content-Length header field or
- the terminal zero-length chunk (when Transfer-Encoding of chunked is used).
A response that has neither chunked transfer coding nor Content-Length is complete only if a valid closure alert has been received. 🚨 Treating an incomplete message as complete could expose implementations to attack. Not worth the risk, bestie 😰
A client detecting an incomplete close SHOULD recover gracefully. 🌼
Clients MUST send a closure alert before closing the connection. Clients that do not expect to receive any more data MAY choose not to wait for the server's closure alert and simply close the connection, thus generating an incomplete close on the server side. 💌
Servers SHOULD be prepared to receive an incomplete close from the client, since the client can often locate the end of server data. 🧚 Servers MUST attempt to initiate an exchange of closure alerts with the client before closing the connection. Servers MAY close the connection after sending the closure alert, thus generating an incomplete close on the client side. 🦋
- Enclosing Messages as Data
10.1. Media Type message/http ✨
okay so the message/http media type 💖 can wrap up a single HTTP request or response message, as long as it follows the MIME rules for every message type about line length and encodings 🎀. Because lines have to stay short, field values inside message/http can use line folding (obs-fold), the way Section 5.2 describes, to stretch one field value over several lines 🦋. A recipient of message/http data MUST replace any obsolete line folding with one or more SP characters when the message is consumed 💅.
Here's the registration info, bestie 🍓:
- Type name:
message - Subtype name:
http - Required parameters: N/A
- Optional parameters:
version,msgtypeversion: the HTTP-version number of the enclosed message (like"1.1"). If it's not there, the version can be figured out from the first line of the body 🌸.msgtype: the message type, which is"request"or"response". If it's not there, the type can be figured out from the first line of the body 🧚.
- Encoding considerations: only
"7bit","8bit", or"binary"are permitted - Security considerations: see Section 11
- Interoperability considerations: N/A
- Published specification: RFC 9112 (see Section 10.1)
- Applications that use this media type: N/A
- Fragment identifier considerations: N/A
- Additional information:
- Magic number(s): N/A
- Deprecated alias names for this type: N/A
- File extension(s): N/A
- Macintosh file type code(s): N/A
- Person and email address to contact for further information: See Authors' Addresses section.
- Intended usage: COMMON
- Restrictions on usage: N/A
- Author: See Authors' Addresses section.
- Change controller: IESG
10.2. Media Type application/http 🦋
the application/http media type is for enclosing a pipeline of one or more HTTP request or response messages 🎀. The messages can't be intermixed, so keep each one in its own lane, periodt 💅.
- Type name:
application - Subtype name:
http - Required parameters: N/A
- Optional parameters:
version,msgtypeversion: the HTTP-version number of the enclosed messages (like"1.1"). If it's not there, the version can be figured out from the first line of the body 🌈.msgtype: the message type, which is"request"or"response". If it's not there, the type can be figured out from the first line of the body ⭐.
- Encoding considerations: HTTP messages enclosed by this type are in
"binary"format; use of an appropriate Content-Transfer-Encoding is required when transmitted via email 📬 - Security considerations: see Section 11
- Interoperability considerations: N/A
- Published specification: RFC 9112 (see Section 10.2)
- Applications that use this media type: N/A
- Fragment identifier considerations: N/A
- Additional information:
- Deprecated alias names for this type: N/A
- Magic number(s): N/A
- File extension(s): N/A
- Macintosh file type code(s): N/A
- Person and email address to contact for further information: See Authors' Addresses section.
- Intended usage: COMMON
- Restrictions on usage: N/A
- Author: See Authors' Addresses section.
- Change controller: IESG
11. Security Considerations 🔥
okay this section is seriously important, so pay attention 🫶. It's here to tell developers, information providers, and users about the known security risks tied to HTTP message syntax and parsing 😭. Security stuff about HTTP semantics, content, and routing lives in [HTTP], so go check there too 🧚.
11.1. Response Splitting 💀
Response splitting (a.k.a. CRLF injection) is a super common move in a bunch of attacks on Web usage 🙈. It takes advantage of the line-based nature of HTTP message framing and of the way requests get matched to responses in order on persistent connections [Klein] 🔥. It can be especially damaging when requests go through a shared cache 😭.
Here's how it goes down. Response splitting exploits a weakness in servers (usually inside an application server) where an attacker can send encoded data inside some parameter of the request. That data is later decoded and echoed back inside any of the response header fields 👀. If the decoded data is crafted so the response looks finished and a new response has started, the response has been split. Then the content in that fake second response is controlled by the attacker 💅. The attacker can make any other request on the same persistent connection and trick the recipients (including intermediaries) into thinking the second half of the split is the real answer to the second request 💀.
For example, a parameter in the request-target might be read by an application server and reused in a redirect. That means the same parameter gets echoed in the Location header field of the response. If the application decodes the parameter and doesn't encode it properly when it goes into the response field, the attacker can send encoded CRLF octets and other content that makes one response look like two or more responses 😱.
A common defense is to filter requests for data that looks like encoded CR and LF (e.g., "%0D" and "%0A"). But that only works if the application server is just doing URI decoding, and not something sneakier like charset transcoding, XML entity translation, base64 decoding, sprintf reformatting, etc. 🙃 A more effective mitigation is to stop anything other than the server's core protocol libraries from sending a CR or LF within the header section. That means limiting the output of header fields to APIs that filter out bad octets, and not letting application servers write straight to the protocol stream 🛡️✨.
11.2. Request Smuggling 🍓
Request smuggling ([Linhart]) is a technique that takes advantage of differences in how various recipients parse the protocol 🦋. It hides extra requests (which might otherwise be blocked or disabled by policy) inside an apparently harmless request 🕵️. Like response splitting, request smuggling can lead to a whole bunch of attacks on HTTP usage 😭.
This specification added new requirements on request parsing, especially around message framing in Section 6.3, to make request smuggling less effective 🫶✨.
11.3. Message Integrity 🔐
HTTP doesn't define one specific mechanism for keeping messages intact. Instead, it relies on the error-detection abilities of the underlying transport protocols, plus length or chunk-delimited framing to detect completeness 🧩. Historically, the lack of a single integrity mechanism was justified by how informal most HTTP communication was 🙈. But HTTP is now used so much for information access that it's showing up in environments where checking message integrity is crucial 😤.
The mechanisms that come with the "https" scheme, like authenticated encryption, protect messages against modification 🔒💖. Care is still needed to make sure connection closure can't be used to truncate messages (see Section 9.8) ⚠️. User agents might refuse to accept incomplete messages or treat them specially. For example, a browser being used to look at medical history or drug interaction information needs to tell the user when the protocol detects that such info is incomplete, expired, or corrupted during transfer 🩺. These mechanisms might be turned on selectively through user agent extensions, or by the presence of message integrity metadata in a response 🧚.
The "http" scheme gives no protection against accidental or malicious modification of messages 💀.
Extensions to the protocol might help reduce the risk of unwanted modification of messages by intermediaries, even when the "https" scheme is used 🌸. Integrity might be assured by using message authentication codes or digital signatures that get selectively added to messages through extensible metadata fields 🔏✨.
11.4. Message Confidentiality 🤫
HTTP relies on the underlying transport protocols to provide message confidentiality when that's wanted 🔐. HTTP was specifically designed to be independent of the transport protocol, so it can be used over many forms of encrypted connection 🦋. The transport is picked either by the choice of URI scheme or by user agent configuration 🎀.
The "https" scheme can be used to identify resources that need a confidential connection, as described in Section 4.2.2 of [HTTP] 👑.
12. IANA Considerations 🌈
The change controller for the following registrations is: "IETF (iesg@ietf.org) - Internet Engineering Task Force" 🧚.
12.1. Field Name Registration 🏷️
IANA has added the following field names to the "Hypertext Transfer Protocol (HTTP) Field Name Registry" at <https://www.iana.org/assignments/http-fields>, as described in Section 18.4 of [HTTP] ✨.
+===================+===========+=========+============+
| Field Name | Status | Section | Comments |
+===================+===========+=========+============+
| Close | permanent | 9.6 | (reserved) |
+-------------------+-----------+---------+------------+
| MIME-Version | permanent | B.1 | |
+-------------------+-----------+---------+------------+
| Transfer-Encoding | permanent | 6.1 | |
+-------------------+-----------+---------+------------+
Table 112.2. Media Type Registration 📮
IANA has updated the "Media Types" registry at <https://www.iana.org/assignments/media-types> with the registration information in Sections 10.1 and 10.2 for the media types "message/http" and "application/http", respectively 💌.
12.3. Transfer Coding Registration 🎀
IANA has updated the "HTTP Transfer Coding Registry" at <https://www.iana.org/assignments/http-parameters/> with the registration procedure of Section 7.3 and the content coding names summarized in the table below 🗂️.
+============+===========================================+=========+
| Name | Description | Section |
+============+===========================================+=========+
| chunked | Transfer in a series of chunks | 7.1 |
+------------+-------------------------------------------+---------+
| compress | UNIX "compress" data format [Welch] | 7.2 |
+------------+-------------------------------------------+---------+
| deflate | "deflate" compressed data ([RFC1951]) | 7.2 |
| | inside the "zlib" data format ([RFC1950]) | |
+------------+-------------------------------------------+---------+
| gzip | GZIP file format [RFC1952] | 7.2 |
+------------+-------------------------------------------+---------+
| trailers | (reserved) | 12.3 |
+------------+-------------------------------------------+---------+
| x-compress | Deprecated (alias for compress) | 7.2 |
+------------+-------------------------------------------+---------+
| x-gzip | Deprecated (alias for gzip) | 7.2 |
+------------+-------------------------------------------+---------+
Table 2Note: the coding nametrailersis reserved because its use would conflict with the keywordtrailersin the TE header field (Section 10.1.4 of [HTTP]) ⚠️💅
12.4. ALPN Protocol ID Registration 🦋
okay so IANA has updated the "TLS Application-Layer Protocol Negotiation (ALPN) Protocol IDs" registry at https://www.iana.org/assignments/tls-extensiontype-values/ with a new entry for us 💖 here it is:
+==========+=============================+===========+
| Protocol | Identification Sequence | Reference |
+==========+=============================+===========+
| HTTP/1.1 | 0x68 0x74 0x74 0x70 0x2f | RFC 9112 |
| | 0x31 0x2e 0x31 ("http/1.1") | |
+----------+-----------------------------+-----------+
Table 3so basically HTTP/1.1 gets the identification sequence 0x68 0x74 0x74 0x70 0x2f 0x31 0x2e 0x31, which spells "http/1.1", and it points right back to RFC 9112 🎀 the registry is literally the official ID card for the protocol ✨
13. References 📚
13.1. Normative References
these are the ones you really need to read, bestie 🧐 the full list:
[CACHING] Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke,
Ed., "HTTP Caching", STD 98, RFC 9111,
DOI 10.17487/RFC9111, June 2022,
<https://www.rfc-editor.org/info/rfc9111>.
[HTTP] Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke,
Ed., "HTTP Semantics", STD 97, RFC 9110,
DOI 10.17487/RFC9110, June 2022,
<https://www.rfc-editor.org/info/rfc9110>.
[RFC1950] Deutsch, P. and J-L. Gailly, "ZLIB Compressed Data Format
Specification version 3.3", RFC 1950,
DOI 10.17487/RFC1950, May 1996,
<https://www.rfc-editor.org/info/rfc1950>.
[RFC1951] Deutsch, P., "DEFLATE Compressed Data Format Specification
version 1.3", RFC 1951, DOI 10.17487/RFC1951, May 1996,
<https://www.rfc-editor.org/info/rfc1951>.
[RFC1952] Deutsch, P., "GZIP file format specification version 4.3",
RFC 1952, DOI 10.17487/RFC1952, May 1996,
<https://www.rfc-editor.org/info/rfc1952>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax
Specifications: ABNF", STD 68, RFC 5234,
DOI 10.17487/RFC5234, January 2008,
<https://www.rfc-editor.org/info/rfc5234>.
[RFC7405] Kyzivat, P., "Case-Sensitive String Support in ABNF",
RFC 7405, DOI 10.17487/RFC7405, December 2014,
<https://www.rfc-editor.org/info/rfc7405>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[TLS13] Rescorla, E., "The Transport Layer Security (TLS) Protocol
Version 1.3", RFC 8446, DOI 10.17487/RFC8446, August 2018,
<https://www.rfc-editor.org/info/rfc8446>.
[URI] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform
Resource Identifier (URI): Generic Syntax", STD 66,
RFC 3986, DOI 10.17487/RFC3986, January 2005,
<https://www.rfc-editor.org/info/rfc3986>.
[USASCII] American National Standards Institute, "Coded Character
Set -- 7-bit American Standard Code for Information
Interchange", ANSI X3.4, 1986.
[Welch] Welch, T., "A Technique for High-Performance Data
Compression", IEEE Computer 17(6),
DOI 10.1109/MC.1984.1659158, June 1984,
<https://ieeexplore.ieee.org/document/1659158/>.13.2. Informative References
these are the extra-helpful reads, the background and history 🌸
[HTTP/1.0] Berners-Lee, T., Fielding, R., and H. Frystyk, "Hypertext
Transfer Protocol -- HTTP/1.0", RFC 1945,
DOI 10.17487/RFC1945, May 1996,
<https://www.rfc-editor.org/info/rfc1945>.
[Klein] Klein, A., "Divide and Conquer - HTTP Response Splitting,
Web Cache Poisoning Attacks, and Related Topics", March
2004, <https://packetstormsecurity.com/papers/general/
whitepaper_httpresponse.pdf>.
[Linhart] Linhart, C., Klein, A., Heled, R., and S. Orrin, "HTTP
Request Smuggling", June 2005,
<https://www.cgisecurity.com/lib/HTTP-Request-
Smuggling.pdf>.
[RFC2045] Freed, N. and N. Borenstein, "Multipurpose Internet Mail
Extensions (MIME) Part One: Format of Internet Message
Bodies", RFC 2045, DOI 10.17487/RFC2045, November 1996,
<https://www.rfc-editor.org/info/rfc2045>.
[RFC2046] Freed, N. and N. Borenstein, "Multipurpose Internet Mail
Extensions (MIME) Part Two: Media Types", RFC 2046,
DOI 10.17487/RFC2046, November 1996,
<https://www.rfc-editor.org/info/rfc2046>.
[RFC2049] Freed, N. and N. Borenstein, "Multipurpose Internet Mail
Extensions (MIME) Part Five: Conformance Criteria and
Examples", RFC 2049, DOI 10.17487/RFC2049, November 1996,
<https://www.rfc-editor.org/info/rfc2049>.
[RFC2068] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., and T.
Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1",
RFC 2068, DOI 10.17487/RFC2068, January 1997,
<https://www.rfc-editor.org/info/rfc2068>.
[RFC2557] Palme, J., Hopmann, A., and N. Shelness, "MIME
Encapsulation of Aggregate Documents, such as HTML
(MHTML)", RFC 2557, DOI 10.17487/RFC2557, March 1999,
<https://www.rfc-editor.org/info/rfc2557>.
[RFC5322] Resnick, P., Ed., "Internet Message Format", RFC 5322,
DOI 10.17487/RFC5322, October 2008,
<https://www.rfc-editor.org/info/rfc5322>.
[RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer
Protocol (HTTP/1.1): Message Syntax and Routing",
RFC 7230, DOI 10.17487/RFC7230, June 2014,
<https://www.rfc-editor.org/info/rfc7230>.
[RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for
Writing an IANA Considerations Section in RFCs", BCP 26,
RFC 8126, DOI 10.17487/RFC8126, June 2017,
<https://www.rfc-editor.org/info/rfc8126>.Appendix A. Collected ABNF 🧩
okay this is the whole grammar in one place, so you can see every rule together ✨ quick heads-up first: list rules get expanded per Section 5.6.1 of [HTTP] 💅
BWS = <BWS, see [HTTP], Section 5.6.3>
HTTP-message = start-line CRLF *( field-line CRLF ) CRLF [
message-body ]
HTTP-name = %x48.54.54.50 ; HTTP
HTTP-version = HTTP-name "/" DIGIT "." DIGIT
OWS = <OWS, see [HTTP], Section 5.6.3>
RWS = <RWS, see [HTTP], Section 5.6.3>
Transfer-Encoding = [ transfer-coding *( OWS "," OWS transfer-coding
) ]
absolute-URI = <absolute-URI, see [URI], Section 4.3>
absolute-form = absolute-URI
absolute-path = <absolute-path, see [HTTP], Section 4.1>
asterisk-form = "*"
authority = <authority, see [URI], Section 3.2>
authority-form = uri-host ":" port
chunk = chunk-size [ chunk-ext ] CRLF chunk-data CRLF
chunk-data = 1*OCTET
chunk-ext = *( BWS ";" BWS chunk-ext-name [ BWS "=" BWS chunk-ext-val
] )
chunk-ext-name = token
chunk-ext-val = token / quoted-string
chunk-size = 1*HEXDIG
chunked-body = *chunk last-chunk trailer-section CRLF
field-line = field-name ":" OWS field-value OWS
field-name = <field-name, see [HTTP], Section 5.1>
field-value = <field-value, see [HTTP], Section 5.5>
last-chunk = 1*"0" [ chunk-ext ] CRLF
message-body = *OCTET
method = token
obs-fold = OWS CRLF RWS
obs-text = <obs-text, see [HTTP], Section 5.6.4>
origin-form = absolute-path [ "?" query ]
port = <port, see [URI], Section 3.2.3>
query = <query, see [URI], Section 3.4>
quoted-string = <quoted-string, see [HTTP], Section 5.6.4>
reason-phrase = 1*( HTAB / SP / VCHAR / obs-text )
request-line = method SP request-target SP HTTP-version
request-target = origin-form / absolute-form / authority-form /
asterisk-form
start-line = request-line / status-line
status-code = 3DIGIT
status-line = HTTP-version SP status-code SP [ reason-phrase ]
token = <token, see [HTTP], Section 5.6.2>
trailer-section = *( field-line CRLF )
transfer-coding = <transfer-coding, see [HTTP], Section 10.1.4>
uri-host = <host, see [URI], Section 3.2.2>Appendix B. Differences between HTTP and MIME 🎀
so HTTP/1.1 borrows a bunch of stuff from the Internet Message Format [RFC5322] and MIME [RFC2045], so a message body can show up in an open variety of representations with extensible fields 🌈 BUT some of those constructs got reinterpreted so they fit interactive communication better, which means HTTP uses MIME bits a little differently. these choices were made to optimize performance over binary connections, give more freedom with new media types, make date comparisons easier, and play nice with common implementations 💅
this appendix spells out exactly where HTTP and MIME don't match. if you're a proxy or gateway talking to a strict MIME environment, pay attention here 👑 you'll need to know these differences and do the right conversions where needed 🫶
B.1. MIME-Version
HTTP is not a MIME-compliant protocol, no cap 🙈 BUT messages are allowed to include a single MIME-Version header field to say which version of MIME was used to build the message. using MIME-Version means the message fully conforms to MIME as defined in [RFC2045] ✨ and when senders export HTTP messages to strict MIME environments, they're responsible for making sure things are fully conformant, where that's possible 💖
B.2. Conversion to Canonical Form
MIME wants an Internet mail body part converted to canonical form before it gets sent, as described in Section 4 of [RFC2049]. it also says content of type "text" should show line breaks as CRLF and forbids using CR or LF anywhere outside of line break sequences [RFC2046] 😭 HTTP is way more chill about this: it doesn't care whether a line break inside content is CRLF, bare CR, or bare LF 🦋
a proxy or gateway going from HTTP to a strict MIME environment ought to translate all the line breaks inside text media types into the RFC 2049 canonical form of CRLF. heads up though, this can get messy because of Content-Encoding, and because HTTP lets you use some charsets that don't use octets 13 and 10 for CR and LF 🙈
converting line breaks will break any cryptographic checksums on the original content, unless that content was already in canonical form. so canonical form is recommended for any content that uses checksums in HTTP 🔐
B.3. Conversion of Date Formats
HTTP/1.1 only uses a restricted set of date formats (Section 5.6.7 of [HTTP]) so date comparison stays simple 🧚 proxies and gateways coming from other protocols ought to make sure any Date header field in a message matches one of the HTTP/1.1 formats, and rewrite the date if it doesn't 🍓
B.4. Conversion of Content-Encoding
MIME doesn't have anything that matches HTTP's Content-Encoding header field 😭 since it acts like a modifier on the media type, proxies and gateways going from HTTP to MIME-compliant protocols ought to either change the value of the Content-Type header field or decode the representation before forwarding the message 💜
(some experimental Internet mail apps used a media-type parameter of ;conversions=<content-coding> to do the same job as Content-Encoding. just know that this parameter isn't part of the MIME standards 🙃)
B.5. Conversion of Content-Transfer-Encoding ✨
okay so HTTP just doesn't use the Content-Transfer-Encoding field from MIME at all 🙈 which means if a proxy or gateway is coming from a MIME-compliant protocol and heading to HTTP, it needs to strip out any Content-Transfer-Encoding before handing the response over to the HTTP client. bye bye, field 💅
going the other way is a bigger deal. proxies and gateways going from HTTP to MIME-compliant protocols are responsible for making sure the message is in the right format and encoding for safe transport on that protocol. "safe transport" here just means whatever the limits of the protocol being used are, so no vibes-based guessing 🧐 And if transforming and labeling the data with an appropriate Content-Transfer-Encoding would make it more likely to get through safely over the destination protocol, that proxy or gateway ought to do it. it's giving responsible middleman energy 🫶
B.6. MHTML and Line Length Limitations 🌸
if you're an HTTP implementation that shares code with MHTML [RFC2557] implementations, you need to keep MIME line length limitations in mind 👀 HTTP doesn't have that limit, so HTTP does not fold long lines. no folding, no drama 💖
now, MHTML messages that get carried over HTTP still follow every MHTML rule, including the line length limits and folding, canonicalization, and so on. the reason is that HTTP transfers message-bodies without changing them, and (except for the "multipart/byteranges" type, see Section 14.6 of [HTTP]) it doesn't interpret the content or any MIME header lines that might be hiding inside. HTTP is basically the delivery truck that doesn't open the boxes 📦🚚
Appendix C. Changes from Previous RFCs 🎀
C.1. Changes from HTTP/0.9
okay the HTTP/0.9 tea ☕ HTTP/0.9 couldn't do header fields in a request at all, so it had zero way to support name-based virtual hosts (picking a resource by looking at the Host header field) 😭 any server doing name-based virtual hosts ought to turn off HTTP/0.9 support. also, most requests that look like HTTP/0.9 are actually just badly built HTTP/1.x requests, usually because a client didn't encode the request-target properly. not a real 0.9 request, just a sloppy client being sloppy 💀
C.2. Changes from HTTP/1.0
C.2.1. Multihomed Web Servers 🌈
the requirements that clients and servers support the Host header field (Section 7.2 of [HTTP]), report an error when it's missing from an HTTP/1.1 request, and accept absolute URIs (Section 3.2) are some of the biggest glow-ups HTTP/1.1 brought to the table ✨
back in the HTTP/1.0 days, clients assumed one IP address = one server. there was no real way to tell which server a request was meant for besides the IP address it was sent to 🙃 The Host header field showed up while HTTP/1.1 was being built, and even though most HTTP/1.0 browsers adopted it super fast, extra requirements were added to all HTTP/1.1 requests so it would be fully adopted. and as of this writing, most HTTP-based services depend on the Host header field to target requests. it's literally everywhere now 🔥
C.2.2. Keep-Alive Connections 🦋
in HTTP/1.0, the client opens a connection before the request, and the server closes it after sending the response. some implementations did the explicitly negotiated ("Keep-Alive") version of persistent connections described in Section 19.7.1 of [RFC2068] though, so they could keep things open 👀
some clients and servers might want to stay compatible with those older persistent connection approaches by explicitly negotiating with a Connection: keep-alive request header field. the catch is that some experimental HTTP/1.0 persistent connection implementations are faulty. for example, if an HTTP/1.0 proxy doesn't understand Connection, it'll wrongly forward that header field to the next inbound server, and that leads to a hung connection 😩
one attempted fix was a Proxy-Connection header field, aimed specifically at proxies. in practice that was also unworkable, because proxies are often deployed in multiple layers, so you get the same problem again 🙃
so, clients are encouraged not to send the Proxy-Connection header field in any requests. 🚫
clients are also encouraged to think carefully before using Connection: keep-alive in requests. yes, it can enable persistent connections with HTTP/1.0 servers, but clients using it will need to watch the connection for "hung" requests (which means the client ought to stop sending the header field). and clients ought not use this mechanism at all when a proxy is in the middle 🧠
C.2.3. Introduction of Transfer-Encoding 💜
HTTP/1.1 introduces the Transfer-Encoding header field (Section 6.1). transfer codings need to be decoded before an HTTP message gets forwarded over a MIME-compliant protocol. decode first, forward second 🎀
C.3. Changes from RFC 7230 🍓
most of the sections about HTTP's design goals, history, architecture, conformance criteria, protocol versioning, URIs, message routing, and header fields have moved over to [HTTP]. this document is now slimmed down to just the messaging syntax and the connection management requirements that are specific to HTTP/1.1 ✂️
bare CRs are now prohibited outside of content. (Section 2.2) no more sneaky bare CRs 🙅♀️
the ABNF definition of authority-form changed from the more general authority component of a URI (where port is optional) to the specific host:port format that CONNECT requires. (Section 3.2.3) it's stricter now, bestie 💅
recipients are required to avoid smuggling/splitting attacks when they process an ambiguous message framing. (Section 6.1) this is a serious one 😤 security first, always 🔐
in the ABNF for chunked extensions, (bad) whitespace around ";" and "=" has been brought back. whitespace was removed in [RFC7230], but that change turned out to break existing implementations, so it's back. (Section 7.1.1) sometimes the old ways are the right ways 🥹
trailer field semantics now go beyond the specifics of chunked transfer coding. the decoding algorithm for chunked (Section 7.1.3) got updated to encourage storing or forwarding trailer fields separately from the header section. trailer fields may only be merged into the header section if the recipient knows the corresponding field definition permits it and defines how to merge, and otherwise the trailer fields get discarded instead of merged. the trailer part is now called the trailer section, to be more consistent with the header section and more distinct from a body part. (Section 7.1.2) ✨
transfer coding parameters called "q" are disallowed, so they don't clash with the use of ranks in the TE header field. (Section 7.3) no "q" allowed, period 🙅♀️💖
Acknowledgements 🫶
go check out the "Acknowledgements" appendix in [HTTP], because it applies to this document too. everyone who helped out gets their flowers 🌷
Index 📚
A C D F G H M O R T X
A
absolute-form (of request-target) Section 3.2.2
application/http Media Type *_Section 10.2_*
asterisk-form (of request-target) Section 3.2.4
authority-form (of request-target) Section 3.2.3
C
chunked (Coding Format) Section 6.1; Section 6.3
chunked (transfer coding) *_Section 7.1_*
close Section 9.3; *_Section 9.6_*
compress (transfer coding) *_Section 7.2_*
Connection header field Section 9.6
Content-Length header field Section 6.2
Content-Transfer-Encoding header field Appendix B.5
D
deflate (transfer coding) *_Section 7.2_*
F
Fields
Close *_Section 9.6, Paragraph 4_*
MIME-Version *_Appendix B.1_*
Transfer-Encoding *_Section 6.1_*
G
Grammar
ALPHA *_Section 1.2_*
CR *_Section 1.2_*
CRLF *_Section 1.2_*
CTL *_Section 1.2_*
DIGIT *_Section 1.2_*
DQUOTE *_Section 1.2_*
HEXDIG *_Section 1.2_*
HTAB *_Section 1.2_*
HTTP-message *_Section 2.1_*
HTTP-name *_Section 2.3_*
HTTP-version *_Section 2.3_*
LF *_Section 1.2_*
OCTET *_Section 1.2_*
SP *_Section 1.2_*
Transfer-Encoding *_Section 6.1_*
VCHAR *_Section 1.2_*
absolute-form Section 3.2; *_Section 3.2.2_*
asterisk-form Section 3.2; *_Section 3.2.4_*
authority-form Section 3.2; *_Section 3.2.3_*
chunk *_Section 7.1_*
chunk-data *_Section 7.1_*
chunk-ext Section 7.1; *_Section 7.1.1_*
chunk-ext-name *_Section 7.1.1_*
chunk-ext-val *_Section 7.1.1_*
chunk-size *_Section 7.1_*
chunked-body *_Section 7.1_*
field-line *_Section 5_*; Section 7.1.2
field-name Section 5
field-value Section 5
last-chunk *_Section 7.1_*
message-body *_Section 6_*
method *_Section 3.1_*
obs-fold *_Section 5.2_*
origin-form Section 3.2; *_Section 3.2.1_*
reason-phrase *_Section 4_*
request-line *_Section 3_*
request-target *_Section 3.2_*
start-line *_Section 2.1_*
status-code *_Section 4_*
status-line *_Section 4_*
trailer-section Section 7.1; *_Section 7.1.2_*
gzip (transfer coding) *_Section 7.2_*
H
Header Fields
MIME-Version *_Appendix B.1_*
Transfer-Encoding *_Section 6.1_*
header line Section 2.1
header section Section 2.1
headers Section 2.1
M
Media Type
application/http *_Section 10.2_*
message/http *_Section 10.1_*
message/http Media Type *_Section 10.1_*
method *_Section 3.1_*
MIME-Version header field *_Appendix B.1_*
O
origin-form (of request-target) Section 3.2.1
R
request-target *_Section 3.2_*
T
Transfer-Encoding header field *_Section 6.1_*
X
x-compress (transfer coding) *_Section 7.2_*
x-gzip (transfer coding) *_Section 7.2_*Authors' Addresses 💌
here are the fabulous people behind this doc, with all their contact details 💖
Roy T. Fielding (editor)
Adobe
345 Park Ave
San Jose, CA 95110
United States of America
Email: fielding@gbiv.com
URI: https://roy.gbiv.com/
Mark Nottingham (editor)
Fastly
Prahran
Australia
Email: mnot@mnot.net
URI: https://www.mnot.net/
Julian Reschke (editor)
greenbytes GmbH
Hafenweg 16
48155 Münster
Germany
Email: julian.reschke@greenbytes.de
URI: https://greenbytes.de/tech/webdav/that's a wrap on RFC 9112 bestie 🌈✨ you made it through the whole thing and honestly? iconic behavior 👑💅
📦 where this page comes from
- ietf: rfc9112 2022-06
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms