RFC 9651

Structured Field Values for HTTP

📅 published September 2024, from ietf

📜 RFC 9651: Structured Field Values for HTTP 🎀

Okay so bestie, this one is Internet Engineering Task Force (IETF) stuff, and it's by M. Nottingham (Cloudflare) and P-H. Kamp (The Varnish Cache Project) 👑. It's dated September 2024 and it's a Standards Track document, so it's the real deal, not a vibes-only draft 💅. It obsoletes RFC 8941, meaning it replaces that older version, and it doesn't update anything else 🦋. The ISSN is 2070-1721 ✨.

Abstract ✨

Okay so the big idea is making HTTP header and trailer fields way easier and safer to define and handle 🫶. These fields are called "Structured Fields", "Structured Headers", or "Structured Trailers", and this doc gives you a set of data types plus the algorithms that go with them. It's built for people writing brand new HTTP fields 🌸.

This document obsoletes RFC 8941 💖.

Status of This Memo ✨

This is an Internet Standards Track document 💅.

It's a product of the Internet Engineering Task Force (IETF), it represents the consensus of the IETF community, it got public review, and the Internet Engineering Steering Group (IESG) approved it for publication 🎀. If you want more on Internet Standards, check Section 2 of RFC 7841 🧚.

Info about the current status, errata, and how to send feedback lives at https://www.rfc-editor.org/info/rfc9651 🔥.

Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved 💜.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they stand on the publication date. Read them carefully, bestie, because they explain your rights and restrictions 🙈. Code Components pulled out of this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come with no warranty as described in the Revised BSD License 💅.

Table of Contents 🦋

1.  Introduction
  1.1.  Intentionally Strict Processing
  1.2.  Notational Conventions
2.  Defining New Structured Fields
  2.1.  Example
  2.2.  Error Handling
  2.3.  Preserving Extensibility
  2.4.  Using New Structured Types in Extensions
3.  Structured Data Types
  3.1.  Lists
    3.1.1.  Inner Lists
    3.1.2.  Parameters
  3.2.  Dictionaries
  3.3.  Items
    3.3.1.  Integers
    3.3.2.  Decimals
    3.3.3.  Strings
    3.3.4.  Tokens
    3.3.5.  Byte Sequences
    3.3.6.  Booleans
    3.3.7.  Dates
    3.3.8.  Display Strings
4.  Working with Structured Fields in HTTP
  4.1.  Serializing Structured Fields
    4.1.1.  Serializing a List
    4.1.2.  Serializing a Dictionary
    4.1.3.  Serializing an Item
    4.1.4.  Serializing an Integer
    4.1.5.  Serializing a Decimal
    4.1.6.  Serializing a String
    4.1.7.  Serializing a Token
    4.1.8.  Serializing a Byte Sequence
    4.1.9.  Serializing a Boolean
    4.1.10. Serializing a Date
    4.1.11. Serializing a Display String
  4.2.  Parsing Structured Fields
    4.2.1.  Parsing a List
    4.2.2.  Parsing a Dictionary
    4.2.3.  Parsing an Item
    4.2.4.  Parsing an Integer or Decimal
    4.2.5.  Parsing a String
    4.2.6.  Parsing a Token
    4.2.7.  Parsing a Byte Sequence
    4.2.8.  Parsing a Boolean
    4.2.9.  Parsing a Date
    4.2.10. Parsing a Display String
5.  IANA Considerations
6.  Security Considerations
7.  References
  7.1.  Normative References
  7.2.  Informative References
Appendix A.  Frequently Asked Questions
  A.1.  Why Not JSON?
Appendix B.  Implementation Notes
Appendix C.  ABNF
Appendix D.  Changes from RFC 8941
Acknowledgements
Authors' Addresses

1. Introduction 🌈

Okay so defining the syntax of a new HTTP header (or trailer) field is genuinely exhausting 😭. Even with the guidance in Section 16.3.2 of [HTTP], a prospective field author still runs into a ton of decisions and pitfalls 💀.

And once a field exists, people keep writing bespoke parsers and serializers, because every field handles what looks like the same syntax a little differently 🙈.

So this doc brings in a shared set of data structures for defining new HTTP field values 🎀. It gives a generic, abstract model, plus a concrete serialization for expressing that model in HTTP [HTTP] header and trailer fields ✨.

A field defined as a "Structured Header" or "Structured Trailer" (or a "Structured Field" if it can be either) uses the types from this spec for its syntax and basic handling rules 💅. That makes it easier to define and easier to implement 🫶.

Future HTTP versions can also define other serializations of this same abstract model, so fields using it can be sent more efficiently without being redefined 🦋.

It's not a goal of this document to redefine the syntax of existing HTTP fields 🙅‍♀️. The mechanisms here are only meant for fields that explicitly opt in 💖.

Section 2 explains how to specify a Structured Field 📜. Section 3 defines the abstract data types you can use in Structured Fields 🧚. Those types get serialized to and parsed from HTTP field values using the algorithms in Section 4 🔥.

1.1. Intentionally Strict Processing 💅

This spec is intentionally strict 😤. It defines parsing and serialization as step-by-step algorithms, and the only error handling it allows is failing the entire operation ❌.

The point is faithful implementation and good interop 🫶. If one implementation tried to be helpful by accepting sloppier input, it would push everyone else into making similar (but probably subtly different) workarounds, and that makes interop worse 💀.

So strictness is a feature, not a bug 👑. It lets the producer catch and fix non-conformant input early, and it avoids the interop and security problems that could otherwise happen 🔥.

Also, if a field gets appended to by several parties (like intermediaries or different components of the sender), one party's bad value is likely to make the whole field fail parsing 😬.

1.2. Notational Conventions 🎀

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be read as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here 💖.

This doc uses the VCHAR, SP, DIGIT, ALPHA, and DQUOTE rules from [RFC5234] to describe characters and/or their ASCII bytes, depending on context 🧚. It also uses the tchar and OWS rules from [HTTP] for the same job ✨.

The spec uses algorithms to describe parsing and serialization 🔥. When parsing HTTP fields, implementations MUST behave indistinguishably from following the algorithms 💅.

For serialization, the algorithms define the recommended way to produce fields 🌸. Implementations MAY differ from that behavior, as long as the output still gets handled correctly by the parsing algorithm in Section 4.2 🦋.

2. Defining New Structured Fields 👑

To specify an HTTP field as a Structured Field, its authors need to do the following 📜:

  • Normatively reference this specification. Recipients and generators of the field need to know the requirements of this document are in effect 🎀.
  • Say whether the field is a Structured Header (header section only, the common case), a Structured Trailer (trailer section only), or a Structured Field (both) 🦋.
  • Specify the type of the field value: List (Section 3.1), Dictionary (Section 3.2), or Item (Section 3.3) 💅.
  • Define the semantics of the field value ✨.
  • Specify any extra constraints on the field value, and what happens when those constraints get violated 🔥.

Usually a field definition picks the top-level type (List, Dictionary, or Item) and then defines what types and constraints are allowed inside it 💖. For example, a List-typed header might have all Integer members, or a mix of types 🧚. An Item-typed header might only allow Strings, and maybe only strings starting with the letter "Q" or strings in lowercase 🌈. Inner Lists (Section 3.1.1) are only valid when a field definition explicitly allows them 🫶.

Fields using the Display String type should carefully specify their allowed Unicode code points, for example by naming a profile from [PRECIS] 🎀.

Field definitions can only use this spec for the entire field value, not a portion of it 🙅‍♀️.

Specs can refer to a field name as a "Structured Header name", "Structured Trailer name", or "Structured Field name", whichever fits 💜. They can also call the field value a "Structured Header value", "Structured Trailer value", or "Structured Field value" as needed ✨.

This spec sets minimums for the length or number of various structures that implementations support 💅. In most cases it doesn't set maximums, but authors should know that HTTP implementations do impose limits on the size of individual fields, the total number of fields, and/or the size of the whole header or trailer section 😭.

2.1. Example 🧚

A fictitious Foo-Example header field could be specified like this 💖:

|  42.  Foo-Example Header Field
|
|  The Foo-Example HTTP header field conveys information about how
|  much Foo the message has.
|
|  Foo-Example is an Item Structured Header Field [RFC9651].  Its
|  value MUST be an Integer (Section 3.3.1 of [RFC9651]).
|
|  Its value indicates the amount of Foo in the message, and it MUST
|  be between 0 and 10, inclusive; other values MUST cause the entire
|  header field to be ignored.
|
|  The following parameter is defined:
|
|  *  A parameter whose key is "foourl", and whose value is a String
|     (Section 3.3.3 of [RFC9651]), conveying the Foo URL for the
|     message.  See below for processing requirements.
|
|  "foourl" contains a URI-reference (Section 4.1 of [RFC3986]).  If
|  its value is not a valid URI-reference, the entire header field
|  MUST be ignored.  If its value is a relative reference
|  (Section 4.2 of [RFC3986]), it MUST be resolved (Section 5 of
|  [RFC3986]) before being used.

And an example of the field in use 🌸:

Foo-Example: 2; foourl="https://foo.example.com/"

Part 1 of 7 done, bestie! 🫶 Say "next" and I'll keep going with Section 2.2 💅.

2.2. Error Handling 💅

okay so, when parsing fails, the whole field gets ignored (see Section 4.2) 🙈 and field definitions can't override that, bestie, because if they could, generic software would literally break 😭 they can only add extra constraints, like limiting the numeric range of Integers and Decimals, the format of Strings and Tokens, the types allowed in a Dictionary's values, or how many Items are in a List 🎀

when a field-specific constraint gets violated, the whole field is ignored too 💀 unless the field definition says otherwise with its own handling rules. so like, if a header field is defined as an Item that's required to be an Integer, but you get a String instead, it should be ignored unless that field's definition explicitly says something different ✨

2.3. Preserving Extensibility 🦋

Structured Fields are built to be extensible, and honestly that's so iconic 👑 because experience has shown that even stuff nobody planned for ends up needing the allowed syntax and semantics of a field to change in a controlled way 🌈

both Items and Inner Lists take Parameters as an extensibility mechanism 🧚 so their values can later grow to hold more info if they need to 💖 to keep forward compatibility, field specs are discouraged from making an unrecognized parameter an error condition 🫶

field specifications are required to be an Item, List, or Dictionary to preserve extensibility 💅 fields that get defined as some other type (like Integer) are assumed to be Items, meaning they allow Parameters 🍓

to make sure this extensibility sticks around in the future, and to encourage consumers to use a complete parser implementation 🔥, a field definition can say that "grease" parameters get added by senders 🌸 a spec could stipulate that all parameters matching a defined pattern are reserved for this, and then encourage senders to include them on some portion of requests ✨ this helps discourage recipients from writing a parser that ignores Parameters altogether 🙈

specs that use Dictionaries can also stay forward compatible by requiring that the presence of unknown keys, along with their value and type, be ignored 💜 later specs can then add more keys and put constraints on them as needed 🦋

an extension to a Structured Field can require that an entire field value be ignored by a recipient that understands the extension if the constraints it defines on that value aren't met 😭💅

2.4. Using New Structured Types in Extensions 👑

because a field definition needs to reference a specific RFC for Structured Fields, the types usable in its value are limited to the ones defined in that RFC 💖 for example, a field whose definition references this document can have a value using the Date type (Section 3.3.7) ✨ but a field whose definition references RFC 8941 can't, because it'll be treated as invalid (and therefore discarded) by implementations of that specification 💀

this limit also applies to future extensions of a field 🎀 like, a field defined with a reference to RFC 8941 can't use the Date type, because some recipients might still be running a parser based on RFC 8941 to process it 🙈

BUT, this document is designed to be backward compatible with RFC 8941 🫶 a parser that implements the requirements here can also parse valid Structured Fields whose definitions reference RFC 8941 🌈

upgrading a Structured Fields implementation to support a newer revision of the spec (like this document) means some field values that were invalid under the earlier RFC might become valid when processed 🧚

for example, a field instance might contain a syntactically valid Date (Section 3.3.7), even though that field's definition doesn't accommodate Dates 😭 an implementation based on RFC 8941 would fail parsing that field instance because Dates aren't defined in that spec 💀 if that implementation got upgraded to this specification, parsing would now succeed ✨ in some cases the resulting Date value gets rejected by field-specific logic, but values in fields that are otherwise ignored (like extension parameters) might not get detected, and the field might end up being accepted and processed 🔥

3. Structured Data Types 🍓

this section gives an overview of the abstract types Structured Fields use, plus a quick description and examples of how each one gets serialized into textual HTTP fields 💅 Section 4 is where the details of parsing from and serializing into textual HTTP fields live 🌸

in summary 🎀:

  • there are three top-level types an HTTP field can be defined as: Lists, Dictionaries, and Items ✨
  • Lists and Dictionaries are containers, and their members can be Items or Inner Lists (which are themselves arrays of Items) 💖
  • both Items and Inner Lists can be Parameterized with key/value pairs 🦋

3.1. Lists 🧚

Lists are arrays of zero or more members, each of which can be an Item (Section 3.3) or an Inner List (Section 3.1.1), and both of those can be Parameterized (Section 3.1.2) 💜

an empty List is denoted by not serializing the field at all 🙈 which means fields defined as Lists have a default empty value 🫶

when serialized as a textual HTTP field, each member is separated by a comma and optional whitespace 💅 for example, a field whose value is defined as a List of Tokens could look like:

Example-List: sugar, tea, rum

lowkey important: Lists can have their members split across multiple lines of the same header or trailer section, as per Section 5.3 of [HTTP] 🔥 for example, these are equivalent:

Example-List: sugar, tea, rum

and

Example-List: sugar, tea
Example-List: rum

BUT individual members of a List cannot be safely split between lines 💀 see Section 4.2 for details ✨

Parsers MUST support Lists containing at least 1024 members 🎀 field specifications can constrain the types and cardinality of individual List values as they require 👑

3.1.1. Inner Lists 🌈

an Inner List is an array of zero or more Items (Section 3.3) 💖 both the individual Items and the Inner List itself can be Parameterized (Section 3.1.2) 🦋

when serialized in a textual HTTP field, Inner Lists are denoted by surrounding parentheses, and their values are delimited by one or more spaces 🧚 a field whose value is defined as a List of Inner Lists of Strings could look like:

Example-List: ("foo" "bar"), ("baz"), ("bat" "one"), ()

the last member in this example is an empty Inner List 💅

a header field whose value is defined as a List of Inner Lists with Parameters at both levels could look like:

Example-List: ("foo"; a=1;b=2);lvl=5, ("bar" "baz");lvl=1

Parsers MUST support Inner Lists containing at least 256 members 🍓 field specifications can constrain the types and cardinality of individual Inner List members as they require 🌸

3.1.2. Parameters ✨

Parameters are an ordered map of key-value pairs associated with an Item (Section 3.3) or Inner List (Section 3.1.1) 💖 the keys are unique within the scope of the Parameters they occur in, and the values are bare items (meaning they themselves can't be parameterized; see Section 3.3) 🎀

Implementations MUST provide access to Parameters both by index and by key 🔥 Specifications MAY use either means of accessing them 🫶

note that parameters are ordered, and parameter keys cannot contain uppercase letters 🙈

when serialized in a textual HTTP field, a Parameter is separated from its Item or Inner List and other Parameters by a semicolon 💅 for example:

Example-List: abc;a=1;b=2; cde_456, (ghi;jk=4 l);q="9";r=w

Parameters whose value is Boolean (see Section 3.3.6) true MUST omit that value when serialized 💀 for example, the "a" parameter here is true, while the "b" parameter is false:

Example-Integer: 1; a; b=?0

note that this requirement only applies to serialization 🦋 parsers are still required to correctly handle the true value when it shows up in a parameter 💜

Parsers MUST support at least 256 parameters on an Item or Inner List 🧚 and support parameter keys with at least 64 characters 🌈 Field specifications can constrain the order of individual parameters, as well as their values' types, as required 👑

3.2. Dictionaries 💖

Dictionaries are ordered maps of key-value pairs, where the keys are short textual strings and the values are Items (Section 3.3) or arrays of Items, and both of those can be Parameterized (Section 3.1.2) ✨ there can be zero or more members, and their keys are unique within the scope of the Dictionary they occur in 🎀

Implementations MUST provide access to Dictionaries both by index and by key 🔥 Specifications MAY use either means of accessing the members 🫶

same as Lists, an empty Dictionary is represented by omitting the entire field 🙈 which implies that fields defined as Dictionaries have a default empty value 💅

typically, a field specification defines the semantics of a Dictionary by specifying the allowed types for individual members by their keys, plus whether their presence is required or optional 🌸 Recipients MUST ignore members whose keys are undefined or unknown, unless the field's specification specifically disallows them 💀

when serialized as a textual HTTP field, members are ordered as serialized and separated by a comma with optional whitespace 🍓 Member keys cannot contain uppercase characters 🙈 keys and values are separated by "=" (without whitespace) ✨ for example:

Example-Dict: en="Applepie", da=:w4ZibGV0w6ZydGU=:

note that in this example, the final "=" is there because of the Byte Sequence; see Section 3.3.5 💖

Members whose value is Boolean (see Section 3.3.6) true MUST omit that value when serialized 🦋 for example, here both "b" and "c" are true:

Example-Dict: a=?0, b, c; foo=bar

note that this requirement only applies to serialization 🧚 parsers are still required to correctly handle the true Boolean value when it shows up in Dictionary values 💜

a Dictionary with a member whose value is an Inner List of Tokens:

Example-Dict: rating=1.5, feelings=(joy sadness)

a Dictionary with a mix of Items and Inner Lists, some with parameters:

Example-Dict: a=(1 2), b=3, c=4;aa=bb, d=(5 6);valid

note that Dictionaries can have their members split across multiple lines of the same header or trailer section 🌈 for example, these are equivalent:

Example-Dict: foo=1, bar=2

and

Example-Dict: foo=1
Example-Dict: bar=2

BUT individual members of a Dictionary cannot be safely split between lines 💀 see Section 4.2 for details ✨

Parsers MUST support Dictionaries containing at least 1024 key/value pairs and keys with at least 64 characters 👑 field specifications can constrain the order of individual Dictionary members, as well as their values' types, as required 🎀

3.3. Items 🦋

an Item can be an Integer (Section 3.3.1), a Decimal (Section 3.3.2), a String (Section 3.3.3), a Token (Section 3.3.4), a Byte Sequence (Section 3.3.5), a Boolean (Section 3.3.6), or a Date (Section 3.3.7) 🧚 it can have associated parameters (Section 3.1.2) 💖

for example, a header field defined to be an Item that is an Integer might look like:

Example-Integer: 5

or with parameters:

Example-Integer: 5; foo=bar

3.3.1. Integers 🔢✨

Okay so Integers range from -999,999,999,999,999 to 999,999,999,999,999 inclusive, which is up to fifteen digits, signed 💖. That range is there for IEEE 754 compatibility [IEEE754], so they play nice with floats 🦋

Here's an example:

Example-Integer: 42

Integers bigger than 15 digits can still be supported in a few ways: by using a String (Section 3.3.3), a Byte Sequence (Section 3.3.5), or a parameter on an Integer that acts as a scaling factor 🧚

You can serialize Integers with leading zeros (like "0002" or "-01") and signed zero ("-0"), but these distinctions may not be preserved by implementations 💅

Also, the commas in Integers are only there in this section's prose for readability 🙈 They're not valid in the wire format, so don't put them in your actual headers!

3.3.2. Decimals 🍓

Decimals are numbers with an integer part and a fractional part 🌸 The integer part has at most 12 digits, and the fractional part has at most three digits.

For example, a header whose value is defined as a Decimal could look like this:

Example-Decimal: 4.5

Leading zeros ("0002.5", "-01.334"), trailing zeros ("5.230", "-0.40"), and signed zero ("-0.0") are all serializable, but these distinctions may not be preserved by implementations 🙃

Heads up: the serialization algorithm (Section 4.1.5) rounds input that has more than three digits of precision in the fractional part. If you want a different rounding strategy, the field definition should specify it to happen before serialization 🧐

3.3.3. Strings 🎀

Strings are zero or more printable ASCII [RFC0020] characters, meaning the range %x20 to %x7E. Tabs, newlines, carriage returns, and friends are all excluded 🚫

Non-ASCII characters aren't directly supported in Strings, because they cause a bunch of interoperability headaches 😭 With a few exceptions, field values don't need them anyway. If a field does need to show non-ASCII content, use a Display String (Section 3.3.8) instead ✨

In a textual HTTP field, Strings get wrapped in double quotes, and a backslash ("\") is used to escape double quotes and backslashes. Like so:

Example-String: "hello world"

Only DQUOTE works as a delimiter here, so single quotes don't count 🙅‍♀️ Also, only DQUOTE and "\" can be escaped, and any other character after "\" MUST cause parsing to fail 💀

Parsers MUST support Strings (after any decoding) with at least 1024 characters 💪

3.3.4. Tokens 🌈

Tokens are short textual words that start with an alphabetic character or "*", followed by zero or more token characters. The token characters are the same ones allowed by the token ABNF rule defined in [HTTP], plus ":" and "/" 🧩

Here's one:

Example-Token: foo123/456

Parsers MUST support Tokens with at least 512 characters 🙌

Tokens are mostly defined so they play nicely with the data model of existing HTTP fields. They might need some extra steps to use in certain implementations, so for new fields, you're encouraged to use Strings instead 🫶

3.3.5. Byte Sequences 🦋

Byte Sequences can be conveyed in Structured Fields 📦

In a textual HTTP field, a Byte Sequence is wrapped in colons and encoded using base64 ([RFC4648], Section 4) 🔐 Here's what it looks like:

Example-ByteSequence: :cHJldGVuZCB0aGlzIGlzIGJpbmFyeSBjb250ZW50Lg==:

Parsers MUST support Byte Sequences with at least 16384 octets after decoding 🧠

3.3.6. Booleans ✅

Boolean values can be conveyed in Structured Fields too 💜

In a textual HTTP field, a Boolean starts with a "?", followed by a "1" for true or a "0" for false:

Example-Boolean: ?1

Plot twist: in Dictionary (Section 3.2) and Parameter (Section 3.1.2) values, Boolean true is shown just by leaving the value out entirely 🤯

3.3.7. Dates 📅

Dates can be conveyed in Structured Fields 🗓️

Their data model is kind of like Integers. They represent a delta in seconds from 1970-01-01T00:00:00Z, which can be negative, and leap seconds are excluded. Because of that, the serialization looks a lot like Integers, except it's marked with a leading "@" so you can tell them apart 🕰️

Example-Date: @1659578233

Parsers MUST support Dates whose values include all days in years 1 to 9999. In delta seconds from 1970-01-01T00:00:00Z, that's -62,135,596,800 to 253,402,214,400 🌌

3.3.8. Display Strings 💬

Display Strings are a lot like Strings, in that they hold zero or more characters. The difference is that they allow Unicode scalar values (basically every Unicode code point except surrogates), while Strings don't 🌍

Display Strings are meant for values that get shown to end users, so they might need to carry non-ASCII content. It is NOT RECOMMENDED to use them when a String (Section 3.3.3) or Token (Section 3.3.4) would do the job, because Unicode brings processing considerations (like normalization) and security considerations (like homograph attacks) that make it way harder to handle correctly 😵‍💫

Note that Display Strings don't tell you what language the value is in. If you need that, you can handle it separately, like with a parameter 🌐

In textual HTTP fields, Display Strings look similar to Strings, except non-ASCII characters are percent-encoded. There's a leading "%" so people (and parsers) know it's not a regular String 🎀

Example-DisplayString: %"This is intended for display to %c3%bcsers."

Also check out Section 6 for more security considerations when handling Display Strings 🔒

4. Working with Structured Fields in HTTP 🛠️

This section explains how to serialize and parse the abstract types from Section 3 into textual HTTP field values, plus other encodings that are compatible with them (for example, in HTTP/2 [HTTP/2] before compression with HPACK [HPACK]) 🧃

4.1. Serializing Structured Fields 🧁

Given a structure defined in this specification, return an ASCII string that's suitable for use in an HTTP field value ✨

  1. If the structure is a Dictionary or List and its value is empty (meaning it has no members), do not serialize the field at all. That means you omit both the field-name and the field-value 🙈
  2. If the structure is a List, let output_string be the result of running Serializing a List (Section 4.1.1) with the structure.
  3. Else, if the structure is a Dictionary, let output_string be the result of running Serializing a Dictionary (Section 4.1.2) with the structure.
  4. Else, if the structure is an Item, let output_string be the result of running Serializing an Item (Section 4.1.3) with the structure.
  5. Else, fail serialization 💔
  6. Return output_string converted into an array of bytes, using ASCII encoding [RFC0020] 🫧

4.1.1. Serializing a List 📜

Given an array of (member_value, parameters) tuples as input_list, return an ASCII string suitable for use in an HTTP field value 🌷

  1. Let output be an empty string.
  2. For each (member_value, parameters) of input_list:
    1. If member_value is an array, append the result of running Serializing an Inner List (Section 4.1.1.1) with (member_value, parameters) to output.
    2. Otherwise, append the result of running Serializing an Item (Section 4.1.3) with (member_value, parameters) to output.
    3. If more member_values remain in input_list:
      1. Append "," to output.
      2. Append a single SP to output.
  3. Return output 🎀

4.1.1.1. Serializing an Inner List 🪆

Given an array of (member_value, parameters) tuples as inner_list, and parameters as list_parameters, return an ASCII string suitable for use in an HTTP field value 🌸

  1. Let output be the string "(".
  2. For each (member_value, parameters) of inner_list:
    1. Append the result of running Serializing an Item (Section 4.1.3) with (member_value, parameters) to output.
    2. If more values remain in inner_list, append a single SP to output.
  3. Append ")" to output.
  4. Append the result of running Serializing Parameters (Section 4.1.1.2) with list_parameters to output.
  5. Return output 💖

4.1.1.2. Serializing Parameters 🏷️

Given an ordered Dictionary as input_parameters (where each member has a param_key and a param_value), return an ASCII string suitable for use in an HTTP field value 🦋

  1. Let output be an empty string.
  2. For each param_key with a value of param_value in input_parameters:
    1. Append ";" to output.
    2. Append the result of running Serializing a Key (Section 4.1.1.3) with param_key to output.
    3. If param_value is not Boolean true:
      1. Append "=" to output.
      2. Append the result of running Serializing a bare Item (Section 4.1.3.1) with param_value to output.
  3. Return output ✨

4.1.1.3. Serializing a Key 🔑

Given a key as input_key, return an ASCII string suitable for use in an HTTP field value 🗝️

  1. Convert input_key into a sequence of ASCII characters. If conversion fails, fail serialization.
  2. If input_key contains characters that aren't in lcalpha, DIGIT, "_", "-", ".", or "*", fail serialization 🙅‍♀️
  3. If the first character of input_key is not lcalpha or "*", fail serialization.
  4. Let output be an empty string.
  5. Append input_key to output.
  6. Return output 🌈

4.1.2. Serializing a Dictionary 📖

Given an ordered Dictionary as input_dictionary (where each member has a member_key and a tuple value of (member_value, parameters)), return an ASCII string suitable for use in an HTTP field value 💜

  1. Let output be an empty string.
  2. For each member_key with a value of (member_value, parameters) in input_dictionary:
    1. Append the result of running Serializing a Key (Section 4.1.1.3) with the member's member_key to output.
    2. If member_value is Boolean true:
      1. Append the result of running Serializing Parameters (Section 4.1.1.2) with parameters to output.
    3. Otherwise:
      1. Append "=" to output.
      2. If member_value is an array, append the result of running Serializing an Inner List (Section 4.1.1.1) with (member_value, parameters) to output.
      3. Otherwise, append the result of running Serializing an Item (Section 4.1.3) with (member_value, parameters) to output.
    4. If more members remain in input_dictionary:
      1. Append "," to output.
      2. Append a single SP to output.
  3. Return output 🎀

4.1.3. Serializing an Item 🧸

Given an Item as bare_item and Parameters as item_parameters, return an ASCII string suitable for use in an HTTP field value 🫶

  1. Let output be an empty string.
  2. Append the result of running Serializing a Bare Item (Section 4.1.3.1) with bare_item to output.
  3. Append the result of running Serializing Parameters (Section 4.1.1.2) with item_parameters to output.
  4. Return output 👑

4.1.3.1. Serializing a Bare Item ✨

okay bestie, this is where we take a bare Item (input_item) and turn it into an ASCII string that's safe to drop into an HTTP field value 🎀 we check what type it is and hand it off to the right serializer. it's like a sorting hat 🧚

  1. If input_item is an Integer, return the result of running Serializing an Integer (Section 4.1.4) with input_item.
  2. If input_item is a Decimal, return the result of running Serializing a Decimal (Section 4.1.5) with input_item.
  3. If input_item is a String, return the result of running Serializing a String (Section 4.1.6) with input_item.
  4. If input_item is a Token, return the result of running Serializing a Token (Section 4.1.7) with input_item.
  5. If input_item is a Byte Sequence, return the result of running Serializing a Byte Sequence (Section 4.1.8) with input_item.
  6. If input_item is a Boolean, return the result of running Serializing a Boolean (Section 4.1.9) with input_item.
  7. If input_item is a Date, return the result of running Serializing a Date (Section 4.1.10) with input_item.
  8. If input_item is a Display String, return the result of running Serializing a Display String (Section 4.1.11) with input_item.
  9. Otherwise, fail serialization. 💀

4.1.4. Serializing an Integer 🔢

this one's giving simple. we take an Integer (input_integer) and make it an ASCII string 💖

  1. If input_integer is not an integer in the range of -999,999,999,999,999 to 999,999,999,999,999 inclusive, fail serialization.
  2. Let output be an empty string.
  3. If input_integer is less than (but not equal to) 0, append "-" to output.
  4. Append input_integer's numeric value represented in base 10 using only decimal digits to output.
  5. Return output.

4.1.5. Serializing a Decimal 🦋

decimals are a little more work, bestie, because we have to round and count digits so nothing goes sideways 🌸 take a decimal number (input_decimal) and make it an ASCII string.

  1. If input_decimal is not a decimal number, fail serialization.
  2. If input_decimal has more than three significant digits to the right of the decimal point, round it to three decimal places, rounding the final digit to the nearest value, or to the even value if it is equidistant.
  3. If input_decimal has more than 12 significant digits to the left of the decimal point after rounding, fail serialization.
  4. Let output be an empty string.
  5. If input_decimal is less than (but not equal to) 0, append "-" to output.
  6. Append input_decimal's integer component represented in base 10 (using only decimal digits) to output; if it is zero, append "0".
  7. Append "." to output.
  8. If input_decimal's fractional component is zero, append "0" to output.
  9. Otherwise, append the significant digits of input_decimal's fractional component represented in base 10 (using only decimal digits) to output.
  10. Return output.

4.1.6. Serializing a String 🎀

strings get the most babysitting, periodt 💅 we only allow ASCII, we only allow certain characters, and then we wrap the whole thing in quotes and escape the sneaky ones. take a String (input_string) and make it an ASCII string.

  1. Convert input_string into a sequence of ASCII characters; if conversion fails, fail serialization.
  2. If input_string contains characters in the range %x00-1f or %x7f-ff (i.e., not in VCHAR or SP), fail serialization.
  3. Let output be the string DQUOTE.
  4. For each character char in input_string:
    1. If char is "\" or DQUOTE:
      1. Append "\" to output.
      2. Append char to output.
  5. Append DQUOTE to output.
  6. Return output.

4.1.7. Serializing a Token 🌈

tokens are the cool, no-quotes-needed kind 😎 we just check the first character and the rest, then tack it on.

  1. Convert input_token into a sequence of ASCII characters; if conversion fails, fail serialization.
  2. If the first character of input_token is not ALPHA or "*", or the remaining portion contains a character not in tchar, ":", or "/", fail serialization.
  3. Let output be an empty string.
  4. Append input_token to output.
  5. Return output.

4.1.8. Serializing a Byte Sequence 🧚

byte sequences get base64-encoded and wrapped in colons, but there are a couple of padding rules we have to follow, so pay attention 🙈

  1. If input_bytes is not a sequence of bytes, fail serialization.
  2. Let output be an empty string.
  3. Append ":" to output.
  4. Append the result of base64-encoding input_bytes as per [RFC4648], Section 4, taking account of the requirements below.
  5. Append ":" to output.
  6. Return output.

The encoded data is required to be padded with "=", as per [RFC4648], Section 3.2.

Likewise, encoded data SHOULD have pad bits set to zero, as per [RFC4648], Section 3.5, unless it is not possible to do so due to implementation constraints.

4.1.9. Serializing a Boolean 🌸

booleans are the simplest little guys ever. a question mark, then a 1 or a 0. that's it, that's the tea 🍓

  1. If input_boolean is not a boolean, fail serialization.
  2. Let output be an empty string.
  3. Append "?" to output.
  4. If input_boolean is true, append "1" to output.
  5. If input_boolean is false, append "0" to output.
  6. Return output.

4.1.10. Serializing a Date 📅

dates are just integers with an @ in front, which is honestly iconic 👑

  1. Let output be "@".
  2. Append to output the result of running Serializing an Integer with input_date (Section 4.1.4).
  3. Return output.

4.1.11. Serializing a Display String 🫶

display strings are Unicode, so we first turn them into UTF-8 bytes, then percent-encode the risky bytes and leave the rest alone. no cap, this is the most detailed one in the whole section 🥹

  1. If input_sequence is not a sequence of Unicode code points, fail serialization.
  2. Let byte_array be the result of applying UTF-8 encoding (Section 3 of [UTF8]) to input_sequence. If encoding fails, fail serialization.
  3. Let encoded_string be a string containing "%" followed by DQUOTE.
  4. For each byte in byte_array:
    1. If byte is %x25 ("%"), %x22 (DQUOTE), or in the ranges %x00-1f or %x7f-ff:
      1. Append "%" to encoded_string.
      2. Let encoded_byte be the result of applying base16 encoding (Section 8 of [RFC4648]) to byte, with any alphabetic characters converted to lowercase.
      3. Append encoded_byte to encoded_string.
    2. Otherwise, decode byte as an ASCII character and append the result to encoded_string.
  5. Append DQUOTE to encoded_string.
  6. Return encoded_string.

Note that [UTF8] prohibits the encoding of code points between U+D800 and U+DFFF (surrogates); if they occur in input_sequence, serialization will fail. 💀

4.2. Parsing Structured Fields 🔍

okay bestie, parsing is where the edge cases hide, and they can cause interoperability or even security problems. so this section lays out exactly how to do it safely 🛡️

when a receiving implementation parses HTTP fields that are known to be Structured Fields, it is important that care be taken, as there are a number of edge cases that can cause interoperability or even security problems. this section specifies the algorithm for doing so.

Given an array of bytes as input_bytes that represent the chosen field's field-value (which is empty if that field is not present) and field_type (one of "dictionary", "list", or "item"), return the parsed field value.

  1. Convert input_bytes into an ASCII string input_string; if conversion fails, fail parsing.
  2. Discard any leading SP characters from input_string.
  3. If field_type is "list", let output be the result of running Parsing a List (Section 4.2.1) with input_string.
  4. If field_type is "dictionary", let output be the result of running Parsing a Dictionary (Section 4.2.2) with input_string.
  5. If field_type is "item", let output be the result of running Parsing an Item (Section 4.2.3) with input_string.
  6. Discard any leading SP characters from input_string.
  7. If input_string is not empty, fail parsing.
  8. Otherwise, return output.

When generating input_bytes, parsers MUST combine all field lines in the same section (header or trailer) that case-insensitively match the field name into one comma-separated field-value, as per Section 5.2 of [HTTP]; this assures that the entire field value is processed correctly. 🧩

For Lists and Dictionaries, this has the effect of correctly concatenating all of the field's lines, as long as individual members of the top-level data structure are not split across multiple field instances. The parsing algorithms for both types allow tab characters, since these might be used to combine field lines by some implementations. 🌷

Strings split across multiple field lines will have unpredictable results, because one or more commas (with optional whitespace) will become part of the string output by the parser. Since concatenation might be done by an upstream intermediary, the results are not under the control of the serializer or the parser, even when they are both under the control of the same party. 😭

Tokens, Integers, Decimals, and Byte Sequences cannot be split across multiple field lines because the inserted commas will cause parsing to fail. 🙅‍♀️

Parsers MAY fail when processing a field value spread across multiple field lines, when one of those lines does not parse as that field. For example, a parsing handling an Example-String field that's defined as an sf-string is allowed to fail when processing this field section:

Example-String: "foo
Example-String: bar"

If parsing fails, either the entire field value MUST be ignored (i.e., treated as if the field were not present in the section), or alternatively the complete HTTP message MUST be treated as malformed. This is intentionally strict to improve interoperability and safety, and field specifications that use Structured Fields are not allowed to loosen this requirement. 🚫

Note that this requirement does not apply to an implementation that is not parsing the field; for example, an intermediary is not required to strip a failing field from a message before forwarding it. 🦋

4.2.1. Parsing a List 📜

Given an ASCII string as input_string, return an array of (item_or_inner_list, parameters) tuples. input_string is modified to remove the parsed value.

  1. Let members be an empty array.
  2. While input_string is not empty:
    1. Append the result of running Parsing an Item or Inner List (Section 4.2.1.1) with input_string to members.
    2. Discard any leading OWS characters from input_string.
    3. If input_string is empty, return members.
    4. Consume the first character of input_string; if it is not ",", fail parsing.
    5. Discard any leading OWS characters from input_string.
    6. If input_string is empty, there is a trailing comma; fail parsing.
  3. No structured data has been found; return members (which is empty). 🫥

4.2.1.1. Parsing an Item or Inner List 🧁

Given an ASCII string as input_string, return the tuple (item_or_inner_list, parameters), where item_or_inner_list can be either a single bare item or an array of (bare_item, parameters) tuples. input_string is modified to remove the parsed value.

  1. If the first character of input_string is "(", return the result of running Parsing an Inner List (Section 4.2.1.2) with input_string.
  2. Return the result of running Parsing an Item (Section 4.2.3) with input_string.

4.2.1.2. Parsing an Inner List 🧚

okay so this is the inner list parser 💅 Given an ASCII string as input_string, it hands back the tuple (inner_list, parameters), where inner_list is an array of (bare_item, parameters) tuples. It also chews through input_string and removes whatever it parsed 🫶

  1. Consume the first character of input_string; if it is not "(", fail parsing.
  2. Let inner_list be an empty array.
  3. While input_string is not empty:
    1. Discard any leading SP characters from input_string.
    2. If the first character of input_string is ")":
      1. Consume the first character of input_string.
      2. Let parameters be the result of running Parsing Parameters (Section 4.2.3.2) with input_string.
      3. Return the tuple (inner_list, parameters).
    3. Let item be the result of running Parsing an Item (Section 4.2.3) with input_string.
    4. Append item to inner_list.
    5. If the first character of input_string is not SP or ")", fail parsing.
  4. The end of the Inner List was not found; fail parsing.

4.2.2. Parsing a Dictionary 🍓

this one is giving key-value vibes 🌈 Given an ASCII string as input_string, it returns an ordered map whose values are (item_or_inner_list, parameters) tuples. Same deal: input_string gets modified to remove the parsed value.

  1. Let dictionary be an empty, ordered map.
  2. While input_string is not empty:
    1. Let this_key be the result of running Parsing a Key (Section 4.2.3.3) with input_string.
    2. If the first character of input_string is "=":
      1. Consume the first character of input_string.
      2. Let member be the result of running Parsing an Item or Inner List (Section 4.2.1.1) with input_string.
    3. Otherwise:
      1. Let value be Boolean true.
      2. Let parameters be the result of running Parsing Parameters (Section 4.2.3.2) with input_string.
      3. Let member be the tuple (value, parameters).
    4. If dictionary already contains a key this_key (comparing character for character), overwrite its value with member.
    5. Otherwise, append key this_key with value member to dictionary.
    6. Discard any leading OWS characters from input_string.
    7. If input_string is empty, return dictionary.
    8. Consume the first character of input_string; if it is not ",", fail parsing.
    9. Discard any leading OWS characters from input_string.
    10. If input_string is empty, there is a trailing comma; fail parsing.
  3. No structured data has been found; return dictionary (which is empty).

Note that when duplicate Dictionary keys are encountered, all but the last instance are ignored. so the last one wins, sorry earlier copies 🙈

4.2.3. Parsing an Item 🎀

Given an ASCII string as input_string, this returns a (bare_item, parameters) tuple, and input_string gets modified to remove the parsed value.

  1. Let bare_item be the result of running Parsing a Bare Item (Section 4.2.3.1) with input_string.
  2. Let parameters be the result of running Parsing Parameters (Section 4.2.3.2) with input_string.
  3. Return the tuple (bare_item, parameters).

4.2.3.1. Parsing a Bare Item 💜

Given an ASCII string as input_string, this returns a bare Item. Again, input_string is modified to remove the parsed value. The first character decides which parser takes over, so it's basically a bouncer at the door 🚪

  1. If the first character of input_string is a "-" or a DIGIT, return the result of running Parsing an Integer or Decimal (Section 4.2.4) with input_string.
  2. If the first character of input_string is a DQUOTE, return the result of running Parsing a String (Section 4.2.5) with input_string.
  3. If the first character of input_string is an ALPHA or "*", return the result of running Parsing a Token (Section 4.2.6) with input_string.
  4. If the first character of input_string is ":", return the result of running Parsing a Byte Sequence (Section 4.2.7) with input_string.
  5. If the first character of input_string is "?", return the result of running Parsing a Boolean (Section 4.2.8) with input_string.
  6. If the first character of input_string is "@", return the result of running Parsing a Date (Section 4.2.9) with input_string.
  7. If the first character of input_string is "%", return the result of running Parsing a Display String (Section 4.2.10) with input_string.
  8. Otherwise, the item type is unrecognized; fail parsing.

4.2.3.2. Parsing Parameters ✨

Given an ASCII string as input_string, this returns an ordered map whose values are bare Items. input_string gets modified to remove the parsed value.

  1. Let parameters be an empty, ordered map.
  2. While input_string is not empty:
    1. If the first character of input_string is not ";", exit the loop.
    2. Consume the ";" character from the beginning of input_string.
    3. Discard any leading SP characters from input_string.
    4. Let param_key be the result of running Parsing a Key (Section 4.2.3.3) with input_string.
    5. Let param_value be Boolean true.
    6. If the first character of input_string is "=":
      1. Consume the "=" character at the beginning of input_string.
      2. Let param_value be the result of running Parsing a Bare Item (Section 4.2.3.1) with input_string.
    7. If parameters already contains a key param_key (comparing character for character), overwrite its value with param_value.
    8. Otherwise, append key param_key with value param_value to parameters.
  3. Return parameters.

Note that when duplicate parameter keys are encountered, all but the last instance are ignored.

4.2.3.3. Parsing a Key 🦋

Given an ASCII string as input_string, this returns a key, and input_string gets modified to remove the parsed value.

  1. If the first character of input_string is not lcalpha or "*", fail parsing.
  2. Let output_string be an empty string.
  3. While input_string is not empty:
    1. If the first character of input_string is not one of lcalpha, DIGIT, "_", "-", ".", or "*", return output_string.
    2. Let char be the result of consuming the first character of input_string.
    3. Append char to output_string.
  4. Return output_string.

4.2.4. Parsing an Integer or Decimal 🔥

Given an ASCII string as input_string, this returns an Integer or Decimal, and input_string is modified to remove the parsed value.

NOTE: This algorithm parses both Integers (Section 3.3.1) and Decimals (Section 3.3.2), and returns the corresponding structure. one parser, two number types, very efficient 🧠

  1. Let type be "integer".
  2. Let sign be 1.
  3. Let input_number be an empty string.
  4. If the first character of input_string is "-", consume it and set sign to -1.
  5. If input_string is empty, there is an empty integer; fail parsing.
  6. If the first character of input_string is not a DIGIT, fail parsing.
  7. While input_string is not empty:
    1. Let char be the result of consuming the first character of input_string.
    2. If char is a DIGIT, append it to input_number.
    3. Else, if type is "integer" and char is ".":
      1. If input_number contains more than 12 characters, fail parsing.
      2. Otherwise, append char to input_number and set type to "decimal".
    4. Otherwise, prepend char to input_string, and exit the loop.
    5. If type is "integer" and input_number contains more than 15 characters, fail parsing.
    6. If type is "decimal" and input_number contains more than 16 characters, fail parsing.
  8. If type is "integer":
    1. Let output_number be an Integer that is the result of parsing input_number as an integer.
  9. Otherwise:
    1. If the final character of input_number is ".", fail parsing.
    2. If the number of characters after "." in input_number is greater than three, fail parsing.
    3. Let output_number be a Decimal that is the result of parsing input_number as a decimal number.
  10. Let output_number be the product of output_number and sign.
  11. Return output_number.

4.2.5. Parsing a String 💅

Given an ASCII string as input_string, this returns an unquoted String, and input_string is modified to remove the parsed value.

  1. Let output_string be an empty string.
  2. If the first character of input_string is not DQUOTE, fail parsing.
  3. Discard the first character of input_string.
  4. While input_string is not empty:
    1. Let char be the result of consuming the first character of input_string.
    2. If char is a backslash ("\"):
      1. If input_string is now empty, fail parsing.
      2. Let next_char be the result of consuming the first character of input_string.
      3. If next_char is not DQUOTE or "\", fail parsing.
      4. Append next_char to output_string.
    3. Else, if char is DQUOTE, return output_string.
    4. Else, if char is in the range %x00-1f or %x7f-ff (i.e., it is not in VCHAR or SP), fail parsing.
    5. Else, append char to output_string.
  5. Reached the end of input_string without finding a closing DQUOTE; fail parsing.

4.2.6. Parsing a Token 🌸

Given an ASCII string as input_string, this returns a Token, and input_string is modified to remove the parsed value.

  1. If the first character of input_string is not ALPHA or "*", fail parsing.
  2. Let output_string be an empty string.
  3. While input_string is not empty:
    1. If the first character of input_string is not in tchar, ":", or "/", return output_string.
    2. Let char be the result of consuming the first character of input_string.
    3. Append char to output_string.
  4. Return output_string.

4.2.7. Parsing a Byte Sequence 🫧

Given an ASCII string as input_string, this returns a Byte Sequence, and input_string is modified to remove the parsed value.

  1. If the first character of input_string is not ":", fail parsing.
  2. Discard the first character of input_string.
  3. If there is not a ":" character before the end of input_string, fail parsing.
  4. Let b64_content be the result of consuming content of input_string up to but not including the first instance of the character ":".
  5. Consume the ":" character at the beginning of input_string.
  6. If b64_content contains a character not included in ALPHA, DIGIT, "+", "/", and "=", fail parsing.
  7. Let binary_content be the result of base64-decoding [RFC4648] b64_content, synthesizing padding if necessary (note the requirements about recipient behavior below). If base64 decoding fails, parsing fails.
  8. Return binary_content.

okay so base64 libraries aren't always strict 😭 Some of them can't reject data that's missing its "=" padding (see [RFC4648], Section 3.2), so parsers SHOULD NOT fail when that padding isn't there, unless they can't be configured to behave that way 🙏

same energy for the pad bits: some base64 implementations can't reject data whose pad bits aren't zero (see [RFC4648], Section 3.5), so parsers SHOULD NOT fail when non-zero pad bits show up, unless they can't be configured to do so 🍃

BUT here's the plot twist 👑 This specification does not relax the requirements in Sections 3.1 and 3.3 of [RFC4648]; therefore, parsers MUST fail on characters outside the base64 alphabet and on line feeds in encoded data. 🚫

4.2.8. Parsing a Boolean 🦋

okay so this one is a Boolean parser and it's giving simple 💅 Given an ASCII string as input_string, you return a Boolean, and input_string gets modified to remove the parsed value.

  1. If the first character of input_string is not ?, fail parsing.
  2. Discard the first character of input_string.
  3. If the first character of input_string matches 1, discard the first character, and return true.
  4. If the first character of input_string matches 0, discard the first character, and return false.
  5. No value has matched; fail parsing.

4.2.9. Parsing a Date 📅

bestie, dates are parsed with a leading @ ✨ Given an ASCII string as input_string, you return a Date, and input_string gets modified to remove the parsed value.

  1. If the first character of input_string is not @, fail parsing.
  2. Discard the first character of input_string.
  3. Let output_date be the result of running Parsing an Integer or Decimal (Section 4.2.4) with input_string.
  4. If output_date is a Decimal, fail parsing.
  5. Return output_date.

4.2.10. Parsing a Display String 🎀

this one is the main character of the parsers 👑 Given an ASCII string as input_string, you return a sequence of Unicode code points, and input_string gets modified to remove the parsed value. It's a little bit of a journey, so hold on tight 😭

  1. If the first two characters of input_string are not % followed by ", fail parsing.
  2. Discard the first two characters of input_string.
  3. Let byte_array be an empty byte array.
  4. While input_string is not empty:
    1. Let char be the result of consuming the first character of input_string.
    2. If char is in the range %x00-1f or %x7f-ff (i.e., it is not in VCHAR or SP), fail parsing.
    3. If char is %:
      1. Let octet_hex be the result of consuming two characters from input_string. If there are not two characters, fail parsing.
      2. If octet_hex contains characters outside the range %x30-39 or %x61-66 (i.e., it is not in 0-9 or lowercase a-f), fail parsing.
      3. Let octet be the result of hex decoding octet_hex (Section 8 of [RFC4648]).
      4. Append octet to byte_array.
    4. If char is DQUOTE:
      1. Let unicode_sequence be the result of decoding byte_array as a UTF-8 string (Section 3 of [UTF8]). Fail parsing if decoding fails.
      2. Return unicode_sequence.
    5. Otherwise, if char is not % or DQUOTE:
      1. Let byte be the result of applying ASCII encoding to char.
      2. Append byte to byte_array.
  5. Reached the end of input_string without finding a closing DQUOTE; fail parsing.

5. IANA Considerations 🌸

so IANA added a little note to the "Hypertext Transfer Protocol (HTTP) Field Name Registry" 📝 Here's exactly what it says:

The "Structured Type" column indicates the type of the field (per
RFC 9651), if any, and may be "Dictionary", "List", or "Item".

Note that field names beginning with characters other than ALPHA
or "*" will not be able to be represented as a Structured Fields
Token and therefore may be incompatible with being mapped into
field values that refer to it.

A new column, "Structured Type", has been added to the registry, and every existing registry entry listed in Table 1 got its indicated Structured Type too ✨

+==========================================+=================+
| Field Name                               | Structured Type |
+==========================================+=================+
| Accept-CH                                | List            |
+------------------------------------------+-----------------+
| Cache-Status                             | List            |
+------------------------------------------+-----------------+
| CDN-Cache-Control                        | Dictionary      |
+------------------------------------------+-----------------+
| Cross-Origin-Embedder-Policy             | Item            |
+------------------------------------------+-----------------+
| Cross-Origin-Embedder-Policy-Report-Only | Item            |
+------------------------------------------+-----------------+
| Cross-Origin-Opener-Policy               | Item            |
+------------------------------------------+-----------------+
| Cross-Origin-Opener-Policy-Report-Only   | Item            |
+------------------------------------------+-----------------+
| Origin-Agent-Cluster                     | Item            |
+------------------------------------------+-----------------+
| Priority                                 | Dictionary      |
+------------------------------------------+-----------------+
| Proxy-Status                             | List            |
+------------------------------------------+-----------------+

Table 1: Existing Fields

6. Security Considerations 🔥

okay bestie, read this part slowly, it's serious 🙈

The size of most types defined by Structured Fields is not limited, which means extremely large fields could be an attack vector (for example, for resource consumption) 💀 To mitigate that, most HTTP implementations limit the sizes of individual fields as well as the overall header or trailer section size.

Parties with the ability to inject new HTTP fields can change the meaning of a Structured Field 😱 In some circumstances that will make parsing fail, but it is not possible to reliably fail in all such circumstances.

The Display String type can convey any possible Unicode code point without sanitization. That means it might contain unassigned code points, control points (including NUL), or noncharacters 🫣 So applications consuming Display Strings need to consider strategies like filtering or escaping untrusted content before displaying it. See [PRECIS] and [UNICODE-SECURITY].

7. References 📚

7.1. Normative References

7.2. Informative References

Appendix A. Frequently Asked Questions 💖

A.1. Why Not JSON? 🙃

okay so the earliest Structured Fields proposals were based on JSON [RFC8259] 😬 The catch is that making JSON fit for HTTP fields meant senders and recipients had to implement extra special handling, which is a lot.

JSON has issues with large numbers and with objects that have duplicate members. There is advice for avoiding these (e.g., [RFC7493]), but it can't be relied upon 🙈

JSON strings are Unicode strings by default, and that causes a bunch of potential interoperability problems (e.g., in comparison) 🌈 You can advise implementers to avoid non-ASCII content where it isn't needed, but that's hard to enforce.

Then there's JSON's ability to nest content to arbitrary depths. The memory that takes might be unsuitable for embedded and other limited server deployments, so it needs to be limited somehow. The thing is, existing JSON implementations have no such limits, and even if a limit gets specified, some field definition will probably find a reason to break it 💀

JSON is so widely adopted and implemented that it's really hard to impose extra constraints across every implementation, and some deployments would fail to enforce them, which hurts interoperability. Basically, if it looks like JSON, people will be tempted to run a JSON parser/serializer on field values 😭

A major goal for Structured Fields is improving interoperability and simplifying implementation, so these concerns led to a format that needs a dedicated parser and serializer 🧚 There were also widely shared feelings that JSON just doesn't "look right" in HTTP fields 💅

Appendix B. Implementation Notes ✨

okay bestie, so a generic implementation of this spec should expose the top-level serialize (Section 4.1) and parse (Section 4.2) functions 💅 They don't have to literally be functions, though. You could build it as an object with methods for each of the top-level types, and that's totally valid 🧚

For interoperability, generic implementations need to be complete and follow the algorithms closely, see Section 1.1 🫶 The community also keeps a shared test suite at <https://github.com/httpwg/structured-field-tests>, so you can check your work. It's giving accountability 😭

Implementers, listen up 🎀 Dictionaries and Parameters are order-preserving maps. Some fields might not mean anything based on ordering, but the order should still be exposed so applications that need it can use it.

Also, keep Tokens and Strings separate! Most languages have built-in types that map pretty well to the other types, but you might need a wrapper "token" object or a parameter on your functions so the two types don't blur together 🦋

The serialization algorithm isn't strictly limited to the data types in Section 3 in every case. For example, Decimals are designed to take broader input and round it to allowed values. Main character energy 👑

Implementations can limit the size of different structures, as long as they respect the minimums defined for each type. When a structure goes over an implementation limit, it fails parsing or serialization. No cap, that's a hard stop 💀

Appendix C. ABNF 🌸

This section uses the Augmented Backus-Naur Form (ABNF) notation [RFC5234] to illustrate the expected syntax of Structured Fields. Real talk though, it can't be used to validate them, because it doesn't capture every requirement 🙈

This section is non-normative. If the ABNF and the parsing algorithms ever disagree, the specified algorithms take precedence. The algorithms win, periodt 💜

sf-list       = list-member *( OWS "," OWS list-member )
list-member   = sf-item / inner-list

inner-list    = "(" *SP [ sf-item *( 1*SP sf-item ) *SP ] ")"
                parameters

parameters    = *( ";" *SP parameter )
parameter     = param-key [ "=" param-value ]
param-key     = key
key           = ( lcalpha / "*" )
                *( lcalpha / DIGIT / "_" / "-" / "." / "*" )
lcalpha       = %x61-7A ; a-z
param-value   = bare-item

sf-dictionary = dict-member *( OWS "," OWS dict-member )
dict-member   = member-key ( parameters / ( "=" member-value ))
member-key    = key
member-value  = sf-item / inner-list

sf-item   = bare-item parameters
bare-item = sf-integer / sf-decimal / sf-string / sf-token
            / sf-binary / sf-boolean / sf-date / sf-displaystring

sf-integer       = ["-"] 1*15DIGIT
sf-decimal       = ["-"] 1*12DIGIT "." 1*3DIGIT
sf-string        = DQUOTE *( unescaped / "%" / bs-escaped ) DQUOTE
sf-token         = ( ALPHA / "*" ) *( tchar / ":" / "/" )
sf-binary        = ":" base64 ":"
sf-boolean       = "?" ( "0" / "1" )
sf-date          = "@" sf-integer
sf-displaystring = "%" DQUOTE *( unescaped / "\" / pct-encoded )
                   DQUOTE

base64       = *( ALPHA / DIGIT / "+" / "/" ) *"="

unescaped    = %x20-21 / %x23-24 / %x26-5B / %x5D-7E
bs-escaped   = "\" ( DQUOTE / "\" )

pct-encoded  = "%" lc-hexdig lc-hexdig
lc-hexdig = DIGIT / %x61-66 ; 0-9, a-f

Appendix D. Changes from RFC 8941 🦋

This revision of the "Structured Field Values for HTTP" specification has made the following changes, bestie:

  • Added the Date Structured Type. (Section 3.3.7) 📅
  • Stopped encouraging use of ABNF in definitions of new Structured Fields. (Section 2) 🙅‍♀️
  • Moved ABNF to an informative appendix. (Appendix C) 📎
  • Added a "Structured Type" column to the "Hypertext Transfer Protocol (HTTP) Field Name Registry". (Section 5) 🗂️
  • Refined parse failure handling. (Section 4.2) 🛠️
  • Added the Display String Structured Type. (Section 3.3.8) 💖

Acknowledgements 🫶

Many thanks to Matthew Kerwin for his detailed feedback and careful consideration during the development of this specification. Absolute legend 👑

Thanks also to Ian Clelland, Roy Fielding, Anne van Kesteren, Kazuho Oku, Evert Pot, Julian Reschke, Martin Thomson, Mike West, and Jeffrey Yasskin for their contributions. Everyone who helped make this happen is iconic ✨

Authors' Addresses 📮

Mark Nottingham
Cloudflare
Prahran VIC
Australia
Email: mnot@mnot.net
URI:   https://www.mnot.net/


Poul-Henning Kamp
The Varnish Cache Project
Email: phk@varnish-cache.org
📦 where this page comes from