RFC 7464

JavaScript Object Notation (JSON) Text Sequences

📅 published February 2015, from ietf

RFC 7464: JavaScript Object Notation (JSON) Text Sequences 🦋

Okay bestie, here's the header scoop 💅 This is RFC 7464, titled JavaScript Object Notation (JSON) Text Sequences. It's from the Internet Engineering Task Force (IETF), it was written by N. Williams (Cryptonector), and it's a Standards Track document dated February 2015. Its ISSN is 2070-1721. The header doesn't list anything it obsoletes or updates, so no drama there ✨

Abstract

This doc explains the JSON text sequence format and its media type, application/json-seq 🎀 A JSON text sequence is any number of JSON texts, all encoded in UTF-8. Each one gets a leading ASCII Record Separator (0x1E) and ends with an ASCII Line Feed (0x0A). Simple, but so powerful 😭

Status of This Memo

This is an Internet Standards Track document 💖

It's a product of the IETF, which means it represents the consensus of the IETF community. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). If you want more on Internet Standards, check out Section 2 of RFC 5741 🌸

For the current status, any errata, and how to send feedback, go to http://www.rfc-editor.org/info/rfc7464 🧚

Copyright Notice

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they stand on the date of publication. Please read those carefully, because they spell out your rights and restrictions with respect to this document 📜 Code Components pulled out of this document must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions. They come without warranty, as described in the Simplified BSD License 🌈

1. Introduction and Motivation .....................................2
   1.1. Conventions Used in This Document ..........................2
2. JSON Text Sequence Format .......................................3
   2.1. JSON Text Sequence Parsing .................................3
   2.2. JSON Text Sequence Encoding ................................4
   2.3. Incomplete/Invalid JSON Texts Need Not Be Fatal ............4
   2.4. Top-Level Values: numbers, true, false, and null ...........5
3. Security Considerations .........................................6
4. IANA Considerations .............................................6
5. Normative References ............................................7
Acknowledgements ...................................................8
Author's Address ...................................................8

1. Introduction and Motivation ✨

JSON ([RFC7159]) is a super handy serialization format, no notes 💅 But when you're serializing a big sequence of values as an array, or a sequence that might be indeterminate in length or never-ending, JSON gets really hard to work with 😩

Picture a sequence of one million values, each maybe one kilobyte once encoded. That's roughly one gigabyte 😳 People often want to process a dataset like that incrementally, so they can start producing results without reading all of it first. The usual JSON way to do that is a "streaming" parser. The problem is those aren't widely available, aren't widely used, and aren't easy to use 🙈

So this doc describes "JSON text sequences" 🎀 These aren't JSON texts themselves. They're made up of (possibly) JSON texts. The best part is that they can be parsed, and produced, incrementally without needing a streaming parser or a streaming encoder 🦋

1.1. Conventions Used in This Document

Here's how the key words work in this document. The words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" get interpreted as described in [RFC2119] 🔥

2. JSON Text Sequence Format 🌸

There are two different sets of ABNF rules for defining JSON text sequences: one for parsers and one for encoders 💜 Having two sets lets parsers recover from sequences where some elements got truncated, for whatever reason. The parser syntax is written in terms of octet strings, which then get interpreted as JSON texts if possible. The encoder syntax, on the other hand, assumes sequence elements are not truncated.

JSON text sequences MUST use UTF-8 encoding. Other encodings of JSON (that's UTF-16 and UTF-32) MUST NOT be used 🚫

2.1. JSON Text Sequence Parsing

The ABNF [RFC5234] for the JSON text sequence parser is in Figure 1 👑

input-JSON-sequence = *(1*RS possible-JSON)
RS = %x1E; "record separator" (RS), see RFC 20
         ; Also known as: Unicode Character INFORMATION SEPARATOR
         ;                TWO (U+001E)
possible-JSON = 1*(not-RS); attempt to parse as UTF-8-encoded
                          ; JSON text (see RFC 7159)
not-RS = %x00-1d / %x1f-ff; any octets other than RS

Figure 1: JSON Text Sequence ABNF

In plain English: you've got a series of octet strings, and each one holds any octet except a record separator (RS) (0x1E) [RFC20]. Every octet string starts with an RS byte. Each octet string in the sequence should be parsed as a JSON text in UTF-8 encoding [RFC3629] ✨

Now, if parsing one of those octet strings as a UTF-8-encoded JSON text fails, the parser SHOULD nonetheless keep going through the rest of the sequence 💖 The parser can report those failures to applications, and the app might then decide to stop parsing the sequence. Multiple consecutive RS octets don't mean empty sequence elements between them, so they can be ignored 🧚

This document doesn't define a way to reliably identify a text sequence by its position. For example, that would matter when you're sending individual array elements as their own text sequences. So for apps where truncation is a possibility, intended sequence elements can get truncated, or even go missing entirely. That means a reference to the nth element would be unreliable 😬

There's no end-of-sequence indicator. Just saying 🫶

2.2. JSON Text Sequence Encoding

The ABNF for the JSON text sequence encoder is in Figure 2 🍓

JSON-sequence = *(RS JSON-text LF)
RS = %x1E; see RFC 20
         ; Also known as: Unicode Character INFORMATION SEPARATOR
         ;                TWO (U+001E)
LF = %x0A; "line feed" (LF), see RFC 20
JSON-text = <given by RFC 7159, using UTF-8 encoding>

Figure 2: JSON Text Sequence ABNF

In prose: you can have any number of JSON texts, each encoded in UTF-8 [RFC3629]. Each one gets preceded by one ASCII RS character and followed by a line feed (LF) 🌈 RS is an ASCII control character, so it can only show up inside JSON strings in escaped form (see [RFC7159]). And since RS can't appear in JSON texts in any other form, RS unambiguously marks the start of every element in the sequence. RS is enough to unambiguously delimit every top-level JSON value type except numbers 👀 That's why each JSON text gets a trailing LF too: it lets you detect truncated JSON texts that are just a number at the top level. More on that in Section 2.4 🦋

JSON text sequence encoders are expected to make sure their sequence elements are properly formed. If the encoder does the JSON text encoding itself, the elements will naturally come out properly formed ✨ If the encoder instead accepts JSON texts that are already encoded, it ought to parse them before adding them to a sequence 💅

Random tip-ish note from the text: on some systems you can type RS with "ctrl-^", and on some systems or apps the right combo might be "ctrl-v ctrl-^". This is handy when you're building a sequence by hand in a text editor 🎀

2.3. Incomplete/Invalid JSON Texts Need Not Be Fatal 💀

Per Section 2.1, JSON text sequence parsers should not abort when an octet string contains a malformed JSON text. Instead, the parser should skip to the next RS 🙌 This can happen where, say, data appended to log files gets truncated by the filesystem (for example, because of a crash or an administrative process termination) 😭

Incremental JSON text parsers can be used too, though of course, failing to parse a given text might happen after the parser has already produced some incremental results 🌸

Sequence parsers should have an option to warn about truncated JSON texts ⭐

2.4. Top-Level Values: numbers, true, false, and null 🌈

Objects, arrays, and strings are self-delimited in JSON texts. Numbers and the values 'true', 'false', and 'null' are not. Only whitespace can delimit those last four kinds of values 🧚

JSON text sequences use 0x0A as a "canary" octet to detect truncation 🐤

Parsers MUST check that any JSON texts that are a top-level number, or that might be 'true', 'false', or 'null', include JSON whitespace after that value (at least one byte matching the "ws" ABNF rule from [RFC7159]). If they don't, the JSON text may have been truncated. And the LF after each JSON text matches the "ws" ABNF rule, so that's how it works out 💖

Parsers MUST drop JSON-text sequence elements that consist of non-self-delimited top-level values that may have been truncated (the ones not delimited by whitespace). Parsers can report such texts as warnings, and optionally include the parsed text and/or the original octet string 📣

Here are some examples 🍓 '<RS>123<RS>' might have been meant to carry the top-level number 1234, but it got truncated. Similarly, '<RS>true<RS>' might have been meant to carry the invalid text 'trueish'. And '<RS>truefalse<RS>' is NOT two top-level values, 'true' and 'false'. It's just not a valid JSON text 🙈

Implementations may produce a value when parsing '<RS>"foo"<RS>', because their JSON text parser might be able to consume bytes incrementally. Since the JSON text here is a self-delimiting top-level value, the parser can produce the result without consuming an additional byte. Implementations like that ought to skip to the next RS byte, and they can report any intervening non-whitespace bytes 👑

Detecting truncation of non-self-delimited sequence elements (numbers, true, false, and null) is only possible when the sequence encoder produces or receives complete JSON texts. Implementations where the sequence encoder isn't also in charge of encoding the individual JSON texts should make sure those JSON texts are complete 🫶

Part 1 of 2 💖 More RFC goodness coming up next! 🌸

3. Security Considerations 🔐✨

Okay bestie, this part is SERIOUS 😭 All the security considerations of JSON [RFC7159] apply here too, so go read those as well 💅. Also, big heads up: this format provides no cryptographic integrity protection of any kind 💀 so nothing in here is checking whether your data got messed with.

As usual, parsers must operate on input that is assumed to be untrusted 🙈. That means parsers must fail gracefully when they get malicious inputs. No crashing, no chaos, just graceful failure 🫶.

Okay, now the lowkey tricky part 👀. Incremental JSON text parsers can hand you partial results and only later say "nope, the rest of this text didn't parse" 😳. So a sequence parser that uses an incremental JSON text parser might treat <RS>"foo"<LF>456<LF><RS> as a sequence with one element ("foo"), while a sequence parser that uses a non-incremental JSON text parser might treat that exact same sequence as being empty 🤯. This mismatch, plus texts that fail to parse and just get ignored, can be used to smuggle data past sequence parsers that don't warn about JSON text failures. Not okay 😤, so don't let your parser quietly ignore failures!!

One more thing that is absolutely iconic and also a little scary 🎀. Repeatedly parsing and re-encoding a JSON text sequence can add (or strip) trailing LF bytes from individual sequence element JSON texts. That can break signature validation 💔. JSON has no canonical form for JSON texts, so the JSON text sequence format doesn't have one either. Periodt 💅.

4. IANA Considerations 📋🌸

The MIME media type for JSON text sequences is application/json-seq. Here is the full registration template, copied exactly, because the details matter ✨:

   The MIME media type for JSON text sequences is application/json-seq.

   Type name: application

   Subtype name: json-seq

   Required parameters: N/A

   Optional parameters: N/A

   Encoding considerations: binary

   Security considerations: See RFC 7464, Section 3.

   Interoperability considerations: Described herein.

   Published specification: RFC 7464.

   Applications that use this media type:

      <https://stedolan.github.io/jq>
      <https://github.com/mapbox/cligj>
      <https://github.com/hildjj/json-text-sequence>

   Fragment identifier considerations: N/A

   Additional information:

   o  Deprecated alias names for this type: N/A

   o  Magic number(s): N/A

   o  File extension(s): N/A

   o  Macintosh file type code(s): N/A

   Person & email address to contact for further information:

      json@ietf.org

   Intended usage: COMMON

   Author: Nicolas Williams (nico@cryptonector.com)

   Change controller: IETF

5. Normative References 📚💜

These are the references this document leans on, listed exactly as written so you can find every single one:

   [RFC20]    Cerf, V., "ASCII format for network interchange", STD 80,
              RFC 20, October 1969,
              <http://www.rfc-editor.org/info/rfc20>.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997,
              <http://www.rfc-editor.org/info/rfc2119>.

   [RFC3629]  Yergeau, F., "UTF-8, a transformation format of ISO
              10646", STD 63, RFC 3629, November 2003,
              <http://www.rfc-editor.org/info/rfc3629>.

   [RFC5234]  Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax
              Specifications: ABNF", STD 68, RFC 5234, January 2008,
              <http://www.rfc-editor.org/info/rfc5234>.

   [RFC7159]  Bray, T., Ed., "The JavaScript Object Notation (JSON) Data
              Interchange Format", RFC 7159, March 2014,
              <http://www.rfc-editor.org/info/rfc7159>.

Acknowledgements 🫶🎀

Big love to everyone who helped make this happen 🥹. Phillip Hallam-Baker proposed using JSON text sequences for logfiles and pointed out the need for resynchronization. Stephen Dolan created <https://github.com/stedolan/jq>, which uses something like JSON text sequences (with LF as the separator between texts on output, and requiring only such whitespace as needed to disambiguate on input) 🍓. Carsten Bormann suggested the use of ASCII RS, and Joe Hildebrand suggested the use of LF in addition to RS for disambiguating top-level number values 🧚. Paul Hoffman shepherded the document 👑. Many others contributed reviews and comments on the JSON Working Group mailing list, which is honestly such a glow-up for a document 🌈.

Author's Address 🦋

   Nicolas Williams
   Cryptonector, LLC

   EMail: nico@cryptonector.com

📦 where this page comes from

ietf: rfc7464 2015-02
💖 girlified by claude-haiku-5-5 on 2026-10-18: a language model's rewrite of the original, under its terms