RFC 8785

JSON Canonicalization Scheme (JCS)

📅 published June 2020, from ietf

RFC 8785: JSON Canonicalization Scheme (JCS) 🎀

Okay bestie, this one is RFC 8785, all about the JSON Canonicalization Scheme (JCS) 💖. Here's the title block tea ☕:

  • Stream: Independent Submission 🦋
  • Category: Informational (so it's not an Internet Standards Track spec, just published for info) ✨
  • Authors: A. Rundgren (Independent), B. Jordan (Broadcom), and S. Erdtman (Spotify AB) 👑
  • Date: June 2020 🌸
  • ISSN: 2070-1721
  • Obsoletes / Updates: the title block doesn't list anything it obsoletes or updates 🙈

Abstract

Okay so, hashing and signing need the data to look exactly the same every single time, or the results won't match up 😭. One fix is to make a canonical version of the data. The cute part is that you can still send the data in its original form over the "wire" 🧚, and the producer and consumer can each canonicalize their own copy and still get consistent results. It's giving reliability 💅.

This document describes the JSON Canonicalization Scheme (JCS) 🎀. It explains how to build a canonical version of JSON data by leaning on the strict serialization rules for JSON primitives that ECMAScript already defines, by keeping the data inside the Internet JSON (I-JSON) subset, and by sorting object properties in a deterministic way ✨.

Status of This Memo

Heads up: this document is not an Internet Standards Track spec 🙈. It's published purely for information 🌈.

This is a contribution to the RFC Series, independent of any other RFC stream 🦋. The RFC Editor picked to publish it at their discretion and makes no statement about how valuable it is for implementation or deployment 💅. Documents the RFC Editor approves for publication are not candidates for any level of Internet Standard, see Section 2 of RFC 7841 📚.

For info on the current status, any errata, and how to give feedback, check https://www.rfc-editor.org/info/rfc8785 🔥.

Copyright (c) 2020 IETF Trust and the persons identified as the document authors. All rights reserved 💜.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as in effect on the date of publication. Please read these carefully, because they spell out your rights and restrictions for this document 🫶.

Table of Contents

   1.  Introduction
   2.  Terminology
   3.  Detailed Operation
     3.1.  Creation of Input Data
     3.2.  Generation of Canonical JSON Data
       3.2.1.  Whitespace
       3.2.2.  Serialization of Primitive Data Types
         3.2.2.1.  Serialization of Literals
         3.2.2.2.  Serialization of Strings
         3.2.2.3.  Serialization of Numbers
       3.2.3.  Sorting of Object Properties
       3.2.4.  UTF-8 Generation
   4.  IANA Considerations
   5.  Security Considerations
   6.  References
     6.1.  Normative References
     6.2.  Informative References
   Appendix A.  ECMAScript Sample Canonicalizer
   Appendix B.  Number Serialization Samples
   Appendix C.  Canonicalized JSON as "Wire Format"
   Appendix D.  Dealing with Big Numbers
   Appendix E.  String Subtype Handling
     E.1.  Subtypes in Arrays
   Appendix F.  Implementation Guidelines
   Appendix G.  Open-Source Implementations
   Appendix H.  Other JSON Canonicalization Efforts
   Appendix I.  Development Portal
   Acknowledgements
   Authors' Addresses

1. Introduction ✨

Alright, so this document is all about the JSON Canonicalization Scheme (JCS) 🎀. It defines how to create a canonical representation of JSON [RFC8259] data by building on the strict serialization methods for JSON primitives that ECMAScript [ECMA-262] defines, by keeping the data inside the I-JSON [RFC7493] subset, and by using deterministic property sorting 🧚. The output is a "hashable" representation of JSON data that cryptographic methods can use 💖. The paragraphs after this one walk through the main design considerations 👑.

Cryptographic operations like hashing and signing need the data to be in an invariant format so they're reliably repeatable 😭. One way to get there is to turn the data into a format with a simple, fixed representation, like base64url [RFC4648] 🌸. That's how JSON Web Signature (JWS) [RFC7515] handled it. Another option is to make a canonical version of the data, which is what the XML signature [XMLDSIG] standard did 💅.

The big advantage of a canonicalizing scheme is that the data can stay in its original form, and that's the whole core idea behind JCS 🦋. In other words, canonicalization lets a JSON object stay a JSON object even after it's signed 🔥. That makes system design, documentation, and logging way simpler 🫶.

To avoid "reinventing the wheel" 🛞, JCS relies on the serialization of JSON primitives (strings, numbers, and literals) as ECMAScript (aka JavaScript) [ECMA-262] defines them, starting with version 6 🌈.

Seasoned XML developers might remember the pain of getting XML signatures to validate 😵. That usually came down to different readings of the super intricate XML canonicalization rules, plus the equally complicated Web Services security standards 💀. JCS should avoid those same problems, and here's why:

  • JSON doesn't have a namespace concept or default values 💜.
  • The data is constrained to the I-JSON [RFC7493] subset. That gets rid of the need for special parsers just to handle canonicalization 🎀.
  • JCS-compatible serialization of JSON primitives is already supported by most web browsers and by Node.js [NODEJS] 🌸.
  • The full JCS spec is supported by multiple open-source implementations (see Appendix G) ✨. Appendix F also has implementation guidelines 📚.

JCS is compatible with some existing systems that rely on JSON canonicalization, like the JSON Web Key (JWK) Thumbprint [RFC7638] and Keybase [KEYBASE] 🔑.

If you're curious about uses outside of cryptography, check out [JSONCOMP] 🧚.

This document is aimed at JSON tool vendors and at designers of JSON-based cryptographic solutions 💅. The reader is assumed to know ECMAScript well, including the "JSON" object 🤓.

2. Terminology 📖

Note that this document is not on the IETF standards track 🙈. Even so, a conformant implementation is supposed to stick to the specified behavior, for security and interoperability reasons 🔐. This text uses BCP 14 to describe that necessary behavior 💖.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here 🎀.

3. Detailed Operation 🛠️

This section covers the details of creating a canonical JSON representation and how JCS handles them 🧚.

Appendix F describes the RECOMMENDED way of adding JCS support to existing JSON tools 🌟.

3.1. Creation of Input Data 📥

Data that's going to be canonically serialized usually comes from one of two places 🦋:

  • Parsing previously generated JSON data 📜.
  • Programmatically creating the data 🤖.

No matter which method you use, the data to be serialized MUST be adapted for I-JSON [RFC7493] formatting 💅. In practice that looks like this:

  • JSON objects MUST NOT exhibit duplicate property names 🙅.
  • JSON string data MUST be expressible as Unicode [UNICODE] 🌈.
  • JSON number data MUST be expressible as IEEE 754 [IEEE754] double-precision values 🔢. If an application needs more precision or longer integers than IEEE 754 double precision offers, it is RECOMMENDED to represent those numbers as JSON strings. Appendix D has the details on how to do that in an interoperable and extensible way 🧁.

There's one more constraint: parsed JSON string data MUST NOT be altered during subsequent serializations 🔒. For more on that, see Appendix E 💜.

Note: Unicode has a feature where certain character sequences can get rearranged, called "Unicode Normalization" [UCNORM] 🙈. JCS-compliant string processing doesn't take that into account. So every component involved in a scheme that depends on JCS MUST preserve Unicode string data "as is" 🫶.

3.2. Generation of Canonical JSON Data 🧁

The subsections below walk you through the steps to create a canonical JSON representation of the data from the previous section ✨.

Appendix A has sample code for an ECMAScript-based canonicalizer that matches the JCS spec 💻.

3.2.1. Whitespace ⬜

Whitespace between JSON tokens MUST NOT be emitted 🚫.

3.2.2. Serialization of Primitive Data Types 🔤

Okay, picture this JSON object getting parsed 👀:

  {
    "numbers": [333333333.33333329, 1E30, 4.50,
                2e-3, 0.000000000000000000000000001],
    "string": "\u20ac$\u000F\u000aA'\u0042\u0022\u005c\\\"\/",
    "literals": [null, true, false]
  }

If that parsed data then gets serialized with a serializer that's compliant with ECMAScript's "JSON.stringify()", the result would be (with a line wrap added just for display purposes) pretty different from the original 😳:

  {"numbers":[333333333.3333333,1e+30,4.5,0.002,1e-27],"string":
  "€$\u000f\nA'B\"\\\\\"/","literals":[null,true,false]}

Why the difference? JSON [RFC8259] is very tolerant about input data, but output data (as ECMAScript defines it) has one fixed representation 🎯. As the example shows, numbers get rounded too 🙃.

The subsections below describe how JCS serializes primitive JSON data types. This part is identical to ECMAScript's version 🪞. In the (unlikely) event that a future ECMAScript version invalidates any of these serialization methods, it'll be up to the developer community to either stick with this specification or write a new one 🔮.

3.2.2.1. Serialization of Literals 🧸

In accordance with JSON [RFC8259], the literals "null", "true", and "false" MUST be serialized as null, true, and false, respectively 💖.

3.2.2.2. Serialization of Strings 🎀

For JSON string data (which includes JSON object property names too), each Unicode code point MUST be serialized as described below (see Section 24.3.2.2 of [ECMA-262]) 💅:

  • If the Unicode value falls within the traditional ASCII control character range (U+0000 through U+001F), it MUST be serialized using lowercase hexadecimal Unicode notation (\uhhhh) unless it's one of the predefined JSON control characters U+0008, U+0009, U+000A, U+000C, or U+000D. Those MUST be serialized as \b, \t, \n, \f, and \r, respectively 🌸.
  • If the Unicode value is outside the ASCII control character range, it MUST be serialized "as is", unless it's equivalent to U+005C (\) or U+0022 ("). Those MUST be serialized as \\ and \", respectively 🦋.

Finally, the resulting sequence of Unicode code points MUST be enclosed in double quotes (") 🫶.

Note: Invalid Unicode data like "lone surrogates" (e.g., U+DEAD) can cause interoperability problems, including broken signatures 💀. So occurrences of such data MUST cause a compliant JCS implementation to terminate with an appropriate error 🚨.

3.2.2.3. Serialization of Numbers

okay so numbers are where it gets spicy 🌶️ ECMAScript is built on the IEEE 754 [IEEE754] double-precision standard for representing JSON number data, so that data MUST be serialized according to Section 7.1.12.1 of [ECMA-262], including the "Note 2" enhancement 🔥

The algorithm itself isn't spelled out in this document, since it's pretty complicated 🙈 If you're implementing JCS-compliant number serialization, Google's implementation in V8 [V8] can be a reference. Ryu [RYU] is another compatible reference implementation, and it's what the JCS open-source Java implementation in Appendix G uses. Appendix B has a set of IEEE 754 sample values with their matching JSON serialization ✨

Note: Since Not a Number (NaN) and Infinity are not permitted in JSON, occurrences of NaN or Infinity MUST cause a compliant JCS implementation to terminate with an appropriate error. 💀

3.2.3. Sorting of Object Properties

so even after the last step cleaned up the primitive types, it still wouldn't count as "canonical" 😭 the reason is that JSON object properties aren't in lexicographic (alphabetical) order yet

Using the sample from Section 3.2.2, a properly canonicalized version should (with a line wrap added for display purposes only) read as:

{"literals":[null,true,false],"numbers":[333333333.3333333,
1e+30,4.5,0.002,1e-27],"string":"€$\u000f\nA'B\"\\\\\"/"}

here are the rules for sorting JSON object properties under JCS, bestie 👑

  • JSON object properties MUST be sorted recursively, which means that JSON child Objects MUST have their properties sorted as well.
  • JSON array data MUST also be scanned for the presence of JSON objects (if an object is found, then its properties MUST be sorted), but array element order MUST NOT be changed.

when a JSON object is about to get its properties sorted, these measures MUST be adhered to 🎀

  • The sorting process is applied to property name strings in their "raw" (unescaped) form. That is, a newline character is treated as U+000A.
  • Property name strings are formatted as arrays of UTF-16 [UNICODE] code units. The sorting is based on pure value comparisons, where code units are treated as unsigned integers, independent of locale settings.
  • Property name strings either have different values at some index that is a valid index for both strings, or their lengths are different, or both. If they have different values at one or more index positions, let k be the smallest such index; then, the string whose value at position k has the smaller value, as determined by using the "<" operator, lexicographically precedes the other string. If there is no index position at which they differ, then the shorter string lexicographically precedes the longer string.

in plain English, property names get sorted in ascending order like this 🦋

        ""
        "a"
        "aa"
        "ab"

the reason the sorting is based on UTF-16 code units is that it maps directly onto the string type in ECMAScript (the stuff in web browsers and Node.js), Java, and .NET 🌈 JSON also only supports escape sequences written as UTF-16 code units, so handling that stuff is kind of a must-know anyway. Systems that store string data some other way will need to convert JSON property name strings into arrays of UTF-16 code units before sorting. That conversion from UTF-8 or UTF-32 to UTF-16 is defined by the Unicode [UNICODE] standard 💜

to check that your JCS sorting is correct, you can use this JSON test data 🧚

{
  "\u20ac": "Euro Sign",
  "\r": "Carriage Return",
  "\ufb33": "Hebrew Letter Dalet With Dagesh",
  "1": "One",
  "\ud83d\ude00": "Emoji: Grinning Face",
  "\u0080": "Control",
  "\u00f6": "Latin Small Letter O With Diaeresis"
}

and this is the expected argument order after the property strings are sorted 💖

"Carriage Return"
"One"
"Control"
"Latin Small Letter O With Diaeresis"
"Euro Sign"
"Emoji: Grinning Face"
"Hebrew Letter Dalet With Dagesh"
Note: For the purpose of obtaining a deterministic property order, sorting of data encoded in UTF-8 or UTF-32 would also work, but the outcome for JSON data like above would differ and thus be incompatible with this specification. However, in practice, property names are rarely defined outside of 7-bit ASCII, making it possible to sort string data in UTF-8 or UTF-32 format without conversion to UTF-16 and still be compatible with JCS. Whether or not this is a viable option depends on the environment JCS is used in. 🙈

3.2.4. UTF-8 Generation

last step! 🎉 to get a platform-independent representation, the result of the previous step MUST be encoded in UTF-8 🫶

applied to the sample in Section 3.2.3, this should give you the following bytes (shown in hexadecimal) 🍓

7b 22 6c 69 74 65 72 61 6c 73 22 3a 5b 6e 75 6c 6c 2c 74 72
75 65 2c 66 61 6c 73 65 5d 2c 22 6e 75 6d 62 65 72 73 22 3a
5b 33 33 33 33 33 33 33 33 33 2e 33 33 33 33 33 33 33 2c 31
65 2b 33 30 2c 34 2e 35 2c 30 2e 30 30 32 2c 31 65 2d 32 37
5d 2c 22 73 74 72 69 6e 67 22 3a 22 e2 82 ac 24 5c 75 30 30
30 66 5c 6e 41 27 42 5c 22 5c 5c 5c 5c 5c 22 2f 22 7d

this data is meant to be used as input to cryptographic methods 🔐

4. IANA Considerations ✨

this document has no IANA actions, so nothing to see here for the registry folks 💅

5. Security Considerations 🔐

okay, this section is SERIOUS 😭 sanity checks on input data are crucial, otherwise you could end up overflowing buffers and messing with the integrity of the whole system. it's giving disaster movie 💀

when JCS is applied to signature schemes like the one in Appendix F, applications MUST perform the following operations before acting upon received data:

  1. Parse the JSON data and verify that it adheres to I-JSON.
  2. Verify the data for correctness according to the conventions defined by the ecosystem where it is to be used. This also includes locating the property holding the signature data.
  3. Verify the signature.

if any of these steps fail, the operation in progress MUST be aborted. no exceptions, periodt 🛑

6. References 📚

6.1. Normative References

6.2. Informative References

Appendix A. ECMAScript Sample Canonicalizer ✨

Okay bestie, here's a little ECMAScript canonicalizer so you can see the JCS core in action 💅 The code is only there to highlight the algorithm, so error handling and UTF-8 generation were skipped on purpose 🫶

     ////////////////////////////////////////////////////////////
     // Since the primary purpose of this code is highlighting //
     // the core of the JCS algorithm, error handling and      //
     // UTF-8 generation were not implemented.                 //
     ////////////////////////////////////////////////////////////
     var canonicalize = function(object) {

         var buffer = '';
         serialize(object);
         return buffer;

         function serialize(object) {
             if (object === null || typeof object !== 'object' ||
                 object.toJSON != null) {
                 /////////////////////////////////////////////////
                 // Primitive type or toJSON, use "JSON"        //
                 /////////////////////////////////////////////////
                 buffer += JSON.stringify(object);

             } else if (Array.isArray(object)) {
                 /////////////////////////////////////////////////
                 // Array - Maintain element order              //
                 /////////////////////////////////////////////////
                 buffer += '[';
                 let next = false;
                 object.forEach((element) => {
                     if (next) {
                         buffer += ',';
                     }
                     next = true;
                     /////////////////////////////////////////
                     // Array element - Recursive expansion //
                     /////////////////////////////////////////
                     serialize(element);
                 });
                 buffer += ']';

             } else {
                 /////////////////////////////////////////////////
                 // Object - Sort properties before serializing //
                 /////////////////////////////////////////////////
                 buffer += '{';
                 let next = false;
                 Object.keys(object).sort().forEach((property) => {
                     if (next) {
                         buffer += ',';
                     }
                     next = true;
                     /////////////////////////////////////////////
                     // Property names are strings, use "JSON"  //
                     /////////////////////////////////////////////
                     buffer += JSON.stringify(property);
                     buffer += ':';
                     //////////////////////////////////////////
                     // Property value - Recursive expansion //
                     //////////////////////////////////////////
                     serialize(object[property]);
                 });
                 buffer += '}';
             }
         }
     };

Appendix B. Number Serialization Samples 🍓

Here's a table of ECMAScript-compatible number serialization samples, edge cases included 🧚 The "IEEE 754" column is the internal ECMAScript representation of the "Number" data type. That's based on the IEEE 754 [IEEE754] standard using 64-bit (double-precision) values, shown here in hexadecimal.

   +==================+===========================+====================+
   |     IEEE 754     |    JSON Representation    |      Comment       |
   +==================+===========================+====================+
   | 0000000000000000 | 0                         | Zero               |
   +------------------+---------------------------+--------------------+
   | 8000000000000000 | 0                         | Minus zero         |
   +------------------+---------------------------+--------------------+
   | 0000000000000001 | 5e-324                    | Min pos number     |
   +------------------+---------------------------+--------------------+
   | 8000000000000001 | -5e-324                   | Min neg number     |
   +------------------+---------------------------+--------------------+
   | 7fefffffffffffff | 1.7976931348623157e+308   | Max pos number     |
   +------------------+---------------------------+--------------------+
   | ffefffffffffffff | -1.7976931348623157e+308  | Max neg number     |
   +------------------+---------------------------+--------------------+
   | 4340000000000000 | 9007199254740992          | Max pos int    (1) |
   +------------------+---------------------------+--------------------+
   | c340000000000000 | -9007199254740992         | Max neg int    (1) |
   +------------------+---------------------------+--------------------+
   | 4430000000000000 | 295147905179352830000     | ~2**68         (2) |
   +------------------+---------------------------+--------------------+
   | 7fffffffffffffff |                           | NaN            (3) |
   +------------------+---------------------------+--------------------+
   | 7ff0000000000000 |                           | Infinity       (3) |
   +------------------+---------------------------+--------------------+
   | 44b52d02c7e14af5 | 9.999999999999997e+22     |                    |
   +------------------+---------------------------+--------------------+
   | 44b52d02c7e14af6 | 1e+23                     |                    |
   +------------------+---------------------------+--------------------+
   | 44b52d02c7e14af7 | 1.0000000000000001e+23    |                    |
   +------------------+---------------------------+--------------------+
   | 444b1ae4d6e2ef4e | 999999999999999700000     |                    |
   +------------------+---------------------------+--------------------+
   | 444b1ae4d6e2ef4f | 999999999999999900000     |                    |
   +------------------+---------------------------+--------------------+
   | 444b1ae4d6e2ef50 | 1e+21                     |                    |
   +------------------+---------------------------+--------------------+
   | 3eb0c6f7a0b5ed8c | 9.999999999999997e-7      |                    |
   +------------------+---------------------------+--------------------+
   | 3eb0c6f7a0b5ed8d | 0.000001                  |                    |
   +------------------+---------------------------+--------------------+
   | 41b3de4355555553 | 333333333.3333332         |                    |
   +------------------+---------------------------+--------------------+
   | 41b3de4355555554 | 333333333.33333325        |                    |
   +------------------+---------------------------+--------------------+
   | 41b3de4355555555 | 333333333.3333333         |                    |
   +------------------+---------------------------+--------------------+
   | 41b3de4355555556 | 333333333.3333334         |                    |
   +------------------+---------------------------+--------------------+
   | 41b3de4355555557 | 333333333.33333343        |                    |
   +------------------+---------------------------+--------------------+
   | becbf647612f3696 | -0.0000033333333333333333 |                    |
   +------------------+---------------------------+--------------------+
   | 43143ff3c1cb0959 | 1424953923781206.2        | Round to even  (4) |
   +------------------+---------------------------+--------------------+

      Table 1: ECMAScript-Compatible JSON Number Serialization Samples

Notes:

(1) Okay so for the most compliance with the ECMAScript "JSON" object, values that are meant to be treated as true integers SHOULD be in the range -9007199254740991 to 9007199254740991. But how you use numbers in your app doesn't change how the JCS algorithm works 🙈

(2) Even though a set of specific integers like 2**68 could be seen as having extended precision, the JCS/ECMAScript number serialization algorithm doesn't take that into account 💀

(3) Values out of range are not permitted in JSON. See Section 3.2.2.3 🚫

(4) This number is exactly 1424953923781206.25, but after the "Note 2" rule from Section 3.2.2.3 it gets truncated and rounded to the closest even value 🎀

For a more exhaustive validation of a JCS number serializer, you may test against a file (currently) available in the development portal (see Appendix I) that has a big set of sample values 📂 Another option is running V8 [V8] as a live reference alongside a program that generates a ton of random IEEE 754 values 🌈

Appendix C. Canonicalized JSON as "Wire Format" 🦋

Since the result of the canonicalization process (see Section 3.2.4) is fully valid JSON, it can also be used as "Wire Format" ✨ It's just an option though, because cryptographic schemes based on JCS would, in most cases, not depend on the externally supplied JSON data already being canonicalized 🫶

Honestly, the ECMAScript standard way of serializing objects with "JSON.stringify()" gives you a more "logical" format, where properties stay in the order they were created or received 💅 Here's an address record that could benefit from that standard serialization:

     {
       "name": "John Doe",
       "address": "2000 Sunset Boulevard",
       "city": "Los Angeles",
       "zip": "90001",
       "state": "CA"
     }

Using canonicalization, the properties above would come out in the order "address", "city", "name", "state", and "zip". That makes the data a bit fuzzy for a human (developer or tech support) to read 😭 Canonicalization also squishes the JSON data into a single line of text, which may be less than ideal for debugging and logging 🙈

Appendix D. Dealing with Big Numbers 👑

There are a few issues with the JSON number type, and they're all shown in this sample object 🌸

     {
       "giantNumber": 1.4e+9999,
       "payMeThis": 26000.33,
       "int64Max": 9223372036854775807
     }

Even though this sample conforms to JSON [RFC8259], applications would normally use different native data types for storing "giantNumber" and "int64Max" 🧠 Also, monetary data like "payMeThis" would presumably not rely on floating-point data types, because of rounding issues with decimal arithmetic 💸

The established way of handling this kind of "overloading" of the JSON number type (at least in an extensible way) is through mapping mechanisms, which tell parsers what to do with different properties based on their name. But that greatly limits the value of using the JSON number type outside its original, somewhat constrained JavaScript context 😬 The ECMAScript "JSON" object doesn't support mappings to the JSON number type either.

So, because of all that, numbers that don't have a natural place in the current JSON ecosystem MUST be wrapped using the JSON string type 🎀 This is pretty much a de facto standard for open systems ✨ It also applies to other data types that JSON doesn't directly support, like "DateTime" objects described in Appendix E 🌈

Using a system that uses the JSON string type, whether programmatic like

     var obj = JSON.parse('{"giantNumber": "1.4e+9999"}');
     var biggie = new BigNumber(obj.giantNumber);

or declarative schemes like OpenAPI [OPENAPI], JCS puts no limits on applications, including when they use ECMAScript 🥰

Appendix E. String Subtype Handling ✨

Okay so JSON only gives you a few data types 💅, which is why people keep stuffing subtypes (dates, big numbers, that kind of thing) into plain JSON strings 🎀. Lowkey genius, until it isn't. Depending on how you parse the JSON, this can cause interoperability problems 😭. Any JCS-compliant app that wants to reach a wider audience MUST deal with this.

Let's say the schema designer gave the property "big" a "BigInt" subtype and "time" a "DateTime" subtype, and "val" is supposed to be a JCS-compliant JSON number. Here's the object:

{
  "time": "2019-01-28T07:45:10Z",
  "big": "055",
  "val": 3.5
}

To parse it, you'd use this ECMAScript statement 🧚:

var object = JSON.parse(JSON_object_featured_as_a_string);

Then you pull out the actual data. For subtypes, you also have to convert the parsed result (an ECMAScript object) into the real type 🦋:

... = new Date(object.time); // Date object
... = BigInt(object.big);    // Big integer
... = object.val;            // JSON/JS number

Heads up 👀: the "BigInt" data type is currently only natively supported by V8 [V8].

If you canonicalize "object" with the sample code in Appendix A, you get this string 💖:

{"big":"055","time":"2019-01-28T07:45:10Z","val":3.5}

Technically correct with respect to JCS, but there's another way to parse JSON data, and it works with ECMAScript too 🙈. Check this out:

// "BigInt" requires the following code to become JSON serializable
BigInt.prototype.toJSON = function() {
    return this.toString();
};

// JSON parsing using a "stream"-based method
var object = JSON.parse(JSON_object_featured_as_a_string,
    (k,v) => k == 'time' ? new Date(v) : k == 'big' ? BigInt(v) : v
);

Run the canonicalizer from Appendix A on this "object" and you get this instead 💀:

{"big":"55","time":"2019-01-28T07:45:10.000Z","val":3.5}
Bestie, look at those strings. The arguments for "big" and "time" have changed from the original, and an application that depends on JCS would presumably fail 😭.

The reason is that stream- and schema-based JSON parsers usually swap the original string argument on the fly for the native subtype, and when that gets serialized again, it can follow a different, platform-dependent pattern 🌈.

So here's the rule 🫶: stream- and schema-based parsing MUST treat subtypes as "pure" (immutable) JSON string types, and do the actual conversion to the designated native type in a separate, later step. In modern platforms like Go, Java, and C#, you can get this done with moderate effort by combining annotations, getters, and setters ✨. Here's a snippet in C#/Json.NET showing part of a class that serializes as a JSON object:

// The "pure" string solution uses a local
// string variable for JSON serialization while
// exposing another type to the application
[JsonProperty("amount")]
private string _amount;

[JsonIgnore]
public decimal Amount {
    get { return decimal.Parse(_amount); }
    set { _amount = value.ToString(); }
}

In the app, "Amount" is accessed like any other property, but in JSON contexts it's actually a quoted string 🎀.

Note: The example above also covers the numeric constraints implied by I-JSON. The C# "decimal" data type is pretty different from IEEE 754 double precision, so it's a whole vibe 👑.

E.1. Subtypes in Arrays

Since the JSON array construct lets you mix any JSON data types you want, you may end up needing custom parsing and serialization code to handle subtypes anyway 💅.

Appendix F. Implementation Guidelines 🧚

The dream scenario is building JCS support straight into JSON serializers (parsers need no changes). Canonicalization would just be another "mode" for a JSON serializer 🌸. Sadly, that's not the case yet. The good news is that JCS support can be added through externally supplied canonicalizer software that acts as a post processor on top of existing JSON serializers ✨. This setup also means the JCS implementer doesn't have to worry about how the underlying data is represented in JSON 💖.

The post processor idea makes signature schemes like this one possible 🔥:

  1. Create the data to be signed.
  2. Serialize the data using existing JSON tools.
  3. Let the external canonicalizer process the serialized data and return canonicalized result data.
  4. Sign the canonicalized data.
  5. Add the resulting signature value to the original JSON data through a designated signature property.
  6. Serialize the completed (now signed) JSON object using existing JSON tools.

A matching verification scheme goes like this 🦋:

  1. Parse the signed JSON data using existing JSON tools.
  2. Read and save the signature value from the designated signature property.
  3. Remove the signature property from the parsed JSON object.
  4. Serialize the remaining JSON data using existing JSON tools.
  5. Let the external canonicalizer process the serialized data and return canonicalized result data.
  6. Verify that the canonicalized data matches the saved signature value using the algorithm and key used for creating the signature.

A canonicalizer like this is basically just a "filter" 🧹, and it could work with a bunch of very different cryptographic schemes 🌈.

If your JSON serializer has JCS support built in, you can drop the serialization step before canonicalization for both processes 👑.

Appendix G. Open-Source Implementations 💜

These open-source implementations have been verified to be compatible with JCS 🫶:

Appendix H. Other JSON Canonicalization Efforts 🍓

There are (and have been) other efforts to create "Canonical JSON" 🌷. Here's a list of URLs for some of them:

Every one of these builds on text-level JSON-to-JSON transformations. The big perk of text-level canonicalization is that it can be neutral to the flavor of JSON you're using 🦋. But these schemes also mean major changes to the JSON parsing process, which is probably the biggest hurdle to adoption 😬. JCS was designed to work with existing JSON tools, at the cost of some JSON and application constraints 💅.

Appendix I. Development Portal 🌈

The JCS specification is being developed right now at https://github.com/cyberphone/ietf-json-canon 🛠️.

JCS source code and a ton of test data live at https://github.com/cyberphone/json-canonicalization 🎀.

Acknowledgements 🫶

Building on ECMAScript number serialization was originally proposed by James Manger 👑. That idea eventually led to adopting the entire ECMAScript serialization scheme for JSON primitives ✨.

Other people who gave valuable input to this specification include Scott Ananian, Tim Bray, Ben Campbell, Adrian Farell, Richard Gibson, Bron Gondwana, John-Mark Gurney, Mike Jones, John Levine, Mark Miller, Matthew Miller, Mark Nottingham, Mike Samuel, Jim Schaad, Robert Tupelo-Schneck, and Michal Wadas 💖.

For real-world concept verification, the software and support for number serialization from Ulf Adams, Tanner Gooding, and Remy Oudompheng was super helpful 🧚.

Authors' Addresses 🦋

Anders Rundgren
Independent
Montpellier
France

Email: anders.rundgren.net@gmail.com
URI:   https://www.linkedin.com/in/andersrundgren/


Bret Jordan
Broadcom
1320 Ridder Park Drive
San Jose, CA 95131
United States of America

Email: bret.jordan@broadcom.com


Samuel Erdtman
Spotify AB
Birger Jarlsgatan 61, 4tr
SE-113 56 Stockholm
Sweden

Email: erdtman@spotify.com

That's the whole spec, bestie! 🌈✨ JSON Canonicalization, done and dusted 💅👑

📦 where this page comes from