RFC 7049
Concise Binary Object Representation (CBOR)
Internet Engineering Task Force (IETF) 🌸
Okay bestie, here's the title block 🎀 This is RFC 7049, titled Concise Binary Object Representation (CBOR) 🦋.
- Authors: C. Bormann (Universitaet Bremen TZI) and P. Hoffman (VPN Consortium) 💜
- Category: Standards Track 👑
- ISSN: 2070-1721
- Date: October 2013 📅
- Obsoletes / Updates: the header doesn't list any RFCs being obsoleted or updated, so there's nothing to fill in there ✨
Abstract 💖
So CBOR (Concise Binary Object Representation) is a data format, and its whole vibe is built around a few design goals. First, it can have an extremely small code size 🤏. Second, the messages come out fairly small 📦. Third, it can be extended without needing version negotiation 🦋. Those goals are what make it different from older binary formats like ASN.1 and MessagePack, and it's honestly iconic 💅.
Status of This Memo 📜
This is an Internet Standards Track document 👑.
This document is a product of the Internet Engineering Task Force (IETF) 🌈. It represents the consensus of the IETF community, it got public review, and the Internet Engineering Steering Group (IESG) approved it for publication ✅. If you want more info on Internet Standards, check Section 2 of RFC 5741 📚.
Info about the current status of this document, any errata, and how to give feedback on it lives at http://www.rfc-editor.org/info/rfc7049 🔗.
Copyright Notice ©️
Copyright (c) 2013 IETF Trust and the persons identified as the document authors. All rights reserved 💜.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they stand on the date of publication. Please read these carefully, because they explain your rights and restrictions for this document 🫶. Code Components pulled out of this document must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions. They also come with no warranty, as the Simplified BSD License describes 🔥.
Table of Contents 🗂️
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Objectives . . . . . . . . . . . . . . . . . . . . . . . 4
1.2. Terminology . . . . . . . . . . . . . . . . . . . . . . . 5
2. Specification of the CBOR Encoding . . . . . . . . . . . . . 6
2.1. Major Types . . . . . . . . . . . . . . . . . . . . . . . 7
2.2. Indefinite Lengths for Some Major Types . . . . . . . . . 9
2.2.1. Indefinite-Length Arrays and Maps . . . . . . . . . . 9
2.2.2. Indefinite-Length Byte Strings and Text Strings . . . 11
2.3. Floating-Point Numbers and Values with No Content . . . . 12
2.4. Optional Tagging of Items . . . . . . . . . . . . . . . . 14
2.4.1. Date and Time . . . . . . . . . . . . . . . . . . . . 16
2.4.2. Bignums . . . . . . . . . . . . . . . . . . . . . . . 16
2.4.3. Decimal Fractions and Bigfloats . . . . . . . . . . . 17
2.4.4. Content Hints . . . . . . . . . . . . . . . . . . . . 18
2.4.4.1. Encoded CBOR Data Item . . . . . . . . . . . . . 18
2.4.4.2. Expected Later Encoding for CBOR-to-JSON
Converters . . . . . . . . . . . . . . . . . . . 18
2.4.4.3. Encoded Text . . . . . . . . . . . . . . . . . . 19
2.4.5. Self-Describe CBOR . . . . . . . . . . . . . . . . . 19
3. Creating CBOR-Based Protocols . . . . . . . . . . . . . . . . 20
3.1. CBOR in Streaming Applications . . . . . . . . . . . . . 20
3.2. Generic Encoders and Decoders . . . . . . . . . . . . . . 21
3.3. Syntax Errors . . . . . . . . . . . . . . . . . . . . . . 21
3.3.1. Incomplete CBOR Data Items . . . . . . . . . . . . . 22
3.3.2. Malformed Indefinite-Length Items . . . . . . . . . . 22
3.3.3. Unknown Additional Information Values . . . . . . . . 23
3.4. Other Decoding Errors . . . . . . . . . . . . . . . . . . 23
3.5. Handling Unknown Simple Values and Tags . . . . . . . . . 24
3.6. Numbers . . . . . . . . . . . . . . . . . . . . . . . . . 24
3.7. Specifying Keys for Maps . . . . . . . . . . . . . . . . 25
3.8. Undefined Values . . . . . . . . . . . . . . . . . . . . 26
3.9. Canonical CBOR . . . . . . . . . . . . . . . . . . . . . 26
3.10. Strict Mode . . . . . . . . . . . . . . . . . . . . . . . 28
4. Converting Data between CBOR and JSON . . . . . . . . . . . . 29
4.1. Converting from CBOR to JSON . . . . . . . . . . . . . . 29
4.2. Converting from JSON to CBOR . . . . . . . . . . . . . . 30
5. Future Evolution of CBOR . . . . . . . . . . . . . . . . . . 31
5.1. Extension Points . . . . . . . . . . . . . . . . . . . . 32
5.2. Curating the Additional Information Space . . . . . . . . 33
6. Diagnostic Notation . . . . . . . . . . . . . . . . . . . . . 33
6.1. Encoding Indicators . . . . . . . . . . . . . . . . . . . 34
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 35
7.1. Simple Values Registry . . . . . . . . . . . . . . . . . 35
7.2. Tags Registry . . . . . . . . . . . . . . . . . . . . . . 35
7.3. Media Type ("MIME Type") . . . . . . . . . . . . . . . . 36
7.4. CoAP Content-Format . . . . . . . . . . . . . . . . . . . 37
7.5. The +cbor Structured Syntax Suffix Registration . . . . . 37
8. Security Considerations . . . . . . . . . . . . . . . . . . . 38
9. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 38
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 39
10.1. Normative References . . . . . . . . . . . . . . . . . . 39
10.2. Informative References . . . . . . . . . . . . . . . . . 40
Appendix A. Examples . . . . . . . . . . . . . . . . . . . . . . 41
Appendix B. Jump Table . . . . . . . . . . . . . . . . . . . . . 45
Appendix C. Pseudocode . . . . . . . . . . . . . . . . . . . . . 48
Appendix D. Half-Precision . . . . . . . . . . . . . . . . . . . 50
Appendix E. Comparison of Other Binary Formats to CBOR's Design
Objectives . . . . . . . . . . . . . . . . . . . . . 51
E.1. ASN.1 DER, BER, and PER . . . . . . . . . . . . . . . . . 52
E.2. MessagePack . . . . . . . . . . . . . . . . . . . . . . . 52
E.3. BSON . . . . . . . . . . . . . . . . . . . . . . . . . . 53
E.4. UBJSON . . . . . . . . . . . . . . . . . . . . . . . . . 53
E.5. MSDTP: RFC 713 . . . . . . . . . . . . . . . . . . . . . 53
E.6. Conciseness on the Wire . . . . . . . . . . . . . . . . . 531. Introduction ✨
Okay so there are hundreds of standardized formats for binary representation of structured data (also called binary serialization formats) 🤯. Some are made for specific domains of information, and others are generalized for arbitrary data. In the IETF, the best-known ones in the general category are ASN.1's BER and DER [ASN.1] 🍓.
CBOR follows some specific design goals that current formats don't really nail 💅. Its underlying data model is an extended version of the JSON data model [RFC4627]. Important note: this is not a proposal to extend the grammar in RFC 4627 in general, because that would cause a big backwards incompatibility with JSON documents that are already out there 🙈. Instead, this document just defines its own data model that starts from JSON 🧚.
Appendix E lists some existing binary formats and talks about how well they do, or don't, fit the design objectives of the Concise Binary Object Representation (CBOR) 🌈.
1.1. Objectives 🎯
The objectives of CBOR are listed roughly in decreasing order of importance. Here they are, bestie 🫶:
- The representation must be able to unambiguously encode most common data formats used in Internet standards 💜.
- It must represent a reasonable set of basic data types and structures using binary encoding. "Reasonable" here is mostly shaped by what JSON can do, plus the major addition of binary byte strings 🦋. The structures supported are limited to arrays and trees; loops and lattice-style graphs are not supported 🙅♀️.
- There is no requirement that all data formats be uniquely encoded. That means it's totally acceptable for the number "7" to be encoded in multiple different ways 🤷♀️.
- The code for an encoder or decoder must be able to be compact, so it can run on systems with very limited memory, processor power, and instruction sets 🔋.
- An encoder and a decoder need to be implementable in a very small amount of code (for example, in class 1 constrained nodes as defined in [CNN-TERMS]) 🤏.
- The format should use contemporary machine representations of data (for example, not requiring binary-to-decimal conversion) 🖥️.
- Data must be able to be decoded without a schema description 🧠.
- Similar to JSON, encoded data should be self-describing so that a generic decoder can be written 👑.
- The serialization must be reasonably compact, but data compactness is secondary to code compactness for the encoder and decoder 📦.
- "Reasonable" here is bounded by JSON as an upper bound in size, and by implementation complexity as a lower bound. Using either general compression schemes or extensive bit-fiddling violates the complexity goals 🙅♀️.
- The format must be applicable to both constrained nodes and high-volume applications 🔥.
- This means it must be reasonably frugal in CPU usage for both encoding and decoding. This is relevant both for constrained nodes and for potential usage in applications with a very high volume of data 💨.
- The format must support all JSON data types for conversion to and from JSON 🔁.
- It must support a reasonable level of conversion as long as the data represented is within the capabilities of JSON. It must be possible to define a unidirectional mapping towards JSON for all types of data 🌸.
- The format must be extensible, and the extended data must be decodable by earlier decoders 🦋.
- The format is designed for decades of use 🕰️.
- The format must support a form of extensibility that allows fallback, so that a decoder that does not understand an extension can still decode the message 🛟.
- The format must be able to be extended in the future by later IETF standards ⭐.
1.2. Terminology ✨
okay so first, the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document mean what RFC 2119, BCP 14 [RFC2119] says they mean. They're the requirement levels for compliant CBOR implementations, so they're kinda the rules of the game 🎀
The word "byte" is used in its now-customary sense as a synonym for "octet" 🦋 And all the multi-byte values get encoded in network byte order, which is most significant byte first, also known as "big-endian" 💖
Here's the vocab the spec uses:
- Data item: a single piece of CBOR data 🧚 A data item can hold zero, one, or more nested data items inside it. The term covers both the data item in its representation format and the abstract idea a decoder can pull out of it 🌸
- Decoder: a process that decodes a CBOR data item and makes it available to an application. Formally, a decoder has a parser that breaks up the input using CBOR's syntax rules, plus a semantic processor that gets the data ready for the application 💅
- Encoder: a process that generates the representation format of a CBOR data item from application information 🍓
- Data Stream: a sequence of zero or more data items that isn't further assembled into a bigger containing data item. The independent data items in a data stream are sometimes called "top-level data items" 🌈
- Well-formed: a data item that follows CBOR's syntactic structure. A well-formed data item uses the initial bytes, and the byte strings and/or data items that those values imply, as CBOR defines them, and it isn't followed by extraneous data 🫶
- Valid: a data item that is well-formed AND also follows the semantic restrictions that apply to CBOR data items 👑
- Stream decoder: a process that decodes a data stream and makes each data item in the sequence available to an application as it's received ⭐
When the spec explains bit arithmetic or data types, it uses the notation you'd know from the C programming language, with one twist: ** means exponentiation 🧠 Binary numbers get the 0b prefix, kinda like how 0x marks hexadecimal. You can add underscores to a binary number purely for readability. So 0b00100001 (0x21) might be written 0b001_00001 to show how the bits in the byte are meant to be read. In that example it's split into a three-bit part and a five-bit part 🦋
2. Specification of the CBOR Encoding 💜
a CBOR-encoded data item is structured and encoded the way this section describes ✨ The whole encoding is summed up in Table 5 💅
The initial byte of each data item holds two things. The high-order 3 bits are the major type (see Section 2.1), and the low-order 5 bits are the additional information 🎀 If the additional information is less than 24, it's used directly as a small unsigned integer. If it's 24 to 27, the additional bytes for a variable-length integer come right after it. Values 24 to 27 mean that length is a 1-, 2-, 4-, or 8-byte unsigned integer, respectively. Additional information value 31 is for indefinite-length items, which Section 2.2 covers. And additional information values 28 to 30 are reserved for future expansion 🔮
For all additional information values, the resulting integer gets interpreted based on the major type. Sometimes it's the actual data. For example, in integer types, the integer is the value itself 🌸 Other times it gives length info. For example, in byte strings it tells you the length of the byte string data that follows 🫧
A CBOR decoder can be built on a jump table covering all 256 defined values for the initial byte (Table 5) 🦋 A decoder in a constrained implementation can instead use the structure of the initial byte and the bytes after it, for more compact code. Appendix C has a rough impression of how that could look 🧚
2.1. Major Types 🍓
Here are the major types, along with the additional information and other bytes that go with each one 💖
- Major type 0: an unsigned integer. The 5-bit additional information is either the integer itself (for additional information values 0 through 23) or the length of additional data. Additional information 24 means the value is represented in an additional
uint8_t, 25 means auint16_t, 26 means auint32_t, and 27 means auint64_t. For example, the integer 10 is written as the one byte0b000_01010(major type 0, additional information 10). The integer 500 would be0b000_11001(major type 0, additional information 25) followed by the two bytes0x01f4, which is 500 in decimal 🌈 - Major type 1: a negative integer. It follows the rules for unsigned integers (major type 0), except the value is then -1 minus the encoded unsigned integer. For example, the integer -500 would be
0b001_11001(major type 1, additional information 25) followed by the two bytes0x01f3, which is 499 in decimal 🙈 - Major type 2: a byte string. The string's length in bytes is represented using the rules for positive integers (major type 0). For example, a byte string whose length is 5 would have an initial byte of
0b010_00101(major type 2, additional information 5 for the length), followed by 5 bytes of binary content. A byte string whose length is 500 would have 3 initial bytes:0b010_11001(major type 2, additional information 25 to indicate a two-byte length), followed by the two bytes0x01f4for a length of 500, and then 500 bytes of binary content 🫶 - Major type 3: a text string, specifically a string of Unicode characters encoded as UTF-8 [RFC3629]. This type has the same format as byte strings (major type 2). That means the length gives the number of bytes, not characters. This type exists for systems that need to interpret or display human-readable text. It lets you tell apart unstructured bytes from text that has a specified repertoire and encoding 📖 Unlike formats such as JSON, the Unicode characters in this type are never escaped. So a newline character (U+000A) is always represented in a string as the byte
0x0a, and never as the bytes0x5c6e(the characters\andn) or as0x5c7530303061(the characters\,u,0,0,0, anda) 💅 - Major type 4: an array of data items. Arrays are also called lists, sequences, or tuples. The array's length follows the rules for byte strings (major type 2), except the length counts the number of data items, not the length in bytes the array takes up. Items in an array don't all have to be the same type 🍓 For example, an array that contains 10 items of any type would have an initial byte of
0b100_01010(major type 4, additional information 10 for the length), followed by the 10 remaining items ✨ - Major type 5: a map of pairs of data items. Maps are also called tables, dictionaries, hashes, or objects (in JSON). A map is made of pairs of data items, where each pair is a key immediately followed by a value. The map's length follows the rules for byte strings (major type 2), except it counts the number of pairs, not the bytes the map takes up. For example, a map that contains 9 pairs would have an initial byte of
0b101_01001(major type 5, additional information 9 for the number of pairs), followed by the 18 remaining items. The first item is the first key, the second item is the first value, the third item is the second key, and so on. A map with duplicate keys may be well-formed, but it is not valid, and so it causes indeterminate decoding. See also Section 3.7 🙃 - Major type 6: optional semantic tagging of other major types. See Section 2.4 🌸
- Major type 7: floating-point numbers and simple data types that need no content, plus the "break" stop code. See Section 2.3 🦋
These eight major types make a simple table showing which of the 256 possible values for the initial byte of a data item are used (Table 5) 🧚
In major types 6 and 7, a lot of the possible values are reserved for future specification. Section 7 has more info on these values 🔮
2.2. Indefinite Lengths for Some Major Types 🌈
Four CBOR items can be encoded with an indefinite length using additional information value 31: arrays, maps, byte strings, and text strings 💖 This is handy when the encoding of the item has to start before you know how many items are in the array or map, or the total length of the string. (People often call this "streaming" within a data item 🫧)
Indefinite-length arrays and maps get handled differently from indefinite-length byte strings and text strings 💜
2.2.1. Indefinite-Length Arrays and Maps ✨
Okay so indefinite-length arrays and maps are literally just opened without saying how many items are coming 🎀. You use the additional information value of 31 and that's it. The initial major type and additional information byte gets followed by the elements, just like they would in any other array or map 💅. The end is marked with a "break" stop code, placed wherever the next data item would normally go. The "break" is encoded with major type 7 and additional information value 31 (0b111_11111), but it's not a data item itself. It's just a syntactic feature that closes the array or map 😭. So the "break" comes after the last item, and it can't show up anywhere else in place of a data item. Basically, indefinite-length arrays and maps look identical to regular ones, except they start with 31 and end with the "break" stop code. Iconic 🦋
Arrays and maps with indefinite lengths can have any number of items (for arrays) or key/value pairs (for maps) before the "break" stop code 🍓. Nesting is totally allowed, with no restriction at all. The thing to remember is that a "break" only ends a single item. So nested indefinite-length items need exactly as many "break" stop codes as there are type bytes starting an indefinite-length item 🧚.
Let's say an encoder wants to represent the abstract array [1, [2, 3], [4, 5]]. The definite-length encoding would be 0x8301820203820405:
83 -- Array of length 3
01 -- 1
82 -- Array of length 2
02 -- 2
03 -- 3
82 -- Array of length 2
04 -- 4
05 -- 5Indefinite-length encoding can be applied independently to each of the three arrays in this data item, as required, which leads to representations like these:
0x9f018202039f0405ffff
9F -- Start indefinite-length array
01 -- 1
82 -- Array of length 2
02 -- 2
03 -- 3
9F -- Start indefinite-length array
04 -- 4
05 -- 5
FF -- "break" (inner array)
FF -- "break" (outer array)0x9f01820203820405ff
9F -- Start indefinite-length array
01 -- 1
82 -- Array of length 2
02 -- 2
03 -- 3
82 -- Array of length 2
04 -- 4
05 -- 5
FF -- "break"0x83018202039f0405ff
83 -- Array of length 3
01 -- 1
82 -- Array of length 2
02 -- 2
03 -- 3
9F -- Start indefinite-length array
04 -- 4
05 -- 5
FF -- "break"0x83019f0203ff820405
83 -- Array of length 3
01 -- 1
9F -- Start indefinite-length array
02 -- 2
03 -- 3
FF -- "break"
82 -- Array of length 2
04 -- 4
05 -- 5Here's an indefinite-length map that happens to have two key/value pairs 🌸:
0xbf6346756ef563416d7421ff
BF -- Start indefinite-length map
63 -- First key, UTF-8 string length 3
46756e -- "Fun"
F5 -- First value, true
63 -- Second key, UTF-8 string length 3
416d74 -- "Amt"
21 -- -2
FF -- "break"2.2.2. Indefinite-Length Byte Strings and Text Strings 🫶
Indefinite-length byte strings and text strings are basically a bunch of zero or more definite-length byte or text strings (called "chunks") that together get treated as one contiguous string 🧩. They open with the major type and additional information value of 31, and then comes a series of byte or text strings with definite lengths (the chunks). The series ends with the "break" stop code (0b111_11111) wherever the next chunk would have gone. The chunks get concatenated together, and the overall length of the string is the sum of the lengths of all the chunks ✨. So an indefinite-length string is encoded similarly to an indefinite-length array of its chunks, except the major type is that of a (text or byte) string and matches the major types of its chunks. Cute, right? 🎀
For indefinite-length byte strings, every data item (chunk) between the indefinite-length indicator and the "break" MUST be a definite-length byte string item. If the parser sees any other item type before it sees the "break", that's an error 🚨.
Let's say we have this sequence:
0b010_11111 0b010_00100 0xaabbccdd 0b010_00011 0xeeff99 0b111_111115F -- Start indefinite-length byte string
44 -- Byte string of length 4
aabbccdd -- Bytes content
43 -- Byte string of length 3
eeff99 -- Bytes content
FF -- "break"After decoding, this turns into a single byte string with seven bytes: 0xaabbccddeeff99 💖.
Text strings with indefinite lengths act the same as byte strings with indefinite lengths, except all their chunks MUST be definite-length text strings 💜. This means the bytes of a single UTF-8 character can't be split across chunks. A new chunk can only start at a character boundary 🔥.
2.3. Floating-Point Numbers and Values with No Content 🌈
Major type 7 is for two kinds of data: floating-point numbers, and "simple values" that don't need any content at all 🧚. Each value of the 5-bit additional information in the initial byte has its own separate meaning, as defined in Table 1. Like the major types for integers, items of this major type don't carry any content data. All the information is in the initial bytes 💅.
+-------------+--------------------------------------------------+
| 5-Bit Value | Semantics |
+-------------+--------------------------------------------------+
| 0..23 | Simple value (value 0..23) |
| | |
| 24 | Simple value (value 32..255 in following byte) |
| | |
| 25 | IEEE 754 Half-Precision Float (16 bits follow) |
| | |
| 26 | IEEE 754 Single-Precision Float (32 bits follow) |
| | |
| 27 | IEEE 754 Double-Precision Float (64 bits follow) |
| | |
| 28-30 | (Unassigned) |
| | |
| 31 | "break" stop code for indefinite-length items |
+-------------+--------------------------------------------------+
Table 1: Values for Additional Information in Major Type 7Just like with all the other major types, the 5-bit value 24 signals a single-byte extension. It's followed by an additional byte that represents the simple value. (To keep things less confusing, only the values 32 to 255 are used 🙈.) This keeps the structure of the initial bytes intact. As with the other major types, the length of these always depends on the additional information in the first byte. Table 2 lists the values assigned and available for simple types 🍓.
+---------+-----------------+
| Value | Semantics |
+---------+-----------------+
| 0..19 | (Unassigned) |
| | |
| 20 | False |
| | |
| 21 | True |
| | |
| 22 | Null |
| | |
| 23 | Undefined value |
| | |
| 24..31 | (Reserved) |
| | |
| 32..255 | (Unassigned) |
+---------+-----------------+
Table 2: Simple ValuesThe 5-bit values 25, 26, and 27 are for 16-bit, 32-bit, and 64-bit IEEE 754 binary floating-point values 🦋. These floating-point values are encoded in the additional bytes of the appropriate size. (See Appendix D for some info about 16-bit floating point 👑.)
2.4. Optional Tagging of Items ✨
okay so in CBOR, a data item can optionally get a tag 🎀 to give it extra meaning while keeping its structure intact. The tag is major type 6, and its integer value tells you which tag it is. The (sole) data item that follows is carried as the content data 💅. If a tag needs structured data, that structure gets encoded into the nested data item. The definition of a tag usually restricts what kinds of nested data item or items it can carry, so it's not a free-for-all 🙈
The initial bytes of the tag follow the rules for positive integers (major type 0). The tag is then followed by a single data item of any type. For example, say a byte string of length 12 gets marked with a tag to say it's a positive bignum (Section 2.4.2). That would be 0b110_00010 (major type 6, additional information 2 for the tag), followed by 0b010_01100 (major type 2, additional information of 12 for the length), followed by the 12 bytes of the bignum 🔥
Decoders don't have to understand tags 🧚 so tags might not be super useful when the app creating a CBOR data item and the app decoding the stream already know what every item means. Their main job in this spec is to define common data types like dates. A secondary perk is optional tagging for when the decoder is a generic CBOR decoder that could benefit from hints about what the items contain. Understanding the semantic tags is optional for a decoder. It can just hop over the tag's initial bytes and interpret the tagged data item itself 💖
A tag always applies to the item directly after it. So if tag A is followed by tag B, which is followed by data item C, tag A applies to the result of applying tag B to data item C. Basically, a tagged item is a data item made of a tag and a value. The content of the tagged item is the data item (the value) being tagged 🦋
IANA keeps a registry of tag values, as described in Section 7.2. Table 3 lists the initial values, and the rest of this section explains their definitions 🌈
+--------------+------------------+---------------------------------+
| Tag | Data Item | Semantics |
+--------------+------------------+---------------------------------+
| 0 | UTF-8 string | Standard date/time string; see |
| | | Section 2.4.1 |
| | | |
| 1 | multiple | Epoch-based date/time; see |
| | | Section 2.4.1 |
| | | |
| 2 | byte string | Positive bignum; see Section |
| | | 2.4.2 |
| | | |
| 3 | byte string | Negative bignum; see Section |
| | | 2.4.2 |
| | | |
| 4 | array | Decimal fraction; see Section |
| | | 2.4.3 |
| | | |
| 5 | array | Bigfloat; see Section 2.4.3 |
| | | |
| 6..20 | (Unassigned) | (Unassigned) |
| | | |
| 21 | multiple | Expected conversion to |
| | | base64url encoding; see |
| | | Section 2.4.4.2 |
| | | |
| 22 | multiple | Expected conversion to base64 |
| | | encoding; see Section 2.4.4.2 |
| | | |
| 23 | multiple | Expected conversion to base16 |
| | | encoding; see Section 2.4.4.2 |
| | | |
| 24 | byte string | Encoded CBOR data item; see |
| | | Section 2.4.4.1 |
| | | |
| 25..31 | (Unassigned) | (Unassigned) |
| | | |
| 32 | UTF-8 string | URI; see Section 2.4.4.3 |
| | | |
| 33 | UTF-8 string | base64url; see Section 2.4.4.3 |
| | | |
| 34 | UTF-8 string | base64; see Section 2.4.4.3 |
| | | |
| 35 | UTF-8 string | Regular expression; see |
| | | Section 2.4.4.3 |
| | | |
| 36 | UTF-8 string | MIME message; see Section |
| | | 2.4.4.3 |
| | | |
| 37..55798 | (Unassigned) | (Unassigned) |
| | | |
| 55799 | multiple | Self-describe CBOR; see |
| | | Section 2.4.5 |
| | | |
| 55800+ | (Unassigned) | (Unassigned) |
+--------------+------------------+---------------------------------+
Table 3: Values for Tags2.4.1. Date and Time 🕰️
Tag value 0 is for date/time strings that follow the standard format described in [RFC3339], as refined by Section 3.3 of [RFC4287].
Tag value 1 is for a numerical representation of seconds relative to 1970-01-01T00:00Z in UTC time. (For the non-negative values that the Portable Operating System Interface (POSIX) defines, the number of seconds is counted the same way as POSIX "seconds since the epoch" [TIME_T].) The tagged item can be a positive or negative integer (major types 0 and 1), or a floating-point number (major type 7 with additional information 25, 26, or 27). Note that the number can be negative (a time before 1970-01-01T00:00Z) and, if it's a floating-point number, it can indicate fractional seconds 🌸
2.4.2. Bignums 🔢
Bignums are integers that don't fit into the basic integer representations provided by major types 0 and 1. They're encoded as a byte string data item, which is interpreted as an unsigned integer n in network byte order. For tag value 2, the value of the bignum is n. For tag value 3, the value of the bignum is -1 - n. Decoders that understand these tags MUST be able to decode bignums that have leading zeroes. Non-negotiable 😤
For example, the number 18446744073709551616 (2**64) is represented as 0b110_00010 (major type 6, tag 2), followed by 0b010_01001 (major type 2, length 9), followed by 0x010000000000000000 (one byte 0x01 and eight bytes 0x00). In hexadecimal:
C2 -- Tag 2
29 -- Byte string of length 9
010000000000000000 -- Bytes content2.4.3. Decimal Fractions and Bigfloats 🧮
Decimal fractions combine an integer mantissa with a base-10 scaling factor. They're most useful when an application needs the exact representation of a decimal fraction such as 1.1, because lots of decimal fractions have no exact representation in binary floating point 🙃
Bigfloats combine an integer mantissa with a base-2 scaling factor. They're binary floating-point values that can exceed the range or the precision of the three IEEE 754 formats supported by CBOR (Section 2.3). Bigfloats can also be used by constrained applications that need some basic binary floating-point capability without having to support IEEE 754 💜
A decimal fraction or a bigfloat is represented as a tagged array that contains exactly two integer numbers: an exponent e and a mantissa m. Decimal fractions (tag 4) use base-10 exponents, so the value of a decimal fraction data item is m*(10e). Bigfloats (tag 5) use base-2 exponents, so the value of a bigfloat data item is m*(2e). The exponent e MUST be represented in an integer of major type 0 or 1, while the mantissa can also be a bignum (Section 2.4.2) 🎯
Here's an example of a decimal fraction: the number 273.15 could be represented as 0b110_00100 (major type of 6 for the tag, additional information of 4 for the type of tag), followed by 0b100_00010 (major type of 4 for the array, additional information of 2 for the length of the array), followed by 0b001_00001 (major type of 1 for the first integer, additional information of 1 for the value of -2), followed by 0b000_11001 (major type of 0 for the second integer, additional information of 25 for a two-byte value), followed by 0b0110101010110011 (27315 in two bytes). In hexadecimal:
C4 -- Tag 4
82 -- Array of length 2
21 -- -2
19 6ab3 -- 27315And an example of a bigfloat: the number 1.5 could be represented as 0b110_00101 (major type of 6 for the tag, additional information of 5 for the type of tag), followed by 0b100_00010 (major type of 4 for the array, additional information of 2 for the length of the array), followed by 0b001_00000 (major type of 1 for the first integer, additional information of 0 for the value of -1), followed by 0b000_00011 (major type of 0 for the second integer, additional information of 3 for the value of 3). In hexadecimal:
C5 -- Tag 5
82 -- Array of length 2
20 -- -1
03 -- 3Decimal fractions and bigfloats don't have a way to represent Infinity, -Infinity, or NaN. If you need those in place of a decimal fraction or bigfloat, the IEEE 754 half-precision representations from Section 2.3 can be used 🚨
For constrained applications, when there's a choice between representing a specific number as an integer or as a decimal fraction or bigfloat (such as when the exponent is small and non-negative), there's a quality-of-implementation expectation that the integer representation gets used directly 🧠
2.4.4. Content Hints 💡
The tags in this section are for content hints that generic CBOR processors might use.
2.4.4.1. Encoded CBOR Data Item 📦
Sometimes it's beneficial to carry an embedded CBOR data item that isn't meant to be decoded right away, at the time the enclosing data item is being parsed. Tag 24 (CBOR data item) can be used to tag the embedded byte string as a data item encoded in CBOR format ✨
2.4.4.2. Expected Later Encoding for CBOR-to-JSON Converters ✨
okay so tags 21 to 23 are the encoder basically saying "hey, this byte string will probably get turned into text later, and here's how I think it should look" 💅 When a converter turns CBOR into JSON, some JSON setups want strings in base64, base64url, and so on. The encoder can just keep the raw bytes instead of doing the encoding itself, which keeps the message smaller, keeps the encoder code tiny, or both 🫶 And since the encoder doesn't know whether the converter is going to be generic, it's really saying what it thinks is the proper way to turn binary into JSON strings.
The tagged thing can be a byte string or any other data item. If it's some other data item, the tag applies to all of the byte string data items inside it, except for any byte strings sitting in a nested data item that has its own expected conversion tag 🌈
These three tags suggest conversions to three of the base data encodings from [RFC4648]. For base64url, padding is not used (see Section 3.2 of RFC 4648), so all the trailing equals signs ("=") get stripped off the base64url-encoded string 😭 Later tags might be defined for other RFC 4648 data encodings, or for other ways to encode binary data in strings.
2.4.4.3. Encoded Text
Some text strings hold data in formats that are super widely used on the Internet, and sometimes the decoder can validate those and present them to the application in a proper form 🦋 So there are tags for some of these formats:
- Tag 32 is for URIs, as defined in [RFC3986];
- Tags 33 and 34 are for base64url- and base64-encoded text strings, as defined in [RFC4648];
- Tag 35 is for regular expressions in Perl Compatible Regular Expressions (PCRE) / JavaScript syntax [ECMA262].
- Tag 36 is for MIME messages (including all headers), as defined in [RFC2045];
Note that tags 33 and 34 differ from 21 and 22 in that the data is transported in base-encoded form for the former and in raw byte string form for the latter. This is the key difference, bestie, so don't mix them up 🙈
2.4.5. Self-Describe CBOR 🎀
In lots of applications, it's obvious from the context that CBOR is being used to encode a data item. For instance, a specific protocol might say CBOR is used, or a media type gets indicated that specifies its use. But there can be applications where that context isn't available, like when CBOR data is stored in a file and there's no disambiguating metadata in use. In that case, it can help to have some distinguishing characteristics right in the data itself 🧚
Tag 55799 is defined for exactly this purpose. It doesn't add any special semantics to the data item that follows. The semantics of a data item tagged with tag 55799 are exactly identical to the semantics of the data item itself, so it's giving "no plot twist, just a label" ✨
The serialization of this tag is 0xd9d9f7, which doesn't appear to be in use as a distinguishing mark for frequently used file types. In particular, it's not a valid start of a Unicode text in any Unicode encoding if it's followed by a valid CBOR data item 💅
For instance, a decoder might be able to parse both CBOR and JSON. A decoder like that would need to mechanically tell the two formats apart. An easy way for an encoder to help out is to tag the entire CBOR item with tag 55799, because the serialization of that tag will never be found at the beginning of a JSON text 🫶
3. Creating CBOR-Based Protocols 🌸
Data formats like CBOR are often used in environments where there's no format negotiation. A specific design goal of CBOR is to not need any included or assumed schema: a decoder can take a CBOR item and decode it with no other knowledge. Iconic, honestly 👑
Of course, in real-world implementations, the encoder and the decoder will have a shared view of what should be in a CBOR data item. For example, an agreed-to format might be "the item is an array whose first value is a UTF-8 string, second value is an integer, and subsequent values are zero or more floating-point numbers" or "the item is a map that has byte strings for keys and contains at least one pair whose key is 0xab01" 💖
This specification puts no restrictions on CBOR-based protocols. An encoder can be capable of encoding as many or as few types of values as the protocol it's used in requires, and a decoder can be capable of understanding as many or as few types of values as the protocols it's used in require. This lack of restrictions is what lets CBOR be used in extremely constrained environments 🦋
This section discusses some considerations in creating CBOR-based protocols. It is advisory only, and it explicitly excludes any language from RFC 2119 other than words that could be interpreted as "MAY" in the sense of RFC 2119. Lowkey a vibe check, but a helpful one 🍓
3.1. CBOR in Streaming Applications 🌊
In a streaming application, a data stream may be made up of a sequence of CBOR data items concatenated back-to-back. In that kind of environment, the decoder immediately begins decoding a new data item if data is found after the end of a previous data item 💜
Not all of the bytes that make up a data item may be immediately available to the decoder. Some decoders will buffer additional data until a complete data item can be presented to the application. Other decoders can present partial information about a top-level data item to an application, like the nested data items that could already be decoded, or even parts of a byte string that hasn't completely arrived yet 👀
Note that some applications and protocols won't want to use indefinite-length encoding. Using indefinite-length encoding lets an encoder skip marshalling all the data for counting, but it requires a decoder to allocate increasing amounts of memory while it waits for the end of the item. That might be fine for some applications but not others 🙈
3.2. Generic Encoders and Decoders 🎀
A generic CBOR decoder can decode all well-formed CBOR data and present it to an application. CBOR data is well-formed if it uses the initial bytes, as well as the byte strings and/or data items implied by their values, in the manner defined by CBOR, and no extraneous data follows (Appendix C). That's the bar, bestie ✨
Even though CBOR tries to minimize these cases, not all well-formed CBOR data is valid. For example, the format excludes simple values below 32 that are encoded with an extension byte. Also, specific tags may make semantic constraints that can be violated, such as by including a tag in a bignum tag, or by following a byte string within a date tag. Finally, the data may be invalid, like invalid UTF-8 strings or date strings that don't conform to [RFC3339] 😭 There's no requirement that generic encoders and decoders make unnatural choices in their application interface just so they can process invalid data. Generic encoders and decoders are expected to forward simple values and tags even if their specific codepoints weren't registered at the time the encoder or decoder was written (Section 3.5).
Generic decoders provide ways to present well-formed CBOR values, both valid and invalid, to an application. The diagnostic notation (Section 6) may be used to present well-formed CBOR values to humans 🧚
Generic encoders provide an application interface that lets the application specify any well-formed value, including simple values and tags unknown to the encoder. Main character energy, no notes 💅
3.3. Syntax Errors 💔
A decoder that runs into a CBOR data item that isn't well-formed can generally choose to completely fail the decoding (issue an error and/or stop processing altogether). It can also substitute the problematic data and data items using a decoder-specific convention that clearly indicates there's been a problem, or take some other action entirely 🫶
3.3.1. Incomplete CBOR Data Items 🥺
The representation of a CBOR data item has a specific length, determined by its initial bytes and by the structure of any data items enclosed in it. If less data is available, this can be treated as a syntax error. A decoder may also implement incremental parsing, meaning it decodes the data item as far as it's available and presents the data found so far (such as in an event-based interface), with the option of continuing the decoding once more data arrives 🌈
Examples of incomplete data items include:
- A decoder expects a certain number of array or map entries but instead encounters the end of the data.
- A decoder processes what it expects to be the last pair in a map and comes to the end of the data.
- A decoder has just seen a tag and then encounters the end of the data.
- A decoder has seen the beginning of an indefinite-length item but encounters the end of the data before it sees the "break" stop code.
3.3.2. Malformed Indefinite-Length Items 🚫
Examples of malformed indefinite-length data items include:
- Within an indefinite-length byte string or text, a decoder finds an item that is not of the appropriate major type before it finds the "break" stop code.
- Within an indefinite-length map, a decoder encounters the "break" stop code immediately after reading a key (the value is missing).
Another error is finding a "break" stop code at a point in the data where there is no immediately enclosing (unclosed) indefinite-length item. Not it 💀
3.3.3. Unknown Additional Information Values 🔮
At the time of writing, some additional information values are unassigned and reserved for future versions of this document (see Section 5.2). Since the overall syntax for these additional information values isn't defined yet, a decoder that sees an additional information value it doesn't understand can't continue parsing. Zero chance of guessing that one, so the decoder has to stop there ✨
3.4. Other Decoding Errors 💅
Okay so, a CBOR data item can be totally well-formed (the syntax is perfect, bestie) but still have a problem when you try to interpret the data inside it in the CBOR data model 🤔 A decoder that finds one of these might issue a warning, might stop processing altogether, might handle the error and hand the problematic value to the app as such, or take some other kind of action. It's giving "your call" energy 💖
Some of the problems that can pop up:
Duplicate keys in a map: Generic decoders (Section 3.2) give apps data in the native CBOR data model. That model has maps (key-value mappings with unique keys), not multimaps (key-value mappings where multiple entries can have the same key) 🙈 So a generic decoder that gets a CBOR map item with duplicate keys will decode to a map with only one instance of that key, or it might stop processing altogether. On the flip side, a "streaming decoder" may not even be able to notice (Section 3.7).
Inadmissible type on the value following a tag: Tags (Section 2.4) say what kind of data item is supposed to come after them. For example, the tags for positive or negative bignums are supposed to be put on byte strings 🧚 A decoder that turns the tagged data item into a native representation (a native big integer in this example) is expected to check the type of the data item being tagged. Even decoders without those native representations in their environment may do the check for the tags they know, and react appropriately ✨
Invalid UTF-8 string: A decoder might or might not want to verify that the sequence of bytes in a UTF-8 string (major type 3) is actually valid UTF-8, and react appropriately 🌸
3.5. Handling Unknown Simple Values and Tags 🦋
A decoder that meets a simple value (Section 2.3) it doesn't recognize is in a bit of a pickle 🥺 That could be a value added to the IANA registry after the decoder was deployed, or a value the decoder chose not to implement. It might issue a warning, might stop processing altogether, might handle the error by making the unknown value available to the app as such (as is expected of generic decoders), or take some other action.
A decoder that meets a tag (Section 2.4) it doesn't recognize has even more options, bestie 🎀 That could be a tag added to the IANA registry after the decoder was deployed, or a tag the decoder chose not to implement. It might issue a warning, might stop processing altogether, might handle the error and present the unknown tag value together with the contained data item to the app (as is expected of generic decoders), might ignore the tag and simply present only the contained data item to the app, or take some other type of action.
3.6. Numbers 🔢
For this specification, all number representations for the same numeric value are equivalent ✨ So an encoder can encode a floating-point value of 0.0 as the integer 0. BUT (lowercase, no cap, ngl) that also means an app that expects integers only might find floating-point values, if the encoder decides they're more desirable, like when the float is more compact than a 64-bit integer 🙈
An application or protocol that uses CBOR might restrict the representations of numbers. For instance, a protocol that only deals with integers might say that floating-point numbers may not be used, and that its decoders don't need to handle floating-point numbers. Similarly, a protocol or application that uses CBOR might say that decoders need to be able to handle either type of number 💅
CBOR-based protocols should take into account that different language environments have different restrictions on the range and precision of representable numbers 🌈 For example, the JavaScript number system treats all numbers as floating point, which may result in silent loss of precision when decoding integers with more than 53 significant bits. A protocol that uses numbers should define its expectations for handling non-trivial numbers in decoders and receiving applications.
A CBOR-based protocol that includes floating-point numbers can restrict which of the three formats (half-precision, single-precision, and double-precision) are to be supported. For an integer-only application, a protocol may want to completely exclude floating-point values 🍓
A CBOR-based protocol designed for compactness may want to exclude specific integer encodings that are longer than necessary for the application, such as to save the need to implement 64-bit integers. There is an expectation that encoders will use the most compact integer representation that can represent a given value. However, a compact application should accept values that use a longer-than-needed encoding (such as encoding "0" as 0b000_11101 followed by two bytes of 0x00) as long as the application can decode an integer of the given size 🧠
3.7. Specifying Keys for Maps 🔑
The encoding and decoding applications need to agree on what types of keys will be used in maps 🫶 In applications that need to interwork with JSON-based applications, keys probably should be limited to UTF-8 strings only. Otherwise, there has to be a specified mapping from the other CBOR types to Unicode characters, and this often leads to implementation errors 😭 In applications where keys are numeric in nature and numeric ordering of keys matters, using the numbers directly as keys is useful.
If multiple types of keys are to be used, consideration should be given to how those types would be represented in the programming environments involved. For example, in JavaScript objects, a key of integer 1 cannot be distinguished from a key of string "1". This means that, if integer keys are used, the simultaneous use of string keys that look like numbers needs to be avoided. Again, this leads to the conclusion that keys should be of a single CBOR type 🙃
Decoders that deliver data items nested within a CBOR data item immediately on decoding them ("streaming decoders") often don't keep the state needed to ascertain uniqueness of a key in a map. Similarly, an encoder that can start encoding data items before the enclosing data item is completely available ("streaming encoder") may want to significantly reduce its overhead by relying on its data source to maintain uniqueness ✨
A CBOR-based protocol should make an intentional decision about what to do when a receiving application does see multiple identical keys in a map. The resulting rule in the protocol should respect the CBOR data model: it cannot prescribe a specific handling of the entries with identical keys, except that it might have a rule that identical keys in a map indicate a malformed map, and that the decoder has to stop with an error. Duplicate keys are also prohibited by CBOR decoders that are using strict mode (Section 3.10) 🚫
The CBOR data model for maps does not allow ascribing semantics to the order of the key/value pairs in the map representation. Thus, it would be a very bad practice to define a CBOR-based protocol in such a way that changing the key/value pair order in a map would change the semantics, apart from trivial aspects (cache usage, etc.) 💀 (A CBOR-based protocol can prescribe a specific order of serialization, such as for canonicalization.)
Applications for constrained devices that have maps with 24 or fewer frequently used keys should consider using small integers (and those with up to 48 frequently used keys should consider also using small negative integers), because the keys can then be encoded in a single byte 🔋
3.8. Undefined Values 🌙
In some CBOR-based protocols, the simple value (Section 2.3) of Undefined might be used by an encoder as a substitute for a data item with an encoding problem. That way the rest of the enclosing data items can be encoded without harm 🫧
3.9. Canonical CBOR 💎
Some protocols may want encoders to only emit CBOR in a particular canonical format. Those protocols might also have the decoders check that their input is canonical. They're free to define what "canonical" means and what encoders and decoders are expected to do. This section lists some suggestions for such protocols 🎀
If a protocol considers "canonical" to mean that two encoder implementations starting with the same input data will produce the same CBOR output, the following four rules would suffice 👑
- Integers must be as small as possible.
0to23and-1to-24must be expressed in the same byte as the major type.24to255and-25to-256must be expressed only with an additionaluint8_t.256to65535and-257to-65536must be expressed only with an additionaluint16_t.65536to4294967295and-65537to-4294967296must be expressed only with an additionaluint32_t.
- The expression of lengths in major types 2 through 5 must be as short as possible. The rules for these lengths follow the above rule for integers.
- The keys in every map must be sorted lowest value to highest. Sorting is performed on the bytes of the representation of the key data items without paying attention to the 3/5 bit splitting for major types. (Note that this rule allows maps that have keys of different types, even though that is probably a bad practice that could lead to errors in some canonicalization implementations.) The sorting rules are:
- If two keys have different lengths, the shorter one sorts earlier.
- If two keys have the same length, the one with the lower value in (byte-wise) lexical order sorts earlier.
- Indefinite-length items must be made into definite-length items.
If a protocol allows for IEEE floats, additional canonicalization rules might need to be added 🌊 One example rule might be to have all floats start as a 64-bit float, then do a test conversion to a 32-bit float. If the result is the same numeric value, use the shorter value and repeat the process with a test conversion to a 16-bit float. (This rule selects 16-bit float for positive and negative Infinity as well.) Also, there are many representations for NaN. If NaN is an allowed value, it must always be represented as 0xf97e00 🧊
CBOR tags present additional considerations for canonicalization 🏷️ The absence or presence of tags in a canonical format is determined by the optionality of the tags in the protocol. In a CBOR-based protocol that allows optional tagging anywhere, the canonical format must not allow them. In a protocol that requires tags in certain places, the tag needs to appear in the canonical format. A CBOR-based protocol that uses canonicalization might instead say that all tags that appear in a message must be retained, regardless of whether they are optional ✨
3.10. Strict Mode 🛡️
okay so some apps don't need canonicalization ([Section 3.9](#)) 💅 but they still need every decoder to land on the same (semantically equivalent) result, even when the data is sketchy 👀 this matters when one app, like a firewall or other guard, makes a call based on data that a different app decodes on its own 🔥
normally the sender is the one responsible for not sending data that can be decoded in more than one way 🙄 but an attacker could totally craft CBOR data so different decoders read it differently and then exploit that gap 💀 so generic decoders used where this could be a problem need a strict mode, where the receiver is also responsible for rejecting ambiguously decodable data 🫶 firewalls and other security systems that decode CBOR are expected to only decode in strict mode, no exceptions, periodt ✨
a decoder in strict mode reliably rejects any data that other decoders could read in different ways 🦋 it also reliably rejects data items with syntax errors ([Section 3.3](#)) and puts in the effort to reliably detect other decoding errors ([Section 3.4](#)) 💖 specifically, a strict decoder needs an API that reports an error (and does not return data) for a CBOR data item containing any of the following:
- a map (major type
5) that has more than one entry with the same key 🙈 - a tag that is used on a data item of the incorrect type 😭
- a data item that is incorrectly formatted for the type given to it, such as invalid UTF-8 or data that cannot be interpreted with the specific tag that it has been tagged with 💅
a strict-mode decoder has two options when it runs into a tag or simple value it doesn't recognize 🧚
- it can report an error (and not return data) 💀
- it can hand the unknown item to the calling application (the type, the value, and, for tags, the decoded tagged data item) along with an indication that the decoder didn't recognize that tag or simple value 🌈
the second option is the one that's also right for non-strict decoders, and it's lowkey iconic because it supports forward compatibility with newly registered tags and simple values 🎀 the encoder doesn't have to be updated at the same time as the calling application ✨ (for this to work, the decoder's API needs a way to mark unknown items so the calling application can handle them however makes sense for its program 🫶)
some of this checking can cost a lot (duplicate detection for maps especially), so strict mode support is not required of every CBOR decoder 🙏
some encoders depend on their applications to hand them input data in a way that guarantees unambiguously decodable CBOR output 🍓 and a generic encoder may also want its own strict mode, where it reliably limits its output to unambiguously decodable CBOR, no matter whether its application is giving it API-conformant data 👑
4. Converting Data between CBOR and JSON 🍓
this section is non-normative advice about converting between CBOR and JSON 🌸 converter implementations can use whichever bits of it they want, no pressure 💖
it's worth knowing that a JSON text is a sequence of characters, not an encoded sequence of bytes, while a CBOR data item is made of bytes, not characters 🧠
4.1. Converting from CBOR to JSON 🦋
most CBOR types have direct JSON cousins 🫶 but some don't, so anyone building a CBOR-to-JSON converter has to think about what to do with those 😭 the non-normative advice below handles them by turning them into a single substitute value, like a JSON null ✨
- an integer (major type
0or1) becomes a JSON number 🔥 - a byte string (major type
2) that is not embedded in a tag that specifies a proposed encoding gets encoded in base64url without padding and becomes a JSON string 🌈 - a UTF-8 string (major type
3) becomes a JSON string 💅 note that JSON requires escaping certain characters (RFC 4627, Section 2.5): quotation mark (U+0022), reverse solidus (U+005C), and the "C0 control characters" (U+0000throughU+001F) 🙈 every other character gets copied unchanged into the JSON UTF-8 string ✨ - an array (major type
4) becomes a JSON array 💖 - a map (major type
5) becomes a JSON object 🎀 this only works directly if all the keys are UTF-8 strings 🧚 a converter could also turn other keys into UTF-8 strings (like converting integers into strings with their decimal representation), but that introduces a danger of key collision 💀 - false (major type
7, additional information20) becomes a JSONfalse🫶 - true (major type
7, additional information21) becomes a JSONtrue🌸 - null (major type
7, additional information22) becomes a JSONnull👑 - a floating-point value (major type
7, additional information25through27) becomes a JSON number if it is finite (that is, it can be represented in a JSON number) 🔥 if the value is non-finite (NaN, or positive or negative Infinity), it is represented by the substitute value 😭 - any other simple value (major type
7, any additional information value not yet discussed) is represented by the substitute value 🙄 - a bignum (major type
6, tag value2or3) is represented by encoding its byte string in base64url without padding, and becomes a JSON string 🌈 for tag value3(negative bignum), a "~" (ASCII tilde) is inserted before the base-encoded value 🦋 (turning it into a binary blob instead of a number stops a likely numeric overflow for the JSON decoder, which is the whole point 🧠) - a byte string with an encoding hint (major type
6, tag value21through23) is encoded as described and becomes a JSON string 💅 - for all other tags (major type
6, any other tag value), the embedded CBOR item is represented as a JSON value, and the tag value gets ignored 🧚 - indefinite-length items get made definite before conversion ✨
4.2. Converting from JSON to CBOR 🎀
every JSON value, once decoded, maps directly into one or more CBOR values 💖 but just like any CBOR generation, you have to make decisions about number representation 🤓 in a suggested conversion:
- JSON numbers without fractional parts (integer numbers) are represented as integers (major types
0and1, possibly major type6tag value2and3), choosing the shortest form 🔥 integers longer than an implementation-defined threshold (usually either 32 or 64 bits) may instead be represented as floating-point values 🌈 (if the JSON came from a JavaScript implementation, its precision is already capped at 53 bits maximum 🙈) - numbers with fractional parts are represented as floating-point values 🦋 the shortest exact floating-point representation is preferred, so for instance
1.5is represented in a 16-bit floating-point value (though not every implementation can efficiently find the minimum form 😭) there may be an implementation-defined limit to precision that affects the represented values 💅 and decimal representation should only be used if a protocol specifies it ✨
CBOR was designed to be more compact than JSON in general 💖 one implementation idea that might pop into your head is doing the JSON-to-CBOR encoding in place, in a single buffer 🧠 but this strategy has to carefully handle some pathological cases 💀 for example, some strings with no or very few escapes that are longer (or much longer) than 255 bytes may expand when encoded as UTF-8 strings in CBOR 😱 similarly, some binary floating-point representations might cause expansion from short decimal representations (1.1, 1e9) in JSON 🙃 this can be really hard to get right, and any resulting vulnerabilities may be exploited by an attacker 🚨
5. Future Evolution of CBOR 🌈
successful protocols evolve over time ✨ new ideas show up, implementation platforms improve, related protocols get developed and evolve, and new requirements from applications and protocols keep getting added 🦋 so facilitating protocol evolution is an important design consideration for any protocol development 💅
for protocols that will use CBOR, CBOR gives them some useful mechanisms to help them evolve 🫶 the best practices for this are well known, especially from JSON format development of JSON-based protocols 🙌 so those best practices are outside the scope of this specification 🧚
BUT facilitating the evolution of CBOR itself is very much within its scope 🔥 CBOR is designed to provide a stable basis for CBOR-based protocol development and to be able to evolve 👑
a successful protocol may live for decades, so CBOR needs to be designed for decades of use and evolution 💖 this section gives some guidance for that evolution 🌸 it's necessarily more subjective than other parts of this document 🙈 it's also necessarily incomplete, or it would turn into a textbook on protocol development 📚💀
5.1. Extension Points 🧩
in protocol design, opportunities for evolution often show up as extension points 🦋 for example, there may be a codepoint space that isn't fully allocated from the start, and the protocol is designed to tolerate and embrace implementations that start using more codepoints than were initially allocated 🌈
sizing the codepoint space can be tricky, because the range needed can be hard to predict 🤔 so an attempt should be made to make the codepoint space large enough that it can slowly be filled over the protocol's intended lifetime ✨
CBOR has three major extension points 👑
- the "simple" space (values in major type
7). Of the 24 efficient (and 224 slightly less efficient) values, only a small number have been allocated 🙈 implementations that receive an unknown simple data item may be able to process it as such, as long as the structure of the value really is simple 💅 the IANA registry in Section 7.1 is the appropriate way to handle extensibility of this codepoint space 🫶 - the "tag" space (values in major type
6). Again, only a small part of this codepoint space has been allocated, and the space is abundant (although the early numbers are more efficient than the later ones) 🦋 implementations that receive an unknown tag can choose to simply ignore it, or to process it as an unknown tag wrapping the following data item 🧚 the IANA registry in Section 7.2 is the appropriate way to handle extensibility of this codepoint space 🌸 - the "additional information" space. An implementation that receives an unknown additional information value has no way to continue parsing, so allocating codepoints to this space is a major step 😱 there are also very few codepoints left 💀
5.2. Curating the Additional Information Space 🌸
Okay so, the human brain just cannot leave a gap alone 🙈 Give it a little empty spot and it wants to fill it in, no cap. We're pretty sure the leftover slots in the additional information codepoint space are going to be a magnet for new ideas, simply because they exist.
This specification does not manage that space with an IANA registry. Instead, any allocation out of it can only happen by updating this specification 💅
For an additional information value of n >= 24, the size of the additional data is typically 2**(n-24) bytes. So additional information values 28 and 29 are basically the candidates for 128-bit and 256-bit quantities, in case the protocol ever needs them ✨ That leaves 30 as the only additional information value available for general allocation, and there should be a seriously good reason to allocate it before it gets assigned through an update of this protocol 👑
6. Diagnostic Notation 🦋
CBOR is a binary interchange format, but for documentation and debugging (especially when people are helping each other debug) this section defines a simple human-readable diagnostic notation. All actual interchange always happens in the binary format, bestie 💜
Just so we're clear, this is a diagnostic format and it is not meant to be parsed. That's why there's no formal definition (no ABNF) here. If you're an implementer who wants a text-based format for representing CBOR data items in configuration files, you may also want to look at YAML [YAML] 🧚
The diagnostic notation is loosely based on JSON as it is defined in RFC 4627, and it stretches JSON wherever it needs to 🌈
It borrows the JSON syntax for numbers (integer and floating point), true, false, null, UTF-8 strings, arrays, and maps. (Maps are called objects in JSON, but the diagnostic notation extends JSON by letting any data item sit in the key position, which is so iconic.) undefined is written as in JavaScript. The non-finite floating-point numbers Infinity, -Infinity, and NaN are written exactly as in this sentence. That's also how they're written in JavaScript, even though JSON doesn't allow them 💀
A tagged item is written as an integer for the tag, followed by the item in parentheses. For example, an RFC 3339 (ISO 8601) date could be notated like this:
0("2013-03-21T20:04:00Z")or the equivalent relative time like this:
1(1363896240)Byte strings are notated in one of the base encodings, without padding, in single quotes. They get a prefix: h for base16, b32 for base32, h32 for base32hex, and b64 for base64 or base64url. The actual encodings don't overlap, so the string stays unambiguous ✨ For example, the byte string 0x12345678 could be written as h'12345678', b32'CI2FM6A', or b64'EjRWeA'.
Unassigned simple values are written as simple() with the right integer in the parentheses. For instance, simple(42) means major type 7, value 42 🎀
6.1. Encoding Indicators 🧚
Sometimes it's helpful to show which of several alternative representations was actually used. For example, a data item written as 1.5 by a diagnostic decoder might have been encoded as a half-, single-, or double-precision float 🤔
The convention is that anything starting with an underscore, plus all the following characters that are alphanumeric or underscores, is an encoding indicator. Anyone who doesn't care about that info can just ignore it. Encoding indicators are always optional 🫶
A single underscore can go right after the opening brace of a map or the opening bracket of an array, to show that the data item was represented in indefinite-length format. For example, [_ 1, 2] has an indicator saying an indefinite-length representation was used for the data item [1, 2] ✨
An underscore followed by a decimal digit n means the preceding item (or, for arrays and maps, the item starting with the preceding bracket or brace) was encoded with an additional information value of 24+n. For example, 1.5_1 is a half-precision floating-point number, while 1.5_3 is encoded as double precision. This encoding indicator is not shown in Appendix A. (Also, the encoding indicator "_" is really just an abbreviation of the full form "_7", which is not used, so no tea there 🍓)
As a special case, byte and text strings of indefinite length can be notated like this:
(_ h'0123', h'4567')
(_ "foo", "bar")7. IANA Considerations 🎀
IANA has created two registries for new CBOR values. They're separate (not under an umbrella registry) and they follow the rules in [RFC5226]. IANA also assigned a new MIME media type and an associated Constrained Application Protocol (CoAP) Content-Format entry, so CBOR is officially getting its whole glow-up 👑
7.1. Simple Values Registry 💖
IANA created the "Concise Binary Object Representation (CBOR) Simple Values" registry. The initial values are shown in Table 2.
New entries in the range 0 to 19 are assigned by Standards Action. It's suggested that these Standards Actions start allocating values at 16, so the lower numbers stay free for contiguous blocks (if any) 🌈
New entries in the range 32 to 255 are assigned by Specification Required.
7.2. Tags Registry 🦋
IANA created the "Concise Binary Object Representation (CBOR) Tags" registry. The initial values are shown in Table 3.
New entries in the range 0 to 23 are assigned by Standards Action. New entries in the range 24 to 255 are assigned by Specification Required. New entries in the range 256 to 18446744073709551615 are assigned by First Come First Served. The template for registration requests looks like this:
- Data item
- Semantics (short form)
First Come First Served requests should also include:
- Point of contact
- Description of semantics (URL)
The description is optional, and the URL can point to something like an Internet-Draft or a web page 💅
7.3. Media Type ("MIME Type") 💜
The Internet media type [RFC6838] for CBOR data is application/cbor. Here's the full template, copied exactly:
Type name: application
Subtype name: cbor
Required parameters: n/a
Optional parameters: n/a
Encoding considerations: binary
Security considerations: See Section 8 of this document
Interoperability considerations: n/a
Published specification: This document
Applications that use this media type: None yet, but it is expected
that this format will be deployed in protocols and applications.
Additional information:
Magic number(s): n/a
File extension(s): .cbor
Macintosh file type code(s): n/a
Person & email address to contact for further information:
Carsten Bormann
cabo@tzi.org
Intended usage: COMMON
Restrictions on usage: none
Author:
Carsten Bormann <cabo@tzi.org>
Change controller:
The IESG <iesg@ietf.org>7.4. CoAP Content-Format 🌸
Here's the CoAP entry, verbatim:
Media Type: application/cbor
Encoding: -
Id: 60
Reference: [RFC7049]7.5. The +cbor Structured Syntax Suffix Registration ✨
Here are the registration details, copied exactly:
Name: Concise Binary Object Representation (CBOR)
+suffix: +cbor
References: [RFC7049]
Encoding Considerations: CBOR is a binary format.
Interoperability Considerations: n/a
Contact: Apps Area Working Group (apps-discuss@ietf.org)
Author/Change Controller: The Apps Area Working Group.
The IESG has change control over this registration.
Security Considerations: See Section 8 of this documentThe fragment identifier rules are the juiciest part, so here they are. The syntax and semantics of fragment identifiers specified for +cbor SHOULD be the same as the ones specified for "application/cbor". (When this document was published, there was no fragment identification syntax defined for "application/cbor", so it's a bit of a blank slate 🙈)
For a specific "xxx/yyy+cbor", the syntax and semantics for fragment identifiers SHOULD be processed like this:
- For cases defined in +cbor, where the fragment identifier resolves per the +cbor rules, process it as specified in +cbor.
- For cases defined in +cbor, where the fragment identifier does not resolve per the +cbor rules, process it as specified in "xxx/yyy+cbor".
- For cases not defined in +cbor, process it as specified in "xxx/yyy+cbor".
8. Security Considerations 💀
Okay, this section is serious, so please pay attention 🚨 A network-facing application can have vulnerabilities in how it processes incoming data. Complex parsers are famously a likely source of these, and the consequences can be dramatic: a node could be remotely crashed, or even have arbitrary code remotely executed on it. CBOR tries to shrink those opportunities by keeping parser complexity down, and by giving the entire range of encodable values a meaning wherever it can ✨
Resource exhaustion attacks are another danger. An attacker might try to lure a decoder into allocating very big data items (strings, arrays, maps), or try to exhaust the stack depth by setting up deeply nested items. Decoders need appropriate resource management to mitigate these attacks. Items with very large sizes can also try to exploit integer overflow vulnerabilities, so watch out for those too 😭
Applications where a CBOR data item is examined by a gatekeeper function and later used by a different application can also be vulnerable when multiple interpretations of the data item are possible. For example, an attacker could use duplicate keys in maps and precision issues in numbers to make the gatekeeper base its decisions on a different interpretation than the one the second application will use. Protocols used in a security context should be defined so that these multiple interpretations are reliably reduced to a single one. To help with this, encoder and decoder implementations used in such contexts should provide at least one strict mode of operation (Section 3.10). That's the main character move for security 👑
9. Acknowledgements 🫶
CBOR was inspired by MessagePack. MessagePack was developed and promoted by Sadayuki Furuhashi ("frsyuki"). This reference to MessagePack is purely for attribution. CBOR is not meant to be a version of or a replacement for MessagePack, since it has different design goals and requirements 💖
The need for functionality beyond the original MessagePack Specification became obvious to lots of people at about the same time, around 2012. BinaryPack is a minor derivation of MessagePack that Eric Zhang developed for the binaryjs project. A similar but different extension was made by Tim Caswell for his msgpack-js and msgpack-js-browser projects. Many people also contributed to the recent discussion about extending MessagePack to separate text string representation from byte string representation 🌈
The way additional information is encoded in CBOR was inspired by the encoding of length information that Klaus Hartke designed for CoAP 🧚
This document also incorporates suggestions from many people, especially Dan Frost, James Manger, Joe Hildebrand, Keith Moore, Matthew Lepinski, Nico Williams, Phillip Hallam-Baker, Ray Polk, Tim Bray, Tony Finch, Tony Hansen, and Yaron Sheffer. Iconic behaviour from all of them 👑
10. References ✨
10.1. Normative References
[ECMA262] European Computer Manufacturers Association, "ECMAScript Language Specification 5.1 Edition", ECMA Standard ECMA-262, June 2011, http://www.ecma-international.org/publications/files/ecma-st/ECMA-262.pdf.
[RFC2045] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies", RFC 2045, November 1996.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC3339] Klyne, G., Ed. and C. Newman, "Date and Time on the Internet: Timestamps", RFC 3339, July 2002.
[RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO 10646", STD 63, RFC 3629, November 2003.
[RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform Resource Identifier (URI): Generic Syntax", STD 66, RFC 3986, January 2005.
[RFC4287] Nottingham, M., Ed. and R. Sayre, Ed., "The Atom Syndication Format", RFC 4287, December 2005.
[RFC4648] Josefsson, S., "The Base16, Base32, and Base64 Data Encodings", RFC 4648, October 2006.
[RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 5226, May 2008.
[TIME_T] The Open Group Base Specifications, "Vol. 1: Base Definitions, Issue 7", Section 4.15 'Seconds Since the Epoch', IEEE Std 1003.1, 2013 Edition, 2013, http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_15.
10.2. Informative References
[ASN.1] International Telecommunication Union, "Information Technology -- ASN.1 encoding rules: Specification of Basic Encoding Rules (BER), Canonical Encoding Rules (CER) and Distinguished Encoding Rules (DER)", ITU-T Recommendation X.690, 1994.
[BSON] Various, "BSON - Binary JSON", 2013, http://bsonspec.org/.
[CNN-TERMS] Bormann, C., Ersue, M., and A. Keranen, "Terminology for Constrained Node Networks", Work in Progress, July 2013.
[MessagePack] Furuhashi, S., "MessagePack", 2013, http://msgpack.org/.
[RFC0713] Haverty, J., "MSDTP-Message Services Data Transmission Protocol", RFC 713, April 1976.
[RFC4627] Crockford, D., "The application/json Media Type for JavaScript Object Notation (JSON)", RFC 4627, July 2006.
[RFC6838] Freed, N., Klensin, J., and T. Hansen, "Media Type Specifications and Registration Procedures", BCP 13, RFC 6838, January 2013.
[UBJSON] The Buzz Media, "Universal Binary JSON Specification", 2013, http://ubjson.org/.
[YAML] Ben-Kiki, O., Evans, C., and I. Net, "YAML Ain't Markup Language (YAML[TM]) Version 1.2", 3rd Edition, October 2009, http://www.yaml.org/spec/1.2/spec.html.
Appendix A. Examples ✨
Time for the examples! 🧚 The following table provides some CBOR-encoded values in hexadecimal (right column), together with diagnostic notation for these values (left column). Note that the string "\u00fc" is one form of diagnostic notation for a UTF-8 string containing the single Unicode character U+00FC, LATIN SMALL LETTER U WITH DIAERESIS (u umlaut). Similarly, "\u6c34" is a UTF-8 string in diagnostic notation with a single character U+6C34 (CJK UNIFIED IDEOGRAPH-6C34, often representing "water"), and "\ud800\udd51" is a UTF-8 string in diagnostic notation with a single character U+10151 (GREEK ACROPHONIC ATTIC FIFTY STATERS). (Note that all these single-character strings could also be represented in native UTF-8 in diagnostic notation, just not in an ASCII-only specification like the present one.) In the diagnostic notation provided for bignums, their intended numeric value is shown as a decimal number (such as 18446744073709551616) instead of showing a tagged byte string (such as 2(h'010000000000000000')). 💖
Okay bestie, this is the examples table, and it's lowkey the best part. 💖 Each row pairs a human-readable diagnostic value with its exact encoded bytes in hex, so you can check your encoder against it line by line. 🧚
+------------------------------+------------------------------------+
| Diagnostic | Encoded |
+------------------------------+------------------------------------+
| 0 | 0x00 |
| | |
| 1 | 0x01 |
| | |
| 10 | 0x0a |
| | |
| 23 | 0x17 |
| | |
| 24 | 0x1818 |
| | |
| 25 | 0x1819 |
| | |
| 100 | 0x1864 |
| | |
| 1000 | 0x1903e8 |
| | |
| 1000000 | 0x1a000f4240 |
| | |
| 1000000000000 | 0x1b000000e8d4a51000 |
| | |
| 18446744073709551615 | 0x1bffffffffffffffff |
| | |
| 18446744073709551616 | 0xc249010000000000000000 |
| | |
| -18446744073709551616 | 0x3bffffffffffffffff |
| | |
| -18446744073709551617 | 0xc349010000000000000000 |
| | |
| -1 | 0x20 |
| | |
| -10 | 0x29 |
| | |
| -100 | 0x3863 |
| | |
| -1000 | 0x3903e7 |
| | |
| 0.0 | 0xf90000 |
| | |
| -0.0 | 0xf98000 |
| | |
| 1.0 | 0xf93c00 |
| | |
| 1.1 | 0xfb3ff199999999999a |
| | |
| 1.5 | 0xf93e00 |
| | |
| 65504.0 | 0xf97bff |
| | |
| 100000.0 | 0xfa47c35000 |
| | |
| 3.4028234663852886e+38 | 0xfa7f7fffff |
| | |
| 1.0e+300 | 0xfb7e37e43c8800759c |
| | |
| 5.960464477539063e-8 | 0xf90001 |
| | |
| 0.00006103515625 | 0xf90400 |
| | |
| -4.0 | 0xf9c400 |
| | |
| -4.1 | 0xfbc010666666666666 |
| | |
| Infinity | 0xf97c00 |
| | |
| NaN | 0xf97e00 |
| | |
| -Infinity | 0xf9fc00 |
| | |
| Infinity | 0xfa7f800000 |
| | |
| NaN | 0xfa7fc00000 |
| | |
| -Infinity | 0xfaff800000 |
| | |
| Infinity | 0xfb7ff0000000000000 |
| | |
| NaN | 0xfb7ff8000000000000 |
| | |
| -Infinity | 0xfbfff0000000000000 |
| | |
| false | 0xf4 |
| | |
| true | 0xf5 |
| | |
| null | 0xf6 |
| | |
| undefined | 0xf7 |
| | |
| simple(16) | 0xf0 |
| | |
| simple(24) | 0xf818 |
| | |
| simple(255) | 0xf8ff |
| | |
| 0("2013-03-21T20:04:00Z") | 0xc074323031332d30332d32315432303a |
| | 30343a30305a |
| | |
| 1(1363896240) | 0xc11a514b67b0 |
| | |
| 1(1363896240.5) | 0xc1fb41d452d9ec200000 |
| | |
| 23(h'01020304') | 0xd74401020304 |
| | |
| 24(h'6449455446') | 0xd818456449455446 |
| | |
| 32("http://www.example.com") | 0xd82076687474703a2f2f7777772e6578 |
| | 616d706c652e636f6d |
| | |
| h'' | 0x40 |
| | |
| h'01020304' | 0x4401020304 |
| | |
| "" | 0x60 |
| | |
| "a" | 0x6161 |
| | |
| "IETF" | 0x6449455446 |
| | |
| "\"\\" | 0x62225c |
| | |
| "\u00fc" | 0x62c3bc |
| | |
| "\u6c34" | 0x63e6b0b4 |
| | |
| "\ud800\udd51" | 0x64f0908591 |
| | |
| [] | 0x80 |
| | |
| [1, 2, 3] | 0x83010203 |
| | |
| [1, [2, 3], [4, 5]] | 0x8301820203820405 |
| | |
| [1, 2, 3, 4, 5, 6, 7, 8, 9, | 0x98190102030405060708090a0b0c0d0e |
| 10, 11, 12, 13, 14, 15, 16, | 0f101112131415161718181819 |
| 17, 18, 19, 20, 21, 22, 23, | |
| 24, 25] | |
| | |
| {} | 0xa0 |
| | |
| {1: 2, 3: 4} | 0xa201020304 |
| | |
| {"a": 1, "b": [2, 3]} | 0xa26161016162820203 |
| | |
| ["a", {"b": "c"}] | 0x826161a161626163 |
| | |
| {"a": "A", "b": "B", "c": | 0xa5616161416162614261636143616461 |
| "C", "d": "D", "e": "E"} | 4461656145 |
| | |
| (_ h'0102', h'030405') | 0x5f42010243030405ff |
| | |
| (_ "strea", "ming") | 0x7f657374726561646d696e67ff |
| | |
| [_ ] | 0x9fff |
| | |
| [_ 1, [2, 3], [_ 4, 5]] | 0x9f018202039f0405ffff |
| | |
| [_ 1, [2, 3], [4, 5]] | 0x9f01820203820405ff |
| | |
| [1, [2, 3], [_ 4, 5]] | 0x83018202039f0405ff |
| | |
| [1, [_ 2, 3], [4, 5]] | 0x83019f0203ff820405 |Every row here is a little snapshot of exactly how the bytes should look, so the table is the source of truth for these examples. ✨
| | |
| [_ 1, 2, 3, 4, 5, 6, 7, 8, | 0x9f0102030405060708090a0b0c0d0e0f |
| 9, 10, 11, 12, 13, 14, 15, | 101112131415161718181819ff |
| 16, 17, 18, 19, 20, 21, 22, | |
| 23, 24, 25] | |
| | |
| {_ "a": 1, "b": [_ 2, 3]} | 0xbf61610161629f0203ffff |
| | |
| ["a", {_ "b": "c"}] | 0x826161bf61626163ff |
| | |
| {_ "Fun": true, "Amt": -2} | 0xbf6346756ef563416d7421ff |
+------------------------------+------------------------------------+Table 4: Examples of Encoded CBOR Data Items 🎀
Appendix B. Jump Table ✨
okay so this is the cheat sheet bestie 🧚 the jump table is basically a lookup for what each initial byte means! 💖 heads up though, it's not showing every single byte. the reserved ones for future extension are skipped, and it only shows a selection of the bytes that are used for optional features 🙈 also, all unsigned integers in here are in network byte order, so keep that in mind when you're reading the hex 💅
+-----------------+-------------------------------------------------+
| Byte | Structure/Semantics |
+-----------------+-------------------------------------------------+
| 0x00..0x17 | Integer 0x00..0x17 (0..23) |
| | |
| 0x18 | Unsigned integer (one-byte uint8_t follows) |
| | |
| 0x19 | Unsigned integer (two-byte uint16_t follows) |
| | |
| 0x1a | Unsigned integer (four-byte uint32_t follows) |
| | |
| 0x1b | Unsigned integer (eight-byte uint64_t follows) |
| | |
| 0x20..0x37 | Negative integer -1-0x00..-1-0x17 (-1..-24) |
| | |
| 0x38 | Negative integer -1-n (one-byte uint8_t for n |
| | follows) |
| | |
| 0x39 | Negative integer -1-n (two-byte uint16_t for n |
| | follows) |
| | |
| 0x3a | Negative integer -1-n (four-byte uint32_t for n |
| | follows) |
| | |
| 0x3b | Negative integer -1-n (eight-byte uint64_t for |
| | n follows) |
| | |
| 0x40..0x57 | byte string (0x00..0x17 bytes follow) |
| | |
| 0x58 | byte string (one-byte uint8_t for n, and then n |
| | bytes follow) |
| | |
| 0x59 | byte string (two-byte uint16_t for n, and then |
| | n bytes follow) |
| | |
| 0x5a | byte string (four-byte uint32_t for n, and then |
| | n bytes follow) |
| | |
| 0x5b | byte string (eight-byte uint64_t for n, and |
| | then n bytes follow) |
| | |
| 0x5f | byte string, byte strings follow, terminated by |
| | "break" |
| | |
| 0x60..0x77 | UTF-8 string (0x00..0x17 bytes follow) |
| | |
| 0x78 | UTF-8 string (one-byte uint8_t for n, and then |
| | n bytes follow) |
| | |
| 0x79 | UTF-8 string (two-byte uint16_t for n, and then |
| | n bytes follow) |
| | |
| 0x7a | UTF-8 string (four-byte uint32_t for n, and |
| | then n bytes follow) |
| | |
| 0x7b | UTF-8 string (eight-byte uint64_t for n, and |
| | then n bytes follow) |
| | |
| 0x7f | UTF-8 string, UTF-8 strings follow, terminated |
| | by "break" |
| | |
| 0x80..0x97 | array (0x00..0x17 data items follow) |
| | |
| 0x98 | array (one-byte uint8_t for n, and then n data |
| | items follow) |
| | |
| 0x99 | array (two-byte uint16_t for n, and then n data |
| | items follow) |
| | |
| 0x9a | array (four-byte uint32_t for n, and then n |
| | data items follow) |
| | |
| 0x9b | array (eight-byte uint64_t for n, and then n |
| | data items follow) |
| | |
| 0x9f | array, data items follow, terminated by "break" |
| | |
| 0xa0..0xb7 | map (0x00..0x17 pairs of data items follow) |
| | |
| 0xb8 | map (one-byte uint8_t for n, and then n pairs |
| | of data items follow) |
| | |
| 0xb9 | map (two-byte uint16_t for n, and then n pairs |
| | of data items follow) |
| | |
| 0xba | map (four-byte uint32_t for n, and then n pairs |
| | of data items follow) |
| | |
| 0xbb | map (eight-byte uint64_t for n, and then n |
| | pairs of data items follow) |
| | |
| 0xbf | map, pairs of data items follow, terminated by |
| | "break" |
| | |
| 0xc0 | Text-based date/time (data item follows; see |
| | Section 2.4.1) |
| | |
| 0xc1 | Epoch-based date/time (data item follows; see |
| | Section 2.4.1) |
| | |
| 0xc2 | Positive bignum (data item "byte string" |
| | follows) |
| | |
| 0xc3 | Negative bignum (data item "byte string" |
| | follows) |
| | |
| 0xc4 | Decimal Fraction (data item "array" follows; |
| | see Section 2.4.3) |
| | |
| 0xc5 | Bigfloat (data item "array" follows; see |
| | Section 2.4.3) |
| | |
| 0xc6..0xd4 | (tagged item) |
| | |
| 0xd5..0xd7 | Expected Conversion (data item follows; see |
| | Section 2.4.4.2) |
| | |
| 0xd8..0xdb | (more tagged items, 1/2/4/8 bytes and then a |
| | data item follow) |
| | |
| 0xe0..0xf3 | (simple value) |
| | |
| 0xf4 | False |
| | |
| 0xf5 | True |
| | |
| 0xf6 | Null |
| | |
| 0xf7 | Undefined |
| | |
| 0xf8 | (simple value, one byte follows) |
| | |
| 0xf9 | Half-Precision Float (two-byte IEEE 754) |
| | |
| 0xfa | Single-Precision Float (four-byte IEEE 754) |
| | |
| 0xfb | Double-Precision Float (eight-byte IEEE 754) |
| | |
| 0xff | "break" stop code |
+-----------------+-------------------------------------------------+Table 5: Jump Table for Initial Byte 🦋
Appendix C. Pseudocode ✨
Okay so, the well-formedness of a CBOR item can be checked with the pseudocode in Figure 1 🔍 The data counts as well-formed if and only if:
- the pseudocode does not "fail";
- after the pseudocode runs, no bytes are left in the input (except in streaming applications)
Before you run it, there are a few prerequisites to know about 💅
take(n)reads n bytes from the input data and returns them as a byte string. If n bytes are no longer available,take(n)fails.uint()converts a byte string into an unsigned integer by interpreting the byte string in network byte order.- Arithmetic works as in C.
- All variables are unsigned integers of sufficient range.
well_formed (breakable = false) {
// process initial bytes
ib = uint(take(1));
mt = ib >> 5;
val = ai = ib & 0x1f;
switch (ai) {
case 24: val = uint(take(1)); break;
case 25: val = uint(take(2)); break;
case 26: val = uint(take(4)); break;
case 27: val = uint(take(8)); break;
case 28: case 29: case 30: fail();
case 31:
return well_formed_indefinite(mt, breakable);
}
// process content
switch (mt) {
// case 0, 1, 7 do not have content; just use val
case 2: case 3: take(val); break; // bytes/UTF-8
case 4: for (i = 0; i < val; i++) well_formed(); break;
case 5: for (i = 0; i < val*2; i++) well_formed(); break;
case 6: well_formed(); break; // 1 embedded data item
}
return mt; // finite data item
}
well_formed_indefinite(mt, breakable) {
switch (mt) {
case 2: case 3:
while ((it = well_formed(true)) != -1)
if (it != mt) // need finite embedded
fail(); // of same type
break;
case 4: while (well_formed(true) != -1); break;
case 5: while (well_formed(true) != -1) well_formed(); break;
case 7:
if (breakable)
return -1; // signal break out
else fail(); // no enclosing indefinite
default: fail(); // wrong mt
}
return 0; // no break out
}
Figure 1: Pseudocode for Well-Formedness CheckNote that the rest of the complexity in a complete CBOR decoder is about presenting parsed data to the application in an appropriate form 🌸
Major types 0 and 1 were designed so that they can be encoded in C from a signed integer without an if-then-else for positive/negative 🧚 (Figure 2). The trick is that (-1-n), the transformation for major type 1, is the same as ~n (bitwise complement) in C unsigned arithmetic. So ~n can be written as (-1)^n for the negative case, while 0^n leaves n unchanged for non-negative. The sign of a number can be turned into -1 for negative and 0 for non-negative (0 or positive) by arithmetic-shifting the number by one bit less than its bit length (for example, by 63 for 64-bit numbers). It's giving clever 🤓
void encode_sint(int64_t n) {
uint64t ui = n >> 63; // extend sign to whole length
mt = ui & 0x20; // extract major type
ui ^= n; // complement negatives
if (ui < 24)
*p++ = mt + ui;
else if (ui < 256) {
*p++ = mt + 24;
*p++ = ui;
} else
...
Figure 2: Pseudocode for Encoding a Signed IntegerAppendix D. Half-Precision 🦋
Half-precision floating-point numbers were only added to IEEE 754 in 2008, so today's programming platforms often still have limited support for them 🥺 The good news is that it's very easy to include at least decoding support, even without platform support. An example of a small decoder for half-precision floating-point numbers in the C language is shown in Figure 3. A similar program for Python is in Figure 4. This code assumes that the 2-byte value has already been decoded as an (unsigned short) integer in network byte order (as would be done by the pseudocode in Appendix C) 💜
#include <math.h>
double decode_half(unsigned char *halfp) {
int half = (halfp[0] << 8) + halfp[1];
int exp = (half >> 10) & 0x1f;
int mant = half & 0x3ff;
double val;
if (exp == 0) val = ldexp(mant, -24);
else if (exp != 31) val = ldexp(mant + 1024, exp - 25);
else val = mant == 0 ? INFINITY : NAN;
return half & 0x8000 ? -val : val;
}
Figure 3: C Code for a Half-Precision Decoderimport struct
from math import ldexp
def decode_single(single):
return struct.unpack("!f", struct.pack("!I", single))[0]
def decode_half(half):
valu = (half & 0x7fff) << 13 | (half & 0x8000) << 16
if ((half & 0x7c00) != 0x7c00):
return ldexp(decode_single(valu), 112)
return decode_single(valu | 0x7f800000)
Figure 4: Python Code for a Half-Precision DecoderAppendix E. Comparison of Other Binary Formats to CBOR's Design Objectives 🌈
Binary formats have a history about as long as computers themselves 🕰️ Different formats have had different objectives. In most cases the objectives were never stated, although you can sometimes guess them from the context where the format was first used. Some formats were meant to be universally usable, but history has proven that no binary format meets the needs of all protocols and applications 😭
CBOR is different from many of these because it started with a set of objectives and tried to meet just those ✨ This section compares a few of the dozens of formats against CBOR's objectives, so you can decide whether you want CBOR or something else for a particular protocol or application 🎀
Just so we're clear, this isn't a criticism of any format 🫶 To the best of our knowledge, no format before CBOR was meant to cover CBOR's objectives in the priority we have assigned them. A quick recap of the objectives from Section 1.1 goes like this:
- unambiguous encoding of most common data formats from Internet standards
- code compactness for encoder or decoder
- no schema description needed
- reasonably compact serialization
- applicability to constrained and unconstrained applications
- good JSON conversion
- extensibility
E.1. ASN.1 DER, BER, and PER
[ASN.1] has many serializations. In the IETF, DER and BER are the most common. The serialized output isn't particularly compact for many items, and the code needed to decode numeric items can get complex on a constrained device 🥴
Few (if any) IETF protocols have adopted one of the several variants of Packed Encoding Rules (PER). There could be many reasons for this, but one that gets commonly stated is that PER uses the schema even to parse the surface structure of the data stream, which requires significant tool support. There are also different versions of the ASN.1 schema language in use, which has hampered adoption too 💀
E.2. MessagePack
[MessagePack] is a concise, widely implemented counted binary serialization format. It's similar in many properties to CBOR, although somewhat less regular 🍓 While the data model can be used to represent JSON data, MessagePack has also been used in many remote procedure call (RPC) applications and for long-term storage of data.
MessagePack has been essentially stable since it was first published around 2011, and it has not yet had a transition. Its evolution is impeded by an imperative to maintain complete backwards compatibility with existing stored data, while only a few bytecodes are still available for extension. Over the years the MessagePack user community has repeatedly asked to separate binary and text strings in the encoding. That recently led to an extension proposal that would leave MessagePack's "raw" data ambiguous between its usages for binary and text data. The extension mechanism for MessagePack remains unclear 🤷♀️
E.3. BSON
[BSON] is a data format that was developed for storing JSON-like maps (JSON objects) in the MongoDB database. Its major distinguishing feature is the capability for in-place update, which means it forgoes a compact representation 🧁 BSON uses a counted representation except for map keys, which are null-byte terminated. While BSON can be used to represent JSON-like objects on the wire, its specification is dominated by the requirements of the database application and has become somewhat baroque. The status of how BSON extensions will be implemented remains unclear 🙈
E.4. UBJSON
[UBJSON] has a design goal of making JSON faster and somewhat smaller, using a binary format that is limited to exactly the data model JSON uses 🧚♀️ So there is expressly no intention to support, for example, binary data. There is, however, a "high-precision number", expressed as a character string in JSON syntax. UBJSON is not optimized for code compactness, and its type byte coding is optimized for human recognition rather than for compact representation of native types such as small integers. Although UBJSON is mostly counted, it provides a reserved "unknown-length" value to support streaming of arrays and maps (JSON objects). Within these containers, UBJSON also has a "Noop" type for padding 🌸
E.5. MSDTP: RFC 713
Message Services Data Transmission (MSDTP) is a very early example of a compact message format. It is described in [RFC0713], written in 1976. It's included here for its historical value, not because it was ever widely used 📼
E.6. Conciseness on the Wire
Okay, so while CBOR's design objective of code compactness for encoders and decoders is a higher priority than its objective of conciseness on the wire, a lot of people focus on wire size 👀 Table 6 shows some encoding examples for the simple nested array [1, [2, 3]]. Where the encoding supports some form of indefinite-length encoding, [_ 1, [2, 3]] (indefinite length on the outer array) is also shown 💖
+---------------+-------------------------+-------------------------+
| Format | [1, [2, 3]] | [_ 1, [2, 3]] |
+---------------+-------------------------+-------------------------+
| RFC 713 | c2 05 81 c2 02 82 83 | |
| | | |
| ASN.1 BER | 30 0b 02 01 01 30 06 02 | 30 80 02 01 01 30 06 02 |
| | 01 02 02 01 03 | 01 02 02 01 03 00 00 |
| | | |
| MessagePack | 92 01 92 02 03 | |
| | | |
| BSON | 22 00 00 00 10 30 00 01 | |
| | 00 00 00 04 31 00 13 00 | |
| | 00 00 10 30 00 02 00 00 | |
| | 00 10 31 00 03 00 00 00 | |
| | 00 00 | |
| | | |
| UBJSON | 61 02 42 01 61 02 42 02 | 61 ff 42 01 61 02 42 02 |
| | 42 03 | 42 03 45 |
| | | |
| CBOR | 82 01 82 02 03 | 9f 01 82 02 03 ff |
+---------------+-------------------------+-------------------------+
Table 6: Examples for Different Levels of ConcisenessAuthors' Addresses ✨
Okay bestie, this is the final stop on our RFC 7049 tour 🎀 and it's giving contact card vibes. These are the people who wrote the spec, and yes, you can literally reach out to them 💅
Carsten Bormann Universitaet Bremen TZI Postfach 330440 D-28359 Bremen Germany
Phone: +49-421-218-63921 EMail: cabo@tzi.org
Paul Hoffman VPN Consortium
EMail: paul.hoffman@vpnc.org
That's the whole document, babe 🦋 Carsten and Paul built CBOR, and honestly the fact that you made it all the way here is iconic 👑🫶
📦 where this page comes from
- ietf: rfc7049 2013-10
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms