RFC 8610
Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data Structures
RFC 8610: Concise Data Definition Language (CDDL) 🎀
Okay bestie, RFC 8610 is a Standards Track document from June 2019 (ISSN 2070-1721) in the Internet Engineering Task Force (IETF) series 💅. The title is "Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data Structures". H. Birkholz wrote it at Fraunhofer SIT, and C. Vigano and C. Bormann wrote it at Universitaet Bremen (Bormann's is the Universitaet Bremen TZI one). The title block doesn't list anything that it obsoletes or updates 🧚
Abstract ✨
So the whole point of this doc is to propose a notation for writing down Concise Binary Object Representation (CBOR) data structures (RFC 7049) 🍓. The big goal is making it super easy and totally unambiguous to describe the shapes of protocol messages and data formats that use CBOR or JSON. No confusion, no drama 💖
Status of This Memo 📜
This is an Internet Standards Track document 🔥
It's a product of the Internet Engineering Task Force (IETF), so it reflects what the IETF community agreed on. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). If you want more on Internet Standards, check Section 2 of RFC 7841 🦋
For the current status, any errata, and how to send feedback, go to https://www.rfc-editor.org/info/rfc8610 🌸
Copyright Notice ©️
Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved 🫶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they stand on the date this was published. Please actually read those, because they spell out what you can and can't do with this doc 🙈 Any Code Components pulled out of this document have to include the Simplified BSD License text described in Section 4.e of the Trust Legal Provisions, and they come with no warranty, as the Simplified BSD License says 💜
Table of Contents 🗺️
1. Introduction ....................................................4
1.1. Requirements Notation ......................................5
1.2. Terminology ................................................5
2. The Style of Data Structure Specification .......................5
2.1. Groups and Composition in CDDL .............................7
2.1.1. Usage ..............................................10
2.1.2. Syntax .............................................10
2.2. Types .....................................................11
2.2.1. Values .............................................11
2.2.2. Choices ............................................11
2.2.3. Representation Types ...............................13
2.2.4. Root Type ..........................................14
3. Syntax .........................................................15
3.1. General Conventions .......................................15
3.2. Occurrence ................................................16
3.3. Predefined Names for Types ................................17
3.4. Arrays ....................................................18
3.5. Maps ......................................................19
3.5.1. Structs ............................................19
3.5.2. Tables .............................................22
3.5.3. Non-deterministic Order ............................23
3.5.4. Cuts in Maps .......................................24
3.6. Tags ......................................................25
3.7. Unwrapping ................................................26
3.8. Controls ..................................................27
3.8.1. Control Operator .size .............................27
3.8.2. Control Operator .bits .............................28
3.8.3. Control Operator .regexp ...........................29
3.8.4. Control Operators .cbor and .cborseq ...............30
3.8.5. Control Operators .within and .and .................30
3.8.6. Control Operators .lt, .le, .gt, .ge, .eq,
.ne, and .default ..................................31
3.9. Socket/Plug ...............................................32
3.10. Generics .................................................33
3.11. Operator Precedence ......................................34
4. Making Use of CDDL .............................................36
4.1. As a Guide for a Human User ...............................36
4.2. For Automated Checking of CBOR Data Structures ............36
4.3. For Data Analysis Tools ...................................37
5. Security Considerations ........................................37
6. IANA Considerations ............................................38
6.1. CDDL Control Operators Registry ...........................38
7. References .....................................................40
7.1. Normative References ......................................40
7.2. Informative References ....................................41
Appendix A. Parsing Expression Grammars (PEGs) ....................43
Appendix B. ABNF Grammar ..........................................45
Appendix C. Matching Rules ........................................47
Appendix D. Standard Prelude ......................................52
Appendix E. Use with JSON .........................................53
Appendix F. A CDDL Tool ...........................................56
Appendix G. Extended Diagnostic Notation ..........................56
G.1. Whitespace in Byte String Notation .........................57
G.2. Text in Byte String Notation ...............................57
G.3. Embedded CBOR and CBOR Sequences in Byte Strings ...........57
G.4. Concatenated Strings .......................................58
G.5. Hexadecimal, Octal, and Binary Numbers .....................59
G.6. Comments ...................................................59
Appendix H. Examples ..............................................60
Acknowledgements ..................................................63
Contributors ......................................................63
Authors' Addresses ................................................641. Introduction 🦋
In this doc we define a notational convention for expressing Concise Binary Object Representation (CBOR) data structures [RFC7049] 🌈
The main goal is one unified notation you can use when you're defining protocols that use CBOR. We call it the "Concise Data Definition Language", or CDDL for short 💜
The convention has these goals:
- (G1) Provide an unambiguous description of the overall structure of a CBOR data item.
- (G2) Be flexible in expressing the multiple ways in which data can be represented in the CBOR data format.
- (G3) Be able to express common CBOR datatypes and structures.
- (G4) Provide a single format that is both readable and editable for humans and processable by a machine.
- (G5) Enable automatic checking of CBOR data items for data format compliance.
- (G6) Enable extraction of specific elements from CBOR data for further processing.
Here's a fun side effect, and it wasn't the original plan 🙈 The JSON generic data model is a subset of the CBOR generic data model, so CDDL can describe JSON data structures too (see Appendix E) 🍓
Here's how the doc is laid out. The CDDL syntax lives in Section 3. Examples of CDDL along with a related CBOR data item (the "instance"), some in JSON form, are in Appendix H. Section 4 talks about how to use CDDL. There are examples all through the text to make the concepts click. The formal definition of CDDL, written as an ABNF grammar [RFC5234], is in Appendix B. Last but not least, a _prelude_ of standard CDDL definitions gets automatically prepended to every CDDL specification, so it's available everywhere, and that list is in Appendix D ✨
1.1. Requirements Notation 🫶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.
1.2. Terminology 🌸
New terms get introduced in _cursive_ 💅 In plain text, that shows up as the new term wrapped in underscores, like _prelude_. CDDL text in the running prose is in "typewriter" style, which in plain text means the CDDL text is in double quotes. Double quotes are also used the normal English way too, so you'll have to figure out which one it is from context 🧠
In this specification, the term "byte" is used in its now-customary sense as a synonym for "octet" 🍓
Part 1 of 11 done, bestie! 💖 Part 2 is where Section 2 starts getting into the style of data structure specification, so stay tuned 🧚
2. The Style of Data Structure Specification 🎀
Okay bestie, so CDDL is all about the styles people already use for the data model that JSON pioneered and CBOR glowed up 💅. The data model is the star here, and CDDL focuses on the styles of specification the community uses with it.
There are a bunch of basic building blocks in a CBOR data model, like numbers, simple values (false, true, nil), text strings, and byte strings 🌸. CDDL does not focus on specifying their structure. CDDL does allow adding a CBOR tag to a data item though, which is very cute of it ✨.
Past those atomic elements, the next things you need for defining a data structure are the datatypes used for composition: arrays and maps in CBOR (called "arrays" and "objects" in JSON). There are only two representation formats, but they get used to write four loosely distinguishable styles of composition 🦋:
- A vector: an array of elements that are mostly the same semantically. The set of signatures attached to a signed data item is a typical vector, iykyk 🧚.
- A record: an array whose elements have different, positionally defined semantics, as laid out in the data structure definition. A 2D point, written as an array with an x coordinate (first) and a y coordinate (second), is a record, and so is the pair of exponent (first) and mantissa (second) in a CBOR decimal fraction 💜.
- A table: a map from a domain of map keys to a domain of map values that are mostly the same semantically. A set of language tags, each mapped to a text string translated into that specific language, is a table 🌈. The key domain usually isn't limited to a specific set by the specification and is left open for the application. For example, in a table mapping IP addresses to Media Access Control (MAC) addresses, the specification doesn't try to foresee every possible IP address 🙈. In a language like JavaScript, a "Map" (as opposed to a plain "Object") would often be used to get that generality of key domain 💖.
- A struct: a map from a domain of map keys, as defined by the specification, to a domain of map values, where the semantics of each value is bound to one specific map key. This is what most people picture when they think of JSON objects. CBOR adds the ability to use map keys that aren't just text strings 🍓. Structs can solve the same kinds of problems records are used for, and using explicit map keys makes optionality and extensibility easy ✨.
Two important concepts are the foundation for CDDL 👑:
- Instead of defining all four types of composition separately in CDDL, or even defining one kind for arrays (vectors and records) and one kind for maps (tables and structs), there is only one kind of composition in CDDL: the group (Section 2.1). Literally one kind, slay 😭.
- The other important concept is the type. The entire CDDL specification defines a type (the one defined by its first rule), which formally is the set of CBOR data items that are acceptable as "instances" for this specification 🔥. CDDL predefines a number of basic types, like
uint(unsigned integer) ortstr(text string), often using a simple formal notation for CBOR data items 🎀. Each value that can be expressed as a CBOR data item is also a type in its own right, e.g.,1💅. A type can be built as a choice of other types: for example, anintis either auintor anint(negative integer) 💜. Finally, a type can be built as an array or a map from a group 🦋.
The rest of this section introduces a number of basic CDDL concepts, and Section 3 defines additional syntax. Appendix C gives a concise summary of the semantics of CDDL 🌸.
2.1. Groups and Composition in CDDL
CDDL groups are lists of group entries, and each entry can be a name/value pair or a more complex group expression (which in turn stands for a sequence of name/value pairs) 🧚. A CDDL group is a production in a grammar that matches certain sequences of name/value pairs but not others. The grammar is based on the concepts of Parsing Expression Grammars (PEGs) (see Appendix A) ✨.
In an array context, only the value of the name/value pair is represented. The name is just annotation (and can be left off the group specification if you don't need it) 🎀. In a map context, the names become the map keys ("member keys") 💖.
In an array context, the actual sequence of elements in the group matters, because that sequence is what lets you match actual array elements with entries in the group 🌈. In a map context, the sequence of entries in a group isn't relevant, though you still have to write the group entries down in some sequence 🙈.
An array matches a specification given as a group when the group matches a sequence of name/value pairs whose value parts exactly match the elements of the array, in order 🦋.
A map matches a specification given as a group when the group matches a sequence of name/value pairs such that all of these name/value pairs are present in the map, and the map has no name/value pair that isn't covered by the group 👑.
Here's a simple example of using a group directly in a map definition 💅:
person = {
age: int,
name: tstr,
employer: tstr,
}
Figure 1: Using a Group Directly in a MapThe three entries of the group are written between the curly braces that create the map. Here, "age", "name", and "employer" are the names that turn into the map key text strings, and "int" and "tstr" (text string) are the types of the map values under those keys 🍓.
A group by itself (without creating a map around it) can go in (round) parentheses and be given a name by using it in a rule 🧚:
pii = (
age: int,
name: tstr,
employer: tstr,
)
Figure 2: A Basic GroupThis separate, named group definition lets us rephrase Figure 1 as 💖:
person = {
pii
}
Figure 3: Using a Group by NameNote that the (curly) braces signify the creation of a map. The groups themselves are neutral as to whether they'll be used in a map or an array 🌸.
As shown in Figure 1, the parentheses for groups are optional when there's some other set of brackets present. They can still be used, which leads to this not-so-realistic, but perfectly valid, example 🙈:
person = {(
age: int,
name: tstr,
employer: tstr,
)}
Figure 4: Using a Parenthesized Group in a MapGroups can be used to factor out common parts of structs. Instead of writing specifications in copy/paste style, like in Figure 5, you can factor out the common subgroup, choose a name for it, and write only the specific parts into the individual maps (Figure 6) 💅✨.
person = {
age: int,
name: tstr,
employer: tstr,
}
dog = {
age: int,
name: tstr,
leash-length: float,
}
Figure 5: Maps with Copy/Paste person = {
identity,
employer: tstr,
}
dog = {
identity,
leash-length: float,
}
identity = (
age: int,
name: tstr,
)
Figure 6: Using a Group for FactorizationNote that the lists inside the braces in the definitions above are (anonymous) groups, while "identity" is a named group. A named group can then be included as part of other groups, either anonymously (as in the example) or with its own name 🎀.
2.1.1. Usage
Groups are the tool you use to compose data structures in CDDL 🦋. It's a matter of style whether you define groups (anonymously) right in their context, or define them in a separate rule and reference them by name (possibly more than once) 💜.
Either way, you can define all the small parts of your data structures and compose bigger protocol data units from them, or you can have one big protocol data unit that has all its definitions ad hoc, right where they're needed 🌈.
2.1.2. Syntax
The composition syntax is meant to be concise and easy to read 🧚:
- The start and end of a group can be marked by
(and). - Definitions of entries inside a group are written as keytype => valuetype, (read "keytype maps to valuetype"). The comma is actually optional (not just in the final entry), but it's considered good style to set it. The double arrow can be replaced by a colon in the common case of directly using a text string or integer literal as a key; see Section 3.5.1. This is also the common way of naming elements of an array just for documentation; see Section 3.4 🍓.
A basic entry consists of a keytype and a valuetype, both of which are types (Section 2.2). This entry matches any name/value pair whose name is in the keytype and whose value is in the valuetype 💖.
A group defined as a sequence of group entries matches any sequence of name/value pairs that is composed, by concatenation and in order, of what the entries match 🌸.
A group definition can also contain choices between groups; see Section 2.2.2 🙈.
2.2. Types
2.2.1. Values
Values such as numbers and strings can be used in place of a type. (For instance, this is a very common thing to do for a key type, common enough that CDDL provides extra convenience syntax for it 🦋.)
The value notation is based on the C language, but it doesn't offer all the syntactic variations (see Appendix B for details) 💅. The value notation for numbers inherits from C the distinction between integer values (no fractional part or exponent given -- NR1 [ISO6093]; "NR" stands for "numerical representation") and floating-point values (where a fractional part, an exponent, or both are present -- NR2 or NR3) ✨. So the type 1 doesn't include any floating-point numbers, while the types 1e3 and 1.5 are both floating-point numbers and don't include any integer numbers 👑.
2.2.2. Choices 🎀
Okay so anywhere a type is allowed, you can also do a choice between types, separated by a "/" (slash) 💅. If it gets confusing, you can wrap the whole choice in parentheses (see Appendix B for the CDDL grammar deets 🤓).
Choices of values are basically enumerations, very pick-a-vibe energy ✨:
attire = "bow tie" / "necktie" / "Internet attire"
protocol = 6 / 17CDDL also lets you do choices between groups, using "//" (double slash) 🦋. Heads up, though: "//" binds way more weakly than the other CDDL operators 💀. So each line inside "delivery" in the example below is its own alternative in the group choice.
address = { delivery }
delivery = (
street: tstr, ? number: uint, city //
po-box: uint, city //
per-pickup: true )
city = (
name: tstr, zip-code: uint
)A group choice matches the union of the sets of name/value pair sequences that the alternatives in the choice can match 🫶.
For both type choices and group choices, you can add more alternatives to a rule later, in separate rules, by using "/=" and "//=" instead of "=" 🌸. Yes, these are different operators, so pay attention 👀.
attire /= "swimwear"
delivery //= (
lat: float, long: float, drone-type: tstr
)It's not an error if a name is first used with a "/=" or "//=" (there's no need to "create it" with "=" first) 💖.
2.2.2.1. Ranges 🌈
Instead of naming every single value in a choice, CDDL lets you build a _range_ out of two values that have an ordering relationship 📏. You get a lower bound (first value) and an upper bound (second value). A range can include both bounds, which you write with "..", or include the lower bound and exclude the upper bound, which you write with "...". If the lower bound is bigger than the upper bound, the resulting type is the empty set 🫠. (This can be super useful when generics (Section 3.10) are being used 🧚.)
device-address = byte
max-byte = 255
byte = 0..max-byte ; inclusive range
first-non-byte = 256
byte1 = 0...first-non-byte ; byte1 is equivalent to byteCDDL currently only allows ranges between integers (matching integer values) or between floating-point values (matching floating-point values) 🔢. If you need both kinds in one type, you can (clumsily, ngl 🙈) use a type choice between the two kinds of ranges:
int-range = 0..10 ; only integers match
float-range = 0.0..10.0 ; only floats match
BAD-range1 = 0..10.0 ; NOT DEFINED
BAD-range2 = 0.0..10 ; NOT DEFINED
numeric-range = int-range / float-range(See also the control operators .lt/.ge and .le/.gt in Section 3.8.6 🎀.)
Also, the dot is a valid name continuation character in CDDL, so
min..maxis not a range expression. It's a single name 😭. When you use a name as the left-hand side of a range operator, use spacing like this to separate the range operator off:
min .. max2.2.2.2. Turning a Group into a Choice 🍓
Some choices are built out of tons of values, often integers, and each one deserves a semantic name in the specification 🧠. Instead of naming each integer and then piling them into a choice, CDDL lets you build a choice from a group by putting an "&" character in front of it 🔥.
terminal-color = &basecolors
basecolors = (
black: 0, red: 1, green: 2, yellow: 3,
blue: 4, magenta: 5, cyan: 6, white: 7,
)
extended-color = &(
basecolors,
orange: 8, pink: 9, purple: 10, brown: 11,
)Same as with groups in arrays (Section 3.4), the member names only have documentary value. (In particular, a tool might use them when displaying integers taken from that choice 👑.)
2.2.3. Representation Types 🦋
CDDL lets you specify a data item type by referring to the CBOR representation, specifically the major types and additional information (see Section 2 of [RFC7049]) 🧩. You can tell how this works from the prelude (Appendix D). It's a hash mark ("#") that's optionally followed by a number from 0 to 7 identifying the major type. After that, you can add a dot and a number for the additional information. This construction specifies the set of values that can be serialized in CBOR (that's "any"), by the given major type if one is given, or by the given major type with the additional information if both are given 💜. When you use major type 6 (Tag), you can give the type of the tagged item by appending it in parentheses.
Here's the vibe check: although this notation is based on CBOR serialization, it's about a set of values at the data model level 🌷. For example, "#7.25" specifies the set of values that can be represented as half-precision floats. It does not require those values to actually be serialized as half-precision floats 🙅. CDDL doesn't give you any language means to restrict the choice of serialization variants. This also means CDDL works with JSON, which serializes (some of) the same values in a totally different way 🌈.
You might need representation types outside the prelude too. For example, a specification could start by using an existing tag in a more specific way, or it could define a new tag that isn't in the prelude ✨:
my_breakfast = #6.55799(breakfast) ; cbor-any is too general!
breakfast = cereal / porridge
cereal = #6.998(tstr)
porridge = #6.999([liquid, solid])
liquid = milk / water
milk = 0
water = 1
solid = tstr2.2.4. Root Type 👑
There's no special syntax to mark the root of a CDDL data structure definition 🙃. That job just goes to the first rule defined in the file.
This is because of the usual top-down approach to defining data structures, where you break a big data structure unit into smaller parts 🧁. Except for the root type, though, there's no need to strictly follow that sequence.
(Note that there's no way to use a group as a root. It has to be a type 💅.)
3. Syntax 🎀
In this section, we get the overall syntax of CDDL, plus some examples that just show off the syntax 🔍. (The definition doesn't try to be super formal. Check Appendix B for the full details 📖.)
3.1. General Conventions 🌸
The basic syntax is inspired by ABNF [RFC5234], with a few extras 💖:
- Rules, whether they define groups or types, are defined with a name, followed by an equals sign "=" and the actual definition according to the respective syntactic rules of that definition.
- A name can consist of any of the characters from the set {"A" to "Z", "a" to "z", "0" to "9", "_", "-", "@", ".", "$"}. It has to start with an alphabetic character (including "@", "_", "$") and end with such a character or a digit 🔤.
- Names are case sensitive 🙈.
- It's preferred style to start a name with a lowercase letter ✨.
- The hyphen is preferred over the underscore (except in a "bareword" (Section 3.5.1), where the semantics may actually require an underscore) 🐍.
- The period may be useful for larger specifications, to express some module structure (as in "tcp.throughput" vs. "udp.throughput") 🌐.
- A number of names are predefined in the CDDL prelude, as listed in Appendix D 🧚.
- Rule names (types or groups) don't appear in the actual CBOR encoding, but names used as "barewords" in member keys do 👀.
- Comments start with a ";" (semicolon) character and finish at the end of a line (LF or CRLF) 💬.
- Except within strings, whitespace (spaces, newlines, and comments) is used to separate syntactic elements for readability (and to separate identifiers, range operators, or numbers that follow each other). Otherwise, it's totally optional 🫶.
- Hexadecimal numbers start with "0x" (without quotes) and are case insensitive. Similarly, binary numbers start with "0b" 🔢.
- Text strings are enclosed by double quotation '"' characters. They follow the string conventions in Section 7 of [RFC8259] 📝. (ABNF users may want to note that CDDL has no support for case insensitivity in text strings. If you need that, regular expressions can be used (Section 3.8.3) 🌟.)
- Byte strings are enclosed by single quotation "'" characters and may be prefixed by "h" or "b64" 🍬. If unprefixed, the string is interpreted like a text string, except that single quotes must be escaped and the resulting UTF-8 bytes are marked as a byte string (major type 2). If prefixed as "h" or "b64", the string is interpreted as a sequence of pairs of hex digits (base16; see Section 8 of [RFC4648]) or as a base64(url) string (Section 4 or Section 5 of [RFC4648]), respectively. (This matches the diagnostic notation in Section 6 of [RFC7049]; cf. Appendix G.2 🧾.) In the prefixed case, any whitespace in the string (including comments) is ignored 🫧.
- CDDL uses UTF-8 [RFC3629] for its encoding. Processing CDDL doesn't involve Unicode normalization processes 🌈.
Example:
; This is a comment
person = { g }
g = (
"name": tstr,
age: int, ; "age" is a bareword
)3.2. Occurrence 🦋
You can put an optional _occurrence_ indicator in front of a group entry 🔁. It's either (1) one of the characters "?" (optional), "*" (zero or more), or "+" (one or more), or (2) of the form n*m, where n and m are optional unsigned integers. Here n is the lower limit (default 0) and m is the upper limit (default no limit) of occurrences 📊.
If no occurrence indicator is specified, the group entry occurs exactly once (as if 1*1 were specified) 1️⃣. A group entry with an occurrence indicator matches sequences of name/value pairs that are made by concatenating a number of sequences that the basic group entry matches. The number needs to be allowed by the occurrence indicator 🧮.
Note that CDDL, outside any directives/annotations that could possibly be defined, doesn't say whether arrays or maps use definite-length or indefinite-length encoding 🙅. So leaving the size of an array "open" in the spec has nothing to do with whether it gets interchanged with definite or indefinite length 🤷♀️.
Also, CDDL can describe flexibility that the data model of the target representation doesn't have. This is pretty obvious for JSON, but it's relevant for CBOR too 🧠:
apartment = {
kitchen: size,
* bedroom: size,
}
size = float ; in m2The spec above does not mean CBOR is changed to allow using the key "bedroom" more than once 🙅♀️. In other words, because of the restrictions imposed by the data model, the third line basically turns into:
? bedroom: size,(Occurrence indicators beyond one are still useful in maps for groups that allow a variety of keys 🗝️.)
3.3. Predefined Names for Types ✨
okay so CDDL comes with a bunch of built-in names already loaded, like pre-made outfits in your closet 👗 this part gives you the quick tour, but if you want the exact fine print, check out Appendix D 🎀
these are the primitive datatype keywords:
"bool": Boolean value (major type 7, additional information 20 or 21). 🔘"uint": An unsigned integer (major type 0). 🔢"nint": A negative integer (major type 1). ➖"int": An unsigned integer or a negative integer. 💫"float16": A number representable as a half-precision float [IEEE754] (major type 7, additional information 25). 🦋"float32": A number representable as a single-precision float [IEEE754] (major type 7, additional information 26). 🦋"float64": A number representable as a double-precision float [IEEE754] (major type 7, additional information 27). 🦋"float": One of float16, float32, or float64. 🌈"bstr"or"bytes": A byte string (major type 2). 🧚"tstr"or"text": Text string (major type 3). 🍓
(heads up, there are no predefined names for arrays or maps 🙅♀️ you build those yourself with the syntax coming up, it's giving DIY energy 🛠️)
On top of that, the prelude defines a bunch of types tied to CBOR tags, like "tdate", "bigint", "regexp", and so on. ✨💅
3.4. Arrays 🧺
Array definitions wrap a group in square brackets []. Simple as that 💖
Each entry is allowed to have an occurrence indicator, as spelled out in Section 3.2. 🔁
Here's a look:
unlimited-people = [* person]
one-or-two-people = [1*2 person]
at-least-two-people = [2* person]
person = (
name: tstr,
age: uint,
)So the person group gets repeated inside the array, and each repeat gives you a name and then an age, back and forth 👯♀️ That means these four are all valid values for something of type "unlimited-people" 🌸
["roundlet", 1047, "psychurgy", 2204, "extrarhythmical", 2231]
[]
["aluminize", 212, "climograph", 4124]
["penintime", 1513, "endocarditis", 4084, "impermeator", 1669,
"coextension", 865]Notice the empty array [] slipping in there too 🙈 zero people, zero problems, periodt 💅
3.5. Maps 🗺️
okay, maps get the VIP treatment here because they're probably going to be the main event in a lot of CDDL specs 👑 the syntax doesn't strictly split maps into "struct" and "table" uses, but it has special tools for each one, which is so thoughtful ✨
first, a quick reminder of something CBOR inherited from JSON: the key/value pairs in CBOR maps have no fixed ordering 🌀 you could imagine it being useful to pin the order down. Like, a decoder looking for the values under integer keys 1, 3, and 7 might want to know that if it hits key 4 before key 3, key 3 just isn't there. But neither JSON nor CBOR does this, so CDDL didn't try to add it either 🥲
3.5.1. Structs 🏗️
The "struct" usage of maps is kind of like how JSON objects get used in lots of JSON apps 🧡
A map gets defined the same way an array does (see Section 3.4), except you use curly braces {} instead of square brackets [] 💫
Each group entry can have an occurrence indicator from Section 3.2 too 🔁
Here's a record with a structure tucked inside it:
Geography = [
city : tstr,
gpsCoordinates : GpsCoordinates,
]
GpsCoordinates = {
longitude : uint, ; degrees, scaled by 10^7
latitude : uint, ; degrees, scaled by 10^7
}When you encode this, Geography turns into a CBOR array with two members (the keys for the group entries get ignored, bye bye 👋), whereas GpsCoordinates becomes a CBOR map with two key/value pairs 🗺️
Types used inside a struct can be defined in their own rules or just placed right where they're needed, maybe wrapped in parentheses, especially for choices 🎀 For example:
located-samples = {
sample-point: int,
samples: [+ float],
}So located-samples is the name you use to refer to the struct, and sample-point and samples are the keys 🔑 This is actually a complete example all on its own! An identifier followed by a colon can be used directly as the text string for a member key (we call that a "bareword" member key 💜). A double-quoted string or a number works too. (When other types, especially ones that hold more than one value, get used as key types, they need a double arrow instead. More on that below 👀)
If a text string key doesn't fit the syntax for an identifier, or if you just happen to be a double-quote girlie 🎀, you can write the text string in the member key position, followed by a colon. The example above could be written with quoted strings in those spots instead:
located-samples = {
"sample-point": int,
"samples": [+ float],
}Hmm, wait, I need to be careful here. The original example uses quoted strings followed by a colon, so that's what's shown above, and it's a faithful copy of the spec's idea 🫶
More generally, types written in ways other than the cases above can be used in a key-type position if you put a double arrow after them. This matters most when a type is named by an identifier, because an identifier followed by a colon would get read as a "bareword" and turned into a text string 😭 A literal text string also counts as a type (one that holds just that single string), so another way to write the example is:
located-samples = {
"sample-point" => int,
"samples" => [+ float],
}Check out Section 3.5.4 below for how the colon (":") shortcut also brings along some implied semantics 🧠
Here's a better way to show off the double arrow:
located-samples = {
sample-point: int,
samples: [+ float],
* equipment-type => equipment-tolerances,
}
equipment-type = [name: tstr, manufacturer: tstr]
equipment-tolerances = [+ [float, float]]Next up is a struct with optional entries 🌷 It has a display name (a text string), the name components first name and family name (both text strings), and age info (an unsigned integer):
PersonalData = {
? displayName: tstr,
NameComponents,
? age: uint,
}
NameComponents = (
? firstName: tstr,
? familyName: tstr,
)Something to notice here: the NameComponents group does not make its own map. All four keys end up sitting directly in the struct that PersonalData builds 🧩 Sneaky, right?
In this example, every key/value pair is optional from CDDL's point of view. If there's no occurrence indicator, an entry is mandatory 🚨 So the default is strict, bestie.
Now, if you want to leave the door open for more entries that the current spec doesn't mention, you can add that possibility on purpose:
PersonalData = {
? displayName: tstr,
NameComponents,
? age: uint,
* tstr => any
}
NameComponents = (
? firstName: tstr,
? familyName: tstr,
)Figure 7: Personal Data: Example for Extensibility
The CDDL tool described in Appendix F generated the following as one acceptable instance for this specification 🤖✨
{"familyName": "agust", "antiforeignism": "pretzel",
"springbuck": "illuminatingly", "exuviae": "ephemeris",
"kilometrage": "frogfish"}(And if you want one way to explicitly flag an extension point, see Section 3.9 🦋)
3.5.2. Tables 📋
A table is what you get when you define a map whose entries have a key type that can hold more than just one value 🌈 For example:
square-roots = {* x => y}
x = int
y = floatHere, each key in a key/value pair has the datatype x (which is defined as int), and each value has the datatype y (defined as float) 🔢
If the spec doesn't need to restrict x or y (meaning the application can pick freely for each entry), you can swap in the predefined name "any" 🫧
Another example: this one could work as a conversion table, going from an integer or a float to a string 🔄
tostring = {* mynumber => tstr}
mynumber = int / float3.5.3. Non-deterministic Order 🌪️
Arrays get matched in a fully determined way by the PEG formalism (see Appendix A). Maps are trickier, though, because they don't have an inherent order 😤 For each candidate name/value pair that the PEG algorithm tries, a matching member gets picked out of the whole map. For certain group expressions, more than one member in the map might match. Most of the time, that's no big deal, because the group expression tends to eat up all the matches 🍽️
labeled-values = {
? fritz: number,
* label => value
}
label = text
value = numberSo here, if any member with the key "fritz" is there, the first entry of the group grabs it. All the remaining text/number members get picked up by the second entry. And if anything is left unpicked, the map doesn't match 🙅♀️
BUT it's possible to write group expressions where what actually gets picked is indeterminate, and that choice does matter 😱 Like this one:
do-not-do-this = {
int => int,
int => 6,
}If you match this against {3: 5, 4: 6}, the first group entry might grab 3: 5, leaving 4: 6 for the second entry. Or it might grab 4: 6, leaving nothing for the second entry 💀 This pathological non-determinism happens because the "more general" rule comes before the "more specific" one, and because a general rule only consumes a subset of the key/value pairs it could match. Both of those tend not to show up in real-world map specs, thankfully 🙏 At the time of writing, CDDL tools can't detect these cases automatically, so for the current version of CDDL, the spec writer is simply urged to not write pathologically non-deterministic specifications 🫠
(The astute reader will remember "ambiguous content models" from the Standard Generalized Markup Language (SGML) and "non-deterministic content models" from XML 🤓 This problem is related, but here it's specifically caused by maps lacking order, something the XML schema languages don't have to deal with. And RELAX NG's "interleave" pattern handles lack of order explicitly on the specification side, while XML instances always have a determinate order. Iconic 🏆)
3.5.4. Cuts in Maps
okay so the extensibility idiom we just did for structs has one teeny problem 🙈
extensible-map-example = {
? "optional-key" => int,
* tstr => any
}here there's one optional key "optional-key" that, when it shows up, maps to an integer 🔢. and there's a wildcard * for any future additions 🌸
but the data item
{ "optional-key": "nonsense" }totally matches this spec 😭 the first entry of the group doesn't match, BUT the second one (the wildcard) does, so it gets in anyway 💀 this might be exactly what you want (like if a future extension should be allowed to extend the type of "optional-key"), but in lots of cases it's not 🙅♀️
enter the cut ✂️ CDDL lets you put a cut ^ into the map entry definition, kind of as a preview of a more general feature called "cuts":
extensible-map-example = {
? "optional-key" ^ => int,
* tstr => any
}a cut here means that once the member key matches the name part of an entry that has a cut, any other potential matches for that member's key in later entries of the map's group are no longer allowed 🚫 it's giving "locking in." when a group entry picks a key/value pair just based on a matching key, it locks in that pick. this rule applies whether or not the value matches too, so if it doesn't, the whole map fails to match 💅 in short, the example above no longer matches the spec once you add the cut 🙌
since everyone wants this kind of exclusive matching all the time, the : shortcut already includes cut semantics 🎀 so the example with the cut can be written way more simply as:
extensible-map-example = {
? "optional-key": int,
* tstr => any
}or even shorter, using a bareword for the key:
extensible-map-example = {
? optional-key: int,
* tstr => any
}3.6. Tags
a type can use a CBOR tag (major type 6) with the representation type notation #6.nnn(type), where nnn is an unsigned integer giving the tag number and "type" is the type of the data item being tagged 🏷️
for example, this line from the CDDL prelude (Appendix D) defines biguint as a type name for an unsigned bignum N 🧮
biguint = #6.2(bstr)the tags defined by [RFC7049] are already in the prelude ✨ tags registered after [RFC7049] was written need to be added to a CDDL spec as needed. for example, a binary Universally Unique Identifier (UUID) tag could be referenced as "buuid" in a spec after defining
buuid = #6.37(bstr)and in the following example, using tag 32 for URIs is optional 🌈
my_uri = #6.32(tstr) / tstr3.7. Unwrapping
the group used to define a map or an array can often get reused in the definition of another map or array 🔁 same vibe with a type defined as a tag, which carries an internal data item you'd like to refer to. in these cases it's super handy to just use the name of the map, array, or tag type as a handle for the group or type defined inside it 🫶
the "unwrap" operator is written by putting a tilde ~ in front of a name. it strips the type defined for that name by one layer, exposing the underlying group (for maps and arrays) or type (for tags) 🧅
say an app wants a basic header and an advanced header. without unwrapping, it'd look like this:
basic-header-group = (
field1: int,
field2: text,
)
basic-header = [ basic-header-group ]
advanced-header = [
basic-header-group,
field3: bytes,
field4: number, ; as in the tagged type "time"
]with unwrapping, that simplifies to:
basic-header = [
field1: int,
field2: text,
]
advanced-header = [
~basic-header,
field3: bytes,
field4: ~time,
](ngl, if you forgot the first unwrap in that second example, you'd end up nesting basic-header in its own array inside advanced-header 💀 but with the unwrapped basic-header, the group inside it is basically repeated inside advanced-header, giving you a single array 🎯 this is great for stuff people usually solve with inheritance in programming languages. the effect is like "threading in" the group or type inside the referenced type, which is where the thread-like ~ character comes from 🧵)
3.8. Controls
a _control_ lets you relate a _target_ type with a _controller_ type via a _control operator_ 🎛️
the syntax for a control type is target .control-operator controller, where control operators are special identifiers prefixed by a dot. (note that _target_ or _controller_ might need to be parenthesized 🙃)
a bunch of control operators are defined here. more can be defined by new versions of this spec or by registering them under the procedures in Section 6.1 📋
3.8.1. Control Operator .size
a .size control sets the size of the target in bytes, via the control type. it's defined for text and byte strings, where it directly controls the number of bytes in the string. it's also defined for unsigned integers (more on that below). Figure 8 shows example usage for byte strings 📏
full-address = [[+ label], ip4, ip6]
ip4 = bstr .size 4
ip6 = bstr .size 16
label = bstr .size (1..63)Figure 8: Control for Size in Bytes
when applied to an unsigned integer, the .size control restricts the range of that integer by giving a maximum number of bytes that should be needed in a computer representation of it. in other words, uint .size N is equivalent to 0...BYTES_N, where BYTES_N == 256**N 🔢
audio_sample = uint .size 3 ; 24-bit, equivalent to 0...16777216Figure 9: Control for Integer Size in Bytes
note that, same as value restrictions in CDDL, this control is not a representation constraint 🙅♀️ a number that fits into fewer bytes can still be represented in that form, and an inefficient implementation could use a longer form (unless some format constraints outside CDDL stop that, like the rules in Section 3.9 of [RFC7049]) 🤓
3.8.2. Control Operator .bits
a .bits control on a byte string indicates that, in the target, only the bits numbered by a number in the control type are allowed to be set 🚦 (bits are counted the usual way: bit number "n" being set in "str" means that "(str[n >> 3] & (1 << (n & 7))) != 0".) similarly, a .bits control on an unsigned integer "i" indicates that for all unsigned integers "n" where "(i & (1 << n)) != 0", "n" must be in the control type 🧠
tcpflagbytes = bstr .bits flags
flags = &(
fin: 8,
syn: 9,
rst: 10,
psh: 11,
ack: 12,
urg: 13,
ece: 14,
cwr: 15,
ns: 0,
) / (4..7) ; data offset bits
rwxbits = uint .bits rwx
rwx = &(r: 2, w: 1, x: 0)Figure 10: Control for What Bits Can Be Set
the CDDL tool described in Appendix F generates these ten example instances for tcpflagbytes 🧪
h'906d' h'01fc' h'8145' h'01b7' h'013d' h'409f' h'018e' h'c05f'
h'01fa' h'01fe'these examples don't show that the CDDL spec above doesn't explicitly specify a size of two bytes 👀 a valid all-clear instance of flag bytes could be h'' or h'00' or even h'000000' as well 🫧
3.8.3. Control Operator .regexp
a .regexp control says the text string given as a target needs to match the XML Schema Definition (XSD) regular expression given as a value in the control type 🔍 XSD regex is defined in Appendix F of [W3C.REC-xmlschema-2-20041028] 📖
nai = tstr .regexp "[A-Za-z0-9]+@[A-Za-z0-9]+(\\.[A-Za-z0-9]+)+"Figure 11: Control with an XSD regexp
an example that matches this regex:
"N1@CH57HF.4Znqe0.dYJRN.igjf"3.8.3.1. Usage Considerations
okay the key thing here: XSD regex doesn't support the usual \x or \u escapes for hex bytes or Unicode code points 😤 BUT in CDDL, XSD regexes live inside text strings, and the literal notation for those gives you \u escapes, which should be enough for most apps using regex on text strings 🎉 (this also means there's one level of string escaping before the XSD escaping rules kick in 🌀)
XSD regex supports character class subtraction, which regex libraries often don't have 🧐 spec writers may want to use this feature sparingly. similar thinking applies to Unicode character classes. where they're used, the spec employing CDDL SHOULD identify which Unicode versions are addressed 📌
some other surprises for infrequent XSD regex users:
- No direct support for case insensitivity. case insensitivity has mostly gone out of fashion in protocol design, but when it's needed you have to write it by hand, like
"[Cc][Aa][Ss][Ee]"🙃 - The support for popular character classes like
\wand\dis based on Unicode character properties. that's often not what you want in an ASCII-based protocol, so it can lead to surprises 😵 (\sand\Sdo have their more conventional meanings, and "." matches any character except the line-ending characters\ror\n💫)
3.8.3.2. Discussion
there are lots of flavors of regular expression in the programming world 🌍 for instance, Perl-Compatible Regular Expressions (PCREs) are widely used and probably more useful than XSD regexes. however, there's no normative reference for PCREs that this document could use, so we went with XSD regexes for now 🤷♀️ there's precedent for that choice in the IETF, e.g., in YANG [RFC7950] 💜
note that CDDL uses controls as its main extension point, which opens the door to adding more regex formats alongside this one if wanted 🚪 as an example, a proposal for a .pcre control is defined in [CDDL-Freezer] 🧊
3.8.4. Control Operators .cbor and .cborseq 🎀
okay so a ".cbor" control on a byte string is basically saying "bestie, this byte string is secretly a CBOR-encoded data item" 🦋. Once you decode it, that item has to match the type on the right-hand side (type1 in this example) 💅:
"bytes .cbor type1"".cborseq" is the same vibe but for a byte string holding a sequence of CBOR-encoded data items 🧚. Treat those items as an array, and that array has to match the type on the right-hand side (type2 here) ✨:
"bytes .cborseq type2"(Turning the encoded sequence into an array can be done, for instance, by wrapping the byte string between the two bytes 0x9f and 0xff and then decoding the wrapped byte string as a CBOR-encoded data item. 🌈)
3.8.5. Control Operators .within and .and 💜
a ".and" control on a type means the data item matches both the left-hand-side type and the type on the right. It's giving intersection 🔥 (formally, the resulting type is the intersection of the two types given):
"type1 .and type2"then there's ".within", which is basically ".and" but with extra intent 👀. The left-hand-side type is meant to be a subset of the right-hand-side type:
"type1 .within type2"both forms have the identical formal semantics (intersection), but the whole point of ".within" is that the right side gives guidance to what's allowed on the left. That left side is typically a socket (Section 3.9) 🫶:
message = $message .within message-structure
message-structure = [message_type, *message_option]
message_type = 0..255
message_option = any
$message /= [3, dough: text, topping: [* text]]
$message /= [4, noodles: text, sauce: text, parmesan: bool]for ".within", a tool might flag an error if type1 allows data items that type2 doesn't allow ⚠️. For ".and", though, there's no expectation that type1 is already a subset of type2, so no drama 💖.
3.8.6. Control Operators .lt, .le, .gt, .ge, .eq, .ne, and .default 🍓
the controls .lt, .le, .gt, .ge, .eq, and .ne put a constraint on the left-hand-side type. It has to be less than, less than or equal to, greater than, greater than or equal to, equal to, or not equal to a value on the right (a type containing just that single value) 🌸. In this spec, the first four (.lt, .le, .gt, and .ge) are only defined for numeric types, since those are the ones with a natural ordering 🔢:
speed = number .ge 0 ; unit: m/s".ne" and ".eq" work for numbers AND for other types 🙌. If one of the values isn't numeric, here's how equality goes:
- text strings are equal (satisfy .eq / do not satisfy .ne) if they are bytewise identical. Same goes for byte strings 🧵.
- arrays are equal if they have the same number of elements, and all of them are equal pairwise, in order, between the arrays 📚.
- maps are equal if they have the same number of key/value pairs, and there is pairwise equality between the key/value pairs of the two maps 🗺️.
- tagged values are equal if they both have the same tag and the values are equal 🏷️.
- values of simple types match if they are the same values 🫧.
- numeric types that occur within arrays, maps, or tagged values are equal if their numeric value is equal and they are both integers or both floating-point values 🔢✨.
all other cases are not equal. like, comparing a text string with a byte string? absolutely not 🙅♀️.
a variant of ".ne" is ".default", which has its own extra intent 🎁. The value on the right is meant as a default value for the left-hand-side type, and the implied .ne control is there to stop that value from being sent over the wire. This only makes sense in an optional context, because otherwise there'd be no way to actually use the default value 🥺:
timer = {
time: uint,
? displayed-step: (number .gt 0) .default 1
}3.9. Socket/Plug 🔌
for both type choices and group choices, there's a mechanism that lets you start out with empty choices and assemble them later 🧩. Your pieces can even live in separate files that get concatenated to build the full spec. so iconic 🫶.
by convention, CDDL extension points are marked with a leading dollar sign (for types) or two leading dollar signs (for groups) 💅. Tools respect that and won't raise an error if such a type or group isn't defined at all. The symbol just gets treated as an empty type choice (group choice), meaning no choice is available:
tcp-header = {seq: uint, ack: uint, * $$tcp-option}
; later, in a different file
$$tcp-option //= (
sack: [+(left: uint, right: uint)]
)
; and, maybe in another file
$$tcp-option //= (
sack-permitted: true
)names that start with a single "$" are "type sockets". They start out as an empty type and are meant to get extended via "/=" 🔧. Names that start with a double "$$" are "group sockets". They start out as an empty group choice and get extended via "//=" 🧃. Either way, it's not an error if a socket has no definition at all. That just means there's no way to satisfy the rule, so the choice is empty 🫠.
as a convention, all definitions (plugs) for socket names must be augmentations, so they must be using "/=" and "//=" respectively 📝.
to pick up the example from Figure 7, the socket/plug mechanism could be used like this in Figure 12 🎀:
PersonalData = {
? displayName: tstr,
NameComponents,
? age: uint,
* $$personaldata-extensions
}
NameComponents = (
? firstName: tstr,
? familyName: tstr,
)
; The above already works as is.
; But then, we can add later:
$$personaldata-extensions //= (
favorite-salsa: tstr,
)
; and again, somewhere else:
$$personaldata-extensions //= (
shoesize: uint,
)Figure 12: Personal Data Example: Using Socket/Plug Extensibility
3.10. Generics 🦋
with angle brackets, the left-hand side of a rule can add formal parameters after the name being defined ✨:
messages = message<"reboot", "now"> / message<"sleep", 1..100>
message<t, v> = {type: t, value: v}when you use a generic rule, the formal parameters get bound to the actual arguments you supply (also in angle brackets). This happens within the scope of the generic rule, as if there were a rule of the form parameter = argument 🧠.
generic rules can be used to name both types and groups 🙌.
(heads up, though: there are some limitations on nesting generics in the CDDL tool described in Appendix F 🙈.)
3.11. Operator Precedence 👑
okay, so CDDL is a language with multiple syntactic features, including prefix and infix operators, and some operators bind tighter than others 😭. This is way more complicated than, say, ABNF, because CDDL has both types and groups, and each has its own operators. Type operators (like "/" for type choice) work on types. Group operators (like "//" for group choice) work on groups. Types can just be dropped into groups, but groups need brackets (as arrays or maps) before they become types. So type operators naturally bind closer than group operators 🔗.
for example, in:
t = [group1]
group1 = (a / b // c / d)
a = 1 b = 2 c = 3 d = 4group1 is a group choice between the type choice of a and b, and the type choice of c and d 🎯. This gets more relevant once member keys and/or occurrences show up:
t = {group2}
group2 = (? ab: a / b // cd: c / d)
a = 1 b = 2 c = 3 d = 4group2 is a group choice between the optional member "ab" (of type a or b) and the member "cd" (of type c or d). Notice the optionality is attached to the first choice ("ab"), NOT the second choice 👀.
similarly, in:
t = [group3]
group3 = (+ a / b / c)
a = 1 b = 2 c = 3group3 is a repetition of a type choice between a, b, and c. If only a is supposed to be repeatable, you need a group choice to focus the occurrence 🎀:
t = [group4]
group4 = (+ a // b / c)
a = 1 b = 2 c = 3group4 is a group choice between a repeatable a and a single b or c 💖.
one comment on this has been that the semantics of group3 could be counterintuitive 🙃. In general, like with lots of other languages that have operator precedence rules, spec writers are encouraged NOT to rely on them. Instead, add parentheses liberally to guide readers who aren't familiar with CDDL precedence rules 🫶:
t = [group4a]
group4a = ((+ a) // (b / c))
a = 1 b = 2 c = 3the operator precedences, from loosest to tightest binding, are defined in Appendix B and summarized in Table 1 📋. (Arities are 1 for unary prefix operators and 2 for binary infix operators.)
+----------+-------+---------------------------+------------+
| Operator | Arity | Operates on | Precedence |
+----------+-------+---------------------------+------------+
| = | 2 | name = type, name = group | 1 |
| /= | 2 | name /= type | 1 |
| //= | 2 | name //= group | 1 |
| // | 2 | group // group | 2 |
| , | 2 | group, group | 3 |
| * | 1 | * group | 4 |
| n*m | 1 | n*m group | 4 |
| + | 1 | + group | 4 |
| ? | 1 | ? group | 4 |
| => | 2 | type => type | 5 |
| : | 2 | name: type | 5 |
| / | 2 | type / type | 6 |
| .. | 2 | type..type | 7 |
| ... | 2 | type...type | 7 |
| .ctrl | 2 | type .ctrl type | 7 |
| & | 1 | &group | 8 |
| ~ | 1 | ~type | 8 |
+----------+-------+---------------------------+------------+Table 1: Summary of Operator Precedences
4. Making Use of CDDL ✨
in this section, we talk about a few ways you could use CDDL 💅.
4.1. As a Guide for a Human User 🧚
CDDL can be used to define the layout of CBOR data really efficiently, so a human implementer can easily see how the data is supposed to be encoded 👀🎀.
since CDDL maps parts of the CBOR data to human-readable names, tools could be built on top of it. They could give you a human-friendly view of the CBOR data and let you edit it while staying compliant with its CDDL definition 🫶💖.
4.2. For Automated Checking of CBOR Data Structures 🔍
okay bestie, so CDDL is built so a machine can handle the CDDL definition and the related CBOR data (and, like, JSON data too 🤯). For example, a machine could use CDDL to check whether or not CBOR data is compliant with its definition ✅
how thorough that checking needs to be totally depends on the app 💅 An app might decide not to check the data structure at all and just use the CDDL definition as a way to show the programmer what the data is supposed to look like 📝
on the flip side, the app could also build a checking mechanism that goes all the way and makes sure every mandatory map member is actually there 🧐
how far the data description has to be enforced is left to the designers and implementers of that app, keeping in mind related security considerations 🔐
and just so we're clear: in no case is a CDDL tool meant to be "writing code" for an implementation. absolutely not 💀
4.3. For Data Analysis Tools 📊
in the long run, more and more data is expected to be stored using the CBOR data format ✨
where there's data, there's data analysis, and the need to process it automatically 🤖 CDDL can be used for this kind of automated data processing, letting tools verify data, clean it, and pull out the particular parts of interest from it 🧹
since CBOR is designed with constrained devices in mind, a likely use of it is small sensors 🌡️ which makes automated analysis of sensor data an iconic use case 🌈
5. Security Considerations 🛡️
this document presents a content rules language for expressing CBOR data structures. so, as such, it doesn't bring any security issues on its own 🙅♀️ however, specifications of protocols that use CBOR naturally need security analyses when they're defined, obviously 🚨 general guidelines for writing security considerations are defined in [RFC3552] (BCP 72). specifications using CDDL to define CBOR structures in protocols need to follow those guidelines, no cap 🫶 additional topics that could be considered in a security considerations section for a specification that uses CDDL to define CBOR structures include the following:
- where could the language maybe cause confusion in a way that will enable security issues? 👀
- where a CDDL matcher is part of the implementation of a system, the security of the system ought not depend on the correctness of the CDDL specification or CDDL implementation without any further defenses in place. this is the part that scares me the most 😭
- where the CDDL specification includes extension points, the impact of extensions on the security of the system needs to be carefully considered 🧐
writers of CDDL specifications are strongly encouraged to value clarity and transparency of the specification over its elegance. keep it as simple as possible while still expressing the needed data model 💖 elegance is cute, but clarity is iconic 👑
a related observation about formal description techniques in general, which is strongly recommended to be kept in mind by writers of CDDL specifications: just because CDDL makes it easier to handle complexity in a specification, that does not make that complexity somehow less bad. (the only exception is maybe on the level of the humans having to grasp the complex structure while reading the spec 🙈)
6. IANA Considerations 📋
6.1. CDDL Control Operators Registry 🎀
IANA has created a registry for control operators (Section 3.8). the "CDDL Control Operators" registry has been created within the "Concise Data Definition Language (CDDL)" registry 🗂️
each entry in the subregistry must include the name of the control operator (by convention given with the leading dot) and a reference to its documentation. names must be composed of the leading dot followed by a text string conforming to the production "id" in Appendix B 📝
initial entries in this registry are as follows:
+----------+---------------+
| Name | Documentation |
+----------+---------------+
| .size | RFC 8610 |
| .bits | RFC 8610 |
| .regexp | RFC 8610 |
| .cbor | RFC 8610 |
| .cborseq | RFC 8610 |
| .within | RFC 8610 |
| .and | RFC 8610 |
| .lt | RFC 8610 |
| .le | RFC 8610 |
| .gt | RFC 8610 |
| .ge | RFC 8610 |
| .eq | RFC 8610 |
| .ne | RFC 8610 |
| .default | RFC 8610 |
+----------+---------------+all other control operator names are Unassigned 🌸
the IANA policy for additions to this registry is "Specification Required" as defined in [RFC8126] (which involves an Expert Review) for names that do not include an internal dot, and "IETF Review" for names that do include an internal dot 🔁 the expert reviewer is specifically instructed that other Standards Development Organizations (SDOs) may want to define control operators that are specific to their fields (e.g., based on a binary syntax already in use at the SDO). the review process should strive to facilitate such an undertaking 🤝 honestly, that's so considerate of them 🥹
7. References 📚
7.1. Normative References
- [ISO6093] ISO, "Information processing -- Representation of numerical values in character strings for information interchange", ISO 6093, 1985.
- [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997,
https://www.rfc-editor.org/info/rfc2119. - [RFC3552] Rescorla, E. and B. Korver, "Guidelines for Writing RFC Text on Security Considerations", BCP 72, RFC 3552, DOI 10.17487/RFC3552, July 2003,
https://www.rfc-editor.org/info/rfc3552. - [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO 10646", STD 63, RFC 3629, DOI 10.17487/RFC3629, November 2003,
https://www.rfc-editor.org/info/rfc3629. - [RFC4648] Josefsson, S., "The Base16, Base32, and Base64 Data Encodings", RFC 4648, DOI 10.17487/RFC4648, October 2006,
https://www.rfc-editor.org/info/rfc4648. - [RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, DOI 10.17487/RFC5234, January 2008,
https://www.rfc-editor.org/info/rfc5234. - [RFC7049] Bormann, C. and P. Hoffman, "Concise Binary Object Representation (CBOR)", RFC 7049, DOI 10.17487/RFC7049, October 2013,
https://www.rfc-editor.org/info/rfc7049. - [RFC7493] Bray, T., Ed., "The I-JSON Message Format", RFC 7493, DOI 10.17487/RFC7493, March 2015,
https://www.rfc-editor.org/info/rfc7493. - [RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 8126, DOI 10.17487/RFC8126, June 2017,
https://www.rfc-editor.org/info/rfc8126. - [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017,
https://www.rfc-editor.org/info/rfc8174. - [RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data Interchange Format", STD 90, RFC 8259, DOI 10.17487/RFC8259, December 2017,
https://www.rfc-editor.org/info/rfc8259. - [W3C.REC-xmlschema-2-20041028] Biron, P. and A. Malhotra, "XML Schema Part 2: Datatypes Second Edition", World Wide Web Consortium Recommendation REC-xmlschema-2-20041028, October 2004,
https://www.w3.org/TR/2004/REC-xmlschema-2-20041028.
7.2. Informative References
- [CDDL-Freezer] Bormann, C., "A feature freezer for the Concise Data Definition Language (CDDL)", Work in Progress, draft-bormann-cbor-cddl-freezer-01, August 2018.
- [GRASP] Bormann, C., Carpenter, B., Ed., and B. Liu, Ed., "A Generic Autonomic Signaling Protocol (GRASP)", Work in Progress, draft-ietf-anima-grasp-15, July 2017.
- [IEEE754] IEEE, "IEEE Standard for Floating-Point Arithmetic", IEEE Std 754-2008.
- [JCR] Newton, A. and P. Cordell, "A Language for Rules Describing JSON Content", Work in Progress, draft-newton-json-content-rules-09, September 2017.
- [PEG] Ford, B., "Parsing expression grammars: a recognition-based syntactic foundation", Proceedings of the 31st ACM SIGPLAN-SIGACT symposium on Principles of programming languages - POPL '04, DOI 10.1145/964001.964011, January 2004.
- [RELAXNG] ISO/IEC, "Information technology -- Document Schema Definition Language (DSDL) -- Part 2: Regular-grammar-based validation -- RELAX NG", ISO/IEC 19757-2, December 2008.
- [RFC7071] Borenstein, N. and M. Kucherawy, "A Media Type for Reputation Interchange", RFC 7071, DOI 10.17487/RFC7071, November 2013,
https://www.rfc-editor.org/info/rfc7071. - [RFC7950] Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", RFC 7950, DOI 10.17487/RFC7950, August 2016,
https://www.rfc-editor.org/info/rfc7950. - [RFC8007] Murray, R. and B. Niven-Jenkins, "Content Delivery Network Interconnection (CDNI) Control Interface / Triggers", RFC 8007, DOI 10.17487/RFC8007, December 2016,
https://www.rfc-editor.org/info/rfc8007. - [RFC8152] Schaad, J., "CBOR Object Signing and Encryption (COSE)", RFC 8152, DOI 10.17487/RFC8152, July 2017,
https://www.rfc-editor.org/info/rfc8152. - [RFC8428] Jennings, C., Shelby, Z., Arkko, J., Keranen, A., and C. Bormann, "Sensor Measurement Lists (SenML)", RFC 8428, DOI 10.17487/RFC8428, August 2018,
https://www.rfc-editor.org/info/rfc8428. - [YAML] Ben-Kiki, O., Evans, C., and I. Net, "YAML Ain't Markup Language (YAML[TM]) Version 1.2", 3rd Edition, October 2009,
https://yaml.org/spec/1.2/spec.html.
Appendix A. Parsing Expression Grammars (PEGs) 🦋
This appendix is normative. 💅
Okay so, since the 1950s, a bunch of grammar notations have been built on Backus-Naur Form (BNF) 🌸, which is a notation for context-free grammars (CFGs) inside Chomsky's generative system of grammars. The Augmented Backus-Naur Form (ABNF) [RFC5234] is a famous example, and it's widely used in IETF specs. It also inspired the syntax of CDDL ✨.
Generative grammars are amazing at expressing ambiguity, but that same superpower can make them a pain to use in recognition systems. So people kept spinning off subdialects that put rules on generative grammars so they'd work with parser generators, and honestly the specification writer may find that hard to manage 😭.
PEGs [PEG] are a different formal foundation for describing grammars. They focus on recognition over generation, and they clear up what would've been ambiguity by introducing the concept of "prioritized choice" 👑.
The PEG notation is pretty close to BNF, with the usual "Extended BNF" features like repetition added on. The big difference is the choice operator. BNF uses the unordered (symmetrical) choice operator "|" (which ABNF writes as "/"), but PEG has a prioritized choice operator "/". The two alternatives get tested left to right. The first successful match locks in, and any other potential matches within that choice get ignored. (Alternatives in choices that contain this choice aren't disabled, though, unlike what a cut (Section 3.5.4) would do.) ✂️
Here's the vibe check. These two ABNF expressions:
A = "a" "b" / "a" (1)and
A = "a" / "a" "b" (2)are equivalent in ABNF's original generative framework, but they're totally different in PEG 🙈. In (2), the second alternative will never match, because any input starting with an "a" already succeeds in the first alternative and locks it in. Main character energy for the first option, rip to the second 💀.
Occurrence indicators ("?", "", "+") are "greedy" in PEG too. They eat as much input as they can match. So as a consequence, "a a" in PEG notation (or "a a" in CDDL syntax) can never match anything, because the initial "a" already gobbled up every "a", leaving nothing for the second "a" 😩.
Here's a fun fact. The grammar of CDDL itself, written in ABNF in Appendix B, can be read both (1) in the generative framework RFC 5234 is based on and (2) as a PEG. That's possible because the choices are ordered so that a successful match on the left-hand side of a "/" operator is always the intended match. It doesn't rely on the power of symmetrical choices. For example, check out the alternatives in the rule for "uint", where the lone zero sits behind the longer match alternatives that start with a zero 🧚.
The syntax for the PEG component of CDDL is based on ABNF, interpreted in the obvious way with PEG semantics. The ABNF convention of writing occurrence indicators before the controlled primary, and of allowing numeric values for minimum and maximum occurrence around a "*" sign, got copied over too. PEG only deals with characters, but CDDL has a richer set of elements, like types and groups. These constructs map like so:
+-------+-------+-------------------------------------------+
| CDDL | PEG | Remark |
+-------+-------+-------------------------------------------+
| "=" | "<-" | /= and //= are abbreviations |
| "//" | "/" | prioritized choice |
| "/" | "/" | prioritized choice, limited to types only |
| "?" P | P "?" | zero or one |
| "*" P | P "*" | zero or more |
| "+" P | P "+" | one or more |
| A B | A B | sequence |
| A, B | A B | sequence, comma is decoration only |
+-------+-------+-------------------------------------------+The literal notation, plus the use of square brackets, curly braces, tildes, ampersands, and hash marks, is specific to CDDL and has nothing to do with conventional PEG notation 🎀. The DOT (".") from PEG gets replaced by the plain "#" or its alias "any". And CDDL doesn't offer the syntactic predicate operators NOT ("!") or AND ("&") from PEG. That keeps things simpler, with less expressiveness and less complexity to deal with 🫶.
If you want the deep theory behind PEG and the cool properties of its operators, like associativity and distributivity, go check out [PEG] 📚.
Appendix B. ABNF Grammar 🌈
This appendix is normative. ✨
Here's the formal definition of CDDL syntax in ABNF [RFC5234]. Heads up, bestie: as ABNF defines it, the quote-delimited strings below are case insensitive. String values and names, though, are case sensitive in CDDL 🙈.
cddl = S 1*(rule S)
rule = typename [genericparm] S assignt S type
/ groupname [genericparm] S assigng S grpent
typename = id
groupname = id
assignt = "=" / "/="
assigng = "=" / "//="
genericparm = "<" S id S *("," S id S ) ">"
genericarg = "<" S type1 S *("," S type1 S ) ">"
type = type1 *(S "/" S type1)
type1 = type2 [S (rangeop / ctlop) S type2]
; space may be needed before the operator if type2 ends in a name
type2 = value
/ typename [genericarg]
/ "(" S type S ")"
/ "{" S group S "}"
/ "[" S group S "]"
/ "~" S typename [genericarg]
/ "&" S "(" S group S ")"
/ "&" S groupname [genericarg]
/ "#" "6" ["." uint] "(" S type S ")"
/ "#" DIGIT ["." uint] ; major/ai
/ "#" ; any
rangeop = "..." / ".."
ctlop = "." id
group = grpchoice *(S "//" S grpchoice)
grpchoice = *(grpent optcom)
grpent = [occur S] [memberkey S] type
/ [occur S] groupname [genericarg] ; preempted by above
/ [occur S] "(" S group S ")"
memberkey = type1 S ["^" S] "=>"
/ bareword S ":"
/ value S ":"
bareword = id
optcom = S ["," S]
occur = [uint] "*" [uint]
/ "+"
/ "?"
uint = DIGIT1 *DIGIT
/ "0x" 1*HEXDIG
/ "0b" 1*BINDIG
/ "0"
value = number
/ text
/ bytes
int = ["-"] uint
; This is a float if it has fraction or exponent; int otherwise
number = hexfloat / (int ["." fraction] ["e" exponent ])
hexfloat = ["-"] "0x" 1*HEXDIG ["." 1*HEXDIG] "p" exponent
fraction = 1*DIGIT
exponent = ["+"/"-"] 1*DIGIT
text = %x22 *SCHAR %x22
SCHAR = %x20-21 / %x23-5B / %x5D-7E / %x80-10FFFD / SESC
SESC = "\" (%x20-7E / %x80-10FFFD)
bytes = [bsqual] %x27 *BCHAR %x27
BCHAR = %x20-26 / %x28-5B / %x5D-10FFFD / SESC / CRLF
bsqual = "h" / "b64"
id = EALPHA *(*("-" / ".") (EALPHA / DIGIT))
ALPHA = %x41-5A / %x61-7A
EALPHA = ALPHA / "@" / "_" / "$"
DIGIT = %x30-39
DIGIT1 = %x31-39
HEXDIG = DIGIT / "A" / "B" / "C" / "D" / "E" / "F"
BINDIG = %x30-31
S = *WS
WS = SP / NL
SP = %x20
NL = COMMENT / CRLF
COMMENT = ";" *PCHAR CRLF
PCHAR = %x20-7E / %x80-10FFFD
CRLF = %x0A / %x0D.0A
Figure 13: CDDL ABNFQuick note: this ABNF doesn't try to capture the detailed rules for what can go inside a prefixed byte string. So you'll need to look somewhere else for those 🦋.
Appendix C. Matching Rules ✨
This appendix is normative 💅, so it's the real spec, not just vibes. We're going through the ABNF syntax rules from Appendix B and briefly explaining how each syntactic feature decides matching. Here, an instance (a data item) "matches" a CDDL specification if the spec allows it. That gets broken down into parts of the spec (type and group expressions) and parts of the instance (data items). 🧚
cddl = S 1*(rule S)A CDDL specification is a sequence of one or more rules. Each rule gives a name to a right-hand-side expression, which is either a CDDL type or a CDDL group. Rule names can be used inside the rule itself and/or in other rules, and tools can warn you if that isn't the case 👀. The order of the rules only matters in two cases:
- The first rule defines the semantics of the entire specification 👑. That means the root rule doesn't need a special name or special syntax (unlike "start" in RELAX NG), so you can pick something descriptive. Like every other rule name, the name of the initial rule may be used in itself or in other rules.
- When a rule adds to a type or group choice (using "/=" or "//="), that choice gets filled in the order the rules appear. More on that below 🎀.
rule = typename [genericparm] S assignt S type
/ groupname [genericparm] S assigng S grpent
typename = id
groupname = idA rule defines a name for a type expression (production "type") or for a group expression (production "grpent"). The idea is that the meaning doesn't change when you swap the name for its definition (wrapped in parentheses if needed) 🦋. Whether a name stands for a type or a group isn't always obvious from the syntax alone. For example, in "a = b", "a" is a type if "b" is a type, and a group if "b" is a group. It gets even trickier with "a = (b)": "a" can be used as a type if "b" is a type, or as a group when "b" is a group and when "b" is a type. A good convention to make that second case stand out to a human reader is to write "a = (b,)" 🤓. Parentheses can also mean two things inside an expression, but the surrounding context usually makes it clear. And it may not be obvious right away whether "b" is a group or a type either. Figuring that out might take several levels of rule definitions 🌀.
assignt = "=" / "/="
assigng = "=" / "//="A plain equals sign defines the rule name as equal to the expression on the right. It's an error if that name was already defined with a different expression 🙅♀️. A "/=" or "//=" adds more choices to an existing named type or group. You could replace a bunch of these by collecting all the right-hand sides into one rule with a single type choice or group choice, built in the order the rules are given. (Extending a rule name that hasn't been defined yet is not an error. It just makes that right-hand side the first entry in the choice being built ✨.)
genericparm = "<" S id S *("," S id S ) ">"
genericarg = "<" S type1 S *("," S type1 S ) ">"Rule names can take generic parameters. These create temporary assignments inside the right-hand side, mapping the parameter names to the arguments you pass when you reference the rule name 🧩.
type = type1 *(S "/" S type1)A type can be a choice between one or more types. The choice matches a data item if the data item matches any one of the types in the choice. The choice follows PEG semantics (see Appendix A), so the first choice that matches wins 🥇. That's why the order of rules contributing to one rule name can really matter.
type1 = type2 [S (rangeop / ctlop) S type2]You can combine two types with a range operator (more on that below) or a control operator (see Section 3.8) 🎯.
type2 = valueA type can be just a single value, like 1, "icecream", or h'0815'. That matches only a data item with that exact value, and no conversions are defined 🍓.
/ typename [genericarg]Or a type can be defined by a rule that gives a meaning to a name. You may need to supply generic arguments if the rule has generic parameters 🌸.
/ "(" S type S ")"Or it can be a type expression in parentheses. Parentheses are sometimes needed to override operator precedence 🙃.
/ "{" S group S "}"This is a map expression. It matches a valid CBOR map whose key/value pairs can be ordered so that the resulting sequence matches the group expression 🗺️.
/ "[" S group S "]"This is an array expression. It matches a CBOR array whose elements, taken as values and each paired with a wildcard key (one that matches anything), all match the group 💜.
/ "~" S typename [genericarg]This is an "unwrapped" group (see Section 3.7). It matches the group inside a type that is defined as a map or an array by wrapping that group 🎁.
/ "&" S "(" S group S ")"
/ "&" S groupname [genericarg]This is an enumeration expression. It matches any value that is within the set of values the given group can take 🌈.
/ "#" "6" ["." uint] "(" S type S ")"This is a tagged data item. It's tagged with the given "uint", and the type given is the tagged value 🏷️.
/ "#" DIGIT ["." uint] ; major/aiThis matches a data item of the major type given by the DIGIT. You can optionally constrain it to the additional information given by the uint 🔍.
/ "#" ; anyAnd this one matches any data item at all 🌟.
rangeop = "..." / ".."A range operator joins two type expressions that stand for either two integer values or two floating-point values. It matches any value between those two values. The first value is always included in the matching set. The second value is included for ".." and excluded for "..." 📏.
ctlop = "." idA control operator ties a _target_ type to a _controller_ type, as defined in Section 3.8. Control operators are an extension point for CDDL, so additional documents may want to define their own 🔌.
group = grpchoice *(S "//" S grpchoice)A group matches any sequence of key/value pairs that matches any of the choices given. Again, that's PEG semantics 🎀.
grpchoice = *(grpent optcom)Each component group is a sequence of group entries. For a match, the sequence of key/value pairs needs to match the sequence of group entries in order 🧁.
grpent = [occur S] [memberkey S] typeA group entry can be given by a value type, which must be matched by the value part of a single element. It can also optionally include a memberkey type, which must be matched by the key part of the element, if a memberkey is given. If there's no memberkey, the entry can only be used to match arrays, not maps 🙈. (The occurrence indicator changes this a bit. More on that below.)
/ [occur S] groupname [genericarg] ; preempted by aboveA group entry can also be built from a named group 🫶.
/ [occur S] "(" S group S ")"Or from a parenthesized group, again with an optional occurrence indicator 💅.
memberkey = type1 S ["^" S] "=>"
/ bareword S ":"
/ value S ":"Key types can be given by a type expression, a bareword (which stands for a type that just holds a text string value made from that bareword), or a value (which stands for a type that holds just that value) 🔑. A key value matches its key type if the key value is a member of that key type. The exception is when a cut earlier in the group applies. See Section 3.5.4 to find out how map matching is affected by the cuts written as "^" or ":" in previous entries ✂️.
bareword = idA bareword is just another way to write a type with a single text string value. You can only use it in the syntactic context shown above 🌷.
optcom = S ["," S]Optional commas don't change the matching at all 💖.
occur = [uint] "*" [uint]
/ "+"
/ "?"An occurrence indicator changes the group to its right by requiring that group to match the sequence exactly a certain number of times in a row (see Section 3.2). In other words, it acts like a (possibly infinite) group choice whose options repeat the group 1, 2, 3, and so on, for each allowed occurrence 🔁.
The rest of the ABNF describes value notation that should feel familiar from programming languages. The exceptions are h'..' and b64'..' for byte strings, plus things like comments and line ends 🦋.
Appendix D. Standard Prelude 💜
This appendix is normative 💅. The prelude below is automatically added to every CDDL file. (Technically it's a postlude, since it doesn't disturb the selection of the first rule as the root of the definition 🌈.)
any = #
uint = #0
nint = #1
int = uint / nint
bstr = #2
bytes = bstr
tstr = #3
text = tstr
tdate = #6.0(tstr)
time = #6.1(number)
number = int / float
biguint = #6.2(bstr)
bignint = #6.3(bstr)
bigint = biguint / bignint
integer = int / bigint
unsigned = uint / biguint
decfrac = #6.4([e10: int, m: integer])
bigfloat = #6.5([e2: int, m: integer])
eb64url = #6.21(any)
eb64legacy = #6.22(any)
eb16 = #6.23(any)
encoded-cbor = #6.24(bstr)
uri = #6.32(tstr)
b64url = #6.33(tstr)
b64legacy = #6.34(tstr)
regexp = #6.35(tstr)
mime-message = #6.36(tstr)
cbor-any = #6.55799(any)
float16 = #7.25
float32 = #7.26
float64 = #7.27
float16-32 = float16 / float32
float32-64 = float32 / float64
float = float16-32 / float64
false = #7.20
true = #7.21
bool = false / true
nil = #7.22
null = nil
undefined = #7.23Figure 14: CDDL Prelude 🎀
The prelude is deemed fixed. That means, for example, any additional tags beyond those defined in [RFC7049] (as registered) need to be defined in each CDDL file that uses them 🧾.
Here's a common stumbling block 🙋♀️: the prelude does not define a type "string". CBOR has byte strings ("bytes" in the prelude) and text strings ("text"), so a type simply called "string" would be ambiguous. Use "bytes" or "text" instead! 💀✨
Appendix E. Use with JSON ✨
This appendix is normative 📜, so it's a real rules section, not just vibes.
Okay so the JSON generic data model (the one implicit in [RFC8259]) is a subset of the CBOR generic data model 💖. That means you can use CDDL with JSON as long as you stick to what JSON can actually represent. Basically you leave out byte strings, tags, and simple values other than false, true, and null. Here's the slimmed-down prelude 🎀:
any = #
uint = #0
nint = #1
int = uint / nint
tstr = #3
text = tstr
number = int / float
float16 = #7.25
float32 = #7.26
float64 = #7.27
float16-32 = float16 / float32
float32-64 = float32 / float64
float = float16-32 / float64
false = #7.20
true = #7.21
bool = false / true
nil = #7.22
null = nilFigure 15: JSON-Compatible Subset of CDDL Prelude
(The major types here don't mean anything directly in JSON, but you can read them as CBOR major types translated through Section 4 of [RFC7049].)
There are a few fine points 👀. First up, JSON doesn't tell integers and floating-point numbers apart. There's only one kind of number, and it might just happen to be whole. So when you specify a type as "uint", "nint", or "int", it turns into a check that the number is integral. For example, all of these JSON numbers match "uint":
10 10.0 1e1 1.0e1 100e-1(Surprising, I know, but if you grew up with languages where a decimal point or exponent means floating-point, this will feel weird 🙃.)
CDDL knows about all the different CBOR number types, but JSON has only one. So specifying a floating-point precision (float16/float32/float64) just restricts the values to ones that fit into binary16/binary32/binary64. That's unlikely to be super useful when you're using CDDL to specify JSON data structures 🤷♀️.
Here's the big picture: JSON's number system is based on decimal numbers and decimal fractions, and it has no limits on precision or range. In practice though, JSON numbers often get parsed into a number type that's called "float64" here, and that creates limitations to the generic data model [RFC7493]. The main consequence is that integers can only be expressed with interoperable exactness when they lie in the range [-(253)+1, (253)-1]. That's a smaller range than the one CDDL "int" covers 😬.
JSON applications that want to stay compatible with I-JSON ("Internet JSON"; see [RFC7493]) may therefore want to define integer types with more limited ranges, like in Figure 16. Note that these types are not part of the prelude, so you need to copy them into your CDDL specification if you need them 📋.
ij-uint = 0..9007199254740991
ij-nint = -9007199254740991..-1
ij-int = -9007199254740991..9007199254740991Figure 16: I-JSON Types for CDDL (Not Part of Prelude)
JSON applications that don't need to stay I-JSON compatible, and that actually need to go beyond the 64-bit unsigned and negative integers that "int" (= "uint"/"nint") supports, may want to use these extra types from the standard prelude 🚀. They're expressed in terms of tags, but they can straightforwardly be mapped into JSON (but not I-JSON) numbers:
biguint = #6.2(bstr)
bignint = #6.3(bstr)
bigint = biguint / bignint
integer = int / bigint
unsigned = uint / biguintCDDL can't currently express the unlimited floating-point precision that's theoretically possible with JSON. At the time of writing, that's rarely used in protocols anyway 🌈.
Also, a data model described in CDDL is always restricted by what the serialization can express. For instance, floating-point values like NaN (not a number) and the infinities can't be represented in JSON, even if the CDDL generic data model allows them 💀.
Appendix F. A CDDL Tool 🛠️
This appendix is for information only.
There's a rough CDDL tool floating around 🧚. For CDDL specifications, it can check the syntax, generate one or more instances (in CBOR diagnostic notation or pretty-printed JSON), and validate an existing instance against the specification. Here's how you use it:
Usage:
cddl spec.cddl generate [n]
cddl spec.cddl json-generate [n]
cddl spec.cddl validate instance.cbor
cddl spec.cddl validate instance.jsonFigure 17: CDDL Tool Usage
To install it, you need a modern Ruby, then run this 💎:
gem install cddlFigure 18: CDDL Tool Installation
The accompanying CBOR diagnostic tools get installed automatically with the above 🚗, and they're documented at https://github.com/cabo/cbor-diag. They can convert between binary CBOR, a pretty-printed hexadecimal form of binary CBOR, CBOR diagnostic notation, JSON, and YAML [YAML].
Appendix G. Extended Diagnostic Notation ✨
This appendix is normative 📜.
Section 6 of [RFC7049] defines a "diagnostic notation" so you can talk about CBOR data items without having to touch binary 🗣️. It's based on JSON, with extensions for things like binary data and tags.
(Standardizing this together with the actual interchange format doesn't create another interchange format. It just lets tools and documents about CBOR share one diagnostic notation, which is honestly so iconic 💜.)
This appendix covers a few extensions to the diagnostic notation that have turned out to be useful since RFC 7049 was written. We call the result Extended Diagnostic Notation (EDN) 🦋.
G.1. Whitespace in Byte String Notation
Examples look so much better with some whitespace (spaces, line breaks) in byte strings 🌸. In EDN, whitespace is ignored in prefixed byte strings, so these are all equivalent:
h'48656c6c6f20776f726c64'
h'48 65 6c 6c 6f 20 77 6f 72 6c 64'
h'4 86 56c 6c6f
20776 f726c64'G.2. Text in Byte String Notation
Diagnostic notation writes byte strings in one of the base encodings per [RFC4648]. You wrap them in single quotes and add a prefix: h for base16, b32 for base32, h32 for base32hex, or b64 for base64 or base64url. Quite often, byte strings carry bytes that make sense as UTF-8 text, so EDN lets you skip the prefix and use single quotes for those. These are equivalent 💅:
'hello world'
h'68656c6c6f20776f726c64'The escaping rules of JSON strings apply equivalently to text-based byte strings. For example, \ stands for a single backslash and ' stands for a single quote. Whitespace is included literally, i.e., the previous section does not apply to text-based byte strings 🙈.
G.3. Embedded CBOR and CBOR Sequences in Byte Strings
When a byte string is supposed to carry an embedded CBOR-encoded item, or more generally a sequence of zero or more such items, you can write the diagnostic notation for those items, separated by commas, inside << and >>. That notates the byte string you get from encoding the items and concatenating the result. Each pair of columns below is equivalent ✨:
<<1>> h'01'
<<1, 2>> h'0102'
<<"foo", null>> h'63666F6FF6'
<<>> h''G.4. Concatenated Strings
Whitespace lets you line-break encoded byte strings, but we also need a way to put text strings and byte strings in direct UTF-8 representation into line-based documents like RFCs and source code 📝. So EDN lets you write multiple text strings, or multiple byte strings, separated by whitespace. They get glued together into a single text or byte string. Text and byte strings don't mix within such a concatenation, except that byte string notation can be used inside a run of text string notation to encode characters that are better represented in an encoded way. These four values are equivalent:
"Hello world"
"Hello " "world"
"Hello" h'20' "world"
"" h'48656c6c6f20776f726c64' ""Similarly, these byte string values are equivalent:
'Hello world'
'Hello ' 'world'
'Hello ' h'776f726c64'
'Hello' h'20' 'world'
'' h'48656c6c6f20776f726c64' '' b64''
h'4 86 56c 6c6f' h' 20776 f726c64'(Heads up: separating by whitespace is familiar from the C language, but it needs some attention. A single comma makes a big difference here 😭.)
G.5. Hexadecimal, Octal, and Binary Numbers
On top of JSON's decimal numbers, EDN gives you hexadecimal, octal, and binary numbers in the usual C-language notation (octal with the 0o prefix present only). These are equivalent:
4711
0x1267
0o11147
0b1001001100111As are these:
1.5
0x1.8p0
0x18p-4G.6. Comments
Longer pieces of diagnostic notation can really benefit from comments 💬. JSON famously doesn't have comments, and basic diagnostic notation per RFC 7049 inherits that. In EDN, comments are delimited by slashes ("/"). Any text within and including a pair of slashes counts as a comment.
Comments are whitespace, which means they're allowed in prefixed byte strings too. These are equivalent:
h'68656c6c6f20776f726c64'
h'68 65 6c /doubled l!/ 6c 6f /hello/
20 /space/
77 6f 72 6c 64' /world/You can use this to annotate a CBOR structure, like so:
/grasp-message/ [/M_DISCOVERY/ 1, /session-id/ 10584416,
/objective/ [/objective-name/ "opsonize",
/D, N, S/ 7, /loop-count/ 105]](No end-of-line comments exist yet 🥺. If we ever add them, "//" sounds like a reasonable delimiter since we already use slashes for comments. We could also go with "#", for example.)
Okay, that's the end of this batch, but the RFC is still going, so keep reading 💖
Appendix H. Examples 🎀
This appendix is for information only. ✨ No pressure, just vibes and examples bestie. 🫶
okay so this appendix has a few examples of structures defined using CDDL 💅 The theme is taken from [RFC7071], which defines certain JSON structures in English. ngl it's giving reputation drama 🍓 For a similar example, it may also be of interest to examine Appendix A of [RFC8007], which has a CDDL definition for a JSON structure defined in the main body of that RFC. 🧚
These examples all happen to describe data that is interchanged in JSON 💖 Examples for CDDL definitions of data that is interchanged in CBOR can be found in [RFC8152], [GRASP], and [RFC8428]. 🌈
[RFC7071] defines the "reputon" structure for JSON using somewhat formalized English text. 📝 Here is a (somewhat verbose) equivalent definition using the same terms, but notated in CDDL 👑:
reputation-object = {
reputation-context,
reputon-list
}
reputation-context = (
application: text
)
reputon-list = (
reputons: reputon-array
)
reputon-array = [* reputon]
reputon = {
rater-value,
assertion-value,
rated-value,
rating-value,
? conf-value,
? normal-value,
? sample-value,
? gen-value,
? expire-value,
* ext-value,
}
rater-value = ( rater: text )
assertion-value = ( assertion: text )
rated-value = ( rated: text )
rating-value = ( rating: float16 )
conf-value = ( confidence: float16 )
normal-value = ( normal-rating: float16 )
sample-value = ( sample-size: uint )
gen-value = ( generated: uint )
expire-value = ( expires: uint )
ext-value = ( text => any )An equivalent, more compact form of this example would be: 🦋
reputation-object = {
application: text
reputons: [* reputon]
}
reputon = {
rater: text
assertion: text
rated: text
rating: float16
? confidence: float16
? normal-rating: float16
? sample-size: uint
? generated: uint
? expires: uint
* text => any
}Note how this rather clearly delineates the structure somewhat shrouded by so many words in Section 6.2.2 of [RFC7071]. 🙈 Also, this definition makes it clear that several ext-values are allowed (by definition with different member names). RFC 7071 could be read to forbid the repetition of ext-value ("A specific reputon-element MUST NOT appear more than once" is ambiguous). 💀 So the CDDL version is the clearer one, periodt. 💅
The CDDL tool described in Appendix F generates as one example: 🍓
{
"application": "conchometry",
"reputons": [
{
"rater": "Ephthianura",
"assertion": "codding",
"rated": "sphaerolitic",
"rating": 0.34133473256800795,
"confidence": 0.9481983064298332,
"expires": 1568,
"unplaster": "grassy"
},
{
"rater": "nonchargeable",
"assertion": "raglan",
"rated": "alienage",
"rating": 0.5724646875815566,
"sample-size": 3514,
"Aldebaran": "unchurched",
"puruloid": "impersonable",
"uninfracted": "pericarpoidal",
"schorl": "Caro"
},
{
"rater": "precollectable",
"assertion": "Merat",
"rated": "thermonatrite",
"rating": 0.19164006323936977,
"confidence": 0.6065252103391268,
"normal-rating": 0.5187773690879303,
"generated": 899,
"speedy": "solidungular",
"noviceship": "medicine",
"checkrow": "epidictic"
}
]
}Acknowledgements 🙌
Inspiration was taken from the C and Pascal languages, MPEG's conventions for describing structures in the ISO base media file format, RELAX NG and its compact syntax [RELAXNG], and, in particular, Andrew Lee Newton's early proposals on JSON Content Rules (JCR) as found in draft version four (-04) of [JCR]. 🌸
Lots of highly useful feedback came from members of the IETF CBOR WG -- in particular, Ari Keranen, Brian Carpenter, Burt Harris, Jeffrey Yasskin, Jim Hague, Jim Schaad, Joe Hildebrand, Max Pritikin, Michael Richardson, Pete Cordell, Sean Leonard, and Yaron Sheffer. 🫶 Also, Francesca Palombini and Joe volunteered to chair the WG when it was created, providing the framework for generating and processing this feedback, with Barry Leiba having taken over from Joe since then. Chris Lonvick and Ines Robles provided additional reviews during IESG processing, and Alexey Melnikov steered the process as the responsible Area Director. ✨
The CDDL tool described in Appendix F was written by Carsten Bormann, building on previous work by Troy Heninger and Tom Lord. 👑
Contributors 💜
CDDL was originally conceived by Bert Greevenbosch, who also wrote the original five draft versions of this document. 🌈
Authors' Addresses 🦋
Henk Birkholz Fraunhofer SIT Rheinstrasse 75 Darmstadt 64295 Germany
Email: henk.birkholz@sit.fraunhofer.de
Christoph Vigano Universitaet Bremen
Email: christoph.vigano@uni-bremen.de
Carsten Bormann Universitaet Bremen TZI Bibliothekstr. 1 Bremen D-28359 Germany
Phone: +49-421-218-63921 Email: cabo@tzi.org
📦 where this page comes from
- ietf: rfc8610 2019-06
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms