RFC 9682

Updates to the Concise Data Definition Language (CDDL) Grammar

📅 published November 2024, from ietf

RFC 9682: Updates to the Concise Data Definition Language (CDDL) Grammar 💖

Series: Internet Engineering Task Force (IETF) 🌸 Request for Comments: 9682 ✨ Author: C. Bormann, Universität Bremen TZI 🧚 Updates: RFC 8610 🔥 Category: Standards Track 👑 Date: November 2024 🍓 ISSN: 2070-1721 🎀

Abstract

okay bestie, so the Concise Data Definition Language (CDDL) is the thing that lets you describe the shape of protocol messages and data formats that are stored as Concise Binary Object Representation (CBOR) or JSON 💅. It's defined in RFCs 8610 and 9165, and it's honestly so clean and unambiguous ✨.

This document is an update to RFC 8610 🦋. It fixes up the errata reports that came in and makes some other small fixes to the ABNF grammar that CDDL uses 🫶.

Status of This Memo

This is an Internet Standards Track document.

This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 7841.

Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at https://www.rfc-editor.org/info/rfc9682.

Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.

Table of Contents

  • 1. Introduction
    • 1.1. Conventions and Definitions
  • 2. Clarifications and Changes Based on Errata Reports
    • 2.1. Updates to String Literal Grammar
      • 2.1.1. Erratum ID 6527 (Text String Literals)
      • 2.1.2. Erratum ID 6278 (Consistent String Literals)
      • 2.1.3. Addressing Erratum ID 6526 and Erratum ID 6543
    • 2.2. Examples Demonstrating the Updated String Syntaxes
  • 3. Small Enabling Grammar Changes
    • 3.1. Empty Data Models
    • 3.2. Non-Literal Tag Numbers and Simple Values
  • 4. Security Considerations
  • 5. IANA Considerations
  • 6. References
    • 6.1. Normative References
    • 6.2. Informative References
  • Appendix A. Updated Collected ABNF for CDDL
  • Appendix B. Details about Covering Erratum ID 6543
    • B.1. Change Proposed by Erratum ID 6543
    • B.2. No Further Change Needed after Updating String Literal Grammar
  • Acknowledgments
  • Author's Address

1. Introduction ✨

so CDDL, as defined in [RFC8610] and [RFC9165], is basically the way to describe structures for protocol messages and data formats that are represented in CBOR or JSON 🌈. It's easy and unambiguous, which is iconic 💖.

This document updates [RFC8610] by fixing errata reports and making other small fixes to the ABNF grammar defined for CDDL 🔥. The body of this document explains and shows the motivation for the updates 🎀. The updated collected ABNF syntax in Figure 11 in Appendix A replaces the collected ABNF syntax in Appendix B of [RFC8610] 💅.

1.1. Conventions and Definitions

The terminology from [RFC8610] applies 🧚. The grammar in [RFC8610] is based on ABNF, which is defined in [STD68] and [RFC7405] 🦋.

2. Clarifications and Changes Based on Errata Reports 🔥

okay so a bunch of errata reports came in about the details of text string and byte string literal syntax, like [Err6527] and [Err6543] 😭. We're fixing those in this section by updating the ABNF for these literal syntaxes ✨. The changes described in [Err6526] also need to be applied, because backslashes got lost during the RFC publication process of Appendix G.2 of [RFC8610], which totally garbled the text explaining backslash escaping 💀.

These changes mirror the way existing implementations have already dealt with the errata reports 🫶. This document also takes the chance, while cleaning up the string literal grammar, to add a backward-compatible syntax for hexadecimal escapes 🌸. This last change is not automatically forward compatible. That means CDDL specifications that use this syntax won't necessarily work with existing implementations until those implementations are updated, which is recommended by this specification 🙈.

2.1. Updates to String Literal Grammar

2.1.1. Erratum ID 6527 (Text String Literals)

The ABNF used in [RFC8610] for the content of text string literals is kinda permissive 👀:

; ABNF from RFC 8610:
text = %x22 *SCHAR %x22
SCHAR = %x20-21 / %x23-5B / %x5D-7E / %x80-10FFFD / SESC
SESC = "\" (%x20-7E / %x80-10FFFD)

  Figure 1: Original ABNF from RFC 8610 for Strings with Permissive
           ABNF for SESC (Which Did Not Allow Hex Escapes)

This allows almost any non-C0 character to be escaped by a backslash, but it critically misses the \uXXXX and \uHHHH\uLLLL forms that JSON allows for specifying characters in hex. Those should apply here according to item 6 of Section 3.1 of [RFC8610] 😤. (Note that CDDL imports the unwieldy \uHHHH\uLLLL syntax from JSON, which represents Unicode code points beyond U+FFFF by making them look like UTF-16 surrogate pairs. CDDL text strings do not use UTF-16 or surrogates 🙃.)

Both problems can be solved by updating the SESC rule. This document uses the opportunity to add a popular form of directly specifying characters in strings using hexadecimal escape sequences of the form \u{hex}, where hex is the hexadecimal representation of the Unicode scalar value 🦄. The result is the new set of rules defining SESC in Figure 2 💖.

; new rules collectively defining SESC:
SESC = "\" ( %x22 / "/" / "\" /                 ; \" \/ \\
             %x62 / %x66 / %x6E / %x72 / %x74 / ; \b \f \n \r \t
             (%x75 hexchar) )                   ; \uXXXX
hexchar = "{" (1*"0" [ hexscalar ] / hexscalar) "}" /
          non-surrogate / (high-surrogate "\" %x75 low-surrogate)
non-surrogate = ((DIGIT / "A"/"B"/"C" / "E"/"F") 3HEXDIG) /
                ("D" %x30-37 2HEXDIG )
high-surrogate = "D" ("8"/"9"/"A"/"B") 2HEXDIG
low-surrogate = "D" ("C"/"D"/"E"/"F") 2HEXDIG
hexscalar = "10" 4HEXDIG / HEXDIG1 4HEXDIG
          / non-surrogate / 1*3HEXDIG
HEXDIG1 = DIGIT1 / "A" / "B" / "C" / "D" / "E" / "F"

  Figure 2: Update to String ABNF in Appendix B of [RFC8610]: Allow
                             Hex Escapes
Notes: In ABNF, strings such as "A", "B", etc., are case insensitive, as is intended here. The rules above could have also used %s"b", etc., instead of %x62, but didn't, in order to maximize compatibility with ABNF tools.

Now that SESC is more restrictively formulated, the BCHAR rule used in the ABNF syntax for byte string literals also needs an update 🧁:

; ABNF from RFC 8610:
bytes = [bsqual] %x27 *BCHAR %x27
BCHAR = %x20-26 / %x28-5B / %x5D-10FFFD / SESC / CRLF
bsqual = "h" / "b64"

               Figure 3: ABNF from RFC 8610 for BCHAR

With the SESC updated as above, \' is no longer allowed in BCHAR and now needs to be explicitly included there; see Figure 4 ✨.

2.1.2. Erratum ID 6278 (Consistent String Literals)

Updating BCHAR is also a chance to address [Err6278], which points out an inconsistency in how U+007F (DEL) gets treated between SCHAR and BCHAR 🔍. U+007F isn't printable, so including it in a byte string literal is just as confusing as including it in a text string literal 😵‍💫. That means it should be excluded from BCHAR, just like it is from SCHAR. The same reasoning applies to the C1 control characters, so the updated ABNF actually excludes the entire range from U+007F to U+009F 🌊. The same reasoning also applies to text in comments (PCHAR). For completeness, all of these rules should also explicitly exclude the code points that have been set aside for UTF-16 surrogates 🫧.

; new rules for SCHAR, BCHAR, and PCHAR:
SCHAR = %x20-21 / %x23-5B / %x5D-7E / NONASCII / SESC
BCHAR = %x20-26 / %x28-5B / %x5D-7E / NONASCII / SESC / "\'" / CRLF
PCHAR = %x20-7E / NONASCII
NONASCII = %xA0-D7FF / %xE000-10FFFD

     Figure 4: Update to ABNF in Appendix B of [RFC8610]: BCHAR,
                           SCHAR, and PCHAR

(Note that, apart from addressing these inconsistencies, there is no attempt to further exclude non-printable characters from the ABNF. Doing that properly would drag in complexity from the ongoing evolution of the Unicode standard [UNICODE], which isn't needed here 🤓.)

2.1.3. Addressing Erratum ID 6526 and Erratum ID 6543

The changes above also cover [Err6543] (a proposal to split qualified byte string literals off from UTF-8 byte string literals) and [Err6526] (the lost backslashes). See Appendix B for the details 🎀.

2.2. Examples Demonstrating the Updated String Syntaxes

The CDDL example in Figure 5 shows off the different escaping techniques that are now available for (byte and text) strings in CDDL 💅. Obviously, in the literals for a and x, there's no need to escape the second character, an o, as \u{6f}. That's just for demonstration 😂. Similarly, as shown in c and z, there's also no need to escape the "🁳" (DOMINO TILE VERTICAL-02-02, U+1F073) or the "⌘" (PLACE OF INTEREST SIGN, U+2318) 🌸. Escaping them can still be convenient, though, if you want to limit the character repertoire of a CDDL file itself to ASCII [STD80] 🧚.

start = [a, b, c, x, y, z]

; "🁳", DOMINO TILE VERTICAL-02-02, and
; "⌘", PLACE OF INTEREST SIGN, in a text string:
a = "D\u{6f}mino's \u{1F073} + \u{2318}"      ; \u{}-escape 3 chars
b = "Domino's \uD83C\uDC73 + \u2318"          ; escape JSON-like
c = "Domino's 🁳 + ⌘"                          ; unescaped

; in a byte string given as text, the ' needs to be escaped:
x = 'D\u{6f}mino\u{27}s \u{1F073} + \u{2318}' ; \u{}-escape 4 chars
y = 'Domino\'s \uD83C\uDC73 + \u2318'         ; escape JSON-like
z = 'Domino\'s 🁳 + ⌘'                         ; escape ' only

   Figure 5: Example Text and Byte String Literals with Various Escaping
                                 Techniques

In this example, the rules a to c and x to z all produce strings with byte-wise identical content. a to c are text strings and x to z are byte strings 🥹. Figure 6 shows this by displaying the output generated from the start rule in Figure 5, using pretty-printed hexadecimal 🌈.

86                                      # array(6)
   73                                   # text(19)
      446f6d696e6f277320f09f81b3202b20e28c98 # "Domino's 🁳 + ⌘"
   73                                   # text(19)
      446f6d696e6f277320f09f81b3202b20e28c98 # "Domino's 🁳 + ⌘"
   73                                   # text(19)
      446f6d696e6f277320f09f81b3202b20e28c98 # "Domino's 🁳 + ⌘"
   53                                   # bytes(19)
      446f6d696e6f277320f09f81b3202b20e28c98 # "Domino's 🁳 + ⌘"
   53                                   # bytes(19)
      446f6d696e6f277320f09f81b3202b20e28c98 # "Domino's 🁳 + ⌘"
   53                                   # bytes(19)
      446f6d696e6f277320f09f81b3202b20e28c98 # "Domino's 🁳 + ⌘"

      Figure 6: Generated CBOR from CDDL Example (Pretty-Printed
                             Hexadecimal)

3. Small Enabling Grammar Changes ✨

okay so this section is all about tiny grammar tweaks 💅 each subsection is a small change that makes certain kinds of specs possible. these changes are backward compatible, meaning CDDL files that follow [RFC8610] still match the updated grammar 🫶 but they're not necessarily forward compatible, so CDDL specs that use these changes might not be processable by existing [RFC8610] implementations 💀

3.1. Empty Data Models

bestie, [RFC8610] says a CDDL file needs at least one rule 🙃

; ABNF from RFC 8610:
cddl = S 1*(rule S)

Figure 7: ABNF from RFC 8610 for Top-Level Rule cddl

and honestly that makes sense when the file has to stand alone 🎯 a CDDL data model needs at least one rule so it has an entry point (aka a start rule).

but with CDDL modules [CDDL-MODULES], CDDL files can also include directives, and those directives might be where all the rules in the module actually come from ✨ any other rule content in the file then has to be available for directive processing, so forcing at least one rule gets clunky 😩

so this update extends the grammar like in Figure 8 and turns "at least one rule" into a semantic constraint 🌸 it only has to be fulfilled after all directives are processed.

; new top-level rule:
cddl = S *(rule S)

Figure 8: Update to Top-Level ABNF in Appendices B and C of RFC 8610

3.2. Non-Literal Tag Numbers and Simple Values

so here's the existing tag syntax in CDDL 🦋

; extracted from the ABNF in RFC 8610:
type2 =/ "#" "6" ["." uint] "(" S type S ")"

Figure 9: Original ABNF from RFC 8610 for Tag Syntax

which means tag numbers can only be written as literal numbers (uints) 🙈 but some specs work with whole ranges of tag numbers. for example, [RFC9277] has a range of tag numbers 1668546817 (0x63740101) to 1668612095 (0x6374FFFF) to tag specific content formats. right now that just can't be expressed in CDDL 😭 the same goes for simple values (#7.xx).

so this update extends the syntax to this:

; new rules collectively defining the tagged case:
type2 =/ "#" "6" ["." head-number] "(" S type S ")"
      / "#" "7" ["." head-number]
head-number = uint / ("<" type ">")

Figure 10: Update to Tag and Simple Value ABNF in Appendices B and C of RFC 8610

for #6, the head-number is the tag number 🏷️ for #7, the head-number is the simple value if it's in the ranges 0..23 or 32..255 (per Section 3.3 of RFC 8949 [STD94], simple values 24..31 aren't used). for 24..31, the head-number is the "additional information", so e.g. #7.25 or #7.<25> is a float16, etc. (all ranges here are inclusive ✨)

so that range can be written in a CDDL fragment like this:

ct-tag<content> = #6.<ct-tag-number>(content)
ct-tag-number = 1668546817..1668612095
; or use 0x63740101..0x6374FFFF

Note 1 🌿: this syntax reuses the angle bracket syntax for generics. it's innocuous because a generic parameter or argument only ever shows up after a rule name (id), while here it shows up after the "." (dot) character. (whether that could confuse humans can be debated, and the example above deliberately uses generics too 🙃)

Note 2 🌿: the updated ABNF makes it a bit more explicit that the number after the optional dot is the value of the argument. for tags and simple values, it's not the CBOR "additional information" like it is with other uses of # in CDDL. (adding this to Section 2.2.3 of [RFC8610] is the subject of [Err6575]; it's correctly noted in Section 3.6 of [RFC8610].) in hindsight, maybe a different character than the dot should've been picked for this special case, but changing the grammar in this document would've been too disruptive 😭

4. Security Considerations 🔐

the grammar fixes and updates in this document aren't believed to create additional security considerations ✨ the security considerations in Section 5 of [RFC8610] still apply. the big thing is that the potential for confusion goes up in an environment that mixes CDDL tools, some updated and some not, especially based on Section 2 👀

attackers may want to exploit that confusion by crafting CDDL models that get interpreted differently by different parts of a system 😈 there's going to be a transition period from the details that [RFC8610] handled in a less well-defined way to the updated grammar defined here. that transition might offer one (but not the only) type of opportunity for attacks that rely on differences between implementations 🚨 implementations that use CDDL models operationally already need to figure out the provenance (and so the authenticity and integrity) and applicability of the models they use. at the time of writing, models are expected to generally be processed by a software developer inside a software development environment. so developers are advised to treat CDDL models with the same care as any other source code 🫡

5. IANA Considerations 🌈

this document has no IANA actions ✨ nothing to see here for the registry gremlins 🧚

6. References 📚

6.1. Normative References

6.2. Informative References

Appendix A. Updated Collected ABNF for CDDL 🎀

This appendix is normative 💅, so it's official and every line counts. It has the full ABNF from [RFC8610] with all of the updates from the present document already baked in ✨

cddl = S *(rule S)
rule = typename [genericparm] S assignt S type
     / groupname [genericparm] S assigng S grpent

typename = id
groupname = id

assignt = "=" / "/="
assigng = "=" / "//="

genericparm = "<" S id S *("," S id S ) ">"
genericarg = "<" S type1 S *("," S type1 S ) ">"

type = type1 *(S "/" S type1)

type1 = type2 [S (rangeop / ctlop) S type2]
; space may be needed before the operator if type2 ends in a name

type2 = value
      / typename [genericarg]
      / "(" S type S ")"
      / "{" S group S "}"
      / "[" S group S "]"
      / "~" S typename [genericarg]
      / "&" S "(" S group S ")"
      / "&" S groupname [genericarg]
      / "#" "6" ["." head-number] "(" S type S ")"
      / "#" "7" ["." head-number]
      / "#" DIGIT ["." uint]                ; major/ai
      / "#"                                 ; any
head-number = uint / ("<" type ">")

rangeop = "..." / ".."

ctlop = "." id

group = grpchoice *(S "//" S grpchoice)

grpchoice = *(grpent optcom)

grpent = [occur S] [memberkey S] type
       / [occur S] groupname [genericarg]  ; preempted by above
       / [occur S] "(" S group S ")"

memberkey = type1 S ["^" S] "=>"
          / bareword S ":"
          / value S ":"

bareword = id

optcom = S ["," S]

occur = [uint] "*" [uint]
      / "+"
      / "?"

uint = DIGIT1 *DIGIT
     / "0x" 1*HEXDIG
     / "0b" 1*BINDIG
     / "0"

value = number
      / text
      / bytes

int = ["-"] uint

; This is a float if it has fraction or exponent; int otherwise
number = hexfloat / (int ["." fraction] ["e" exponent ])
hexfloat = ["-"] "0x" 1*HEXDIG ["." 1*HEXDIG] "p" exponent
fraction = 1*DIGIT
exponent = ["+"/"-"] 1*DIGIT

text = %x22 *SCHAR %x22
SCHAR = %x20-21 / %x23-5B / %x5D-7E / NONASCII / SESC

SESC = "\" ( %x22 / "/" / "\" /                 ; \" \/ \\
             %x62 / %x66 / %x6E / %x72 / %x74 / ; \b \f \n \r \t
             (%x75 hexchar) )                   ; \uXXXX

hexchar = "{" (1*"0" [ hexscalar ] / hexscalar) "}" /
          non-surrogate / (high-surrogate "\" %x75 low-surrogate)
non-surrogate = ((DIGIT / "A"/"B"/"C" / "E"/"F") 3HEXDIG) /
                ("D" %x30-37 2HEXDIG )
high-surrogate = "D" ("8"/"9"/"A"/"B") 2HEXDIG
low-surrogate = "D" ("C"/"D"/"E"/"F") 2HEXDIG
hexscalar = "10" 4HEXDIG / HEXDIG1 4HEXDIG
          / non-surrogate / 1*3HEXDIG

bytes = [bsqual] %x27 *BCHAR %x27
BCHAR = %x20-26 / %x28-5B / %x5D-7E / NONASCII / SESC / "\'" / CRLF
bsqual = "h" / "b64"

id = EALPHA *(*("-" / ".") (EALPHA / DIGIT))
ALPHA = %x41-5A / %x61-7A
EALPHA = ALPHA / "@" / "_" / "$"
DIGIT = %x30-39
DIGIT1 = %x31-39
HEXDIG = DIGIT / "A" / "B" / "C" / "D" / "E" / "F"
HEXDIG1 = DIGIT1 / "A" / "B" / "C" / "D" / "E" / "F"
BINDIG = %x30-31

S = *WS
WS = SP / NL
SP = %x20
NL = COMMENT / CRLF
COMMENT = ";" *PCHAR CRLF
PCHAR = %x20-7E / NONASCII
NONASCII = %xA0-D7FF / %xE000-10FFFD
CRLF = %x0A / %x0D.0A

Figure 11: ABNF for CDDL as Updated

Appendix B. Details about Covering Erratum ID 6543 🦋

This appendix is informative, so think of it as the "let me explain the drama" section 🧚

[Err6543] points out that the ABNF in [RFC8610] for the content of byte string literals kind of lumps together byte strings written as text with byte strings written in base16 (hex) or base64. (But see also the updated BCHAR rule in Figure 4 ✨)

; ABNF from RFC 8610:
bytes = [bsqual] %x27 *BCHAR %x27
BCHAR = %x20-26 / %x28-5B / %x5D-10FFFD / SESC / CRLF

Figure 12: Original ABNF from RFC 8610 for BCHAR

B.1. Change Proposed by Erratum ID 6543 💌

Erratum ID 6543 suggests handling the two cases in separate ABNF rules. (With an updated SESC, BCHAR obviously needs updating as well, as shown above.)

; Proposal from Erratum ID 6543:
bytes = %x27 *BCHAR %x27
      / bsqual %x27 *QCHAR %x27
BCHAR = %x20-26 / %x28-5B / %x5D-10FFFD / SESC / CRLF
QCHAR = DIGIT / ALPHA / "+" / "/" / "-" / "_" / "=" / WS

Figure 13: Proposal from Erratum ID 6543 to Split the Byte String Rules

This could cause a subtle change, and it's hiding in the WS rule 👀

; ABNF from RFC 8610:
WS = SP / NL
SP = %x20
NL = COMMENT / CRLF
COMMENT = ";" *PCHAR CRLF
PCHAR = %x20-7E / %x80-10FFFD
CRLF = %x0A / %x0D.0A

Figure 14: ABNF Definition of WS from RFC 8610

Here's the tea: this allows any non-C0 character in a comment, so this fragment becomes possible 🙈

foo = h'
   43424F52 ; 'CBOR'
   0A       ; LF, but don't use CR!
'

The current text is not unambiguously saying whether the three apostrophes need to be escaped with a \ or not, as in:

foo = h'
   43424F52 ; \'CBOR\'
   0A       ; LF, but don\'t use CR!
'

...which would be supported by the existing ABNF in [RFC8610].

B.2. No Further Change Needed after Updating String Literal Grammar ✨

This document takes the simpler route 😌 It leaves the processing of the content of the byte string literal to a semantic step that comes after processing the syntax of the bytes and BCHAR rules, as updated by Figures 2 and 4 in Section 2.1 (updates prompted by the combination of [Err6527] and [Err6278]).

So the rules in Figure 14 (as updated by Figure 4) get applied to the result of this processing where bsqual is given as h or b64.

Note that this approach also works well with the use of byte strings in Section 3 of [RFC9165]. It does need some care when copying and pasting into CDDL models from ABNF that contains single quotes (which may also hide as apostrophes in comments). These need to be escaped or possibly replaced by %x27.

Finally, the approach taken lends support to extending bsqual in CDDL, similar to the way this is done for CBOR diagnostic notation in [EDN-LITERALS]. (Note that, at the time of writing, the processing of string literals is quite similar for both CDDL and Extended Diagnostic Notation (EDN). The difference is that CDDL has end-of-line comments that are ";" based, and EDN has two comment syntaxes: one in-line "/" based and one end-of-line "#" based.)

Acknowledgments 🫶

Many thanks go to the submitters of the errata reports addressed in this document 💖 In one of the ensuing discussions, Doug Ewell proposed defining an ABNF rule NONASCII, and we've included the essence of it. Special thanks to the reviewers Marco Tiloca, Christian Amsüss (Shepherd Review and further guidance), Orie Steele (AD Review and further guidance), and Éric Vyncke (detailed IESG review) 👑

Author's Address 💌

Carsten Bormann Universität Bremen TZI Postfach 330440 D-28359 Bremen Germany Phone: +49-421-218-63921 Email: cabo@tzi.org

📦 where this page comes from