RFC 9165

Additional Control Operators for the Concise Data Definition Language (CDDL)

📅 published December 2021, from ietf

📜 RFC 9165: Additional Control Operators for the Concise Data Definition Language (CDDL) 🎀

  • Number: RFC 9165 ✨
  • Authors: C. Bormann, Universität Bremen TZI 💅
  • Category: Standards Track 🦋
  • Date: December 2021 🌸
  • ISSN: 2070-1721
  • Obsoletes / Updates: The title block doesn't list anything here, so this RFC doesn't say it obsoletes or updates any other document 🧚

Abstract ✨

okay so bestie, CDDL (standardized in RFC 8610) gets its main extension power from "control operators" 💖. This doc adds a bunch that weren't ready when 8610 was finished: .plus, .cat, and .det for building constants 🔥; .abnf/.abnfb for putting ABNF (RFC 5234 and RFC 7405) inside CDDL specs 🌈; and .feature for flagging that an instance uses a non-basic feature 🫶.

Status of This Memo 📝

This is an Internet Standards Track document 💅.

It's a product of the Internet Engineering Task Force (IETF) and represents the consensus of the IETF community 👑. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). For more on Internet Standards, check Section 2 of RFC 7841 💜.

For the current status, any errata, and how to send feedback, go to https://www.rfc-editor.org/info/rfc9165 🎀.

Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved 🌸.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as in effect on the date of publication 🦋. Please read those carefully, because they spell out your rights and restrictions for this document 🙈. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as described in the Revised BSD License 💅.

Table of Contents 🧚

1.  Introduction
  1.1.  Terminology
2.  Computed Literals
  2.1.  Numeric Addition
  2.2.  String Concatenation
  2.3.  String Concatenation with Dedenting
3.  Embedded ABNF
4.  Features
5.  IANA Considerations
6.  Security Considerations
7.  References
  7.1.  Normative References
  7.2.  Informative References
Acknowledgements
Author's Address

1. Introduction 💖

So CDDL, standardized in [RFC8610], gives you "control operators" as its main language extension point (Section 3.8 of [RFC8610]) 🌈.

This doc defines a bunch of control operators that weren't ready yet when [RFC8610] got finished. Here's the lineup 👑:

   +==========+==================================================+
   | Name     | Purpose                                          |
   +==========+==================================================+
   | .plus    | Numeric addition                                 |
   +----------+--------------------------------------------------+
   | .cat     | String concatenation                             |
   +----------+--------------------------------------------------+
   | .det     | String concatenation, pre-dedenting              |
   +----------+--------------------------------------------------+
   | .abnf    | ABNF in CDDL (text strings)                      |
   +----------+--------------------------------------------------+
   | .abnfb   | ABNF in CDDL (byte strings)                      |
   +----------+--------------------------------------------------+
   | .feature | Indicates name of feature used (extension point) |
   +----------+--------------------------------------------------+

              Table 1: New Control Operators in this Document

1.1. Terminology 📚

Here's the key-word rule 🔑. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here 💅.

This spec uses terminology from [RFC8610] 🦋. For control operators specifically, "target" means the left-hand side operand and "controller" means the right-hand side operand 🌸. "Tool" refers to tools along the lines of the one described in Appendix F of [RFC8610] ✨. Also, the data model underlying CDDL has text strings and byte strings as two separate types, and they're collectively called "strings" 🧚.

The term "ABNF" in this spec stands for the combination of [RFC5234] and [RFC7405]. In other words, the ABNF control operators defined here let you use the case-sensitive extensions from [RFC7405] 🔥.

2. Computed Literals 🧮

CDDL as defined in [RFC8610] has no way to compute literals 🙈. To cover a big chunk of use cases, this spec adds three control operators: .plus for numeric addition, .cat for string concatenation, and .det for string concatenation with dedenting of both sides (target and controller) 🌈.

For these operators, like all control operators, targets and controllers are types 💖. So the result is formally a function of the elements of the cross-product of the two types. Not all tools may be able to handle non-unique targets or controllers 🫶.

2.1. Numeric Addition 🔢

In lots of cases, you need numbers relative to a base number in a specification 🌸. The .plus control identifies a number built by adding the numeric values of the target and the controller ✨.

The target and controller both MUST be numeric. If the target is a floating point number and the controller is an integer (or the other way around), the sum gets converted into the type of the target. Converting from a floating point number to an integer selects its floor, which is the largest integer less than or equal to the floating point number, i.e., rounding towards negative infinity 👑.

interval<BASE> = (
  BASE => int             ; lower bound
  (BASE .plus 1) => int   ; upper bound
  ? (BASE .plus 2) => int ; tolerance
)

X = 0
Y = 3
rect = {
  interval<X>
  interval<Y>
}

Figure 1: An Example of Addition to a Base Value

Figure 1 has the generic definition of a CDDL group called interval, which gives a lower bound, an upper bound, and optionally a tolerance 🦋. The parameter BASE lets you use several of these interval groups in one map without conflicts, by giving each entry of each interval its own labels 🧚. The rule rect combines two of these interval groups into one map: one group for the X dimension (labels 0, 1, and 2) and one for the Y dimension (labels 3, 4, and 5) 💜.

2.2. String Concatenation 🎀

It's super useful to build string literals out of component literals that are defined in different places in the spec 💅.

The .cat control identifies a string built by concatenating the target and the controller 🌈. The target and controller both MUST be strings. The result has the same type as the target. The concatenation runs on the bytes of both strings. If the target is a text string, the result of that concatenation MUST be valid UTF-8 🫶.

c = "foo" .cat '
  bar
  baz
'
; on a system where the newline is \n, is the same string as:
b = "foo\n  bar\n  baz\n"

Figure 2: An Example of Concatenation of Text and Byte Strings

Figure 2 builds a text string named c by concatenating the target text string "foo" with the controller byte string written as a text form byte string literal 🍓. (This idiom is handy when the text string has newlines, which, as the example for b shows, are harder to read in the format that pure CDDL text strings inherit from JSON 🙈.)

2.3. String Concatenation with Dedenting 🧚

Multi-line string literals for all sorts of uses, including embedded ABNF (Section 3), need to sit flush left, at least partially 🌸. But having some indentation in your source code for the literal can make it way more readable, like in Figure 3 ✨.

oid = bytes .abnfb ("oid" .det cbor-tags-oid)
roid = bytes .abnfb ("roid" .det cbor-tags-oid)

cbor-tags-oid = '
  oid = 1*arc
  roid = *arc
  arc = [nlsb] %x00-7f
  nlsb = %x81-ff *%x80-ff
'

Figure 3: An Example of Dedenting Concatenation

The control operator .det works like .cat, except that both arguments (target and controller) get independently dedented before the concatenation happens 💖.

For the first rule in Figure 3, the result is equivalent to Figure 4 👑.

oid = bytes .abnfb 'oid
oid = 1*arc
roid = *arc
arc = [nlsb] %x00-7f
nlsb = %x81-ff *%x80-ff
'

Figure 4: Dedenting Example: Result of First .det

For this spec, "dedenting" means the following 🦋:

  1. Find the smallest amount of leftmost blank space (the number of leading space characters) present in all the non-blank lines, and
  2. remove exactly that number of leading space characters from each line. For blank lines (blank space only, or empty), there may be fewer leading space characters than that amount, or none at all, in which case all the leading space is removed 🌈.
The name .det is a shortcut for "dedenting cat" 💅. The maybe more obvious name .dedcat wasn't picked, because it's longer and might invoke unpleasant images 🙈.

Sometimes you only need to dedent one item. You can do that by using the operator with an empty string, like "" .det rhs or lhs .det "". You can also combine that with a .cat: in the construct lhs .cat ("" .det rhs), only rhs gets dedented 🫶.

Part 2 picks up with Section 3, Embedded ABNF 🧚🦋

  1. Embedded ABNF ✨

Okay so bestie, ABNF is a super popular way for IETF protocols to define what their text strings are allowed to look like [RFC5234] [RFC7405] 💅. Like, it's kinda essential, and it's giving "why would I rewrite that as a regex" energy 🙈. If you wanted to define a text string type in CDDL using existing ABNF, you'd usually have to translate it into a regular expression first, and sometimes that's just not even possible 😭. So this section is the glow-up: ABNF gets added to CDDL the same way regular expressions did, by defining a .abnf control operator 🦋. The target is usually text (or some restriction on it), and the controller is the text of an ABNF specification 🫶.

There are a few little snags, and here's how we fix them 🎀

  • ABNF can work on byte sequences AND on UTF-8 text strings that get read as Unicode scalar sequences 🌈. So this spec defines two control operators: .abnfb for ABNF that's about byte sequences, and .abnf for sequences of Unicode scalar values (code points) written as UTF-8 text strings ✨. Both operators can be applied to either string type. The ABNF gets applied to the bytes of the string and reads them as bytes (.abnfb) or as code points in a UTF-8 text string (.abnf) 💖. The controller string MUST be a string. When it's a byte string, it MUST be valid UTF-8, and it's read as the text string with the exact same bytes 🍓.
  • ABNF is a list of rules, not one single expression (those are called "elements" in [RFC5234]) 👑. The fix is to require the controller string to be one valid "element", then zero or more valid "rules" separated from that element by a newline 🧚. So you can build the controller by putting a piece of valid ABNF after an "element" that picks from that ABNF, plus a newline ✨.
  • ABNF needs newlines, and writing newlines inside CDDL text strings is tedious and makes the ABNF basically unreadable 💀. The workaround uses the .cat operator from Section 2.2 and the syntax for text in byte strings 🌸. Like always with ABNF, the syntax of ABNF itself (_not_ the syntax expressed in ABNF!) is relaxed so a single line feed counts as a newline 🙈:
CRLF = %x0A / %x0D.0A
  • Lots of ABNF rules get reused in different spots, especially staples like DIGIT and ALPHA 🩷. (Heads up: every rule referenced has to be defined in each ABNF operator controller string. There is NO implicit import of core ABNF rules from [RFC5234] or other rules 😤.) You can compose this with .cat, and/or with .det if there's indentation to clean up 🧹.

These points all come together in Figure 5, which uses ABNF from [RFC3339] to define one of each of the Concise Binary Object Representation (CBOR) tags from [RFC8943] and [RFC8949] 🦋.

; for RFC 8943
Tag1004 = #6.1004(text .abnf full-date)
; for RFC 8949
Tag0 = #6.0(text .abnf date-time)

full-date = "full-date" .cat rfc3339
date-time = "date-time" .cat rfc3339

; Note the trick of idiomatically starting with a newline, separating
;   off the element in the concatenations above from the rule-list
rfc3339 = '
   date-fullyear   = 4DIGIT
   date-month      = 2DIGIT  ; 01-12
   date-mday       = 2DIGIT  ; 01-28, 01-29, 01-30, 01-31 based on
                             ; month/year
   time-hour       = 2DIGIT  ; 00-23
   time-minute     = 2DIGIT  ; 00-59
   time-second     = 2DIGIT  ; 00-58, 00-59, 00-60 based on leap sec
                             ; rules
   time-secfrac    = "." 1*DIGIT
   time-numoffset  = ("+" / "-") time-hour ":" time-minute
   time-offset     = "Z" / time-numoffset

   partial-time    = time-hour ":" time-minute ":" time-second
                     [time-secfrac]
   full-date       = date-fullyear "-" date-month "-" date-mday
   full-time       = partial-time time-offset

   date-time       = full-date "T" full-time
' .det rfc5234-core

rfc5234-core = '
   DIGIT          =  %x30-39 ; 0-9
   ; abbreviated here
'

Figure 5: An Example of Employing ABNF from RFC 3339 for Defining CBOR Tags 🎀

  1. Features ✨

Commonly, the kind of validation that languages like CDDL give you is just a Boolean result: valid or invalid 🥱. In fast-changing environments, that's way too basic 😤. The data models a CDDL spec describes may keep getting enhanced with extra features, and even a spec that doesn't describe a specific future feature yet can be useful if it names the extension point the feature can use, accepts such extensions, and marks them as extensions 💜.

The .feature control annotates the target as using the feature named by the controller 🦋. The controller is usually a string 🫶. A tool that validates an instance against that spec can mark the instance as using a feature the spec annotated 🌈.

To be more specific, the tool's diagnostic output might show the controller (right-hand side) as a feature name and the target (left-hand side) as a feature detail 💅. But sometimes the target has way too much detail, and the spec might want to hint that the tool should use something more limited 🙈. In that case, the controller should be an array: the first element is the feature name (the thing that would otherwise be the whole controller), and the second element is the detail (usually another string) 🎀. Check out Figure 6 👀.

foo = {
  kind: bar / baz .feature (["foo-extensions", "bazify"])
}
bar = ...
baz = ... ; complex stuff that doesn't all need to be in the detail

Figure 6: Providing Explicit Detail with .feature

Figure 7 shows what a definition of a person could look like, with potential extensions beyond name and organization marked as further-person-extension 🧚. Extensions that are known when this definition is written can be collected into $$person-extensions 💖. But future extensions would be deemed invalid unless the wildcard at the end of the map is added 😭. Those extensions could then be specifically examined by a user or by a tool that uses the validation result, and the label (map key) actually used makes a fine feature detail for the tool's diagnostic output 🌸.

Leaving out the entire extension point would mean instances that use an extension get marked as wholesale invalid, which makes the whole validation approach way less useful 💀. Leaving the extension point in but not marking its use as special would make mistakes (like using the label "organisation" instead of "organization") invisible 🙈. Yikes.

person = {
  ? name: text
  ? organization: text
  $$person-extensions
  * (text .feature "further-person-extension") => any
}

$$person-extensions //= (? bloodgroup: text)

Figure 7: Map Extensibility with .feature

Figure 8 shows another example where .feature handles type extensibility ✨.

allowed-types = number / text / bool / null
              / [* number] / [* text] / [* bool]
              / (any .feature "allowed-type-extension")

Figure 8: Type Extensibility with .feature

A CDDL tool can just report the set of features being used, and the control only gives info to the process that asked for the validation 🦋. You could also imagine a tool that takes arguments, so it can accept certain features and reject others (enable/disable) 💖. That latter approach could, for instance, be used for a JSON/CBOR switch, as Figure 9 shows, using Sensor Measurement Lists (SenML) [RFC8428] as the example data model used with both JSON and CBOR 🧜‍♀️.

SenML-Record = {
; ...
  ? v => number
; ...
}
v = JC<"v", 2>
JC<J,C> = J .feature "json" / C .feature "cbor"

Figure 9: Describing Variants with .feature

It's still unclear whether the enable/disable approach can lead to new CDDL idioms 🤔. The language currently has no way to enforce mutually exclusive use of features, which this example would need 🥺.

  1. IANA Considerations 📋

IANA has registered the contents of Table 2 into the "CDDL Control Operators" registry of [IANA.cddl] 🌷:

                      +==========+===========+
                      | Name     | Reference |
                      +==========+===========+
                      | .plus    | RFC 9165  |
                      +----------+-----------+
                      | .cat     | RFC 9165  |
                      +----------+-----------+
                      | .det     | RFC 9165  |
                      +----------+-----------+
                      | .abnf    | RFC 9165  |
                      +----------+-----------+
                      | .abnfb   | RFC 9165  |
                      +----------+-----------+
                      | .feature | RFC 9165  |
                      +----------+-----------+

Table 2: New Control Operators

  1. Security Considerations 🚨

The security considerations of [RFC8610] apply 🔐.

Both [RFC5234] and [RFC7405] say security is truly believed to be irrelevant to their documents, but using formal description techniques can not only simplify a specification, it can also sometimes complicate it 😱. This can lead to security problems in implementations and in the specification itself 💀. Just like CDDL itself, ABNF should be applied judiciously, and overly complex (or "cute") constructions should be avoided 🙅‍♀️. (Yes, I'm looking at you, cute constructions 😤.)

  1. References 📚

7.1. Normative References 🌟

[IANA.cddl] IANA, "Concise Data Definition Language (CDDL)", <https://www.iana.org/assignments/cddl>.

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/info/rfc2119>.

[RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, DOI 10.17487/RFC5234, January 2008, <https://www.rfc-editor.org/info/rfc5234>.

[RFC7405] Kyzivat, P., "Case-Sensitive String Support in ABNF", RFC 7405, DOI 10.17487/RFC7405, December 2014, <https://www.rfc-editor.org/info/rfc7405>.

[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <https://www.rfc-editor.org/info/rfc8174>.

[RFC8610] Birkholz, H., Vigano, C., and C. Bormann, "Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data Structures", RFC 8610, DOI 10.17487/RFC8610, June 2019, <https://www.rfc-editor.org/info/rfc8610>.

7.2. Informative References ✨

okay so these are the informative refs, aka the bonus reading list that's helpful but not required to get the spec 💅

Acknowledgements 🫶

Jim Schaad suggested several improvements, bestie, so big thanks to him 🙌 The .feature feature was developed out of a discussion with Henk Birkholz, and Paul Kyzivat helped isolate the need for .det 🦋

.det is an abbreviation for "dedenting cat" 🐱 but Det is also the name of a German TV cartoon character created in the 1960s, so the name has a whole secret life 📺✨

Author's Address 🎀

Carsten Bormann Universität Bremen TZI Postfach 330440 D-28359 Bremen Germany

Phone: +49-421-218-63921 Email: cabo@tzi.org 💌

📦 where this page comes from