RFC 9277

On Stable Storage for Items in Concise Binary Object Representation (CBOR)

📅 published August 2022, from ietf

RFC 9277 🎀 On Stable Storage for Items in Concise Binary Object Representation (CBOR)

Okay bestie, the title block is giving everything 💅. This is RFC 9277, called On Stable Storage for Items in Concise Binary Object Representation (CBOR) 💖. It's a Standards Track doc (ISSN 2070-1721) dated August 2022. M. Richardson from Sandelman Software Works and C. Bormann from Universität Bremen TZI wrote it together 🫶. It doesn't list anything it obsoletes or updates, so no drama there ✨.

Abstract 🌸

Okay so the abstract is short and sweet 🍓. It defines a stored ("file") format for CBOR data items that plays nicely with common systems that recognize file types, like the Unix file(1) command 🔍.

Status of This Memo 📜

Okay so this is officially an Internet Standards Track document, no cap 👑.

This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). If you want more info on Internet Standards, check out Section 2 of RFC 7841 🧚.

For the current status of this document, any errata, and how to send feedback, go to https://www.rfc-editor.org/info/rfc9277 🔗.

Copyright (c) 2022 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as in effect on the date this document was published. Please actually read these documents carefully, because they explain your rights and restrictions for this document 🙈. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions. They're also provided without warranty, as described in the Revised BSD License.

Table of Contents 🗺️

1.  Introduction
  1.1.  Terminology
  1.2.  Requirements for a Magic Number
2.  Protocol
  2.1.  The CBOR-Protocol-Specific Tag
  2.2.  Enveloping Method: CBOR Tag Wrapped
    2.2.1.  Example
  2.3.  Enveloping Method: Labeled CBOR Sequence
    2.3.1.  Example
3.  Security Considerations
4.  IANA Considerations
  4.1.  Labeled CBOR Sequence Tag
  4.2.  CBOR-Labeled Non-CBOR Data Tag
  4.3.  CBOR Tags for CoAP Content-Format Numbers
5.  References
  5.1.  Normative References
  5.2.  Informative References
Appendix A.  Advice to Protocol Designer
  A.1.  Is the on-wire format new?
  A.2.  Can many items be trivially concatenated?
  A.3.  Are there tags at the start?
Appendix B.  CBOR Tags for CoAP Content Formats
  B.1.  Content-Format Tag Examples
Appendix C.  Example from Openswan
Appendix D.  Using CBOR Labels for Non-CBOR Data
  D.1.  Content-Format Tag Examples
Acknowledgements
Contributors
Authors' Addresses

1. Introduction ✨

Okay so since basically the beginning of computing, operating systems have wanted ways to mark which programs can open which files 🖥️. In Unix, everything is just a stream of bytes, so figuring out what's inside a stream became a bit of a guessing game 🎲.

Take the Unix file(1) command. It's been around since 1973 [FILE], and for decades it's been able to spot lots of file formats just by looking at the contents 🕵️‍♀️.

Lots of systems (Linux, macOS, Windows) pick the right app based on the file contents when they can't figure it out another way 💅. Back in classical Mac OS, for example, a resource fork was kept separately from the file data and held the file type info. That meant the OS ideally never had to look inside the file data to know the media type ✨.

A lot of other systems use file extensions instead. Plenty of common Web servers get their media-type info from file extensions 🌈.

Having a media type tied to the file contents can dodge some of the brittleness of that approach 🫶. When files get separated from their type info, like when you're doing forensics on a damaged system, being able to tell what kind of info a file holds can become super important 😭.

A popular way to identify a file's type from its contents is to put a "magic number" at the start of the file [MAGIC]. The media type registration template [RFC6838] asks for a magic number, if one is available, and also asks for a file extension 📝.

The file(1) command often struggles because it can recognize the overall encoding without recognizing the content being encoded 🙃. For example, an Android Package Kit APK (the thing used to transfer and store an app) might get identified as a ZIP file. Also, OpenOffice and MSOffice files are both ZIP files full of XML files, so the identification might stop at "ZIP" 💀.

CBOR is showing up in more and more kinds of artifacts, so calling everything just "CBOR" probably isn't enough anymore 🍓. This document gives a way to put a magic number at the beginning of a CBOR format file. A CBOR format might use a single CBOR data item or a CBOR sequence of data items [RFC8742], so two possible enveloping methods are presented. A CBOR Protocol designer picks one of them. (A CBOR Protocol is a specification that uses CBOR as its encoding 🧠.)

This document also gives advice to CBOR Protocol designers on choosing one of these mechanisms for identifying their contents. That advice is informative 🦋.

A third method is also proposed, where a CBOR format tag gets prepended to identify non-CBOR files. Appendix D covers that one in more detail, because it isn't about identifying media types that contain CBOR-encoded data items 🎀. It includes a simple way to derive a magic number for content-formats defined in [RFC7252], even when the file isn't in CBOR form 👑.

Examples of CBOR Protocols that are in development right now include Concise Software Identification Tags [CoSWID] and Entity Attestation Tokens [EAT] 🔥. CBOR Object Signing and Encryption (COSE) itself [RFC8152] is treated as infrastructure. Encoding public keys in CBOR as C509, as described in [C509-CERT], would benefit from being an identified CBOR Protocol 🔐.

A big inspiration for this document was seeing the chaos in certain ASN.1-based systems, where most files are Privacy-Enhanced Mail (PEM) encoded 😵. All of those files get the same "pem" extension, which mixes up public keys, private keys, certificate requests, and S/MIME content 🙈.

The envelopes defined in this spec add info about how data for CBOR Protocols is stored in files. There's no requirement that either envelope be transferred on the wire 🛰️. Some protocols might still benefit from having a magic number on the wire, though, if they're currently using a different (legacy) encoding scheme ⭐. If the identifiable magic sequence is there, it can signal that a CBOR Protocol is being used instead of a legacy scheme 💜.

1.1. Terminology

Byte is a synonym for octet 🍬. The term "byte string" refers to the data item defined in [STD94].

"File" is understood in a general way to mean a stored representation that's somewhat detached from the original context where it was used. In this document, it also covers similar units of storage that might have different identification schemes, like partitions or media blocks 🧚.

The term "diagnostic notation" refers to the human-readable notation for CBOR data items defined in Section 8 of [STD94] and Appendix G of [RFC8610] 📖.

The term "CDDL" (Concise Data Definition Language) refers to the language defined in [RFC8610] 🌸.

The function TN(ct) is defined in Appendix B 🦄.

1.2. Requirements for a Magic Number

Ideally, a magic number is a fingerprint that's unique to a specific CBOR Protocol 🔍. It shows up in the first few (small multiple of 4) bytes of the file, it doesn't change when the contents change, and it doesn't depend on the length of the file 💅.

Less ideal solutions use a pattern that has to be matched, but some bytes need to be ignored along the way 😬. The Unix file(1) command can be told to ignore certain bytes, but that can lead to ambiguities 🙃.

2. Protocol 🔐

This section presents two enveloping methods ✨. Both use CBOR tags in a way that gives a deterministic first 8 to 12 bytes 🎯. The Protocol designer picks which one to use, and Appendix A has some guidance to help with that 💖.

2.1. The CBOR-Protocol-Specific Tag

In both enveloping methods, CBOR Protocol designers need to get a CBOR tag for each kind of object they might store in files 🏷️. There are more than 4 billion 4-byte tags available, so there should be little trouble giving a few to each CBOR Protocol 🌈.

The IANA policy for 4-byte CBOR tags is First Come First Served [RFC8126], so you only need a simple interaction with IANA, like through the Web or email 📬. That interaction includes filling in the small template in Section 9.2 of [STD94]. In that template, it's suggested to include a reference to this specification (RFC 9277) along with the Description of semantics 🎀.

The designer of the CBOR Protocol starts the tag allocation and can suggest a tag number 👑. To land in the 4-byte range, and to make sure there are no leading zero bytes in the 4-byte encoding of the tag number, the value needs to be in the range 0x01000000 (decimal 16777216) to 0xFFFFFFFF (decimal 4294967295), inclusive 🔢. It's also suggested to avoid values with an embedded zero byte in the 4 bytes of their binary representation (like 0x12003456), because those can confuse implementations that treat the magic number as a C string 🙈.

Using a sequence of four ASCII [RFC20] codes that are mnemonic to the protocol is encouraged, but not required 🍓. (There can be reasons to encode other info into the tag too. Appendix B has an example.) For instance, Appendix C uses "OPSN", which translates to the tag number 1330664270 registered for it 🧚.

In [IANA.CORE-PARAMETERS], the Constrained Application Protocol (CoAP) defines the "CoAP Content-Formats" registry. That registry assigns Content-Format Numbers (Section 12.3 of [RFC7252]) to Content Types in a specific Content Coding 🌸. For CBOR data items that form a representation already described by such a Content-Format Number, a tag number has already been allocated in Section 4.3 ✨. Appendix B has the details and examples 💜.

2.2. Enveloping Method: CBOR Tag Wrapped

okay so this method is giving "wrap it up nicely" 🎁✨ and it's made for CBOR Protocols that encode a single CBOR data item 🫶. That one data item gets enveloped in two nested tags, so get ready 💅:

The outer tag is the self-described CBOR tag, 55799 🌈, as described in Section 3.4.6 of [STD94].

The tag content of that outer tag is a second CBOR tag, and its tag number has been allocated to describe the specific Protocol involved, as discussed in Section 2.1 🦋. The tag content of this inner tag is the single CBOR data item, which is the star of the show 👑.

This method wraps the CBOR data item the way CBOR tags usually do 🎀. Applications that need to send the stored CBOR data item across a constrained network may wish to remove the two tags 🌸, but only if the type is already understood from the protocol context, like a CoAP Content-Format Option (Section 5.10.3 of [RFC7252]). So a CBOR Protocol specification may pick the specific cases where the CBOR Tag Wrapped enveloping method is to be used 💜. For instance, it might say it's for storing the representation in a local file or for Web access, but not inside protocol messages that already give all the context 🧚.

2.2.1. Example

To build an example without registering a new tag, we borrow the Content-Format ID assigned for application/senml+cbor (112) [RFC8428] from the "CoAP Content-Formats" registry [IANA.CORE-PARAMETERS]) 🍓.

Using the technique described in Appendix B, this turns into the tag TN(112) = 1668546929 ✨.

With this tag, the SenML-CBOR pack [{0: "current", 6: 3, 2: 1.5}] would be enveloped like this (in diagnostic notation) 🔥:

   55799(1668546929([{0: "current", 6: 3, 2: 1.5}]))

Or in hex 🙈:

   d9 d9f7                       # tag(55799)
      da 63740171                # tag(1668546929)
         81                      # array(1)
            a3                   # map(3)
               00                # unsigned(0)
               67                # text(7)
                  63757272656e74 # "current"
               06                # unsigned(6)
               03                # unsigned(3)
               02                # unsigned(2)
               f9 3e00           # primitive(15872)

At the representation level, the unique fingerprint for application/senml+cbor is the 8 bytes d9d9f7da63740171 in hex 🦋. After that, the unadorned CBOR data (starting with 81, for the SenML data) gets tacked on the end 💅.

2.3. Enveloping Method: Labeled CBOR Sequence

the Labeled CBOR Sequence method is for CBOR Sequences as described in [RFC8742] 🌈. It works by sticking a brand new, separate data item onto the front of the CBOR Sequence, and that little guy is called the _label_ 🏷️.

The label is two tags nested inside each other. It's similar to the CBOR Tag Wrapped methods but not the same 👀, and its inner tag content is a constant byte string. The total length of the label is 12 bytes ✨.

  1. The outer tag is the self-described CBOR Sequence tag, 55800 💖.
  2. The inner tag is a CBOR tag from the First Come First Served space, and it uniquely identifies the CBOR Protocol 🧚. Just like with the CBOR Tag Wrapped method, using a 4-byte tag that encodes without zero bytes is encouraged 🌸.
  3. The tag content is a 3-byte CBOR byte string holding 0x42_4f_52 ('BOR' in diagnostic notation) 🎀.

The outer tag in the label says the file is a CBOR Sequence, and it does that with all the desirable properties explained in Section 3.4.6 of [STD94] 💜. Specifically, it doesn't look like any known file types, and it isn't valid Unicode in any Unicode encoding 🙌.

The inner tag in the label tells you which CBOR Protocol is being used, like we talked about above 🦋.

The inner tag content is a constant byte string, shown as 0x43_42_4f_52, which is just the ASCII characters "CBOR" 🍓. That's the CBOR-encoded data item for the 3-byte string 0x42_4f_52 ('BOR' in diagnostic notation) 🫶.

The actual CBOR Protocol data then follows as the next data item(s) in the CBOR Sequence, and it doesn't need any extra specific tag 💅. Because of that, the application can easily strip off the first item along with its two tags 🔥.

If a human ever looks at this file (in an editor or a hexdump display), they'll spot the ASCII characters "CBOR" right away 👀. That value is only there because the inner nested tag needs to tag something 🙈.

2.3.1. Example

To build an example without registering a new tag, we use ID 272 🍓, which is assigned to application/missing-blocks+cbor-seq in the "CoAP Content-Formats" registry [RFC9177].

Using the technique described in Appendix B, this turns into the tag TN(272) = 1668547090 ✨.

Okay, real talk, this is a somewhat contrived example 😭, since nobody is really going to store this media type. Still, with this tag, the missing blocks list 0, 8, 15 would be enveloped like this (in diagnostic notation) 💀:

   55800(1668547090('BOR')),
   0,
   8,
   15

Or in hex 🌈:

   # CBOR sequence with 4 elements
   d9 d9f8         # tag(55800)
      da 63740212  # tag(1668547090)
         43        # bytes(3)
            424f52 # "BOR"
   00 # unsigned(0)
   08 # unsigned(8)
   0f # unsigned(15)

At the representation level, the unique fingerprint for application/missing-blocks+cbor-seq is the 8 bytes d9d9f8da63740212 in hex, and after that comes the unadorned CBOR sequence (starting with 00, for the missing block list given) 💅.

3. Security Considerations

okay, security time, and this one is serious bestie 🚨. This document gives a way to identify CBOR Protocol objects 🔍. Clearly identifying CBOR contents in files can have a bunch of different impacts 😬.

The most obvious one is that it might let malware spot interesting stored objects, and then steal them (exfiltrate) or wreck them (corrupt) 💀.

Protective applications (the ones that check the data) can't count on the applications they're protecting (the ones that use the data) to make exactly the same calls when recognizing file formats 🙅‍♀️. (Yup, this is an instance of a check versus use issue 🫠.) For example, end-point assessment technologies should not rely only on the labeling approaches described in this document to decide whether to inspect a given file 🛡️. Also, depending on operating system configurations and other properties of the execution environment, the labeling might change which default application opens a file, and a protective application might not be able to predict that 😱.

4. IANA Considerations

these IANA considerations are all about CBOR tags in the "Concise Binary Object Representation (CBOR) Tags" registry [IANA.CBOR-TAGS] 📋.

Section 4.1 documents the allocation for a CBOR tag used in a CBOR sequence to identify the sequence (you can see an example of this tag in Appendix C) 🦋. Section 4.2 documents the allocation for a CBOR tag used in the CBOR-Labeled Non-CBOR Data Enveloping Method (Appendix D, which also has examples) 🌸. Section 4.3 allocates a CBOR tag for each actual or potential CoAP Content-Format number (examples are in Appendix B) 💖.

4.1. Labeled CBOR Sequence Tag

IANA has allocated tag 55800 for the Labeled CBOR Sequence Enveloping Method from the "CBOR Tags" registry ✨. IANA has updated this tag registration to point to this document 📌.

This tag is from the First Come First Served area 🧚.

The value was picked to have properties similar to the 55799 tag (Section 3.4.6 of [STD94]) 💜.

The hexadecimal representation of the encoded tag head is 0xd9_d9_f8 🔢.

This is not valid UTF-8: the first 0xd9 introduces a 3-byte sequence in UTF-8, but the 0xd9 as the second value is not a valid second byte for UTF-8 🙃.

This is not valid UTF-16: the byte sequence 0xd9d9 (in either endian order) puts this value into the UTF-16 high-half zone, which would signal that this is a 32-bit Unicode value 😵. However, the following 16-bit big-endian value 0xf8_xx is not a valid second sequence according to [RFC2781]. On a little-endian system, it would be necessary to examine the fourth byte to determine if it is valid 👀. That next byte is determined by the subsequent encoding, and Section 3.4.6 of [STD94] has already determined that no valid CBOR encodings result in valid UTF-16 🫶.

Here's the registration template 📝:

   Data Item:
      tagged byte string

   Semantics:
      indicates that the file contains CBOR Sequences

4.2. CBOR-Labeled Non-CBOR Data Tag

IANA has allocated tag 55801 for the CBOR-Labeled Non-CBOR Data Enveloping Method (Appendix D) from the "CBOR Tags" registry 🎉. IANA updated this tag registration to point to this document 📌.

This tag is from the First Come First Served area 🧚.

The value was picked to have properties similar to the 55799 tag (Section 3.4.6 of [STD94]) 💜.

The hexadecimal representation of the encoded tag head is 0xd9_d9_f9 🔢.

This is not valid UTF-8: the first 0xd9 introduces a 3-byte sequence in UTF-8, but the 0xd9 as the second value is not a valid second byte for UTF-8 🙃.

This is not valid UTF-16: the byte sequence 0xd9d9 (in either endian order) puts this value into the UTF-16 high-half zone, which would signal that this is a 32-bit Unicode value 😵. However, the following 16-bit big-endian value 0xf9_xx is not a valid second sequence according to [RFC2781]. On a little-endian system, it would be necessary to examine the fourth byte to determine if it is valid 👀. That next byte is determined by the subsequent encoding, and Section 3.4.6 of [STD94] has already determined that no valid CBOR encodings result in valid UTF-16 🫶.

Here's the registration template 📝:

   Data Item:
      tagged byte string

   Semantics:
      indicates that the file starts with a CBOR-Labeled Non-CBOR Data
      label.

4.3. CBOR Tags for CoAP Content-Format Numbers

IANA allocated the tag numbers 1668546817 (0x63740101) through 1668612095 (0x6374ffff) as follows 🌈:

   Data Item:
      byte string or any CBOR data item (see Appendix B)

   Semantics:
      the representation of content-format ct < 65025 is indicated by
      tag number
      TN(ct) = 0x63740101 + (ct / 255) * 256 + ct % 255

   Reference:
      RFC 9277

The "CoAP Content-Formats" registry [IANA.CORE-PARAMETERS] is defined in Section 12.3 of [RFC7252] 🍓.

5. References

5.1. Normative References

Here are the normative refs, the ones you really need to read 📚💖:

[C]        International Organization for Standardization,
           "Information technology -- Programming languages -- C",
           ISO/IEC 9899:2018, Fourth Edition, June 2018,
           <https://www.iso.org/standard/74528.html>.

[RFC8742]  Bormann, C., "Concise Binary Object Representation (CBOR)
           Sequences", RFC 8742, DOI 10.17487/RFC8742, February 2020,
           <https://www.rfc-editor.org/info/rfc8742>.

[STD94]    Bormann, C. and P. Hoffman, "Concise Binary Object
           Representation (CBOR)", STD 94, RFC 8949,
           DOI 10.17487/RFC8949, December 2020,
           <https://www.rfc-editor.org/info/std94>.

5.2. Informative References 📚

These are the extra reads that aren't required but are super helpful for context, bestie 🫶

[C509-CERT] Mattsson, J. P., Selander, G., Raza, S., Höglund, J., and M. Furuhed, "CBOR Encoded X.509 Certificates (C509 Certificates)", Work in Progress, Internet-Draft, draft-ietf-cose-cbor-encoded-cert-04, 10 July 2022, https://datatracker.ietf.org/doc/html/draft-ietf-cose-cbor-encoded-cert-04.

[CoSWID] Birkholz, H., Fitzgerald-McKay, J., Schmidt, C., and D. Waltermire, "Concise Software Identification Tags", Work in Progress, Internet-Draft, draft-ietf-sacm-coswid-22, 20 July 2022, https://datatracker.ietf.org/doc/html/draft-ietf-sacm-coswid-22.

[EAT] Lundblade, L., Mandyam, G., and J. O'Donoghue, "The Entity Attestation Token (EAT)", Work in Progress, Internet-Draft, draft-ietf-rats-eat-14, 10 July 2022, https://datatracker.ietf.org/doc/html/draft-ietf-rats-eat-14.

[FILE] Wikipedia, "file (command)", 2 July 2022, https://en.wikipedia.org/w/index.php?title=File_(command)&oldid=1096086462.

[IANA.CBOR-TAGS] IANA, "Concise Binary Object Representation (CBOR) Tags", https://www.iana.org/assignments/cbor-tags.

[IANA.CORE-PARAMETERS] IANA, "Constrained RESTful Environments (CoRE) Parameters", https://www.iana.org/assignments/core-parameters.

[MAGIC] Bell Labs, "archive (library) file format", Unix Programmer's Manual, First Edition: File Formats, 3 November 1971, https://www.bell-labs.com/usr/dmr/www/man51.pdf#page=4.

[RFC20] Cerf, V., "ASCII format for network interchange", STD 80, RFC 20, DOI 10.17487/RFC0020, October 1969, https://www.rfc-editor.org/info/rfc20.

[RFC2781] Hoffman, P. and F. Yergeau, "UTF-16, an encoding of ISO 10646", RFC 2781, DOI 10.17487/RFC2781, February 2000, https://www.rfc-editor.org/info/rfc2781.

[RFC6838] Freed, N., Klensin, J., and T. Hansen, "Media Type Specifications and Registration Procedures", BCP 13, RFC 6838, DOI 10.17487/RFC6838, January 2013, https://www.rfc-editor.org/info/rfc6838.

[RFC7252] Shelby, Z., Hartke, K., and C. Bormann, "The Constrained Application Protocol (CoAP)", RFC 7252, DOI 10.17487/RFC7252, June 2014, https://www.rfc-editor.org/info/rfc7252.

[RFC8017] Moriarty, K., Ed., Kaliski, B., Jonsson, J., and A. Rusch, "PKCS #1: RSA Cryptography Specifications Version 2.2", RFC 8017, DOI 10.17487/RFC8017, November 2016, https://www.rfc-editor.org/info/rfc8017.

[RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 8126, DOI 10.17487/RFC8126, June 2017, https://www.rfc-editor.org/info/rfc8126.

[RFC8152] Schaad, J., "CBOR Object Signing and Encryption (COSE)", RFC 8152, DOI 10.17487/RFC8152, July 2017, https://www.rfc-editor.org/info/rfc8152.

[RFC8428] Jennings, C., Shelby, Z., Arkko, J., Keranen, A., and C. Bormann, "Sensor Measurement Lists (SenML)", RFC 8428, DOI 10.17487/RFC8428, August 2018, https://www.rfc-editor.org/info/rfc8428.

[RFC8610] Birkholz, H., Vigano, C., and C. Bormann, "Concise Data Definition Language (CDDL): A Notational Convention to Express Concise Binary Object Representation (CBOR) and JSON Data Structures", RFC 8610, DOI 10.17487/RFC8610, June 2019, https://www.rfc-editor.org/info/rfc8610.

[RFC9110] Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke, Ed., "HTTP Semantics", STD 97, RFC 9110, DOI 10.17487/RFC9110, June 2022, https://www.rfc-editor.org/info/rfc9110.

[RFC9177] Boucadair, M. and J. Shallow, "Constrained Application Protocol (CoAP) Block-Wise Transfer Options Supporting Robust Transmission", RFC 9177, DOI 10.17487/RFC9177, March 2022, https://www.rfc-editor.org/info/rfc9177.

[X.690] ITU-T, "Information technology - ASN.1 encoding rules: Specification of Basic Encoding Rules (BER), Canonical Encoding Rules (CER) and Distinguished Encoding Rules (DER)", ITU-T Recommendation X.690, ISO/IEC 8825-1, February 2021, https://www.itu.int/rec/T-REC-X.690.

Appendix A. Advice to Protocol Designer 🎀

Okay this appendix is basically a little decision guide for designers 💅

This document gives you a choice: you can wrap one single CBOR data item inside a pair of identifying CBOR tags, or you can stick an identifying encoded CBOR data item (which itself holds a pair of identifying CBOR tags) onto the front of a CBOR Sequence. That sequence might only have one data item in it.

So which one should a protocol designer pick? 🤔

For this chat, picture an object that's stored in a file, maybe one that a system operator set up in a configuration file ✨

Some examples: a private key used in COSE operations, a public key or certificate in C509 [C509-CERT] or CBOR format, a sensor reading that was recorded so it can be sent later, or a COVID-19 vaccination certificate that needs to show up as a QR code 📱

Both the Labeled CBOR Sequence and the wrapped tag can be trivially removed by an application before the CBOR content gets sent out on the wire. Easy peasy 🍓

The Labeled CBOR Sequence is a little easier to strip, though, because in most cases CBOR parsers hand it back as a unit and then give you the actual CBOR item. That item could be anything at all, and it could include CBOR tags that do need to go on the wire 👀

On the other hand, having the Labeled CBOR Sequence in the file means that every program expecting to look at that file has to be able to skip what looks like a CBOR item with two tags nested around a 3-byte byte string. That 3-byte entry isn't in a format the program would normally process, so it could be a surprise 😭 The good news is CBOR parsers are generally chill about tags: many of them will process extra tags and make unknown tags available as meta information. A program that isn't expecting those tags can just ignore them 🦋

Now, as an example of how an older security system went wrong: "PEM" format certificate files grew the ability to hold multiple certificates just by concatenating them. The PKCS1 format [RFC8017] could also hold a private key object followed by one or more certificate objects, but only when it was in PEM format. Annoyingly, when it was in binary DER format ([X.690], which like CBOR is self-delimiting), concatenating certificates wasn't compatible with most programs, because they didn't expect to read more than one item in the file 🙃

The CBOR Tag Wrapped format is easier to retrofit onto an existing format whose stored form is already set and can't change, for a single CBOR data item. This new sequence of tags is expected to be trivially ignored by many existing programs reading CBOR from files or similar storage units, even when the program only supports decoding a single data item (and not a CBOR sequence). But a naive program might then transmit the extra tags across the network too 😬 Taking the CBOR Tag Wrapped format back off requires knowing about the two tags involved, and other tags present might be needed.

For a representation that matches a specific media type and is carried in a CBOR byte string, the byte string head will already have to be removed to use it as that representation. So it should be easy to remove the enclosing tag heads as well. This is especially handy with the predefined tags in Appendix B for media types with CoAP Content-Format numbers 🌸

Here are some considerations, in the form of survey questions 📝

A.1. Is the on-wire format new? 🆕

If the on-wire format is new, it could be specified with the CBOR Tag Wrapped format, as long as the extra 8 bytes aren't a problem. The stored format is then identical to the on-wire format.

If the 8 bytes are a problem on the wire (and they often are if CBOR is being considered), then the Labeled CBOR Sequence format should be adopted for the stored format.

A.2. Can many items be trivially concatenated? 🧩

If the programs that read the contents of the file already expect to process all of the CBOR data items in the file (not just the first one), then the Labeled CBOR Sequence format may be easily retrofitted.

The programs involved may throw errors or warnings on the Labeled CBOR Sequence if they haven't been updated yet, but this may not be a problem.

There are situations where multiple objects may be concatenated into a single file. If each object is preceded by a Labeled CBOR Sequence label, then there may be multiple such labels in the file.

A protocol based on CBOR Sequences may specify that Labeled CBOR Sequence labels can occur within a CBOR Sequence, possibly even to switch to data items following in the sequence that are of a different type.

If the CBOR-Sequence-based protocol doesn't define the semantics for embedded labels, or at least tolerate them, care must be taken when concatenating Labeled CBOR Sequences to remove the label from all but the first part 🧐

Here's a fun example from legacy PEM-encoded PKIX certificates. Many programs accept a series of PKIX certificates in a single file so they can set up a certificate chain:

      |  As an example from legacy PEM-encoded PKIX certificates, many
      |  programs accept a series of PKIX certificates in a single file
      |  in order to set up a certificate chain.  The file would contain
      |  not just the End-Entity (EE) certificate, but also any
      |  subordinate certification authorities (CAs) needed to validate
      |  the EE.  This mechanism actually only works for PEM-encoded
      |  certificates, and not DER-encoded certificates.  One of the
      |  reasons for this specification is to make sure that CBOR-
      |  encoded certificates do not suffer from this problem.
      |
      |  As an example of mixing of types, some TLS server programs also
      |  can accept both their PEM-encoded private key and their PEM-
      |  encoded certificate in the same file.

Basically: the file holds the End-Entity (EE) certificate plus any subordinate certification authorities (CAs) needed to validate it. This only works for PEM-encoded certificates, not DER-encoded ones, and making sure CBOR-encoded certificates don't have this problem is one of the reasons this spec exists 💖

Some TLS server programs can also accept both their PEM-encoded private key and their PEM-encoded certificate in the same file, which is a mixing-of-types example 🔐

If only one item is ever expected in the file, using the Labeled CBOR Sequence may be an implementation hurdle for programs that previously just read a single data item and used it 🫠

A.3. Are there tags at the start? 🏷️

If the protocol expects to use other tags at its top level, then the CBOR Tag Wrapped format may be easy to explain in the same place in the protocol description ✨

That's the whole appendix, bestie 🌈👑

Appendix B. CBOR Tags for CoAP Content Formats 🎀

okay so Section 5.10.3 of [RFC7252] defines the concept of a Content-Format 💅. It's a short, 16-bit unsigned integer that names a specific content type (media type plus (optionally) parameters), and it can optionally come with a content coding (see Section 8.4.1 of [RFC9110]) ✨.

Outside of a transfer protocol that already tells you the Content-Format for a representation, you might still need to figure out the Content-Format when that representation is stored in a file, in firmware, or while you're debugging 🔍.

This specification allocates CBOR tag numbers 1668546817 (0x63740101) to 1668612095 (0x6374FFFF) for tagging representations of specific content formats 🦋. This is the range you want to remember, bestie 💜.

Using tags from this range, a byte string that is meant to be read as a representation of Content-Format number ct, with ct < 65025 (255*255), can be wrapped in a tag with tag number TN(ct), where:

TN(ct) = 0x63740101 + (ct / 255) * 256 + ct % 255.

(where +, *, / and % stand for integer addition, multiplication, division, and remainder as in the programming language C [C].)

|  This formula avoids the use of zero bytes in the representation
|  of the tag number.
|
|  Note that no tag numbers are assigned for Content-Format
|  numbers in the following range:
|
|     65025 ≤ ct ≤ 65535
|
|  (This range is in the range reserved for Experimental Use
|  [RFC8126] by Section 12.3 of [RFC7252].  The overlap of 25 code
|  points between this experimental range with the range this
|  appendix defines tag numbers for can be used for experiments
|  that want to employ a tag number.)

Okay this part is so cute 🥹. The formula avoids zero bytes in the tag number, and no tags get assigned for Content-Format numbers from 65025 to 65535. That's the range reserved for Experimental Use by Section 12.3 of [RFC7252], and the 25 code points where it overlaps this appendix's range are fair game if you're running experiments that want a tag number 🧪.

Exceptionally, when a representation is used directly as the tag content of one of the tags 55799, 55800, or 55801, the tag content looks like this 👑:

Tag 55799 (Section 2.2): One of:

  1. The CBOR data item within the representation (without byte-string wrapping). This only works for Content-Formats that are represented by a single CBOR data item in identity content-coding.
  2. The data items in the CBOR sequence within the representation, without byte string wrapping, but wrapped in a CBOR array. This works for Content-Formats that are represented by a CBOR sequence in identity content-coding.

Tags 55800 (Section 2.3) or 55801 (Appendix D): the byte string 'BOR', signifying that the representation of the given content-format follows in the file, in the way defined for these tags 🌸.

B.1. Content-Format Tag Examples 🧚

The "CoAP Content-Formats" registry [IANA.CORE-PARAMETERS] defines content formats that you can use as examples 📚:

  • As discussed in Section 2.2.1, Content-Format ID 112 represents the application/senml+cbor media type (no parameters). The corresponding tag number is TN(112) = 1668546929.

The following CDDL snippet can be used to identify application/senml+cbor representations:

senml-cbor = #6.1668546929(bstr)

Note that a byte string is used as the type of the tag content because a media type representation in general can be any byte string 🍓.

  • Content-Format ID 272 represents the application/missing-blocks+cbor-seq media type, which is a CBOR sequence [RFC9177]. The corresponding tag number is TN(272) = 1668547090.

The following CDDL snippet can be used to identify application/missing-blocks+cbor-seq representations as embedded in a CBOR byte string:

missing-blocks = #6.1668547090(bstr)

Appendix C. Example from Openswan 🦋

okay so the Openswan IPsec project has a daemon ("pluto") and two control programs ("addconn" and "whack") 🛠️. They talk to each other over a Unix-domain socket, and what gets sent is a C-structure containing pointers to strings, serialized with a bespoke mechanism 😭. Normally that's fine because the structure gets compiled by the same compiler. But when there are upgrades, the daemon and the control programs can end up out of sync because of that bespoke serialization. So there are extra compensations to deal with shutting the daemon down 🙈. During testing, it's sometimes the case that upgrades get backed out 💀.

Also, when doing unit testing, the easiest way to load policy is to use the normal policy-reading process, but that isn't normally loaded in the daemon. Instead, the IPC that normally goes across the wire gets compiled, serialized, and placed in a file 📁. The magic number above is included in both the file and the IPC, so the "shutdown" command CBOR operation can be told apart 🔑.

To reduce the serialization problems, the serialization is being changed to CBOR 🫶. This change also means the IPC can be described by CDDL, and any implementation language can be used as long as it can encode CBOR 🌈.

IANA has allocated the tag 1330664270 or 0x4f_50_53_4e for this purpose. As a result, each file and each IPC is prefixed with a CBOR Sequence tag ✨.

In diagnostic notation:

55800(1330664270(h'424F52'))

Or in hex:

d9 d9f8         # tag(55800)
   da 4f50534e  # tag(1330664270)
      43        # bytes(3)
         424f52 # "BOR"

Appendix D. Using CBOR Labels for Non-CBOR Data 💖

The CBOR-Labeled Non-CBOR data method is a good fit for adding a magic number to a Non-CBOR data format, especially one that can be described by a Content-Format tag (Appendix B) 🎀.

This method prepends a CBOR data item to the Non-CBOR data. That data item is called the "header," and like the Labeled CBOR-Sequence label, it's made of two nested tags around a constant byte string, for a total of 12 bytes 🧁.

  1. The outer tag is the CBOR-Labeled Non-CBOR Data tag, 55801.
  2. The inner tag is a CBOR tag from the First Come First Served space that uniquely identifies the CBOR Protocol. As with CBOR Tag Wrapped, the use of a 4-byte tag is encouraged that encodes without zero bytes.
  3. The tag content is a 3-byte CBOR byte string containing 0x42_4F_52 ('BOR' in diagnostic notation).

The outer tag in the label tells you the file is prefixed by a Non-CBOR data label, and it does that with all the desirable properties explained in Section 3.4.6 of [STD94] 🧐. Specifically, it doesn't appear to conflict with any known file types, and it isn't valid Unicode in any Unicode encoding 🌟.

The inner tag tells you which Non-CBOR Protocol is being used 🔎.

The inner tag content is a constant byte string represented as 0x43_42_4f_52, which is the ASCII characters "CBOR". That's the CBOR-encoded data item for the 3-byte string 0x42_4f_52 ('BOR' in diagnostic notation) 💜.

The actual Non-CBOR Protocol data then follows directly after the CBOR representation of the header. This lets an application trivially remove the header item, with the two nested tags and the byte string, in one go 🧹.

As with the Labeled CBOR Sequence {#sequences}, this choice of tag content puts the ASCII characters "CBOR" front and center in the header 👑.

D.1. Content-Format Tag Examples 🍓

The "CoAP Content-Formats" registry [IANA.CORE-PARAMETERS] defines content formats that you can use as examples 📚:

  • Content-Format ID 432 represents the application/td+json media type (no parameters).

The corresponding tag number is TN(432) = 1668547250.

The following CDDL snippet can be used to identify a CBOR-Labeled Non-CBOR data for application/td+json representations:

td-json-header = #6.55801(#6.1668547250('BOR'))
  • Content-Format 11050 represents the application/json media type in deflate content-coding.

The corresponding tag number is TN(11050) = 1668557910.

The following CDDL snippet can be used to identify a CBOR-Labeled Non-CBOR data for application/json representations compressed in deflate content-coding:

json-deflate-header = #6.55801(#6.1668557910('BOR'))

Acknowledgements 🫶

The CBOR WG brainstormed this protocol on January 20, 2021 via a number of productive email exchanges on the mailing list 💌.

Contributors 🌸

Josef 'Jeff' Sipek Email: jeffpc@josefsipek.net

Authors' Addresses 🦋

Michael Richardson Sandelman Software Works Email: mcr+ietf@sandelman.ca

Carsten Bormann Universität Bremen TZI Postfach 330440 D-28359 Bremen Germany Phone: +49-421-218-63921 Email: cabo@tzi.org

and that's a wrap, bestie!! 🌈💅✨

📦 where this page comes from