RFC 9193

Sensor Measurement Lists (SenML) Fields for Indicating Data Value Content-Format

📅 published June 2022, from ietf

RFC 9193 🎀 Sensor Measurement Lists (SenML) Fields for Indicating Data Value Content-Format

Okay bestie, the title block is giving a lot, so here's the tea ☕. This one comes from the IETF (Internet Engineering Task Force) and it's a Standards Track doc, aka the real deal 👑. It was published in June 2022 and it has the ISSN 2070-1721 🌸. The authors are A. Keränen from Ericsson and C. Bormann from Universität Bremen TZI 💅. The title block doesn't list anything this one obsoletes or updates, so there's no drama there ✨.

Abstract

So here's the vibe: SenML media types can carry all kinds of values, from numbers and text strings to arbitrary binary Data Values 🌈. To make those binary Data Values easier to process, this document defines a pair of new SenML fields that say what content format they're in. That means their Internet media type (parameters included) and any content codings that were applied 🦋.

Status of This Memo

This is an Internet Standards Track document. 💖

This doc is a product of the IETF, so it represents the consensus of the IETF community 🫶. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG) ✨. If you want more on Internet Standards, check out Section 2 of RFC 7841 📚.

Info about the current status of this document, any errata, and how to send feedback is at https://www.rfc-editor.org/info/rfc9193 🔗.

Copyright (c) 2022 IETF Trust and the persons identified as the document authors. All rights reserved ©️.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as in effect on the date this document was published. Please read these carefully, bestie, because they spell out your rights and restrictions for this document 📜. Code Components extracted from this document must include Revised BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty as described in the Revised BSD License 💀.

Table of Contents

   1.  Introduction
     1.1.  Evolution
   2.  Terminology
   3.  SenML Content-Format ("ct") Field
   4.  SenML Base Content-Format ("bct") Field
   5.  Examples
   6.  ABNF
   7.  Security Considerations
   8.  IANA Considerations
   9.  References
     9.1.  Normative References
     9.2.  Informative References
   Acknowledgments
   Authors' Addresses

1. Introduction ✨

The SenML media types [RFC8428] can carry all sorts of data 🌈. In the example in Figure 1, a temperature value, an indication of whether a lock is open, and a Data Value (with SenML field "vd") read from a Near Field Communication (NFC) reader all get sent in one SenML Pack 🔐. The example uses the SenML JSON representation, so the "vd" (Data Value) field is encoded as a base64url string (without padding), as per Section 5 of [RFC8428] 🍓.

[
  {"bn":"urn:dev:ow:10e2073a01080063:","n":"temp","u":"Cel","v":7.1},
  {"n":"open","vb":false},
  {"n":"nfc-reader","vd":"aGkgCg"}
]

Figure 1: SenML Pack with Unidentified Binary Data

The receiver is supposed to know how to read the data in the "vd" field based on context, like the name of the data source and out-of-band knowledge of the application 🧠. But that context isn't always easy to get, especially when the Pack gets passed along over time and through multiple entities 😭. So to make automatic interpretation possible, it's useful to be able to indicate an Internet media type and, optionally, content codings right inside the SenML Record 💅.

The Constrained Application Protocol (CoAP) Content-Format (Section 12.3 of [RFC7252]) gives you this info as a single unsigned integer 🦋. For instance, [RFC8949] defines Content-Format number 60 for Content-Type application/cbor 🫧. You can put this Content-Format number in the Record, which Figure 2 shows 👀. All the registered CoAP Content-Format numbers are listed in the "CoAP Content-Formats" registry [IANA.core-parameters], as specified by Section 12.3 of [RFC7252] 📋. Also note that, at the time of writing, this registry only allows for zero or one content coding. Nothing in this document has to change if the registry gets extended to allow sequences of content codings 🌸.

{"n":"nfc-reader", "vd":"gmNmb28YKg", "ct":"60"}

Figure 2: SenML Record with Binary Data Identified as CBOR

In this example SenML Record, the Data Value holds a string "foo" and a number 42, encoded in a Concise Binary Object Representation (CBOR) [RFC8949] array 🧚. Since this example uses the JSON format of SenML, the Data Value holding the binary CBOR value is base64 encoded (Section 5 of [RFC4648]) 🙈. After base64 decoding, the Data Value shows up in CBOR diagnostic notation in Figure 3 🔍.

82           # array(2)
   63        # text(3)
      666F6F # "foo"
   18 2A     # unsigned(42)

Figure 3: Example Data Value in CBOR Diagnostic Notation

1.1. Evolution

Just like SenML in general, the creator of a SenML Pack has no expectation of knowing (or having negotiated with) each consumer of that Pack 🌍. Those consumers might be really far away in space, and especially far away in time ⏳. So the SenML creator generally has no way of knowing whether the consumer knows:

  • each specific Media-Type-Name used,
  • each parameter and each parameter value used,
  • each content coding in use, and
  • each Content-Format number in use for a combination of these.

What SenML, along with the new fields defined here, actually guarantees is that a recipient implementation knows when it needs an update to understand these field values and the values they control 🔥. Registries are used to evolve these name spaces in a controlled way 📈. A consumer can process SenML Packs even when it doesn't understand all the information in them, and that information can usually be kept through this processing so it's still useful for later consumers 🫶.

2. Terminology 📖

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

Media type: A registered label for representations (byte strings) prepared for interchange [RFC1590] [RFC6838], identified by a Media-Type-Name 💌.

Media-Type-Name: A combination of a type-name and a subtype-name registered in [IANA.media-types], as per [RFC6838], which is conventionally identified by the two names separated by a slash 🎀.

Content-Type: A Media-Type-Name, optionally associated with parameters (Section 5 of [RFC2045], separated from the Media-Type-Name and from each other by a semicolon). In HTTP and many other protocols, it's used in a Content-Type header field 🌷.

Content coding: A name registered in the "HTTP Content Coding Registry" [IANA.http-parameters], as specified by Sections 16.6.1 and 18.6 of [RFC9110], indicating an encoding transformation with semantics further specified in Section 8.4.1 of [RFC9110]. Confusingly, in HTTP, content coding values live in a header field called Content-Encoding. But "content coding" is the correct term for the process and the registered values 🙃.

Content format: The combination of a Content-Type and zero or more content codings, identified by either (1) a numeric identifier defined in the "CoAP Content-Formats" registry [IANA.core-parameters], as per Section 12.3 of [RFC7252] (referred to as Content-Format number), or (2) a Content-Format-String 🧁.

Content-Format-String: The string representation of the combination of a Content-Type and zero or more content codings 🍬.

Content-Format-Spec: The string representation of a content format. It's either a Content-Format-String or the (decimal) string representation of a Content-Format number 🪄.

Readers should also be familiar with the terms and concepts discussed in [RFC8428] 📚.

3. SenML Content-Format ("ct") Field 🎀

When a SenML Record has a Data Value field ("vd"), the Record MAY also include a Content-Format indication field, using label "ct". The value of this field is a Content-Format-Spec, which is one of the following:

  • a CoAP Content-Format number in decimal form with no leading zeros (except for the value "0" itself). This value represents an unsigned integer in the range of 0-65535, similar to the "ct" attribute defined in Section 7.2.1 of [RFC7252] for CoRE Link Format [RFC6690] 🔢.
  • a Content-Format-String containing a Content-Type and zero or more content codings (see below) 🫧.

The syntax of this field is formally defined in Section 6 📐.

The CoAP Content-Format number is a simple and efficient way to say what type the data is 💨. But some Internet media types and their content coding and parameter alternatives don't have assigned CoAP Content-Format numbers. So using Content-Type plus zero or more content codings is allowed too 🌈. Both methods use a string value in the "ct" field, which keeps its data type consistent across uses. When the "ct" field contains only digits, it's read as a CoAP Content-Format number 🔎.

To say that one or more content codings are used with a Content-Type, you append each content coding value to the Content-Type value (media type and parameters, if any), separated by an "@" sign. They go in the order the content codings were applied, which is the same order as in Section 8.4 of [RFC9110]. For example (using a content coding value of "deflate", as defined in Section 8.4.1.2 of [RFC9110]):

text/plain; charset=utf-8@deflate

If there's no "@" sign after the media type and parameters, then no content coding has been specified, and the "identity" content coding is used. That means no encoding transformation is applied 🙌.

4. SenML Base Content-Format ("bct") Field 🍓

The Base Content-Format field, label "bct", gives a default value for the Content-Format field (label "ct") within its range 🫶. The range of the base field includes the Record that contains it, up to (but not including) the next Record with a "bct" field, if there is one, or up to the end of the Pack otherwise. To resolve this base field (Section 4.6 of [RFC8428]), you add its value with the label "ct" to every Record in this range that has a "vd" field but doesn't already have a Content-Format ("ct") field 🧩.

Figure 4 is a variation of Figure 2 with multiple records. The "nfc-reader" records resolve to the base field value "60", and the "iris-photo" record overrides that with the "image/png" media type. (The actual data is left out for brevity 🙈.)

[
  {"n":"nfc-reader", "vd":"gmNmb28YKg",
   "bct":"60", "bt":1627430700},
  {"n":"nfc-reader", "vd":"gmNiYXIYKw", "t":10},
  {"n":"iris-photo", "vd":".....", "ct":"image/png", "t":10},
  {"n":"nfc-reader", "vd":"gmNiYXoYLA", "t":20}
]

Figure 4: SenML Pack with the bct Field

5. Examples ✨

okay so these are the valid values for the ct and bct fields, with the explanations in parentheses so you know what each one means 💅

  • "60" (CoAP Content-Format number for application/cbor) 🧚
  • "0" (CoAP Content-Format number for text/plain with parameter charset=utf-8) 🌸
  • "application/json" (JSON Content-Type, which is equivalent to the CoAP Content-Format number "50") 💖
  • "application/json@deflate" (JSON Content-Type with deflate as the content coding, which is equivalent to the CoAP Content-Format number "11050") 🫶
  • "application/json@deflate@aes128gcm" (JSON Content-Type with deflate followed by aes128gcm as the content codings) 🔥
  • "text/csv" (Comma-Separated Values (CSV) [RFC4180] Content-Type) 🦋
  • "text/csv;header=present@gzip" (CSV with a header row, using gzip as the content coding) 👑

6. ABNF

this section gives a formal definition of the syntax of Content-Format-Spec strings using ABNF notation [RFC5234] 🎀 It has three new rules, plus a bunch of rules that were collected and adapted from other RFCs [RFC9110] [RFC6838] [RFC5234] [RFC8866]. The grammar is below, and it's giving very precise ✨

; New in this document

Content-Format-Spec = Content-Format-Number / Content-Format-String

Content-Format-Number = "0" / (POS-DIGIT *DIGIT)
Content-Format-String   = Content-Type *("@" Content-Coding)

; Cleaned up from RFC 9110,
; leaving only SP as blank space,
; removing legacy 8-bit characters, and
; leaving the parameter as mandatory with each semicolon:

Content-Type   = Media-Type-Name *( *SP ";" *SP parameter )
parameter      = token "=" ( token / quoted-string )

token          = 1*tchar
tchar          = "!" / "#" / "$" / "%" / "&" / "'" / "*"
               / "+" / "-" / "." / "^" / "_" / "`" / "|" / "~"
               / DIGIT / ALPHA
quoted-string  = %x22 *( qdtext / quoted-pair ) %x22
qdtext         = SP / %x21 / %x23-5B / %x5D-7E
quoted-pair    = "\" ( SP / VCHAR )

; Adapted from Section 8.4.1 of RFC 9110

Content-Coding   = token

; Adapted from various specs

Media-Type-Name = type-name "/" subtype-name

; From RFC 6838

type-name = restricted-name
subtype-name = restricted-name

restricted-name = restricted-name-first *126restricted-name-chars
restricted-name-first  = ALPHA / DIGIT
restricted-name-chars  = ALPHA / DIGIT / "!" / "#" /
                         "$" / "&" / "-" / "^" / "_"
restricted-name-chars =/ "." ; Characters before first dot always
                             ; specify a facet name
restricted-name-chars =/ "+" ; Characters after last plus always
                             ; specify a structured syntax suffix


; Boilerplate from RFC 5234 and RFC 8866

DIGIT     =  %x30-39           ; 0 - 9
POS-DIGIT =  %x31-39           ; 1 - 9
ALPHA     =  %x41-5A / %x61-7A ; A - Z / a - z
SP        =  %x20
VCHAR     =  %x21-7E           ; printable ASCII (no SP)

Figure 5: ABNF Syntax of Content-Format-Spec

7. Security Considerations 💀

this part is SUPER important, bestie 😭 Just because the data says what its media type is does not let a consuming application skip checking its inputs. Properly checking inputs is still on the app. 🙈

Also, an attacker can supply crafted SenML data that names media types they picked. That can expose vulnerabilities in the handlers for those media types. This includes "decompression bombs", which are compressed data crafted to blow up into extremely large data items when decompressed. Scary, right? Be careful out there 🫶

8. IANA Considerations 🦋

IANA has assigned the following new labels in the "SenML Labels" subregistry of the "Sensor Measurement Lists (SenML)" registry [IANA.senml] ✨ (as defined in Section 12.2 of [RFC8428]) for the Content-Format indication, as shown in Table 1:

 +=====================+=======+===========+==========+===========+
 |                Name | Label | JSON Type | XML Type | Reference |
 +=====================+=======+===========+==========+===========+
 | Base Content-Format | bct   | String    | string   | RFC 9193  |
 +---------------------+-------+-----------+----------+-----------+
 |      Content-Format | ct    | String    | string   | RFC 9193  |
 +---------------------+-------+-----------+----------+-----------+

          Table 1: IANA Registration for New SenML Labels

Note that, per Section 12.2 of [RFC8428], no CBOR labels nor Efficient XML Interchange (EXI) schemaId values (EXI ID column) are supplied 🍓

9. References 📚

9.1. Normative References

9.2. Informative References

Acknowledgments 🫶

big thanks to the authors' crew 💜 The authors would like to thank Sérgio Abreu for the discussions that led to the design of this extension, and Isaac Rivera for reviews and feedback. Klaus Hartke suggested not burdening this document with a separate mandatory-to-implement version of the fields. Alexey Melnikov, Jim Schaad, and Thomas Fossati gave helpful comments at Working Group Last Call. Marco Tiloca asked for clarifying and using the term Content-Format-Spec 🌈

Authors' Addresses 🎀

Ari Keränen Ericsson FI-02420 Jorvas Finland Email: ari.keranen@ericsson.com

Carsten Bormann Universität Bremen TZI Postfach 330440 D-28359 Bremen Germany Phone: +49-421-218-63921 Email: cabo@tzi.org

📦 where this page comes from