RFC 7641
Observing Resources in the Constrained Application Protocol (CoAP)
RFC 7641: Observing Resources in the Constrained Application Protocol (CoAP) 💖
Part 1 of 6 🧚 Next up: Section 2, The Observe Option.
Okay bestie, let's get into it! 🎀 This is RFC 7641 from the IETF, and it's all about letting CoAP clients observe a resource, which means they get fresh updates whenever the server's copy changes. It's giving "subscribe and chill" for tiny devices 🦋
Here's the title block tea ☕:
- Request for Comments: 7641
- Author: K. Hartke, Universitaet Bremen TZI
- Category: Standards Track (so it's an official Internet standard, periodt 👑)
- Date: September 2015
- ISSN: 2070-1721
- Obsoletes or updates: nothing is listed for this document in its header 🙈
Abstract ✨
So the Constrained Application Protocol (CoAP) is a RESTful app protocol built for constrained nodes and networks 🧚 Resource state on a CoAP server can change over time, right? This doc adds a simple extension that lets clients "observe" resources. Basically, a client grabs a representation of a resource and the server keeps it updated for a while 💅
The protocol is best-effort when it sends new representations to clients, and it gives you eventual consistency between what each client sees and what the resource actually is on the server. Iconic, no cap 🌈
Status of This Memo 🌸
This is an Internet Standards Track document. It's a product of the Internet Engineering Task Force (IETF), so it reflects the consensus of the IETF community. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG) 🫶
More info on Internet Standards is in Section 2 of RFC 5741. For the current status, any errata, and how to give feedback, check http://www.rfc-editor.org/info/rfc7641 💜
Copyright Notice 🦋
Copyright (c) 2015 IETF Trust and the persons identified as the document authors. All rights reserved. 🎀
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they stand on the publication date. Read these carefully, because they spell out your rights and restrictions for this document. Code Components pulled from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come without warranty as described in that license 📜
Table of Contents 🍓
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1. Background . . . . . . . . . . . . . . . . . . . . . . . 4
1.2. Protocol Overview . . . . . . . . . . . . . . . . . . . . 4
1.3. Consistency Model . . . . . . . . . . . . . . . . . . . . 6
1.4. Observable Resources . . . . . . . . . . . . . . . . . . 7
1.5. Requirements Notation . . . . . . . . . . . . . . . . . . 8
2. The Observe Option . . . . . . . . . . . . . . . . . . . . . 9
3. Client-Side Requirements . . . . . . . . . . . . . . . . . . 10
3.1. Request . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.2. Notifications . . . . . . . . . . . . . . . . . . . . . . 10
3.3. Caching . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.4. Reordering . . . . . . . . . . . . . . . . . . . . . . . 12
3.5. Transmission . . . . . . . . . . . . . . . . . . . . . . 13
3.6. Cancellation . . . . . . . . . . . . . . . . . . . . . . 13
4. Server-Side Requirements . . . . . . . . . . . . . . . . . . 14
4.1. Request . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.2. Notifications . . . . . . . . . . . . . . . . . . . . . . 14
4.3. Caching . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.4. Reordering . . . . . . . . . . . . . . . . . . . . . . . 16
4.5. Transmission . . . . . . . . . . . . . . . . . . . . . . 17
5. Intermediaries . . . . . . . . . . . . . . . . . . . . . . . 20
6. Web Linking . . . . . . . . . . . . . . . . . . . . . . . . . 20
7. Security Considerations . . . . . . . . . . . . . . . . . . . 21
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 22
9. References . . . . . . . . . . . . . . . . . . . . . . . . . 22
9.1. Normative References . . . . . . . . . . . . . . . . . . 22
9.2. Informative References . . . . . . . . . . . . . . . . . 22
Appendix A. Examples . . . . . . . . . . . . . . . . . . . . . . 24
A.1. Client/Server Examples . . . . . . . . . . . . . . . . . 24
A.2. Proxy Examples . . . . . . . . . . . . . . . . . . . . . 28
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . 30
Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 301. Introduction 🌈
1.1. Background 🧚
CoAP [RFC7252] is meant to give you RESTful services [REST], kind of like HTTP [RFC7230] but way lighter. It cuts down on implementation complexity and packet size so these services work in a highly constrained network of highly constrained nodes [RFC7228]. Small devices, big vibes 🌸
The REST model is a client swapping representations of resources with a server. A representation captures the current or intended state of a resource. The server is the authority for representations of resources in its namespace. A client that cares about a resource sends a request to the server, and the server replies with a representation of the resource as it was at request time 💅
That model gets awkward when a client wants a current representation over a period of time. HTTP's existing options, like repeated polling or HTTP long polling [RFC6202], add a lot of complexity and/or overhead, so they don't fit well in constrained environments 🙈
This document extends the CoAP core protocol with a way for a CoAP client to "observe" a resource on a CoAP server. The client gets a representation of the resource and asks the server to keep it updated for as long as the client is interested 🦋
The protocol keeps REST's architectural properties. It scales well and stays efficient thanks to caches and proxies 🫶 It does not try to solve every problem the HTTP solutions solve, and it doesn't replace publish/subscribe networks, which handle a much more general problem [RFC5989]. Know your lane, bestie 👑
1.2. Protocol Overview 💜
The protocol is built on the well-known observer design pattern [GOF]. In that pattern, components called "observers" register at a specific, known provider called the "subject", because they want to be notified whenever the subject's state changes. The subject manages its own list of registered observers. If an observer cares about several subjects, it must register separately with each one. Okay, that's a lot of registering 📝
Observer Subject
| |
| Registration |
+------------------->|
| |
| Notification |
|<-------------------+
| |
| Notification |
|<-------------------+
| |
| Notification |
|<-------------------+
| |Figure 1: The Observer Design Pattern 🎀
Here's how CoAP realizes the observer design pattern:
- Subject: In CoAP, the subject is a resource in a CoAP server's namespace. Its state can change over time, from rare updates to constant state transformations ✨
- Observer: An observer is a CoAP client that wants a current representation of the resource at any given time 💖
- Registration: A client registers its interest in a resource by sending an extended GET request to the server. Along with a representation of the target resource, this request makes the server add the client to the resource's list of observers 🌸
- Notification: Whenever the resource's state changes, the server notifies each client on the resource's list of observers. Each notification is an extra CoAP response that the server sends in reply to the one extended GET request. It includes a complete, updated representation of the new resource state 🔥
Figure 2 is a little story: a client registers for a resource and gets three notifications. The first one has the current state from the moment of registration, and then two more come in as the resource state changes. Both the registration request and the notifications are marked by the Observe Option defined in this document. In notifications, the Observe Option also carries a sequence number so you can detect reordering. Every notification carries the token the client picked, so the client can match each one back to its request 🧚
Client Server
| |
| GET /temperature |
| Token: 0x4a | Registration
| Observe: 0 |
+------------------->|
| |
| 2.05 Content |
| Token: 0x4a | Notification of
| Observe: 12 | the current state
| Payload: 22.9 Cel |
|<-------------------+
| |
| 2.05 Content |
| Token: 0x4a | Notification upon
| Observe: 44 | a state change
| Payload: 22.8 Cel |
|<-------------------+
| |
| 2.05 Content |
| Token: 0x4a | Notification upon
| Observe: 60 | a state change
| Payload: 23.1 Cel |
|<-------------------+
| |Figure 2: Observing a Resource in CoAP 📈
Note: In this document, "Cel" stands for "degrees Celsius". 🌡️
A client stays on the list of observers for as long as the server can tell it's still interested in the resource. The server may send a notification in a confirmable CoAP message to ask the client for an acknowledgement. If the client deregisters, rejects a notification, or the transmission of a notification times out after several attempts, the client counts as no longer interested. The server then removes it from the list of observers 👋✨
1.3. Consistency Model 🦋
okay so while a client is on the observers list for a resource, the whole goal of the protocol is to keep what the client sees as close to the server's actual state as possible ✨
but ngl, they will end up out of sync sometimes, and that's just life 💀 there are three main ways it happens:
First, there's always some latency between the resource state changing and the notification arriving 🐢
Second, CoAP messages carrying notifications can get lost, so the client keeps assuming an old state until a new notification shows up 😭
And third, the server might wrongly decide the client isn't interested anymore. Then it stops sending notifications, and the client is stuck with an old state until it registers its interest again 🙈
so here's how the protocol handles all that drama:
- It takes a best-effort approach to sending the current representation after a state change. Clients should see the new state as soon as possible, and they should see as many states as possible 🏃♀️ BUT congestion control caps this, so a client can't count on seeing every single state a resource goes through. no cap, it's not guaranteed 💅
- It labels each notification with a maximum duration during which it's acceptable for the observed state and the actual state to be out of sync. Once the notification's age hits that limit, the client can't use the enclosed representation until it gets a new notification ⏳
- It's built on eventual consistency. The protocol guarantees that if the resource doesn't undergo a new state change, all registered observers will eventually have a current representation of the latest state 🌈
1.4. Observable Resources 🌸
okay this part is so cute bc the server is the authority on when resources change their state, and therefore when observers get notified 👑 the protocol doesn't give you any built-in way to set up triggers or thresholds. it's up to the server to expose observable resources that change state in a way that actually matters for the app 🧚
for example, imagine a CoAP server with a temperature sensor attached. it could expose any of these:
<coap://server/temperature>, which changes its state every few seconds to the current reading of the temperature sensor 🌡️<coap://server/temperature/felt>, which changes its state to "COLD" whenever the reading drops below a certain pre-configured threshold, and to "WARM" whenever the reading goes above a second, slightly higher threshold 🥶🥵<coap://server/temperature/critical?above=42>, which changes its state based on the client-specified parameter value. it either changes every few seconds to the current reading if the temperature is above the threshold, or to "OK" when the reading drops below it 🚨<coap://server/?query=select+avg(temperature)+from+Sensor.window:time(30sec)>, which accepts expressions of arbitrary complexity and changes its state accordingly 🤯
so basically, if you design resources that change state only under certain conditions, the client only gets updated when those conditions happen, instead of being flooded with raw sensor data every second 🫶 and because you can parameterize resources, it's not limited to conditions the server picked. the client can specify arbitrarily complex queries too 💖
the designer gets to choose exactly the right level of complexity for the app and the devices involved. it's not a "one size fits all" mechanism baked into the protocol, and I'm obsessed with that flexibility 😭✨
1.5. Requirements Notation 📜
the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. 🧐
2. The Observe Option 🎀
the Observe Option has the following properties. Its meaning depends on whether it's included in a GET request or in a response 👀
+-----+---+---+---+---+---------+--------+--------+---------+
| No. | C | U | N | R | Name | Format | Length | Default |
+-----+---+---+---+---+---------+--------+--------+---------+
| 6 | | x | - | | Observe | uint | 0-3 B | (none) |
+-----+---+---+---+---+---------+--------+--------+---------+
C=Critical, U=Unsafe, N=No-Cache-Key, R=Repeatable
Table 1: The Observe Optionwhen it's included in a GET request, the Observe Option extends the GET method. so it doesn't just retrieve a current representation of the target resource, it also asks the server to add or remove an entry in the resource's list of observers, depending on the option value. the list entry is made of the client endpoint and the token the client put in the request 🧩 the possible values are:
- 0 (register) adds the entry to the list, if it's not already there ✅
- 1 (deregister) removes the entry from the list, if it's there ❌
the Observe Option is not critical for processing the request. if the server can't or won't add a new entry to the list of observers, the request just falls back to a normal GET request, and the response doesn't include the Observe Option 🫠
the Observe Option is not part of the Cache-Key. so a cacheable response obtained with an Observe Option in the request can satisfy a request without one, and vice versa 🔄 when a stored response with an Observe Option is used to satisfy a normal GET request, the option MUST be removed before the response is returned. no exceptions, bestie 🚫
when it's included in a response, the Observe Option marks the message as a notification. that means a matching entry exists in the list of observers, and the server will notify the client about changes to the resource state 📣 the option value is a sequence number used for reordering detection (see Sections 3.4 and 4.4). 🔢
the value of the Observe Option is encoded as an unsigned integer in network byte order, using a variable number of bytes ('uint' option format). see Section 3.2 of RFC 7252 [RFC7252]. 🧠
3. Client-Side Requirements 💜
3.1. Request 💌
a client registers its interest in a resource by sending a GET request with an Observe Option set to 0 (register). if the server replies with a 2.xx response that also includes an Observe Option, the server has successfully added an entry with the client endpoint and request token to the list of observers of the target resource. and then the client will get notified about changes to the resource state 🥳
here's the fun part: just like a fresh response can satisfy a request without contacting the server, the stream of updates from one observation request can satisfy another request, observation or normal GET, as long as the target resource is the same 🌊 a client MUST aggregate such requests and MUST NOT register more than once for the same target resource. the target resource is identified by all the options in the request that are part of the Cache-Key. that includes, for example, the full request URI and the Accept Option 🗂️
3.2. Notifications 🔔
notifications are extra responses the server sends in reply to the single extended GET request that created the registration. every notification includes the token the client put in the request. the only difference between a notification and a normal response is the presence of the Observe Option 🎯
notifications usually come with a 2.05 (Content) response code. they include an Observe Option with a sequence number for reordering detection (see Section 3.4), plus a payload in the same Content-Format as the initial response. if the client put one or more ETag Options in the GET request (see Section 3.3), notifications can use a 2.03 (Valid) response code instead of 2.05 (Content). those ones include an Observe Option with a sequence number but no payload 🫧
and if the resource changes in a way that would make a normal GET request return a non-2.xx response (like when the resource gets deleted), the server sends a notification with an appropriate response code (such as 4.04 Not Found). then it removes the client's entry from the list of observers of the resource. non-2.xx responses do not include an Observe Option 👋
3.3. Caching 🗃️
since notifications are just additional responses to a GET request, they take part in caching as defined in Section 5.6 of RFC 7252 [RFC7252]. both the freshness model and the validation model are supported 💯
3.3.1. Freshness 🥬
a client MAY store a notification like a response in its cache, and use a stored notification that's still fresh without contacting the server 🧊 just like a response, a notification is considered fresh while its age is not greater than the value indicated by the Max-Age Option (and no newer notification or response has been received) ⏱️
the server will do its best to keep the observed state as closely in sync with the actual state as it can. but a client can't rely on observing every single state a resource might go through. for example, if the network is congested or the state changes faster than the network can handle, the server can skip notifications for any number of intermediate states 🚦
the server uses the Max-Age Option to indicate an age up to which it's acceptable for the observed state and the actual state to be inconsistent. if the age of the latest notification becomes greater than its indicated Max-Age, then the client MUST NOT assume that the enclosed representation reflects the actual resource state 🙅♀️
to make sure it has a current representation and/or to re-register its interest in a resource, a client MAY send a new GET request with the same token as the original at any time. all options MUST be identical to those in the original request, except for the set of ETag Options 🔁 it is RECOMMENDED that the client doesn't send the request while it still has a fresh notification or response for the resource in its cache. also, the client SHOULD wait at least a random amount of time between 5 and 15 seconds after Max-Age expires, to reduce collisions with other clients 🎲
3.3.2. Validation 🪪
when a client has one or more notifications stored in its cache for a resource, it can use the ETag Option in the GET request to give the server a chance to select a stored notification to be used ✨
the client MAY include an ETag Option for each stored response that's applicable in the GET request. whenever the observed resource changes to a representation identified by one of those ETag Options, the server can select a stored response by sending a 2.03 (Valid) notification with an appropriate ETag Option, instead of a 2.05 (Content) notification 🫶
a client implementation needs to keep all candidate responses in its cache until it's no longer interested in the target resource, or until it re-registers with a new set of entity tags 🗄️💅
3.4. Reordering 🔀
Notifications can show up in a different order than they were sent 🙃 Since the whole goal is to keep the observed state as close to the actual state as possible, a client MUST consider the notification that was sent most recently as the freshest, regardless of the order it arrived in. Main character energy: the newest one wins, even if it shows up late.
So the server sets the value of the Observe Option in each notification to the 24 least significant bits of a strictly increasing sequence number ✨ That gives the client a way to order them. An incoming notification counts as sent more recently than the freshest notification so far when one of these conditions is met:
(V1 < V2 and V2 - V1 < 2^23) or
(V1 > V2 and V1 - V2 > 2^23) or
(T2 > T1 + 128 seconds)Here V1 is the value of the Observe Option in the freshest notification so far, V2 is the value of the Observe Option in the incoming notification, T1 is a client-local timestamp for the freshest notification so far, and T2 is a client-local timestamp for the incoming notification.
Design Note: The first two conditions check that V1 is less than V2 using 24-bit serial number arithmetic [RFC1982] 🧮 The third condition covers the case where the server builds its serial numbers off a local clock. If enough time has passed between the two messages, the gap between V1 and V2 could get bigger than the largest integer it makes sense to add to a 24-bit serial number. So after 128 seconds with no notification at all, the client doesn't need to dig into the sequence numbers. It can just assume the incoming notification is newer than the freshest one it has so far 💅
Design Note (continued): The 128-second duration was picked as a nice round number that's bigger than MAX_LATENCY (Section 4.8.2 of RFC 7252 [RFC7252]) 🌸
3.5. Transmission 📨
A notification can be confirmable or non-confirmable, which just means it can be sent in a confirmable or a non-confirmable message 💜 The message type used for a notification doesn't depend on the type used for the request, or on the type of any previous notification. Each one is its own thing.
If a client doesn't recognize the token in a confirmable notification, it MUST NOT acknowledge the message and SHOULD reject it with a Reset message 🙅♀️ Otherwise, the client MUST acknowledge the message as usual. For a non-confirmable notification, rejecting the message with a Reset message is OPTIONAL.
An acknowledgement tells the server that the client is still alive and still wants more notifications 🫶 If the server doesn't get an acknowledgement back for a confirmable notification, it'll assume the client isn't interested anymore and will eventually remove that entry from the list of observers (Section 4.5). Ghosted, but make it server-side 😭
3.6. Cancellation 🧹
A client that doesn't care about a resource anymore can just "forget" the observation 👋 When the server sends the next notification, the client won't recognize the token in it, so it'll send back a Reset message. That makes the server remove the matching entry from its list of observers. Basically, the entries in those lists get "garbage collected" by the server 🗑️✨
Implementation Note: Messages can get lost, so the Reset message might never reach the server. The client may end up rejecting several notifications, each with its own Reset message, until the server finally removes the entry and stops sending notifications. Patience, bestie 🐢
Sometimes you want to cancel an observation and free up the server's resources sooner, and that's totally a thing 🦋 In that case, a client MAY explicitly deregister by sending a GET request whose Token field is set to the token of the observation being cancelled, with an Observe Option set to 1 (deregister). All other options MUST be identical to the ones in the registration request, except for the set of ETag Options. When the server gets a request like this, it removes any matching entry from the list of observers and then processes the GET request as usual.
4. Server-Side Requirements 🖥️✨
4.1. Request 💌
A GET request with an Observe Option set to 0 (register) asks the server to do two things: return a current representation of the target resource, and add the client to the list of observers for that resource 🥰 If that works out, the server returns a current representation of the resource and MUST keep this representation updated (as described in Section 1.3) for as long as the client is on the list of observers.
The entry in the list of observers is keyed by the client endpoint and the token the client put in the request. If there's already an entry with a matching endpoint/token pair (which happens, for example, when a client wants to reinforce its interest in a resource 💪), the server MUST NOT add a new entry. It MUST replace or update the existing one instead.
A server that can't or won't add a new entry to the list of observers for a resource MAY silently ignore the registration request and process the GET request as usual 🙈 The resulting response MUST NOT include an Observe Option. Its absence tells the client that it won't be notified of changes to the resource, so it'll need to poll the resource for its state instead 🔁
If the Observe Option in a GET request is set to 1 (deregister), the server MUST remove any existing entry with a matching endpoint/token pair from the list of observers, then process the GET request as usual. The resulting response MUST NOT include an Observe Option.
4.2. Notifications 🔔
A client gets told about changes to the resource state through additional responses the server sends in reply to the GET request ✨ Each notification response, including the initial one, MUST echo the token the client put in the GET request. If there are multiple entries in the list of observers, the order the clients get notified in is not defined. The server can use any method it likes to pick the order 🎲
A notification SHOULD have a 2.05 (Content) or 2.03 (Valid) response code. But if the state of a resource changes in a way that would make a normal GET request return a non-2.xx response at that time (like when the resource gets deleted 💀), the server SHOULD notify the client with an appropriate response code (such as 4.04 Not Found). After that, it MUST remove the associated entry from the list of observers of the resource.
The Content-Format in a 2.xx notification MUST be the same as the one used in the initial response to the GET request. If the server can't keep sending notifications in that format, it SHOULD send a notification with a 4.06 (Not Acceptable) response code, and then it MUST remove the associated entry from the list of observers of the resource 🚫
A 2.xx notification MUST include an Observe Option with a sequence number as specified in Section 4.4 below. A non-2.xx notification MUST NOT include an Observe Option 🎀
4.3. Caching 🗂️
Notifications are just extra responses the server sends in reply to a GET request, so they're subject to caching as defined in Section 5.6 of RFC 7252 [RFC7252] 🧠
4.3.1. Freshness ⏳
After sending the initial response, the server MUST keep the resource state observed by the client as closely in sync with the actual resource state as it possibly can 🎯
Going out of sync sometimes can't be avoided, so the server MUST indicate for each representation an age up to which it's okay for the observed state and the actual state to be inconsistent. That age depends on the application, and it MUST be specified in notifications using the Max-Age Option ⏰
When the resource doesn't change and the client already has a current representation, the server doesn't need to send a notification. But if the client doesn't get one, it can't tell whether the observed state and the actual state are still in sync 😬 So once the age of the latest notification goes past its indicated Max-Age, the client doesn't have a usable representation of the resource state anymore. The server MAY want to prevent that by sending a new notification with the unchanged representation and a new Max-Age, just before the earlier Max-Age runs out 🫧
4.3.2. Validation ✅
A client can include a set of entity tags in its request using the ETag Option 🏷️ When an observed resource changes state and the origin server is about to send a 2.05 (Content) notification, then whenever that notification has an entity tag in the set the client specified, the server MAY send a 2.03 (Valid) response with an appropriate ETag Option instead ✨
4.4. Reordering 🔀
Messages can get reordered, so the client needs a way to figure out whether a notification arrived later than a newer one. For that, the server MUST set the value of the Observe Option in each notification it sends to the 24 least significant bits of a strictly increasing sequence number. The sequence number MAY start at any value and MUST NOT increase so fast that it goes up by more than 2^23 within less than 256 seconds 🚀
The sequence number picked for a notification MUST be greater than that of any earlier notification sent to the same client, with the same token, for the same resource. The value of the Observe Option MUST be current at the time of transmission. If a notification gets retransmitted, the server MUST update the option's value to the sequence number that's current at that time, before the retransmission 🔄
Implementation Note: A simple implementation that meets these requirements is to grab a timestamp from a local clock. The sequence number is then the timestamp in ticks, where 1 tick = (256 seconds)/(2^23) = 30.52 microseconds ⏱️ The clock doesn't have to reflect the current time or date, which is nice.
Implementation Note (continued): Another valid approach is to keep a 24-bit unsigned integer variable per resource, and increment it each time the resource changes state. This works as long as the resource changes state fewer than 2^23 times in the first 256 seconds after every state change. The bonus is that the Observe Option doesn't need updating on retransmission when the resource state didn't change 🎁
Design Note: The 24-bit option value and the 256-second time span theoretically allow for up to 65536 notifications per second. Constrained nodes often have pretty imprecise clocks, though, and the inaccuracies on the client and server sides may cancel out or stack up. Because of that, the maximum notification rate gets reduced to 32768 notifications per second 📉 That's still way above the highest known design objective of around 1 kHz (most CoAP applications sit several orders of magnitude below that), and it allows total clock inaccuracies of up to -50/+100% 💖
4.5. Transmission 💌
okay so a notification can ride in either a confirmable or a non-confirmable message 🫶 which one is usually up to the app, and the server can even pick a different one for each notification 💅
for resources that change in a kinda predictable, regular way, non-confirmable messages are giving the vibe ✨ and for resources that rarely change, confirmable messages are the move 🌸 the server can mix these two approaches depending on how often the state changes and how important each notification is. literally a choose-your-fighter situation 🎀
A server MAY choose to skip sending a notification if it knows another one is coming soon, like when a resource's state is changing super fast 🏃♀️💨 it also MAY choose to send more than one notification for the same resource state. but above all, the server MUST ensure that a client on the list of observers of a resource eventually sees the latest state if the resource doesn't go through a new change in state 😭🫶 this is the most important rule in the whole section, bestie, don't forget it 👑
for example, when state changes come in bursts, the server can skip some notifications 🌪️ send the rest in non-confirmable messages, and then make sure the client sees the latest change by repeating the last notification in a confirmable message once the burst is over 🦋 very organized, very iconic
the client's acknowledgement of a confirmable notification tells the server the client still wants more notifications 💖 but if a client rejects a confirmable or non-confirmable notification with a Reset message, or if the last retransmission attempt of a confirmable notification times out, then the client is considered no longer interested 💀 and the server MUST remove that entry from the list of observers. ghosted, periodt 🙈
Implementation Note: to properly process aResetmessage that rejects a non-confirmable notification, a server needs to remember the message IDs of the non-confirmable notifications it sends. this may be challenging for a server with constrained resources 😩 however, sinceResetmessages are sent unreliably, the client has to be ready in case theResetmessages never reach the server 🥹 so a server can always pretend that aResetmessage rejecting a non-confirmable notification got lost. if a server does this, it could speed up cancellation by sending the following notifications to that client in confirmable messages 🚀
a server that sends notifications mostly in non-confirmable messages MUST send a notification in a confirmable message instead of a non-confirmable one at least every 24 hours 🕰️ this stops a client that left or stopped caring from hanging around on the list of observers forever. no lingering exes allowed 💅
4.5.1. Congestion Control 🚦
basic congestion control for CoAP comes from the exponential back-off mechanism in Section 4.2 of RFC 7252 [RFC7252] and the limits in Section 4.7 of RFC 7252 [RFC7252]. but CoAP only puts the congestion control responsibility on the clients for simple request/response interactions: rate-limiting request transmission implicitly controls the transmission of the responses 🧠 when one single request can yield a potentially infinite number of notifications, the server needs to take on some of that responsibility too 🫡
in order not to cause congestion, servers MUST strictly limit the number of simultaneous outstanding notifications/responses they transmit to a given client to NSTART (1 by default; see Section 4.7 of RFC 7252 [RFC7252]) 🛑 an outstanding notification/response is either a confirmable message that hasn't been acknowledged yet and whose last retransmission attempt hasn't timed out, or a non-confirmable message whose waiting time from the following rate-limiting rules hasn't elapsed yet ⏳
the server SHOULD NOT send more than one non-confirmable notification per round-trip time (RTT) to a client on average 🌊 if the server can't keep an RTT estimate for a client, it SHOULD NOT send more than one non-confirmable notification every 3 seconds, and it SHOULD use an even less aggressive rate when possible (see also Section 3.1.2 of RFC 5405 [RFC5405]) 🐢 be chill, be gentle 🫶
further congestion control optimizations and considerations are expected in the future with advanced CoAP congestion control mechanisms 🔮 stay tuned, main character era incoming ✨
4.5.2. Advanced Transmission 🏎️
the state of an observed resource can change while the number of simultaneous outstanding notifications/responses to a client on the list of observers is greater than or equal to NSTART 😤 in that case, the server can't notify the client of the new state right away and has to wait for an outstanding notification/response to finish first. rude, but it is what it is 🙄
if there's an outstanding notification/response that the server is sending to the client and it's about the changed resource, then it's desirable for the server to stop working toward getting the representation of the old state to the client and start sending the current representation instead 💫 that way, the state the client sees stays closer in sync with what's actually happening at the server 🎯
for this purpose, the server MAY optimize the transmission process by aborting the transmission of the old notification (but not before the current transmission attempt is completed) and starting a new transmission for the new notification (but with the retransmission timer and counter of the aborted transmission kept) ✂️🩷
more specifically, a server MAY supersede an outstanding transmission that pertains to an observation like this:
- wait for the current (re)transmission attempt to be acknowledged, rejected, or to time out (confirmable transmission); or, wait for the waiting time to elapse or the transmission to be rejected (non-confirmable transmission).
- if the transmission is rejected or it was the last attempt to retransmit a notification, remove the associated entry from the list of observers of the observed resource 👋
- if the entry is still on the list of observers, start transmitting a new notification with a representation of the current resource state. if the resource changed its state more than once in the meantime, the notifications for the in-between states get quietly skipped 🤫
- the new notification is sent with a new Message ID and the following transmission parameters: if the previous (re)transmission attempt timed out, keep its transmission parameters, increment the retransmission counter, and double the timeout; otherwise, initialize the transmission parameters as usual (see Section 4.2 of RFC 7252 [RFC7252]) 🧮
it's possible the server later gets an acknowledgement for a confirmable notification it superseded this way 🥰 even though that doesn't prove the client is up to date, it's still valuable because it shows the client is still interested in the resource. so the server should avoid accidentally removing that entry from the list of observers 💜
5. Intermediaries 🧚
a client may be interested in a resource in the namespace of a server that's reached through a chain of one or more CoAP intermediaries 🌐 in that case, the client registers its interest with the first intermediary toward the server, acting like it's talking to the server itself, as specified in Section 3 💁♀️ it's then the intermediary's job to give the client a current representation of the target resource and keep it updated whenever the resource state changes, as specified in Section 4 🛠️
to do this job, the intermediary SHOULD use the protocol specified in this document, playing the client role and registering its own interest in the target resource with the next hop toward the server 🫶 if the response from the next hop doesn't include an Observe Option, the intermediary MAY resort to polling the next hop or MAY itself return a response without an Observe Option 🔁
the communication between each pair of hops is independent 🧩 each hop in the server role MUST decide on its own how many notifications to send, which message type to use, and so on 🙌 each hop MUST generate its own values for the Observe Option in notifications and MUST set the value of the Max-Age Option according to the age of its local current representation ⏰
if two or more clients have registered interest in a resource with an intermediary, the intermediary MUST register itself only once with the next hop and fan out the notifications it receives to all registered clients 📣 this saves the next hop from sending the same notifications over and over, which is what makes it scale 📈🌸
an intermediary isn't required to act on behalf of a client to observe a resource; an intermediary MAY observe a resource, for example, just to keep its own cache fresh 🧊
See Appendix A.2 for examples. 📖
6. Web Linking 🔗
a web link [RFC5988] to a resource accessible over CoAP (for example, in a link-format document [RFC6690]) MAY include the target attribute "obs" 🎀
the "obs" attribute, when present, is a hint that the link's destination is useful for observation, so it should get a nice graphical representation in a user interface, for example 🖼️✨ but this is only a hint, not a promise that the Observe Option can actually be used to do the observation 🙃 a client may need to fall back to polling the resource if the Observe Option isn't returned in the response to the GET request 🔄
a value MUST NOT be given for the "obs" attribute; any value that's present MUST be ignored by parsers 🚫 the "obs" attribute MUST NOT appear more than once in a given link-value; any occurrences after the first MUST be ignored by parsers 🙅♀️💅
7. Security Considerations 🔐
Okay bestie, the security chapter from CoAP itself still applies here, so go read Section 11 of [RFC7252] too 🔒
Observing resources can absolutely make amplification attacks worse 😬 Notifications can be way bigger than the request that set them up, and the way this protocol works can trigger a huge pile of notifications. So without client authentication, a server MUST strictly limit how many notifications it sends between receiving acknowledgements that confirm the client's actual interest in the data. Basically, any notifications sent in non-confirmable messages MUST be interspersed with confirmable messages 🛡️ Also, an attacker may still fake those acknowledgements if the confirmable messages are predictable enough, so keep that in mind 👀
The protocol is best-effort when it comes to keeping the state a client observes in sync with the actual resource state at the server 🫠 That means the client and server can get out of sync sometimes. Depending on how sensitive the observed resource is, working with old state might be a real security threat 💀 So the client must be careful not to use a representation after its Max-Age expires, and the server must set the Max-Age Option to a sensible value ⏰
Also, attackers may try to drain the resources the server has for keeping track of its observers for each resource 🧠 Servers may want to put access controls on creating that state 🚪 As a degraded fallback, the server can always just process the request as a normal GET request (without an Observe Option) if it's unwilling or unable to add a client to the list of observers of a resource. That includes when system resources are exhausted or getting close to exhausted 🫶
Intermediaries must be careful to make sure notifications can't be used to create a loop 🔁 One simple way to break any loops is to use caches for forwarding notifications in intermediaries 💅
Resources can be observed over CoAP that's secured by Datagram Transport Layer Security (DTLS), using any of the security modes described in Section 9 of RFC 7252 🔐 DTLS is indicated by the "coaps" URI scheme. Lastly, and this one is a MUST, all notifications resulting from a GET request with an Observe Option MUST be returned within the same epoch of the same connection as the request 🌸
8. IANA Considerations 📋
The following entry has been added to the CoAP Option Numbers registry 🎀
+--------+---------+-----------+
| Number | Name | Reference |
+--------+---------+-----------+
| 6 | Observe | RFC 7641 |
+--------+---------+-----------+9. References 📚
9.1. Normative References
These are the must-read ones, the ones the spec leans on hardest 🤓
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<http://www.rfc-editor.org/info/rfc2119>.
[RFC5988] Nottingham, M., "Web Linking", RFC 5988,
DOI 10.17487/RFC5988, October 2010,
<http://www.rfc-editor.org/info/rfc5988>.
[RFC7252] Shelby, Z., Hartke, K., and C. Bormann, "The Constrained
Application Protocol (CoAP)", RFC 7252,
DOI 10.17487/RFC7252, June 2014,
<http://www.rfc-editor.org/info/rfc7252>.9.2. Informative References
These are the background reads, good for going deeper 🧁
[GOF] Gamma, E., Helm, R., Johnson, R., and J. Vlissides,
"Design Patterns: Elements of Reusable Object-Oriented
Software", Addison-Wesley Professional Computing Series,
1994.
[REST] Fielding, R., "Architectural Styles and the Design of
Network-based Software Architectures", Ph.D. Dissertation,
University of California, Irvine, 2000,
<http://www.ics.uci.edu/~fielding/pubs/dissertation/
fielding_dissertation.pdf>.
[RFC1982] Elz, R. and R. Bush, "Serial Number Arithmetic", RFC 1982,
DOI 10.17487/RFC1982, August 1996,
<http://www.rfc-editor.org/info/rfc1982>.
[RFC5405] Eggert, L. and G. Fairhurst, "Unicast UDP Usage Guidelines
for Application Designers", BCP 145, RFC 5405,
DOI 10.17487/RFC5405, November 2008,
<http://www.rfc-editor.org/info/rfc5405>.
[RFC5989] Roach, A., "A SIP Event Package for Subscribing to Changes
to an HTTP Resource", RFC 5989, DOI 10.17487/RFC5989,
October 2010, <http://www.rfc-editor.org/info/rfc5989>.
[RFC6202] Loreto, S., Saint-Andre, P., Salsano, S., and G. Wilkins,
"Known Issues and Best Practices for the Use of Long
Polling and Streaming in Bidirectional HTTP", RFC 6202,
DOI 10.17487/RFC6202, April 2011,
<http://www.rfc-editor.org/info/rfc6202>.
[RFC6690] Shelby, Z., "Constrained RESTful Environments (CoRE) Link
Format", RFC 6690, DOI 10.17487/RFC6690, August 2012,
<http://www.rfc-editor.org/info/rfc6690>.
[RFC7228] Bormann, C., Ersue, M., and A. Keranen, "Terminology for
Constrained-Node Networks", RFC 7228,
DOI 10.17487/RFC7228, May 2014,
<http://www.rfc-editor.org/info/rfc7228>.
[RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer
Protocol (HTTP/1.1): Message Syntax and Routing",
RFC 7230, DOI 10.17487/RFC7230, June 2014,
<http://www.rfc-editor.org/info/rfc7230>.Appendix A. Examples 🧚
A.1. Client/Server Examples
Okay, this is where it all clicks 🦋 Here's a CoAP Observe conversation, frame by frame. The left column is what the client thinks the temperature is, the middle is the actual messages, and the right column is what's really happening on the server 💅
Observed CLIENT SERVER Actual
t State | | State
____________ | | ____________
1 | |
2 unknown | | 18.5 Cel
3 +----->| Header: GET 0x41011633
4 | GET | Token: 0x4a
5 | | Uri-Path: temperature
6 | | Observe: 0 (register)
7 | |
8 | |
9 ____________ |<-----+ Header: 2.05 0x61451633
10 | 2.05 | Token: 0x4a
11 18.5 Cel | | Observe: 9
12 | | Max-Age: 15
13 | | Payload: "18.5 Cel"
14 | |
15 | | ____________
16 ____________ |<-----+ Header: 2.05 0x51457b50
17 | 2.05 | 19.2 Cel Token: 0x4a
18 19.2 Cel | | Observe: 16
29 | | Max-Age: 15
20 | | Payload: "19.2 Cel"
21 | | Figure 3: A Client Registers and Receives One Notification of the
Current State and One of a New State upon a State Change Observed CLIENT SERVER Actual
t State | | State
____________ | | ____________
22 | |
23 19.2 Cel | | 19.2 Cel
24 | | ____________
25 | X----+ Header: 2.05 0x51457b51
26 | 2.05 | 19.7 Cel Token: 0x4a
27 | | Observe: 25
28 | | Max-Age: 15
29 | | Payload: "19.7 Cel"
30 | |
31 ____________ | |
32 | |
33 19.2 Cel | |
34 (stale) | |
35 | |
36 | |
37 | |
38 +----->| Header: GET 0x41011634
39 | GET | Token: 0xb2
40 | | Uri-Path: temperature
41 | | Observe: 0 (register)
42 | |
43 | |
44 ____________ |<-----+ Header: 2.05 0x61451634
45 | 2.05 | Token: 0xb2
46 19.7 Cel | | Observe: 44
47 | | Max-Age: 15
48 | | ETag: 0x78797a7a79
49 | | Payload: "19.7 Cel"
50 | | Figure 4: The Client Re-registers after Max-Age Ends Observed CLIENT SERVER Actual
t State | | State
____________ | | ____________
51 | |
52 19.7 Cel | | 19.7 Cel
53 | |
54 | | ____________
55 | crash
56 |
57 |
58 |
59 ____________ |
60 |
61 19.7 Cel |
62 (stale) |
63 | reboot____________
64 | |
65 | | 20.0 Cel
66 | |
67 +----->| Header: GET 0x41011635
68 | GET | Token: 0xf9
69 | | Uri-Path: temperature
70 | | Observe: 0 (register)
71 | | ETag: 0x78797a7a79
72 | |
73 | |
74 ____________ |<-----+ Header: 2.05 0x61451635
75 | 2.05 | Token: 0xf9
76 20.0 Cel | | Observe: 74
77 | | Max-Age: 15
78 | | Payload: "20.0 Cel"
79 | |
80 | | ____________
81 ____________ |<-----+ Header: 2.03 0x5143aa0c
82 | 2.03 | 19.7 Cel Token: 0xf9
83 19.7 Cel | | Observe: 81
84 | | ETag: 0x78797a7a79
85 | | Max-Age: 15
86 | | Figure 5: The Client Re-registers and Gives the Server the
Opportunity to Select a Stored Response Observed CLIENT SERVER Actual
t State | | State
____________ | | ____________
87 | |
88 19.7 Cel | | 19.7 Cel
89 | |
90 | | ____________
91 ____________ |<-----+ Header: 2.05 0x4145aa0f
92 | 2.05 | 19.3 Cel Token: 0xf9
93 19.3 Cel | | Observe: 91
94 | | Max-Age: 15
95 | | Payload: "19.3 Cel"
96 | |
97 | |
98 +- - ->| Header: 0x7000aa0f
99 | |
100 | |
101 | |
102 | | ____________
103 | |
104 | | 19.0 Cel
105 | |
106 ____________ | |
107 | |
108 19.3 Cel | |
109 (stale) | |
110 | | Figure 6: The Client Rejects a Notification and Thereby Cancels the
ObservationA.2. Proxy Examples
Now the fun part: a proxy sitting in the middle, keeping its cache fresh and passing the observation along 🧚 Bestie, this is so iconic 😭
CLIENT PROXY SERVER
| | |
| +----->| Header: GET 0x41015fb8
| | GET | Token: 0x1a
| | | Uri-Host: sensor.example
| | | Uri-Path: status
| | | Observe: 0 (register)
| | |
| |<-----+ Header: 2.05 0x61455fb8
| | 2.05 | Token: 0x1a
| | | Observe: 42
| | | Max-Age: 60
| | | Payload: "ready"
| | |
+----->| | Header: GET 0x41011633
| GET | | Token: 0x9a
| | | Proxy-Uri: coap://sensor.example/status
| | |
|<-----+ | Header: 2.05 0x61451633
| 2.05 | | Token: 0x9a
| | | Max-Age: 53
| | | Payload: "ready"
| | |
| |<-----+ Header: 2.05 0x514505fc0
| | 2.05 | Token: 0x1a
| | | Observe: 135
| | | Max-Age: 60
| | | Payload: "busy"
| | |
+----->| | Header: GET 0x41011634
| GET | | Token: 0x9b
| | | Proxy-Uri: coap://sensor.example/status
| | |
|<-----+ | Header: 2.05 0x61451634
| 2.05 | | Token: 0x9b
| | | Max-Age: 49
| | | Payload: "busy"
| | | Figure 7: A Proxy Observes a Resource to Keep its Cache Up to Date CLIENT PROXY SERVER
| | |
+----->| | Header: GET 0x41011635
| GET | | Token: 0x6a
| | | Proxy-Uri: coap://sensor.example/status
| | | Observe: 0 (register)
| | |
|<- - -+ | Header: 0x60001635
| | |
| +----->| Header: GET 0x4101af90
| | GET | Token: 0xaa
| | | Uri-Host: sensor.example
| | | Uri-Path: status
| | | Observe: 0 (register)
| | |
| |<-----+ Header: 2.05 0x6145af90
| | 2.05 | Token: 0xaa
| | | Observe: 67
| | | Max-Age: 60
| | | Payload: "ready"
| | |
|<-----+ | Header: 2.05 0x4145af94
| 2.05 | | Token: 0x6a
| | | Observe: 17346
| | | Max-Age: 60
| | | Payload: "ready"
| | |
+- - ->| | Header: 0x6000af94
| | |
| |<-----+ Header: 2.05 0x51455a20
| | 2.05 | Token: 0xaa
| | | Observe: 157
| | | Max-Age: 60
| | | Payload: "busy"
| | |
|<-----+ | Header: 2.05 0x5145af9b
| 2.05 | | Token: 0x6a
| | | Observe: 17436
| | | Max-Age: 60
| | | Payload: "busy"
| | | Figure 8: A Client Observes a Resource through a ProxyAcknowledgements 🎀
Big love to Carsten Bormann, who was an original author of this doc and gets major credit for significant contributions 👑
Thanks a ton to Daniele Alessandrelli, Jari Arkko, Peter A. Bigot, Angelo P. Castellani, Gilbert Clark, Esko Dijk, Thomas Fossati, Brian Frank, Bert Greevenbosch, Jeroen Hoebeke, Cullen Jennings, Matthias Kovatsch, Barry Leiba, Salvatore Loreto, Charles Palmer, Akbar Rahman, Zach Shelby, and Floris Van den Abeele for helpful comments and discussions that have shaped the document 🫶
This work was supported in part by Klaus Tschira Foundation, Intel, Cisco, and Nokia. 💜
Author's Address 💌
Klaus Hartke
Universitaet Bremen TZI
Postfach 330440
Bremen D-28359
Germany
Phone: +49-421-218-63905
Email: hartke@tzi.org📦 where this page comes from
- ietf: rfc7641 2015-09
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms