RFC 8895
Application-Layer Traffic Optimization (ALTO) Incremental Updates Using Server-Sent Events (SSE)
Title Block 🎀
RFC 8895, Application-Layer Traffic Optimization (ALTO) Incremental Updates Using Server-Sent Events (SSE) 💖
- Authors: W. Roome (Nokia Bell Labs) and Y. Yang (Yale University) 🧑💻
- Category: Standards Track ✨
- ISSN: 2070-1721
- Date: November 2020 📅
- Obsoletes / Updates: the title block doesn't list any documents this one obsoletes or updates 🙈
The IETF wrote this one, so yes, it's an official Internet Standards Track document 🌈
Abstract 🦋
okay so the Application-Layer Traffic Optimization (ALTO) protocol (RFC 7285) gives client apps network information resources, which is basically network info that helps them make smart choices about using network resources 🧠 this doc adds a way for an ALTO server to push updates to clients, and it's giving two big perks 💅
- updates can be incremental 🧩 so if only a tiny section of a resource changes, the server just sends the changes
- updates can be immediate ⚡ so the server sends them the moment they're ready
Status of This Memo 🌸
This is an Internet Standards Track document 💜
This document is a product of the Internet Engineering Task Force (IETF) 🫶 it represents the consensus of the IETF community, got public review, and was approved for publication by the Internet Engineering Steering Group (IESG) 👑 if you want more on Internet Standards, check out Section 2 of RFC 7841 📚
Info about the current status of this doc, any errata, and how to give feedback lives at:
https://www.rfc-editor.org/info/rfc8895
Copyright Notice 🌈
Copyright (c) 2020 IETF Trust and the persons identified as the document authors. All rights reserved. 🔒
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents, which you can find at https://trustee.ietf.org/license-info (as in effect on the date of publication) 📜 please actually read these, bestie, because they explain your rights and restrictions 🙏 also, any Code Components pulled out of this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come without warranty as described in the Simplified BSD License ⚠️
Table of Contents 🧚
1. Introduction
2. Terms
2.1. Requirements Language
3. Background
3.1. Incremental Encoding: JSON Merge Patch
3.1.1. JSON Merge Patch Encoding
3.1.2. JSON Merge Patch ALTO Messages
3.2. Incremental Encoding: JSON Patch
3.2.1. JSON Patch Encoding
3.2.2. JSON Patch ALTO Messages
3.3. Multiplexing and Server Push: HTTP/2
3.4. Server Push: Server-Sent Event
4. Overview of Approach and High-Level Protocol Message Flow
4.1. Update Stream Service Message Flow
4.2. Stream Control Service Message Flow
4.3. Service Announcement and Management Message Flow
5. Update Messages: Data Update and Control Update Messages
5.1. Generic ALTO Update Message Structure
5.2. ALTO Data Update Message
5.3. ALTO Control Update Message
6. Update Stream Service
6.1. Media Type
6.2. HTTP Method
6.3. Capabilities
6.4. Uses
6.5. Request: Accept Input Parameters
6.6. Response
6.7. Additional Requirements on Update Stream Service
6.7.1. Event Sequence Requirements
6.7.2. Cross-Stream Consistency Requirements
6.7.3. Multipart Update Requirements
6.8. Keep-Alive Messages
7. Stream Control Service
7.1. URI
7.2. Media Type
7.3. HTTP Method
7.4. IRD Capabilities & Uses
7.5. Request: Accept Input Parameters
7.6. Response
8. Examples
8.1. Example: IRD Announcing Update Stream Services
8.2. Example: Simple Network and Cost Map Updates
8.3. Example: Advanced Network and Cost Map Updates
8.4. Example: Endpoint Property Updates
8.5. Example: Multipart Message Updates
9. Operation and Processing Considerations
9.1. Considerations for Choosing Data Update Messages
9.2. Considerations for Client Processing Data Update Messages
9.3. Considerations for Updates to Filtered Cost Maps
9.4. Considerations for Updates to Ordinal Mode Costs
9.5. Considerations for SSE Text Formatting and Processing
10. Security Considerations
10.1. Update Stream Server: Denial-of-Service Attacks
10.2. ALTO Client: Update Overloading or Instability
10.3. Stream Control: Spoofed Control Requests and Information
Breakdown
11. Requirements on Future ALTO Services to Use This Design
12. IANA Considerations
12.1. application/alto-updatestreamparams+json Media Type
12.2. application/alto-updatestreamcontrol+json Media Type
13. Appendix: Design Decision: Not Allowing Stream Restart
14. References
14.1. Normative References
14.2. Informative References
Acknowledgments
Contributors
Authors' Addresses1. Introduction ✨
okay bestie, so ALTO (that's [RFC7285]) gives client apps network information resources so they can make informed decisions about using network resources 💅 for example, an ALTO server provides network and cost maps 🗺️ a network map splits the endpoints into a manageable number of sets, and each set gets a Provider-Defined Identifier (PID) 🏷️ a cost map gives directed costs between PIDs 💸 so a client can work out the cost between two endpoints by first using the network map to find each endpoint's PID, then using the cost map to look up the costs between those PIDs 🧠 and the client can use those costs to pick communicating endpoints with low network costs 🎯 the way that's so smart 😭
here's the tea though: ALTO only has a client pull model 🫠 there's no built-in way for a client to get updates to network information resources, except by re-fetching them over and over 🔁 and if a resource is big but only some parts change a lot (like some entries in a cost map), re-fetching the whole thing is super inefficient 💀
so this doc introduces a way for an ALTO server to push incremental updates to clients 🦋 putting server push and incremental updates together gets you two perks: (1) updates can be small, since if only a small section changes, the server just sends the changes, and (2) updates can be immediate, since the server sends them as soon as they're available ⚡
this is mainly for GET-mode network and cost maps, but the same mechanism can also update POST-mode ALTO services, like the ALTO endpoint property and endpoint cost services 💜 it can even support brand-new ALTO services from future extensions, but a future service has to satisfy the requirements in Section 11 🧚
quick roadmap, iykyk 🗺️
- Section 3 gives background on the basic techniques: (1) JSON merge patch and JSON patch for incremental updates, and (2) Server-Sent Events (SSE) [SSE] for server push 🧩
- Section 4 gives a non-normative overview of the design 🌷
- Section 5 defines the individual messages in an update stream 📨
- Section 6 defines the update stream service 🌊
- Section 7 defines the stream control service 🎛️
- Section 8 gives several examples for both types of services 🍓
- Section 9 covers operation and processing considerations for both ALTO servers and clients 🛠️
- Section 13 discusses a design feature that is not supported 🙅♀️
- Section 10 discusses security issues 🔐 (dramatic, yes, but important!)
- Sections 11 and 12 cover the requirements for future ALTO services to use SSE, and the IANA considerations, respectively 📋
part 1 is wrapping up here, but part 2 is coming with Section 2 (Terms) and Section 3 (Background) 🫶✨
2. Terms ✨
okay so before we dive in, bestie, there are some extra terms this doc uses on top of the ones from [RFC7285] 💅 here they are:
- Update Stream: a reliable, in-order connection that works with HTTP/1.x between an ALTO client and an ALTO server, so the server can push a sequence of update messages to the client using [SSE] 🦋
- Update Stream Server: an ALTO server that provides an update stream. This doc calls it an ALTO update stream server, or update stream server for short. The ALTO server in this doc is a general server that offers all kinds of services. It can be an update stream server or a stream control server (more on that below), and it can also be a server that provides the ALTO Information Resource Directory (IRD) 🌸
- Update Message: a message that is either a data update message or a control update message 💜
- Data Update Message: an update message for a single ALTO information resource. The update stream server sends it to the ALTO client whenever that resource changes. A data update message is either a full-replacement message or an incremental-change message. "Full replacement" is shorthand for a full-replacement message, and "incremental change" is shorthand for an incremental-change message 🎀
- Full Replacement: a data update message for a resource that encodes the content of the resource in its original ALTO encoding 🍓
- Incremental Change: a data update message that only specifies the difference between the new content and the previous version. In this doc, an incremental change can be encoded using either JSON merge patch or JSON patch 🧚
- Stream Control Service: a service that gives the ALTO client of an update stream an HTTP URI. The client uses it to send stream control requests to the ALTO server when resources get added to or removed from the update messages it receives. The ALTO server creates a new stream control resource for each update stream instance, gives it a unique URI, and sends that URI to the client as the first event in the stream. (Heads up: the stream control service in ALTO has nothing to do with the similarly named Stream Control Transmission Protocol [RFC4960], so don't mix them up 🙈)
- Stream Control: shorthand for stream control service 🌈
- Stream Control Server: an ALTO server that provides the stream control service ⭐
- Substream-ID: an ALTO client can assign a unique substream-id when it asks for a resource to be added to the update messages from an update stream. The server puts the substream-id in each update event for that resource. This lets a client use one update stream to get updates for multiple requests for the same resource (meaning the same resource-id in an ALTO IRD). For example, that works for a POST-mode resource with different input parameters 👑
- Data-ID: a subfield of the "event" field of [SSE] that identifies the ALTO data (object) to be updated. For an ALTO resource that returns a multipart response, the data-id that identifies the data (object) is the substream-id, plus the Content-ID of the object in the multipart response. The data-id of a single-part response is just the substream-id 🫶
- Control Update Message: an update message that tells the ALTO client about control information for the update stream. The update stream server sends it to notify the client. It can be triggered by something internal to the server, like server overloading, which means the update stream server will stop sending updates for an information resource. It can also happen because a client sent a request through the stream control service. The first message of an update stream is a control update message that gives the ALTO client a control URI. The client can use that URI to send stream control requests to the stream control server 💖
2.1. Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be read as described in BCP 14 [RFC2119] [RFC8174], but only when they show up in all capitals, just like they do here 🔥
3. Background
okay this part is the foundation, so pay attention 🧠 the design needs two basic techniques: encoding incremental changes, and server push. For incremental changes, the existing techniques are JSON merge patch and JSON patch, and this design uses both. For server push, the existing techniques are HTTP/2 and [SSE]. This design borrows some features from HTTP/2, but it uses [SSE] as the basic server-push design. The rest of this section gives a non-normative summary of JSON merge patch, JSON patch, HTTP/2, and [SSE] ✨
3.1. Incremental Encoding: JSON Merge Patch
so the server doesn't have to keep sending complete data every time, it needs a way to encode incremental changes, and JSON merge patch is one of those ways. [RFC7396] defines the encoding of incremental changes (called JSON merge patch objects) that's meant to be used by the HTTP PATCH method [RFC5789]. This document only adopts the JSON merge patch object encoding. It doesn't use the HTTP PATCH method, since the updates get sent as events instead of HTTP methods. Also, the updates flow from server to client, and PATCH semantics are more about client to server 🦋 Below is a non-normative summary of JSON merge patch objects. For the normative definition, check out [RFC7396] 💜
3.1.1. JSON Merge Patch Encoding
basically, a JSON merge patch message is a JSON merge patch object (called a patch in [RFC7396]). It describes how to turn one JSON value into another using a recursive merge patch algorithm 🌸 Here's how it works. You treat the two JSON values (the original first, then the updated one) as trees of nested JSON objects, which are dictionaries of name/value pairs. The leaves are the values that aren't JSON objects, like arrays, strings, and numbers. The path to each leaf is the sequence of keys that leads to it. When the second tree has a different value for a leaf at some path, or adds a new leaf, the patch gets a leaf at that path with the new value. When a leaf in the first tree doesn't exist in the second tree, the JSON merge patch tree gets a leaf with a JSON "null" value. So in the patch, null as the value of a name/value pair deletes the element with that "name" from the original JSON value 🍓 The patch has no entry for any leaf that has the same value in both versions. For the formal way to apply a patch, look at the MergePatch pseudocode at the beginning of Section 2 of [RFC7396]. Here's the tea: when all the leaf values are simple scalars, JSON merge patch is a really efficient way to represent incremental changes 😭 It's less efficient when leaf values are arrays, because JSON merge patch replaces the whole array even if only one entry changed 💀
3.1.2. JSON Merge Patch ALTO Messages
to show you JSON merge patch in action, and to prove it actually works for ALTO, the sections below apply it to two key ALTO messages 🎀
3.1.2.1. JSON Merge Patch Network Map Messages
Section 11.2.1.6 of [RFC7285] defines the format of an ALTO network map message. Let's say we start with a simple example ALTO message that sends an initial network map:
{
"meta" : {
"vtag": {
"resource-id" : "my-network-map",
"tag" : "da65eca2eb7a10ce8b059740b0b2e3f8eb1d4785"
}
},
"network-map" : {
"PID1" : {
"ipv4" : [ "192.0.2.0/24", "198.51.100.0/25" ]
},
"PID2" : {
"ipv4" : [ "198.51.100.128/25" ]
},
"PID3" : {
"ipv4" : [ "0.0.0.0/0" ],
"ipv6" : [ "::/0" ]
}
}
}Now check out this JSON merge patch update message. It (1) adds an ipv4 prefix "203.0.113.0/25" and an ipv6 prefix "2001:db8:8000::/33" to "PID1", (2) deletes "PID2", and (3) gives the network map a new "tag" 🌈
{
"meta" : {
"vtag" : {
"tag" : "a10ce8b059740b0b2e3f8eb1d4785acd42231bfe"
}
},
"network-map": {
"PID1" : {
"ipv4" : [ "192.0.2.0/24", "198.51.100.0/25",
"203.0.113.0/25" ],
"ipv6" : [ "2001:db8:8000::/33" ]
},
"PID2" : null
}
}Applying the JSON merge patch update to the initial network map gives you this ALTO network map, which is equivalent to it:
{
"meta" : {
"vtag": {
"resource-id" : "my-network-map",
"tag" : "a10ce8b059740b0b2e3f8eb1d4785acd42231bfe"
}
},
"network-map" : {
"PID1" : {
"ipv4" : [ "192.0.2.0/24", "198.51.100.0/25",
"203.0.113.0/25" ],
"ipv6" : [ "2001:db8:8000::/33" ]
},
"PID3" : {
"ipv4" : [ "0.0.0.0/0" ],
"ipv6" : [ "::/0" ]
}
}
}3.1.2.2. JSON Merge Patch Cost Map Messages
Section 11.2.3.6 of [RFC7285] defines the format of an ALTO cost map message. Here's a simple example ALTO message for an initial cost map 💖
{
"meta" : {
"dependent-vtags" : [
{"resource-id": "my-network-map",
"tag": "a10ce8b059740b0b2e3f8eb1d4785acd42231bfe"
}
],
"cost-type" : {
"cost-mode" : "numerical",
"cost-metric": "routingcost"
},
"vtag": {
"resource-id" : "my-cost-map",
"tag" : "3ee2cb7e8d63d9fab71b9b34cbf764436315542e"
}
},
"cost-map" : {
"PID1": { "PID1": 1, "PID2": 5, "PID3": 10 },
"PID2": { "PID1": 5, "PID2": 1, "PID3": 15 },
"PID3": { "PID1": 20, "PID2": 15 }
}
}This next JSON merge patch message updates the example cost map. It (1) updates the "tag" field of the cost map, (2) changes the cost of PID1->PID2 to 9 instead of 5, (3) makes the cost of PID3->PID1 no longer available, and (4) defines the cost of PID3->PID3 as 1 👑
{
"meta" : {
"vtag": {
"tag": "c0ce023b8678a7b9ec00324673b98e54656d1f6d"
}
}
"cost-map" : {
"PID1" : { "PID2" : 9 },
"PID3" : { "PID1" : null, "PID3" : 1 }
}
}So applying the JSON merge patch to the initial cost map gives you this ALTO cost map, which is equivalent to it:
{
"meta" : {
"dependent-vtags" : [
{"resource-id": "my-network-map",
"tag": "a10ce8b059740b0b2e3f8eb1d4785acd42231bfe"
}
],
"cost-type" : {
"cost-mode" : "numerical",
"cost-metric": "routingcost"
},
"vtag": {
"resource-id": "my-cost-map",
"tag": "c0ce023b8678a7b9ec00324673b98e54656d1f6d"
}
},
"cost-map" : {
"PID1": { "PID1": 1, "PID2": 9, "PID3": 10 },
"PID2": { "PID1": 5, "PID2": 1, "PID3": 15 },
"PID3": { "PID2": 15, "PID3": 1 }
}
}3.2. Incremental Encoding: JSON Patch
3.2.1. JSON Patch Encoding ✨
okay so JSON merge patch has a glow-up problem 💅 It's bad at arrays. It treats the whole array as ONE object, so the only thing it can do is replace the entire array. Changing one tiny thing, like deleting one element from a huge array, means resending the whole thing, which is so inefficient 😭 Look at the example in Section 3.1.2.1. To add one new entry to the ipv4 array for PID1, the server has to send a whole brand-new array. not it.
There's another issue too. JSON merge patch can't set a value to null, because its processing algorithm (MergePatch in Section 3.1.1) reads a null as "remove this" 🙈 But some ALTO resources can legitimately have null values, so an update might want to change something to null. Main character problem.
JSON patch [RFC6902] fixes all of this 🦋 It defines a set of operators for modifying a JSON object. For the normative definition, go to [RFC6902], bestie.
3.2.2. JSON Patch ALTO Messages 🎀
To show you both how JSON patch works and how it's different from JSON merge patch, the next sections apply JSON patch to the same updates we saw in Section 3.1.2. Same drama, new outfit 👑
3.2.2.1. JSON Patch Network Map Messages 🌸
First up, the same network map update from Section 3.1.2.1. Here's how it looks encoded with JSON patch:
[
{
"op": "replace",
"path": "/meta/vtag/tag",
"value": "a10ce8b059740b0b2e3f8eb1d4785acd42231bfe"
},
{
"op": "add",
"path": "/network-map/PID1/ipv4/2",
"value": "203.0.113.0/25"
}
{
"op": "add",
"path": "/network-map/PID1/ipv6",
"value": ["2001:db8:8000::/33"]
},
{
"op": "remove",
"path": "/network-map/PID2"
}
]3.2.2.2. JSON Patch Cost Map Messages 🧚
Compared with JSON merge patch, JSON patch is NOT as efficient for cost map updates 🥲 Let's look at the cost map update from Section 3.1.2.2. Encoded with JSON patch, it becomes:
[
{
"op": "replace",
"path": "/meta/vtag/tag",
"value": "c0ce023b8678a7b9ec00324673b98e54656d1f6d"
},
{
"op": "replace",
"path": "/cost-map/PID1/PID2",
"value": 9
},
{
"op": "remove",
"path": "/cost-map/PID3/PID1"
},
{
"op": "replace",
"path": "/cost-map/PID3/PID3",
"value": 1
}
]3.3. Multiplexing and Server Push: HTTP/2 💜
okay HTTP/2 [RFC7540] has two iconic features: multiplexing and server push 🔥 Multiplexing means a client and a server can send multiple HTTP requests and responses over one single TCP connection. They can be interleaved block by block (frame by frame), which is how HTTP/2 dodges the head-of-line blocking problem that HTTP/1.1 runs into. To get the same win, this design adds substream-id so a client can receive updates for multiple resources. HTTP/2 also has a server-push facility, which lets a server send asynchronous updates ✨
Now, even with those two features, this design goes with something compatible with HTTP/1.x, because HTTP/1.x is simpler 🫶 A design built on HTTP/2 would probably need a more complex HTTP/2 client library. In that case, one way to use server push for updates is for the update stream server to send each data update message as its own server-push item, and let the client apply each update as it arrives. An HTTP/2 client library doesn't necessarily tell the client application when the server pushes a resource. Instead, the library might cache the pushed resource and only hand it over when the client explicitly requests that URI 🙈 It's also more likely that an HTTP/2 design runs into trouble with a proxy sitting between the client and the server, because server push is optional and any proxy in between can disable it. For the intended use of server push, that's not a dealbreaker: eventually the client will request those resources, so disabling push just adds a delay. But it does mean Server Push is not suitable for resources the client doesn't know to ask for 💀
So this design leaves an HTTP/2-based design as future work, and focuses on ALTO updates over HTTP/1.x and [SSE] 🌈
3.4. Server Push: Server-Sent Event 🦋
Server-Sent Events (SSE) are techniques that can work with HTTP/1.1. Heads up: the summary below is non-normative, so for the real definition, check [SSE] 🎀
SSE lets a server send new data to a client through "server push". The client opens an HTTP [RFC7230] [RFC7231] connection to the server and keeps it open. The server keeps sending messages down it. Each message has one or more lines, where a line ends with a carriage return immediately followed by a new line, a carriage return NOT immediately followed by a new line, or a new line NOT immediately preceded by a carriage return. A message ends with a blank line (two line terminators in a row) 💖
Each line in a message looks like "field-name: string value". Lines with a blank field name (so, lines that start with a colon) get ignored, and so do lines with no colon at all. The protocol defines three field names: event, id, and data. If a message has more than one "data" line, the value of the data field is all those lines concatenated together. There can be only one "event" and one "id" line per message. The "data" field is required; the others are optional 🫶
Figure 1 is a sample SSE stream, starting with the client request. The server sends three events and then closes the stream 🍓
(Client request)
GET /stream HTTP/1.1
Host: example.com
Accept: text/event-stream
(Server response)
HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: text/event-stream
event: start
id: 1
data: hello there
event: middle
id: 2
data: let's chat some more ...
data: and more and more and ...
event: end
id: 3
data: goodbye
Figure 1: A Sample SSE Stream4. Overview of Approach and High-Level Protocol Message Flow 👑
With all that background covered, this section gives a non-normative overview of the update mechanisms and message flow that later sections of this document will define. Think of it as the preview trailer before the full season 🎬 Figure 2 shows the main components and the overall message flow.
-------------------------------------------------------------------
| |
| +-------+ +-------+ 1. init request +------+ |
| | | | | <-------------- | | |
| | | | | --------------> | | |
| 3.add/ | | | | 1'. control URI | | |
| remove | | | | | | |
| resource |Stream | |Update | | | |
-------->|Control| private |Stream | 2a. data update |Client| --
|Server |<------->|Server | messages | |
-------- | | | | ---------------> | | <-
| response | | | | ---------------> | | |
| | | | | 2b. control update| | |
| +-------+ +-------+ messages +------+ |
| |
-------------------------------------------------------------------
Figure 2: ALTO SSE Architecture and Message Flow4.1. Update Stream Service Message Flow ✨
okay so the building block of this whole update mechanism is the update stream service (defined in Section 6) 💖 and every update stream service is a POST-mode service that provides update streams. it's giving foundation 🧱
btw the lines that look like ** ... ** are just how this doc labels message flows in this section and the ones after it 🦋
Initial request: client -> update server: when an ALTO client wants an update stream service, it sets up a persistent connection to the update stream server and sends an initial update-stream request (defined in Section 6.5) 🎀 that's what creates the update stream. this initial request is labeled "1. init request" in Figure 2 ✨
an update stream can send updates for GET-mode resources (like ALTO network and cost maps) AND POST-mode resources (like ALTO endpoint property service) 🌈 and to avoid making a million update streams, the client can use ONE update stream for lots of requests 🧚 the client might ask for updates to the same resource but with different parameters for a POST-mode resource, and it can also bundle updates for multiple resources into a single stream. each request's updates are called a substream, so the server needs a way to say which substream an update belongs to 💅 the client gives each request in an update stream its own unique substream-id, and the server puts that substream-id in every update it sends. iconic behavior honestly 👑
Data updates: update server -> client: the whole point of an update stream is to keep pushing data value changes to the ALTO client for a set of resources 🔥 and that set is decided by what the client asked for. this doc calls those messages data update messages (defined in Section 5.2) 🫶 an update stream can update one or more requests, but each data update message only updates ONE request, and it gets sent as a Server-Sent Event (SSE), as defined by [SSE] ✨
a data update message is either a full replacement or an incremental change 💜 a full replacement uses the JSON message format from the ALTO protocol. for incremental changes there can be several encodings. the current design supports JSON merge patch [RFC7396] or JSON patch [RFC6902] to describe changes to the resource 🦋 future documents might define more incremental mechanisms too. the update stream server gets to decide when to send data update messages, and whether it sends a full replacement or an incremental change 🌸 those choices can differ from resource to resource and from update to update, no cap.
ngl this part is so cute: since the transport is compatible with HTTP/1.x, data update messages are delivered reliably and in order 🧁 and because delivery is lossless and sequential, the server can know the client's exact state and work out the right incremental updates 🎀 bestie that's the magic ✨ Figure 2 shows examples of data update messages (labeled "2a. data update messages") in the overall message flow 💖
Control updates: update server -> client: an update stream can run for a LONG time, so the status at the update stream server can change while the stream is alive 🕰️ like the server hits an error or needs to shut down for maintenance 🛠️ to keep things robust and flexible, this doc lets the update stream server send control update messages (defined in Section 5.3) alongside data update messages to the ALTO client 🌈 Figure 2 shows that both data updates and control updates can come from the server to the client (labeled "2b. control update messages") 🫶
4.2. Stream Control Service Message Flow 🎀
Stream control: client -> stream control server: the update stream server can trigger control changes, but in a flexible design the ALTO client can start control changes too 💅 specifically, by adding or removing ALTO resources that receive updates. the client does this through the stream control service (defined in Section 7) 🦋 you COULD use the same HTTP connection to send control requests, but that needs stronger server support, like HTTP pipeline 🙈 so for more flexibility, this doc introduces the stream control service 🧚 the update stream server uses its first message to hand over the URI of the stream control service (labeled "1': control URI" in Figure 2) ✨
with that URI, the ALTO client can ask the stream control server to tell the update stream server to do one of three things 👑:
- send data update messages for additional resources,
- stop sending data update messages for resources that were already requested, or
- gracefully stop and close the update stream altogether 💀
4.3. Service Announcement and Management Message Flow 🌸
Service announcements: IRD server -> client: an update server can offer any number of update stream services, and each update stream can cover a given subset of the ALTO server's resources 🍓 the ALTO server's Information Resource Directory (IRD) is where the update stream services get defined and where each one declares which resources it sends updates for 💜 the ALTO server picks the resource set for each service. it is recommended that if a resource depends on other resource(s) (marked with the "uses" attribute defined in [RFC7285]), those other resource(s) should also be part of that same update stream 🫶 so the update stream for a cost map should ALSO send updates for the network map that cost map depends on. makes total sense, slay 💅
Service management (server): an ALTO client can request any number of update streams at once 🌈 but every update stream eats up resources on the update stream server 🍽️ so the server may require client authorization and/or authentication, limit how many update streams are open, close inactive streams, or redirect the ALTO client to another update stream server 🚦 protective energy, honestly iconic 👑
5. Update Messages: Data Update and Control Update Messages 🧚
this section defines what update messages look like when they're sent from the server to the client 💖 it starts with the generic structure of update messages (Section 5.1). then it covers the details of data update messages (Section 5.2) and control update messages (Section 5.3) ✨ the next two sections use these messages to define the update stream service (Section 6) and the stream control service (Section 7) 🎀
5.1. Generic ALTO Update Message Structure 🦋
okay so data update and control update messages both have the same basic structure 🫶 each message has a data field, which is usually a JSON object 💜 and an event field that comes right before the data field and says which media type encodes the data field 🌸
a data update message also needs extra info to say WHICH ALTO data (object) the update applies to 🍓 to keep things generic, this doc uses a data-id to identify the ALTO data (object) being updated, see below 🔥
so the event field of an ALTO update message can have two subfields, media-type and data-id, separated by a comma (',', U+002C) 💅:
media-type [ ',' data-id ]according to Section 4.2 of [RFC6838], a comma is NOT allowed in a media-type name, so there's zero ambiguity when decoding the two subfields 🙌
note that an update message does not use the SSE "id" field. bestie, don't go looking for it there 💀
5.2. ALTO Data Update Message ✨
a data update message is sent when a monitored resource changes 🌈 as we just talked about, the event field of a data update message has two subfields: 'media-type' and 'data-id' 🎀
the 'media-type' subfield depends on whether the update is a complete spec of the identified data or an incremental patch (like a JSON merge patch or JSON patch) describing the changes from the last version of the data, if that's possible 🦋 this doc calls those full replacement and incremental change, respectively 💖 a full replacement is encoded as its defining document says (for network and cost maps, that's [RFC7285]) and uses the media type defined there 🫶 JSON merge patch is encoded per [RFC7396], with the media type application/merge-patch+json ✨ and JSON patch is encoded per [RFC6902], with media type application/json-patch+json 💅
the 'data-id' subfield tells you which ALTO data the data update message applies to 🧚
first up: the case where the resource has only ONE JSON object. for example, an ALTO client can ask for data updates for both a cost map resource (object) AND its dependent network map resource (object) in the same update stream 🌸 so to tell the updates apart, the client gives every resource receiving data updates its own substream-id 🍓 substream-ids MUST be unique within an update stream, but they don't need to be globally unique 🔥 a substream-id is encoded as a JSON string in the same format as the type ResourceID (Section 10.2 of [RFC7285]) 👑 this doc uses the type SubstreamID to mean a string of that format 💜 and for a single JSON object, its substream-id IS the 'data-id' 🫶
here's an example: say the ALTO client assigns substream-id "1" to its request for network map updates and substream-id "2" to the cost map 💖 then the substream-ids are the data-ids that say which objects get updated 🦋 Figure 3 shows some examples of ALTO data update messages 🎀:
event: application/alto-networkmap+json,1
data: { ... full network map message ... }
event: application/alto-costmap+json,2
data: { ... full cost map message ... }
event: application/merge-patch+json,2
data: { ... JSON merge patch update for the cost map ... }Figure 3: Examples of ALTO Data Update Messages 🌈
now the other case: a resource might include MULTIPLE JSON objects 🧚 this doc covers resources made of multiple components (parts), encoded with the media type "multipart/related" [RFC2387] 💅 each part of that multipart response MUST be an HTTP message with a Content-ID header and a JSON object body 🌸 and every component that needs the update stream service (defined in Section 6) MUST be identified by a unique Content-ID, which is defined in its defining document 🍓
for a resource using "multipart/related", the 'data-id' subfield MUST be the substream-id, then the '.' separator (U+002E), then the unique Content-ID, in that order 🔥 bestie, that's the full recipe for the ID 💀✨
5.3. ALTO Control Update Message
Okay so control update messages come with the media type application/alto-updatestreamcontrol+json 💖, and the data is an UpdateStreamControlEvent:
object {
[String control-uri;]
[SubstreamID started<1..*>;]
[SubstreamID stopped<1..*>;]
[String description;]
} UpdateStreamControlEvent;- control-uri: this is the URI that gives you stream control for this update stream (see Section 7) 🎀. The server sends a control update message to tell the client about the control-uri. This one gets sent once and MUST be the first event in an update stream 🥹. If the URI value is NULL, the update stream server doesn't support stream control for that update stream. Otherwise, the server gives you stream control through the URI it provided.
- started: a list of substream-ids of resources. It tells the ALTO client that the update stream server is about to start sending data update messages for each listed resource ✨.
- stopped: another list of substream-ids of resources. This one tells the ALTO client that the update stream server will no longer send data update messages for the listed resources 😭. There can be a bunch of reasons a server stops sending these, like a request from the ALTO client through stream control (Section 6.7.1), or an internal server event.
- description: a non-normative, human-readable text that explains the control event 📝. When a server stops sending data update messages for a resource, it is RECOMMENDED that the server use the description field to give details. Multiple things can trigger a "stopped" event, so check the list above. The whole point of this field is to give developers and/or admins a readable explanation so they can figure out potential problems 🔍.
6. Update Stream Service 🌊
An update stream service returns a stream of update messages, as defined in Section 5. An ALTO server's IRD (Information Resource Directory) MAY define one or more update stream services, and ALTO clients use those to request new update stream instances 💅. An IRD entry that defines an update stream service MUST define the media type, HTTP method, and capabilities and uses, as follows.
6.1. Media Type
The media type of an ALTO update stream service is text/event-stream, as defined by [SSE] 🦋.
6.2. HTTP Method
An ALTO update stream service is requested using the HTTP POST method 📮.
6.3. Capabilities
The capabilities are defined as an object of type UpdateStreamCapabilities:
object {
IncrementalUpdateMediaTypes incremental-change-media-types;
Boolean support-stream-control;
} UpdateStreamCapabilities;
object-map {
ResourceID -> String;
} IncrementalUpdateMediaTypes;So if this update stream can provide data update messages with incremental changes for a resource, the incremental-change-media-types field has an entry for that resource-id 🧩. The value is the supported media types of the incremental change, separated by commas. Normally it'll be application/merge-patch+json, application/json-patch+json, or application/merge-patch+json,application/json-patch+json, because (as Section 5 explains) those are the only incremental change types this document defines. Future extensions might define other types of incremental changes too 🔮.
When picking the media types to encode incremental changes for a resource, the update stream server MUST consider the limitations of the encoding 🙈. For example, if a JSON merge patch says a field's value is null, the semantics are that the field gets removed from the target, so the field is no longer defined (undefined). See the MergePatch algorithm in Section 3.1.1 for how null values get processed. That might not be the result you actually want, though, when null and undefined mean different things for the resource 💀. In that case, the update stream server MUST choose JSON patch over JSON merge patch, as long as JSON patch is listed as a capability of the update stream server. If the server doesn't support JSON patch to handle that case, it then needs to send a full replacement.
The support-stream-control field says whether the given update stream supports stream control. If support-stream-control is "true", the update stream server will use the stream control specified in this document 👑. Otherwise, the update stream server MAY use other mechanisms to provide the same functionality as stream control.
6.4. Uses
The "uses" attribute MUST be an array with the resource-ids of every resource this update stream can provide updates for 🍓. Each resource specified in "uses" MUST support full replacement. The update stream server can always send a full replacement, and the ALTO client MUST accept full replacement.
This set can be any subset of the ALTO server's resources, and it may include resources defined in linked IRDs ✨. Still, it is RECOMMENDED that the ALTO server picks a set that is closed under the resource dependency relationship. Here's what that means: if an update stream's "uses" set includes resource R1, and R1 depends on ("uses") resource R0, then the update stream's "uses" set SHOULD include R0 as well as R1 🫶. For example, an update stream for a cost map SHOULD also provide updates for the network map that the cost map depends on 🗺️.
6.5. Request: Accept Input Parameters
An ALTO client sets up the parameters for a new update stream by sending an HTTP POST body with the media type application/alto-updatestreamparams+json. That body contains a JSON object of type UpdateStreamReq:
object {
[AddUpdatesReq add;]
[SubstreamID remove<0..*>;]
} UpdateStreamReq;
object-map {
SubstreamID -> AddUpdateReq;
} AddUpdatesReq;
object {
ResourceID resource-id;
[JSONString tag;]
[Boolean incremental-changes;]
[Object input;]
} AddUpdateReq;- add: this specifies the resources (and their parameters) that the ALTO client wants updates for 🎯. Within the same update stream, the ALTO client MUST assign a substream-id that's unique in the scope of the update stream (Section 5.2) for each entry, and use those substream-ids as the keys in the
addfield. - resource-id: the resource-id of an ALTO resource. It MUST be in the update stream's "uses" list (Section 6.4) 🔑. If the resource-id is a GET-mode resource with a version tag (or "vtag"), as defined in Sections 6.3 and 10.3 of [RFC7285], and the ALTO client has previously retrieved a version of that resource from the update stream server, the ALTO client MAY set the
tagfield to the tag part of the client's version of that resource. If that version isn't current, the update stream server MUST send a full replacement before sending any incremental changes, as described in Section 6.7.1 😤. If that version is still current, the update stream server MAY skip the initial full replacement 🌸. - incremental-changes: this is where the ALTO client says whether it's willing to receive incremental changes from the update stream server for this substream 💜. If
incremental-changesis "true", the update stream server MAY send incremental changes for this substream. In that case, the client MUST support all incremental methods from the set announced in the server's capabilities for this resource. See Section 6.3 for how the server announces its potential incremental methods. If a client doesn't support all of the incremental methods from the set announced in the server's capabilities, the client can setincremental-changesto "false", and then the update stream server MUST NOT send incremental changes for that substream 🙅♀️. The default value forincremental-changesis "true", so if you want to suppress incremental changes, the ALTO client MUST explicitly setincremental-changesto "false". An alternative design would be finer-grained control, letting a client pick a subset of the incremental methods from the set announced in the server's capabilities. This document doesn't adopt that design, because it adds complexity to the server, which is more likely to be the bottleneck 🥱. Also, the ALTO client can't suppress full replacement 🚫. When the ALTO client setsincremental-changesto "false", the update stream server MUST send a full replacement instead of an incremental change. The update stream server MAY wait until more changes are available and then send a single full replacement with those changes. So an ALTO client that declines incremental changes may not get updates as quickly as one that accepts them 🐢. - input: if the resource is a POST-mode service that requires input, the ALTO client MUST set the
inputfield to a JSON object with the parameters that the resource expects 📥. - remove: this is used in update stream control requests (Section 7) and is not allowed in the update stream request 🙅. If it's included in the request anyway, the update stream server SHOULD ignore this field.
If a request has any errors, the update stream server MUST NOT create an update stream 🚨. The update stream server will also send an error response to the ALTO client, as specified in Section 6.6. No stream, error response, all the drama 😭.
6.6. Response ✨
okay so if the update stream request has any errors, the update stream server MUST return an HTTP "400 Bad Request" to the ALTO client 💅 the body follows the generic ALTO error response format from Section 8.5.2 of [RFC7285]. so an example ALTO error response looks like this:
HTTP/1.1 400 Bad Request
Content-Length: 131
Content-Type: application/alto-error+json
Connection: Closed
{
"meta":{
"code": "E_INVALID_FIELD_VALUE",
"field": "add/my-network-map/resource-id",
"value": "my-networkmap/#"
}
}bestie, note that "field" and "value" are optional fields 🫶 but if the "value" field exists, the "field" field MUST exist. no skipping that one!
- if an update stream request doesn't have an "add" field saying which resources you want, the error code MUST be E_MISSING_FIELD and the "field" field SHOULD be "add" 😭 the update stream server MUST close the stream without sending any events.
- if the "resource-id" field is invalid or isn't associated with the update stream, the error code MUST be E_INVALID_FIELD_VALUE 🙈 the "field" field SHOULD be the full path of the "resource-id" field, and the "value" field SHOULD be the invalid resource-id. if there's more than one invalid resource-ids, the update stream server SHOULD pick one and return it. the update stream server MUST close the stream (i.e., TCP connection) without sending any events.
- if the resource is a POST-mode service that needs input, the client MUST set the "input" field to a JSON object with the parameters that resource expects 🍓 if the "input" field is missing or invalid, the update stream server MUST return the same error response that resource would return for missing or invalid input (see [RFC7285]). in that case, the update stream server MUST close the update stream without sending any events. if the input for several POST-mode resources is missing or invalid, the update stream server MUST pick one and return it.
the response to a valid request is a stream of update messages ✨ Section 5 defines the update messages, and [SSE] defines how they get encoded into a stream.
an update stream server SHOULD send updates only when the underlying values change. however, it can be hard for an update stream server to guarantee that in every circumstance 🥺 so a client MUST NOT assume that an update message represents an actual change. ngl that's a key vibe check.
6.7. Additional Requirements on Update Stream Service 👑
6.7.1. Event Sequence Requirements
- the first event MUST be a control update message with the URI of the update stream control service (see Section 7) for this update stream 🦋 note that the value of the control-uri can be "null", which means there's no control stream service.
- as soon as possible after the ALTO client starts the connection, the update stream server checks the "tag" field for each added update request. if the "tag" field isn't specified in an added update request, the update stream server MUST first send a full replacement for the request 💜 if the "tag" field IS specified, the client can accept incremental changes, and the server can compute an incremental update based on the "tag" (the server needs to make sure that for a POST resource with input, the "tag" should indicate the correct result for different inputs). the update stream server MAY omit the initial full replacement ✨
- if this update stream provides updates for resource-ids R0 and R1, and R1 depends on R0, then the update stream server MUST send the update for R0 before sending the related updates for R1 🌈 for example, say an update stream provides updates to a network map and its dependent cost maps. when the network map changes, the update stream server MUST send the network map update before the cost map updates.
- when the ALTO client uses the stream control service to stop updates for one or more resources (Section 7), the ALTO client MUST send a stream control request 🔥 the update stream server MUST send a control update message whose "stopped" field has the substream-ids of all stopped resources.
6.7.2. Cross-Stream Consistency Requirements
if multiple ALTO clients create multiple update streams from the same update stream resource with the same update request parameters (i.e., same resource and same input), the update stream server MUST send the same updates to all of them 🫶 however, the update stream server MAY pack data items into different patch events, as long as the net result of applying those updates is the same.
for example, say two different ALTO clients create two different update streams for the same cost map, and the update stream server processes three separate cost point updates with a brief pause between each one 🎀 the server MUST send all three new cost points to both clients. but the update stream server MAY send a single patch event (with all three cost points) to one ALTO client while sending three separate patch events (with one cost point per event) to the other one. the vibes are different but the facts are the same 💅
an update stream server MAY offer several different update stream resources that provide updates to the same underlying resource (that is, a resource-id may appear in the "uses" field of more than one update stream resource). in that case, those update stream resources MUST return the same update 🌸
6.7.3. Multipart Update Requirements
this design allows any valid media type for full replacement. so it supports ALTO resources using multipart to contain multiple JSON objects 🧚 this gets you the push benefit but not the incremental encoding benefit of SSE.
JSON patch and merge patch give you the incremental encoding benefit, but they can only be applied to a single JSON object. so if an update stream service supports a resource providing a multipart media type (we call it a multipart resource), then the update stream service needs to handle the issue that a full multipart resource message can include multiple JSON objects. to deal with that, when an update stream service says it supports JSON patch or merge patch incremental updates for a multipart resource, the service MUST ensure that:
- each part of a multipart message is a single JSON object,
- each part is specified by a static Content-ID in the initial full message,
- each data update event applies to only one part, and
- each data update specifies
substream-id.content-idas the "event" field of the event, to identify the part to be updated.
6.8. Keep-Alive Messages 💖
in an SSE stream, any line that starts with a colon (U+003A) character is a comment, and an ALTO client MUST ignore that line [SSE] 🙈 as [SSE] recommends, an update stream server SHOULD send a comment line (or an event) every 15 seconds to keep ALTO clients and proxy servers from dropping the HTTP connection. TCP also has a Keep-Alive function, but the interval between TCP Keep-Alive messages can depend on the OS configuration and varies, so the recommended SSE Keep-Alive lets the SSE client detect the status of the update stream server with more certainty 🦋
7. Stream Control Service 🎀
a stream control service lets an ALTO client remove resources from the set monitored by an update stream, or add more resources to that set 🍓 it also lets an ALTO client gracefully shut down an update stream.
when an update stream server creates a new update stream, and it supports stream control for that update stream, the update stream server creates a stream control service for that update stream ✨ an ALTO client uses the stream control service to remove resources from the update stream instance or to request updates for more resources. an ALTO client cannot get the stream control service through the IRD 😭 instead, the first event the update stream server sends to the ALTO client has the URI for the associated stream control service (see Section 5.3).
each stream control request is its own individual HTTP request. the ALTO client MAY send multiple stream control requests to the stream control server over the same HTTP connection 🌈
7.1. URI
the URI for a stream control service, by itself, MUST uniquely specify the update stream instance it controls. the stream control server MUST NOT use other properties of an HTTP request, such as cookies or the client's IP address, to determine the update stream. and an update stream server MUST NOT reuse a control service URI once the associated update stream has been closed 🔥
the ALTO client MUST evaluate a relative control URI reference [RFC3986] (for example, a URI reference without a host or with a relative path) in the context of the URI used to create the update stream. the stream control service's host MAY be different from the update stream's host 👑
it's expected that there's an internal mechanism that maps a stream control URI to the unique update stream instance it controls. for example, the update stream service may assign a unique, internal stream id to each update stream instance. the exact mechanism is left up to the update stream service provider 🧚
security alert, bestie! 🚨 to stop an attacker from forging a stream control URI and sending bogus requests to mess with other update streams, the service should consider two security issues. first, if http, not https, is used, the stream control URI can be exposed to an on-path attacker. to deal with that, in a setting where the path from the server to the client can go through such an attacker, the server SHOULD use https. second, even without direct exposure, an off-path attacker may guess valid stream control URIs. to deal with that, the server SHOULD choose stream control URIs with enough randomness to make guessing hard, and the server SHOULD introduce mechanisms that detect repeated guesses that signal an attack (e.g., keeping track of the number of failed stream control attempts). please go read the W3C's "Good Practices for Capability URLs" at https://www.w3.org/TR/capability-urls/ 💀
7.2. Media Type
an ALTO stream control response does not have a specific media type 🌸
7.3. HTTP Method
an ALTO update stream control resource is requested using the HTTP POST method 💜
7.4. IRD Capabilities & Uses
None (stream control services do not appear in the IRD). no cap, that's the whole story 🫶
7.5. Request: Accept Input Parameters 📝
okay so a stream control service takes the same input media type and input parameters as the update stream service (Section 6.5) 💖. The only difference? It also accepts the remove field 💅.
If remove is specified, it's an array of substream-ids that the ALTO client previously added to this update stream. An empty remove array is equivalent to a list of all currently active resources, so the update stream server responds by removing all resources and closing the stream 💀.
An ALTO client MAY use the add field to add additional resources ✨. The ALTO client MUST assign a unique substream-id to each additional resource. Substream-ids MUST be unique over the lifetime of this update stream, so an ALTO client MUST NOT reuse a previously removed substream-id 🔒. The processing of an add resource is the same as discussed in Sections 6.5 and 6.6 🦋.
If a request has any errors, the update stream server MUST NOT add or remove any resources from the associated update stream 🚫. Also, the stream control server will return an error response to the client, as specified in Section 7.6 🎀.
7.6. Response 🌸
The stream control server MUST process the add field before the remove field 🧚. If the request removes all active resources without adding any additional resources, the update stream server MUST close the update stream. Thus, an update stream cannot have zero resources 👑.
If the request has any errors, the stream control server MUST return an HTTP 400 Bad Request to the ALTO client. The body part of the response follows the generic ALTO error response format specified in Section 8.5.2 of [RFC7285]. An error response has the same format as specified in Section 6.6. Detailed error code and error information are specified as below 🙈.
- If the
addrequest does not satisfy the requirements in Section 6.5, the stream control server MUST return the ALTO error message defined in Section 6.6. - If any substream-id in the
removefield was not added in a prior request, the error code of the error message MUST beE_INVALID_FIELD_VALUE, thefieldfield SHOULD beremove, and thevaluefield SHOULD be an array of the invalid substream-ids. Thus, it is illegal toaddandremovethe same substream-id in the same request. However, it is legal to remove a substream-id twice. To support the preceding checking, the update stream server MUST keep track of previously used but now closed substream-ids 🧠. - If any substream-id in the
addfield has been used before in this stream, the error code of the error message MUST beE_INVALID_FIELD_VALUE, thefieldfield SHOULD beadd, and thevaluefield SHOULD be an array of invalid substream-ids. - If the request has a non-empty
addfield and aremovefield with an empty list of substream-ids (to replace all active resources with a new set, the client MUST explicitly enumerate the substream-ids to be removed), the error code of the error message MUST beE_INVALID_FIELD_VALUE, thefieldfield SHOULD beremove, and thevaluefield SHOULD be an empty array.
If the request is valid but the associated update stream has been closed, then the stream control server MUST return an HTTP 404 Not Found 😭.
If the request is valid and the stream control server successfully processes the request without error, the stream control server should return either an HTTP 202 Accepted response or an HTTP 204 No Content response. The difference is that for the latter case, the stream control server is sure that the update stream server has also processed the request ✅. Regardless of a 202 or 204 HTTP response, the final updates of related resources will be notified by the update stream server using its control update message(s), due to the modular design 🌈.
8. Examples 🍓
8.1. Example: IRD Announcing Update Stream Services 🌈
Below is an example IRD announcing three update stream services 🎀. The first, which is named "update-my-costs", provides updates for the network map, the "routingcost" and "hopcount" cost maps, and a Filtered Cost Map resource. The second, which is named "update-my-prop", provides updates to the endpoint properties service. The third, which is named "update-my-pv", provides updates to a nonstandard ALTO service returning a multipart response 💜.
Note that in the "update-my-costs" update stream shown in the example IRD, the update stream server uses JSON patch for network map, and it uses JSON merge patch to update the other resources. Also, the update stream will only provide full replacements for "my-simple-filtered-cost-map" 🦋.
Also, note that this IRD defines two Filtered Cost Map resources. They use the same cost types, but "my-filtered-cost-map" accepts cost constraint tests, while "my-simple-filtered-cost-map" does not. To avoid the issues discussed in Section 9.3, the update stream provides updates for the second but not the first 🙅♀️.
This IRD also announces a nonstandard ALTO service, which is named "my-pv". This service accepts an extended endpoint cost request as an input and returns a multipart response, including an endpoint cost resource and a property map resource. This document does not rely on any other design details of this new service. In this document, the "my-pv" service is only used to illustrate how the update stream service provides updates to an ALTO resource returning a multipart response 🧚♀️.
"my-network-map": {
"uri": "https://alto.example.com/networkmap",
"media-type": "application/alto-networkmap+json",
},
"my-routingcost-map": {
"uri": "https://alto.example.com/costmap/routingcost",
"media-type": "application/alto-costmap+json",
"uses": ["my-networkmap"],
"capabilities": {
"cost-type-names": ["num-routingcost"]
}
},
"my-hopcount-map": {
"uri": "https://alto.example.com/costmap/hopcount",
"media-type": "application/alto-costmap+json",
"uses": ["my-networkmap"],
"capabilities": {
"cost-type-names": ["num-hopcount"]
}
},
"my-filtered-cost-map": {
"uri": "https://alto.example.com/costmap/filtered/constraints",
"media-type": "application/alto-costmap+json",
"accepts": "application/alto-costmapfilter+json",
"uses": ["my-networkmap"],
"capabilities": {
"cost-type-names": ["num-routingcost", "num-hopcount"],
"cost-constraints": true
}
},
"my-simple-filtered-cost-map": {
"uri": "https://alto.example.com/costmap/filtered/simple",
"media-type": "application/alto-costmap+json",
"accepts": "application/alto-costmapfilter+json",
"uses": ["my-networkmap"],
"capabilities": {
"cost-type-names": ["num-routingcost", "num-hopcount"],
"cost-constraints": false
}
},
"my-props": {
"uri": "https://alto.example.com/properties",
"media-type": "application/alto-endpointprops+json",
"accepts": "application/alto-endpointpropparams+json",
"capabilities": {
"prop-types": ["priv:ietf-bandwidth"]
}
},
"my-pv": {
"uri": "https://alto.example.com/endpointcost/pv",
"media-type": "multipart/related;
type=application/alto-endpointcost+json",
"accepts": "application/alto-endpointcostparams+json",
"capabilities": {
"cost-type-names": [ "path-vector" ],
"ane-properties": [ "maxresbw", "persistent-entities" ]
}
},
"update-my-costs": {
"uri": "https://alto.example.com/updates/costs",
"media-type": "text/event-stream",
"accepts": "application/alto-updatestreamparams+json",
"uses": [
"my-network-map",
"my-routingcost-map",
"my-hopcount-map",
"my-simple-filtered-cost-map"
],
"capabilities": {
"incremental-change-media-types": {
"my-network-map": "application/json-patch+json",
"my-routingcost-map": "application/merge-patch+json",
"my-hopcount-map": "application/merge-patch+json"
},
"support-stream-control": true
}
},
"update-my-props": {
"uri": "https://alto.example.com/updates/properties",
"media-type": "text/event-stream",
"uses": [ "my-props" ],
"accepts": "application/alto-updatestreamparams+json",
"capabilities": {
"incremental-change-media-types": {
"my-props": "application/merge-patch+json"
},
"support-stream-control": true
}
},
"update-my-pv": {
"uri": "https://alto.example.com/updates/pv",
"media-type": "text/event-stream",
"uses": [ "my-pv" ],
"accepts": "application/alto-updatestreamparams+json",
"capabilities": {
"incremental-change-media-types": {
"my-pv": "application/merge-patch+json"
},
"support-stream-control": true
}
}8.2. Example: Simple Network and Cost Map Updates ✨
Okay bestie, picture the IRD we just looked at, with all those update streams announced 💖. Now the ALTO client actually asks for something, using the update stream resource "update-my-costs" 🦋. It wants updates for the network map and the "routingcost" cost map, but it's ghosting the "hopcount" cost map 💀. It uses the ALTO server's resource-ids as its substream-ids 🎀. The client doesn't give the network map a "tag", so the update stream server has to send a full replacement for the network map AND the cost map 😭. The client also never sets "incremental-changes" to "false", so it defaults to "true". That means the server sends patch updates for both maps 🔥.
POST /updates/costs HTTP/1.1
Host: alto.example.com
Accept: text/event-stream,application/alto-error+json
Content-Type: application/alto-updatestreamparams+json
Content-Length: 155
{ "add": {
"my-network-map": {
"resource-id": "my-network-map"
},
"my-routingcost-map": {
"resource-id": "my-routingcost-map"
}
}
}
HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: text/event-stream
event: application/alto-updatestreamcontrol+json
data: {"control-uri":
data: "https://alto.example.com/updates/streams/3141592653589"}
event: application/alto-networkmap+json,my-network-map
data: {
data: "meta" : {
data: "vtag": {
data: "resource-id" : "my-network-map",
data: "tag" : "da65eca2eb7a10ce8b059740b0b2e3f8eb1d4785"
data: }
data: },
data: "network-map" : {
data: "PID1" : {
data: "ipv4" : [ "192.0.2.0/24", "198.51.100.0/25" ]
data: },
data: "PID2" : {
data: "ipv4" : [ "198.51.100.128/25" ]
data: },
data: "PID3" : {
data: "ipv4" : [ "0.0.0.0/0" ],
data: "ipv6" : [ "::/0" ]
data: }
data: }
data: }
data: }
event: application/alto-costmap+json,my-routingcost-map
data: {
data: "meta" : {
data: "dependent-vtags" : [{
data: "resource-id": "my-network-map",
data: "tag": "da65eca2eb7a10ce8b059740b0b2e3f8eb1d4785"
data: }],
data: "cost-type" : {
data: "cost-mode" : "numerical",
data: "cost-metric": "routingcost"
data: },
data: "vtag": {
data: "resource-id" : "my-routingcost-map",
data: "tag" : "3ee2cb7e8d63d9fab71b9b34cbf764436315542e"
data: }
data: },
data: "cost-map" : {
data: "PID1": { "PID1": 1, "PID2": 5, "PID3": 10 },
data: "PID2": { "PID1": 5, "PID2": 1, "PID3": 15 },
data: "PID3": { "PID1": 20, "PID2": 15 }
data: }
data: }Then, right after those events, the update stream server keeps the tea coming ☕ and sends more events whenever the maps change 🌸. Here's a small glow-up for the cost map: PID1->PID2 goes from 5 to 9 📈, PID3->PID1 is no longer available 🙅♀️, and PID3->PID3 is now defined as 1 ✨.
event: application/merge-patch+json,my-routingcost-map
data: {
data: "meta" : {
data: "vtag": {
data: "tag": "c0ce023b8678a7b9ec00324673b98e54656d1f6d"
data: }
data: },
data: "cost-map": {
data: "PID1" : { "PID2" : 9 },
data: "PID3" : { "PID1" : null, "PID3" : 1 }
data: }
data: }Next up, a change to the network map 🌈: the ipv4 prefix "203.0.113.0/25" gets added to PID1. That also shakes up the cost map 😳. The server decides to send an incremental change for the network map, but a full replacement for the cost map instead of an incremental one 💅.
event: application/json-patch+json,my-network-map
data: {
data: {
data: "op": "replace",
data: "path": "/meta/vtag/tag",
data: "value" :"a10ce8b059740b0b2e3f8eb1d4785acd42231bfe"
data: },
data: {
data: "op": "add",
data: "path": "/network-map/PID1/ipv4/2",
data: "value": "203.0.113.0/25"
data: }
data: }
event: application/alto-costmap+json,my-routingcost-map
data: {
data: "meta" : {
data: "vtag": {
data: "tag": "c0ce023b8678a7b9ec00324673b98e54656d1f6d"
data: }
data: },
data: "cost-map" : {
data: "PID1": { "PID1": 1, "PID2": 3, "PID3": 7 },
data: "PID2": { "PID1": 12, "PID2": 1, "PID3": 9 },
data: "PID3": { "PID1": 14, "PID2": 8 }
data: }
data: }8.3. Example: Advanced Network and Cost Map Updates 🧚
This one is basically the same vibe as the last example, just with a glow-up 💜. This time the ALTO client wants the "hopcount" cost map too, AND it already knows the current version tag of the network map 🙌. Because of that, the update stream server doesn't have to send the full network map data update message at the start of the stream 🥳. The client picks the substream-ids "net", "routing", and "hops" for these resources 🎀. The server sends the stream control URI and the full cost maps first, then updates for the network map and cost maps as they become available ⭐.
POST /updates/costs HTTP/1.1
Host: alto.example.com
Accept: text/event-stream,application/alto-error+json
Content-Type: application/alto-updatestreamparams+json
Content-Length: 244
{ "add": {
"net": {
"resource-id": "my-network-map",
"tag": "a10ce8b059740b0b2e3f8eb1d4785acd42231bfe"
},
"routing": {
"resource-id": "my-routingcost-map"
},
"hops": {
"resource-id": "my-hopcount-map"
}
}
}
HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: text/event-stream
event: application/alto-updatestreamcontrol+json
data: {"control-uri":
data: "https://alto.example.com/updates/streams/2718281828459"}
event: application/alto-costmap+json,routing
data: { ... full routingcost cost map message ... }
event: application/alto-costmap+json,hops
data: { ... full hopcount cost map message ... }
(pause)
event: application/merge-patch+json,routing
data: {"cost-map": {"PID2" : {"PID3" : 31}}}
event: application/merge-patch+json,hops
data: {"cost-map": {"PID2" : {"PID3" : 4}}}Okay so if the ALTO client decides it's over the "hopcount" cost map 🙅♀️, it can send a "remove" request on the stream control URI 💅:
POST /updates/streams/2718281828459 HTTP/1.1
Host: alto.example.com
Accept: text/plain,application/alto-error+json
Content-Type: application/alto-updatestreamparams+json
Content-Length: 24
{
"remove": [ "hops" ]
}
HTTP/1.1 204 No Content
Content-Length: 0
(stream closed without sending data content)The update stream server then sends a "stopped" control update message on the original request stream 📣, so the client knows updates have stopped for that resource:
event: application/alto-updatestreamcontrol+json
data: {
data: "stopped": ["hops"]
data: }Now here's a messy one 😬. Below is an invalid stream control request. The "remove" field includes an undefined substream-id, so the stream control server sends back an error response to the ALTO client 🚫.
POST /updates/streams/2718281828459 HTTP/1.1
Host: alto.example.com
Accept: text/plain,application/alto-error+json
Content-Type: application/alto-updatestreamparams+json
Content-Length: 31
{
"remove": [ "properties" ]
}
HTTP/1.1 400 Bad Request
Content-Length: 89
Content-Type: application/alto-error+json
{
"meta":{
"code": "E_INVALID_FIELD_VALUE",
"field": "remove",
"value": "properties"
}If the ALTO client doesn't need any updates anymore and wants to shut the update stream down gracefully 🕊️, it can send a "remove" request with an empty array:
POST /updates/streams/2718281828459 HTTP/1.1
Host: alto.example.com
Accept: text/plain,application/alto-error+json
Content-Type: application/alto-updatestreamparams+json
Content-Length: 17
{
"remove": [ ]
}
HTTP/1.1 204 No Content
Content-Length: 0
(stream closed without sending data content)Finally, the update stream server sends a final control update message on the original request stream to let the client know that all updates are stopped 😭🫶, and then it closes the stream:
event: application/alto-updatestreamcontrol+json
data: {
data: "stopped": ["net", "routing"]
data: }
(server closes stream)8.4. Example: Endpoint Property Updates ✨
okay so this one is about an ALTO client asking for updates on two property sets, priv:ietf-bandwidth for one group of endpoints and priv:ietf-load for another 🧚 the update stream server immediately sends full replacements with the property values for every endpoint, and then it sends data update messages for individual endpoints whenever their values change 💅
POST /updates/properties HTTP/1.1
Host: alto.example.com
Accept: text/event-stream
Content-Type: application/alto-updatestreamparams+json
Content-Length: 511
{ "add": {
"props-1": {
"resource-id": "my-props",
"input": {
"properties" : [ "priv:ietf-bandwidth" ],
"endpoints" : [
"ipv4:198.51.100.1",
"ipv4:198.51.100.2",
"ipv4:198.51.100.3"
]
}
},
"props-2": {
"resource-id": "my-props",
"input": {
"properties" : [ "priv:ietf-load" ],
"endpoints" : [
"ipv6:2001:db8:100::1",
"ipv6:2001:db8:100::2",
"ipv6:2001:db8:100::3"
]
}
}
}
}
HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: text/event-stream
event: application/alto-updatestreamcontrol+json
data: {"control-uri":
data: "https://alto.example.com/updates/streams/1414213562373"}
event: application/alto-endpointprops+json,props-1
data: { "endpoint-properties": {
data: "ipv4:198.51.100.1" : { "priv:ietf-bandwidth": "13" },
data: "ipv4:198.51.100.2" : { "priv:ietf-bandwidth": "42" },
data: "ipv4:198.51.100.3" : { "priv:ietf-bandwidth": "27" }
data: } }
event: application/alto-endpointprops+json,props-2
data: { "endpoint-properties": {
data: "ipv6:2001:db8:100::1" : { "priv:ietf-load": "8" },
data: "ipv6:2001:db8:100::2" : { "priv:ietf-load": "2" },
data: "ipv6:2001:db8:100::3" : { "priv:ietf-load": "9" }
data: } }
(pause)
event: application/merge-patch+json,props-1
data: { "endpoint-properties":
data: {"ipv4:198.51.100.1" : {"priv:ietf-bandwidth": "3"}}
data: }
(pause)
event: application/merge-patch+json,props-2
data: { "endpoint-properties":
data: {"ipv6:2001:db8:100::3" : {"priv:ietf-load": "7"}}
data: }if the ALTO client needs priv:ietf-bandwidth and priv:ietf-load for more endpoints, it can just send an "add" request on the stream control URI 🦋
POST /updates/streams/1414213562373" HTTP/1.1
Host: alto.example.com
Accept: text/plain,application/alto-error+json
Content-Type: application/alto-updatestreamparams+json
Content-Length: 448
{ "add": {
"props-3": {
"resource-id": "my-props",
"input": {
"properties" : [ "priv:ietf-bandwidth" ],
"endpoints" : [
"ipv4:198.51.100.4",
"ipv4:198.51.100.5"
]
}
},
"props-4": {
"resource-id": "my-props",
"input": {
"properties" : [ "priv:ietf-load" ],
"endpoints" : [
"ipv6:2001:db8:100::4",
"ipv6:2001:db8:100::5"
]
}
}
}
}
HTTP/1.1 204 No Content
Content-Length: 0
(stream closed without sending data content)the update stream server sends full replacements for the two new resources, and then incremental changes for all four requests as they come in 🌈
event: application/alto-endpointprops+json,props-3
data: { "endpoint-properties": {
data: "ipv4:198.51.100.4" : { "priv:ietf-bandwidth": "25" },
data: "ipv4:198.51.100.5" : { "priv:ietf-bandwidth": "31" },
data: } }
event: application/alto-endpointprops+json,props-4
data: { "endpoint-properties": {
data: "ipv6:2001:db8:100::4" : { "priv:ietf-load": "6" },
data: "ipv6:2001:db8:100::5" : { "priv:ietf-load": "4" },
data: } }
(pause)
event: application/merge-patch+json,props-3
data: { "endpoint-properties":
data: {"ipv4:198.51.100.5" : {"priv:ietf-bandwidth": "15"}}
data: }
(pause)
event: application/merge-patch+json,props-2
data: { "endpoint-properties":
data: {"ipv6:2001:db8:100::2" : {"priv:ietf-load": "9"}}
data: }
(pause)
event: application/merge-patch+json,props-4
data: { "endpoint-properties":
data: {"ipv6:2001:db8:100::4" : {"priv:ietf-load": "3"}}
data: }8.5. Example: Multipart Message Updates 🎀
this one is about an ALTO client asking for a nonstandard ALTO service that returns a multipart response 💜 the update stream server immediately sends full replacements of that multipart response, and after that it sends data update messages for each individual part whenever the ALTO data (object) inside it changes 🫶
POST /updates/pv HTTP/1.1
Host: alto.example.com
Accept: text/event-stream
Content-Type: application/alto-updatestreamparams+json
Content-Length: 382
{
"add": {
"ecspvsub1": {
"resource-id": "my-pv",
"input": {
"cost-type": {
"cost-mode": "array",
"cost-metric": "ane-path"
},
"endpoints": {
"srcs": [ "ipv4:192.0.2.2" ],
"dsts": [ "ipv4:192.0.2.89", "ipv4:203.0.113.45" ]
},
"ane-properties": [ "maxresbw", "persistent-entities" ]
}
}
}
}
HTTP/1.1 200 OK
Connection: keep-alive
Content-Type: text/event-stream
event: application/alto-updatestreamcontrol+json
data: {"control-uri":
data: "https://alto.example.com/updates/streams/1414"}
event: multipart/related;boundary=example-pv;
type=application/alto-endpointcost+json,ecspvsub1
data: --example-pv
data: Content-ID: ecsmap
data: Content-Type: application/alto-endpointcost+json
data:
data: { ... data (object) of an endpoint cost map ... }
data: --example-pv
data: Content-ID: propmap
data: Content-Type: application/alto-propmap+json
data:
data: { ... data (object) of a property map ... }
data: --example-pv--
(pause)
event: application/merge-patch+json,ecspvsub1.ecsmap
data: { ... merge patch for updates of ecspvsub1.ecsmap ... }
event: application/merge-patch+json,ecspvsub1.propmap
data: { ... merge patch for updates of ecspvsub1.propmap ... }9. Operation and Processing Considerations ⚙️💖
9.1. Considerations for Choosing Data Update Messages 🍓
okay this part is all about picking the right update messages, and it's lowkey a big deal 💅 the update stream server should be cognizant of what its update schedule does, and that covers both timing (when and what to trigger an update for) and message format (given an update, send a full replacement or an incremental change) ✨ the schedule really affects both overhead and freshness of info. to keep overhead down, the server may batch a bunch of updates for resources that change a lot, sending cumulative updates or a full replacement after a while 🌸 but the update stream server should be cognizant that batching makes information less fresh, and the server should also think about how those delays affect client behavior (more on client timeouts while waiting for updates of dependent resources below) 🙈
now for incremental updates, this design lets you use either JSON patch or JSON merge patch 🦋 for cost maps, endpoint costs, and endpoint properties, JSON merge patch is clearly better than JSON patch, no debate. it's more space efficient and simpler to apply, so there's zero advantage to letting a server use JSON patch for those resources 😭
network maps are not as clear-cut though 👀
first, small changes, like moving a prefix from one PID to another. JSON patch could say it as a simple insertion and deletion, but JSON merge patch would have to replace the entire array of prefixes for both PIDs. BUT to process a JSON patch update, the ALTO client would have to keep track of the index of each prefix for every PID 💀 logically, the prefixes in a PID are an unordered set, not an array, and apart from handling updates a client has no reason to keep array indexes. so using JSON patch for network maps would mean clients store extra data they don't otherwise need 🧚
second, bigger changes, like removing half the prefixes from a PID. JSON merge patch would just send a new array for that PID, while JSON patch would have to send a list of remove operations and delete the prefixes one by one 🔥
so the bottom line: each update stream server can decide for itself whether to use JSON merge patch or JSON patch based on the changes it's making to network maps 👑✨
9.2. Considerations for Client Processing Data Update Messages 📨
Okay so when an ALTO client gets a full replacement for a resource, the move is to swap the old version for the new one. When it gets an incremental change, it applies those patches to the current version. Simple, right? ✨
BUT (and it's a big but 😭) resources can depend on other resources. Cost maps depend on network maps, so an ALTO client MUST NOT use a dependent resource if the resource it depends on has changed. There are at least two ways to handle this. The examples below use network and cost maps, but these techniques apply to any dependent resources. 🧩
Quick heads-up: when a network map changes, the update stream server MUST send the network map update message before it sends the updates for the dependent cost maps (see Section 6.7.1). 👑
Approach one: the ALTO client buffers the network map update message and keeps using the previous network map and its associated cost maps until it has received the update messages for all dependent cost maps. Then it applies all the network and cost map updates atomically. 🎀
Approach two: the ALTO client MAY update the network map immediately. In that case the cost maps using that network map become invalid, because they no longer match the current network map. So the ALTO client MUST mark each such dependent cost map as temporarily invalid, and MUST NOT use each such cost map until it receives a cost map update message saying it's based on the new network map version tag. 🚫
The update stream server SHOULD send updates for dependent resources (the cost maps in the example above) in a timely fashion. But if the ALTO client doesn't get the updates it expects, a simple recovery move is to close the update stream connection, throw away the dependent resources, and reestablish the update stream. The ALTO client MAY keep the version tag of the last version of any tagged resources and give those version tags when it requests the new update stream. Then, if a version is still current, the update stream server won't resend that resource. 🔁
Not as efficient as possible, I know, but this recovery method is simple and reliable. 💖
9.3. Considerations for Updates to Filtered Cost Maps 🔍
If an update stream provides updates to a Filtered Cost Map that allows constraint tests, then an ALTO client MAY request updates to a Filtered Cost Map request with a constraint test. Here's the tea: when a cost changes, the update stream server MUST send an update if the new value satisfies the test. If the new value doesn't, it depends on the previous value. If the previous value didn't satisfy the test either, the update stream server SHOULD NOT send an update to the ALTO client. But if the previous value did satisfy it, the update stream server MUST send an update with a "null" value to let the ALTO client know this cost no longer meets the criteria. 🙈
Plot twist: a server can dodge all this drama by offering update streams only for Filtered Cost Maps that do not allow constraint tests. 💅
9.4. Considerations for Updates to Ordinal Mode Costs 📊
For an ordinal mode cost map, a change to a single cost point may require updating many other costs. As an extreme example, suppose the lowest cost changes to the highest cost. With a numerical mode cost map, only that one cost changes. But with an ordinal mode cost map, every cost might change 😱
This document allows an update stream server to offer incremental updates for ordinal mode cost maps. Still, update stream server implementors should be aware that incremental updates for ordinal costs are more complicated than for numerical costs, and ALTO clients should be aware that small changes may result in large updates.
An update stream server can avoid this complication by only offering full replacements for ordinal cost maps. 🫶
9.5. Considerations for SSE Text Formatting and Processing 🌊
SSE was designed for events that consist of relatively small amounts of line-oriented text data, and SSE clients frequently read input one line at a time. But an update stream sends a full cost map as a single events, and a cost map may involve megabytes, if not tens of megabytes, of text. That has implications the ALTO client and the update stream server may want to consider. 🧐
First, some SSE client libraries read all the data for an event into memory and then present it to the client as a character array. But a client may not have enough memory to hold the entire JSON text for a large cost map. So an ALTO client SHOULD consider using an SSE library that presents the event data in manageable chunks, so the ALTO client can parse the cost map incrementally and store the underlying data in a more compact format.
Second, an SSE client library may use a low-level, generic socket read library that stores each line of event data, just in case the higher-level parser needs the line delimiters as part of the protocol formatting. If a server sends a complete cost map as one single line, that can create a multi-megabyte data "line", and a line that long may require complex memory management at the client. It is RECOMMENDED that an update stream server limit the lengths of data lines. 📏
Third, an SSE server may use a library that puts line breaks in places that would have semantic consequences for the ALTO updates; see Section 11. The update stream server implementation MUST ensure that no line breaks are introduced to change the semantics. ⚠️
10. Security Considerations 🔐
Buckle up, this part is serious! The security considerations of the base protocol (Section 15 of [RFC7285]) fully apply to this extension. For example, the same authenticity and integrity considerations (Section 15.1 of [RFC7285]) still fully apply, and the same considerations for the privacy of ALTO users (Section 15.4 of [RFC7285]) also still fully apply. 🔒
The additional services (the addition of update streams and stream control URIs) provided by this extension extend the attack surface described in Section 15.1.1 of [RFC7285]. Below, we talk through the additional risks and their remedies.
10.1. Update Stream Server: Denial-of-Service Attacks 🚨
Allowing persistent update stream connections enables a new class of Denial-of-Service attacks. For the update stream server, an ALTO client might create an unreasonable number of update stream connections, or add an unreasonable number of substream-ids to one update stream.
To avoid these attacks on the update stream server, the server SHOULD choose to limit the number of active streams and reject new requests when that threshold is reached. An update stream server SHOULD also choose to limit the number of active substream-ids on any given stream, or limit the total number of substream-ids used over the lifetime of a stream and reject any stream control request that would exceed those limits. In these cases, the update stream server SHOULD return the HTTP status 503 Service Unavailable.
It's important to note that these approaches aren't the only possibilities. For example, it may be possible for the update stream server to use somewhat more clever logic involving IP reputation, rate-limiting, and splitting the overall threshold into smaller thresholds that apply to subsets of potential clients. 🧠
While these techniques stop update stream DoS attacks from disrupting the update stream server's other services, they do make it easier for a DoS attack to disrupt the update stream service itself. Therefore, an update stream server MAY prefer to restrict update stream services to authorized clients, as discussed in Section 15 of [RFC7285].
Alternatively, an update stream server MAY return the HTTP status 307 Temporary Redirect to redirect the client to another ALTO server that can better handle a large number of update streams. ↪️
10.2. ALTO Client: Update Overloading or Instability 🥵
The availability of continuous updates can also cause overload for an ALTO client, in particular one with limited processing capabilities. The current design does not include any flow control mechanisms for the client to reduce the update rates from the server. Under overloading, the client MAY choose to remove the information resources with high update rates.
Also, under overloading, the client may no longer be able to detect whether information is still fresh or has become stale. In such a case, the client should be careful in how it uses the information, so it avoids stability or efficiency issues. 🫠
10.3. Stream Control: Spoofed Control Requests and Information Breakdown 🕵️
An outside party that can read the update stream response, or that can observe stream control requests, can obtain the control URI and use it to send fraudulent "remove" requests, disabling updates for the valid ALTO client. This can be avoided by encrypting the update stream and stream control requests (see Section 15 of [RFC7285]). Also, the update stream server echoes the "remove" requests on the update stream, so the valid ALTO client can detect unauthorized requests.
In general, because the architecture allows the update stream server and the stream control server to be different entities, the additional risks should be evaluated and remedied. For example, the private communication path between the servers may be attacked, resulting in a risk of communications breakdown between them, as well as invalid or spoofed messages claiming to be on that private communications path. Proper security mechanisms, including confidentiality, authenticity, and integrity mechanisms, should be considered. 🛡️
11. Requirements on Future ALTO Services to Use This Design 📋
Although this design is quite flexible, it has underlying requirements.
The key requirements are that (1) each data update message is for a single resource and (2) an incremental change can be applied only to a resource that is a single JSON object, as both JSON merge patch and JSON patch can apply only to a single JSON object. So if a future ALTO resource can contain multiple objects, then either each individual object also has a resource-id, or an extension to this design is made.
At the low-level encoding level, a new line in SSE has its own semantics. Hence, this design requires that resource encoding does not include new lines that can be confused with SSE encoding. In particular, the data update message MUST NOT include event: or data: at a new line as part of the data message. 🚫
If an update stream provides updates to a Filtered Cost Map that allows constraint tests, the requirements for such services are stated in Section 9.3. 🔍
12. IANA Considerations 🌸
This document defines two new media types: application/alto-updatestreamparams+json, as described in Section 6.5, and application/alto-updatestreamcontrol+json, as described in Section 5.3. All other media types used in this document have already been registered, either for ALTO, JSON merge patch, or JSON patch. 🎀
12.1. application/alto-updatestreamparams+json Media Type
Type name:applicationSubtype name:alto-updatestreamparams+jsonRequired parameters:N/AOptional parameters:N/AEncoding considerations:Encoding is exactly the same as for theapplication/jsonmedia type, so check [RFC8259] for the details 📝Security considerations:Security stuff for generating and consuming ALTO Protocol messages is covered in Section 10 of RFC 8895 and Section 15 of [RFC7285]. Go read it, it's serious 😭Interoperability considerations:RFC 8895 spells out the format of conforming messages and how to interpret them 🎀Published specification:Section 6.5 of RFC 8895.Applications that use this media type:ALTO servers and ALTO clients, whether they run standalone or are embedded inside other applications ✨Fragment identifier considerations:N/AAdditional information:Deprecated alias names for this type:N/AMagic number(s):N/AFile extension(s):RFC 8895 uses the media type to refer to protocol messages, so it doesn't need a file extension 💅Macintosh file type code(s):N/A
Person & email address to contact for further information:See Authors' Addresses section.Intended usage:COMMONRestrictions on usage:N/AAuthor:See Authors' Addresses section.Change controller:Internet Engineering Task Force (mailto:iesg@ietf.org).
12.2. application/alto-updatestreamcontrol+json Media Type
Type name:applicationSubtype name:alto-updatestreamcontrol+jsonRequired parameters:N/AOptional parameters:N/AEncoding considerations:Same deal as before: encoding is identical to theapplication/jsonmedia type, so see [RFC8259] 💖Security considerations:Security for generating and consuming ALTO Protocol messages is discussed in Section 10 of RFC 8895 and Section 15 of [RFC7285]. Yes, it's the same line again, and yes, it matters 🔥Interoperability considerations:RFC 8895 specifies format of conforming messages and the interpretation thereof 🦋Published specification:Section 5.3 of RFC 8895.Applications that use this media type:ALTO servers and ALTO clients, standalone or embedded in other applications 🌸Fragment identifier considerations:N/AAdditional information:Deprecated alias names for this type:N/AMagic number(s):N/AFile extension(s):RFC 8895 uses the media type to refer to protocol messages, so no file extension needed 🧚Macintosh file type code(s):N/A
Person & email address to contact for further information:See Authors' Addresses section.Intended usage:COMMONRestrictions on usage:N/AAuthor:See Authors' Addresses section.Change controller:Internet Engineering Task Force (mailto:iesg@ietf.org).
13. Appendix: Design Decision: Not Allowing Stream Restart 🙈
okay so picture this: an update stream gets closed by accident. When the ALTO client reconnects, the update stream server has to resend the full maps. Ngl, that is clearly inefficient 😩 The SSE specification lets an update stream server give each event an id, so the client can say "hey, here's the id of the last event I got" and the server picks up at the next one. Iconic, right?
BUT that mechanism adds extra complexity. The server has to save SSE messages in a buffer in case clients reconnect. And it's never going to be perfect: if the ALTO client waits too long to reconnect, or sends an invalid ID, the server has to resend the complete maps anyway. Not ideal, bestie.
Here's the thing though, this is unlikely to be a problem in practice. ALTO clients that want continuous updates for big resources, like full network and cost maps, are probably things like P2P trackers. Those are well connected to the network, so they rarely drop connections 💜
Mobile devices absolutely can and do drop connections, and they will have to reconnect. BUT mobile devices won't need continuous updates for multi-megabyte cost maps. If they need continuous updates at all, it's for small queries, like the costs from a small set of media servers the device can stream the currently playing movie from. So if a mobile device drops and re-establishes the update stream, the update stream server only has to retransmit a small amount of redundant data 🍓
So basically: using event ids to avoid resending the full map adds a whole lot of complexity to fix a situation that is super rare. The complexity isn't worth the benefit. ✨
Now, the update stream service does let the ALTO client specify the tag of the last received version of any tagged resource. If that tag is still current, the update stream server doesn't need to retransmit the full resource. So ALTO clients can use this to avoid retransmitting full network maps. Cost maps aren't tagged, though, so this won't work for them 😭 Of course, the ALTO protocol could be extended by adding version tags to cost maps, which would solve the retransmission-on-reconnect problem. However, adding tags to cost maps might bring a new set of complications. Tradeoffs, tradeoffs 🫶
14. References
14.1. Normative References
Here's the full normative list, copied exactly so nothing gets lost in the vibes 💅
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC2387] Levinson, E., "The MIME Multipart/Related Content-type",
RFC 2387, DOI 10.17487/RFC2387, August 1998,
<https://www.rfc-editor.org/info/rfc2387>.
[RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform
Resource Identifier (URI): Generic Syntax", STD 66,
RFC 3986, DOI 10.17487/RFC3986, January 2005,
<https://www.rfc-editor.org/info/rfc3986>.
[RFC6838] Freed, N., Klensin, J., and T. Hansen, "Media Type
Specifications and Registration Procedures", BCP 13,
RFC 6838, DOI 10.17487/RFC6838, January 2013,
<https://www.rfc-editor.org/info/rfc6838>.
[RFC6902] Bryan, P., Ed. and M. Nottingham, Ed., "JavaScript Object
Notation (JSON) Patch", RFC 6902, DOI 10.17487/RFC6902,
April 2013, <https://www.rfc-editor.org/info/rfc6902>.
[RFC7285] Alimi, R., Ed., Penno, R., Ed., Yang, Y., Ed., Kiesel, S.,
Previdi, S., Roome, W., Shalunov, S., and R. Woundy,
"Application-Layer Traffic Optimization (ALTO) Protocol",
RFC 7285, DOI 10.17487/RFC7285, September 2014,
<https://www.rfc-editor.org/info/rfc7285>.
[RFC7396] Hoffman, P. and J. Snell, "JSON Merge Patch", RFC 7396,
DOI 10.17487/RFC7396, October 2014,
<https://www.rfc-editor.org/info/rfc7396>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC8259] Bray, T., Ed., "The JavaScript Object Notation (JSON) Data
Interchange Format", STD 90, RFC 8259,
DOI 10.17487/RFC8259, December 2017,
<https://www.rfc-editor.org/info/rfc8259>.
[SSE] Hickson, I., "Server-Sent Events", W3C Recommendation,
February 2015, <https://www.w3.org/TR/eventsource/>.14.2. Informative References
And the informative ones, for when you want to go deeper 🌈
[RFC4960] Stewart, R., Ed., "Stream Control Transmission Protocol",
RFC 4960, DOI 10.17487/RFC4960, September 2007,
<https://www.rfc-editor.org/info/rfc4960>.
[RFC5789] Dusseault, L. and J. Snell, "PATCH Method for HTTP",
RFC 5789, DOI 10.17487/RFC5789, March 2010,
<https://www.rfc-editor.org/info/rfc5789>.
[RFC7230] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer
Protocol (HTTP/1.1): Message Syntax and Routing",
RFC 7230, DOI 10.17487/RFC7230, June 2014,
<https://www.rfc-editor.org/info/rfc7230>.
[RFC7231] Fielding, R., Ed. and J. Reschke, Ed., "Hypertext Transfer
Protocol (HTTP/1.1): Semantics and Content", RFC 7231,
DOI 10.17487/RFC7231, June 2014,
<https://www.rfc-editor.org/info/rfc7231>.
[RFC7540] Belshe, M., Peon, R., and M. Thomson, Ed., "Hypertext
Transfer Protocol Version 2 (HTTP/2)", RFC 7540,
DOI 10.17487/RFC7540, May 2015,
<https://www.rfc-editor.org/info/rfc7540>.Acknowledgments ✨
Huge thanks to Dawn Chen (Tongji University), Shawn Lin (Tongji University), and Xiao Shi (Yale University) for their work on an earlier version of this document 🫶
Contributors 💜
Sections 2, 5.1, 5.2, and 8.5 of this document are built on contributions from Jingxuan Jensen Zhang, and he's considered an author too 👑
Authors' Addresses 🦋
Wendy Roome
Nokia Bell Labs (Retired)
124 Burlington Rd
Murray Hill, NJ 07974
United States of America
Phone: +1-908-464-6975
Email: wendy@wdroome.com
Y. Richard Yang
Yale University
51 Prospect St
New Haven, CT
United States of America
Email: yry@cs.yale.edu📦 where this page comes from
- ietf: rfc8895 2020-11
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms