RFC 8847
Protocol for Controlling Multiple Streams for Telepresence (CLUE)
RFC 8847 🎀 Protocol for Controlling Multiple Streams for Telepresence (CLUE)
okay bestie, this is the title block 💖 so you know who's behind it and what kind of doc it is:
- Number: RFC 8847 (the Request for Comments number)
- Stream: Internet Engineering Task Force (IETF) 🌸
- Authors: R. Presta and S P. Romano, both from the University of Napoli 🇮🇹
- Category: Experimental 🧪
- ISSN: 2070-1721
- Date: January 2021
- Obsoletes / Updates: the title block doesn't list anything it obsoletes or updates ✨
Abstract 💌
okay so CLUE stands for Controlling Multiple Streams for Telepresence 🎀 it's an application protocol for describing and negotiating a telepresence session, and the design takes the requirements and framework from the IETF CLUE Working Group into account 🦋 A companion document, RFC 8848, digs into the CLUE signaling details and the SIP / Session Description Protocol (SDP) session establishment phase 💅 CLUE messages flow over the CLUE data channel, which is based on reliable and ordered SCTP-over-DTLS transport. ("SCTP" stands for "Stream Control Transmission Protocol".) The message details, plus how CLUE Participants act as Media Providers and/or Media Consumers, are all covered here 🫶
Status of This Memo 📜
ngl this part is a little formal, but the key bit is that this document is not an Internet Standards Track specification 🙈 It's published so people can examine it, try out experimental implementations, and evaluate it 🧪
This document defines an Experimental Protocol for the Internet community. It's a product of the Internet Engineering Task Force (IETF) and represents the consensus of the IETF community 🌈 It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). Not every document the IESG approves is a candidate for any level of Internet Standard, so see Section 2 of RFC 7841 for the details 👑
Info about the current status of this document, any errata, and how to give feedback lives at https://www.rfc-editor.org/info/rfc8847 🔗
Copyright Notice ©️
Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved. ✨
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication. Please review these carefully, because they describe your rights and restrictions with respect to this document 📚 Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they're provided without warranty as described in the Simplified BSD License 💜
Table of Contents 📚
1. Introduction
2. Terminology
3. Conventions
4. Overview of the CLUE Protocol
5. Protocol Messages
5.1. 'options'
5.2. 'optionsResponse'
5.3. 'advertisement'
5.4. 'ack'
5.5. 'configure'
5.6. 'configureResponse'
5.7. Response Codes and Reason Strings
6. Protocol State Machines
6.1. Media Provider's State Machine
6.2. Media Consumer's State Machine
7. Versioning
8. Extensions
8.1. Extension Example
9. XML Schema
10. Call Flow Example
10.1. CLUE Message No. 1: 'options'
10.2. CLUE Message No. 2: 'optionsResponse'
10.3. CLUE Message No. 3: 'advertisement'
10.4. CLUE Message No. 4: 'configure+ack'
10.5. CLUE Message No. 5: 'configureResponse'
10.6. CLUE Message No. 6: 'advertisement'
10.7. CLUE Message No. 7: 'ack'
10.8. CLUE Message No. 8: 'configure'
10.9. CLUE Message No. 9: 'configureResponse'
11. Security Considerations
12. IANA Considerations
12.1. URN Sub-Namespace Registration
12.2. XML Schema Registration
12.3. Media Type Registration for "application/clue+xml"
12.4. CLUE Protocol Registry
12.4.1. CLUE Message Types
12.4.2. CLUE Response Codes
13. References
13.1. Normative References
13.2. Informative References
Acknowledgements
Authors' Addresses1. Introduction ✨
The Controlling Multiple Streams for Telepresence (CLUE) protocol is an application protocol that two CLUE Participants use to make a multimedia telepresence session feel amazing 🌟 The main goals are:
- enabling a Media Provider (MP) to properly announce its current telepresence capabilities to a Media Consumer (MC), in terms of available media captures, groups of encodings, simultaneity constraints, and other information defined in [RFC8845]. 📣
- enabling an MC to request the desired multimedia streams from the offering MP. 🙏
CLUE-capable endpoints are connected by means of the CLUE data channel -- an SCTP-over-DTLS channel that is opened and established as described in [RFC8848] and [RFC8850]. ("SCTP" stands for "Stream Control Transmission Protocol".) CLUE protocol messages that flow over that channel are detailed in this document, both syntactically and semantically 🧚
In Section 4, we get a general overview of the CLUE protocol 🦋 CLUE protocol messages are detailed in Section 5, and the CLUE protocol state machines are introduced in Section 6. Versioning and extensions are discussed in Sections 7 and 8, respectively. The XML schema [W3C.REC-xml-20081126] that defines the CLUE messages is provided in Section 9 🌈
2. Terminology 🧚
This document uses terminology that's also used in [RFC8845] and [RFC7262], so for convenience, here's the list 💕 The definition of "CLUE Participant", also listed below, originates from this document.
- Capture Encoding: A specific encoding of a Media Capture, to be sent via RTP [RFC3550]. 🎞️
- CLUE Participant (CP): An entity able to use the CLUE protocol within a telepresence session. It can be an endpoint or an MCU (Multipoint Control Unit) able to use the CLUE protocol. 🫂
- CLUE-capable device: A device that (1) supports the CLUE data channel [RFC8850], the CLUE protocol, and the principles of CLUE negotiation and (2) seeks CLUE-enabled calls. 📱
- Endpoint: A CLUE-capable device that is the logical point of final termination through receiving, decoding, and rendering, and/or initiation through the capturing, encoding, and sending of media streams. An endpoint consists of one or more physical devices that source and sink media streams, and exactly one participant (as described in [RFC4353]) that, in turn, includes exactly one user agent [RFC3261]. Endpoints can be anything from multiscreen/multicamera rooms to handheld devices. 🏢📱
- Multipoint Control Unit (MCU): A CLUE-capable device that connects two or more endpoints together into one single multimedia conference [RFC7667]. An MCU includes a mixer (as defined in [RFC4353]), without the requirement per [RFC4353] to send media to each participant. 🎪
- Media: Any data that, after suitable encoding, can be conveyed over RTP, including audio, video, or timed text. 🎶
- Media Capture: A source of media, for example from one or more Capture Devices or constructed from other Media streams. 📸
- Media Consumer (MC): A CP (i.e., an Endpoint or an MCU) able to receive Capture Encodings. 👂
- Media Provider (MP): A CP (i.e., an Endpoint or an MCU) able to send Capture Encodings. 📤
- Stream: A Capture Encoding sent from an MP to an MC via RTP [RFC3550]. 🌊
3. Conventions 🎀
okay this part is the rulebook for the capitalized words 🙈 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here 💜 Anything in lowercase is just regular English, so no need to panic 😭
4. Overview of the CLUE Protocol ✨
okay bestie, so the CLUE protocol exists to make CLUE telepresence sessions happen 🎀 It's built to fix some Session Description Protocol (SDP) limitations. With plain SDP you just can't convey the info about the multimedia streams you need for that "being there" rendering experience 🦋 That info lives in the CLUE framework doc [RFC8845] and gets formally defined in the CLUE data model doc [RFC8846]. The CLUE protocol is the mechanism for swapping telepresence info between CPs. Mainly, it gives MPs the messages to advertise their telepresence capabilities, and MCs the messages to pick the telepresence options they want 💅
The CLUE protocol, as defined in this document and described below, is a stateful client-server XML-based application protocol 🧚 CLUE protocol messages travel on a reliable and ordered SCTP-over-DTLS transport channel connecting two CPs. The messages carry info taken from the XML-based CLUE data model [RFC8846]. There are three main communication phases you should know about 👑
- Establishment of the CLUE data channel: In this phase, the CLUE data channel setup happens. If it completes successfully, the CPs are able to communicate and start the initiation phase ✨
- Negotiation of the CLUE protocol version and extensions (initiation phase): The CPs connected via the CLUE data channel agree on the protocol version and extensions to be used during the telepresence session. Special CLUE messages do this job (
optionsandoptionsResponse). The negotiation of the version and extensions can be performed once during the CLUE session and only at this stage. At the end of that basic negotiation, each CP starts its activity as a CLUE MP and/or CLUE MC 🌸 - Description and negotiation of CLUE telepresence capabilities: In this phase, the MP-MC dialogues take place on the data channel by means of the CLUE protocol messages 🦋
As soon as the channel is ready, the CPs must agree on the protocol version and extensions to be used within the telepresence session 🫶 CLUE protocol version numbers have a major version number and a minor version number, both unsigned integers, separated by a dot. Minor version numbers denote backward-compatible changes within a given major version. Different major version numbers generally mean the protocol implementations might not interoperate 😭 So, to correctly establish a CLUE dialogue, the involved CPs must have a major version number in common (see Section 7 for further details). The subset of extensions allowed within the CLUE session is also decided in the initiation phase, and it only includes the extensions supported by both parties. Negotiating the version and extensions is part of the initiation phase. Here's how it goes: the CP that is the CLUE Channel Initiator (CI) sends a CLUE message (options) to the CP that is the Channel Receiver (CR), specifying the supported version and extensions. The CR then answers by selecting the subset of the CI extensions it can support and determining the protocol version to be used 💖
After the negotiation phase is completed, CPs describe and agree on the media flows to be exchanged. In many cases, CPs will want to both transmit and receive media 🌈 So in a call between two CPs (e.g., CPs A and B), there would be two separate message exchange sequences, as follows:
- the one needed to describe and set up the media streams sent from A to B, i.e., the dialogue between A's MP side and B's MC side.
- the one needed to describe and set up the media streams sent from B to A, i.e., the dialogue between B's MP side and A's MC side.
CLUE messages for the media session description and negotiation are designed with the MP side as the server side of the protocol, since it produces and provides media streams. The MC side is the client side, since it requests and receives media streams 🍓 The messages exchanged to set up the telepresence media session are described by focusing on a single MP-MC dialogue.
The MP first advertises its available media captures and encoding capabilities to the MC, along with its simultaneity constraints, according to the information model defined in [RFC8845] 📣 The CLUE message carrying the MP's multimedia offer is the advertisement message. It leverages the XML data model definitions provided in [RFC8846].
The MC selects the desired streams of the MP by using the configure message, which references the info carried in the previously received advertisement 🎯
Besides advertisement and configure, other messages were conceived to provide all the needed mechanisms and operations. They're detailed in the following sections 🧁
5. Protocol Messages ✨
CLUE protocol messages are textual XML-based messages that enable the configuration of the telepresence session 💜 The formal definition of these messages is in the XML schema in Section 9. This section includes non-normative excerpts of the schema to help describe it.
The XML definitions of the CLUE information provided in [RFC8846] are included within some CLUE protocol messages (namely the advertisement and configure messages), so they can use the concepts defined in [RFC8845] 🦋
The CLUE protocol messages are as follows:
optionsoptionsResponseadvertisementackconfigureconfigureResponse
While the options and optionsResponse messages are exchanged between the CPs in the initiation phase, the other messages are involved in MP-MC dialogues. Please note that the word "dialogue" as used in this document is not related to SIP's usage of the same term 🙈 It refers to message exchange sequences between a CLUE MP and a Clue MC.
Each CLUE message inherits a basic structure, as shown in the following excerpt (Figure 1) 💅
<xs:complexType name="clueMessageType" abstract="true">
<xs:sequence>
<xs:element name="clueId" type="xs:string" minOccurs="0"/>
<xs:element name="sequenceNr" type="xs:positiveInteger"/>
</xs:sequence>
<xs:attribute name="protocol" type="xs:string" fixed="CLUE"
use="required"/>
<xs:attribute name="v" type="versionType" use="required"/>
</xs:complexType>
<!-- VERSION TYPE -->
<xs:simpleType name="versionType">
<xs:restriction base="xs:string">
<xs:pattern value="[1-9][0-9]*\.[0-9]+" />
</xs:restriction>
</xs:simpleType>
Figure 1: Structure of a CLUE MessageHere's the info contained in each CLUE message 🌸
clueId: An optional XML element containing the identifier (in the form of a generic string) of the CP within the telepresence system.sequenceNr: An XML element containing the local message sequence number. The sender MUST increment the sequence number by one for each new message sent, and the receiver MUST remember the most recent sequence number received and send back a 402 error if it receives a message with an unexpected sequence number (e.g., sequence number gap, repeated sequence number, sequence number too small). The initial sequence number can be chosen randomly by each party 🎲protocol: A mandatory attribute set to"CLUE", identifying the protocol the messages refer to.v: A mandatory attribute carrying the version of the protocol. The content of thevattribute is the major version number, followed by a dot, and then the minor version number of the CLUE protocol in use. The major number cannot be0, and if it is more than one digit, it cannot start with a0. Allowed values of this kind are"1.3","2.0","20.44", etc. This document describes version1.0✨
Each CP is responsible for creating and updating up to three independent streams of sequence numbers in the messages it sends: (i) one for the messages sent in the initiation phase, (ii) one for the messages sent as an MP (if it is acting as an MP), and (iii) one for the messages sent as an MC (if it is acting as an MC) 🧠
In particular, CLUE response messages (optionsResponse, ack, configureResponse) derive from a base type that inherits from the clueMessageType, defined as follows (Figure 2) 🎀
<xs:complexType name="clueResponseType">
<xs:complexContent>
<xs:extension base="clueMessageType">
<xs:sequence>
<xs:element name="responseCode" type="responseCodeType"/>
<xs:element name="reasonString" type="xs:string" minOccurs="0"/>
</xs:sequence>
</xs:extension>
</xs:complexContent>
</xs:complexType>
Figure 2: Structure of CLUE Response MessagesThe elements <responseCode> and <reasonString> are populated as detailed in Section 5.7 🌈
5.1. 'options' 💖
okay so the 'options' message is the very first thing the CP that is the CI sends to the CP that is the CR, the moment the CLUE data channel is ready 🫶 besides the basic structure stuff, it tells the other side a few big things:
<mediaProvider>: a mandatory boolean set to "true" if the CP is able to act as an MP 🎀<mediaConsumer>: a mandatory boolean set to "true" if the CP is able to act as an MC 🦋<supportedVersions>: the list of supported versions 🌸<supportedExtensions>: the list of supported extensions ✨
here's the XML schema for the message (Figure 3) 👑
<!-- CLUE OPTIONS -->
<xs:complexType name="optionsMessageType">
<xs:complexContent>
<xs:extension base="clueMessageType">
<xs:sequence>
<xs:element name="mediaProvider" type="xs:boolean" />
<xs:element name="mediaConsumer" type="xs:boolean" />
<xs:element name="supportedVersions" type="versionsListType"
minOccurs="0"/>
<xs:element name="supportedExtensions"
type="extensionsListType"
minOccurs="0"/>
<xs:any namespace="##other" processContents="lax"
minOccurs="0"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
<!-- VERSIONS LIST TYPE -->
<xs:complexType name="versionsListType">
<xs:sequence>
<xs:element name="version" type="versionType" minOccurs="1"
maxOccurs="unbounded"/>
<xs:any namespace="##other" processContents="lax" minOccurs="0"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax"/>
</xs:complexType>
<!-- EXTENSIONS LIST TYPE -->
<xs:complexType name="extensionsListType">
<xs:sequence>
<xs:element name="extension" type="extensionType" minOccurs="1"
maxOccurs="unbounded"/>
<xs:any namespace="##other" processContents="lax" minOccurs="0"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax"/>
</xs:complexType>
<!-- EXTENSION TYPE -->
<xs:complexType name="extensionType">
<xs:sequence>
<xs:element name="name" type="xs:string" />
<xs:element name="schemaRef" type="xs:anyURI" />
<xs:element name="version" type="versionType" />
<xs:any namespace="##other" processContents="lax" minOccurs="0"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax"/>
</xs:complexType>Figure 3: Structure of a CLUE 'options' Message
ok the <supportedVersions> part is where it gets spicy 🌶️ it holds the list of versions the CI supports, each one in its own child <version> element. each <version> is a string: the major version number, a dot, then the minor version number (like 1.3 or 2.4) 💅 for each major version the CI supports, there MUST be exactly one <version> element, and it has to carry the highest minor version of that major, because all minor versions are backward compatible 🧚
if there's no <supportedVersions> in the 'options' message, the CI only supports the version in the "v" attribute plus everything with the same major number and a lower minor number. so if "v" says "3.4" and there's no list, that means major version 3, minors 3.0 through 3.4 🍓 and if <supportedVersions> IS there, at least one <version> element MUST be included. in that case the "v" attribute SHOULD be set to the largest minor version of the smallest major version in the list 🔥
why does "v" matter so much? an implementation that gets a major version higher than it understands is supposed to close the connection, because it could totally misread the messages 💀 the minor version matters less, since minors are defined to be both backward AND forward compatible, and you can always pull the value out of the <version> list anyway. what's more useful is knowing the highest minor version supported, since that shows the full feature set 🌈
the <supportedExtensions> element lists the extensions the CI supports. if it's missing from the 'options' message, the CI doesn't support anything beyond what's in its versions. every extension gets its own <extension> element, and each one has a name, an XML schema of reference where it's defined, and the version of the protocol it refers to 🦋
5.2. 'optionsResponse' 🎀
the 'optionsResponse' is what a CR sends back to a CI as a reply to the 'options' message 💌 it has a mandatory response code and a reason string that explains how the 'options' message went. if the responseCode is between 200 and 299 inclusive, the response MUST also include <mediaProvider>, <mediaConsumer>, <version>, and <commonExtensions> elements. for any other response code, it MAY include them 🫶
the <mediaProvider> and <mediaConsumer> elements are booleans about the roles (MP and MC), kind of like what the CI does in the 'options' message 🦋 the <version> element is the highest commonly supported version number, and its content MUST be a string made of the major version number, a dot, and the minor version number (like 1.3 or 2.4) 💅 and last but not least, the commonly supported extensions get copied into the <commonExtensions> element ✨
here's the structure (Figure 4) 👑
<!-- CLUE 'optionsResponse' -->
<xs:complexType name="optionsResponseMessageType">
<xs:complexContent>
<xs:extension base="clueResponseType">
<xs:sequence>
<xs:element name="mediaProvider" type="xs:boolean"
minOccurs="0"/>
<xs:element name="mediaConsumer" type="xs:boolean"
minOccurs="0"/>
<xs:element name="version" type="versionType" minOccurs="0"/>
<xs:element name="commonExtensions" type="extensionsListType"
minOccurs="0"/>
<xs:any namespace="##other" processContents="lax"
minOccurs="0"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>Figure 4: Structure of a CLUE 'optionsResponse' Message
once the CI gets the 'optionsResponse', the version to use is the one in its <version> element 🌸 all the CLUE messages after that MUST use that version number in the "v" attribute. and the extensions allowed in the CLUE dialogue are exactly the ones listed in the <commonExtensions> element of the 'optionsResponse' 🔥
5.3. 'advertisement' 📣
the 'advertisement' message is how each MP shows off its available media captures and related info to its MC 🎬 the MP sends an 'advertisement' as soon as it's ready after the initiation phase wraps up successfully, meaning the CPs have already agreed on the CLUE version and extensions 🤝 during one CLUE session, an MP can send new 'advertisement' messages to replace the old one, like if its telepresence media capabilities change mid-call 🧚 and a new 'advertisement' completely replaces the previous one. no leftovers, no drama 💅
the 'advertisement' structure is defined in the schema below (Figure 5). it holds elements that follow the CLUE data model and describe the MP's telepresence offer 🌈 those elements are:
- media captures (
<mediaCaptures>) 🎥 - encoding groups (
<encodingGroups>) 🍓 - capture scenes (
<captureScenes>) 🌸 - simultaneous sets (
<simultaneousSets>) 🦋 - global views (
<globalViews>) 👑 - represented participants (
<people>) 🫶
each of these is fully described in the CLUE framework document [RFC8845] and formally defined in the CLUE data model document [RFC8846] 📚
<!-- CLUE ADVERTISEMENT MESSAGE TYPE -->
<xs:complexType name="advertisementMessageType">
<xs:complexContent>
<xs:extension base="clueMessageType">
<xs:sequence>
<!-- mandatory -->
<xs:element name="mediaCaptures"
type="dm:mediaCapturesType"/>
<xs:element name="encodingGroups"
type="dm:encodingGroupsType"/>
<xs:element name="captureScenes"
type="dm:captureScenesType"/>
<!-- optional -->
<xs:element name="simultaneousSets"
type="dm:simultaneousSetsType" minOccurs="0"/>
<xs:element name="globalViews" type="dm:globalViewsType"
minOccurs="0"/>
<xs:element name="people"
type="dm:peopleType" minOccurs="0"/>
<xs:any namespace="##other" processContents="lax"
minOccurs="0"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>Figure 5: Structure of a CLUE 'advertisement' Message
5.4. 'ack' 🙈
the 'ack' is the MC's way of saying "got it" to an MP's 'advertisement' 💖 it's sent by an MC to an MP to acknowledge an 'advertisement' message. as you can see in the schema below (Figure 6), the 'ack' has a response code and can also have a reason string describing how the processing of the 'advertisement' went 🎀 the <advSequenceNr> element carries the sequence number of the 'advertisement' the 'ack' is talking about ✨
<!-- 'ack' MESSAGE TYPE -->
<xs:complexType name="advAcknowledgementMessageType">
<xs:complexContent>
<xs:extension base="clueResponseType">
<xs:sequence>
<xs:element name="advSequenceNr" type="xs:positiveInteger"/>
<xs:any namespace="##other" processContents="lax"
minOccurs="0"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>Figure 6: Structure of a CLUE 'ack' Message
5.5. 'configure' 🎀
okay so the 'configure' message is how an MC tells an MP which advertised captures it actually wants to receive 🫶 it's giving "yes queen, send me these" energy. The MC MUST send a 'configure' after it gets an 'advertisement', and it sends one again every time it wants other captures that the MP has already advertised 💅 the shape of the message is in Figure 7, bestie.
<!-- CLUE 'configure' MESSAGE TYPE -->
<xs:complexType name="configureMessageType">
<xs:complexContent>
<xs:extension base="clueMessageType">
<xs:sequence>
<!-- mandatory fields -->
<xs:element name="advSequenceNr" type="xs:positiveInteger"/>
<xs:element name="ack" type="successResponseCodeType"
minOccurs="0"/>
<xs:element name="captureEncodings"
type="dm:captureEncodingsType" minOccurs="0"/>
<xs:any namespace="##other" processContents="lax"
minOccurs="0"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
Figure 7: Structure of a CLUE 'configure' Messagethe <advSequenceNr> element holds the sequence number of the 'advertisement' that this 'configure' is talking about 🦋 so the MP knows exactly which ad you're replying to.
the <ack> element is optional 🌸 when it shows up, it carries a success response code (the ones from Section 5.7), and that means the 'configure' is also acknowledging the referred advertisement with success. it's basically a 'configure+ack' moment ✨ one thing though: the <ack> element MUST NOT be there if an 'ack' message for that specific advertisement sequence number has already gone back to the MP. no double-acking, periodt 🙅♀️
the main event is the list of capture encodings in the <captureEncodings> element (go check [RFC8846] for how that's defined) 👑 it holds a sequence of capture encodings, and each one represents a stream that's going to get set up.
5.6. 'configureResponse' 💜
the 'configureResponse' is the MP's reply to the MC, spilling the tea on how it handled the requests in the 'configure' it just got 🍓 it's got a response code (plus an optional reason string) saying whether things worked or didn't, and if they didn't, the failure details come along too. the <confSequenceNr> element right after that tells you which 'configure' the reply is for 🧚
here's the big rule: there is no partial execution of commands 🔥 like, if the MP can understand every selected capture encoding except one, the whole thing fails and nothing gets instantiated. all or nothing, no half-baked streams.
<!-- 'configureResponse' MESSAGE TYPE -->
<xs:complexType name="configureResponseMessageType">
<xs:complexContent>
<xs:extension base="clueResponseType">
<xs:sequence>
<xs:element name="confSequenceNr"
type="xs:positiveInteger" />
<xs:any namespace="##other" processContents="lax"
minOccurs="0"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax" />
</xs:extension>
</xs:complexContent>
</xs:complexType>
Figure 8: Structure of a CLUE 'configureResponse' Message5.7. Response Codes and Reason Strings 🌈
okay, response codes are three digits, and the FIRST digit is the one that tells you the whole vibe 💅 a code starting with "2" means success ✨ anything that doesn't start with "2" or "1" is an error response, meaning something went wrong while the CLUE request was being processed. "3" codes are about the XML in the message being off (like "Bad syntax" or "Invalid value") 🙈 and "4" codes are about CLUE protocol semantics (think "Invalid sequencing" or "Version not supported") 💀 200, 300, and 400 are the super generic ones in their categories. more codes can show up later, either in future protocol versions or through the extension mechanism, and when they do, they MUST be registered with IANA 🧾. they also MUST NOT override the codes defined in this document, and they MUST respect what the first digit means. no rule-breaking allowed, bestie 😤
this document doesn't define any response codes starting with "1", and those aren't allowed in major version 1 of CLUE 🙅♀️ the 100 to 199 range (inclusive) is saved for future major versions, for delayed or incomplete operations if they ever need it 🕰️ codes starting with "5" through "9" are also reserved for future major versions to make brand new classes of responses, and they're not allowed in version 1 either. and codes starting with "0"? a hard no 🚫
the response codes and reason strings for CLUE version 1 are in Table 1 below 🦋 the "Description" text from the table can go into the <reasonString> element of a response message. implementations can (and are encouraged to!) add more specific descriptions of an error when they can, so go be detailed 🫶
+==========+===============+========================================+
| Response | Reason String | Description |
| Code | | |
+==========+===============+========================================+
| 200 | Success | The request has been |
| | | successfully processed. |
+----------+---------------+----------------------------------------+
| 300 | Low-level | A generic low-level request |
| | request error | error has occurred. |
+----------+---------------+----------------------------------------+
| 301 | Bad syntax | The XML syntax of the message |
| | | is not correct. |
+----------+---------------+----------------------------------------+
| 302 | Invalid value | The message contains an |
| | | invalid parameter value. |
+----------+---------------+----------------------------------------+
| 303 | Conflicting | The message contains values |
| | values | that cannot be used together. |
+----------+---------------+----------------------------------------+
| 400 | Semantic | The received CLUE protocol |
| | errors | message contains semantic |
| | | errors. |
+----------+---------------+----------------------------------------+
| 401 | Version not | The protocol version used in |
| | supported | the message is not supported. |
+----------+---------------+----------------------------------------+
| 402 | Invalid | The received message contains |
| | sequencing | an unexpected sequence number |
| | | (e.g., sequence number gap, |
| | | repeated sequence number, or |
| | | sequence number outdated). |
+----------+---------------+----------------------------------------+
| 403 | Invalid | The clueId used in the |
| | identifier | message is invalid or |
| | | unknown. |
+----------+---------------+----------------------------------------+
| 404 | Advertisement | The sequence number of the |
| | expired | advertisement the 'configure' |
| | | message refers to is out of |
| | | date. |
+----------+---------------+----------------------------------------+
| 405 | Subset choice | The subset choice is not |
| | not allowed | allowed for the specified |
| | | Multiple Content Capture. |
+----------+---------------+----------------------------------------+
Table 1: CLUE Response Codes6. Protocol State Machines ✨
Okay bestie, this is where we get into how CLUE actually moves 💅. CLUE is an application protocol used between two CPs 🎀 to properly configure a multimedia telepresence session. Its messages flow over the CLUE data channel, which is an SCTP-over-DTLS channel set up as shown in [RFC8850]. We're going to look at three state machines: the CP (Figure 9), the MP role (Figure 10 in Section 6.1), and the MC role (Figure 11 in Section 6.2). Endpoints usually want to send and receive media, so they act as both an MP and an MC 🦋. That means there are often two sets of messages flowing in opposite directions, but lowkey those two flows don't interact with each other 🙈. We're only covering the CLUE application logic here. How CLUE and SDP negotiation work together for the media streams being exchanged is discussed in [RFC8848].
+----+
+---------------------->|IDLE|<----------------------------+
| +-+--+ |
| | |
| | start |
| | channel |
| v |
| channel error / +--------+ |
| session ends | CHANNEL| |
+----------------------+ SETUP | |
| +--+-----+ |
| | |
| | channel |
| | established |
| channel error / v OPTIONS phase |
| session ends +-------+ failure |
+-----------------------+OPTIONS+--------------------------+
| +-+-----+
| |
| | OPTIONS phase
| | success
| v
| channel error / +---------+
| session ends | ACTIVE |
+----------------------+ |
| +----+ +------------------+
| | MP | | send/receive |
| +----+ | CLUE messages |
| |<-----------------+
| +----+ |
| | MC | |
| +----+ |
| |
+---------+
Figure 9: CLUE Participant State MachineThe main state machines focus on the behavior of the CP acting as a CLUE CI/CR 🌸.
The starting point is the IDLE state 🍓. While in IDLE, the CLUE data channel is not set up and no CLUE-controlled media is being exchanged between the two CLUE-capable devices in question. (If there's already an ongoing exchange of media streams, those streams are not CLUE controlled right now, so no drama there ✨.)
When the CLUE data channel gets set up ("start channel"), the CP hops from IDLE to the CHANNEL SETUP state 🧚.
If the CLUE data channel is set up successfully ("channel established"), the CP moves from CHANNEL SETUP to the OPTIONS state. But if a "channel error" happens, it bounces back to IDLE. That same move happens if the CLUE-enabled telepresence session ends ("session ends"), which is when an SDP negotiation removes the CLUE data channel 💀.
When the CP is in the OPTIONS state, it's handling the initiation phase, where both parts agree on the version and extensions they'll use in the rest of the CLUE message exchange. If the CP is the CI, it sends an options message and waits for the optionsResponse message. If the CP is the CR, it waits for the options message and, the second it arrives, replies with the optionsResponse message 🫶. If the negotiation completes successfully ("OPTIONS phase success"), the CP moves from OPTIONS to the ACTIVE state 👑. If the initiation phase fails ("OPTIONS phase failure"), the CP goes back to IDLE. The initiation phase might fail for one of these reasons:
- The CI receives an
optionsResponsewith an error response code. - The CI does not receive any
optionsResponse, and a timeout error is raised. - The CR does not receive any
options, and a timeout error is raised.
When the CP is in the ACTIVE state, it starts up the sub-state machines it needs (the MP state machine and the MC state machine) based on the roles it plays in the telepresence session 🌈. Those roles were declared earlier in the options and optionsResponse messages from the initiation phase (see Sections 5.1 and 5.2 for the details). While in ACTIVE, the CP hands off the sending and processing of CLUE messages to the right MP/MC sub-state machines. If the CP receives another options or optionsResponse message at this point, it MUST ignore that message and stay in the ACTIVE state 🙅♀️.
6.1. Media Provider's State Machine
As soon as the MP sub-state machine (Figure 10) gets activated, it's in the ADV state 🦋. In ADV, the MP prepares an advertisement message that reflects its actual telepresence capabilities.
+-----+
+------------>| ADV |<-------------------+
| +-+---+ |
| advertisement| NACK received |
| sent| |
| v |
changed| +--------+ |
telepresence+-------------+WAIT FOR+-----------------+
settings| +----------+ ACK |
| |configure +-+------+
| | +ack |
| |received |ack received
| | v
| | +--------+
+-------------+WAIT FOR|
| | | CONF |
| | +-+------+<-----------------------------+
| | | |
| | |configure received |
| | v |
| +--------->+---------+ error configureResponse sent|
+-------------+CONF |-----------------------------+
| +--------->|RESPONSE +
| | +---------+
| | |
| | |successful
| | |configureResponse
| | |sent
| | |
| | |
| |configure |
| |received v
| | +-----------+
| +----------+ESTABLISHED|
+-------------+-----------+
Figure 10: Media Provider's State MachineAfter the advertisement has been sent ("advertisement sent"), the MP goes from ADV to WAIT FOR ACK 📨. If an ack message with a successful response code shows up ("ack received"), the MP moves on to WAIT FOR CONF ✅. If a NACK arrives (that's an ack message with an error response code), the MP goes back to ADV to prepare a fresh advertisement 😤. While in WAIT FOR ACK, if a configure message with the <ack> element set to "200" arrives ("configure+ack received"), the MP skips right over to CONF RESPONSE ⭐. Any configure+ack messages that refer to out-of-date advertisements (ones with a sequence number lower than the highest one generated so far) MUST be ignored, which means they don't trigger any state transition at all 🙃. And if the MP's telepresence settings change while it's in WAIT FOR ACK ("changed telepresence settings"), it switches from WAIT FOR ACK back to ADV to create a new advertisement.
While the MP is in WAIT FOR CONF, it keeps listening on the channel for a configure request from the MC 👀. When a configure arrives ("configure received"), the MP switches to CONF RESPONSE. If the telepresence settings change in the meantime ("changed telepresence settings"), the MP moves from WAIT FOR CONF back to ADV so it can create a new advertisement to send to the MC.
The MP in CONF RESPONSE processes the received configure in order to produce a configureResponse message 🧁. If the MP processes the MC's configuration successfully, it sends a 200 configureResponse ("successful configureResponse sent") and moves on to ESTABLISHED 🎉. If there are errors while processing the configure, the MP sends a configureResponse carrying an error response code and goes back to WAIT FOR CONF to wait for a new configuration request 🥲. Finally, if the MP's telepresence settings change ("changed telepresence settings"), the MP switches to ADV.
The MP in ESTABLISHED has successfully negotiated the media streams with the MC using CLUE messages 💜. If the MP's telepresence settings change ("changed telepresence settings"), it moves back to ADV. In ESTABLISHED, the CLUE-controlled media streams of the session are the ones described in the last successfully processed configure message.
Messages that aren't shown for a state don't cause that state to change, so if it's not in the diagram, it's not a trigger, periodt 💅.
6.2. Media Consumer's State Machine 🎀
okay so the MC has its own little state machine too, and it's giving main character energy 💅. As soon as the sub-state machine of the MC (Figure 11) gets activated, it starts out in the WAIT FOR ADV state 🌸. Chilling there means it's waiting for an 'advertisement' from the MP. When that 'advertisement' shows up ("ADV received"), the MC moves over to the ADV PROCESSING state ✨. If nothing has arrived yet, it just stays in WAIT FOR ADV. Patience is a vibe 🫶.
+----------+
| WAIT FOR |
| ADV |
+----+-----+<--------+
| |
advertisement| NACK sent|
received| |
v |
+-----------+--------+
| ADV +
| PROCESSING|<-----------------------+
+-+-----+---+ |
| | |
configure+ack | | ack |
sent | | sent |
| v |
| +-----+ |
| |CONF | advertisement received |
+----------------------->| +-------------------------+
|error | +--+--+ |
|configureResponse | | |
|received | |configure |
| | |sent |
| | | |
| v v advertisement |
+------------------+---------------+ received |
+--------->| WAIT FOR +---------------------+
| | CONF RESPONSE+ |
| +-------+-------+ |
| | |
| | |
| |successful |
| |configureResponse |
|configure v |
|sent +-----------+ advertisement received|
+------------+ESTABLISHED+-----------------------+
+-----------+
Figure 11: Media Consumer's State MachineOnce we're in ADV PROCESSING, the MC parses the 'advertisement' 🔍. If it gets processed successfully, there are two possible plot twists 😭. In the first one, the MC sends back a successful 'ack' to the MP ("ack sent") and heads into the CONF state. This usually happens when the MC needs a little more time to put together the 'configure' that goes with the 'advertisement' it just got 🧚. In the second one, the MC is ready right away, so it prepares a 'configure' and sends it straight back to the MP. That message has the <ack> element set to 200 ("configure+ack sent"), and it lets the MC skip ahead directly to WAIT FOR CONF RESPONSE 🚀.
Now, if processing the 'advertisement' fails (bad syntax, missing XML elements, that kind of drama 💀), the MC sends a NACK to the MP. That's an 'ack' carrying an error response code 🙈. It can also optionally explain what went wrong with a proper reason phrase. In this case ("NACK sent"), the MC goes back to WAIT FOR ADV and waits for a fresh 'advertisement' 🌈.
When the MC is in CONF, it gets the 'configure' request ready to send to the MP, based on the 'advertisement' it already acked 🍓. Once the 'configure' is out the door ("configure sent"), the MC moves into WAIT FOR CONF RESPONSE. If a brand new 'advertisement' comes in during all this ("advertisement received"), the MC goes back to ADV PROCESSING 🦋.
In WAIT FOR CONF RESPONSE, the MC is waiting on the MP to reply to whatever 'configure' or 'configure+ack' it sent 👑. If a 200 'configureResponse' comes back ("successful configureResponse received"), that means the MP and the MC agreed on which media streams they're sharing, and the MC can move on to ESTABLISHED 🥳. If an error response comes back instead ("error configureResponse received"), the MC returns to CONF so it can craft a new 'configure' request 😤. And if a new 'advertisement' lands while it's waiting here, the MC switches over to ADV PROCESSING 💜.
When the MC is in ESTABLISHED, the telepresence session has been set up at the CLUE application level, following the MC's preferences ✨. Both sides have agreed on the CLUE-controlled media streams for the call and know all about them 🫶. Even while things are going well, the MC might want to tweak some call settings. If so, it sends a new 'configure' ("configure sent") and moves to WAIT FOR CONF RESPONSE to wait for the new 'configureResponse' 🔥. On the flip side, if the MP sends a new 'advertisement' while the MC is in ESTABLISHED ("advertisement received"), it means something changed on the MP's end, so the MC moves to ADV PROCESSING 👀.
Messages that aren't shown for a given state don't change that state 🌸.
7. Versioning ✨
okay bestie, versioning time 🧠💖. CLUE protocol messages are XML messages that have to follow the CLUE protocol XML schema [RFC8846] 📜. The protocol version matches the schema version. Both the client and the server have to check that the messages they receive comply with the CLUE protocol XML schema 🔍. If a message doesn't pass that check, it can't be processed any further 💀.
The client and the server can only talk to each other if they share exactly the same XML schema 🫂. That schema is tied to the CLUE URN urn:ietf:params:xml:ns:clue-protocol 🎀. If every CLUE-enabled device is using that same schema, schema mismatches won't cause interoperability problems, and that's the dream 🌈.
This document defines version 1.0 of the CLUE protocol schema, written in XML schema version 1.0 [W3C.REC-xml-20081126] 💅. The way versions are used here is similar in spirit to the Extensible Messaging and Presence Protocol (XMPP) [RFC6120] 🦋. A version number has a major part and a minor part, and each one is a non-negative integer 🔢. A change to the major version means the change is not interoperable 😱. A change to the minor version means a schema change that stays backward compatible, either by ignoring unknown XML elements or through some other backward-compatible tweak ✨.
The minor versions of the XML schema MUST be backward compatible, and that's not just about the schema: they have to be backward compatible semantically and procedurally too 🫶. In practice that means each new minor version adds features and functionality on top of the previous ones, in an incremental way, without messing with the basic rules of the earlier version 🍓. So say an MP can "speak" version 1.5 of the protocol, but the MC only understands version 1.4. The MP should have no problem dropping back to version 1.4 and not using any of the 1.5 features 🧚. Version 1.4 is the one that gets spoken, and it has to show up in the v attribute of the CLUE messages that follow 💜. So in this example, the MP MUST use version 1.4 👑. That said, the general IETF "robustness principle" applies too, which says an implementation must be conservative in its sending behavior and liberal in its receiving behavior [RFC1122] 🌸. With that in mind, CPs MUST ignore unknown elements or attributes that aren't part of the negotiated protocol version and its related extensions 🙈✨.
8. Extensions ✨
okay so the standard CLUE protocol XML schema is already super thorough 💅 but new needs can pop up, and that's when we design extensions 🦋 Extensions cover information and behaviors that aren't described in a certain version of the protocol and aren't specified in its related RFC document 🌸 They can be used in a CLUE dialogue, but they MUST be negotiated in the CLUE initiation phase 🫶 Extensions can relate to:
- new information, to be carried in the existing messages. For example, more fields may be added within an existing message.
- new messages. This is the case if there is no proper message for a certain task, so a brand new CLUE message needs to be defined.
As for the first category, there's a split between protocol-specific and data model information 👑 CLUE messages are envelopes carrying both of the following:
- XML elements defined within the CLUE protocol XML schema itself (protocol-specific information).
- other XML elements compliant to the CLUE data model schema (data model information).
When new protocol-specific information is needed somewhere in the protocol messages, it can be added in place of the <any> elements and <anyAttribute> elements the protocol schema has set up for this 💖 The policy the protocol schema currently uses for handling <any> and <anyAttribute> elements is:
elementFormDefault="qualified"attributeFormDefault="unqualified"
The new information must be qualified by namespaces other than "urn:ietf:params:xml:ns:clue-protocol" (the protocol URN) and "urn:ietf:params:xml:ns:clue-info" (the data model URN) ✨ Elements or attributes from unknown namespaces MUST be ignored 🙈
The other matter is data model information 🍓 Data model information is defined by the XML schema associated with the URN "urn:ietf:params:xml:ns:clue-info". Heads up, there are extensibility issues for the XML elements in that schema too, and those get handled with <any> and <anyAttribute> placeholders 💜 New information within data model elements can be added in place of those <any> and <anyAttribute> schema elements, as long as it's properly namespace qualified 🧚
And now the second category of extensions: "extra" CLUE protocol messages, which are messages not envisioned in the latest standard version of the schema, might be needed 😭 In that case, the messages and their associated behavior should be defined in external documents that both communication parties must be aware of 🎀
As shown in Figure 12, the fields of the <extension> element (whether it's new information or new messages) take the following values:
- a name.
- an external XML schema defining the XML information and/or the XML messages representing the extension.
- the major standard version of the protocol that the extension refers to.
<xs:complexType name="extensionType">
<xs:sequence>
<xs:element name="name" type="xs:string" />
<xs:element name="schemaRef" type="xs:anyURI"/>
<xs:element name="version" type="versionType"/>
<xs:any namespace="##other" processContents="lax"
minOccurs="0"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax"/>
</xs:complexType>Figure 12: The <extension> Element
The <extension> element described above is carried within the 'options' and 'optionsResponse' messages to represent the extensions supported by both the CI and the CR 🌈
Extensions MUST be defined in a separate XML schema file and MUST be provided with a companion document describing their semantics and use 📚
8.1. Extension Example
An example of an extension might be a "new" capture attribute (a capture attribute that isn't envisioned in the current standard defining the CLUE data model in [RFC8846]) needed to further describe a video capture 🎥✨
The CLUE data model document [RFC8846] envisions the possibility of adding this kind of "extra" information to the description of a video capture 💅 For convenience, the XML definition of a video capture taken from [RFC8846] is shown in Figure 13 below.
<!-- VIDEO CAPTURE TYPE -->
<xs:complexType name="videoCaptureType">
<xs:complexContent>
<xs:extension base="tns:mediaCaptureType">
<xs:sequence>
<xs:any namespace="##other" processContents="lax"
minOccurs="0"
maxOccurs="unbounded"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>Figure 13: XML Definition of a CLUE Video Capture
So according to that definition, a video capture might have, after the set of generic media capture attributes, a set of new attributes defined elsewhere, meaning in an XML schema defining an extension 🦋 The XML schema defining the extension might look like this (Figure 14):
<?xml version="1.0" encoding="UTF-8" ?>
<xs:schema version="1.0"
targetNamespace="https://example.extensions.com/myVideoExtensions"
xmlns:xs="https://www.w3.org/2001/XMLSchema"
xmlns="https://example.extensions.com/myVideoExtensions"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<!--
This is the new element to be put in place of the <any>
element in the video capture definition
of the CLUE data model schema
-->
<xs:element name="myVideoExtension">
<xs:complexType>
<xs:sequence>
<xs:element ref="newVideoAttribute1"/>
<xs:element ref="newVideoAttribute2"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="newVideoAttribute1" type="xs:string"/>
<xs:element name = "newVideoAttribute2" type = "xs:boolean"/>
</xs:schema>Figure 14: XML Schema Defining an Extension
Using the extension above, a video capture can be described further in the advertisement with the <myVideoExtension> element, which contains two extra pieces of information (<newVideoAttribute1> and <newVideoAttribute2>) on top of the attributes envisioned for a generic media capture 🌸 As this document says, both participants must be aware of the extension schema and its related semantics to use such an extension, and they must negotiate it via the 'options' and 'optionsResponse' messages 🔥
9. XML Schema ✨
okay bestie, this is the part where the messages get their official outfit 👑 the XML schema defines every CLUE message, and it's literally Figure 15 💖
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="https://www.w3.org/2001/XMLSchema"
xmlns="urn:ietf:params:xml:ns:clue-protocol"
xmlns:dm="urn:ietf:params:xml:ns:clue-info"
version="1.0"
targetNamespace="urn:ietf:params:xml:ns:clue-protocol"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<!-- Import data model schema -->
<xs:import namespace="urn:ietf:params:xml:ns:clue-info"/>
<!-- ELEMENT DEFINITIONS -->
<xs:element name="options" type="optionsMessageType" />
<xs:element name="optionsResponse"
type="optionsResponseMessageType"/>
<xs:element name="advertisement" type="advertisementMessageType"/>
<xs:element name="ack" type="advAcknowledgementMessageType"/>
<xs:element name="configure" type="configureMessageType"/>
<xs:element name="configureResponse"
type="configureResponseMessageType"/>
<!-- CLUE MESSAGE TYPE -->
<xs:complexType name="clueMessageType" abstract="true">
<xs:sequence>
<xs:element name="clueId" type="xs:string" minOccurs="0" />
<xs:element name="sequenceNr" type="xs:positiveInteger" />
</xs:sequence>
<xs:attribute name="protocol" type="xs:string" fixed="CLUE"
use="required" />
<xs:attribute name="v" type="versionType" use="required" />
</xs:complexType>
<!-- CLUE RESPONSE TYPE -->
<xs:complexType name="clueResponseType">
<xs:complexContent>
<xs:extension base="clueMessageType">
<xs:sequence>
<xs:element name="responseCode" type="responseCodeType" />
<xs:element name="reasonString" type="xs:string"
minOccurs="0"/>
</xs:sequence>
</xs:extension>
</xs:complexContent>
</xs:complexType>
<!-- VERSION TYPE -->
<xs:simpleType name="versionType">
<xs:restriction base="xs:string">
<xs:pattern value="[1-9][0-9]*\.[0-9]+" />
</xs:restriction>
</xs:simpleType>
<!-- RESPONSE CODE TYPE -->
<xs:simpleType name="responseCodeType">
<xs:restriction base="xs:integer">
<xs:pattern value="[1-9][0-9][0-9]" />
</xs:restriction>
</xs:simpleType>
<!-- SUCCESS RESPONSE CODE TYPE -->
<xs:simpleType name="successResponseCodeType">
<xs:restriction base="xs:integer">
<xs:pattern value="2[0-9][0-9]" />
</xs:restriction>
</xs:simpleType>
<!-- CLUE OPTIONS -->
<xs:complexType name="optionsMessageType">
<xs:complexContent>
<xs:extension base="clueMessageType">
<xs:sequence>
<xs:element name="mediaProvider" type="xs:boolean"/>
<xs:element name="mediaConsumer" type="xs:boolean"/>
<xs:element name="supportedVersions"
type="versionsListType"
minOccurs="0" />
<xs:element name="supportedExtensions"
type="extensionsListType" minOccurs="0"/>
<xs:any namespace="##other" processContents="lax"
minOccurs="0"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
<!-- VERSIONS LIST TYPE -->
<xs:complexType name="versionsListType">
<xs:sequence>
<xs:element name="version" type="versionType" minOccurs="1"
maxOccurs="unbounded"/>
<xs:any namespace="##other" processContents="lax"
minOccurs="0"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax" />
</xs:complexType>
<!-- EXTENSIONS LIST TYPE -->
<xs:complexType name="extensionsListType">
<xs:sequence>
<xs:element name="extension" type="extensionType"
minOccurs="1"
maxOccurs="unbounded"/>
<xs:any namespace="##other" processContents="lax"
minOccurs="0"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax" />
</xs:complexType>
<!-- EXTENSION TYPE -->
<xs:complexType name="extensionType">
<xs:sequence>
<xs:element name="name" type="xs:string" />
<xs:element name="schemaRef" type="xs:anyURI" />
<xs:element name="version" type="versionType" />
<xs:any namespace="##other" processContents="lax"
minOccurs="0"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax"/>
</xs:complexType>
<!-- CLUE 'optionsResponse' -->
<xs:complexType name="optionsResponseMessageType">
<xs:complexContent>
<xs:extension base="clueResponseType">
<xs:sequence>
<xs:element name="mediaProvider" type="xs:boolean"
minOccurs="0"/>
<xs:element name="mediaConsumer" type="xs:boolean"
minOccurs="0"/>
<xs:element name="version" type="versionType"
minOccurs="0"/>
<xs:element name="commonExtensions"
type="extensionsListType" minOccurs="0"/>
<xs:any namespace="##other" processContents="lax"
minOccurs="0"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
<!-- CLUE ADVERTISEMENT MESSAGE TYPE -->
<xs:complexType name="advertisementMessageType">
<xs:complexContent>
<xs:extension base="clueMessageType">
<xs:sequence>
<!-- mandatory -->
<xs:element name="mediaCaptures"
type="dm:mediaCapturesType"/>
<xs:element name="encodingGroups"
type="dm:encodingGroupsType"/>
<xs:element name="captureScenes"
type="dm:captureScenesType"/>
<!-- optional -->
<xs:element name="simultaneousSets"
type="dm:simultaneousSetsType" minOccurs="0"/>
<xs:element name="globalViews" type="dm:globalViewsType"
minOccurs="0"/>
<xs:element name="people"
type="dm:peopleType" minOccurs="0"/>
<xs:any namespace="##other" processContents="lax"
minOccurs="0"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
<!-- 'ack' MESSAGE TYPE -->
<xs:complexType name="advAcknowledgementMessageType">
<xs:complexContent>
<xs:extension base="clueResponseType">
<xs:sequence>
<xs:element name="advSequenceNr"
type="xs:positiveInteger"/>
<xs:any namespace="##other" processContents="lax"
minOccurs="0"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
<!-- CLUE 'configure' MESSAGE TYPE -->
<xs:complexType name="configureMessageType">
<xs:complexContent>
<xs:extension base="clueMessageType">
<xs:sequence>
<!-- mandatory fields -->
<xs:element name="advSequenceNr"
type="xs:positiveInteger"/>
<xs:element name="ack" type="successResponseCodeType"
minOccurs="0"/>
<xs:element name="captureEncodings"
type="dm:captureEncodingsType" minOccurs="0"/>
<xs:any namespace="##other" processContents="lax"
minOccurs="0"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
<!-- 'configureResponse' MESSAGE TYPE -->
<xs:complexType name="configureResponseMessageType">
<xs:complexContent>
<xs:extension base="clueResponseType">
<xs:sequence>
<xs:element name="confSequenceNr"
type="xs:positiveInteger"/>
<xs:any namespace="##other" processContents="lax"
minOccurs="0"/>
</xs:sequence>
<xs:anyAttribute namespace="##other" processContents="lax"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:schema>Figure 15: Schema Defining CLUE Messages 🎀
10. Call Flow Example 🎬
okay so this section walks through the CLUE protocol messages in one call flow 💅 we only show one direction of media, because the other direction is precisely symmetric (mirror image, it's giving 🪞)
+-----+ +-----+
| | | |
| CP1 | | CP2 |
| | | |
+--+--+ +--+--+
| |
| 1. options |
+-------------------->|
| |
| |
|2. optionsResponse |
|<--------------------+
| |
| |
|3. advertisement |
+-------------------->|
| |
| |
|4. configure+ack |
|<--------------------+
| |
| |
|5. configureResponse |
+-------------------->|
| |
| |
|6. advertisement |
+-------------------->|
| |
| |
| 7. ack |
|<--------------------+
| |
| |
|8. configure |
|<--------------------+
| |
| |
|9. configureResponse |
+-------------------->|
| |
| |
. .
. .
. .Two CPs, CP1 and CP2, have already set up the CLUE channel according to [RFC8850] ✨ CP1 is the CI, and CP2 is the CR 🦋
- The initiation phase kicks off (negotiating the CLUE protocol version and extensions 🤝). CP1, as the CI, sends CP2 an
optionsmessage listing what it supports (Section 10.1). CP1 supports (i) version 1.4 with extensions E1, E2, and E3 and (ii) version 2.7 with extensions E4 and E5. Because of that, CP1 sends anoptionsmessage with the "v" attribute set to "1.4" and spells out every supported version and extension in the matching fields. In thatoptionsmessage, CP1 also says it intends to act as both an MP and an MC 🌈 - CP2 supports versions 3.0, 2.9, and 1.9, each with no extensions. Version 2.7 is the best common choice 🫶 So CP2 replies with an
optionsResponsethat picks only version 2.7, and leaves the extensions part blank to say no extensions will be used in this session (Section 10.2). CP2 also says it intends to act as both an MP and an MC 🙌
After the initiation phase is done, CP1 starts as the MP and CP2 starts as the MC 🎀 For simplicity, the rest of the flow only focuses on the dialogue between MP CP1 and MC CP2 💖
- CP1 advertises a telepresence setup like the one in [RFC8846], Section 27 💅: three main video streams from three cameras (the central camera can capture a zoomed-out view of the whole room), a multicontent capture of the loudest segment of the room, and one audio capture for the audio of the whole room (Section 10.3).
- CP2 receives CP1's
advertisement, processes it, and sends back aconfigure+ackdeclaring its interest in the multicontent capture and the audio capture only (Section 10.4) 👀 - CP1 answers the
configure+ackwith aconfigureResponsethat includes a 200 (Success) response code, accepting all of CP2's requests (Section 10.5) ✅ - To reflect the changes in its telepresence offer, CP1 issues a new
advertisement(Section 10.6), this time also adding a composed capture: a big picture for the current speaker plus two picture-in-picture boxes for the previous speakers 📺 (see [RFC8846], Section 28 for more telepresence details 🧚) - CP2 acknowledges that second
advertisementwith anack(Section 10.7) 💌 - Later in the session, CP2 changes the media streams it requests from CP1 by sending a
configurethat replaces the previously selected video streams with the new composed streams CP1 advertised (Section 10.8) 🔄 The streams from the previous configuration keep flowing during the reconfiguration, no drama 😌 - Finally, CP1 accepts CP2's latest request with a
configureResponse(Section 10.9) 🎉
We should also point out that in the depicted flow, three distinct sequence number spaces per sender are involved (two of which appear in the snippets, since we only show one direction of media) 🔢 The jump between the sequence number values in Sections 10.2 and 10.3 is hence correct ✨
10.1. CLUE Message No. 1: 'options' 💖
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<options xmlns="urn:ietf:params:xml:ns:clue-protocol"
xmlns:ns2="urn:ietf:params:xml:ns:clue-info"
xmlns:ns3="urn:ietf:params:xml:ns:vcard-4.0"
xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance"
protocol="CLUE" v="1.4">
<clueId>CP1</clueId>
<sequenceNr>51</sequenceNr>
<mediaProvider>true</mediaProvider>
<mediaConsumer>true</mediaConsumer>
<supportedVersions>
<version>1.4</version>
<version>2.7</version>
</supportedVersions>
<supportedExtensions>
<extension>
<name>E1</name>
<schemaRef>URL_E1</schemaRef>
<version>1.4</version>
</extension>
<extension>
<name>E2</name>
<schemaRef>URL_E2</schemaRef>
<version>1.4</version>
</extension>
<extension>
<name>E3</name>
<schemaRef>URL_E3</schemaRef>
<version>1.4</version>
</extension>
<extension>
<name>E4</name>
<schemaRef>URL_E4</schemaRef>
<version>2.7</version>
</extension>
<extension>
<name>E5</name>
<schemaRef>URL_E5</schemaRef>
<version>2.7</version>
</extension>
</supportedExtensions>
</options>10.2. CLUE Message No. 2: 'optionsResponse' 🌸
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<optionsResponse xmlns="urn:ietf:params:xml:ns:clue-protocol"
xmlns:ns2="urn:ietf:params:xml:ns:clue-info"
xmlns:ns3="urn:ietf:params:xml:ns:vcard-4.0"
xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance"
protocol="CLUE" v="1.4">
<clueId>CP2</clueId>
<sequenceNr>62</sequenceNr>
<responseCode>200</responseCode>
<reasonString>Success</reasonString>
<mediaProvider>true</mediaProvider>
<mediaConsumer>true</mediaConsumer>
<version>2.7</version>
</optionsResponse>10.3. CLUE Message No. 3: 'advertisement' 📺
<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <ns2:advertisement xmlns="urn:ietf:params:xml:ns:clue-info" xmlns:ns2="urn:ietf:params:xml:ns:clue-protocol" xmlns:ns3="urn:ietf:params:xml:ns:vcard-4.0" xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" protocol="CLUE" v="2.7"> <ns2:clueId>CP1</ns2:clueId> <ns2:sequenceNr>11</ns2:sequenceNr> <ns2:mediaCaptures> <mediaCapture xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xsi:type="audioCaptureType" captureID="AC0" mediaType="audio"> <captureSceneIDREF>CS1</captureSceneIDREF> <spatialInformation> <captureOrigin> <capturePoint> <x>0.0</x> <y>0.0</y> <z>10.0</z> </capturePoint> <lineOfCapturePoint> <x>0.0</x> <y>1.0</y> <z>10.0</z> </lineOfCapturePoint> </captureOrigin> </spatialInformation> <individual>true</individual> <encGroupIDREF>EG1</encGroupIDREF> <description lang="en">main audio from the room </description> <priority>1</priority> <lang>it</lang> <mobility>static</mobility> <view>room</view> <capturedPeople> <personIDREF>alice</personIDREF> <personIDREF>bob</personIDREF> <personIDREF>ciccio</personIDREF> </capturedPeople> </mediaCapture> <mediaCapture xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xsi:type="videoCaptureType" captureID="VC0" mediaType="video"> <captureSceneIDREF>CS1</captureSceneIDREF> <spatialInformation> <captureOrigin> <capturePoint> <x>-2.0</x> <y>0.0</y> <z>10.0</z> </capturePoint> </captureOrigin> <captureArea> <bottomLeft> <x>-3.0</x> <y>20.0</y> <z>9.0</z> </bottomLeft> <bottomRight> <x>-1.0</x> <y>20.0</y> <z>9.0</z> </bottomRight> <topLeft> <x>-3.0</x> <y>20.0</y> <z>11.0</z> </topLeft> <topRight> <x>-1.0</x> <y>20.0</y> <z>11.0</z> </topRight> </captureArea> </spatialInformation> <individual>true</individual> <encGroupIDREF>EG0</encGroupIDREF> <description lang="en">left camera video capture </description> <priority>1</priority> <lang>it</lang> <mobility>static</mobility> <view>individual</view> <capturedPeople> <personIDREF>ciccio</personIDREF> </capturedPeople> </mediaCapture> <mediaCapture xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xsi:type="videoCaptureType" captureID="VC1" mediaType="video"> <captureSceneIDREF>CS1</captureSceneIDREF> <spatialInformation> <captureOrigin> <capturePoint> <x>0.0</x> <y>0.0</y> <z>10.0</z> </capturePoint> </captureOrigin> <captureArea> <bottomLeft> <x>-1.0</x> <y>20.0</y> <z>9.0</z> </bottomLeft> <bottomRight> <x>1.0</x> <y>20.0</y> <z>9.0</z> </bottomRight> <topLeft> <x>-1.0</x> <y>20.0</y> <z>11.0</z> </topLeft> <topRight> <x>1.0</x> <y>20.0</y> <z>11.0</z> </topRight> </captureArea> </spatialInformation> <individual>true</individual> <encGroupIDREF>EG0</encGroupIDREF> <description lang="en">central camera video capture </description> <priority>1</priority> <lang>it</lang> <mobility>static</mobility> <view>individual</view> <capturedPeople> <personIDREF>alice</personIDREF> </capturedPeople> </mediaCapture> <mediaCapture xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xsi:type="videoCaptureType" captureID="VC2" mediaType="video"> <captureSceneIDREF>CS1</captureSceneIDREF> <spatialInformation> <captureOrigin> <capturePoint> <x>2.0</x> <y>0.0</y> <z>10.0</z> </capturePoint> </captureOrigin> <captureArea> <bottomLeft> <x>1.0</x> <y>20.0</y> <z>9.0</z> </bottomLeft> <bottomRight> <x>3.0</x> <y>20.0</y> <z>9.0</z> </bottomRight> <topLeft> <x>1.0</x> <y>20.0</y> <z>11.0</z> </topLeft> <topRight> <x>3.0</x> <y>20.0</y> <z>11.0</z> </topRight> </captureArea> </spatialInformation> <individual>true</individual> <encGroupIDREF>EG0</encGroupIDREF> <description lang="en">right camera video capture </description> <priority>1</priority> <lang>it</lang> <mobility>static</mobility> <view>individual</view> <capturedPeople> <personIDREF>bob</personIDREF> </capturedPeople> </mediaCapture> <mediaCapture xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xsi:type="videoCaptureType" captureID="VC3" mediaType="video"> <captureSceneIDREF>CS1</captureSceneIDREF> <spatialInformation> <captureArea> <bottomLeft> <x>-3.0</x> <y>20.0</y> <z>9.0</z> </bottomLeft> <bottomRight> <x>3.0</x> <y>20.0</y> <z>9.0</z> </bottomRight> <topLeft> <x>-3.0</x> <y>20.0</y> <z>11.0</z> </topLeft> <topRight> <x>3.0</x> <y>20.0</y> <z>11.0</z> </topRight> </captureArea> </spatialInformation> <content> <sceneViewIDREF>SE1</sceneViewIDREF> </content> <policy>SoundLevel:0</policy> <encGroupIDREF>EG0</encGroupIDREF> <description lang="en">loudest room segment </description> <priority>2</priority> <lang>it</lang> <mobility>static</mobility> <view>individual</view> </mediaCapture> <mediaCapture xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xsi:type="videoCaptureType" captureID="VC4" mediaType="video"> <captureSceneIDREF>CS1</captureSceneIDREF> <spatialInformation> <captureOrigin> <capturePoint> <x>0.0</x> <y>0.0</y> <z>10.0</z> </capturePoint> </captureOrigin> <captureArea> <bottomLeft> <x>-3.0</x> <y>20.0</y> <z>7.0</z> </bottomLeft> <bottomRight> <x>3.0</x> <y>20.0</y> <z>7.0</z> </bottomRight> <topLeft> <x>-3.0</x> <y>20.0</y> <z>13.0</z> </topLeft> <topRight> <x>3.0</x> <y>20.0</y> <z>13.0</z> </topRight> </captureArea> </spatialInformation> <individual>true</individual> <encGroupIDREF>EG0</encGroupIDREF> <description lang="en">zoomed-out view of all people in the room</description> <priority>2</priority> <lang>it</lang> <mobility>static</mobility> <view>room</view> <capturedPeople> <personIDREF>alice</personIDREF> <personIDREF>bob</personIDREF> <personIDREF>ciccio</personIDREF> </capturedPeople> </mediaCapture> </ns2:mediaCaptures> <ns2:encodingGroups> <encodingGroup encodingGroupID="EG0"> <maxGroupBandwidth>600000</maxGroupBandwidth> <encodingIDList> <encodingID>ENC1</encodingID> <encodingID>ENC2</encodingID> <encodingID>ENC3</encodingID> </encodingIDList> </encodingGroup> <encodingGroup encodingGroupID="EG1"> <maxGroupBandwidth>300000</maxGroupBandwidth> <encodingIDList> <encodingID>ENC4</encodingID>
<encodingID>ENC5</encodingID>
</encodingIDList>
</encodingGroup>
</ns2:encodingGroups>
<ns2:captureScenes>
<captureScene scale="unknown" sceneID="CS1">
<sceneViews>
<sceneView sceneViewID="SE1">
<mediaCaptureIDs>
<mediaCaptureIDREF>VC0</mediaCaptureIDREF>
<mediaCaptureIDREF>VC1</mediaCaptureIDREF>
<mediaCaptureIDREF>VC2</mediaCaptureIDREF>
</mediaCaptureIDs>
</sceneView>
<sceneView sceneViewID="SE2">
<mediaCaptureIDs>
<mediaCaptureIDREF>VC3</mediaCaptureIDREF>
</mediaCaptureIDs>
</sceneView>
<sceneView sceneViewID="SE3">
<mediaCaptureIDs>
<mediaCaptureIDREF>VC4</mediaCaptureIDREF>
</mediaCaptureIDs>
</sceneView>
<sceneView sceneViewID="SE4">
<mediaCaptureIDs>
<mediaCaptureIDREF>AC0</mediaCaptureIDREF>
</mediaCaptureIDs>
</sceneView>
</sceneViews>
</captureScene>
</ns2:captureScenes>
<ns2:simultaneousSets>
<simultaneousSet setID="SS1">
<mediaCaptureIDREF>VC3</mediaCaptureIDREF>
<sceneViewIDREF>SE1</sceneViewIDREF>
</simultaneousSet>
<simultaneousSet setID="SS2">
<mediaCaptureIDREF>VC0</mediaCaptureIDREF>
<mediaCaptureIDREF>VC2</mediaCaptureIDREF>
<mediaCaptureIDREF>VC4</mediaCaptureIDREF>
</simultaneousSet>
</ns2:simultaneousSets>
<ns2:people>
<person personID="bob">
<personInfo>
<ns3:fn>
<ns3:text>Bob</ns3:text>
</ns3:fn>
</personInfo>
<personType>minute taker</personType>
</person>
<person personID="alice">
<personInfo>
<ns3:fn>
<ns3:text>Alice</ns3:text>
</ns3:fn>
</personInfo>
<personType>presenter</personType>
</person>
<person personID="ciccio">
<personInfo>
<ns3:fn>
<ns3:text>Ciccio</ns3:text>
</ns3:fn>
</personInfo>
<personType>chairman</personType>
<personType>timekeeper</personType>
</person>
</ns2:people>
</ns2:advertisement>10.4. CLUE Message No. 4: 'configure+ack' 🎀
Okay, bestie, here's the configure message all dressed up 💅
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ns2:configure xmlns="urn:ietf:params:xml:ns:clue-info"
xmlns:ns2="urn:ietf:params:xml:ns:clue-protocol"
xmlns:ns3="urn:ietf:params:xml:ns:vcard-4.0"
xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance"
protocol="CLUE" v="2.7">
<ns2:clueId>CP2</ns2:clueId>
<ns2:sequenceNr>22</ns2:sequenceNr>
<ns2:advSequenceNr>11</ns2:advSequenceNr>
<ns2:ack>200</ns2:ack>
<ns2:captureEncodings>
<captureEncoding ID="ce123">
<captureID>AC0</captureID>
<encodingID>ENC4</encodingID>
</captureEncoding>
<captureEncoding ID="ce223">
<captureID>VC3</captureID>
<encodingID>ENC1</encodingID>
<configuredContent>
<sceneViewIDREF>SE1</sceneViewIDREF>
</configuredContent>
</captureEncoding>
</ns2:captureEncodings>
</ns2:configure>10.5. CLUE Message No. 5: 'configureResponse' ✨
And now the reply, the configureResponse, serving looks 🫶
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ns2:configureResponse xmlns="urn:ietf:params:xml:ns:clue-info"
xmlns:ns2="urn:ietf:params:xml:ns:clue-protocol"
xmlns:ns3="urn:ietf:params:xml:ns:vcard-4.0"
xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance"
protocol="CLUE" v="2.7">
<ns2:clueId>CP1</ns2:clueId>
<ns2:sequenceNr>12</ns2:sequenceNr>
<ns2:responseCode>200</ns2:responseCode>
<ns2:reasonString>Success</ns2:reasonString>
<ns2:confSequenceNr>22</ns2:confSequenceNr>
</ns2:configureResponse>10.6. CLUE Message No. 6: 'advertisement' 🦋
Next up, the advertisement message, our newest main character 👑
okay the XML is still going, bestie 💅
<content> <sceneViewIDREF>SE1</sceneViewIDREF>
</content>
<policy>SoundLevel:1</policy>
<description lang="en">penultimate loudest room segment
</description>
<lang>it</lang>
<mobility>static</mobility>
<view>individual</view>
</mediaCapture>
<mediaCapture
xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance"
xsi:type="videoCaptureType" captureID="VC6"
mediaType="video">
<captureSceneIDREF>CS1</captureSceneIDREF>
<spatialInformation>
<captureArea>
<bottomLeft>
<x>-3.0</x>
<y>20.0</y>
<z>9.0</z>
</bottomLeft>
<bottomRight>
<x>3.0</x>
<y>20.0</y>
<z>9.0</z>
</bottomRight>
<topLeft>
<x>-3.0</x>
<y>20.0</y>
<z>11.0</z>
</topLeft>
<topRight>
<x>3.0</x>
<y>20.0</y>
<z>11.0</z>
</topRight>
</captureArea>
</spatialInformation>
<content>
<sceneViewIDREF>SE1</sceneViewIDREF>
</content>
<policy>SoundLevel:2</policy>
<description lang="en">last but two loudest room segment
</description>
<lang>it</lang>
<mobility>static</mobility>
<view>individual</view>
</mediaCapture>
<mediaCapture
xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance"
xsi:type="videoCaptureType" captureID="VC7"
mediaType="video">
<captureSceneIDREF>CS1</captureSceneIDREF>
<spatialInformation>
<captureArea>
<bottomLeft>
<x>-3.0</x>
<y>20.0</y>
<z>9.0</z>
</bottomLeft>
<bottomRight>
<x>3.0</x>
<y>20.0</y>
<z>9.0</z>
</bottomRight>
<topLeft>
<x>-3.0</x>
<y>20.0</y>
<z>11.0</z>
</topLeft>
<topRight>
<x>3.0</x>
<y>20.0</y>
<z>11.0</z>
</topRight>
</captureArea>
</spatialInformation>
<content>
<mediaCaptureIDREF>VC3</mediaCaptureIDREF>
<mediaCaptureIDREF>VC5</mediaCaptureIDREF>
<mediaCaptureIDREF>VC6</mediaCaptureIDREF>
</content>
<maxCaptures exactNumber="true">3</maxCaptures>
<encGroupIDREF>EG0</encGroupIDREF>
<description lang="en">big picture of the current
speaker + pips about previous speakers</description>
<priority>3</priority>
<lang>it</lang>
<mobility>static</mobility>
<view>individual</view>
</mediaCapture>
</ns2:mediaCaptures>
<ns2:encodingGroups>
<encodingGroup encodingGroupID="EG0">
<maxGroupBandwidth>600000</maxGroupBandwidth>
<encodingIDList>
<encodingID>ENC1</encodingID>
<encodingID>ENC2</encodingID>
<encodingID>ENC3</encodingID>
</encodingIDList>
</encodingGroup>
<encodingGroup encodingGroupID="EG1">
<maxGroupBandwidth>300000</maxGroupBandwidth>
<encodingIDList>
<encodingID>ENC4</encodingID>
<encodingID>ENC5</encodingID>
</encodingIDList>
</encodingGroup>
</ns2:encodingGroups>
<ns2:captureScenes>
<captureScene scale="unknown" sceneID="CS1">
<sceneViews>
<sceneView sceneViewID="SE1">
<description lang="en">participants' individual
videos</description>
<mediaCaptureIDs>
<mediaCaptureIDREF>VC0</mediaCaptureIDREF>
<mediaCaptureIDREF>VC1</mediaCaptureIDREF>
<mediaCaptureIDREF>VC2</mediaCaptureIDREF>
</mediaCaptureIDs>
</sceneView>
<sceneView sceneViewID="SE2">
<description lang="en">loudest segment of the
room</description>
<mediaCaptureIDs>
<mediaCaptureIDREF>VC3</mediaCaptureIDREF>
</mediaCaptureIDs>
</sceneView>
<sceneView sceneViewID="SE5">
<description lang="en">loudest segment of the
room + pips</description>
<mediaCaptureIDs>
<mediaCaptureIDREF>VC7</mediaCaptureIDREF>
</mediaCaptureIDs>
</sceneView>
<sceneView sceneViewID="SE4">
<description lang="en">room audio</description>
<mediaCaptureIDs>
<mediaCaptureIDREF>AC0</mediaCaptureIDREF>
</mediaCaptureIDs>
</sceneView>
<sceneView sceneViewID="SE3">
<description lang="en">room video</description>
<mediaCaptureIDs>
<mediaCaptureIDREF>VC4</mediaCaptureIDREF>
</mediaCaptureIDs>
</sceneView>
</sceneViews>
</captureScene>
</ns2:captureScenes>
<ns2:simultaneousSets>
<simultaneousSet setID="SS1">
<mediaCaptureIDREF>VC3</mediaCaptureIDREF>
<mediaCaptureIDREF>VC7</mediaCaptureIDREF>
<sceneViewIDREF>SE1</sceneViewIDREF>
</simultaneousSet>
<simultaneousSet setID="SS2">
<mediaCaptureIDREF>VC0</mediaCaptureIDREF>
<mediaCaptureIDREF>VC2</mediaCaptureIDREF>
<mediaCaptureIDREF>VC4</mediaCaptureIDREF>
</simultaneousSet>
</ns2:simultaneousSets>
<ns2:people>
<person personID="bob">
<personInfo>
<ns3:fn>
<ns3:text>Bob</ns3:text>
</ns3:fn>
</personInfo>
<personType>minute taker</personType>
</person>
<person personID="alice">
<personInfo>
<ns3:fn>
<ns3:text>Alice</ns3:text>
</ns3:fn>
</personInfo>
<personType>presenter</personType>
</person>
<person personID="ciccio">
<personInfo>
<ns3:fn>
<ns3:text>Ciccio</ns3:text>
</ns3:fn>
</personInfo>
<personType>chairman</personType>
<personType>timekeeper</personType>
</person>
</ns2:people>
</ns2:advertisement>10.7. CLUE Message No. 7: 'ack' 💌
okay bestie, this one is the little reply-back message 🫶 It's called ack, and it's all about the sequence numbers, the response code, and the reason string, so you can see what came back and how it went ✨
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ack xmlns="urn:ietf:params:xml:ns:clue-protocol"
xmlns:ns2="urn:ietf:params:xml:ns:clue-info"
xmlns:ns3="urn:ietf:params:xml:ns:vcard-4.0"
xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance"
protocol="CLUE" v="2.7">
<clueId>CP2</clueId>
<sequenceNr>23</sequenceNr>
<responseCode>200</responseCode>
<reasonString>Success</reasonString>
<advSequenceNr>13</advSequenceNr>
</ack>10.8. CLUE Message No. 8: 'configure' 🎀
next up is configure 💅 this is the one where the sender picks which capture gets which encoding, and it even says which scene view a capture should show. the vibes are very organised ngl 🌸
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ns2:configure xmlns="urn:ietf:params:xml:ns:clue-info"
xmlns:ns2="urn:ietf:params:xml:ns:clue-protocol"
xmlns:ns3="urn:ietf:params:xml:ns:vcard-4.0"
xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance"
protocol="CLUE" v="2.7">
<ns2:clueId>CP2</ns2:clueId>
<ns2:sequenceNr>24</ns2:sequenceNr>
<ns2:advSequenceNr>13</ns2:advSequenceNr>
<ns2:captureEncodings>
<captureEncoding ID="ce123">
<captureID>AC0</captureID>
<encodingID>ENC4</encodingID>
</captureEncoding>
<captureEncoding ID="ce456">
<captureID>VC7</captureID>
<encodingID>ENC1</encodingID>
<configuredContent>
<sceneViewIDREF>SE5</sceneViewIDREF>
</configuredContent>
</captureEncoding>
</ns2:captureEncodings>
</ns2:configure>10.9. CLUE Message No. 9: 'configureResponse' 🦋
and last but not least in this part, configureResponse 👑 it's the reply to that configure message. It carries the same kind of response code and reason string, plus the configuration sequence number it's answering, so everything lines up perfectly 💖
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ns2:configureResponse xmlns="urn:ietf:params:xml:ns:clue-info"
xmlns:ns2="urn:ietf:params:xml:ns:clue-protocol"
xmlns:ns3="urn:ietf:params:xml:ns:vcard-4.0"
xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance"
protocol="CLUE" v="2.7">
<ns2:clueId>CP1</ns2:clueId>
<ns2:sequenceNr>14</ns2:sequenceNr>
<ns2:responseCode>200</ns2:responseCode>
<ns2:reasonString>Success</ns2:reasonString>
<ns2:confSequenceNr>24</ns2:confSequenceNr>
</ns2:configureResponse>11. Security Considerations ✨
Okay bestie, let's talk security 🔐 CLUE was built with security-by-design from day one, so it's not an afterthought. The standard makes a bunch of things mandatory across the whole CLUE architecture: (i) cryptography and authentication, (ii) protection of all sensitive fields, (iii) mutual authentication between CLUE endpoints, (iv) authorization mechanisms, and (v) native defenses against eavesdropping, selective modification, deletion, and replay (and combos of those) 🛡️ The real vibe is that you can't judge one piece of CLUE's security on its own. You have to look at the whole integrated architecture, periodt 💅
CLUE is an application-level protocol that lets a Media Producer and an MC negotiate a bunch of parameters for setting up a telepresence session. That means a CLUE-enabled telepresence system is exposed to a number of potential threats 😬 Most of them get a deep dive in the CLUE framework document [RFC8845]. Its Security Considerations section covers setup and management issues for (1) the basic case of two CLUE endpoints acting as the MP and the MC, and (2) the fancier scenario with an MCU in the mix 🎀
The CLUE framework document [RFC8845] also points out that CLUE protocol messages can carry sensitive data, so that data SHOULD only be accessed by authenticated endpoints 🫣 Security issues tied to the CLUE data model schema are covered in [RFC8846].
Now, there's some extra info in the CLUE protocol that isn't part of the data model schema, and it can reveal stuff you'd rather keep private 👀 It's mostly swapped during negotiation through the 'options' and 'optionsResponse' messages. In the CP state machine's OPTIONS state, both sides agree on the version and extensions they'll use for the rest of the CLUE message exchange. A sneaky participant could (1) try to dig up a detailed footprint of a specific CLUE implementation during this early setup phase, or (2) pressure the other side into using an outdated version of the protocol that they already know how to break 😤 Basically, listing every version and extension you support could leak details about your implementation. An implementation could choose to hold back some of the versions it supports to avoid that leak, but the tradeoff is that it hurts interoperability 🙃 Also, the OPTIONS state is only reached after the CLUE data channel is successfully set up. So only authenticated parties can exchange 'options' messages and their 'optionsResponse' replies, which seriously shrinks the attack surface for bad actors 🧚
The CLUE framework clearly requires protection for CLUE protocol messages against a malicious agent that can get into the CLUE data channel 🚨 That requirement is met by the CLUE data channel solution in [RFC8850], which protects messages from both recovery and tampering. On the tampering front, any CLUE implementation that follows the CLUE specification MUST rely on messages flowing over a reliable and ordered SCTP-over-DTLS transport channel connecting two CPs 🔥
12. IANA Considerations 🌸
This document registers a new XML namespace, a new XML schema, and the media type for that schema 🎀 It also registers the "CLUE" Application Service tag and the "CLUE" Application Protocol tag, and it sets up registries for CLUE messages and response codes 📋
12.1. URN Sub-Namespace Registration
This section registers a new XML namespace, "urn:ietf:params:xml:ns:clue-protocol" 💖
URI: urn:ietf:params:xml:ns:clue-protocol
Registrant Contact: IESG (iesg@ietf.org).
XML:
<CODE BEGINS>
<?xml version="1.0"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"https://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="https://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<title>CLUE Messages</title>
</head>
<body>
<h1>Namespace for CLUE Messages</h1>
<h2>urn:ietf:params:xml:ns:clue-protocol</h2>
<p>See <a href="https://www.rfc-editor.org/rfc/rfc8847.txt">
RFC 8847</a>.</p>
</body>
</html>
<CODE ENDS>12.2. XML Schema Registration
This section registers an XML schema following the guidelines in [RFC3688] 🧁
URI: urn:ietf:params:xml:schema:clue-protocol
Registrant Contact: IESG (iesg@ietf.org).
Schema: The XML for this schema can be found in Section 9 of this document.
12.3. Media Type Registration for "application/clue+xml"
This section registers the "application/clue+xml" media type 🦋
To: ietf-types@iana.org
Subject: Registration of media type "application/clue+xml"
Media type name: application
Subtype name: clue+xml
Required parameters: (none)
Optional parameters: charset. Same as the charset parameter of "application/xml" as specified in [RFC7303], Section 4.2.
Encoding considerations: Same as the encoding considerations of "application/xml" as specified in [RFC7303], Section 4.2.
Security considerations: This content type is designed to carry protocol data related to telepresence session control. Some of the data could be considered private. This media type does not provide any protection, so other mechanisms, such as those described in Section 11 of this document, are required to protect the data. This media type does not contain executable content 🔒
Interoperability considerations: None.
Published specification: RFC 8847
Applications that use this media type: CLUE Participants.
Additional Information:
- Magic Number(s): (none)
- File extension(s): .xml
- Macintosh File Type Code(s): TEXT
Person & email address to contact for further information: Simon Pietro Romano (spromano@unina.it).
Intended usage: LIMITED USE
Author/Change controller: The IETF
Other information: This media type is a specialization of application/xml [RFC7303], and many of the considerations described there also apply to application/clue+xml 💜
12.4. CLUE Protocol Registry
Per this document, IANA has created new registries for CLUE messages and response codes 🗂️
12.4.1. CLUE Message Types
Here's the rundown on the CLUE messages registry:
Related Registry: CLUE Message Types
Defining RFC: RFC 8847
Registration/Assignment Procedures: Following the policies outlined in [RFC8126], the IANA policy for assigning new values for the CLUE message types for the CLUE protocol is Specification Required.
Registrant Contact: IESG (iesg@ietf.org).
The initial table of CLUE messages is filled in using the CLUE messages described in Section 5 and defined in the XML schema in Section 9 ✨
+===================+=================================+===========+
| Message | Description | Reference |
+===================+=================================+===========+
| options | Sent by the CI to the CR in the | RFC 8847 |
| | initiation phase to specify the | |
| | roles played by the CI, the | |
| | supported versions, and the | |
| | supported extensions. | |
+-------------------+---------------------------------+-----------+
| optionsResponse | Sent by the CI to the CR in | RFC 8847 |
| | reply to an 'options' message, | |
| | to establish the version and | |
| | extensions to be used in the | |
| | subsequent exchange of CLUE | |
| | messages. | |
+-------------------+---------------------------------+-----------+
| advertisement | Sent by the MP to the MC to | RFC 8847 |
| | specify the telepresence | |
| | capabilities of the MP | |
| | expressed according to the CLUE | |
| | framework. | |
+-------------------+---------------------------------+-----------+
| ack | Sent by the MC to the MP to | RFC 8847 |
| | acknowledge the reception of an | |
| | 'advertisement' message. | |
+-------------------+---------------------------------+-----------+
| configure | Sent by the MC to the MP to | RFC 8847 |
| | specify the desired media | |
| | captures among those specified | |
| | in the 'advertisement'. | |
+-------------------+---------------------------------+-----------+
| configureResponse | Sent by the MP to the MC in | RFC 8847 |
| | reply to a 'configure' message | |
| | to communicate whether or not | |
| | the configuration request has | |
| | been successfully processed. | |
+-------------------+---------------------------------+-----------+
Table 2: Initial IANA Table of CLUE Messages12.4.2. CLUE Response Codes
Okay so bestie, this is the big table of CLUE response codes! 💖 It's IANA's registry for them, and the whole thing is the official list of "what went wrong" codes 🎀 ✨
Here's the deal with the registry: new values are assigned under the Specification Required policy 💅, and the defining document is RFC 8847. The point of contact is the IESG at iesg@ietf.org 🫶. Those are the policy rules (from [RFC8126]), and the starter table is copied over from Section 5.7, so the codes are literally the ones from that section 🦋
Registry info:
Related Registry: CLUE Response Codes
Defining RFC: RFC 8847
Registration/Assignment Procedures: Following the policies outlined
in [RFC8126], the IANA policy for assigning new values for the
response codes for CLUE is Specification Required.
Registrant Contact: IESG (iesg@ietf.org).Now the table, which is the main event 🌸 Each row is a number, a default reason string, a description, and a reference (which is RFC 8847 for every single one, iykyk 💜):
+========+===============+==============================+===========+
| Number | Default | Description | Reference |
| | Reason String | | |
+========+===============+==============================+===========+
| 200 | Success | The request has been | RFC 8847 |
| | | successfully | |
| | | processed. | |
+--------+---------------+------------------------------+-----------+
| 300 | Low-level | A generic low-level | RFC 8847 |
| | request error | request error has | |
| | | occurred. | |
+--------+---------------+------------------------------+-----------+
| 301 | Bad syntax | The XML syntax of the | RFC 8847 |
| | | message is not | |
| | | correct. | |
+--------+---------------+------------------------------+-----------+
| 302 | Invalid value | The message contains | RFC 8847 |
| | | an invalid parameter | |
| | | value. | |
+--------+---------------+------------------------------+-----------+
| 303 | Conflicting | The message contains | RFC 8847 |
| | values | values that cannot be | |
| | | used together. | |
+--------+---------------+------------------------------+-----------+
| 400 | Semantic | The received CLUE | RFC 8847 |
| | errors | protocol message | |
| | | contains semantic | |
| | | errors. | |
+--------+---------------+------------------------------+-----------+
| 401 | Version not | The protocol version | RFC 8847 |
| | supported | used in the message is | |
| | | not supported. | |
+--------+---------------+------------------------------+-----------+
| 402 | Invalid | The received message | RFC 8847 |
| | sequencing | contains an unexpected | |
| | | sequence number (e.g., | |
| | | sequence number gap, | |
| | | repeated sequence | |
| | | number, or sequence | |
| | | number outdated). | |
+--------+---------------+------------------------------+-----------+
| 403 | Invalid | The clueId used in the | RFC 8847 |
| | identifier | message is invalid or | |
| | | unknown. | |
+--------+---------------+------------------------------+-----------+
| 404 | Advertisement | The sequence number of | RFC 8847 |
| | expired | the advertisement the | |
| | | 'configure' message | |
| | | refers to is out of | |
| | | date. | |
+--------+---------------+------------------------------+-----------+
| 405 | Subset choice | The subset choice is | RFC 8847 |
| | not allowed | not allowed for the | |
| | | specified Multiple | |
| | | Content Capture. | |
+--------+---------------+------------------------------+-----------+Table 3: Initial IANA Table of CLUE Response Codes 🧚
13. References
13.1. Normative References
Okay the references, bestie, lowkey the most important part for building anything 🤓 Here's the full list of normative ones:
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550,
July 2003, <https://www.rfc-editor.org/info/rfc3550>.
[RFC3688] Mealling, M., "The IETF XML Registry", BCP 81, RFC 3688,
DOI 10.17487/RFC3688, January 2004,
<https://www.rfc-editor.org/info/rfc3688>.
[RFC7303] Thompson, H. and C. Lilley, "XML Media Types", RFC 7303,
DOI 10.17487/RFC7303, July 2014,
<https://www.rfc-editor.org/info/rfc7303>.
[RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for
Writing an IANA Considerations Section in RFCs", BCP 26,
RFC 8126, DOI 10.17487/RFC8126, June 2017,
<https://www.rfc-editor.org/info/rfc8126>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC8845] Duckworth, M., Ed., Pepperell, A., and S. Wenger,
"Framework for Telepresence Multi-Streams", RFC 8845,
DOI 10.17487/RFC8845, January 2021,
<https://www.rfc-editor.org/info/rfc8845>.
[RFC8846] Presta, R. and S P. Romano, "An XML Schema for the
Controlling Multiple Streams for Telepresence (CLUE) Data
Model", RFC 8846, DOI 10.17487/RFC8846, January 2021,
<https://www.rfc-editor.org/info/rfc8846>.
[RFC8848] Hanton, R., Kyzivat, P., Xiao, L., and C. Groves, "Session
Signaling for Controlling Multiple Streams for
Telepresence (CLUE)", RFC 8848, DOI 10.17487/RFC8848,
January 2021, <https://www.rfc-editor.org/info/rfc8848>.
[RFC8850] Holmberg, C., "Controlling Multiple Streams for
Telepresence (CLUE) Protocol Data Channel", RFC 8850,
DOI 10.17487/RFC8850, January 2021,
<https://www.rfc-editor.org/info/rfc8850>.
[W3C.REC-xml-20081126]
Bray, T., Paoli, J., Sperberg-McQueen, M., Maler, E., and
F. Yergeau, "Extensible Markup Language (XML) 1.0 (Fifth
Edition)", World Wide Web Consortium Recommendation REC-
xml-20081126, November 2008,
<https://www.w3.org/TR/2008/REC-xml-20081126>.13.2. Informative References
And these are the informative ones, the extra reading that's good to know about 📚 ✨
[RFC1122] Braden, R., Ed., "Requirements for Internet Hosts -
Communication Layers", STD 3, RFC 1122,
DOI 10.17487/RFC1122, October 1989,
<https://www.rfc-editor.org/info/rfc1122>.
[RFC3261] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston,
A., Peterson, J., Sparks, R., Handley, M., and E.
Schooler, "SIP: Session Initiation Protocol", RFC 3261,
DOI 10.17487/RFC3261, June 2002,
<https://www.rfc-editor.org/info/rfc3261>.
[RFC4353] Rosenberg, J., "A Framework for Conferencing with the
Session Initiation Protocol (SIP)", RFC 4353,
DOI 10.17487/RFC4353, February 2006,
<https://www.rfc-editor.org/info/rfc4353>.
[RFC6120] Saint-Andre, P., "Extensible Messaging and Presence
Protocol (XMPP): Core", RFC 6120, DOI 10.17487/RFC6120,
March 2011, <https://www.rfc-editor.org/info/rfc6120>.
[RFC7262] Romanow, A., Botzko, S., and M. Barnes, "Requirements for
Telepresence Multistreams", RFC 7262,
DOI 10.17487/RFC7262, June 2014,
<https://www.rfc-editor.org/info/rfc7262>.
[RFC7667] Westerlund, M. and S. Wenger, "RTP Topologies", RFC 7667,
DOI 10.17487/RFC7667, November 2015,
<https://www.rfc-editor.org/info/rfc7667>.Acknowledgements
Huge shoutout to all the CLUErs 🌈 who gave their precious feedback and support. The authors specifically thank Paul Kyzivat, Christian Groves, and Scarlett Liuyan 💜🫶
Authors' Addresses
Roberta Presta University of Napoli Via Claudio 21 80125 Napoli Italy
Email: roberta.presta@unina.it
Simon Pietro Romano University of Napoli Via Claudio 21 80125 Napoli Italy
Email: spromano@unina.it
That's the whole doc, bestie! 🦋 You made it to the end of the CLUE spec, and honestly, that's main character energy 👑🎀
📦 where this page comes from
- ietf: rfc8847 2021-01
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms