RFC 8845
Framework for Telepresence Multi-Streams
Framework for Telepresence Multi-Streams 🎀
Okay bestie, here's the spec in full RFC 8845 glory 💅✨
- Authors: M. Duckworth, Ed. (that "Ed." just means editor 🫶), A. Pepperell from Acano 🌸, and S. Wenger from Tencent 💖
- Category: Standards Track 🦋
- Date: January 2021 📅
- ISSN: 2070-1721
- Obsoletes / updates: the title block doesn't list anything this one obsoletes or updates 🙈
Abstract ✨
Telepresence systems are supposed to be built on open standards, but they still don't play nicely together 😭 This doc is a framework for a protocol that lets telepresence devices in a conference interoperate. The protocol shares info about multiple media streams, so the sending system and the receiving system can make good choices about sending, picking, and rendering those streams 🎀 It's used on top of SIP signaling and Session Description Protocol (SDP) negotiation to set up a telepresence session 🧚
Status of This Memo 🌈
This is an Internet Standards Track document 🦋
It's a product of the Internet Engineering Task Force (IETF), and it reflects the consensus of the IETF community. It went through public review and got approved for publication by the Internet Engineering Steering Group (IESG). Want more on Internet Standards? Check Section 2 of RFC 7841 🔥
Info about the current status of this document, any errata, and how to send feedback lives at https://www.rfc-editor.org/info/rfc8845 💜
Copyright Notice 💅
Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved. 🫶
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as in effect on the date of publication. Please read these carefully, because they describe your rights and restrictions for this document. Code Components extracted from this document must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as described in the Simplified BSD License 🙈
Table of Contents 🧚
1. Introduction
2. Requirements Language
3. Definitions
4. Overview and Motivation
5. Description of the Framework/Model
6. Spatial Relationships
7. Media Captures and Capture Scenes
7.1. Media Captures
7.1.1. Media Capture Attributes
7.2. Multiple Content Capture
7.2.1. MCC Attributes
7.3. Capture Scene
7.3.1. Capture Scene Attributes
7.3.2. Capture Scene View Attributes
7.4. Global View List
8. Simultaneous Transmission Set Constraints
9. Encodings
9.1. Individual Encodings
9.2. Encoding Group
9.3. Associating Captures with Encoding Groups
10. Consumer's Choice of Streams to Receive from the Provider
10.1. Local Preference
10.2. Physical Simultaneity Restrictions
10.3. Encoding and Encoding Group Limits
11. Extensibility
12. Examples - Using the Framework (Informative)
12.1. Provider Behavior
12.1.1. Three-Screen Endpoint Provider
12.1.2. Encoding Group Example
12.1.3. The MCU Case
12.2. Media Consumer Behavior
12.2.1. One-Screen Media Consumer
12.2.2. Two-Screen Media Consumer Configuring the Example
12.2.3. Three-Screen Media Consumer Configuring the Example
12.3. Multipoint Conference Utilizing Multiple Content Captures
12.3.1. Single Media Captures and MCC in the Same
Advertisement
12.3.2. Several MCCs in the Same Advertisement
12.3.3. Heterogeneous Conference with Switching and
Composition
12.3.4. Heterogeneous Conference with Voice-Activated
Switching
13. IANA Considerations
14. Security Considerations
15. References
15.1. Normative References
15.2. Informative References
Acknowledgements
Authors' Addresses1. Introduction ✨
Okay so telepresence systems are built on open standards like RTP [RFC3550] and SIP [RFC3261], but they still can't easily interoperate with each other 😭 The big blocker is that there's no standard way to describe and negotiate the use of multiple audio and video streams that make up the media flows. This document gives protocols a framework to get interoperability by handling multiple streams in a standardized way 🦋 It's meant to support the use cases in "Use Cases for Telepresence Multistreams" [RFC7205] and to meet the requirements in "Requirements for Telepresence Multistreams" [RFC7262]. And it isn't limited to telepresence, since it also covers cases using multiple media streams that aren't necessarily telepresence 🌈
The basic session setup for these use cases is based on SIP [RFC3261] and SDP offer/answer [RFC3264]. On top of basic SIP and SDP offer/answer, you need signaling that is ControLling mUltiple streams for tElepresence (CLUE) specific, to exchange the info describing the multiple Media Streams 🎀 Why this framework exists, an overview of the signaling, and the information required to be exchanged all get covered in the later sections of this document 💖 Companion documents cover the signaling details [RFC8848], the data model [RFC8846], and the protocol [RFC8847] 🧚
2. Requirements Language 💜
Here's the BCP 14 part, bestie 🫶 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be read as described in BCP 14 [RFC2119] [RFC8174], but only when they show up in all capitals, just like shown here ✨
3. Definitions ✨
okay bestie, this is the glossary section and it's lowkey the most important part 🤓💅 The terms below get used all over this document and in companion documents. Capitals are how you spot a defined term, so if you see a capitalised word, it means the exact thing defined here 🎀
Advertisement: This is a CLUE message that a Media Provider sends to a Media Consumer. It describes specific aspects of the content of the Media, plus any restrictions on providing certain Streams at the same time 📣
Audio Capture (AC): This is Media Capture for audio 🎧 In the examples in this document it's written as "ACn".
Capture: Same thing as Media Capture ✨
Capture Device: A device that turns physical input, like audio, video, or text, into an electrical signal. Most of the time that signal is fed into a Media encoder 🔌
Capture Encoding: A specific Encoding of a Media Capture. A Media Provider sends it to a Media Consumer via RTP 🚀
Capture Scene: This is a structure that represents a spatial region captured by one or more Capture Devices. Each device captures Media for a portion of the region 🏠 The region can correspond to a real place in physical space, like a room. A Capture Scene has attributes and one or more Capture Scene Views, and each view includes one or more Media Captures 🌸
Capture Scene View (CSV): A list of Media Captures that are all the same Media type. Together they make one way to represent the whole Capture Scene 👀
CLUE: CLUE stands for "ControLling mUltiple streams for tElepresence." That's the name of the IETF working group where this document and some companion documents were developed 🦋 When you see CLUE-* written out, it usually means something the CLUE working group designed. For example, this document may be called the CLUE-framework document both here and elsewhere 💜
CLUE-capable device: A device that supports the CLUE data channel [RFC8850], the CLUE protocol [RFC8847], and the principles of CLUE negotiation. It also seeks out CLUE-enabled calls 🫶
CLUE-enabled call: A call where two CLUE-capable devices have successfully negotiated support for a CLUE data channel in SDP [RFC4566]. A CLUE-enabled call isn't automatically ready to send CLUE-controlled Media. Negotiation of the data channel and of the CLUE protocol must finish first ⏳ Calls between two CLUE-capable devices that haven't yet successfully finished negotiating support for the CLUE data channel in SDP aren't considered CLUE-enabled 🙈
Conference: Used as defined in "A Framework for Conferencing within the Session Initiation Protocol (SIP)" [RFC4353] 🎙️
Configure Message: A CLUE message that a Media Consumer sends to a Media Provider. It specifies which content and Media Streams the Consumer wants to receive, based on the information in a matching Advertisement message 🎯
Consumer: Short for Media Consumer 💕
Encoding: Short for Individual Encoding ✨
Encoding Group: A set of Encoding parameters that represents a total Media Encoding capability. That total gets split up across potentially multiple Individual Encodings 🍰
Endpoint: A CLUE-capable device that is the logical point of final termination. It does this through receiving, decoding, and Rendering, and/or through initiation, which means capturing, encoding, and sending Media Streams 🌈 An Endpoint is made of one or more physical devices that source and sink Media Streams, plus exactly one [RFC4353] Participant. That Participant in turn includes exactly one SIP User Agent 🤖 Endpoints can be anything from multiscreen, multicamera rooms all the way down to handheld devices 📱
Global View: A set of references to one or more CSVs of the same Media type. Those CSVs are defined within Scenes of the same Advertisement. A Global View is a suggestion from the Provider to the Consumer for one set of CSVs that gives a useful representation of all the Scenes in the Advertisement 🌍
Global View List: A list of Global Views included in an Advertisement. One Global View List can include Global Views of different Media types 🗂️
Individual Encoding: A set of parameters that represents a way to encode a Media Capture so it becomes a Capture Encoding 🧩
Multipoint Control Unit (MCU): A CLUE-capable device that connects two or more Endpoints into one single multimedia Conference [RFC7667]. An MCU includes a Mixer like the one described in [RFC4353], but [RFC4353] doesn't require that Mixer to send Media to each participant 🎪
Media: Any data that can be conveyed over RTP after suitable encoding. That includes audio, video, or timed text 🎶
Media Capture (MC): A source of Media. It can come from one or more Capture Devices or be constructed from other Media Streams 📸
Media Consumer: A CLUE-capable device that intends to receive Capture Encodings 👂
Media Provider: A CLUE-capable device that intends to send Capture Encodings 📤
Multiple Content Capture (MCC): A Capture that mixes and/or switches other Captures of a single type, like all audio or all video 🔀 Particular Media Captures may or may not be present in the resulting Capture Encoding, depending on time or space ⏰ In the example cases in this document it's written as "MCCn".
Plane of Interest: The spatial plane within a Scene that contains the most-relevant subject matter 🔍
Provider: Same as a Media Provider 🎁
Render: The process of generating a representation from Media. Examples are displayed motion video or sound coming out of loudspeakers 🖥️🔊
Scene: Same as a Capture Scene 🎬
Simultaneous Transmission Set: A set of Media Captures that can be transmitted at the same time from a Media Provider 🙌
Single Media Capture: A Capture that contains Media from one single source Capture Device. For example, an Audio Capture from a single microphone, or a Video Capture from a single camera 🎤📷
Spatial Relation: The arrangement of two objects in space. It's different from relation in time or other kinds of relationships 📐
Stream: A Capture Encoding that a Media Provider sends to a Media Consumer via RTP [RFC3550] 🌊
Stream Characteristics: The Media Stream attributes that are commonly used in non-CLUE SIP/SDP environments. Examples are Media codec, bitrate, resolution, and profile/level. It also includes CLUE-specific attributes, like the Capture ID or a spatial location 🏷️
Video Capture (VC): Media Capture for video 🎥 In the example cases in this document it's written as VCn.
Video Composite: A single image, normally formed by an RTP mixer inside an MCU, that combines visual elements from separate sources 🖼️✨
4. Overview and Motivation ✨
okay so this section is basically the tour guide 🎀 It gives you an overview of the functional pieces this document defines for a telepresence or multistream system, and it explains why the framework exists in the first place 💖
Two key concepts get introduced here: "Media Provider" and "Media Consumer" 🦋 A Media Provider is the entity that sends the Media, and a Media Consumer is the entity that receives it. A Media Provider provides Media as RTP packets, and a Media Consumer consumes those RTP packets. Both can live in Endpoints or in Multipoint Control Units (MCUs) 🌸 In an Endpoint, a Media Provider is usually the one generating Media for Media Captures, which are typically sourced from cameras, microphones, and the like 📷 The Media Consumer in an Endpoint is usually tied to renderers, like screens and loudspeakers 🔊 In MCUs, Media Providers and Consumers can take the form of outputs and inputs of RTP mixers, RTP translators, and similar devices 🧚 Typically, telepresence devices like Endpoints and MCUs act as both Media Providers and Media Consumers. The Provider side cares about the device's transmitted Media, and the Consumer side cares about its received Media 💅 In a few circumstances, a CLUE-capable device only has Consumer or Provider functionality, like recorder-type Consumers or webcam-type Providers 📼
The motivations for the framework outlined in this document include the following:
- Endpoints in telepresence systems typically have multiple Media Capture and Media Render devices, e.g., multiple cameras and screens. While previous system designs were able to set up calls that would capture Media using all cameras and display Media on all screens, for example, there was no mechanism that could associate these Media Captures with each other in space and time, in a cross-vendor interoperable way. 😭
- The mere fact that there are multiple Media Capture and Media Render devices, each of which may be configurable in aspects such as zoom, leads to the difficulty that a variable number of such devices can be used to capture different aspects of a region. The Capture Scene concept allows for the description of multiple setups for those multiple Media Capture devices that could represent sensible operation points of the physical Capture Devices in a room, chosen by the operator. A Consumer can pick and choose from those configurations based on its rendering abilities and then inform the Provider about its choices. Details are provided in Section 7. 🎬
- In some cases, physical limitations or other reasons disallow the concurrent use of a device in more than one setup. For example, the center camera in a typical three-camera conference room can set its zoom objective to capture either the middle few seats only or all seats of a room, but not both concurrently. The Simultaneous Transmission Set concept allows a Provider to signal such limitations. Simultaneous Transmission Sets are part of the Capture Scene description and are discussed in Section 8. 🙈
- Often, the devices in a room do not have the computational complexity or connectivity to deal with multiple Encoding options simultaneously, even if each of these options is sensible in certain scenarios, and even if the simultaneous transmission is also sensible (i.e., in case of multicast Media distribution to multiple Endpoints). Such constraints can be expressed by the Provider using the Encoding Group concept, which is described in Section 9. 🧠
- Due to the potentially large number of RTP Streams required for a Multimedia Conference involving potentially many Endpoints, each of which can have many Media Captures and Media renderers, it has become common to multiplex multiple RTP Streams onto the same transport address, so as to avoid using the port number as a multiplexing point and the associated shortcomings such as NAT/firewall traversal. The large number of possible permutations of sensible options a Media Provider can make available to a Media Consumer makes a mechanism desirable that allows it to narrow down the number of possible options that a SIP offer/answer exchange has to consider. Such information is made available using protocol mechanisms specified in this document and companion documents. The Media Provider and Media Consumer may use information in CLUE messages to reduce the complexity of SIP offer/answer messages. Also, there are aspects of the control of both Endpoints and MCUs that dynamically change during the progress of a call, such as audio-level-based screen switching, layout changes, and so on, which need to be conveyed. Note that these control aspects are complementary to those specified in traditional SIP-based conference management, such as Binary Floor Control Protocol (BFCP). An exemplary call flow can be found in Section 5. 📞✨
Finally, all this information needs to be conveyed, and the notion of support for it needs to be established 🫶 This is done by negotiating a "CLUE channel", a data channel negotiated early during the initiation of a call. An Endpoint or MCU that rejects the establishment of this data channel, by definition, does not support CLUE-based mechanisms, whereas an Endpoint or MCU that accepts it is indicating support for CLUE as specified in this document and its companion documents 💜
5. Description of the Framework/Model 🌸
The CLUE framework specifies how multiple Media Streams are to be handled in a telepresence Conference 💖
A Media Provider (transmitting Endpoint or MCU) describes specific aspects of the content of the Media and the Media Stream Encodings it can send in an Advertisement 📢 Then the Media Consumer responds to the Media Provider by specifying which content and Media Streams it wants to receive in a Configure message 🎯 The Provider then transmits the asked-for content in the specified Streams 🚀
This Advertisement and Configure typically occur during call initiation, after CLUE has been enabled in a call, but they MAY also happen at any time throughout the call, whenever there is a change in what the Consumer wants to receive or (perhaps less common) what the Provider can send 🔄
An Endpoint or MCU typically acts as both Provider and Consumer at the same time, sending Advertisements and sending Configurations in response to receiving Advertisements. (It is possible to be just one or the other.) 👯♀️
The data model [RFC8846] is based around two main concepts: a Capture and an Encoding 🧩 A Media Capture, such as one of type audio or video, has attributes to describe the content a Provider can send. Media Captures are described in terms of CLUE-defined attributes, such as Spatial Relationships and purpose of the Capture. Providers tell Consumers which Media Captures they can provide, described in terms of the Media Capture attributes 🎀
A Provider organizes its Media Captures into one or more Capture Scenes, each representing a spatial region, such as a room 🏠 A Consumer chooses which Media Captures it wants to receive from the Capture Scenes 🛍️
In addition, the Provider can send the Consumer a description of the Individual Encodings it can send, in terms of identifiers that relate to items in SDP [RFC4566] 🌈
The Provider can also specify constraints on its ability to provide Media, and a sensible design choice for a Consumer is to take these into account when choosing the content and Capture Encodings it requests in the later offer/answer exchange 🧐 Some constraints are due to the physical limitations of a device; for example, a camera may not be able to provide zoom and non-zoom views simultaneously 📸 Other constraints are system based, such as maximum bandwidth 📶
The following diagram illustrates the information contained in an Advertisement 👀
...................................................................
. Provider Advertisement +--------------------+ .
. | Simultaneous Sets | .
. +------------------------+ +--------------------+ .
. | Capture Scene N | +--------------------+ .
. +-+----------------------+ | | Global View List | .
. | Capture Scene 2 | | +--------------------+ .
. +-+----------------------+ | | +----------------------+ .
. | Capture Scene 1 | | | | Encoding Group N | .
. | +---------------+ | | | +-+--------------------+ | .
. | | Attributes | | | | | Encoding Group 2 | | .
. | +---------------+ | | | +-+--------------------+ | | .
. | | | | | Encoding Group 1 | | | .
. | +----------------+ | | | | parameters | | | .
. | | V i e w s | | | | | bandwidth | | | .
. | | +---------+ | | | | | +-------------------+| | | .
. | | |Attribute| | | | | | | V i d e o || | | .
. | | +---------+ | | | | | | E n c o d i n g s || | | .
. | | | | | | | | Encoding 1 || | | .
. | | View 1 | | | | | | || | | .
. | | (list of MCs) | | |-+ | +-------------------+| | | .
. | +----|-|--|------+ |-+ | | | | .
. +---------|-|--|---------+ | +-------------------+| | | .
. | | | | | A u d i o || | | .
. | | | | | E n c o d i n g s || | | .
. v | | | | Encoding 1 || | | .
. +---------|--|--------+ | | || | | .
. | Media Capture N |------>| +-------------------+| | | .
. +-+---------v--|------+ | | | | | .
. | Media Capture 2 | | | | |-+ .
. +-+--------------v----+ |-------->| | | .
. | Media Capture 1 | | | | |-+ .
. | +----------------+ |---------->| | .
. | | Attributes | | |_+ +----------------------+ .
. | +----------------+ |_+ .
. +---------------------+ .
. .
................................................................... Figure 1: Advertisement StructureFigure 2 illustrates the call flow used by a simple system (two Endpoints) in compliance with this document 📞💕 A very brief outline of the call flow is described in the text that follows 👇
In a basic call, two Endpoints first trade an initial offer/answer exchange to set up a basic Media session, such as audio-only, plus a CLUE channel between them. ✨ Once that CLUE channel exists, both Endpoints have agreed to speak CLUE, so they MUST stick to the CLUE protocol suite described here. 💖
+-----------+ +-----------+
| Endpoint1 | | Endpoint2 |
+----+------+ +-----+-----+
| INVITE (BASIC SDP+CLUECHANNEL) |
|--------------------------------->|
| 200 0K (BASIC SDP+CLUECHANNEL)|
|<---------------------------------|
| ACK |
|--------------------------------->|
| |
|<################################>|
| BASIC MEDIA SESSION |
|<################################>|
| |
| CONNECT (CLUE CTRL CHANNEL) |
|=================================>|
| ... |
|<================================>|
| CLUE CTRL CHANNEL ESTABLISHED |
|<================================>|
| |
| ADVERTISEMENT 1 |
|*********************************>|
| ADVERTISEMENT 2 |
|<*********************************|
| |
| CONFIGURE 1 |
|<*********************************|
| CONFIGURE 2 |
|*********************************>|
| |
| REINVITE (UPDATED SDP) |
|--------------------------------->|
| 200 0K (UPDATED SDP)|
|<---------------------------------|
| ACK |
|--------------------------------->|
| |
|<################################>|
| UPDATED MEDIA SESSION |
|<################################>|
| |
v vFigure 2: Basic Information Flow
Now the good stuff 🧚 Over that CLUE channel, the Provider in each Endpoint tells the other side what it's capable of by sending an Advertisement, as spelled out in this doc. 🎀 An Advertisement usually isn't enough on its own to set up all the Media, though. The Consumer gets that info and can use it in a few ways. 💅 It combines it with info from an offer/answer exchange to build a CLUE Configure message, which is how it tells the Provider what it actually wants to receive. It can also tailor the SDP it sends in any later SIP offer/answer exchange, and how it reacts to SDP it gets back. Honestly, doing this is often a sensible implementation choice. 🙈
Spatial relationships between the Media can ride along in the Advertisement, and it's often smart for the Consumer to factor those in when it tailors the SDP. 🌸 The Consumer can also cap how many Encodings it has to reserve resources for, so it doesn't waste anything on Encodings it won't use, since it already knows ahead of time what it really wants from the Advertisement. ✨ And the Advertisement info is useful for local rendering decisions too. 🦋
After that initial CLUE exchange comes a regular SDP offer/answer exchange, which is "plain vanilla SIP" 🍦. This one handles the parts of the Media that weren't negotiated over CLUE, and it also actually sets up the Media transmission itself. That can involve security exchanges, Interactive Connectivity Establishment (ICE), and whatnot. 🔥
During the life of a call, more exchanges MAY happen over the CLUE channel. 🫶 Sometimes those exchanges change how the Provider or Consumer (or both) behave, without any other protocol activity such as more offer/answer exchanges. For example, a Configure message asking the Provider to put a different Capture source into a Capture Encoding, sent over the CLUE channel, shouldn't trigger heavy-handed stuff like SIP re-invites. 💀 Other times, though, an extra offer/answer exchange is needed after the CLUE negotiation. Say both sides decide to go from one screen to a multi-screen call, and the extra video channels need more bandwidth than what was negotiated earlier with offer/answer. Then a new offer/answer exchange is required. 🎀
One big perk of this protocol, which gets specified in more detail in companion documents, is that it gives the Consumer info about the Provider's ability to deliver Media, along with attributes of that Media such as its Spatial Relationship. 👑 What the renderer inside the Consumer does with all that is left unspecified. It's allowed to ignore some of the info the Provider sends, and it can also choose not to Render Media Streams that the Provider makes available. Even so, the Consumer still follows the CLUE protocol, which means it gracefully receives and responds to the Provider's info using a Configure operation. 🌈
A CLUE-capable device can work with a device that doesn't support CLUE. 🧚 The CLUE-capable device can figure out from the result of the initial offer/answer exchange whether the other device supports CLUE and wants to use it. The specific mechanism for that is described in [RFC8848]. If the other device doesn't use CLUE, the CLUE-capable device falls back to behavior that doesn't need CLUE. 💖
As for the Media itself, the Provider and Consumer have an end-to-end communication relationship for (RTP-transported) Media. The mechanisms in this doc and its companion documents don't change how those RTP flows and sessions get set up. In other words, the RTP Media sessions follow the negotiated SDP whether or not CLUE is used. ⭐
6. Spatial Relationships 🌸
For a Consumer to render things properly, it's often necessary (or at least helpful) for it to know the spatial info about the Streams it's receiving. 🦋 CLUE defines a coordinate system that lets Media Providers describe the Spatial Relationships of their Media Captures. That way the Streams can be scaled properly and rendered in a spatially sensible way. 💅 The coordinate system is built on a few principles:
- Each Capture Scene has a distinct coordinate system, unrelated to the coordinate systems of other Scenes. 🎀
- Simple systems that don't have multiple Media Captures to associate spatially don't need to use the coordinate model, although it can still be useful to provide an Area of Capture. ✨
- Coordinates can be in real, physical units (millimeters), have an unknown scale, or have no physical scale. 🔥 Systems that know their physical dimensions (for example, professionally installed Telepresence room systems) MUST provide those real-world measurements, so advanced receiving systems can make the best user experience out of that info. 👑 Systems that don't know specific physical dimensions but still know relative distances MUST use "Unknown Scale". "No Scale" is meant only for cases where Media Captures from different devices (possibly with different scales) get forwarded side by side, like in the case of an MCU. 🫶
- "Millimeters" means the scale is in millimeters. 📏
- "Unknown Scale" means the scale is not necessarily in millimeters, but it's the same for every Capture in the Capture Scene. 🧚
- "No Scale" means the scale could be different for each Capture. 🌈 For example, an MCU Provider that advertises two adjacent Captures and picks sources (which can change quickly) from different Endpoints might use this value. The scale could be different and changing for each Capture. Even so, the areas of capture still represent a Spatial Relation between Captures. 💜
- The coordinate system is right-handed Cartesian X, Y, Z, with the origin at a spatial location the Provider picks. 🍓 The Provider MUST use the same coordinate system, with the same scale and origin, for all coordinates within the same Capture Scene. 🔒
The direction of increasing coordinate values works like this: X increases from left to right, from the point of view of an observer at the front of the room looking toward the back. 🧭 Y increases from the front of the room to the back. Z increases from low to high (aka floor to ceiling). 🏠
Cameras in a Scene usually point in the direction of increasing Y, from front to back. 🎥 There could be multiple cameras pointing in different directions, though. If the physical space doesn't have a well-defined front and back, the Provider picks any direction for X, Y, and Z that stays consistent with right-handed coordinates. 🌟
7. Media Captures and Capture Scenes 💜
This section is all about how Providers can describe their Media content to Consumers. 🎀
7.1. Media Captures 📸
Media Captures are the basic building blocks of Streams that a device can transmit. ✨ What a Media Capture actually represents is super flexible:
- It can represent the immediate output of a physical source (e.g., camera, microphone) or a 'synthetic' source (e.g., laptop computer, DVD player). 💻
- It can represent the output of an audio mixer or video composer. 🎚️
- It can represent a concept, like 'the loudest speaker'. 🔊
- It can represent a conceptual position, like 'the leftmost Stream'. 👈
To tell multiple Capture instances apart, Captures have a unique identity. 🪪 For instance, VC1, VC2, AC1, and AC2 (where VC1 and VC2 are two different Video Captures and AC1 and AC2 are two different Audio Captures). 🦋
Some key points about Media Captures:
- A Media Capture is of a single Media type (e.g., audio or video). 🎧
- A Media Capture is defined in a Capture Scene and is given an Advertisement unique identity. The identity may be referenced outside the Capture Scene that defines it through an MCC. 🔗
- A Media Capture may be associated with one or more CSVs. 🍭
- A Media Capture has exactly one set of spatial information. 📍
- A Media Capture can be the source of at most one Capture Encoding. 💖
Each Media Capture can be associated with attributes that describe what it represents. 🌷
7.1.1. Media Capture Attributes 📸
Okay so Media Capture attributes are the extra details that describe each Capture, basically its little spec sheet 💅. A Provider uses them to describe the Captures for the Consumer of the Advertisement message 🫶. All of these are optional, and they fall into two groups:
- Spatial information, such as
Point of Capture,Point on Line of Capture, andArea of Capture, (all of which, in combination, define the capture field of, for example, a camera) 📷 - Other descriptive info to help the Consumer choose between Captures (e.g., description, presentation, view, priority, language, person information, and type) 🎀
The subsections below define each of the Capture attributes. Let's go 🧚
7.1.1.1. Point of Capture 📍
The Point of Capture attribute is a field with a single Cartesian (X, Y, Z) point value. It describes where the capturing device (like a camera) is located in space 🌸. For an Audio Capture with multiple microphones, the Point of Capture is the nominal midpoint of those microphones 🎤.
7.1.1.2. Point on Line of Capture 🧭
The Point on Line of Capture attribute is a field with a single Cartesian (X, Y, Z) point value. It describes the position in space of a second point on the axis of the capturing device, in the direction it's pointing. The first point is the Point of Capture (see above) ✨.
Put the Point of Capture and Point on Line of Capture together and you get the direction and axis of the capturing device. Think the optical axis of a camera or the axis of a microphone 🔥. The Media Consumer can use this info to adjust how it Renders the received Media, if it so chooses 💖.
For an Audio Capture, the Media Consumer can use this info along with the Audio Capture Sensitivity Pattern to define a three-dimensional volume of capture. That's the space where sounds can be expected to be picked up by the microphone providing this specific Audio Capture 🦋. If the Consumer wants to associate an Audio Capture with a Video Capture, it can compare this volume with the Area of Capture for the video Media. That's a check on whether the Audio Capture is actually spatially associated with the Video Capture 👑. For example, a video Area of Capture that doesn't intersect at all with the audio volume of capture is giving mismatch 💀. The same goes for one that's at such a long radial distance from the microphone Point of Capture that the audio level would be very low 🙈. Both would be inappropriate.
7.1.1.3. Area of Capture 🖼️
The Area of Capture is a field with a set of four (X, Y, Z) points as its value. It describes the spatial location of what is being "captured" 💅. This attribute applies only to Video Captures, not other types of Media 🎬. If you compare the Area of Capture for different Video Captures within the same Capture Scene, a Consumer can figure out the Spatial Relationships between them and Render them correctly 🌈.
The four points MUST be co-planar, forming a quadrilateral. That quadrilateral defines the Plane of Interest for the particular Media Capture 📐.
If the Area of Capture is not specified, it means the Video Capture might be spatially related to other Captures in the same Scene. But there's no detailed info on that relationship 🤷♀️. For a switched Capture that switches between different sections within a larger area, the Area of Capture MUST use coordinates for the larger potential area 🗺️.
7.1.1.4. Mobility of Capture 🏃♀️
The Mobility of Capture attribute tells you whether the Point of Capture, Point on Line of Capture, and Area of Capture values stay the same over time, or are expected to change (potentially a lot) 🔄. The possible values are static, dynamic, and highly dynamic 💜.
Here's an example of "dynamic": a camera on a stand that's occasionally hand-carried and placed at different positions to get the best angle on a work task 📷. A camera worn by someone who moves around the room is "highly dynamic" 🏃♀️. Either way, the Point of Capture, Capture Axis, and Area of Capture change with time ⏰.
The Point of Capture of a static Capture MUST NOT move for the life of the CLUE session. Period 🛑. The Point of Capture of dynamic Captures is marked by a change in position, followed by a reasonable period of stability, on the order of minutes ⏳. highly dynamic Captures have a Point of Capture that is constantly moving 🌀. If the Area of Capture, Point of Capture, and Point on Line of Capture attributes are included with dynamic or highly dynamic Captures, they indicate the spatial info at the time of the Advertisement 📝.
7.1.1.5. Audio Capture Sensitivity Pattern 🎙️
The Audio Capture Sensitivity Pattern attribute applies only to Audio Captures 🔊. It gives info about the nominal sensitivity pattern of the microphone that's the source of the Capture. Possible values include patterns such as omni, shotgun, cardioid, and hyper-cardioid 🦄.
7.1.1.6. Description 💬
The Description attribute is a human-readable description of the Capture, and it could be in multiple languages 🌍.
7.1.1.7. Presentation 🖥️
The Presentation attribute indicates that the Capture comes from a presentation device. That's one that provides supplementary info to a Conference through slides, video, still images, data, etc. 📊. When more is known about the Capture, it MAY be expanded hierarchically to show the different types of presentation Media, like presentation.slides, presentation.image, and so on 🌸.
Note: It is expected that a number of keywords will be defined that provide more detail on the type of presentation. Refer to [RFC8846] for how to extend the model. 📚
7.1.1.8. View 👀
The View attribute is a field with enumerated values. It says what type of view the Capture relates to, and the Consumer can use that to choose which Media Captures it wants to receive 🎯. The possible values are:
Room: Captures the entire Scene
Table: Captures the conference table with seated people
Individual: Captures an individual person
Lectern: Captures the region of the lectern including the
presenter, for example, in a classroom-style conference
room
Audience: Captures a region showing the audience in a classroom-
style conference room7.1.1.9. Language 🌐
The Language attribute indicates one or more languages used in the content of the Media Capture. Captures MAY be offered in different languages for multilingual and/or accessible Conferences 🫶. A Consumer can use this attribute to tell them apart and pick the right one 💅.
The Language attribute is defined and meaningful for both Audio and Video Captures ✨. For Audio Captures, the meaning is obvious 🔉. For a Video Capture, "Language" could be, for example, sign interpretation or text 🤟.
The Language attribute is coded per [RFC5646] 🧾.
7.1.1.10. Person Information 🧑🤝🧑
The Person Information attribute lets a Provider share specific info about the people in a Capture. This works whether or not the Capture has a Presentation attribute 💖. The Provider can gather the info automatically or manually from a variety of sources. The xCard [RFC6351] format is used to convey the info 📇. That covers Identification information (Section 6.2 of [RFC6350]), Communication Information (Section 6.4 of [RFC6350]), and Organizational information (Section 6.6 of [RFC6350]) 💼. A Consumer may then select Captures based on who is in them, either automatically (i.e., via a policy) or manually 🎀. It also lets a Consumer Render info about the people participating in the Conference, or use it for further processing 🧠.
The Provider may supply a minimal set of info or a larger set 🌷. However, it MUST be compliant to [RFC6350] and supply a VERSION and FN property 📋. A Provider may supply multiple xCards per Capture of any KIND (Section 6.1.4 of [RFC6350]) 🃏.
To keep CLUE messages compact 📦, the Provider SHOULD use a URI to point to any LOGO, PHOTO, or SOUND contained in the xCard. The Provider should not transmit the LOGO, PHOTO, or SOUND data in a CLUE message itself 🙅♀️.
Heads up: the SHOULD above is the actual requirement here, so don't skip it 🫣.
7.1.1.11. Person Type 🎭
The Person Type attribute indicates the type of people in the Capture with respect to the meeting agenda. This works whether or not the Capture has a Presentation attribute 📅. A Capture can include multiple people, so the attribute may contain multiple values. However, values MUST NOT be repeated within the attribute 🙅♀️.
An Advertiser associates a person type with an individual Capture when it knows that type is in the Capture 🔍. If the Advertiser can't link a particular type to a Capture with some certainty, that type isn't included 🤏. When a Consumer receives a Capture with a Person Type attribute, it knows with some certainty that the Capture contains that person type 💜. The Capture may contain other person types too, but the Advertiser hasn't been able to determine that 🤷♀️.
The types of Captured people are:
Chair: the person responsible for running the meeting
according to the agenda.
Vice-Chair: the person responsible for assisting the chair in
running the meeting.
Minute Taker: the person responsible for recording the minutes of
the meeting.
Attendee: the person has no particular responsibilities with
respect to running the meeting.
Observer: an Attendee without the right to influence the
discussion.
Presenter: the person scheduled on the agenda to make a
presentation in the meeting. Note: This is not
related to any "active speaker" functionality.
Translator: the person providing some form of translation or
commentary in the meeting.
Timekeeper: the person responsible for maintaining the meeting
schedule.On top of that, the Person Type attribute may contain one or more strings, which lets the Provider indicate custom meeting-specific types 🎀.
7.1.1.12. Priority 🥇
The Priority attribute indicates a relative priority between different Media Captures 👑. The Provider sets this priority, and the Consumer MAY use it to help decide which Captures it wants to receive 🎯.
The Priority attribute is an integer that indicates a relative priority between Captures ✨. For example, you can assign a priority between two presentation Captures so a remote Endpoint can tell which presentation is more important 📊. Priority is assigned at the individual Capture level 🫧. It represents the Provider's view of the relative priority between Captures that have a priority 🔢. The same priority number MAY be used across multiple Captures, which means they're equally important 🤝. If no priority is assigned, no assumptions about the relative importance of that Capture can be made 🙈.
7.1.1.13. Embedded Text 📝
The Embedded Text attribute indicates that a Capture provides embedded textual info 💅. For example, a Video Capture may contain speech-to-text info composed with the video image 🎬✨.
7.1.1.14. Related To ✨
Okay bestie, Related To is the attribute that says "this Capture is adding extra complementary info to another Capture" 💖. Its value is the identity of the other Capture that this one is supplementing 🦋.
For example, a Conference can have translators or facilitators who send an extra audio Stream (a translation, a description, or commentary on the Conference) 🎧. When there are several Captures available, a Consumer may find it useful to pick a complementary Capture instead of, or in addition to, the one it relates to 💅.
7.2. Multiple Content Capture 🌈
The MCC says that one or more Single Media Captures are multiplexed (temporally and/or spatially) or mixed into one Media Capture 🧩. Only one Capture type (audio, video, etc.) is allowed in each MCC instance 🚫. An MCC can hold references to its Single Media Captures (which can have their own attributes), plus attributes of its own 💜. An MCC can also contain other MCCs. The MCC MAY reference Captures from within the Capture Scene that defines it or from other Capture Scenes. The order in which Captures appear inside an MCC doesn't imply anything 🙈. An MCC MAY contain no references to other Captures, which tells you it holds content from multiple sources but gives no info about those sources. MCCs either contain the referenced Captures and no others, or have no referenced Captures and therefore may contain any Capture ✨.
One or more MCCs can also be listed in a CSV 🍓. This lets an Advertiser say that several MCC Captures are used to represent a Capture Scene. Table 14 provides an example of this case.
As outlined in Section 7.1, each MCC instance has its own Capture identity, such as MCC1. That lets every individual Capture inside the MCC be referenced through one MCC identity 🎀.
The example below shows a Multiple Content Capture in action:
+===================+=========================+
| Capture Scene #1 | |
+===================+=========================+
| VC1 | {MC attributes} |
+-------------------+-------------------------+
| VC2 | {MC attributes} |
+-------------------+-------------------------+
| VC3 | {MC attributes} |
+-------------------+-------------------------+
| MCC1(VC1,VC2,VC3) | {MC and MCC attributes} |
+-------------------+-------------------------+
| CSV(MCC1) | |
+-------------------+-------------------------+
Table 1: Multiple Content Capture ConceptThis shows that MCC1 is a single Capture containing VC1, VC2, and VC3, following any MCC1 attributes 🦋.
7.2.1. MCC Attributes
Media Capture attributes can be attached to the MCC instance and also to the Single Media Captures that the MCC references 🌸. A Provider should avoid giving conflicting attribute values between the MCC and its Single Media Captures. When there's a conflict, a Provider should override any attributes that may be present in the individual Captures with the attributes of the MCC 👑.
A Provider MAY include as much or as little of the original source Capture information as it needs 💅.
There are MCC-specific attributes that MUST only be used with Multiple Content Captures 🚨. They're covered in the sections below. The attributes described in Section 7.1.1 MAY also be used with MCCs 🫶.
The spatial attributes of an MCC indicate its Area of Capture and Point of Capture within the Scene, just like any other Media Capture 📍. That spatial info doesn't say anything about how the other Captures are composed inside the MCC 🙃.
For example, you could build a virtual Scene for the MCC Capture with two Video Captures, a MaxCaptures attribute set to 2, and an Area of Capture attribute covering the overall area 🌈. Each individual Capture could then also have an Area of Capture attribute covering a subset of that overall area. The Consumer would then know how each Capture relates to the others in the Scene, but not where the individual Captures sit within the composed Capture 😭.
+===============+===================================+
| Capture Scene | |
| #1 | |
+===============+===================================+
| VC1 | AreaofCapture=(0,0,0)(9,0,0) |
| | (0,0,9)(9,0,9) |
+---------------+-----------------------------------+
| VC2 | AreaofCapture=(10,0,0)(19,0,0) |
| | (10,0,9)(19,0,9) |
+---------------+-----------------------------------+
| MCC1(VC1,VC2) | MaxCaptures=2 |
| | AreaofCapture=(0,0,0)(19,0,0) |
| | (0,0,9)(19,0,9) |
+---------------+-----------------------------------+
| CSV(MCC1) | |
+---------------+-----------------------------------+
Table 2: Example of MCC and Single Media Capture
AttributesThe subsections below describe the MCC-only attributes 🧁.
7.2.1.1. MaxCapture: Maximum Number of Captures within an MCC
The MaxCaptures attribute says the maximum number of individual Captures that can appear in a Capture Encoding at one time 🎬. The actual number at any moment can be less than or equal to that max. It can be used to work out how the Single Media Captures inside the MCC are composed or switched across space and time 🧠.
A Provider can say that the number of Captures in an MCC Capture Encoding is exactly equal (=) to the MaxCaptures value, or that there can be any number up to and including (<=) the MaxCaptures value 💖. This lets a Provider tell apart an MCC that's purely a composition of sources from one that represents switched sources, or switched and composed sources 🔥.
MaxCaptures can be set to one, so only content from one of the sources shows in the MCC Capture Encoding at a time. It can also be set to any value up to the total number of Source Media Captures in the MCC 🌷.
The bullets below explain how the MaxCaptures setting, compared with the number of Captures in the MCC, affects how sources show up in a Capture Encoding:
- A switched case happens when MaxCaptures is set to
<= 1and the number of Captures in the MCC is greater than 1 (or not specified) in the MCC. Zero or one Captures may be switched into the Capture Encoding. Note: zero is allowed because of the<=🙃. - A switched case happens when MaxCaptures is set to
= 1and the number of Captures in the MCC is greater than 1 (or not specified) in the MCC. Only one Capture source is contained in a Capture Encoding at a time 🎯. - A switched and composed case happens when MaxCaptures is set to
<= N(with N > 1) and the number of Captures in the MCC is greater than N (or not specified). The Capture Encoding may contain purely switched sources (so<= 2allows for one source on its own), or it may contain composed and switched sources (a composition of two sources switched between the sources) 🦋. - A switched and composed case happens when MaxCaptures is set to
= N(with N > 1) and the number of Captures in the MCC is greater than N (or not specified). The Capture Encoding contains composed and switched sources (a composition of N sources switched between the sources). It is not possible to have a single source 🚫. - A switched and composed case happens when MaxCaptures is set
<=to the number of Captures in the MCC. The Capture Encoding may contain Media switched between any number (up to the MaxCaptures) of composed sources 🌈. - A composed case happens when MaxCaptures is set
=to the number of Captures in the MCC. All the sources are composed into a single Capture Encoding 🧩.
If this attribute isn't set, the default assumption is that all source Media Capture content can appear at the same time in the Capture Encoding associated with the MCC 💜.
For example, using MaxCaptures equal to 1 on an MCC with three Video Captures, VC1, VC2, and VC3, would mean the Advertiser switches between VC1, VC2, and VC3 in the Capture Encoding, because there can only be one Capture at a time 📺.
7.2.1.2. Policy
The Policy MCC attribute says what criteria the Provider uses to decide when and/or where Media content appears in the Capture Encoding related to the MCC 🎀.
The attribute is a token that names the policy, plus an index that represents an instance of that policy. The same index value can be used for multiple MCCs 🍓.
The tokens are:
SoundLevel: This means the content of the MCC is picked by a sound-level-detection algorithm. The loudest (active) speaker, or a previous speaker depending on the index value, is contained in the MCC 🔊.
RoundRobin: This means the content of the MCC is picked by a time-based algorithm. For example, the Provider gives content from one source for a period of time, then content from another source, and so on 🔁.
An index represents an instance of the policy setting. An index of 0 is the most current instance of the policy, i.e., the active speaker. 1 is the previous instance, i.e., the previous active speaker, and so on 🌟.
The example below shows a case where the Provider gives two Media Streams: one showing the active speaker, and a second Stream showing the previous speaker 🥰.
+==================+=====================+
| Capture Scene #1 | |
+==================+=====================+
| VC1 | |
+------------------+---------------------+
| VC2 | |
+------------------+---------------------+
| MCC1(VC1,VC2) | Policy=SoundLevel:0 |
| | MaxCaptures=1 |
+------------------+---------------------+
| MCC2(VC1,VC2) | Policy=SoundLevel:1 |
| | MaxCaptures=1 |
+------------------+---------------------+
| CSV(MCC1,MCC2) | |
+------------------+---------------------+
Table 3: Example Policy MCC Attribute
Usage7.2.1.3. SynchronizationID: Synchronization Identity ✨
okay so the SynchronizationID MCC attribute is all about how the Captures inside multiple MCCs stay in sync with each other 🎀 if you want the Capture Encodings for some MCCs to hold Captures from the same source at the same time, the Provider should give each of those MCCs the same SynchronizationID. bestie, the Provider is the one who decides what counts as a "source" 💅 so it can squish Single Media Captures together into one combined "source" and then switch them as a unit to keep everything synced by SynchronizationID. 🧚 for example, if the Provider is an MCU, it may decide that each separate CLUE Endpoint is a remote source of Media. and get this, the SynchronizationID can go across Media types too, so you can sync audio-related MCCs with video-related MCCs 🦋 it's giving teamwork.
no SynchronizationID at all? then it's assumed that multiple MCCs might be giving you content from different sources at any given moment 🌸
here's an example, bestie 👀
+=======================+=====================+
| Capture Scene #1 | |
+=======================+=====================+
| VC1 | Description=Left |
+-----------------------+---------------------+
| VC2 | Description=Center |
+-----------------------+---------------------+
| VC3 | Description=Right |
+-----------------------+---------------------+
| AC1 | Description=Room |
+-----------------------+---------------------+
| CSV(VC1,VC2,VC3) | |
+-----------------------+---------------------+
| CSV(AC1) | |
+=======================+=====================+
| Capture Scene #2 | |
+=======================+=====================+
| VC4 | Description=Left |
+-----------------------+---------------------+
| VC5 | Description=Center |
+-----------------------+---------------------+
| VC6 | Description=Right |
+-----------------------+---------------------+
| AC2 | Description=Room |
+-----------------------+---------------------+
| CSV(VC4,VC5,VC6) | |
+-----------------------+---------------------+
| CSV(AC2) | |
+=======================+=====================+
| Capture Scene #3 | |
+=======================+=====================+
| VC7 | |
+-----------------------+---------------------+
| AC3 | |
+=======================+=====================+
| Capture Scene #4 | |
+=======================+=====================+
| VC8 | |
+-----------------------+---------------------+
| AC4 | |
+=======================+=====================+
| Capture Scene #5 | |
+=======================+=====================+
| MCC1(VC1,VC4,VC7) | SynchronizationID=1 |
| | MaxCaptures=1 |
+-----------------------+---------------------+
| MCC2(VC2,VC5,VC8) | SynchronizationID=1 |
| | MaxCaptures=1 |
+-----------------------+---------------------+
| MCC3(VC3,VC6) | MaxCaptures=1 |
+-----------------------+---------------------+
| MCC4(AC1,AC2,AC3,AC4) | SynchronizationID=1 |
| | MaxCaptures=1 |
+-----------------------+---------------------+
| CSV(MCC1,MCC2,MCC3) | |
+-----------------------+---------------------+
| CSV(MCC4) | |
+-----------------------+---------------------+
Table 4: Example SynchronizationID MCC
Attribute Usageso this Advertisement says MCC1, MCC2, MCC3, and MCC4 together make up a Capture Scene 🌈 there are four Capture Encodings, one for each MCC. since MCC1 and MCC2 share a SynchronizationID, their Encodings will together only show content from Capture Scene 1, or only Capture Scene 2, or the combo of VC7 and VC8, at any given moment 😭 in this case the Provider decided the synced sources are Scene #1, Scene #2, and Scene #3 and #4 together. MCC3 has no SynchronizationID, so its Encoding isn't synced with MCC1 or MCC2 🙈 and MCC4 shares the same ID as MCC1 and MCC2, which means the audio is synced with the video. main character sync energy 🔥
7.2.1.4. Allow Subset Choice 🎀
the Allow Subset Choice MCC attribute is a boolean, so it's just true or false 💖 it tells you whether the Provider lets the Consumer pick a specific subset of the Captures that the MCC references. if it's true and the MCC references other Captures, the Consumer MAY pick a specific subset of them (using a Configure message) to be included in the MCC. then the Provider MUST include only that subset, no extras 🫶 but if it's false, or the MCC doesn't reference any other Captures, the Consumer MUST NOT pick a subset. 💅 no cap, that's the whole rule 🍓
7.3. Capture Scene 💅
Okay so, for a Consumer to actually use a Provider's individual Captures well, the Provider sorts them into one or more Capture Scenes 🫶. The structure and contents of those Capture Scenes get sent from the Provider to the Consumer in the Advertisement ✨.
A Capture Scene is a structure for a spatial region with one or more Capture Devices in it, and each device captures Media for a portion of that region 🌸. A Capture Scene includes one or more Capture Scene Views (CSVs), and each CSV holds one or more Media Captures of the same Media type 💜. There can also be Media Captures that aren't in any CSV, which is giving solo main character energy 🦋. For example, a Capture Scene could represent the video of a group of people sitting next to each other, plus the sound of their voices, and those could be some number of VCs and ACs in the CSVs 🎀. An MCU can also describe in Capture Scenes what it builds from the Media Streams it receives 🧚.
A Provider MAY advertise one or more Capture Scenes 💖. What counts as an entire Capture Scene is up to the Provider. A simple Provider might typically use one Capture Scene for participant Media (live video from the room cameras) and another Capture Scene for a computer-generated presentation. In more-complex systems, using additional Capture Scenes is also sensible. For example, a classroom may advertise two Capture Scenes involving live video: one including only the camera capturing the instructor (and associated audio) the other including camera(s) capturing students (and associated audio) 🍓.
A Capture Scene MAY (and typically will) include more than one type of Media. For example, a Capture Scene can include several CSVs for Video Captures and several CSVs for Audio Captures 🌈. A particular Capture MAY be included in more than one CSV 👑.
A Provider MAY express Spatial Relationships between Captures that are included in the same Capture Scene. However, there is no Spatial Relationship between Media Captures from different Capture Scenes. In other words, Capture Scenes each use their own spatial measurement system as outlined in Section 6 📐.
A Provider arranges Captures in a Capture Scene to help the Consumer choose which Captures it wants to Render. The CSVs in a Capture Scene are different alternatives the Provider is suggesting for representing the Capture Scene. Each CSV is given an advertisement-unique identity. The order of CSVs within a Capture Scene has no significance. The Media Consumer can choose to receive all Media Captures from one CSV for each Media type (e.g., audio and video), or it can pick and choose Media Captures regardless of how the Provider arranges them in CSVs. Different CSVs of the same Media type are not necessarily mutually exclusive alternatives. Also, the presence of multiple CSVs (with potentially multiple Encoding options in each view) in a given Capture Scene does not necessarily imply that a Provider is able to serve all the associated Media simultaneously (although building such an over-rich Capture Scene is probably not sensible in many cases) 🙈. What a Provider can send simultaneously is determined through the Simultaneous Transmission Set mechanism, described in Section 8 ✨.
Captures within the same CSV MUST be of the same Media type -- it is not possible to mix audio and Video Captures in the same CSV, for instance 🚫. The Provider MUST be capable of encoding and sending all Captures (that have an Encoding Group) in a single CSV simultaneously. The order of Captures within a CSV has no significance. A Consumer can decide to receive all the Captures in a single CSV, but it could also decide to receive just a subset of those Captures. A Consumer can also decide to receive Captures from different CSVs, all subject to the constraints set by Simultaneous Transmission Sets, as discussed in Section 8 💅.
When a Provider advertises a Capture Scene with multiple CSVs, it is basically signaling that there are multiple representations of the same Capture Scene available 🦋. Sometimes these multiple views would be used at the same time (for instance, a "video view" and an "audio view"). Other times the views would conceptually be alternatives (for instance, a view of three Video Captures covering the whole room versus a view of just a single Video Capture covering only the center of a room). In that latter example, one sensible choice for a Consumer would be to indicate (through its Configure and possibly through an additional offer/answer exchange) the Captures of that CSV that most closely match the Consumer's number of display devices or screen layout 🖥️.
The following is an example of four potential CSVs for an Endpoint-style Provider 🎀:
(VC0, VC1, VC2)- left, center, and right camera Video Captures(MCC3)- Video Capture associated with loudest room segment(VC4)- Video Capture zoomed out view of all people in the room(AC0)- main audio
The first view in this Capture Scene example is a list of Video Captures that have a Spatial Relationship to each other. Working out the order of these Captures (VC0, VC1, and VC2) for rendering purposes is done through their Area of Capture attributes 📍. The second view (MCC3) and the third view (VC4) are alternative representations of the same room's video, which might suit some Consumers' rendering capabilities better 🌸. Including the Audio Capture in the same Capture Scene shows that AC0 is associated with all of those Video Captures, meaning it comes from the same spatial region. So if audio were to be Rendered at all, this audio would be the correct choice, no matter which Video Captures were chosen 🔊.
7.3.1. Capture Scene Attributes
Capture Scene attributes can be applied to Capture Scenes as well as to individual Media Captures. Attributes specified at this level apply to all constituent Captures. Capture Scene attributes include the following:
- Human-readable description of the Capture Scene, which could be in multiple languages;
- xCard Scene information
- Scale information ("Millimeters", "Unknown Scale", "No Scale"), as described in Section 6.
7.3.1.1. Scene Information
The Scene Information attribute gives info about the Capture Scene itself, not the individual participants 🏛️. The Provider may gather the information automatically or manually from a variety of sources. It lets a Provider share things like organizational or geographic info, so a Consumer can figure out which Capture Scenes are of interest and then do Capture selection 🌍. It also lets a Consumer Render information about the Scene or use it for further processing ✨.
As per Section 7.1.1.10, the xCard format is used to convey this information, and the Provider may supply a minimal set of information or a larger set of information 🧾.
To keep CLUE messages compact, the Provider SHOULD use a URI to point to any LOGO, PHOTO, or SOUND contained in the xCard, rather than transmitting the LOGO, PHOTO, or SOUND data in a CLUE message 🪄.
7.3.2. Capture Scene View Attributes
A Capture Scene can include one or more CSVs in addition to the Capture-Scene-wide attributes described above. CSV attributes apply to the CSV as a whole, i.e., to all Captures that are part of the CSV 💜.
CSV attributes include the following:
- A human-readable description (which could be in multiple languages) of the CSV.
7.4. Global View List 🌐
An Advertisement can include an optional Global View list. Each item in this list is a Global View. The Provider can include multiple Global Views, so a Consumer can choose sets of Captures that fit its capabilities or application 🎯. How the Provider makes these suggestions in the Global View list for everything the Scenes can send is up to the Provider. It's very similar to how each CSV represents a particular Scene 👯.
Here's an example: suppose an Advertisement has three Scenes, and each Scene has three CSVs, ranging from one to three Video Captures in each CSV. The Provider is advertising a total of nine Video Captures across three Scenes 📹. The Provider can use the Global View list to suggest alternatives for Consumers that can't receive all nine Video Captures as separate Media Streams. For a Consumer that wants to receive three Video Captures, a Provider might suggest a Global View containing just a single CSV with three Captures and nothing from the other two Scenes. Or a Provider might suggest a Global View containing three different CSVs, one from each Scene, with a single Video Capture in each 🫧.
Some additional rules:
- The ordering of Global Views in the Global View list is insignificant.
- The ordering of CSVs within each Global View is insignificant.
- A particular CSV may be used in multiple Global Views.
- The Provider must be capable of encoding and sending all Captures within the CSVs of a given Global View simultaneously.
Here's a figure showing what the structure of Global Views in a Global View List looks like 🦋:
........................................................
. Advertisement .
. .
. +--------------+ +-------------------------+ .
. |Scene 1 | |Global View List | .
. | | | | .
. | CSV1 (v)<----------------- Global View (CSV 1) | .
. | <-------. | | .
. | | *--------- Global View (CSV 1,5) | .
. | CSV2 (v) | | | | .
. | | | | | .
. | CSV3 (v)<---------*------- Global View (CSV 3,5) | .
. | | | | | | .
. | CSV4 (a)<----------------- Global View (CSV 4) | .
. | <-----------. | | .
. +--------------+ | | *----- Global View (CSV 4,6) | .
. | | | | | .
. +--------------+ | | | +-------------------------+ .
. |Scene 2 | | | | .
. | | | | | .
. | CSV5 (v)<-------' | | .
. | <---------' | .
. | | | (v) = video .
. | CSV6 (a)<-----------' (a) = audio .
. | | .
. +--------------+ .
`......................................................'Figure 3: Global View List Structure
8. Simultaneous Transmission Set Constraints ✨
Okay bestie, so a Provider often has limits on how many Captures it can send at the same time 💅. The first kind of limit comes from the physical capture hardware, and those limits are shown with a Simultaneous Transmission Set. The second kind comes from the encoding resources it has, like bandwidth or video encoding throughput (macroblocks/second). That one gets handled by Individual Encodings and Encoding Groups, which we'll get to soon 🌸.
Some Endpoints or MCUs can send several Captures at once, but sometimes certain Captures just can't be sent together with others. A device might not be able to do two jobs at the same time 🙈. So Provider Advertisements let the Consumer pick one of several mutually exclusive ways to use the device. That's what a Simultaneous Transmission Set is for. It lists all the Captures of one particular Media type (like audio, video, or text) that can be sent at the same time, and each Media type in the Advertisement gets its own set. It's easier to see with an example 🦋.
Picture a room system with three cameras. Each camera can send its own Capture covering two people: VC0, VC1, and VC2. The middle camera can also zoom out with its optical zoom lens and show all six people, which is VC3. But the middle camera can't do both at once. It has to show either the space where two participants sit or all six seats, never both. So VC1 and VC3 can't be sent at the same time 💖.
Simultaneous Transmission Sets are written as sets of the Media Captures the Provider could transmit at the same time, even though that isn't always intuitive 🧚. If a Multiple Content Capture is in a Simultaneous Transmission Set, that means the Capture Encoding attached to it could be transmitted at the same time as the other Captures in that set. It does not mean that every Single Media Capture inside the Multiple Content Capture could be sent together ✨.
In this example, the two Simultaneous Transmission Sets look like Table 5.
+===================+
| Simultaneous Sets |
+===================+
| {VC0, VC1, VC2} |
+-------------------+
| {VC0, VC3, VC2} |
+-------------------+
Table 5: Two
Simultaneous
Transmission SetsIf a Provider advertises one or more mutually exclusive Simultaneous Transmission Sets, then for each Media type, the Consumer MUST ensure that the Media Captures it chooses lie wholly within one of those Simultaneous Transmission Sets 🔥.
A Provider OPTIONALLY can include Simultaneous Transmission Sets in its Advertisement 💜. These constraints apply across all the Capture Scenes in the Advertisement. As a syntax-conformance requirement, the Simultaneous Transmission Sets MUST allow all the Media Captures in any particular CSV to be used at the same time. In the same way, the Simultaneous Transmission Sets MUST reflect the simultaneity that any Global View expresses 👑.
For shorthand, a Provider MAY describe a Simultaneous Transmission Set in terms of CSVs and Capture Scenes 🎀. If a CSV is in a Simultaneous Transmission Set, then every Media Capture in that CSV is in the set too. If a Capture Scene is in a Simultaneous Transmission Set, then all of its CSVs (for the matching Media type) are in the set too. Either way, it boils down to a set of Media Captures of one particular Media type 🌈.
If an Advertisement doesn't include Simultaneous Transmission Sets, then the Provider MUST be able to provide all the Captures from any one CSV of each Media type, from each Capture Scene, at the same time 😭. Likewise, if there are no Simultaneous Transmission Sets but there is a Global View list, then the Provider MUST be able to provide all the Captures from any particular Global View (for each Media type) from that list at the same time 🫶.
If an Advertisement includes multiple CSVs in a Capture Scene, then the Consumer MAY choose one CSV for each Media type, or it MAY choose individual Captures based on the Simultaneous Transmission Sets 💅.
9. Encodings 🦋
Individual Encodings and Encoding Groups are CLUE's tools for letting a Provider signal its limits on sending Captures, or combinations of Captures, to a Consumer ✨. The Consumer can map the Captures it wants onto the Encodings, using whatever Encoding parameters it wants 💖. For how the CLUE Encoding mechanisms relate to the SIP offer/answer exchange, check out Section 5 🌸.
9.1. Individual Encodings
An Individual Encoding is a way to encode a Media Capture as a Capture Encoding, which then gets sent as an encoded Media Stream from the Provider to the Consumer 🎀. Each Individual Encoding has a set of parameters that describe how the Media is encoded.
Different Media types have different parameters, and different encoding algorithms can have different parameters too 🧚. An Individual Encoding can be assigned to at most one Capture Encoding at any given time.
Individual Encoding parameters are represented in SDP [RFC4566], not in CLUE messages 🌈. For example, a video Encoding using H.26x compression technologies can include parameters such as the following:
- Maximum bandwidth;
- Maximum picture size in pixels;
- Maximum number of pixels to be processed per second;
The bandwidth parameter is the only one that directly relates to a CLUE Advertisement, because it can be further constrained by the maximum group bandwidth in an Encoding Group 👑.
9.2. Encoding Group
An Encoding Group includes a set of one or more Individual Encodings, along with parameters that apply to the group as a whole 🦋. By grouping several Individual Encodings together, an Encoding Group adds extra bandwidth constraints for the whole group. A single Encoding Group MAY refer to Encodings for different Media types 💜.
The Encoding Group data structure contains:
- Maximum bitrate for all Encodings in the group combined;
- A list of identifiers for the Individual Encodings belonging to the group.
When the Individual Encodings in a group are instantiated into Capture Encodings, each Capture Encoding has a bitrate that MUST be less than or equal to the max bitrate for that particular Individual Encoding 🔥. The "maximum bitrate for all Encodings in the group" parameter adds one more restriction: the sum of all the individual Capture Encoding bitrates MUST be less than or equal to this group value 😭.
Here's one example of how a Media Provider's Encoding Groups and their contents can be laid out 🧚.
,-------------------------------------------------.
| Media Provider |
| |
| ,--------------------------------------. |
| | ,--------------------------------------. |
| | | ,--------------------------------------. |
| | | | Encoding Group | |
| | | | ,-----------. | |
| | | | | | ,---------. | |
| | | | | | | | ,---------.| |
| | | | | Encoding1 | |Encoding2| |Encoding3|| |
| `.| | | | | | `---------'| |
| `.| `-----------' `---------' | |
| `--------------------------------------' |
`-------------------------------------------------'
Figure 4: Encoding Group StructureA Provider advertises one or more Encoding Groups, and each Encoding Group includes one or more Individual Encodings 🌸. Each Individual Encoding can be a different way of encoding Media. For instance, one Individual Encoding might be 1080p60 video, another could be 720p30, and a third could be 352x288p30, all in H.264 format, for example 🎀.
A typical three-codec/display system might have one Encoding Group per "codec box" (a physical codec connected to one camera and one screen). But a Provider could offer many different numbers of Encoding Groups, and many different Encoding values inside each one 💖.
There's no requirement for every Encoding in an Encoding Group to be instantiated at the same time 🌈.
9.3. Associating Captures with Encoding Groups
Each Media Capture, including MCCs, MAY be associated with one Encoding Group 🦋. To be eligible for configuration, a Media Capture MUST be associated with one Encoding Group, and that group is what gets used to instantiate the Capture into a Capture Encoding ✨. When an MCC is configured, all the Media Captures it references appear in the Capture Encoding according to the attributes of the MCC's chosen Encoding. This lets an Advertiser specify Encoding attributes for Media Captures without having to provide an individual Capture Encoding for each input 💅.
If an Encoding Group is assigned to a Media Capture referenced by the MCC, that means this Capture may also have an individual Capture Encoding 👑.
For example:
+==================+=================+
| Capture Scene #1 | |
+==================+=================+
| VC1 | EncodeGroupID=1 |
+------------------+-----------------+
| VC2 | |
+------------------+-----------------+
| MCC1(VC1,VC2) | EncodeGroupID=2 |
+------------------+-----------------+
| CSV(VC1) | |
+------------------+-----------------+
| CSV(MCC1) | |
+------------------+-----------------+
Table 6: Example Usage of Encoding
with MCC and Source CapturesThis tells us that VC1 may be sent as its own Capture Encoding from EncodeGroupID=1, or it may be sent as part of a Capture Encoding from EncodeGroupID=2 along with VC2 💜.
More than one Capture MAY use the same Encoding Group 🧚.
The maximum number of Capture Encodings that can come from a particular Encoding Group constraint equals the number of Individual Encodings in that group 🌈. The actual number of Capture Encodings used at any time MAY be less than this maximum. Any of the Captures that use a particular Encoding Group can be encoded according to any of the Individual Encodings in that group 🎀.
As a protocol conformance requirement, the Encoding Groups MUST allow all the Captures in a particular CSV to be used at the same time 🫶.
10. Consumer's Choice of Streams to Receive from the Provider ✨
okay bestie, once the Provider sends its Advertisement (the one with the Media Captures and their constraints), the Consumer writes back with a Configure message 💌 The Consumer is totally free to use that Advertisement however it likes, but there are a few obviously sensible moves, and I'll walk you through them below 🦋
If multiple Providers connect to the same Consumer (i.e., in an MCU-less multiparty call), the Consumer is the one responsible for building a Configure for each Provider. Each Configure has to fulfill that Provider's constraints from its Advertisement AND match the Consumer's own capabilities. that's a lot of juggling, periodt 🤹♀️
In an MCU-based multiparty call, the MCU can logically terminate the Advertisement/Configure negotiation. It can hide the characteristics of the receiving Endpoint and rely on its own capabilities (transcoding/transrating/etc.) to create Media Streams that can be decoded at the Endpoint Consumers. 🧚 When the MCU sends Advertisements (for its outgoing ports) and Configures (for its incoming ports, in response to Advertisements received there) is up to the MCU and is implementation dependent, so there's no one set timing rule here 🙈
As a general outline, a Consumer can look at the Advertisement it received and choose which Captures it wants to receive, plus which Individual Encodings it wants the Provider to use to encode those Captures. the main character decisions, lol 👑
On receipt of an Advertisement with an MCC, the Consumer treats the MCC like any other non-MCC Capture, with a few differences:
- The Consumer understands that the MCC is a Capture that includes the referenced individual Captures (or any Captures, if none are referenced), and that these individual Captures are delivered as part of the MCC's Capture Encoding.
- The Consumer may use any of the attributes associated with the referenced individual Captures, and any Capture Scene attributes from where the individual Captures were defined, to choose Captures and to make Rendering decisions. 🎀
- If the MCC attribute Allow Subset Choice is true, the Consumer may or may not choose to receive all the indicated Captures. It can choose to receive just a subset of the Captures the MCC indicates. ✂️
For example, if the Consumer receives:
MCC1(VC1,VC2,VC3){attributes}A Consumer could choose all the Captures within an MCC. however, if it decides it doesn't want VC3, it can return:
MCC1(VC1,VC2)If it wants all the individual Captures, it returns only the MCC identity (i.e., MCC1). If the MCC in the Advertisement does not reference any individual Captures, or the Allow Subset Choice attribute is false, then the Consumer cannot choose what is included in the MCC. it's up to the Provider to decide 💀
A Configure Message includes a list of Capture Encodings. These are the Capture Encodings the Consumer wishes to receive from the Provider. Each Capture Encoding refers to one Media Capture and one Individual Encoding. 🌈
For each Capture the Consumer wants to receive, it configures one of the Encodings in that Capture's Encoding Group. It does this by telling the Provider, in its Configure Message, which Encoding to use for each chosen Capture. Once the Provider gets this Configure from the Consumer, common knowledge is established between them about sensible choices for the Media Streams. 🤝 The setup of the actual Media channels, at least in the simplest case, is left to a following offer/answer exchange. Optimized implementations may speed up the reaction to that offer/answer exchange by reserving resources at the time the CLUE handshake is finalized ⚡
CLUE Advertisements and Configure Messages don't necessarily need a new SDP offer/answer for every CLUE message exchange. But the resulting Encodings sent via RTP must conform to the most-recent SDP offer/answer result. 📝
To meaningfully create and send an initial Configure, the Consumer needs to have received at least one Advertisement, and an SDP offer defining the Individual Encodings, from the Provider. 🎁
The Consumer can also send a Configure at any time during the call. The Configure MUST be valid according to the most recently received Advertisement. The Consumer can send a Configure either in response to a new Advertisement from the Provider or on its own, for example because of a local change in conditions (people leaving the room, connectivity changes, multipoint related considerations). 🚪
When choosing which Media Streams to receive from the Provider, and the encoding characteristics of those Media Streams, the Consumer advantageously takes several things into account: its local preference, simultaneity restrictions, and encoding limits. 💅
10.1. Local Preference
A bunch of local factors shape the Consumer's choice of Media Streams to receive from the Provider:
- If the Consumer is an Endpoint, it would likely choose, where possible, to receive Video and Audio Captures that match the number of display devices and audio system it has. 📺
- If the Consumer is an MCU, it may choose to receive loudest speaker Streams (so it can do its own Media composition) and avoid pre-composed Video Captures. 🎤
- User choice (for instance, picking a new layout) may result in a different set of Captures, or different Encoding characteristics, being required by the Consumer. 🫶
10.2. Physical Simultaneity Restrictions
Physical simultaneity constraints on the Provider often affect its ability to send all the Captures the Consumer would like to receive at the same time. For instance, an MCU connected to a multi-camera room system might prefer to receive both the individual video Streams of the people in the room and an overall view of the room from a single camera. Some Endpoint systems might be able to provide both sets of Streams simultaneously, whereas others might not (if the overall room view were produced by changing the optical zoom level on the center camera, for instance). 📷
10.3. Encoding and Encoding Group Limits
Each of the Provider's Encoding Groups has bandwidth limits, and the potential Encodings inside them have limits on bandwidth, computational complexity, video frame rate, and resolution. When choosing the Captures to receive from a Provider, a Consumer device MUST ensure that the Encoding characteristics requested for each individual Capture fit within the capability of the Encoding it is being configured to use, AND that the combined Encoding characteristics for Captures fit within the capabilities of their associated Encoding Groups. 🧮 Sometimes this means an otherwise "preferred" choice of Capture Encodings gets passed over for different ones. For example, if a set of three Captures could only be provided at a low resolution, a three-screen device could switch to favoring a single, higher quality, Capture Encoding. 🖥️
11. Extensibility 🌸
One important characteristic of the Framework is its extensibility. The standard for interoperability and handling multiple Streams must be future-proof. ✨ The framework itself is inherently extensible through expanding the data model types. For example:
- Adding more types of Media, such as telemetry, can be done by defining additional types of Captures in addition to audio and video. 📡
- Adding new functionalities, such as 3-D Video Captures, may require additional attributes describing the Captures. 🧊
The infrastructure is designed to be extended rather than requiring new infrastructure elements. Extension comes through adding to defined types. 🧩
12. Examples - Using the Framework (Informative) 🎀
This section gives some examples, first from the point of view of the Provider, then the Consumer, then some multipoint scenarios. 🦋
12.1. Provider Behavior
This section shows some examples, in more detail, of how a Provider can use the framework to represent a typical case for telepresence rooms. First, an Endpoint is illustrated, then an MCU case is shown. 📹
12.1.1. Three-Screen Endpoint Provider ✨
Okay bestie, let's look at an Endpoint with three cameras, three displays, and a six-person table 🪑💖
- Each camera can provide one Capture for each 1/3-section of the table. 📸
- A single Capture representing the active speaker can be provided (voice-activity-based camera selection to a given encoder input port implemented locally in the Endpoint). 🎤
- A single Capture representing the active speaker with the other two Captures shown picture in picture (PiP) within the Stream can be provided (again, implemented inside the Endpoint). 🪟
- A Capture showing a zoomed out view of all six seats in the room can be provided. 🔭
The Video and Audio Captures for this Endpoint can be described as follows.
Video Captures:
VC0 (the left camera Stream), Encoding Group=EG0, view=table
VC1 (the center camera Stream), Encoding Group=EG1, view=table
VC2 (the right camera Stream), Encoding Group=EG2, view=table
MCC3 (the loudest panel Stream), Encoding Group=EG1, view=table,
MaxCaptures=1, policy=SoundLevel
MCC4 (the loudest panel Stream with PiPs), Encoding Group=EG1,
view=room, MaxCaptures=3, policy=SoundLevel
VC5 (the zoomed out view of all people in the room), Encoding
Group=EG1, view=room
VC6 (presentation Stream), Encoding Group=EG1, presentationThe following diagram is a top view of the room with three cameras, three displays, and six seats. Each camera captures two people. The six seats are not all in a straight line. 👀
,-. d
( )`--.__ +---+
`-' / `--.__ | |
,-. | `-.._ |_-+Camera 2 (VC2)
( ).' <--(AC1)-+-''`+-+
`-' |_...---'' | |
,-.c+-..__ +---+
( )| ``--..__ | |
`-' | ``+-..|_-+Camera 1 (VC1)
,-. | <--(AC2)..--'|+-+ ^
( )| __..--' | | |
`-'b|..--' +---+ |X
,-. |``---..___ | | |
( )\ ```--..._|_-+Camera 0 (VC0) |
`-' \ <--(AC0) ..-''`-+ |
,-. \ __.--'' | | <----------+
( ) |..-'' +---+ Y
`-' a (0,0,0) origin is under Camera 1
Figure 5: Room Layout Top ViewThe two points labeled 'b' and 'c' are intended to be at the midpoint between the seating positions, and where the fields of view of the cameras intersect. 📍
The Plane of Interest for VC0 is a vertical plane that intersects points 'a' and 'b'.
The Plane of Interest for VC1 intersects points 'b' and 'c'. The plane of interest for VC2 intersects points 'c' and 'd'.
This example uses an area scale of millimeters. 📏
Areas of capture:
bottom left bottom right top left top right
VC0 (-2011,2850,0) (-673,3000,0) (-2011,2850,757) (-673,3000,757)
VC1 ( -673,3000,0) ( 673,3000,0) ( -673,3000,757) ( 673,3000,757)
VC2 ( 673,3000,0) (2011,2850,0) ( 673,3000,757) (2011,3000,757)
MCC3(-2011,2850,0) (2011,2850,0) (-2011,2850,757) (2011,3000,757)
MCC4(-2011,2850,0) (2011,2850,0) (-2011,2850,757) (2011,3000,757)
VC5 (-2011,2850,0) (2011,2850,0) (-2011,2850,757) (2011,3000,757)
VC6 nonePoints of capture:
VC0 (-1678,0,800)
VC1 (0,0,800)
VC2 (1678,0,800)
MCC3 none
MCC4 none
VC5 (0,0,800)
VC6 noneIn this example, the right edge of the VC0 area lines up with the left edge of the VC1 area. It doesn't have to be this way. There could be a gap or an overlap. 🧩 One additional thing to note for this example is the distance from 'a' to 'b' is equal to the distance from 'b' to 'c' and the distance from 'c' to 'd'. All these distances are 1346 mm. This is the planar width of each Area of Capture for VC0, VC1, and VC2.
Note the text in parentheses (e.g., "the left camera Stream") is not explicitly part of the model, it is just explanatory text for this example, and it is not included in the model with the Media Captures and attributes. Also, MCC4 doesn't say anything about how a Capture is composed, so the Media Consumer can't tell based on this Capture that MCC4 is composed of a "loudest panel with PiPs". 🙈
Audio Captures:
Three ceiling microphones are located between the cameras and the table, at the same height as the cameras. The microphones point down at an angle toward the seating positions. 🎙️
- AC0 (left), Encoding Group=EG3
- AC1 (right), Encoding Group=EG3
- AC2 (center), Encoding Group=EG3
- AC3 being a simple pre-mixed audio Stream from the room (mono), Encoding Group=EG3
- AC4 audio Stream associated with the presentation video (mono) Encoding Group=EG3, presentation
Point of Capture: Point on Line of Capture:
AC0 (-1342,2000,800) (-1342,2925,379)
AC1 ( 1342,2000,800) ( 1342,2925,379)
AC2 ( 0,2000,800) ( 0,3000,379)
AC3 ( 0,2000,800) ( 0,3000,379)
AC4 noneThe physical simultaneity information is:
Simultaneous Transmission Set #1 {VC0, VC1, VC2, MCC3, MCC4, VC6}
Simultaneous Transmission Set #2 {VC0, VC2, VC5, VC6}This constraint indicates that it is not possible to use all the VCs at the same time. VC5 cannot be used at the same time as VC1 or MCC3 or MCC4. Also, using every member in the set simultaneously may not make sense -- for example, MCC3 (loudest) and MCC4 (loudest with PiP). 🤷♀️ In addition, there are Encoding constraints that make choosing all of the VCs in a set impossible. VC1, MCC3, MCC4, VC5, and VC6 all use EG1 and EG1 has only three ENCs. This constraint shows up in the Encoding Groups, not in the Simultaneous Transmission Sets. 🧠
In this example, there are no restrictions on which Audio Captures can be sent simultaneously. 🎶
Encoding Groups:
This example has three Encoding Groups associated with the Video Captures. Each group can have three Encodings, but with each potential Encoding having a progressively lower specification. In this example, 1080p60 transmission is possible (as ENC0 has a maxPps value compatible with that). Significantly, as up to three Encodings are available per group, it is possible to transmit some Video Captures simultaneously that are not in the same view in the Capture Scene, for example, VC1 and MCC3 at the same time. ✨ The information below about Encodings is a summary of what would be conveyed in SDP, not directly in the CLUE Advertisement.
encodeGroupID=EG0, maxGroupBandwidth=6000000
encodeID=ENC0, maxWidth=1920, maxHeight=1088, maxFrameRate=60,
maxPps=124416000, maxBandwidth=4000000
encodeID=ENC1, maxWidth=1280, maxHeight=720, maxFrameRate=30,
maxPps=27648000, maxBandwidth=4000000
encodeID=ENC2, maxWidth=960, maxHeight=544, maxFrameRate=30,
maxPps=15552000, maxBandwidth=4000000
encodeGroupID=EG1 maxGroupBandwidth=6000000
encodeID=ENC3, maxWidth=1920, maxHeight=1088, maxFrameRate=60,
maxPps=124416000, maxBandwidth=4000000
encodeID=ENC4, maxWidth=1280, maxHeight=720, maxFrameRate=30,
maxPps=27648000, maxBandwidth=4000000
encodeID=ENC5, maxWidth=960, maxHeight=544, maxFrameRate=30,
maxPps=15552000, maxBandwidth=4000000
encodeGroupID=EG2 maxGroupBandwidth=6000000
encodeID=ENC6, maxWidth=1920, maxHeight=1088, maxFrameRate=60,
maxPps=124416000, maxBandwidth=4000000
encodeID=ENC7, maxWidth=1280, maxHeight=720, maxFrameRate=30,
maxPps=27648000, maxBandwidth=4000000
encodeID=ENC8, maxWidth=960, maxHeight=544, maxFrameRate=30,
maxPps=15552000, maxBandwidth=4000000Figure 6: Example Encoding Groups for Video
For audio, there are five potential Encodings available, so all five Audio Captures can be encoded at the same time. 🔊
encodeGroupID=EG3, maxGroupBandwidth=320000
encodeID=ENC9, maxBandwidth=64000
encodeID=ENC10, maxBandwidth=64000
encodeID=ENC11, maxBandwidth=64000
encodeID=ENC12, maxBandwidth=64000
encodeID=ENC13, maxBandwidth=64000Figure 7: Example Encoding Group for Audio
Capture Scenes:
The following table represents the Capture Scenes for this Provider. Recall that a Capture Scene is composed of alternative CSVs covering the same spatial region. Capture Scene #1 is for the main people Captures, and Capture Scene #2 is for presentation. 🎬
Each row in the table is a separate CSV.
+==================+
| Capture Scene #1 |
+==================+
| VC0, VC1, VC2 |
+------------------+
| MCC3 |
+------------------+
| MCC4 |
+------------------+
| VC5 |
+------------------+
| AC0, AC1, AC2 |
+------------------+
| AC3 |
+==================+
| Capture Scene #2 |
+==================+
| VC6 |
+------------------+
| AC4 |
+------------------+Table 7: Example CSVs
Different Capture Scenes are distinct from each other and do not overlap. A Consumer can choose a view from each Capture Scene. In this case, the three Captures, VC0, VC1, and VC2, are one way of representing the video from the Endpoint. These three Captures should appear adjacent to each other. Alternatively, another way of representing the Capture Scene is with the Capture MCC3, which automatically shows the person who is talking; this is the same for the MCC4 and VC5 alternatives. 🎀
As in the video case, the different views of audio in Capture Scene #1 represent the "same thing", in that one way to receive the audio is with the three Audio Captures (AC0, AC1, and AC2), and another way is with the mixed AC3. The Media Consumer can choose an audio CSV it is capable of receiving. 🎧
The spatial ordering is understood by the Media Capture attribute's Area of Capture, Point of Capture, and Point on Line of Capture. 🗺️
A Media Consumer would likely want to choose a CSV to receive, partially based on how many Streams it can simultaneously receive. A Consumer that can receive three video Streams would probably prefer to receive the first view of Capture Scene #1 (VC0, VC1, and VC2) and not receive the other views. A Consumer that can receive only one video Stream would probably choose one of the other views. 💅
If the Consumer can receive a presentation Stream too, it would also choose to receive the only view from Capture Scene #2 (VC6). 🌸
12.1.2. Encoding Group Example
Okay so this is a cute example of an Encoding Group showing how it can express dependencies between Encodings 🎀 Heads up: the info below is a summary of what would be conveyed in SDP, not directly in the CLUE Advertisement 💅
encodeGroupID=EG0 maxGroupBandwidth=6000000
encodeID=VIDENC0, maxWidth=1920, maxHeight=1088,
maxFrameRate=60, maxPps=62208000, maxBandwidth=4000000
encodeID=VIDENC1, maxWidth=1920, maxHeight=1088,
maxFrameRate=60, maxPps=62208000, maxBandwidth=4000000
encodeID=AUDENC0, maxBandwidth=96000
encodeID=AUDENC1, maxBandwidth=96000
encodeID=AUDENC2, maxBandwidth=96000So the group here is EG0 🦋 It can push up to 6 Mbit/s in total, but no single video Encoding is allowed to go over 4 Mbit/s. That's the whole vibe: the group has a bigger budget, but each video Encoding gets its own cap ✨
EG0 also has room for up to three audio Encodings, AUDENC<0-2> 🎶 Audio and video don't have to live in the same Encoding Group, which is totally fine. BUT if they do share one, then the group's overall maxBandwidth value becomes a limit on the sum of all the audio and video Encodings the Consumer sets up. Main character energy for the group, but it means everything inside is fighting over the same pot 🍓
If a system doesn't want or need to combine bandwidth limits like that, it should instead use separate Encoding Groups for audio and video. That way the audio and video limits don't mess with each other 💜
Here's how it looks when audio and video get their own Encoding Groups 👑
encodeGroupID=EG0 maxGroupBandwidth=6000000
encodeID=VIDENC0, maxWidth=1920, maxHeight=1088,
maxFrameRate=60, maxPps=62208000, maxBandwidth=4000000
encodeID=VIDENC1, maxWidth=1920, maxHeight=1088,
maxFrameRate=60, maxPps=62208000, maxBandwidth=4000000
encodeGroupID=EG1 maxGroupBandwidth=500000
encodeID=AUDENC0, maxBandwidth=96000
encodeID=AUDENC1, maxBandwidth=96000
encodeID=AUDENC2, maxBandwidth=9600012.1.3. The MCU Case
This part shows how an MCU might lay out its Capture Scenes 🌈 The goal is to offer different choices for Consumers that can handle different numbers of Streams. Each MCC is for video. There's one Audio Capture for all the single and multi-screen setups, and it can be associated (think lip-synced 🫦) with any combo of Video Captures (the MCCs) on the Consumer's side.
+==========================+==================================+
| Capture Scene #1 | |
+==========================+==================================+
| MCC | for a one-screen Consumer |
+--------------------------+----------------------------------+
| MCC1, MCC2 | for a two-screen Consumer |
+--------------------------+----------------------------------+
| MCC3, MCC4, MCC5 | for a three-screen Consumer |
+--------------------------+----------------------------------+
| MCC6, MCC7, MCC8, MCC9 | for a four-screen Consumer |
+--------------------------+----------------------------------+
| AC0 | AC representing all participants |
+--------------------------+----------------------------------+
| CSV(MCC0) | |
+--------------------------+----------------------------------+
| CSV(MCC1,MCC2) | |
+--------------------------+----------------------------------+
| CSV(MCC3,MCC4,MCC5) | |
+--------------------------+----------------------------------+
| CSV(MCC6,MCC7,MCC8,MCC9) | |
+--------------------------+----------------------------------+
| CSV(AC0) | |
+--------------------------+----------------------------------+Table 8: MCU Main Capture Scenes
Once a presentation Stream goes live in the Conference, the MCU might re-advertise its Media like this 📽️✨
+==================+======================================+
| Capture Scene #2 | Note |
+==================+======================================+
| VC10 | Video Capture for presentation |
+------------------+--------------------------------------+
| AC1 | Presentation audio to accompany VC10 |
+------------------+--------------------------------------+
| CSV(VC10) | |
+------------------+--------------------------------------+
| CSV(AC1) | |
+------------------+--------------------------------------+Table 9: MCU Presentation Capture Scene
12.2. Media Consumer Behavior
This section shows an example of how a Media Consumer might act when it's deciding which Streams to request from the three-screen Endpoint we just looked at 👀
The receive side of a call has to balance a few things at once: what it needs (based on its number of screens and speakers), what it can decode, how much bandwidth it has, and what the Provider is capable of. All of that goes into optimally configuring the Provider's Streams 🧠 Usually, it'd want to receive and decode Media from each Capture Scene the Provider advertises.
A sane, basic algorithm could go something like this: the Consumer goes through each CSV one at a time and finds the group of Video Captures that best fits its number of screens. (This might also consider which screens are for presentation video rather than "people" video.) Then it picks between the alternative views in the video Capture Scenes, either from hard-coded preferences or from user choice 🎀 Once that's decided, the Consumer figures out how to configure the Provider's Encoding Groups so it makes the best use of the available network bandwidth and its own decoding abilities 💖
12.2.1. One-Screen Media Consumer
MCC3, MCC4, and VC5 are all different views by themselves, not grouped together in one view, so the receiving device has to choose one of them 🫠 It comes down to whether you want to see the most participants at once at roughly equal priority (VC5), a switched view of just the loudest region (MCC3), or a switched view with PiPs (MCC4). An Endpoint device that knows a little bit about these differences could offer a dynamic choice of these options to the user, in-call 💅
12.2.2. Two-Screen Media Consumer Configuring the Example
Mixing systems with an even number of screens, 2n, and systems with 2n+1 cameras (and the other way around) is always a likely trouble spot 😭 Here, the behavior probably depends on whether a "two-screen" system is really a "two-decoder" system. In other words: can only one received Stream be shown per screen, or can more than two Streams be received and spread across the available screen area? Here are three possible behaviors for a two-screen system that learns the far end is "ideally" expressed via three Capture Streams:
- Fall back to receiving just one Stream (
MCC3,MCC4, orVC5, as in the one-screen Consumer case above) and either leave one screen blank or use it for a presentation if/when one becomes active. - Receive three Streams (
VC0,VC1, andVC2) and display them across two screens. You can scale each Capture to 2/3 of a screen and split the center Capture across both screens. Or, if the screens have big bezels, scale each Stream to 1/2 the screen width and height and have a fourth "blank" panel. That fourth panel could be used for any presentation that pops up during the call 🌸 - Receive three Streams, decode all three, and use control information that says which one is most active. Then switch between showing the left and center Streams (one per screen) and the center and right Streams 🔄
If an Endpoint can do all three of these methods, it might be a good idea to let the user pick the display mode 🥰
12.2.3. Three-Screen Media Consumer Configuring the Example
This is the most straightforward case 🧚 The Media Consumer looks for the set of Streams that best matches its available screens, so VC0 plus VC1 plus VC2 should be a perfect match. The spatial ordering gives enough info for each Video Capture to land on the correct screen ✨ The Consumer then needs to divide a single Encoding Group's capability by 3. It can use that either to decide what resolution and frame rate to configure the Provider with, or to set up the individual Video Captures' Encoding Groups in whatever makes the most sense. That means taking into account the receive-side decode capabilities, the overall call bandwidth, the resolution of the screens, and any user preferences (like motion vs. sharpness) 🎬
12.3. Multipoint Conference Utilizing Multiple Content Captures
The MCCs let the MCU build outgoing Advertisements that describe complex Media switching and composition scenarios 🌟 The following sections give several examples.
Note: in the examples, the identities of the CLUE elements (e.g., Captures, Capture Scene) in the incoming Advertisements overlap. That's because there's no coordination between the Endpoints. The MCU is responsible for making these unique in the outgoing Advertisement. 🫶
12.3.1. Single Media Captures and MCC in the Same Advertisement ✨
okay so picture this: 🎀 four Endpoints are in one Conference using CLUE, and there's an MCU in the middle acting as a middlebox 🦋. Every Endpoint has its own CLUE channel going to the MCU. The MCU gets the following Advertisements from them, starting with Endpoint A's Australian conference room 🇦🇺.
+==================+================================+
| Capture Scene #1 | Description=AustralianConfRoom |
+==================+================================+
| VC1 | Description=Audience |
| | EncodeGroupID=1 |
+------------------+--------------------------------+
| CSV(VC1) | |
+------------------+--------------------------------+
Table 10: Advertisement Received from Endpoint A
+==================+===========================+
| Capture Scene #1 | Description=ChinaConfRoom |
+==================+===========================+
| VC1 | Description=Speaker |
| | EncodeGroupID=1 |
+------------------+---------------------------+
| VC2 | Description=Audience |
| | EncodeGroupID=1 |
+------------------+---------------------------+
| CSV(VC1, VC2) | |
+------------------+---------------------------+
Table 11: Advertisement Received from Endpoint B
+==================+=========================+
| Capture Scene #1 | Description=USAConfRoom |
+==================+=========================+
| VC1 | Description=Audience |
| | EncodeGroupID=1 |
+------------------+-------------------------+
| CSV(VC1) | |
+------------------+-------------------------+
Table 12: Advertisement Received from
Endpoint CNote: Endpoint B indicates that it sends two Streams. Yep, Endpoint B is the one bringing the speaker 🎤 along with its audience, so it's sending two Streams total.
okay now the MCU wants to put together a Multiple Content Capture 🌈 that gives a round-robin switched view of the audience from all three Endpoints and the speaker. The Advertisement it could build for that looks like this 👑:
+=======================+================================+
| Capture Scene #1 | Description=AustralianConfRoom |
+=======================+================================+
| VC1 | Description=Audience |
+-----------------------+--------------------------------+
| CSV(VC1) | |
+=======================+================================+
| Capture Scene #2 | Description=ChinaConfRoom |
+=======================+================================+
| VC2 | Description=Speaker |
+-----------------------+--------------------------------+
| VC3 | Description=Audience |
+-----------------------+--------------------------------+
| CSV(VC2, VC3) | |
+=======================+================================+
| Capture Scene #3 | Description=USAConfRoom |
+=======================+================================+
| VC4 | Description=Audience |
+-----------------------+--------------------------------+
| CSV(VC4) | |
+=======================+================================+
| Capture Scene #4 | |
+=======================+================================+
| MCC1(VC1,VC2,VC3,VC4) | Policy=RoundRobin:1 |
| | MaxCaptures=1 |
| | EncodingGroup=1 |
+-----------------------+--------------------------------+
| CSV(MCC1) | |
+-----------------------+--------------------------------+
Table 13: Advertisement Sent to Endpoint F - One Encodingand then there's the alternative 💅, where the MCU wants the speaker to be its own Media Stream and the audiences to be another one. To do that, it gives VC2 in Capture Scene 2 an Encoding Group, and it adds a CSV in Capture Scene #4 like this:
+===================+================================+
| Capture Scene #1 | Description=AustralianConfRoom |
+===================+================================+
| VC1 | Description=Audience |
+-------------------+--------------------------------+
| CSV(VC1) | |
+===================+================================+
| Capture Scene #2 | Description=ChinaConfRoom |
+===================+================================+
| VC2 | Description=Speaker |
| | EncodingGroup=1 |
+-------------------+--------------------------------+
| VC3 | Description=Audience |
+-------------------+--------------------------------+
| CSV(VC2, VC3) | |
+===================+================================+
| Capture Scene #3 | Description=USAConfRoom |
+===================+================================+
| VC4 | Description=Audience |
+-------------------+--------------------------------+
| CSV(VC4) | |
+===================+================================+
| Capture Scene #4 | |
+===================+================================+
| MCC1(VC1,VC3,VC4) | Policy=RoundRobin:1 |
| | MaxCaptures=1 |
| | EncodingGroup=1 |
| | AllowSubset=True |
+-------------------+--------------------------------+
| MCC2(VC2) | MaxCaptures=1 |
| | EncodingGroup=1 |
+-------------------+--------------------------------+
| CSV2(MCC1,MCC2) | |
+-------------------+--------------------------------+
Table 14: Advertisement Sent to Endpoint F - Two
Encodingsso yeah, the Consumer gets to decide 🫶 whether it wants a separate speaker-related Stream at all, and which Endpoints it wants to see. 🙈 Say it wants the second Stream but doesn't want the Australian conference room. It can then ask for these Captures in the Configure message:
+---------------+----------+
| MCC1(VC3,VC4) | Encoding |
+---------------+----------+
| VC2 | Encoding |
+---------------+----------+
Table 15: MCU Case:
Consumer Responsemain character energy, fr fr 🌈✨
12.3.2. Several MCCs in the Same Advertisement ✨
okay so you can use multiple MCCs when multiple Streams are carrying Media from multiple Endpoints 🎀 here's a lowkey iconic example:
Picture a Conference with three Endpoints, D, E, and F 🌸 Each one has three Video Captures covering the left, middle, and right of its conference room. The MCU gets these Advertisements from D and E:
+==================+================================+
| Capture Scene #1 | Description=AustralianConfRoom |
+==================+================================+
| VC1 | CaptureArea=Left |
+------------------+--------------------------------+
| | EncodingGroup=1 |
+------------------+--------------------------------+
| VC2 | CaptureArea=Center |
+------------------+--------------------------------+
| | EncodingGroup=1 |
+------------------+--------------------------------+
| VC3 | CaptureArea=Right |
+------------------+--------------------------------+
| | EncodingGroup=1 |
+------------------+--------------------------------+
| CSV(VC1,VC2,VC3) | |
+------------------+--------------------------------+
Table 16: Advertisement Received from Endpoint D
+==================+===========================+
| Capture Scene #1 | Description=ChinaConfRoom |
+==================+===========================+
| VC1 | CaptureArea=Left |
+------------------+---------------------------+
| | EncodingGroup=1 |
+------------------+---------------------------+
| VC2 | CaptureArea=Center |
+------------------+---------------------------+
| | EncodingGroup=1 |
+------------------+---------------------------+
| VC3 | CaptureArea=Right |
+------------------+---------------------------+
| | EncodingGroup=1 |
+------------------+---------------------------+
| CSV(VC1,VC2,VC3) | |
+------------------+---------------------------+
Table 17: Advertisement Received from Endpoint Eso now the MCU wants to offer Endpoint F three Capture Encodings 🧚 each one holds all the Captures from either Endpoint D or Endpoint E, depending on who's the active speaker. The MCU sends this Advertisement:
+=====================+================================+
| Capture Scene #1 | Description=AustralianConfRoom |
+=====================+================================+
| VC1 | |
+---------------------+--------------------------------+
| VC2 | |
+---------------------+--------------------------------+
| VC3 | |
+---------------------+--------------------------------+
| CSV(VC1,VC2,VC3) | |
+=====================+================================+
| Capture Scene #2 | Description=ChinaConfRoom |
+=====================+================================+
| VC4 | |
+---------------------+--------------------------------+
| VC5 | |
+---------------------+--------------------------------+
| VC6 | |
+---------------------+--------------------------------+
| CSV(VC4,VC5,VC6) | |
+=====================+================================+
| Capture Scene #3 | |
+=====================+================================+
| MCC1(VC1,VC4) | CaptureArea=Left |
| | MaxCaptures=1 |
| | SynchronizationID=1 |
| | EncodingGroup=1 |
+---------------------+--------------------------------+
| MCC2(VC2,VC5) | CaptureArea=Center |
| | MaxCaptures=1 |
| | SynchronizationID=1 |
| | EncodingGroup=1 |
+---------------------+--------------------------------+
| MCC3(VC3,VC6) | CaptureArea=Right |
| | MaxCaptures=1 |
| | SynchronizationID=1 |
| | EncodingGroup=1 |
+---------------------+--------------------------------+
| CSV(MCC1,MCC2,MCC3) | |
+---------------------+--------------------------------+
Table 18: Advertisement Sent to Endpoint F12.3.3. Heterogeneous Conference with Switching and Composition 💅
okay this one is a whole vibe 🌈 picture a Conference where the Endpoints look like this:
- Endpoint A: 4 screens, 3 cameras
- Endpoint B: 3 screens, 3 cameras
- Endpoint C: 3 screens, 3 cameras
- Endpoint D: 3 screens, 3 cameras
- Endpoint E: 1 screen, 1 camera
- Endpoint F: 2 screens, 1 camera
- Endpoint G: 1 screen, 1 camera
this example zooms in on what the person at one of the three-camera, multi-screen Endpoints sees 👑 let's call her User A, at Endpoint A. she has four big display screens. whenever somebody at another site is talking, all the Video Captures from that Endpoint show up on the large screens 😭 if the talker is at a three-camera site, their video from those three cameras fills three of the screens. if the talker is at a single-camera site, that camera's video fills one screen, and the other screens show video from other single-camera Endpoints.
User A also hears audio from the four loudest talkers 🎧
on top of the current speaker, User A can see video from other Endpoints too, just much smaller 🦋 Endpoint A has four screens, so one of them shows up to nine other Media Captures in a tiled layout. when a three-camera Endpoint shows up in the tiled area, all three of its cameras appear together across the screen, with the correct Spatial Relationship between those three images ✨
+---+---+---+ +-------------+ +-------------+ +-------------+
| | | | | | | | | |
+---+---+---+ | | | | | |
| | | | | | | | | |
+---+---+---+ | | | | | |
| | | | | | | | | |
+---+---+---+ +-------------+ +-------------+ +-------------+
Figure 8: Endpoint A - Four-Screen DisplayUser B at Endpoint B has a similar setup, except there are only three screens 🍓 so the nine other Media Captures get spread across the bottom of the three displays, in a PiP format. when a three-camera Endpoint shows up in the PiP area, all three cameras appear together across one screen, with the correct Spatial Relationship.
+-------------+ +-------------+ +-------------+
| | | | | |
| | | | | |
| | | | | |
| +-+ +-+ +-+ | | +-+ +-+ +-+ | | +-+ +-+ +-+ |
| +-+ +-+ +-+ | | +-+ +-+ +-+ | | +-+ +-+ +-+ |
+-------------+ +-------------+ +-------------+
Figure 9: Endpoint B - Three-Screen Display with PiPswhen somebody at a different Endpoint becomes the current speaker, User A and User B both see that new person's video appear in their large screen area 🔥 the previous speaker moves into one of the smaller tiled or PiP areas. the current speaker doesn't see themselves 🙈 they see the previous speaker in their large screen area instead.
one of the main points here is that Endpoints A and B each want to receive three Capture Encodings for their large display areas, and nine Encodings for their smaller areas 💜 A and B can both send the same Configure message to the MCU and receive the same conceptual Media Captures from it. the differences are in how things are Rendered, and that's purely a local matter at A and B 🫶
the Advertisements for this scenario are below 🌸
+=====================+========================+
| Capture Scene #1 | Description=Endpoint x |
+=====================+========================+
| VC1 | EncodingGroup=1 |
+---------------------+------------------------+
| VC2 | EncodingGroup=1 |
+---------------------+------------------------+
| VC3 | EncodingGroup=1 |
+---------------------+------------------------+
| AC1 | EncodingGroup=2 |
+---------------------+------------------------+
| CSV1(VC1, VC2, VC3) | |
+---------------------+------------------------+
| CSV2(AC1) | |
+---------------------+------------------------+
Table 19: Advertisement Received at the MCU from Endpoints A to D
+==================+========================+
| Capture Scene #1 | Description=Endpoint y |
+==================+========================+
| VC1 | EncodingGroup=1 |
+------------------+------------------------+
| AC1 | EncodingGroup=2 |
+------------------+------------------------+
| CSV1(VC1) | |
+------------------+------------------------+
| CSV2(AC1) | |
+------------------+------------------------+
Table 20: Advertisement Received at the MCU from Endpoints E to Grather than focusing on what gets displayed, CLUE focuses more on what the MCU sends 💖 the MCU doesn't know how many screens an Endpoint has, so that's not its problem 😌
since Endpoints A to D each advertise that three Captures make up a Capture Scene, the MCU offers these in a "site switching" mode ✨ that means there are three Multiple Content Captures (and Capture Encodings), each one switching between Endpoints. the MCU switches the applicable Media into the Stream based on voice activity 🎤 and Endpoint A will not see a Capture from itself 🙅♀️
using the MCC concept, the MCU would send Endpoint A the following Advertisement:
The source part of the Advertisement is the list of who's sending what to the MCC, and it's basically the same info you'd get from the Advertisements the MCU received. Two differences: there are no Capture Encodings attached, and the identities have been renumbered. 🦋
+=====================+========================+
| Capture Scene #1 | Description=Endpoint B |
+=====================+========================+
| VC4 | CaptureArea=Left |
+---------------------+------------------------+
| VC5 | CaptureArea=Center |
+---------------------+------------------------+
| VC6 | CaptureArea=Right |
+---------------------+------------------------+
| AC1 | |
+---------------------+------------------------+
| CSV(VC4,VC5,VC6) | |
+---------------------+------------------------+
| CSV(AC1) | |
+=====================+========================+
| Capture Scene #2 | Description=Endpoint C |
+=====================+========================+
| VC7 | CaptureArea=Left |
+---------------------+------------------------+
| VC8 | CaptureArea=Center |
+---------------------+------------------------+
| VC9 | CaptureArea=Right |
+---------------------+------------------------+
| AC2 | |
+---------------------+------------------------+
| CSV(VC7,VC8,VC9) | |
+---------------------+------------------------+
| CSV(AC2) | |
+=====================+========================+
| Capture Scene #3 | Description=Endpoint D |
+=====================+========================+
| VC10 | CaptureArea=Left |
+---------------------+------------------------+
| VC11 | CaptureArea=Center |
+---------------------+------------------------+
| VC12 | CaptureArea=Right |
+---------------------+------------------------+
| AC3 | |
+---------------------+------------------------+
| CSV(VC10,VC11,VC12) | |
+---------------------+------------------------+
| CSV(AC3) | |
+=====================+========================+
| Capture Scene #4 | Description=Endpoint E |
+=====================+========================+
| VC13 | |
+---------------------+------------------------+
| AC4 | |
+---------------------+------------------------+
| CSV(VC13) | |
+---------------------+------------------------+
| CSV(AC4) | |
+=====================+========================+
| Capture Scene #5 | Description=Endpoint F |
+=====================+========================+
| VC14 | |
+---------------------+------------------------+
| AC5 | |
+---------------------+------------------------+
| CSV(VC14) | |
+---------------------+------------------------+
| CSV(AC5) | |
+=====================+========================+
| Capture Scene #6 | Description=Endpoint G |
+=====================+========================+
| VC15 | |
+---------------------+------------------------+
| AC6 | |
+---------------------+------------------------+
| CSV(VC15) | |
+---------------------+------------------------+
| CSV(AC6) | |
+---------------------+------------------------+
Table 21: Advertisement Sent to Endpoint A -
Source PartNow the MCU also advertises site switching for Endpoints B to G, packaged up as three Streams. 🎀
+=====================+==============================+
| Capture Scene #7 | Description=Output3streammix |
+=====================+==============================+
| MCC1(VC4,VC7,VC10, | CaptureArea=Left |
| VC13) | MaxCaptures=1 |
| | SynchronizationID=1 |
| | Policy=SoundLevel:0 |
| | EncodingGroup=1 |
+---------------------+------------------------------+
| MCC2(VC5,VC8,VC11, | CaptureArea=Center |
| VC14) | MaxCaptures=1 |
| | SynchronizationID=1 |
| | Policy=SoundLevel:0 |
| | EncodingGroup=1 |
+---------------------+------------------------------+
| MCC3(VC6,VC9,VC12, | CaptureArea=Right |
| VC15) | MaxCaptures=1 |
| | SynchronizationID=1 |
| | Policy=SoundLevel:0 |
| | EncodingGroup=1 |
+---------------------+------------------------------+
| MCC4() (for audio) | CaptureArea=whole Scene |
| | MaxCaptures=1 |
| | Policy=SoundLevel:0 |
| | EncodingGroup=2 |
+---------------------+------------------------------+
| MCC5() (for audio) | CaptureArea=whole Scene |
| | MaxCaptures=1 |
| | Policy=SoundLevel:1 |
| | EncodingGroup=2 |
+---------------------+------------------------------+
| MCC6() (for audio) | CaptureArea=whole Scene |
| | MaxCaptures=1 |
| | Policy=SoundLevel:2 |
| | EncodingGroup=2 |
+---------------------+------------------------------+
| MCC7() (for audio) | CaptureArea=whole Scene |
| | MaxCaptures=1 |
| | Policy=SoundLevel:3 |
| | EncodingGroup=2 |
+---------------------+------------------------------+
| CSV(MCC1,MCC2,MCC3) | |
+---------------------+------------------------------+
| CSV(MCC4,MCC5,MCC6, | |
| MCC7) | |
+---------------------+------------------------------+
Table 22: Advertisement Sent to Endpoint A -
Switching PartsSo, the three big switched Streams are the ones doing the site switching. MaxCaptures=1 means only one Capture from the MCC gets sent at any given moment. SynchronizationID=1 says the source sending is synchronized. The Provider can choose to group VC13, VC14, and VC15 together for switching based on that SynchronizationID, which means that when it switches one of them into an MCC, it can switch the others too, even though they aren't in the same Capture Scene. Iconic teamwork 🧚
All the audio for the Conference lives in Scene #7. There isn't necessarily a one-to-one relationship between any Audio Capture and any Video Capture in this Scene. Typically, when the loudest talker changes, the MCU switches the audio Streams faster than it switches the video Streams. 🎧
The MCU can also supply nine Media Streams showing the active speaker and the previous eight speakers. Here's what it includes in the Advertisement:
+========================+===========================+
| Capture Scene #8 | Description=Output9stream |
+========================+===========================+
| MCC8(VC4,VC5,VC6,VC7, | MaxCaptures=1 |
| VC8,VC9,VC10,VC11, | Policy=SoundLevel:0 |
| VC12,VC13,VC14,VC15) | EncodingGroup=1 |
+------------------------+---------------------------+
| MCC9(VC4,VC5,VC6,VC7, | MaxCaptures=1 |
| VC8,VC9,VC10,VC11, | Policy=SoundLevel:1 |
| VC12,VC13,VC14,VC15) | EncodingGroup=1 |
+========================+===========================+
| to | to |
+========================+===========================+
| MCC16(VC4,VC5,VC6,VC7, | MaxCaptures=1 |
| VC8,VC9,VC10,VC11, | Policy=SoundLevel:8 |
| VC12,VC13,VC14,VC15) | EncodingGroup=1 |
+------------------------+---------------------------+
| CSV(MCC8,MCC9,MCC10, | |
| MCC11,MCC12,MCC13, | |
| MCC14,MCC15,MCC16) | |
+------------------------+---------------------------+
Table 23: Advertisement Sent to Endpoint A -
9 Switched PartsThat's nine Capture Encodings in total. Each one can hold any Captures from any source site, with a maximum of one Capture at a time, and the policy decides which Capture is the one showing. The MCCs in this Scene don't have any spatial attributes, so no left-center-right vibes here. 💜
Note: The Provider could alternatively provide each of the MCCs above in its own Capture Scene. 🌸
If the MCU also wants a composed Capture Encoding that shows all nine Captures at once, it can add this to the Advertisement:
+========================+=======================+
| Capture Scene #9 | Description=NineTiles |
+========================+=======================+
| MCC13(MCC8,MCC9,MCC10, | MaxCaptures=9 |
| MCC11,MCC12,MCC13, | |
| MCC14,MCC15,MCC16) | EncodingGroup=1 |
+------------------------+-----------------------+
| CSV(MCC13) | |
+------------------------+-----------------------+
Table 24: Advertisement Sent to Endpoint A -
9 Composed PartsNine tiles, one screen, full main character energy. ✨ Since MaxCaptures is 9, the Capture Encoding carries information from nine sources at a time. 👑
The Advertisement to Endpoint B is basically the same, bestie, except Captures from Endpoint A get added and Captures from Endpoint B get removed 🦋. Whether those Captures end up on a four-screen or a three-screen display is up to the Consumer to decide ✨. The Consumer wants to keep Video Captures from the same original source Endpoint together, in the correct spatial order 🗺️, but the MCCs don't have any spatial attributes 😭. So the Consumer needs to link incoming Media packets back to the original individual Captures in the Advertisement (like VC4, VC5, and VC6) so it knows the spatial info it needs for correct placement on the screens 💅. The Provider can use the RTCP CaptureId source description (SDES) item and its associated RTP header extension, as described in [RFC8849], to pass that info along to the Consumer 🫶.
12.3.4. Heterogeneous Conference with Voice-Activated Switching ✨
This example shows how multipoint "voice-activated switching" can be pulled off, with an Endpoint making its own call on which of its outgoing video Streams counts as the "active talker" from that Endpoint 🎤. Then the MCU can decide which one is the active talker across the whole Conference 👑.
Picture a Conference with these vibes:
- Endpoint A - 3 screens, 3 cameras
- Endpoint B - 3 screens, 3 cameras
- Endpoint C - 1 screen, 1 camera
This example zooms in on what the user at Endpoint C sees 👀. That user wants to watch the Video Capture of whoever is talking, without it being composed with any other Video Capture 🎀. Endpoint C can only receive a single video Stream 📺. The tables below show the Advertisements from Endpoints A and B to the MCU, and from the MCU to Endpoint C, that get this done 💖.
+===================+=========================+
| Capture Scene #1 | Description=Endpoint x |
+===================+=========================+
| VC1 | CaptureArea=Left |
| | EncodingGroup=1 |
+-------------------+-------------------------+
| VC2 | CaptureArea=Center |
| | EncodingGroup=1 |
+-------------------+-------------------------+
| VC3 | CaptureArea=Right |
| | EncodingGroup=1 |
+-------------------+-------------------------+
| MCC1(VC1,VC2,VC3) | MaxCaptures=1 |
| | CaptureArea=whole Scene |
| | Policy=SoundLevel:0 |
| | EncodingGroup=1 |
+-------------------+-------------------------+
| AC1 | CaptureArea=whole Scene |
| | EncodingGroup=2 |
+-------------------+-------------------------+
| CSV1(VC1, VC2, | |
| VC3) | |
+-------------------+-------------------------+
| CSV2(MCC1) | |
+-------------------+-------------------------+
| CSV3(AC1) | |
+-------------------+-------------------------+
Table 25: Advertisement Received at the MCU
from Endpoints A and BEndpoints A and B are advertising each individual Video Capture, plus a switched Capture MCC1 that flips between the other three based on who is the active talker 🔄. These Endpoints don't advertise distinct Audio Captures tied to each individual Video Capture, so the MCU (as a Media Consumer) couldn't figure out which Video Capture is the active talker just from the audio Streams 🙈.
+======================+========================+
| Capture Scene #1 | Description=conference |
+======================+========================+
| MCC1() | CaptureArea=Left |
| | MaxCaptures=1 |
| | SynchronizationID=1 |
| | Policy=SoundLevel:0 |
| | EncodingGroup=1 |
+----------------------+------------------------+
| MCC2() | CaptureArea=Center |
| | MaxCaptures=1 |
| | SynchronizationID=1 |
| | Policy=SoundLevel:0 |
| | EncodingGroup=1 |
+----------------------+------------------------+
| MCC3() | CaptureArea=Right |
| | MaxCaptures=1 |
| | SynchronizationID=1 |
| | Policy=SoundLevel:0 |
| | EncodingGroup=1 |
+----------------------+------------------------+
| MCC4() | CaptureArea=whole |
| | Scene |
| | MaxCaptures=1 |
| | Policy=SoundLevel:0 |
| | EncodingGroup=1 |
+----------------------+------------------------+
| MCC5() (for audio) | CaptureArea=whole |
| | Scene |
| | MaxCaptures=1 |
| | Policy=SoundLevel:0 |
| | EncodingGroup=2 |
+----------------------+------------------------+
| MCC6() (for audio) | CaptureArea=whole |
| | Scene |
| | MaxCaptures=1 |
| | Policy=SoundLevel:1 |
| | EncodingGroup=2 |
+----------------------+------------------------+
| CSV1(MCC1,MCC2,MCC3) | |
+----------------------+------------------------+
| CSV2(MCC4) | |
+----------------------+------------------------+
| CSV3(MCC5,MCC6) | |
+----------------------+------------------------+
Table 26: Advertisement Sent from the MCU to
Endpoint CThe MCU advertises one Scene with four video MCCs 🌈. Three of them in CSV1 give a left, center, and right view of the Conference, with site switching 🪞. MCC4 provides a single Video Capture representing a view of the whole Conference ✨. The MCU intends for MCC4 to be switched between all the other original source Captures 🔀. In this example, the MCU's Advertisement isn't giving all the info about all the other Endpoints' Scenes and which of those Captures are included in the MCCs 🤷♀️. The MCU could include all that if it wants to give Consumers more info, but it isn't necessary for this example scenario 💅.
The Provider advertises MCC5 and MCC6 for audio 🎧. Both are switched Captures, with different SoundLevel policies indicating they're the top two dominant talkers 👑. The Provider advertises CSV3 with both MCCs, suggesting the Consumer should use both if it can 💜.
Endpoint C, in its Configure Message to the MCU, requests to receive MCC4 for video and MCC5 and MCC6 for audio 📩. In order for the MCU to get the info it needs to construct MCC4, it has to send Configure Messages to Endpoints A and B asking to receive MCC1 from each of them, along with their AC1 audio 📨. Now the MCU can use audio energy info from the two incoming audio Streams from Endpoints A and B to figure out which of those alternatives is the current talker 🎙️. Based on that, the MCU uses either MCC1 from A or MCC1 from B as the source of MCC4 to send to Endpoint C 🧚.
13. IANA Considerations ✨
This document has no IANA actions. 🌸
14. Security Considerations 🔐
Okay bestie, security is where things get dramatic 😭🔥 Telepresence has a bunch of potential attacks, specifically around the protocols CLUE uses. Conferencing sessions, lots of Endpoints hanging out together, and a ton of capabilities that users often trigger themselves 💅 all add up.
An MCU in a CLUE session can run into many of the same attacks as a conferencing system like the one enabled by the Conference Information Data Model for Centralized Conferencing (XCON) framework [RFC5239]. Examples include an Endpoint trying to listen in on sessions it isn't authorized to join, an Endpoint trying to disconnect or mute other users, and theft of service, where an Endpoint tries to create telepresence sessions it isn't allowed to create 🙈. So it is RECOMMENDED that an MCU implementing the protocols needed to support CLUE follow the security recommendations in the conference control protocol documents. For CLUE, SIP is the conferencing protocol, so the security considerations in [RFC4579] MUST be followed. Other MCU security issues are discussed in the XCON framework [RFC5239]. Using xCard with potentially sensitive information is another reason to implement the recommendations in Section 11 of [RFC5239].
One primary security concern with the CLUE framework introduced in this document is securing the actual protocols and their associated authorization mechanisms. These concerns apply to Endpoint-to-Endpoint sessions as well as sessions involving multiple Endpoints and MCUs. Figure 2 in Section 5 gives a basic look at the flow of information exchange for CLUE and the protocols involved.
As Section 5 describes, CLUE uses SIP/SDP to establish the session before any CLUE-specific information gets exchanged. So the security mechanisms recommended for SIP [RFC3261], including user authentication and authorization, MUST be supported. On top of that, the Media MUST be secured. Datagram Transport Layer Security (DTLS) / Secure Real-time Transport Protocol (SRTP) MUST be supported and SHOULD be used unless the Media, which is based on RTP, is secured by other means (see [RFC7201] [RFC7202]). Media security is also discussed in [RFC8848] and [RFC8849]. Note that SIP call setup happens before any CLUE-specific information is available, so the authentication and authorization are based on SIP mechanisms. The entity that gets authenticated may use the Endpoint identity or the Endpoint user identity. That's an application issue, not a CLUE-specific one ✨
A separate data channel carries the CLUE protocol messages. The contents of those messages are based on information introduced in this document. The CLUE data model [RFC8846] defines the syntax to use, through an XML schema. One type of information that could introduce privacy concerns is the xCard information, described in Section 7.1.1.10. Deciding which xCard information to send in the CLUE channel is an application policy for point-to-point and multipoint calls, based on the authenticated identity, which can be the Endpoint identity or the user of the Endpoint. For example, a telepresence multipoint application can authenticate a user before starting a CLUE exchange with the telepresence system and have a policy per user 🎀
Also, the (text) description field in the Media Capture attribute (Section 7.1.1.6) could possibly reveal sensitive information or specific identities. The same is true for the descriptions in the Capture Scene (Section 7.3.1) and CSV (Section 7.3.2) attributes. An implementation SHOULD give users control over what sensitive information is sent in an Advertisement. One more important thing about the xCard information and the description fields in the Media Capture and CSV attributes: even though the Endpoints involved in the session have been authenticated, there is no assurance that the xCard or description information is authentic. So this information MUST NOT be used to make any authorization decisions 🚫
Other information in the CLUE protocol messages doesn't reveal specific identities, but it can reveal characteristics and capabilities of the Endpoints. That could possibly uniquely identify specific Endpoints. It might also be possible for an attacker to manipulate the information and disrupt CLUE sessions. A DoS attack on the CLUE Endpoints would also be possible if a malicious agent has access to the data channel. So it MUST be possible for the Endpoints to establish a channel that is secure against both message recovery and message modification. More details are in the CLUE data channel solution document [RFC8850] 🔥
There are also security issues tied to authorization for performing actions at the CLUE Endpoints to invoke specific capabilities (e.g., rearranging screens, sharing content, etc.). However, the policies and security for those actions are outside the scope of this document and the overall CLUE solution 🙏
15. References 📚
15.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC3261] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston,
A., Peterson, J., Sparks, R., Handley, M., and E.
Schooler, "SIP: Session Initiation Protocol", RFC 3261,
DOI 10.17487/RFC3261, June 2002,
<https://www.rfc-editor.org/info/rfc3261>.
[RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model
with Session Description Protocol (SDP)", RFC 3264,
DOI 10.17487/RFC3264, June 2002,
<https://www.rfc-editor.org/info/rfc3264>.
[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550,
July 2003, <https://www.rfc-editor.org/info/rfc3550>.
[RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
Description Protocol", RFC 4566, DOI 10.17487/RFC4566,
July 2006, <https://www.rfc-editor.org/info/rfc4566>.
[RFC4579] Johnston, A. and O. Levin, "Session Initiation Protocol
(SIP) Call Control - Conferencing for User Agents",
BCP 119, RFC 4579, DOI 10.17487/RFC4579, August 2006,
<https://www.rfc-editor.org/info/rfc4579>.
[RFC5239] Barnes, M., Boulton, C., and O. Levin, "A Framework for
Centralized Conferencing", RFC 5239, DOI 10.17487/RFC5239,
June 2008, <https://www.rfc-editor.org/info/rfc5239>.
[RFC5646] Phillips, A., Ed. and M. Davis, Ed., "Tags for Identifying
Languages", BCP 47, RFC 5646, DOI 10.17487/RFC5646,
September 2009, <https://www.rfc-editor.org/info/rfc5646>.
[RFC6350] Perreault, S., "vCard Format Specification", RFC 6350,
DOI 10.17487/RFC6350, August 2011,
<https://www.rfc-editor.org/info/rfc6350>.
[RFC6351] Perreault, S., "xCard: vCard XML Representation",
RFC 6351, DOI 10.17487/RFC6351, August 2011,
<https://www.rfc-editor.org/info/rfc6351>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC8846] Presta, R. and S P. Romano, "An XML Schema for the
Controlling Multiple Streams for Telepresence (CLUE) Data
Model", RFC 8846, DOI 10.17487/RFC8846, January 2021,
<http://www.rfc-editor.org/info/rfc8846>.
[RFC8847] Presta, R. and S P. Romano, "Protocol for Controlling
Multiple Streams for Telepresence (CLUE)", RFC 8847,
DOI 10.17487/RFC8847, January 2021,
<https://www.rfc-editor.org/info/rfc8847>.
[RFC8848] Hanton, R., Kyzivat, P., Xiao, L., and C. Groves, "Session
Signaling for Controlling Multiple Streams for
Telepresence (CLUE)", RFC 8848, DOI 10.17487/RFC8848,
January 2021, <https://www.rfc-editor.org/info/rfc8848>.
[RFC8850] Holmberg, C., "Controlling Multiple Streams for
Telepresence (CLUE) Protocol Data Channel", RFC 8850,
DOI 10.17487/RFC8850, January 2021,
<https://www.rfc-editor.org/info/rfc8850>.15.2. Informative References
[RFC4353] Rosenberg, J., "A Framework for Conferencing with the
Session Initiation Protocol (SIP)", RFC 4353,
DOI 10.17487/RFC4353, February 2006,
<https://www.rfc-editor.org/info/rfc4353>.
[RFC7201] Westerlund, M. and C. Perkins, "Options for Securing RTP
Sessions", RFC 7201, DOI 10.17487/RFC7201, April 2014,
<https://www.rfc-editor.org/info/rfc7201>.
[RFC7202] Perkins, C. and M. Westerlund, "Securing the RTP
Framework: Why RTP Does Not Mandate a Single Media
Security Solution", RFC 7202, DOI 10.17487/RFC7202, April
2014, <https://www.rfc-editor.org/info/rfc7202>.
[RFC7205] Romanow, A., Botzko, S., Duckworth, M., and R. Even, Ed.,
"Use Cases for Telepresence Multistreams", RFC 7205,
DOI 10.17487/RFC7205, April 2014,
<https://www.rfc-editor.org/info/rfc7205>.
[RFC7262] Romanow, A., Botzko, S., and M. Barnes, "Requirements for
Telepresence Multistreams", RFC 7262,
DOI 10.17487/RFC7262, June 2014,
<https://www.rfc-editor.org/info/rfc7262>.
[RFC7667] Westerlund, M. and S. Wenger, "RTP Topologies", RFC 7667,
DOI 10.17487/RFC7667, November 2015,
<https://www.rfc-editor.org/info/rfc7667>.
[RFC8849] Even, R. and J. Lennox, "Mapping RTP Streams to
Controlling Multiple Streams for Telepresence (CLUE) Media
Captures", RFC 8849, DOI 10.17487/RFC8849, January 2021,
<https://www.rfc-editor.org/info/rfc8849>.Acknowledgements 🫶
Allyn Romanow and Brian Baldino were authors of early draft versions. Mark Gorzynski also contributed a lot to the initial approach. Many others also contributed, including Christian Groves, Jonathan Lennox, Paul Kyzivat, Rob Hanton, Roni Even, Christer Holmberg, Stephen Botzko, Mary Barnes, John Leslie, and Paul Coverdale 💜
Authors' Addresses 💌
Mark Duckworth (editor)
Email: mrducky73@outlook.com
Andrew Pepperell
Acano
Uxbridge
United Kingdom
Email: apeppere@gmail.com
Stephan Wenger
Tencent
2747 Park Blvd.
Palo Alto, CA 94306
United States of America
Email: stewe@stewe.org📦 where this page comes from
- ietf: rfc8845 2021-01
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms