RFC 7262

Requirements for Telepresence Multistreams

📅 published June 2014, from ietf

RFC 7262: Requirements for Telepresence Multistreams ✨

Okay bestie, this is RFC 7262, and it's called Requirements for Telepresence Multistreams 🎀. It's an IETF document from June 2014, written by A. Romanow (Cisco Systems), S. Botzko (Polycom), and M. Barnes (MLB@Realtime Communications, LLC). Its category is Informational (so it's not Standards Track 💅), and its ISSN is 2070-1721. The title block doesn't list anything it obsoletes or updates, so it's a standalone doc 🦋.

Abstract 🌸

This memo is all about the requirements for specs that make telepresence interoperability happen 💖. It does this by describing the behaviors and protocols for Controlling Multiple Streams for Telepresence (CLUE). It also covers the problem statement and related definitions, so it's a whole package 📦.

Status of This Memo 📝

This document is not an Internet Standards Track specification. It's published for informational purposes 🌈.

It's a product of the Internet Engineering Task Force (IETF), it represents the consensus of the IETF community, and it has had public review. The Internet Engineering Steering Group (IESG) approved it for publication. Not every IESG-approved doc is a candidate for any level of Internet Standard, so check Section 2 of RFC 5741 if you want the full story 🧐.

For the current status of this document, any errata, and how to give feedback, go to http://www.rfc-editor.org/info/rfc7262 ✨.

Copyright (c) 2014 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as in effect on the date of publication. Please read these carefully, because they describe your rights and restrictions for this document 🙈. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they're provided without warranty as described in the Simplified BSD License.

Table of Contents 🧚

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Terminology . . . . . . . . . . . . . . . . . . . . . . . . .   3
   3.  Definitions . . . . . . . . . . . . . . . . . . . . . . . . .   4
   4.  Problem Statement . . . . . . . . . . . . . . . . . . . . . .   5
   5.  Requirements  . . . . . . . . . . . . . . . . . . . . . . . .   6
   6.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  10
   7.  Security Considerations . . . . . . . . . . . . . . . . . . .  10
   8.  Informative References  . . . . . . . . . . . . . . . . . . .  11

1. Introduction ✨

Telepresence systems are seriously good for collaboration 🫶. In a telepresence conference (as this doc uses the term), the goal is an environment that gives users a feeling of (co-located) presence. That's the feeling that a local user is in the same room as other local users and remote parties 🌸. Right now, systems from different vendors often don't interoperate, because they do the same tasks differently. That's covered in the Problem Statement section below (see Section 4) 😭.

The approach in this memo is to set requirements for future specification(s). When an implementation of those specs meets the requirements, it provides interoperability between IETF protocol-based telepresence systems 🦋. It's anticipated that a solution for these requirements will likely involve exchanging adequate information about participating sites. That information isn't currently standardized by the IETF, so it's a whole new vibe 🍓.

The purpose of this doc is to describe the requirements for a spec that enables interworking between different SIP-based [RFC3261] telepresence systems, by exchanging and negotiating appropriate information. In the context of these requirements and the related solution docs, this covers both point-to-point SIP sessions and SIP-based conferences. Those conferences are described in the SIP conferencing framework [RFC4353] and the SIP-based conference control [RFC4579] specifications 💜. Non-IETF protocol-based systems, such as those based on ITU-T Rec. H.323 [ITU.H323], are out of scope 🙅. These requirements are for the specification. They are not requirements on the telepresence systems that implement the solution/protocol that will be specified.

Today, telepresence systems from different vendors can follow radically different architectural approaches while still offering a similar user experience 💅. CLUE won't dictate telepresence architectural and implementation choices. It will, however, describe a protocol architecture for CLUE and how it relates to other protocols. CLUE enables interoperability between telepresence systems by exchanging information about the systems' characteristics. Systems can then use that info to control their behavior so they can interoperate with each other 👑.

A telepresence session needs at least one sending and one receiving endpoint. Multiparty telepresence sessions include more than 2 endpoints, plus centralized infrastructure such as Multipoint Control Units (MCUs) or equivalent 🎀. CLUE specifies the syntax, semantics, and control flow of information, all to enable the best possible user experience at those endpoints ✨.

Sending endpoints, or MCUs, are not mandated to use any of the CLUE specifications that describe their capabilities, attributes, or behavior 🧐. Similarly, it's not envisioned that endpoints or MCUs will ever have to take received information into account. Still, by making as much information available as possible, and by taking into account as much information as has been received or exchanged, MCUs and endpoints are expected to select operation modes that enable the best possible user experience under their constraints 🌈.

The doc structure is as follows. First come the definitions, then a description of the telepresence interoperability problem that led to this work 🧚. After that, the requirements for a spec that fixes the current shortcomings are listed and discussed 💖.

2. Terminology 💅

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119] 🔥.

3. Definitions 🎀

These terms show up all through this doc, and they're a reference for other docs too 📚.

  • Audio Mixing: refers to accumulating scaled audio signals to produce a single audio stream. See "RTP Topologies" [RFC5117] 🎧.
  • Conference: used as defined in "A Framework for Conferencing within the Session Initiation Protocol (SIP)" [RFC4353] 🫶.
  • Endpoint: the logical point of final termination through receiving, decoding and rendering, and/or initiation through capturing, encoding, and sending of media streams. An endpoint is made up of one or more physical devices that source and sink media streams, plus exactly one participant [RFC4353] (which, in turn, includes exactly one SIP user agent). Unlike an endpoint, an MCU can also send and receive media streams, but it's not the initiator or the final terminator in the sense that media is captured or rendered. Endpoints can be anything from multiscreen/multicamera rooms to handheld devices 📱.
  • Endpoint Characteristics: include placement of capture and rendering devices, capture/render angle, resolution of cameras and screens, spatial location, and mixing parameters of microphones. Endpoint characteristics aren't specific to individual media streams sent by the endpoint 🌸.
  • Layout: how rendered media streams are spatially arranged with respect to each other on a telepresence endpoint with a single screen and a single loudspeaker, and how they're arranged on an endpoint with multiple screens or loudspeakers. Note that audio as well as video are part of layout. That means placing audio streams on loudspeakers counts, and so does placing video streams on video screens 🖥️.
  • Local: sender and/or receiver physically co-located ("local") in the context of the discussion 🏠.
  • MCU: Multipoint Control Unit (MCU), a device that connects two or more endpoints together into one single multimedia conference [RFC5117]. An MCU may include a mixer [RFC4353] 🧚.
  • Media: any data that, after suitable encoding, can be conveyed over RTP, including audio, video, or timed text 🎶.
  • Model: a set of assumptions a telepresence system of a given vendor adheres to, and expects the remote telepresence system(s) to also adhere to 🤝.
  • Remote: sender and/or receiver on the other side of the communication channel (depending on context), i.e., not local. A remote can be an endpoint or an MCU 🌍.
  • Render: the process of generating a representation from a media, such as displayed motion video or sound emitted from loudspeakers 📺.
  • Telepresence: an environment that gives non-co-located users or user groups a feeling of (co-located) presence. That's the feeling that a local user is in the same room with other local users and the remote parties. Remote parties are included through multimedia communication with at least audio and video signals of high fidelity 💫.

4. Problem Statement ✨

Okay so, the whole goal of telepresence is that "being there" feeling 🌸. To get it, media inputs have to be sent, received, and coordinated between every system in the call. The catch is that different telepresence systems come up with totally different ways to do this, or they do similar things in wildly different ways 🙃.

They use different techniques, and they describe, control, and negotiate media in their own unique ways 💅. That mismatch is an interoperability problem. Different systems solve the same issues differently, so they aren't directly interoperable. Interworking ends up being hard at best and sometimes straight-up impossible 😭.

It gets worse. Even when extensions are built on common standards like SIP, lots of telepresence systems add proprietary protocol extensions to solve telepresence-related problems 🙈.

You can get some interworking between vendors with transcoding and translation. But those extra devices cost a lot, usually aren't fully automatic, and can cause annoying side effects like extra delay or worse performance 💀. Right now, you also need specialized knowledge to run a telepresence conference with endpoints from different vendors. For example, someone has to configure the transcoding and translating devices. So these conferences often don't start as planned, or they get interrupted by problems that pop up mid-call 😩.

Here's the core problem. Right now, every endpoint renders the audio and video captures it gets based on an implicit model that says how to make a realistic picture of the remote location 🎀. If all the endpoints come from the same vendor, they share the same implicit model and render the received captures correctly ✨. But if the devices come from different vendors, the rendering models can differ, and they usually do 👀. The systems might still connect, but the experience suffers. For example, one system assumes the first video stream comes from the right camera, while the other assumes it comes from the left camera 💀.

Let's say Alice and Bob are at different sites. Alice has to tell Bob about the camera and sound setup at her site, so Bob's receiver can build an accurate picture of her space 🦋. They both need to agree on which characteristics matter and how to represent and communicate them. Those characteristics can include the number, placement, and capture/render angle of cameras, the resolution of cameras and screens, spatial location, and the audio mixing parameters of microphones 🎧.

The telepresence multistream work aims to describe the sender's setup in a way that lets the receiver render it realistically, even if the receiver uses a different rendering model than the sender 🧚.

5. Requirements 📋

Okay, some of these requirements could be handled by existing technology, like the Session Description Protocol (SDP) [RFC4566]. They're still written out here so there's a complete record of what CLUE needs 💖. Whether a requirement needs new work is something the solution development will figure out, and this document doesn't go into it 🤓. Heads up, "solution" in these requirements means the protocol specifications. That includes extensions to existing protocols and any brand-new protocols developed to support the use cases 🔥. A solution might add functionality that doesn't map directly to these requirements, like the detailed information carried in the signaling protocol(s) 💜. When a requirement is directly relevant to a specific use case from [RFC7205], the requirement points to that use case 🌈.

  • REQ-1: The solution MUST support a description of the spatial arrangement of source video images sent in video streams, so the receiver can reproduce the original scene satisfactorily 🖼️. This applies to each site in a point-to-point or a multipoint meeting, and it's about the spatial ordering within a site, not the ordering of images between sites 🙌.
    • This requirement relates to all the use cases described in [RFC7205].
    • REQ-1a: The solution MUST support a means of preserving the order of images in the captured scene. For example, if John is to Susan's right in the image capture, John is also to Susan's right in the rendered image 👯.
    • REQ-1b: The solution MUST support a means of preserving the order of images in the scene in two dimensions, horizontal and vertical 📐.
    • REQ-1c: The solution MUST support a means to identify the relative location, within a scene, of the point of capture of individual video captures in three dimensions 📍.
    • REQ-1d: The solution MUST support a means to identify the area of coverage, within a scene, of individual video captures in three dimensions 🗺️.
  • REQ-2: The solution MUST support a description of the spatial arrangement of captured source audio sent in audio streams, so the receiver can reproduce it in a spatially correct way 🎶. This applies to each site in a point-to-point or a multipoint meeting, and it's about the spatial ordering within a site, not the ordering of channels between sites 🔊.
    • This requirement relates to all the use cases described in [RFC7205], but it's especially important for the Heterogeneous Systems use case 💅.
    • REQ-2a: The solution MUST support a means of preserving the spatial order of audio in the captured scene. For example, if John sounds like he's on Susan's right in the captured audio, John's voice is also placed on Susan's right in the rendered output 🎧.
    • REQ-2b: The solution MUST support a means to identify the number and spatial arrangement of audio channels, including monaural, stereophonic (2.0), and 3.0 (left, center, right) audio channels 🎚️.
    • REQ-2c: The solution MUST support a means to identify the point of capture of individual audio captures in three dimensions 📡.
    • REQ-2d: The solution MUST support a means to identify the area of coverage of individual audio captures in three dimensions 🌐.
  • REQ-3: The solution MUST enable individual audio streams to be associated with one or more video image captures, and individual video image captures to be associated with one or more audio captures, so the right position can be rendered 🫶.
    • This requirement relates to all the use cases described in [RFC7205].
  • REQ-4: The solution MUST enable interoperability between endpoints that have a different number of similar devices. For example, one endpoint might have 1 screen, 1 loudspeaker, 1 camera, and 1 mic, while another has 3 screens, 2 loudspeakers, 3 cameras, and 2 microphones. Or, in a multipoint conference, one endpoint might have 1 screen, another 2 screens, and a third 3 screens 🖥️. This also covers endpoints where the number of devices of a given type is zero 🙃.
    • This requirement relates to the Point-to-Point Meeting: Symmetric and Multipoint Meeting use cases described in [RFC7205].
  • REQ-5: The solution MUST support means of enabling interoperability between telepresence endpoints where cameras have different picture aspect ratios 📸.
  • REQ-6: The solution MUST provide scaling information that enables rendering of a video image at the actual size of the captured scene 📏.
  • REQ-7: The solution MUST support means of enabling interoperability between telepresence endpoints where displays have different resolutions 📺.
  • REQ-8: The solution MUST support methods for handling different bit rates in the same conference ⚡.
  • REQ-9: The solution MUST support means of enabling interoperability between endpoints that send and receive different numbers of media streams 🔀.
    • This requirement relates to the Heterogeneous Systems and Multipoint Meeting use cases.
  • REQ-10: The solution MUST ensure that endpoints that support telepresence extensions can establish a session with a SIP endpoint that doesn't support them. For example, with a SIP endpoint that supports a single audio and a single video stream, an endpoint that supports the telepresence extensions would set up a session with a single audio and single video stream using existing SIP and SDP mechanisms 🤝.
  • REQ-11: The solution MUST support a mechanism for determining whether an endpoint or MCU is capable of telepresence extensions 🕵️.
  • REQ-12: The solution MUST support a means to enable more than two endpoints to take part in a teleconference 👥.
    • This requirement relates to the Multipoint Meeting use case.
  • REQ-13: The solution MUST support both transcoding and switching approaches for providing multipoint conferences 🔁.
  • REQ-14: The solution MUST support mechanisms that allow media from one source endpoint, or from multiple source endpoints, to be sent to a remote endpoint at a particular point in time. Which media is sent at a given point in time may be based on local policy 🎬.
  • REQ-15: The solution MUST provide mechanisms to support the following:
    • Presentations with different media sources
    • Presentations for which the media streams are visible to all endpoints
    • Multiple, simultaneous presentation media streams, including presentation media streams that are spatially related to each other
    • This requirement relates to the Presentation use case. 🎤
  • REQ-16: The specification of any new protocols for the solution MUST provide extensibility mechanisms 🧩.
  • REQ-17: The solution MUST support a mechanism for allowing information about media captures to change during a conference 🔄.
  • REQ-18: The solution MUST provide a mechanism for the secure exchange of information about the media captures 🔐.

6. Acknowledgements 🫶

This document has been shaped by all the comments on the CLUE mailing list and a bunch of discussions 💬. So many people contributed that listing them all isn't possible 🥹. That said, the comments from Roberta Presta, Christian Groves, and Paul Coverdale during WGLC were especially helpful in finishing the WG document 👑.

7. Security Considerations ✨

okay bestie, security is where things get SERIOUS 🫶 REQ-18 is all about making sure the info about media captures gets transported securely 🔐 and it's giving "do not skip this part" energy 💀

so here's the tea: a telepresence session starts with SIP for the basic session setup 📞 and then for a multiparty telepresence session you can use either SIP or the Centralized Conferencing Manipulation Protocol (CCMP) [RFC6503] 🎀 no cap, both are valid choices 💅

for the SIP signaling, the security is handled by the SIP security mechanisms defined in [RFC3261] 🔒 that's the built-in protection for that part 👑

if you're doing conference control with CCMP, the security model and mechanisms in the Centralized Conferencing (XCON) Framework [RFC5239] and CCMP [RFC6503] documents meet the requirement 🌸 so you're covered there too ✨

now the important bit 🚨 if you use ANY additional signaling mechanism to carry the information about media captures, that mechanism needs to define how the information is secured 😭 the details for those mechanisms need to be defined and described in the CLUE framework document and related solution document(s) 🦋 bestie, this is not optional, it's a whole vibe check 💀

8. Informative References 🧚

okay here's the reading list, no skipping 📚

8.1. Informative References

  • [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. 💜
  • [RFC3261] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, "SIP: Session Initiation Protocol", RFC 3261, June 2002. 📞
  • [RFC4353] Rosenberg, J., "A Framework for Conferencing with the Session Initiation Protocol (SIP)", RFC 4353, February 2006. 🎀
  • [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session Description Protocol", RFC 4566, July 2006. 🌈
  • [RFC4579] Johnston, A. and O. Levin, "Session Initiation Protocol (SIP) Call Control - Conferencing for User Agents", BCP 119, RFC 4579, August 2006. 💅
  • [RFC5117] Westerlund, M. and S. Wenger, "RTP Topologies", RFC 5117, January 2008. 🦋
  • [RFC5239] Barnes, M., Boulton, C., and O. Levin, "A Framework for Centralized Conferencing", RFC 5239, June 2008. 🌸
  • [RFC6503] Barnes, M., Boulton, C., Romano, S., and H. Schulzrinne, "Centralized Conferencing Manipulation Protocol", RFC 6503, March 2012. ✨
  • [RFC7205] Romanow, A., Botzko, S., Duckworth, M., and R. Even, "Use Cases for Telepresence Multistreams", RFC 7205, April 2014. 🍓
  • [ITU.H323] ITU-T, "Packet-based Multimedia Communications Systems", ITU-T Recommendation H.323, December 2009. 👑

Authors' Addresses

Allyn Romanow Cisco Systems San Jose, CA 95134 USA

EMail: allyn@cisco.com

Stephen Botzko Polycom Andover, MA 01810 USA

EMail: stephen.botzko@polycom.com

Mary Barnes MLB@Realtime Communications, LLC

EMail: mary.ietf.barnes@gmail.com

that's the whole RFC, bestie 🌈💖 you made it through!

📦 where this page comes from