RFC 7205

Use Cases for Telepresence Multistreams

πŸ“… published April 2014, from ietf

RFC 7205: Use Cases for Telepresence Multistreams πŸŽ€

RFC 7205 πŸ’– was published in April 2014 by the IETF (Internet Engineering Task Force). It's titled Use Cases for Telepresence Multistreams, and its ISSN is 2070-1721. The authors are A. Romanow (Cisco), S. Botzko, M. Duckworth (Polycom), and R. Even, Ed. (Huawei Technologies). The "Ed." means R. Even is the editor. It's Category: Informational, so it isn't a standards-track spec, just a "here's what's going on" doc 🧚. The title block doesn't list anything it obsoletes or updates.

Abstract

okay so telepresence systems are trying to make people who aren't in the same room feel like they're literally sitting together 🫢. They do this with multimedia that has at least high-fidelity audio and video. The problem is that there are a bunch of different techniques for handling those audio and video streams, and when they're not similar, getting different systems to work together is hard, and honestly often just not possible 😭. If you could share info about how the different media streams relate to each other, senders and receivers could make smarter choices so telepresence systems can actually talk to each other. This memo walks through the most typical and important use cases for sending multiple streams in a telepresence conference ✨.

Status of This Memo

This doc is not an Internet Standards Track spec. It's just published for info purposes πŸ“.

It's still a product of the IETF, so it represents the consensus of the IETF community. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). Not every doc the IESG approves is a candidate for any level of Internet Standard, so check out Section 2 of RFC 5741 for that πŸ’….

For the current status, any errata, and how to give feedback, go to http://www.rfc-editor.org/info/rfc7205 πŸ”—.

Copyright (c) 2014 IETF Trust and the persons identified as the document authors. All rights reserved 🌈.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they're in effect on the date this document was published. Please actually read these, because they spell out your rights and restrictions for this document. Any Code Components pulled out of this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come with no warranty, as described in the Simplified BSD License πŸ”₯.

Table of Contents

1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
2.  Overview of Telepresence Scenarios  . . . . . . . . . . . . .   4
3.  Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . .   6
  3.1.  Point-to-Point Meeting: Symmetric . . . . . . . . . . . .   7
  3.2.  Point-to-Point Meeting: Asymmetric  . . . . . . . . . . .   7
  3.3.  Multipoint Meeting  . . . . . . . . . . . . . . . . . . .   9
  3.4.  Presentation  . . . . . . . . . . . . . . . . . . . . . .  10
  3.5.  Heterogeneous Systems . . . . . . . . . . . . . . . . . .  11
  3.6.  Multipoint Education Usage  . . . . . . . . . . . . . . .  12
  3.7.  Multipoint Multiview (Virtual Space)  . . . . . . . . . .  14
  3.8.  Multiple Presentation Streams - Telemedicine  . . . . . .  15
4.  Acknowledgements  . . . . . . . . . . . . . . . . . . . . . .  16
5.  Security Considerations . . . . . . . . . . . . . . . . . . .  16
6.  Informative References  . . . . . . . . . . . . . . . . . . .  16

1. Introduction ✨

Telepresence apps are all about giving you that "being there" feeling during conversational video calls πŸ¦‹. People often call this "immersive telepresence" to set it apart from regular video conferencing and other remote presence stuff that isn't about conversational video, like avatars and robots πŸ€–. The main vibes of telepresence are that it's actual sized, it gives you immersive video, it keeps the interpersonal interaction, and it lets you do non-verbal communication πŸ’œ.

Telepresence systems are built on open standards like RTP [RFC3550], SIP [RFC3261], H.264 [ITU.H264], and the H.323 [ITU.H323] suite of protocols. But they still can't easily work together without an operator stepping in, or without super expensive extra gear that translates from one vendor's protocol to another's 😩.

The basic features that make telepresence so special are implemented in totally different ways across systems. Right now, telepresence systems from different vendors kind of interoperate, but it's not supported in a standards-based way πŸ™ˆ. Making them work together means you need translation and transcoding devices in the architecture. Those devices add latency, which hurts the quality of interpersonal interaction, and that's not a vibe πŸ’€. Using them usually isn't automatic either. It often needs a lot of manual configuration and a detailed understanding of the underlying audio and video streams. Honestly, that's not acceptable if telepresence is going to keep growing. These systems should be as easy to interoperate as telephones are πŸ“ž. So a standard way of describing the multiple streams that make up the media flows, and the fundamental aspects of how they behave, would let telepresence systems work together 🫢.

This doc presents a set of use cases that describe typical scenarios. Requirements will be pulled from these use cases in a separate document πŸ“. The use cases are told from the users' point of view, and they show the user experience that needs to be supported. There are lots of different ways to implement these use cases 🌸.

Tons of different scenarios need support. This document goes into detail on the most common and basic use cases, which should cover most of the requirements. There may be additional scenarios that bring new features and requirements, which could be used to extend the initial work 🌈.

Both point-to-point and multipoint telepresence conferences are considered. In some use cases the number of screens is the same at all sites, and in others it differs from site to site. Both get considered. There's also a use case about displaying presentation material or content πŸŽ€.

The multipoint use cases can include a bunch of different systems, from conference room setups all the way down to handheld devices, and one use case covers exactly that πŸ“±.

This doc is laid out like this: Section 2 gives an overview of scenarios, and Section 3 describes the use cases πŸ—ΊοΈ.

2. Overview of Telepresence Scenarios

This section covers the general characteristics of the use cases and what the scenarios are meant to show πŸ’Ό. The typical setting is a business conference, since that's where telepresence started. Consumer products are being developed too, lately πŸ›οΈ. What we're not covering in these scenarios is the physical infrastructure side of telepresence, like room construction, layout, and decoration. These use cases also don't describe every aspect needed for the best user experience. For example, human factors aren't covered πŸ™ˆ.

We also don't try to precisely define where telepresence systems end and other systems begin, and we don't try to pick the "best" solution for each scenario πŸ’….

Telepresence systems usually have one or more video cameras and encoders, plus one or more big display screens (diagonal around 60 inches) πŸ“Ί. Microphones pick up sound, and the audio codec(s) produce one or more audio streams 🎧. The cameras that capture the telepresence users are called "participant cameras" (and the same goes for screens) πŸ‘©β€πŸ’». There may also be other cameras, like ones for document display. These are called "presentation cameras" or "content cameras", and they usually have different formats, aspect ratios, and frame rates than the participant cameras 🎞️. Presentation streams might show up on participant screens or on auxiliary display screens. A user's computer can also act as a virtual content camera, making an animation or playing a video for the remote participants 🎬.

So we describe a telepresence system as sending one or more video streams, audio streams, and presentation streams to the remote system(s) πŸ’«.

The fundamental parameters that describe today's typical telepresence scenarios include:

  1. The number of participating sites
  2. The number of visible seats at a site
  3. The number of cameras
  4. The number and type of microphones
  5. The number of audio channels
  6. The screen size
  7. The screen capabilities -- such as resolution, frame rate, aspect ratio
  8. The arrangement of the screens in relation to each other
  9. The number of primary screens at each site
  10. Type and number of presentation screens
  11. Multipoint conference display strategies -- for example, the camera-to-screen mappings may be static or dynamic
  12. The camera point of capture
  13. The cameras fields of view and how they spatially relate to each other

Like we said in the intro, the basic features that give telepresence its distinctive characteristics are implemented in totally different ways across systems πŸŒ€.

There's no agreed-upon way to adequately describe the semantics of how streams of various media types relate to each other πŸ˜΅β€πŸ’«. Without a standard for stream semantics that describes the particular roles and activities of each stream in the conference, interoperability is cumbersome at best 🫠.

In a multiple-screen conference, the video and audio streams sent from remote participants must be understood by receivers so they can be presented in a coherent and life-like way 🌟. That includes being able to show remote participants at their actual size for their apparent distance, while keeping correct eye contact and gesticular cues. It also means providing a spatial audio sound stage that matches the displayed video πŸ”Š.

The receiving device, which decides how to render the incoming info, needs to understand a bunch of variables. These include the spatial position of the speaker, the field of view of the cameras, the camera zoom, and which media stream is related to each screen, and so on 🧠. It's not just that individual streams have to be adequately described. To a large extent, that already exists. The real problem is that the semantics of the relationships between the streams have to be communicated πŸ’…. Note that all of this is still required even when the basic aspects of the streams, like bit rate, frame rate, and aspect ratio, are known 🎯. So this problem has way more going on than what you see when interoperating video conferencing systems that only have a single camera and screen πŸ‘‘.

3. Use Cases ✨

okay so the use cases are where we get into the typical setups πŸŽ€ there are a bunch of possible variants, like the audio at each end might be mono or stereo or full surround sound, etc. πŸ”Š

lowkey the classic one is the "full conference room" setup πŸͺ‘ local people sit on one side of the table and the remote people look like they're sitting on the other side. the cameras and screens are usually set up for a panoramic view of the remote room (left to right from the local user's point of view) πŸŒ…

the immersion and non-verbal vibes come from a few key features:

  1. good eye contact, which happens when participants, cameras, and screens are placed carefully πŸ‘€
  2. the camera field of view and screen sizes are matched so the remote room looks full size πŸ“
  3. the left side of each room shows up on the right screen at the far end, and the right side shows up on the left screen. so everyone looks like they're sitting across the table from each other πŸ₯Ή if 2 people on the same site glance at each other, everyone can see it. and if someone at one site gestures at someone on the other site, everyone sees the gesture AND who it's aimed at πŸ’…

3.1. Point-to-Point Meeting: Symmetric

in this one, both sites have the same number of screens, the cameras have fixed fields of view, and there's 1 camera for each screen πŸ“Έ the sound type is the same at each end too. for example, each room could have 3 cameras and 3 screens with stereo sound going both ways 🎧

each screen is paired with its own camera. each camera/screen pair usually goes to a separate codec, which makes an encoded video stream to send to the remote site and gets a similarly encoded stream back 🎬

each system has one or more microphones for capturing audio. sometimes they're stereophonic mics 🎀 in other systems there's a mic in front of each person (or each pair of people). in typical systems, all the mics connect to one codec that sends and receives the audio as stereo or surround sound. the number of mics and audio channels is often different from the number of cameras, and the number of mics is often different from the number of loudspeakers too πŸ™ƒ

the audio can go out as multi-channel (stereo or surround) or as separate mono streams. audio levels should be matched so both sites sound equally loud πŸ”‰ the loudspeaker and mic placement is planned so the sound "stage" (where the sounds seem to come from) lines up with the video 🌈 so if someone at one site talks, the people at the remote site hear her voice coming from her picture 🫢 to make that work, the audio has to get mapped at the receiving end the same way as the video. audio from the right side of the room needs to come out of the loudspeaker(s) on the left side at the remote site, and the other way around πŸ’–

3.2. Point-to-Point Meeting: Asymmetric

this is the scenario where each site has a different number of screens and cameras than the other site 😬 the big thing here is that the number of screens is different between the 2 sites. that creates challenges, and different telepresence systems handle them in different ways πŸ™ˆ

this use case builds on the basic 3-screen-to-3-screen setup. here we're using the common case of 3 screens and 3 cameras at one site, and 1 screen and 1 camera at the other site, connected by a point-to-point call πŸ“ž the screen sizes and camera fields of view are basically similar at both sites, so each camera view is designed to show 2 people sitting side by side. that means the 1-screen room fits up to 2 people at the table, while the 3-screen room can fit up to 6 πŸͺ‘βœ¨

the basic rules for defining left and right and showing where the multiple audio and video streams sit relative to each other are the same as in the 3-3 case. the difference is that handling the mismatch in the number of screens and cameras takes more complicated maneuvers 🧩

the 1-camera room to the 3-screen room: this one's easy, honestly πŸ₯° the usual move is to just use 1 of the 3 screens and keep the second and third screens inactive, or maybe put up the current date πŸ“… that keeps the "full-size" image of the remote side.

the 3-camera room to the 1-screen room: this direction is where it gets spicy 🌢️ there are several ways the video streams can be handled:

  1. the 1-screen system could just show 1 of the 3 camera images, since the receiving side only has 1 screen. 2 people look full size, but 4 people aren't seen at all 😭 which stream gets shown could be fixed, or the users could pick it. it could also be automatic based on who's talking in the 3-screen room, so the people in the 1-screen room always see whoever is speaking πŸŽ™οΈ if the automatic selection happens at the sender, the streams that aren't being displayed can be stopped from being sent, which avoids wasting bandwidth πŸ’Έ
  2. the 1-screen system could be able to receive and decode all 3 streams from all 3 cameras. then it can compose the 3 streams into 1 local image for the one screen πŸ–ΌοΈ all 6 people would be visible, just smaller than full size. this could go along with lowering the resolution of the streams, so encode/decode resources and bandwidth aren't wasted on streams that get shrunk down for display anyway 🧠
  3. the 3-screen system could be able to put all 6 people into a single stream to send to the 1-screen system. for example, it could use PTZ (Pan Tilt Zoom) cameras to physically move the cameras so 1 camera captures the whole room of 6 people πŸŽ₯ or it could recompose the 3 camera images into 1 encoded stream to send over. these options also show all 6 people, just at a reduced size 🌸
  4. or you could mix these approaches, like showing the speaker at full size at the same time as a composite of all 6 participants in a smaller size πŸ‘‘

the receiving telepresence system needs info about the content of the streams it gets in order to make any of these decisions πŸ€“ if the systems can support more than one strategy, the 2 sites need to negotiate to figure out which of the possible variations they'll use in a specific point-to-point call 🀝

3.3. Multipoint Meeting

a multipoint telepresence conference has more than 2 sites participating 🌍 that adds complexity, because media streams from each participant have to show up on the screens of all the other participants πŸ¦‹

there are SO many topologies that could be used to display the streams from multiple sites in a conference πŸ•ΈοΈ

one major goal for telepresence is to keep that "being there" experience ✨ but in multi-site conferences it's often (in fact, usually) not possible to give everyone full-size video, eye contact, and shared perception of gestures and gaze all at once 😩 there are several policies for stream distribution and display. they all give good results, but they all make different compromises πŸ’…

one common policy is called site switching πŸ”€ say the speaker is at site A and everyone else is at various "remote" sites. when the room at site A is shown, all the camera images from site A get forwarded to the remote sites. so at each receiving remote site, all the screens show camera images from site A 🏠 this can keep full-size image display, and it also gives full visual context of the far end, site A. in site switching, there's a fixed relationship between the cameras in each room and the screens in the remote rooms πŸ“ the room or participants being shown get switched from time to time, based on who's speaking or by manual control, like going from site A to site B πŸ”

segment switching is another policy choice πŸ§ƒ in segment switching (still assuming site A is where the speaker is, and "remote" means all the other sites), instead of sending all the images from site A, only the speaker at site A is shown. the camera images of the current speaker and any previous speakers get forwarded to the other sites in the conference 🎀 so the screens at each site are usually showing images from different remote sites, namely the current speaker at site A and the previous ones. this can keep full-size image display AND capture the non-verbal communication between the speakers πŸ₯Ή in segment switching, the display depends on activity in the remote rooms (generally, but not necessarily, based on audio/speech detection) πŸ—£οΈ

a third possibility is to shrink the image so multiple camera views can be composited onto one or more screens πŸͺŸ this doesn't keep full-size display, but it gives the most visual context, since more sites or segments can be seen at once 🌈 usually in this case the display mapping is static, meaning each part of each room shows up in the same spot on the display screens for the whole conference πŸ“Œ

other policies and combinations are also possible πŸ’œ for example, you could have a static display of all the screens from all the remote rooms, with part or all of one screen used to show the current speaker at full size πŸŽ€

3.4. Presentation ✨

Okay so besides the video and audio of everyone in the room, there are extra streams just for presentations πŸŽ€

Today's systems usually give you only one extra video stream for presentations πŸ’… and it's typically half-duplex, which basically means presenters take turns πŸ™ˆ The presentation stream can come straight off a PC screen, or from a multimedia source like a document camera, camcorder, or DVD πŸ“€ In a multipoint meeting, the presentation streams for whatever is currently being presented always go out to every site, so everyone gets to watch together 🫢

Some systems put the presentation on a screen mounted above or below the three participant screens πŸ–₯️ Others have screens on the conference table for watching presentations πŸ“ If there's more than one presentation screen, they usually show identical content, and the placement, number, and size of these screens varies a lot between systems πŸ’–

Presentation audio is handled two ways: some systems pre-mix it with the room audio, and others send a separate presentation audio stream (if the presentation even has audio) πŸ”Š

In H.323 [ITU.H323] systems, H.239 [ITU.H239] is typically used to control the video presentation stream 🧚 In SIP systems, similar control can be done with the Binary Floor Control Protocol (BFCP) [RFC4582] for the presentation token πŸ¦‹ These mechanisms are fine for managing a single presentation stream.

Today's systems are still stuck on one video presentation stream, but there are obvious reasons we'd want multiple ones πŸ‘‘

  1. The meeting convener is usually running through a meeting agenda, and it's super useful for her to show that agenda to everyone πŸ“‹ Remote participants can also make presentations during the meeting, taking turns with each other. The presentations and the agenda can be shown on separate screens, or rescaled and shown together on one screen 🌸
  2. A single multimedia presentation can include multiple video streams that should be shown together 🎬 Imagine a presenter talking about fairness in media coverage. Along with slides that back up her conclusions, she has video excerpts from various news programs to illustrate her findings. She plays the excerpts from a DVD player so she can pause and reposition the video as needed ⏯️
  3. An educator is presenting a multiscreen slide show πŸ–ΌοΈ This requires the images to be placed consistently across the multiple screens at each site ✨

There are many other examples where multiple presentation streams are useful 🌈

3.5. Heterogeneous Systems 🌍

In meetings, people hop in from all kinds of places using all kinds of devices. A multiscreen immersive telepresence conference might include someone on a PC-based video conferencing system, someone calling in by phone, and (soon!) someone on a handheld device πŸ“±

So what experience or view does each of these devices get? πŸ€”

Some devices can handle multiple streams, and others can only handle a single stream. (We're not talking about legacy systems here, but about systems built to join this kind of conference, even though they're single stream only.) With a single video stream, the stream might contain one or more compositions depending on how much screen space the device has. In most cases, an intermediate transcoding device will be relied upon to produce a single stream, maybe with some kind of continuous presence 🧩

Bit rates vary too, with handheld devices and phones having lower bit rates than PC and multiscreen systems πŸ“‰

Layout follows different policies. For example, a handheld device and a PC might both receive the active speaker stream. That decision can be made explicitly by the receiver, or by the sender if it can receive some kind of rendering hint 🎯 Audio works the same way: a receiver gets a mixed stream, or a number of the loudest speakers if mixing isn't available in the network πŸ”ˆ

For the PC-based participant, the experience depends on the application πŸ’» It could be single stream, like a handheld device but with a bigger screen. Or it could be multiple streams, like an immersive telepresence system but with a smaller screen. Control over the streams can happen locally in the software application, or somewhere else and be sent to the application over the network πŸŽ›οΈ

The handheld device is the most extreme case πŸ“± So how will that participant be seen and heard? It should be an equal participant, even though its bandwidth is much lower than an immersive system's πŸ’œ A receiver may choose to display output from a handheld device differently based on its resolution, but that would be true of any low-resolution video stream, for example one coming from a powerful PC on a bad network πŸ“Ά

The handheld device sends and receives a single video stream, which could be a composite or a subset of the conference. It can say what it wants, or it can accept whatever the sender (a conference server or the sending endpoint) thinks is best πŸ™Œ And if the handheld device wants to take any actions, it has to signal them the same way an immersive system does πŸš€

3.6. Multipoint Education Usage πŸŽ“

The big deal with this example is that the multiple video streams aren't being used to create an immersive experience with panoramic views at every site πŸ™…β€β™€οΈ Instead, they're used to let remote students fully take part in a university class. Sometimes the same video stream shows up on several screens in the room, and other times an available stream isn't shown at all πŸ“Ί

The main site is a university auditorium with 3 cameras πŸŽ₯ One camera is focused on the professor at the podium. A second camera is mounted on the wall behind the professor and captures the whole class. The third camera sits right next to the second and is set up to get a close-up of a questioner in the audience. It automatically zooms in on that student using sound localization πŸ”

The auditorium has 3 cameras but only 2 screens πŸ–₯️ One is a large screen at the front so the class can see it. The other is at the rear so the professor can see it. When someone asks a question, the front screen shows the questioner. Otherwise, it shows the professor, so everyone can easily see her πŸ‘©β€πŸ«

The remote sites are typical immersive telepresence rooms, each with 3 camera/screen pairs πŸ’«

Every remote site shows the professor at full size on the center screen πŸ’– A second screen shows the whole classroom view while the professor is talking. When a student asks a question, though, that second screen switches to a full-size close-up of the student. Sometimes the student is in the auditorium, and sometimes the speaking student is at another remote site. Remote systems never show the students who are actually in that room πŸ™ˆ

If someone at a remote site asks a question, the auditorium screen shows that remote student at full size, as if they were sitting in the auditorium itself πŸ™‹β€β™€οΈ The rear screen also shows this questioner, so the professor can see and answer the student without turning her back on the main class πŸ”„

When nobody is asking a question, the rear screen briefly cycles through a full-room view of each remote site, so the professor can keep an eye on the whole class, both remote and local πŸ‘€ She can also use a control on the podium to look at one particular site, choosing either a full-room view or a single-camera view πŸŽ›οΈ

This use case doesn't need any negotiation between the participating sites 🀝 Endpoint devices (and a Multipoint Control Unit (MCU), if there is one) need to know who is speaking and which video stream has that speaker's view πŸ—£οΈ The remote systems need some idea of which stream should go in the center. For the professor to see specific sites (or for the system to cycle through all the sites), the auditorium system also needs to know which sites are available and be able to request a particular view of any site πŸ“‘ Bandwidth gets optimized when video that isn't being shown at a site isn't sent to that site πŸ’Έ

3.7. Multipoint Multiview (Virtual Space) πŸ›‹οΈ

This use case describes a virtual space multipoint meeting with good eye contact and a spatial layout of participants πŸ‘€ It was proposed very early in video conferencing history, back in 1983 by Allardyce and Randal [virtualspace], and it's illustrated in Figure 2-5 of their report πŸ“œ The virtual space idea expands the point-to-point case by having all the multipoint participants "seated" in a virtual room πŸͺ‘ Each participant has a fixed "seat," so each one sees a different view, with a different participant on their left and right πŸ‘― Today, this use case shows up in several telepresence-type video conferencing systems on the market. The report called it "virtual space," and the big difference between then and now is the screen sizes, which are much larger today πŸ“

Virtual space multipoint, as defined here, assumes endpoints with multiple cameras and screens πŸ“Έ Usually, a given endpoint has the same number of cameras and screens, with a camera positioned above each screen 🎯 The big thing about virtual space multipoint is how the cameras are aimed. Each camera is aimed at the same area where the site's participants are sitting, so every camera captures the same group of people from a different angle πŸ”„ That means each endpoint sender offers remote receivers a choice of video streams, with each stream showing a different viewpoint πŸ‘οΈ For example, a camera above the screen on a participant's left might capture the participant's left ear, while a camera above the screen on their right captures their right ear πŸ‘‚

Since a sending endpoint has a camera for each screen, each receiving stream shown on a particular screen gets matched up with the sending stream from the camera above that same screen πŸ”— This matching happens for every screen/camera pair in a meeting πŸ’• The end result is a horizontal row of video images from remote sites, one per screen, with each image paired with the camera above its screen, which gives excellent eye contact 😭✨

3.8. Multiple Presentation Streams - Telemedicine ✨

okay so this one is my favourite πŸ©ΊπŸ’– it's about a surgery room that's hooked up to one or more remote sites, and the remote sites don't all have the same gear. bestie, the local room is giving main character energy: 3 main cameras capture the whole room (the classic 3-camera telepresence setup πŸŽ€), plus a bunch of presentation inputs. there's a surgery camera for a zoomed view of the operation, an endoscopic monitor, a flouroscope (X-ray imaging), an ultrasound diagnostic device, an electrocardiogram (ECG) monitor, and more πŸ”₯. all of these feed in multiple local video presentation streams so the surgeon can keep an eye on the patient and stay on top of the procedure 🫢

the local site might have 3 main screens plus one (or more) presentation screen(s) πŸ–₯️✨ the main screens show the remote experts, and the presentation screens can show several presentation streams from local and remote sites at the same time. the 3 main cameras each cover a different part of the room πŸ“Έ and the surgeon gets to decide how many presentations show up on the local presentation screens, how big they are, and where they go πŸ‘‘ he also picks which local presentation captures get sent out to the remote sites. so the local site can send out multiple presentation captures and can receive multiple presentations about the patient or the procedure. literally so much data, and it all has to make sense 😭

one type of remote site is a single- or dual-screen setup with one camera, used by a consulting expert πŸ’… in the general case, these remote sites can be part of a multipoint telepresence conference 🌈 the presentation screens at the remote sites let the experts zoom in on the details of the operation and related data πŸ” same as the main site, the experts get to choose the number, size, and placement of the presentations on their screens. those screens can show streams from the surgery room, from other remote sites, or from local presentation streams. and the experts can start sending their own presentation streams too, like medical records, pathology data, or their references and analysis 🧚

another remote site type is a typical immersive telepresence room with 3 camera/screen pairs 🌸 this lets even more experts jump into the consult. these rooms can also be used for education, where the teacher (who isn't necessarily the surgeon) and the students are at different remote sites πŸŽ“ the students can watch and learn every detail of the whole procedure, while the teacher explains things and answers questions during the operation. it's giving classroom in the best way πŸ™ˆ

all the remote education sites can display the surgery room πŸ₯ another option is putting the surgery room on the center screen, with the other screens showing the teacher and whichever student is asking a question πŸ’œ for all of the above sites, multiple presentation screens can boost visibility: one screen for the zoomed surgery stream, and the others for medical image streams like MRI images, cardiograms, ultrasonic images, and pathology data πŸ¦‹


4. Acknowledgements 🫢

the document got help from a bunch of amazing people, including Alex Eleftheriadis, Marshall Eubanks, Tommy Andre Nyquist, Mark Gorzynski, Charles Eckel, Nermeen Ismail, Mary Barnes, Pascal Buhler, and Jim Cole πŸ’–

special shoutout to Lennard Xiao, who wrote the text for the telemedicine use case 🩺✨ and to Claudio Allocchio for his super detailed review of the document πŸ”₯


5. Security Considerations πŸ”

okay, real talk bestie 😭 this section is short but it's spicy. while there are likely to be security considerations for any solution for telepresence interoperability, this document has no security considerations πŸ’€ so yeah, nothing here, but any real telepresence solution is definitely going to need to think about security 🚨


6. Informative References πŸ“š

these are the sources that are helpful background reading, not requirements 🌸

  • [ITU.H239] ITU-T, "Role management and additional media channels for H.300-series terminals", ITU-T Recommendation H.239, September 2005. πŸŽ€
  • [ITU.H264] ITU-T, "Advanced video coding for generic audiovisual services", ITU-T Recommendation H.264, April 2013. 🎬
  • [ITU.H323] ITU-T, "Packet-based Multimedia Communications Systems", ITU-T Recommendation H.323, December 2009. πŸ“ž
  • [RFC3261] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, "SIP: Session Initiation Protocol", RFC 3261, June 2002. πŸ¦‹
  • [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, July 2003. 🧚
  • [RFC4582] Camarillo, G., Ott, J., and K. Drage, "The Binary Floor Control Protocol (BFCP)", RFC 4582, November 2006. πŸ’œ
  • [virtualspace] Allardyce, L. and L. Randall, "Development of Teleconferencing Methodologies with Emphasis on Virtual Space Video and Interactive Graphics", April 1983, <http://www.dtic.mil/docs/citations/ADA127738>. 🌈

Authors' Addresses πŸ’Œ

Allyn Romanow Cisco San Jose, CA 95134 US

EMail: allyn@cisco.com

Stephen Botzko Polycom Andover, MA 01810 US

EMail: stephen.botzko@polycom.com

Mark Duckworth Polycom Andover, MA 01810 US

EMail: mark.duckworth@polycom.com

Roni Even (editor) Huawei Technologies Tel Aviv Israel

EMail: roni.even@mail01.huawei.com

and that's a wrap on RFC 7205 πŸŽ‰πŸ’–βœ¨

πŸ“¦ where this page comes from