RFC 7272
Inter-Destination Media Synchronization (IDMS) Using the RTP Control Protocol (RTCP)
RFC 7272 💖 Inter-Destination Media Synchronization (IDMS) Using the RTP Control Protocol (RTCP)
okay bestie, let's start with the header tea ☕🎀 This is RFC 7272, published in June 2014 by the IETF. It's titled Inter-Destination Media Synchronization (IDMS) Using the RTP Control Protocol (RTCP) 🦋
The authors are R. van Brandenburg, H. Stokking, and O. van Deventer from TNO, F. Boronat and M. Montagud from UPV, and K. Gross from AVA Networks ✨ The category is Standards Track, and the ISSN is 2070-1721. The header doesn't list anything this RFC obsoletes or updates 🌸
Abstract
okay so this doc defines a brand-new RTCP Packet Type and an RTCP Extended Report (XR) Block Type, both for achieving Inter-Destination Media Synchronization (IDMS) 🔥 IDMS is the process of syncing playout across multiple media receivers, so everyone's stream lines up 💅
With the RTCP XR IDMS Report Block from this doc, media playout info from everyone in a synchronization group can get collected 🧚 Then an RTCP IDMS Settings Packet can be sent out to distribute a common target playout point. That's the spot every distributed receiver sharing a media experience can sync up to 👑
The iconic use cases are social TV, shared service control (aka apps where two or more geographically separated users watch a media stream together), distance learning, networked video walls, networked loudspeakers, and more 🍓
Status of This Memo
this is an Internet Standards Track document ✨
it's a product of the Internet Engineering Task Force (IETF) and represents the consensus of the IETF community 🫶 It got public review and was approved for publication by the Internet Engineering Steering Group (IESG) 💜 More info on Internet Standards is in Section 2 of RFC 5741 📚
you can check the current status, any errata, and how to give feedback at http://www.rfc-editor.org/info/rfc7272 🌈
Copyright Notice
Copyright (c) 2014 IETF Trust and the persons identified as the document authors. All rights reserved. 🎀
this doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as in effect on the date of publication. Please review those docs carefully, because they describe your rights and restrictions with respect to this document 🙈 Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they're provided without warranty as described in the Simplified BSD License 💅
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Terminology . . . . . . . . . . . . . . . . . . . . . . . 3
2. Rationale . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.1. Applicability of RTCP to IDMS . . . . . . . . . . . . . . 3
2.2. IDMS and ETSI . . . . . . . . . . . . . . . . . . . . . . 4
3. Inter-Destination Media Synchronization (IDMS) Use Cases . . 4
4. Overview of IDMS Operation . . . . . . . . . . . . . . . . . 5
5. Architecture for Inter-Destination Media Synchronization . . 7
5.1. Media Synchronization Application Server (MSAS) . . . . . 7
5.2. Synchronization Client (SC) . . . . . . . . . . . . . . . 8
5.3. Communication between MSAS and SCs . . . . . . . . . . . 8
6. RTCP XR IDMS Report Block . . . . . . . . . . . . . . . . . . 8
7. RTCP Packet Type for IDMS (IDMS Settings Packet) . . . . . . 11
8. Timing and NTP Considerations . . . . . . . . . . . . . . . . 13
9. On the Use of Presentation Timestamps . . . . . . . . . . . . 14
10. SDP Signaling for RTCP IDMS Settings Packet . . . . . . . . . 15
11. SDP Rules . . . . . . . . . . . . . . . . . . . . . . . . . . 16
11.1. Offer/Answer Rules . . . . . . . . . . . . . . . . . . . 16
11.2. Declarative Cases . . . . . . . . . . . . . . . . . . . 17
12. Security Considerations . . . . . . . . . . . . . . . . . . . 17
13. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 18
13.1. RTCP IDMS Packet Type . . . . . . . . . . . . . . . . . 18
13.2. RTCP XR IDMS Report Block . . . . . . . . . . . . . . . 19
13.3. RTCP-IDMS SDP Attribute . . . . . . . . . . . . . . . . 19
13.4. IDMS XR Block SPST Registry . . . . . . . . . . . . . . 19
13.5. Contact Information for Registrations . . . . . . . . . 20
14. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 20
15. References . . . . . . . . . . . . . . . . . . . . . . . . . 21
15.1. Normative References . . . . . . . . . . . . . . . . . . 21
15.2. Informative References . . . . . . . . . . . . . . . . . 211. Introduction ✨
okay so IDMS is about playing media streams at two or more geographically distributed locations in a time-synchronized way 🦋 It works with both unicast and multicast streams, and with any type or combo of streaming media, like audio, video, and text (subtitles) 🌸 [Ishibashi2006] and [Boronat2009] give you an overview of the technologies and algorithms for IDMS 📚
IDMS needs participants in an IDMS session to exchange info on when media arrives and when it gets presented 💖 It may also need signaling to start and maintain IDMS sessions and groups of receivers 🫶
the RTCP spec for IDMS doesn't care which synchronization algorithm you use. That algorithm is out of scope for this document 🙈
1.1. Terminology
the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be read as described in RFC 2119 [RFC2119] 💅
2. Rationale 🌈
2.1. Applicability of RTCP to IDMS
so a big chunk of real-time apps use RTP and RTCP [RFC3550] right now 🔥 RTP gives you end-to-end network transport for real-time data like audio, video, or other data, over multicast or unicast network services 🎀 Its timestamps, sequence numbers, and payload (content) type identification are super useful for rebuilding the original media timing and packet order, and for spotting packet loss at the receiver 👑
the data transport gets a sidekick, RTCP, which lets you monitor data delivery in a way that scales to large groups, plus provides minimal control and identification functions 🧚 RTP receivers and senders send reception quality feedback through RTCP receiver report (RR) and sender report (SR) packets [RFC3550] respectively. Those can also be boosted with extended report (XR) blocks [RFC3611] ✨
here's the vibe: IDMS is all about collecting, summarizing, and distributing RTP packet arrival and presentation times 💜 Since info on arrival and presentation times counts as reception quality feedback, RTCP is a perfect fit for IDMS 😭
2.2. IDMS and ETSI
a first version of IDMS for RTP/RTCP was standardized by ETSI Telecommunications and Internet converged Services and Protocols for Advanced Networking (TISPAN) in [TS183063], which got an IANA registration for an RTCP XR Block Type 🍓 That work was then handed over to the IETF AVTCORE WG for further standardization, using the RTP/RTCP expertise already in AVTCORE. This document is the result of that effort 🦋
the IDMS protocol here has evolved a lot from the version ETSI TISPAN originally specified, but it's still backwards compatible with the ETSI version 🌸 So ETSI decided to update the TS 183 063 document so it references this document as the normative spec of IDMS. That update shows up in newer versions of TS 183 063 (versions higher than 3.5.2) 💖 This document also proposes updating the IANA registration for the RTCP XR IDMS Report Block to point here 🫶 Finally, it proposes an IANA registry for Synchronization Packet Sender Type (SPST) values, so extensions to this document can be registered 👑
3. Inter-Destination Media Synchronization (IDMS) Use Cases 🎀
there are tons of use cases where IDMS might be useful, and this section highlights some of them 🌈 It should be noted that this section is in no way meant to be exhaustive. It's not the full list, bestie 💅
first up is social TV. Social TV is when two or more users consume media content on different devices and at different locations, while also talking in real time 📺 For example, two or more users watch the same TV broadcast on different devices and in different places, while chatting with each other by text, audio, and/or video 💜 A skew in their media playout can wreck the experience. The classic example is one friend seeing a goal in a football match well before or after the other friend(s) 😭⚽
another potential use case is a networked video wall 🦋 A video wall is multiple computer monitors, video projectors, or TV sets tiled together, either touching or overlapping, to form one big screen. Each screen shows a piece of the larger picture 🌸 In some setups, each screen connects to the network on its own and gets its portion of the image from a network-connected video server or video scaler 🧚 Screens refresh at 60 hertz (every 16-2/3 milliseconds) or possibly faster. If the refresh isn't synced, the screens stop acting as one, and users notice tearing and stop seeing a single image 💀
a third scenario is networked loudspeakers, where two or more speakers each connect to the network individually 🔊 You'll find these in large conference rooms, legislative chambers, classrooms (especially ones supporting distance learning), and other large spaces like stadiums 🏟️ Humans are more sensitive to differences in audio delay than video delay, so this use case needs even more accuracy than the video wall one 🙈 Depending on the exact application, that accuracy can need to be in the range of microseconds ✨
4. Overview of IDMS Operation ✨
okay so this section is a tutorial, no rules here 🎀 it just walks through how RTCP makes IDMS happen, like a little story. ngl it's so cute 🦋
Here's the vibe in one diagram:
Alice's . . . . . . .tv:abc.com . . . . . . . Bob's
TV (Sync Client) (Sync Server) Laptop (Sync Client)
| | |
| Media Session | |
|<=====================>| |
| Invite(URL, SyncGroupId) |
|------------------------------------------------->|
| | Media Session Setup |
| |<========================>|
| | |
| Call Setup |
|<================================================>|
| | |
| RTP Packets | RTP Packets |
|<----------------------|------------------------->|
| RR + XR IDMS Report | |
|---------------------->| RR + XR IDMS Report |
| |<-------------------------|
| RTCP IDMS Settings | RTCP IDMS Settings |
|<----------------------|------------------------->|
| | |Figure 1: Example of a Typical IDMS Session 💜
So Alice is chilling on her TV in the living room 🛋️ and she spots that Bob's fave team is playing football 🏈 She's like "bestie, come watch with me!" and sends him an invite. The invite has the link to the media server AND a unique sync-group identifier baked in 🔗✨
Bob's at home too and gets the invite on his laptop. He accepts, and the RTP client on his laptop sets up a session with the media server 💻 A VoIP connection to Alice's TV also gets set up, so they can literally talk about the game while watching it together. the friendship goals are real 🫶
Here's the part I'm obsessed with. As usual with RTP, both Alice's TV client and Bob's laptop client keep sending periodic RTCP RRs to the media server. BUT, to make sure they see the same football moments at the same time, their clients also periodically send an RTCP XR IDMS Report Block to the Sync Server function of the media server 🔥 Those report blocks include timestamps for when each of them received a particular RTP packet, and optionally when they played it out too 🕰️
Next, the Sync Server function looks at all those report blocks and picks a reference client. For example, it can choose the most lagged client as the reference for IDMS 🐢 Then it sends an RTCP IDMS Settings Packet holding that reference client's playout info to the sync clients of both Alice and Bob 📦
In this case, Bob's connection has the longest delay, so the reference client has a delay similar to the one Bob is dealing with 😭 When Alice's RTP client gets this info, it gets to decide what to do. Here it temporarily slows its playout rate until its playout time lines up with the reference client's (which means lining up with Bob) 🐌 Another option for Alice's TV is to just pause until it catches up ⏸️ The exact synchronization algorithm is totally up to the client, so it's their call 💅
And Bob's client? Iconic behavior: it doesn't have to do anything at all 🧚 It's already in sync with the reference client, since the reference is based on Bob's own delay.
Heads up, babe: other sync algorithms might add even more delay than the most delayed client. They might do that to deal with delay variations, or to handle new clients joining an existing sync group, and stuff like that 🙈
One more thing that's kind of the whole secret sauce 🌈 For this to work correctly, the wallclocks of the receivers have to be synchronized with each other. Alice and Bob both report when they receive certain RTP packets (and optionally when they play them out), and so their reports can be correlated with each other, their wallclocks need to match up. No wallclock sync, no vibe ✨
5. Architecture for Inter-Destination Media Synchronization 🦋
The IDMS architecture is based on a sync-maestro architecture [Boronat2009], and it's diagrammed below 🍓 In this particular setup, the Synchronization Client (SC) and the Media Synchronization Application Server (MSAS) show up as extra functionality for the RTP receiver and the RTP sender, respectively 🧩
+-----------------------+ +-----------------------+
| | SR + | |
| RTP Receiver | RTCP | RTP Sender |
| | IDMS | |
| +-----------------+ | <----- | +-----------------+ |
| | | | | | | |
| | Synchronization | | | | Media | |
| | Client | | | | Synchronization | |
| | (SC) | | | | Application | |
| | | | | | Server | |
| | | | RR+XR | | (MSAS) | |
| | | | -----> | | | |
| +-----------------+ | | +-----------------+ |
| | | |
+-----------------------+ +-----------------------+Figure 2: IDMS Architecture Diagram 🌸
5.1. Media Synchronization Application Server (MSAS) 👑
The MSAS is the boss of collecting info 💁♀️ It gathers RTP packet arrival times and presentation times from one or more SCs in a synchronization group by receiving RTCP XR IDMS reports. Then it summarizes all that and passes it back out to the SCs in the group as synchronization settings, using RTCP IDMS Settings Packet messages. For example, it can figure out which SC has the most lagged playout, then use that SC's reported RTP packet arrival time and presentation time as the summary 🐢
Now, the diagram shows the MSAS living inside the RTP sender, but that's not a must-have 🙌 An MSAS could also be its own independent function in the network. Or it could be a master/slave kind of setup where one of the SC devices also acts as the MSAS 🧚 Wherever the MSAS lives, it's super important that it can access the RTP stream the XR reports are about. That's how it correlates the RTCP XR IDMS reports coming from different SCs, and honestly that's giving detective energy 🕵️♀️✨
5.2. Synchronization Client (SC) 🎀
The SC is the one reporting on the vibes of the media stream 📡 It keeps track of RTP packet arrival times and presentation times. It can also receive IDMS Settings Packets, which hold summaries of that same kind of info, and then it uses them to adjust its playout buffer 🎚️ And to send its own news back, the SC sends RTCP XR IDMS reports to the MSAS 💌
5.3. Communication between MSAS and SCs 💜
The MSAS and SCs chat using two different message types 💬 For the SC->MSAS message with a particular client's arrival and playout info, they use an RTCP XR IDMS Report Block (see Section 6) 📊 For the MSAS->SC message with the synchronization settings instructions, there's a brand new RTCP IDMS Settings Packet (see Section 7) 🆕 Two lanes, one for each direction, very organized, very slay 💅
6. RTCP XR IDMS Report Block ✨
okay bestie, this section defines a brand new RTCP XR Block Type, the RTCP XR IDMS Report Block 🦋. SCs use it to send IDMS info back to the MSAS, mainly feedback on when RTP packets arrived and when they got presented 🎀. It's built on top of [RFC3550] and [RFC3611], so it's giving "building on what already works" energy 💅.
Most of the time, an RTP receiver is only in one IDMS session at once, reporting on one RTP stream in one synchronization group 🌸. But sometimes it's more complicated. A receiver might be in multiple sync groups for the same RTP stream, like watching one TV show with different groups at the same time 📺. Even more main character, a receiver might want to sync different RTP streams, either in the same group or across several groups 👑. All of these are totally valid IDMS scenarios, and they all mean the SC sends multiple reports 💜.
This document doesn't invent new rules for when to send RTCP reports 🙅♀️. It just uses the existing rules from [RFC3550]. So when the RTCP reporting timer says an SC can send an IDMS report, the SC SHOULD report on an RTP packet it received since the last RTCP XR IDMS Report Block went out 🍓. Why? Because RTP timestamps roll over, mapping them to NTP timestamps gets a little ambiguous 🙈. Reporting on recent packets is the trick that keeps that ambiguity under control ✨. For more on which packet to pick, check out "Packet Received RTP timestamp" below 🧚.
Here's the layout of the block 💖:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| BT=12 | SPST |Resrv|P| block length=7 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PT | Resrv |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Media Stream Correlation Identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC of media source |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Packet Received NTP timestamp, most significant word |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Packet Received NTP timestamp, least significant word |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Packet Received RTP timestamp |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Packet Presented NTP timestamp |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+The block is 8 32-bit words long, and here's what lives in each field 🎀:
Block Type (BT): 8 bits. It tells you what format the block is in, and its value is always 12 🔥.
Synchronization Packet Sender Type (SPST): 4 bits. This one says what role the packet sender is playing in this specific Extended Report 👑. The values live in an IANA-maintained registry (see Section 13.4), and they're:
SPST=0: reserved for future use 🌙.SPST=1: the packet sender is an SC 💅. It uses this XR to report synchronization status, and the timestamps relate to the SC input 🎧.SPST=2-4: values defined by ETSI TISPAN (see [TS183063]) 🇪🇺.SPST=5-15: unassigned 🫧.
Reserved bits (Resrv): 3 bits. These are saved for future definition 🗝️. Until someone defines them, the bits in this field MUST be set to zero when sent, and the receiver MUST ignore them 🙅♀️. Non-negotiable, bestie.
Packet Presented NTP timestamp flag (P): 1 bit. It's set to 1 if the Packet Presented NTP timestamp field has a value, and 0 if that field is empty 🫧. When P is 0, the receiver SHALL ignore the Packet Presented NTP timestamp 🙈.
Block Length: 16 bits. This is the length of the block in 32-bit words minus one, and it's always 7 because this block is fixed-length 📏.
Payload Type (PT): 7 bits. This identifies the media payload format, according to [RFC3551] 🎶. It's the payload type of the RTP packet being reported on 🎞️. The PT field matters most when the MSAS is neither the media server nor a receiver of the stream, so it's a third-party entity 🧳. In that case, the MSAS needs the PT to figure out how fast the RTP media stream's timestamps move forward, so it can line up reports from different SCs against different RTP timestamp values ⏱️.
Reserved bits (Resrv): 25 bits. These are reserved for future use 🌌. They SHALL be set to 0 when sent, and the receiver MUST ignore them 🧊.
Media Stream Correlation Identifier: 32 bits. This ID is how synchronized media streams get matched up together 🔗. The value 0 (all bits "0") means the field is empty 🫥. The value 2^32-1 (all bits "1") is reserved for future use 🔮. When the RTCP Packet Sender is an SC (SPST=1), this field carries the Synchronization Group Identifier (SyncGroupId) the report applies to 💜.
Synchronization Source (SSRC): 32 bits. This is the SSRC of the media source, copied from the SSRC in the RTP header [RFC3550] of the RTP packet the XR IDMS relates to 🦋.
Packet Received NTP timestamp: 64 bits. This is the wallclock time at the exact moment the first octet of the RTP packet arrived 🕰️. It's formatted using the NTP timestamp format from [RFC5905] ✨. Section 8 explains how this field gets used, so go check that out 👀.
Packet Received RTP timestamp: 32 bits. This is the RTP timestamp from the RTP header [RFC3550] of the packet the XR IDMS relates to 🎬. Here's the tricky bit: several RTP packets in a row can have the same timestamp if they were generated together, like all belonging to one video frame 🎞️. So one receiver might report on the first packet with a given RTP timestamp, while another reports on the last one 😭. That mismatch would make the sync algorithm think both reports are about the same packet, and that's an error 💀. So when an SC reports on one of several consecutive packets with equal timestamps, it SHOULD report on the packet it received with the lowest sequence number 🌈. Quick clarification: "lowest sequence number" here means the first in the sequence of packets just received, not something from way back before the last RTP timestamp wrap-around. (The exception is if that wrap-around happens during the equal-timestamp sequence itself) 🌀.
Packet Presented NTP timestamp: 32 bits. This is the wallclock time at the moment the rendered media unit (like a video frame or audio sample) in the first byte of the associated RTP packet gets presented to the user 🎧📺. It uses the NTP time format but only keeps the least significant 16 bits of the NTP seconds part and the most significant 16 bits of the NTP fractional second part 🧮. If there's no Packet Presented NTP timestamp available, this field SHALL be set to 0 and treated as empty, and the Packet Presented NTP timestamp flag (P) SHALL be set to 0 too 🫧. For NTP epoch and rollover, the Packet Presented NTP timestamp is always considered greater than the Packet Received NTP timestamp, and within 2^16 seconds of it ⏳. "Presented" here means the moment the data actually plays out to the user, like sound coming out of speakers or video showing on a screen 🔊🖥️.
⚠️ The accuracy of the synchronization algorithm can only be as good as the SCs' ability to measure the delay between receiving packets and presenting them to the end user. If SCs don't report presentation timestamps, syncing playout may be limited 🫠.
7. RTCP Packet Type for IDMS (IDMS Settings Packet) ✨
okay so this section is where we define the RTCP packet type for IDMS 🎀 it's the IDMS Settings Packet, and it's how the MSAS tells the receivers of the RTP media stream what sync settings to use 💅 it's built on [RFC3550], obvi. The settings show up as a report that points at a real or hypothetical RTP packet that the MSAS either picked or made up 🦋
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P| Resrv | PT=211 | length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC of packet sender |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| SSRC of media source |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Media Stream Correlation Identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Packet Received NTP timestamp, most significant word |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Packet Received NTP timestamp, least significant word |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Packet Received RTP timestamp |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Packet Presented NTP timestamp, most significant word |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Packet Presented NTP timestamp, least significant word |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+The first 64 bits are the RTCP packet type header, as defined in [RFC3550] 🧚 and the SSRC of the packet sender tells you who sent this specific RTCP packet.
The RTCP IDMS Settings Packet is 7 32-bit words long, with these fields:
PT: 211, as registered by IANA. 🔥
SSRC: 32 bits. The SSRC of the media source gets set to whatever the SSRC identifier of the media source is in the RTP header [RFC3550] of the RTP packet this RTCP IDMS Settings Packet is about 🫶
Media Stream Correlation Identifier: 32 bits. This one is how you match up synchronized media streams 👯 the value 0 (all bits set to "0") means the field is empty, and the value 2^32-1 (all bits set to "1") is reserved for future use. The Media Stream Correlation Identifier holds the SyncGroupId of the group this packet is sent to 💜
Packet Received NTP timestamp: 64 bits. This timestamp is the wallclock time at the reference client the exact moment it got the first octet of the RTP packet this packet relates to 😭 the sync algorithm on the receiving SC can use it to adjust its playout timing and get in sync, like setting the playout delay it needs 🎯 the timestamp is formatted based on the NTP timestamp format from [RFC5905]. Check out Section 8 for more on how this field gets used. Also, RTP timestamps wrap around, so the sender of this packet MUST use recent values. Basically, pick NTP timestamps that reflect the current time, and don't go too far into the future or the past, or you'll create confusion about RTP timestamp wrap around 🙈
Packet Received RTP timestamp: 32 bits. This one holds the RTP timestamp from the RTP header [RFC3550] of the RTP packet the XR IDMS relates to. It SHOULD relate to the first arriving RTP packet with this particular RTP timestamp, in case several consecutive RTP packets share the same RTP timestamp 🌸
Packet Presented NTP timestamp: 64 bits. This is the wallclock time at the reference client the moment it presented the rendered media unit (like a video frame or audio sample) that's in the first octet of the associated RTP packet to the user 🎬 it's formatted based on the NTP timestamp format from [RFC5905]. If there's no Packet Presented NTP timestamp available, this field SHALL be set to 0 and count as empty. This field MAY be left empty if none or only one of the receivers reported presentation timestamps. "Presented" just means the moment the data gets played out to the user of the system ✨
Lowkey the most important bit here: "Presented" means the moment the user actually sees or hears it, not the moment it hits the software. 💀
In some use cases (like phased array transducers), the MSAS might need such precise control over the exact moment of playout that a 32-bit Presented timestamp just won't cut it 😤 that's why this RTCP packet type for IDMS has a 64-bit Presented Timestamp field. Since an MSAS will always add some extra delay on top of the delay reported by the most lagged receiver (to account for packet jitter), the RTCP XR IDMS Report Block that SCs use to report their playout can get away with a 32-bit Presented Timestamp field 🧁
8. Timing and NTP Considerations
okay this one's giving clocks 🕰️ to achieve IDMS, all the receivers involved need synchronized wallclocks so they're all on the same timeline 🫶 this synchronized clock is used for reporting the Packet Received NTP timestamp and the Packet Presented NTP timestamp, and for reading those fields in received IDMS Settings Packets. Different clock sync methods can be used depending on how accurate you need to be. For social TV, the sync accuracy should be on the order of hundreds of milliseconds, and in that case, using NTP correctly on receivers will get the job done in most situations 🍓 as a guideline, to handle clock drift on receivers, receivers should sync their clocks at the start of a synchronized session. If you need high accuracy, the synchronized clocks of different receivers should not drift past the accuracy the sync mechanism needs. In practice, that can mean receivers have to re-sync their clocks over and over during a sync session 😩
Because some use cases need stringent sync to get good audio quality, a high accuracy is needed 🎧 in that case, the global NTP system might not be enough. For better accuracy, you could set up a local NTP server, or use some other more accurate clock sync mechanism, like GPS time or the Precision Time Protocol [IEEE1588-2008] 🛰️
[RFC7273] defines a set of Session Description Protocol (SDP) parameters for signaling the clock synchronization source or sources that are available to and used by the individual receivers 📡 SCs MAY use [RFC7273] to indicate their clock synchronization source or sources in use and available. With these parameters, an SC can say which sync source it's using right now, and it can also list any other sync sources available to it. This lets multiple SCs in an IDMS session use the same or a similar clock source for their session 👯♀️
Applications doing IDMS may or may not be able to pick a sync method for the system clock, since it might be a system-wide setting the application can't change 🙃 how applications handle that is up to the implementation. The application might control the system clock, or it might use a separate application clock or even a separate IDMS session clock. It might also just report on the system clock and the sync method used, without being able to change it 🤷♀️
[RFC7164] gives some guidelines on how RTP senders and receivers should deal with leap seconds ⏱️ when you rely on NTP for clock sync, IDMS is especially sensitive to timing discrepancies caused by leap seconds. It is RECOMMENDED to take the guidelines in [RFC7164] into account when implementing IDMS 💅
9. On the Use of Presentation Timestamps
so a receiver can report on different timing events: packet arrival times, and playout or presentation times 📦 a receiver SHALL report on arrival times, and a receiver MAY report on playout times. Arrival times are pretty easy to report on 💖 and normally, processing and playing out the same media stream takes roughly the same amount of time for different receivers. Syncing on packet arrival times might lose a bit of accuracy, but it'll be good enough for lots of apps, like social TV 📺
If the receivers are controlled in some way (like having the same buffer settings and decoding and rendering times), high accuracy is totally possible 🎯 but if every receiver in a sync session can report on, and therefore sync on, actual presentation times, it'll be even more accurate ✨ whether to implement and use presentation timestamps is up to the applications and implementations of this RTCP extension 🫶
10. SDP Signaling for RTCP IDMS Settings Packet
the SDP attribute rtcp-idms is how you signal that you're using the RTCP IDMS Settings Packet and the associated RTCP XR IDMS Report Block 🎀 it also carries an identifier for the sync group that clients belong to, or will belong to. This attribute is used as a media-level attribute during session setup, which means that if there are multiple related streams, IDMS gets performed on just one of them. The other streams get synced to this reference or master stream using existing inter-stream sync solutions (like lip-sync), meaning sender reports based on a common clock source 🎬 the basic guideline for choosing which stream to use for IDMS is to pick audio over video, since humans are the most sensitive to audio sync problems 🔊 when you're using multi-description or multi-view codecs, the IDMS control should be performed on the base layer 🧱
This SDP attribute is defined like this, using Augmented Backus-Naur Form [RFC5234] 📝
rtcp-idms = "a=" "rtcp-idms" ":" sync-grp CRLF
sync-grp = "sync-group=" SyncGroupId
SyncGroupId = 1*10DIGIT ; Decimal value from 0 through 4294967294
DIGIT = %x30-39SyncGroupId is a 32-bit unsigned integer, written in decimal 🔢 it identifies a group of SCs for IDMS. The value SyncGroupId=0 means an empty SyncGroupId, and the value 4294967295 (2^32-1) is reserved for future use. For a description of which SyncGroupId value to include, see Section 11 💜
Here's an example of the SDP attribute for IDMS 💖
a=rtcp-idms:sync-group=4211. SDP Rules
11.1. Offer/Answer Rules 🎀
okay so the SDP stuff for IDMS follows the rules in [RFC4566] and Section 5 of [RFC3611], except for what's spelled out here 💅. The IDMS side of RTCP is a loosely coupled, collaborative vibe. Receivers send their status info, and the MSAS responds asynchronously with synchronization setting instructions 🔁. So the rtcp-idms attribute basically just says "I can send and receive these RTCP messages" ✨. This section explains how that attribute works in an offer/answer situation.
Most of the time, receivers already know their SyncGroupId before the media session is set up 🗝️. That happens through some means that's outside this document's scope. But senders can also assign a SyncGroupId, and that's supported too. It's useful when the MSAS has group management stuff and is co-located with the sender, or talks to it 🧠. So both senders and receivers can insert the attribute and the SyncGroupId. The attribute can also show up on more than one media stream, which lets an SC join multiple synchronization groups at once 👯. The catch is that this effectively couples two (or more) synchronization groups together 🔗. And if rtcp-idms is inserted more than once for the same media session, each SyncGroupId SHALL only be inserted once 🙅♀️.
To join an IDMS session, the receiver (the SC) puts the rtcp-idms attribute in as a media-level attribute in its SDP offer 🎟️. That offer can be an initial one if the media session is starting out already synchronized. It can also be an update to an existing media session that turns it into an IDMS session 🦋. If the receiver already has a predetermined SyncGroupId, it SHOULD use that value for the SyncGroupId parameter in rtcp-idms 🌸. If the receiver doesn't know which SyncGroupId to use, it MAY leave the SyncGroupId parameter empty by setting its value to 0 🫶.
The sender SHALL include the rtcp-idms attribute in its answer 💖. If the SyncGroupId parameter in the offer isn't empty (isn't equal to 0), the sender SHOULD NOT change the SyncGroupId in its answer 🙌. If the SyncGroupId is empty, the sender SHALL include the proper SyncGroupId in its answer ✨. And if the sender gets an offer with the SyncGroupId parameter set to 0 and can't figure out the proper SyncGroupId, it SHALL remove the attribute from its answer 💀.
A sender that gets an SDP offer without the rtcp-idms attribute can still decide that IDMS applies to that media session 🧚. When that happens, the sender MAY insert the rtcp-idms attribute, including a non-empty SyncGroupId, as part of its answer 🌈.
When a receiver gets an rtcp-idms attribute in the SDP answer from a sender, it SHALL start sending RTCP XR IDMS reports. It has to follow all the normal RTCP rules for sending RTCP XR IDMS Report Blocks while doing that 📊. It SHALL also be ready to start receiving IDMS Settings 👀. As usual, if a receiver doesn't support the attribute (for example, when the MSAS inserted the IDMS attribute), it SHALL ignore the attribute 🙈.
An IDMS session can get different kinds of updates. An update can leave out the rtcp-idms attribute, which ends the receiver's involvement in the synchronization session 👋. An update can also include the rtcp-idms attribute but with a different SyncGroupId. That signals a switch to a different synchronization group 🔀.
11.2. Declarative Cases 📜
Sometimes there's no offer/answer going on at all, just a declarative approach 🙃. In that case, the MSAS simply inserts the rtcp-idms attribute along with a valid SyncGroupId 💅. Any receiver that gets the rtcp-idms attribute in a declarative case like this SHALL start sending RTCP XR IDMS Report Blocks, and it SHALL be ready to start receiving RTCP IDMS Settings Packets 🫶.
12. Security Considerations 🔒
ngl, this section is serious, so grab your snacks and pay attention 🍓. The security considerations from [RFC3611] apply to this document too 🔐.
The RTCP XR IDMS Report Block defined here is used to collect, summarize, and distribute info about when packets were received and when they were played out for streaming media 📡. That info can be used to coordinate media playout across multiple devices 🎬.
Errors in that info can cause undesired behavior, whether they happen by accident or on purpose 😬. For example, say one device wrongly or maliciously reports a playout that's two hours delayed. Then another device in the same synchronization group could decide to delay its own playout by two hours too, just to stay in sync 😭. And the user would most likely think the service was broken 💔.
So the application logic of both SCs and MSASs should check for out-of-bound information 🚨. Differences in playout time that go over configured limits (for example, more than ten seconds) could be a sign of that out-of-bound info 👀.
Aside, but make it loud: out-of-bound checks are your first line of defense, bestie 🛡️.
Beyond checking for out-of-bound info on the endpoints, an IDMS implementation can cut down on its vulnerability to attacks by using source authentication and message integrity measures 🧩. That helps reduce the risk of man-in-the-middle attacks 🥷. [RFC7201] gives an overview of the security options in RTP environments. It also has a set of recommendations for message integrity and source authentication that apply to IDMS 📚. On top of stopping man-in-the-middle attacks from slipping in wrong IDMS reports, the message confidentiality mechanisms in [RFC7201] also stop third parties from figuring out that two or more end hosts are receiving the same stream. They can't tell just by looking at the Media Stream Correlation Identifier 🕵️♀️.
Besides attacking an IDMS session directly by sending incorrect IDMS reports (which adds delays for every device in a synchronization group), there's another weak spot: the clock synchronization method being used ⏰. If an attacker manages to adjust an SC's wallclock, that SC will report incorrect IDMS reports 😱. To prevent these clock synchronization attacks, it is recommended to use a secure time synchronization service 🔐✨.
13. IANA Considerations 🗂️
This document defines a new RTCP packet type, the RTCP IDMS Packet (IDMS Settings), and adds it to the existing Internet Assigned Numbers Authority (IANA) registry of RTCP Control Packet Types 📝. It also defines a new RTCP XR Block Type, the RTCP XR IDMS Report Block, and adds it to the existing IANA registry of RTCP Extended Reports (RTCP XR) Block Types 🧾.
It also defines a new SDP attribute called "rtcp-idms" in the existing IANA registry of SDP Parameters, where it's part of the "att-field (media level only)" 🎯. Last but not least, it defines a brand-new IANA registry that sits under the IANA RTCP Extended Reports (RTCP XR) Block Type Registry: the IDMS XR Block SPST Registry 🌟.
13.1. RTCP IDMS Packet Type 📦
This document assigns the packet type value 211 in the IANA 'RTCP Control Packet types (PT)' registry to the RTCP IDMS Packet (IDMS Settings) 💜.
13.2. RTCP XR IDMS Report Block 📊
This document updates the assignment of value 12 from the RTCP XR Block Type for reporting IDMS information as per [TS183063] to the RTCP XR IDMS Report Block defined in this document 🔄.
The RTCP XR IDMS Report Block has an extensible SPST value field, so a new registry for that field is required 🧩. That new registry is defined in Section 13.4 ✨.
13.3. RTCP-IDMS SDP Attribute 🎀
The SDP attribute "rtcp-idms" defined by this document is registered with the IANA registry of SDP Parameters, like this 🌸:
SDP Attribute ("att-field"):
Attribute name: rtcp-idms
Long form: RTCP IDMS Parameters
Type of name: att-field
Type of attribute: media level
Subject to charset: no
Purpose: see Section 10 of this document
Reference: this document
Values: see this document13.4. IDMS XR Block SPST Registry 🌈
This document defines a new IANA registry that sits under the IANA RTCP Extended Reports (RTCP XR) Block Type Registry: the IDMS XR Block SPST Registry 🦄.
The initial values for the IDMS XR Block SPST Registry are below. Future assignments are made through the Specification Required policy [RFC5226] 📋. The registry is limited to 16 entries (numbered 0-15), and 0 is Reserved 🚫. Values 5-15 are available for assignment 🫧.
In accordance with [RFC5226], a Designated Expert reviews any applications made to IANA for the registry 🧑🏫. When they review new applications, the main criteria are clarity of the specification and, because the SPST value range is relatively small, potential overlap in functionality with existing SPST registrations 🧐.
Value Name Reference
----- ---- ---------
1 Synchronization Client This document, Section 7
2 MSAS [TS183063]
3 SC Prime Input [TS183063]
4 SC Prime Output [TS183063]13.5. Contact Information for Registrations 📮
Here's the contact info for the registrations 💌:
Ray van Brandenburg (ray.vanbrandenburg@tno.nl)
Brassersplein 2
2612CT, Delft, The Netherlands14. Contributors 🫶
Big shoutout to the people who made substantial contributions to this document: Omar Niamut, Fabian Walraven, Ishan Vaishnavi, and Rufael Mekuria 👑. The authors also want to thank Aidan Williams, Colin Perkins, Magnus Westerlund, Roni Even, Peter Musgrave, Ali Begen, Qin Wu, and Rob Koenen for their review comments and contributions to the text 💖✨.
15. References 📚
15.1. Normative References 💜
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, RFC 3550, July 2003.
[RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and
Video Conferences with Minimal Control", STD 65, RFC 3551,
July 2003.
[RFC3611] Friedman, T., Caceres, R., and A. Clark, "RTP Control
Protocol Extended Reports (RTCP XR)", RFC 3611, November
2003.
[RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
Description Protocol", RFC 4566, July 2006.
[RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an
IANA Considerations Section in RFCs", BCP 26, RFC 5226,
May 2008.
[RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax
Specifications: ABNF", STD 68, RFC 5234, January 2008.
[RFC5905] Mills, D., Martin, J., Burbank, J., and W. Kasch, "Network
Time Protocol Version 4: Protocol and Algorithms
Specification", RFC 5905, June 2010.
[RFC7273] Williams, A., Gross, K., van Brandenburg, R., and H.
Stokking, "RTP Clock Source Signalling", RFC 7273, June
2014.15.2. Informative References ✨
okay these are the bonus reads, bestie 📚 they're not required to get the core protocol working, but they're super useful for the deeper context 🫶
[Boronat2009] Boronat, F., Lloret, J., and M. Garcia, "Multimedia group and inter-stream synchronization techniques: a comparative study", Elsevier Information Systems 34, Pages 108-131, March 2009, http://www.sciencedirect.com/science/article/pii/S0306437908000525. 📖✨
[IEEE1588-2008] IEEE, "1588-2008 - IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems", July 2008, http://standards.ieee.org/findstds/standard/1588-2008.html. ⏱️💜
[Ishibashi2006] Ishibashi, Y., Nagasaka, M., and N. Fujiyoshi, "Subjective assessment of fairness among users in multipoint communications", Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertainment technology, Article No. 69, June 2006, http://dl.acm.org/citation.cfm?id=1178905. 🎮🌈
[RFC7164] Gross, K. and R. Brandenburg, "RTP and Leap Seconds", RFC 7164, March 2014. 🗓️🦋
[RFC7201] Westerlund, M. and C. Perkins, "Options for Securing RTP Sessions", RFC 7201, April 2014. 🔐💖
[TS183063] ETSI, "Telecommunications and Internet converged Services and Protocols for Advanced Networking (TISPAN); IMS-based IPTV stage 3 specification", TS 183 063 v3.5.2, March 2011. 📺🍓
Authors' Addresses 🫶
Meet the squad who made this happen, bestie 💅✨
Ray van Brandenburg
TNO
Brassersplein 2
Delft 2612CT
The Netherlands
Phone: +31-88-866-7000
EMail: ray.vanbrandenburg@tno.nl
Hans Stokking
TNO
Brassersplein 2
Delft 2612CT
The Netherlands
Phone: +31-88-866-7000
EMail: hans.stokking@tno.nl
M. Oskar van Deventer
TNO
Brassersplein 2
Delft 2612CT
The Netherlands
Phone: +31-88-866-7000
EMail: oskar.vandeventer@tno.nl
Fernando Boronat
Universitat Politecnica de Valencia (UPV)
Valencia 46730
Spain
Phone: +34 962 849 341
EMail: fboronat@dcom.upv.es
Mario Montagud
Universitat Politecnica de Valencia (UPV)
Valencia 46730
Spain
Phone: +34 962 849 341
EMail: mamontor@posgrado.upv.es
Kevin Gross
AVA Networks
Phone: +1-303-447-0517
EMail: Kevin.Gross@AVAnw.comThat's a wrap, bestie!! 🌈🎀✨ Every single one of these people had a hand in making this spec real, so go give them some love 💜
📦 where this page comes from
- ietf: rfc7272 2014-06
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms