RFC 7728
RTP Stream Pause and Resume
RTP Stream Pause and Resume 💖
Okay bestie, this one is the RFC for pausing and resuming RTP streams, and it's giving real-time multimedia main character energy 🎀
Here's the title block, all in one place ✨
- RFC number: 7728 🦋
- Category: Standards Track, so this is an official Internet Standard 👑
- Date: February 2016 🌸
- Series: IETF, with ISSN 2070-1721
- Updates: RFC 5104 💅
- Obsoletes: nothing is listed in the title block
- Authors: B. Burman, A. Akram, R. Even, and M. Westerlund
- Affiliations: Ericsson (Burman, Akram, and Westerlund) and Huawei Technologies (Even) 🫶
Abstract ✨
Real-time multimedia apps are everywhere now, so it's really important to manage resource usage well, and users want more control over their calls too 😭 This doc explains how a receiver in a multimedia conversation can pause and resume incoming data from a sender, using real-time feedback messages with the Real-time Transport Protocol (RTP) for live data transport. It extends the Codec Control Message (CCM) RTP Control Protocol (RTCP) feedback package in two ways: it explicitly allows and describes specific uses of existing CCMs, and it adds a group of new real-time feedback messages for pausing and resuming RTP data streams. This document updates RFC 5104. 🔥
Status of This Memo
This is an Internet Standards Track document. 👑
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community, it got public review, and the Internet Engineering Steering Group (IESG) approved it for publication 🫶 More info on Internet Standards is in Section 2 of RFC 5741.
You can check the current status of this document, any errata, and how to send feedback at http://www.rfc-editor.org/info/rfc7728. 💜
Copyright Notice
Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved. 🌈
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they stand on the publication date. Please read these carefully, because they describe your rights and restrictions for this document. Code Components pulled out of this document must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come with no warranty, as described in the Simplified BSD License. ⭐
This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The people controlling the copyright in some of that material may not have given the IETF Trust permission to allow modifications outside the IETF Standards Process. Without an adequate license from those copyright holders, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process. The only exceptions are formatting it for publication as an RFC or translating it into languages other than English. 🙈
Table of Contents 🧚
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4
2. Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1. Abbreviations . . . . . . . . . . . . . . . . . . . . . . 5
2.2. Terminology . . . . . . . . . . . . . . . . . . . . . . . 6
2.3. Requirements Language . . . . . . . . . . . . . . . . . . 7
3. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.1. Point to Point . . . . . . . . . . . . . . . . . . . . . 8
3.2. RTP Mixer to Media Sender . . . . . . . . . . . . . . . . 9
3.3. RTP Mixer to Media Sender in Point to Multipoint . . . . 10
3.4. Media Receiver to RTP Mixer . . . . . . . . . . . . . . . 11
3.5. Media Receiver to Media Sender across RTP Mixer . . . . . 11
4. Design Considerations . . . . . . . . . . . . . . . . . . . . 12
4.1. Real-Time Nature . . . . . . . . . . . . . . . . . . . . 12
4.2. Message Direction . . . . . . . . . . . . . . . . . . . . 12
4.3. Apply to Individual Sources . . . . . . . . . . . . . . . 12
4.4. Consensus . . . . . . . . . . . . . . . . . . . . . . . . 13
4.5. Message Acknowledgments . . . . . . . . . . . . . . . . . 13
4.6. Request Retransmission . . . . . . . . . . . . . . . . . 14
4.7. Sequence Numbering . . . . . . . . . . . . . . . . . . . 14
4.8. Relation to Other Solutions . . . . . . . . . . . . . . . 14
5. Solution Overview . . . . . . . . . . . . . . . . . . . . . . 15
5.1. Expressing Capability . . . . . . . . . . . . . . . . . . 16
5.2. PauseID . . . . . . . . . . . . . . . . . . . . . . . . . 16
5.3. Requesting to Pause . . . . . . . . . . . . . . . . . . . 17
5.4. Media Sender Pausing . . . . . . . . . . . . . . . . . . 18
5.5. Requesting to Resume . . . . . . . . . . . . . . . . . . 19
5.6. TMMBR/TMMBN Considerations . . . . . . . . . . . . . . . 20
6. Participant States . . . . . . . . . . . . . . . . . . . . . 22
6.1. Playing State . . . . . . . . . . . . . . . . . . . . . . 22
6.2. Pausing State . . . . . . . . . . . . . . . . . . . . . . 22
6.3. Paused State . . . . . . . . . . . . . . . . . . . . . . 23
6.3.1. RTCP BYE Message . . . . . . . . . . . . . . . . . . 24
6.3.2. SSRC Time-Out . . . . . . . . . . . . . . . . . . . . 24
6.4. Local Paused State . . . . . . . . . . . . . . . . . . . 24
7. Message Format . . . . . . . . . . . . . . . . . . . . . . . 26
8. Message Details . . . . . . . . . . . . . . . . . . . . . . . 28
8.1. PAUSE . . . . . . . . . . . . . . . . . . . . . . . . . . 29
8.2. PAUSED . . . . . . . . . . . . . . . . . . . . . . . . . 30
8.3. RESUME . . . . . . . . . . . . . . . . . . . . . . . . . 31
8.4. REFUSED . . . . . . . . . . . . . . . . . . . . . . . . . 32
8.5. Transmission Rules . . . . . . . . . . . . . . . . . . . 32
9. Signaling . . . . . . . . . . . . . . . . . . . . . . . . . . 33
9.1. Offer/Answer Use . . . . . . . . . . . . . . . . . . . . 37
9.2. Declarative Use . . . . . . . . . . . . . . . . . . . . . 39
10. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 39
10.1. Offer/Answer . . . . . . . . . . . . . . . . . . . . . . 40
10.2. Point-to-Point Session . . . . . . . . . . . . . . . . . 41
10.3. Point to Multipoint Using Mixer . . . . . . . . . . . . 45
10.4. Point to Multipoint Using Translator . . . . . . . . . . 47
11. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 50
12. Security Considerations . . . . . . . . . . . . . . . . . . . 50
13. References . . . . . . . . . . . . . . . . . . . . . . . . . 52
13.1. Normative References . . . . . . . . . . . . . . . . . . 52
13.2. Informative References . . . . . . . . . . . . . . . . . 53
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 54
Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 551. Introduction 🦋
Real-time communication is pulling in more people every day, so more apps keep getting built, and multimedia conversation apps are a big example 💅 Multimedia conversation shows up in lots of forms, like peer-to-peer chat apps and multiparty video conferencing run by central media nodes such as RTP Mixers. 🌸
Multimedia conferencing can involve tons of participants, and each one has their own preferences for the session, not just at the start but during the call too 🙈 This document walks through several multimedia scenarios where a conferencing node or participant chooses to temporarily pause an incoming RTP [RFC3550] stream and resume it later when needed. The receiver doesn't have to end or deactivate the RTP session and start over by negotiating the session parameters again, for example with SIP [RFC3261] and the Session Description Protocol (SDP) [RFC4566] offer/answer [RFC3264]. ✨
Centralized nodes like RTP Mixers or Multipoint Control Units (MCUs) use logic based on voice activity, other measurements, or user input. They can cut down on resource use, both at the sender and in the network, by temporarily pausing RTP streams the RTP Mixer doesn't need right now 🫶 If there are more conference participants than the conference logic has chosen to show to receiving participants at once, some participant RTP streams sent to the RTP Mixer may not need to be forwarded to anyone else. Those streams could be temporarily paused. This gets especially useful when the media sources come in multiple encoding versions (Simulcast) [SDP-SIMULCAST], or with Multi-Session Transmission (MST) of scalable encoding such as Scalable Video Coding (SVC) [RFC6190]. Some of the defined encodings, or some combinations of scalable layers, aren't used or can't be used all the time 🧚 As an example, a centralized node may pause those unused RTP streams without being explicitly asked, maybe because network or processing resources are temporarily limited. It may then also send an explicit indication that the streams are paused. 💜
The set of RTP streams needed at any moment is super dynamic in these scenarios. Using the out-of-band signaling channel to pause, and even more importantly to resume, an RTP stream is hard because of the performance requirements 😭 So the pause and resume signaling should live in the media plane and go directly between the nodes involved. When RTP [RFC3550] is used for media transport, using "Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)" [RFC4585] seems like the right fit. No RTCP feedback message that exists right now explicitly supports pausing and resuming an incoming RTP stream. Since this affects how packets are generated and might even let the encoding process pause, the functionality seems to match Codec Control Messages (CCMs) in the RTP Audio-Visual Profile with Feedback (AVPF) [RFC5104]. This document defines the solution as a CCM extension. 🔥
The Temporary Maximum Media Bitrate Request (TMMBR) message of CCM is what video conferencing systems use for flow control. It would be great to use that method with a bitrate value of zero to pause, whenever possible 💖 This specification updates RFC 5104 by adding the new pause and resume semantics to the TMMBR and Temporary Maximum Media Bitrate Notification (TMMBN) messages. 👑
2. Definitions 🌈
(Part 1 ends here, and the rest of the spec is coming up next, so stay tuned!) ✨
2.1. Abbreviations ✨
Okay bestie, let's get through this acronym soup together 🍓 Here's the full list, copied exactly so you can screenshot it:
AVPF: Audio-Visual Profile with Feedback (RFC 4585)
CCM: Codec Control Message (RFC 5104)
CNAME: Canonical Name (RTCP Source Description)
CSRC: Contributing Source (RTP)
FCI: Feedback Control Information (AVPF)
FIR: Full Intra Refresh (CCM)
FMT: Feedback Message Type (AVPF)
MCU: Multipoint Control Unit
MTU: Maximum Transfer Unit
PT: Payload Type (RTP)
RTP: Real-time Transport Protocol (RFC 3550)
RTCP: RTP Control Protocol (RFC 3550)
RTCP RR: RTCP Receiver Report
RTCP SR: RTCP Sender Report
SDP: Session Description Protocol (RFC 4566)
SIP: Session Initiation Protocol (RFC 3261)
SSRC: Synchronization Source (RTP)
SVC: Scalable Video Coding
TMMBR: Temporary Maximum Media Bitrate Request (CCM)
TMMBN: Temporary Maximum Media Bitrate Notification (CCM)
UA: User Agent (SIP)
UDP: User Datagram Protocol (RFC 768)2.2. Terminology 💅
On top of these, the definitions from RTP [RFC3550], AVPF [RFC4585], CCM [RFC5104], and RTP Taxonomy [RFC7656] also apply in this document. 🫶
CCM [RFC5104] sorted RTCP feedback messages into four types: Request, Command, Indication, and Notification. This document puts PAUSE and RESUME in the Request category, PAUSED as an Indication, and REFUSED as a Notification. Let's break down each one:
- PAUSE: Request from an RTP stream receiver to pause a stream 🙈
- RESUME: Request from an RTP stream receiver to resume a paused stream ✨
- PAUSED: Indication from an RTP stream sender that a stream is paused 🦋
- REFUSED: Notification from an RTP stream sender that a PAUSE or RESUME request will not be honored 💀
Now the people and parts of the picture:
- Mixer: The intermediate RTP node that receives an RTP stream from different endpoints, combines them to make one RTP stream, and forwards them to destinations, in the sense described for Topo-Mixer in "RTP Topologies" [RFC7667]. Main character of centralized conferencing 👑
- Participant: A member that is part of an RTP session, acting as the receiver, sender, or both. 🧚
- Paused sender: An RTP stream sender that has stopped its transmission, i.e., no other participant receives its RTP transmission, based on having received either a PAUSE request, defined in this specification, or a local decision. 🌸
- Pausing receiver: An RTP stream receiver that sends a PAUSE request, defined in this specification, to another participant(s). 💜
- Stream: Used as a short term for RTP stream, unless otherwise noted. 🎀
- Stream receiver: Short for RTP stream receiver; the RTP entity responsible for receiving an RTP stream, usually a Media Depacketizer. 🍓
- Stream sender: Short for RTP stream sender; the RTP entity responsible for creating an RTP stream, usually a Media Packetizer. 🌈
2.3. Requirements Language 📜
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. Take note of these ones, they're the rules-of-the-road words 🔥
3. Use Cases ✨
This section is all about the main use cases for RTP stream pause and resume. Let's go! 🦋
The RTCWEB WG's use case and requirements document [RFC7478] defines the following API requirements in Appendix A, which is also used by the W3C WebRTC WG:
A8 The web API must provide means for the web application to mute/
unmute a stream or stream component(s). When a stream is sent to
a peer, mute status must be preserved in the stream received by
the peer.
A9 The web API must provide means for the web application to cease
the sending of a stream to a peer.So this document gives you ways to save transport usage by stopping the sending of muted streams, and starting them again when they're unmuted 🔇➡️🔊 Here, it is assumed that "mute" above can be taken to apply also to media other than audio. At the time this specification was published, the RTCWEB WG hadn't specified any pause/resume functionality, so this doc is filling that gap 💅
3.1. Point to Point 🎀
This is the most basic use case, with an RTP session containing two endpoints. Each endpoint sends one or more streams.
+---+ +---+
| A |<------->| B |
+---+ +---+
Figure 1: Point to PointHere, RTP stream pause is used to temporarily halt delivery of streams that the sender provides but the receiver doesn't currently use. Think minimized apps where the video isn't actually shown on any display, or it's not being used in any other way, like getting recorded 📼 Since there's only one receiver of the stream, pausing or resuming it doesn't affect anyone except the sender and that single receiver. Very contained, very iconic 💖
3.2. RTP Mixer to Media Sender 🌸
One of the most commonly used topologies in centralized conferencing is based on the RTP Mixer [RFC7667]. The big reason is that it gives a really consistent view of the RTP session to each participant ✨ The Mixer originates its own streams, identified by distinct SSRC values, and any RTP streams sent to the participants are sent using those SSRC values. If the Mixer wants to identify the underlying media sources for its conceptual streams, it can identify them using CSRC. The stream the Mixer provides can be an actual mix of multiple media sources, but it might also be switching received streams as described in Sections 3.6 - 3.8 of "RTP Topologies" [RFC7667].
+---+ +-----------+ +---+
| A |<---->| |<---->| B |
+---+ | | +---+
| Mixer |
+---+ | | +---+
| C |<---->| |<---->| D |
+---+ +-----------+ +---+
Figure 2: RTP Mixer in Unicast OnlyWhich streams from clients B, C, and D get delivered to a given receiver, A, can depend on a few things:
- The RTP Mixer's own logic and measurements, such as voice activity on the incoming audio streams. 🎙️
- The number of sent media sources exceed what is reasonable to present simultaneously at any given receiver. 😵💫
- A human controlling the conference that determines how the media should be mixed. This would be more common in lecture or similar applications where regular listeners may be prevented from breaking into the session unless approved by the moderator. 👩🏫
- The streams may also be part of a Simulcast [SDP-SIMULCAST] or scalable encoded (for Multi-Session Transmission) [RFC6190], thus providing multiple versions that can be delivered by the RTP stream sender. 🌈
These examples show there are lots of reasons a particular stream wouldn't be in use right now, but it still has to be available at very short notice if any dynamic event makes the Mixer pick a different stream 🏃♀️💨
So it would be really beneficial if the Mixer could request the RTP stream sender to pause a particular stream. The Mixer also needs to be able to request that the RTP stream sender resume delivery with minimal delay. 🔥
In some cases, especially when the Mixer sends multiple RTP streams per receiving client, the Mixer may want to pause some of its sent streams, even without being explicitly asked to by the receiving client. Those situations can be caused by a temporary lack of available Mixer network or processing resources, for example. An RTP stream receiver that no longer receives an RTP stream could read that as an error and try to re-establish the stream 😭 That action would likely be undesirable if the stream was actually paused on purpose by the Mixer. Those unwanted receiver actions can be avoided if the Mixer is able to explicitly indicate that an RTP stream is deliberately paused. Smart, right? 🧠
Just like point to point (Section 3.1), there's only a single receiver of the stream, the RTP Mixer, and pausing or resuming a stream doesn't affect anyone except the sender and that single receiver. 🫶
3.3. RTP Mixer to Media Sender in Point to Multipoint 🦋
This use case is similar to the previous section, but this time the RTP Mixer is involved in three domains that need to be kept separate: the Multicast Network (including participants A and C), participant B, and participant D. The difference from above is that A and C share a multicast domain, which is shown below 👀
+-----+
+---+ / \ +-----------+ +---+
| A |<---/ \ | |<---->| B |
+---+ / Multi- \ | | +---+
+ cast +->| Mixer |
+---+ \ Network / | | +---+
| C |<---\ / | |<---->| D |
+---+ \ / +-----------+ +---+
+-----+
Figure 3: RTP Mixer in Point to MultipointIf the RTP Mixer pauses a stream from A, it won't just pause the stream towards itself. It will also stop the stream from arriving at C, which C is heavily impacted by, might not approve of, and should thus have a say on. Tea 🍵
If the Mixer resumes a paused stream from A, it will be resumed towards C as well. In this case, if C isn't interested, it can simply ignore the stream, and it's not impacted as much as above. ✨
In this use case there are several receivers of a stream, so the Mixer must take special care not to pause a stream that is still wanted by some receivers. Major responsibility energy 👑
3.4. Media Receiver to RTP Mixer 🎀
In this use case, the direction of the pause request is the opposite of the two previous use cases. An endpoint in Figure 2 could potentially request to pause the delivery of a given stream. Possible reasons include the ones from the point-to-point case (Section 3.1) above. 💅
When the RTP Mixer is only connected to individual unicast paths, the use case and any considerations are identical to the point-to-point use case. 🧚
However, when the endpoint requesting stream pause is connected to the RTP Mixer through a multicast network, such as A or C in Figure 3, the use case instead becomes identical to the one in Section 3.3, just with the direction of the streams and pause/resume requests reversed. Plot twist 🌀💖
3.5. Media Receiver to Media Sender across RTP Mixer ✨
okay so bestie, this one's a little plot twist 🫣 An endpoint, like A in Figure 2, could technically try to pause one of B's streams by sending a pause request over any of the SSRCs the Mixer uses, pointing at the CSRC that identifies the stream 💅 But the authors weren't vibing with that idea, and honestly? Same. Here's why:
- The Mixer might not include CSRC in its stream indications. 💀
- An endpoint can't rely on the CSRC to correctly identify the stream to be paused when the delivered media is some type of mix. A more elaborate stream identification solution is needed to support this in the general case. 🙈
- The endpoint cannot determine if a given stream is still needed by the RTP Mixer to deliver to another session participant. 😭
So yeah, due to all of that, this use case is excluded from further consideration. Not me saying "no" to the whole idea, just this path 🫶
4. Design Considerations 🎀
This section describes the requirements that this specification needs to meet. Let's get into it 🦋
4.1. Real-Time Nature ⏰
okay so the first section (Section 1) of this spec explains some possible reasons why a receiver might pause an RTP sender 🌸 Pausing and resuming is totally time dependent. A receiver might pause a stream for a while, and then want the sender to start up again later ✨ That means the pause and resume messages have to get to the sender in a timely way, or they won't be purposeful at all 😭
Pausing is arguably less time critical than resuming, since it's mainly about saving resources 💜 Resuming, though? Highkey huge for the end-user's experience, because it's what decides whether the media they expect shows up more or less instantly 🔥 It's also really nice for a receiver to quickly learn that an RTP stream is intentionally paused on the RTP sender's own behalf 🧚
4.2. Message Direction 📬
It's the responsibility of an RTP stream receiver that wants to pause or resume a stream from the sender(s) to transmit PAUSE and RESUME messages 💖 An RTP stream sender that wants to pause itself can often just do it, but sometimes that messes with the receiver, and then an explicit indication that the stream is paused might help 🌈 Any indication that an RTP stream is paused is the sender's responsibility, and the stream receiver may not even need it ✨
4.3. Apply to Individual Sources 🎯
PAUSE and RESUME messages apply to single RTP streams, identified by their SSRC. So the receiver targets the sender's SSRC in its PAUSE and RESUME requests 🦋 If a paused sender starts sending with a new SSRC, receivers will need to send a new PAUSE request to pause it again 🙈 And PAUSED indications refer to just one of the sender's own paused SSRC 💅
4.4. Consensus 🫂
An RTP stream sender should not pause an SSRC that some receiver still wishes to receive. 🛑
The reason? In RTP topologies where a stream is shared between multiple receivers, one receiver on that shared network must not single-handedly cause the stream to be paused without letting all the other receivers voice their opinions 👑 Shared networks can be multicast, a mesh with a joint RTP session, or a transport Translator-based network, for example 🌐 So a newly joining receiver first needs to learn that paused streams exist, and then it should be able to resume any paused stream 🎀 A newly joining receiver can be detected through an RTCP Receiver Report containing both a new SSRC and a CNAME that doesn't already occur in the session 👀 Any single receiver that wants to resume a stream should also cause it to be resumed ✨
There's one important exception 🚨 When the RTP stream sender knows about conditions that make it desirable, or even necessary, to pause the RTP stream on its own behalf, without being explicitly asked, that local consideration in the sender takes precedence over receiver wishes to receive the stream 💅
4.5. Message Acknowledgments 💌
RTP and RTCP don't guarantee reliable data transmission. They use whatever assurance the lower-layer transport protocol can provide 🤷♀️ But that's commonly UDP, which provides no reliability guarantees 😭 So it's possible that a PAUSE and/or RESUME message sent from an RTP endpoint never reaches its destination, i.e., the targeted RTP stream sender 💀
When PAUSE or RESUME reaches the RTP stream sender and actually works (an active sender pauses, or a resuming sender has media data to transmit), it's immediately visible from the arrival or non-arrival of RTP packets for that stream 👀 So no explicit acknowledgments are required in this case 🫶
Sometimes, though, a PAUSE or RESUME message reaches the sender but it can't do anything with it because of some local consideration, like not having any data to transmit 🙃 In that error condition, a negative acknowledgment may be needed to avoid unnecessary retransmission of requests (Section 4.6) 🔁
4.6. Request Retransmission 🔁
When the stream isn't affected the way a PAUSE or RESUME request expected, the request may have been lost, and the sender of the request will need to retransmit it 🥺 The retransmission should take the round-trip time into account, and it will also need to take the normal RTCP bandwidth and timing rules for the RTP session into account when scheduling feedback retransmission 📅
For resume requests or unsolicited paused indications, which are more time critical, the best performance may come from repeating the message as often as possible 💨 Keep going until enough copies have been sent to reach a high probability of delivery, or until there's an explicit indication that the message was delivered 🎉 For resume requests, that explicit indication can be delivery of the RTP stream being asked to resume ✨
4.7. Sequence Numbering 🔢
A PAUSE request message needs a sequence number to tell retransmissions apart from new requests 🧠 A retransmission keeps the sequence number unchanged, while the number goes up every time a new PAUSE request is sent that isn't a retransmission of a previous one 📈
Since RESUME always takes precedence over PAUSE and is even allowed to avoid pausing a stream, strict ordering of PAUSE and RESUME is needed 🙌 So RESUME has to share sequence number space with PAUSE and implicitly reference which PAUSE it refers to 💜 For the same reasons, the explicit PAUSED indication also has to share sequence number space with PAUSE and RESUME 💅
4.8. Relation to Other Solutions 🔍
A performance comparison between SIP/SDP and RTCP signaling technologies was made and included in draft versions of this spec 📊 Using SIP and SDP to carry pause and resume information means they have to traverse the entire signaling path to reach the signaling destination (either the remote endpoint or the entity controlling the RTP Mixer), across any signaling proxies that might also have to process the SDP content to see if they're supposed to act on it 🚧 The bandwidth needed for a SIP/SDP-based signaling solution is in the order of at least 10 times more than an RTCP-based one 😱 Especially for a UA on mobile wireless access, that risks delays that are too long (Section 4.1) for a good user experience, and the bandwidth cost may also be considered infeasible compared to RTCP 💸 RTCP data sent through the media path, which is likely shorter (fewer intermediate nodes) than the signaling path, may still have to traverse a few intermediate nodes anyway 🛤️ The processing and buffering needed in intermediate nodes to forward those RTCP messages is, however, believed to be significantly less than for intermediate nodes in the signaling path 🙌 Based on all that, RTCP is chosen as the signaling protocol for the pause and resume functionality 🦋
5. Solution Overview 🗺️
The proposed solution implements pause and resume functionality by sending AVPF RTCP feedback messages from any RTP session participant that wants to pause or resume a stream aimed at the stream sender, identified by the sender SSRC 💖
This solution reuses CCM TMMBR and TMMBN [RFC5104] to the extent possible, and defines a small set of new RTCP feedback messages where new semantics are needed ✨
A single feedback message specification is used to implement the new messages. The message consists of a number of Feedback Control Information (FCI) blocks, where each block can be a PAUSE request, a RESUME request, a PAUSED indication, a REFUSED notification, or an extension to this specification 🧩 This structure lets a single feedback message handle pause functionality on a number of streams at once 🙌
The PAUSED functionality is also defined so that the RTP stream sender can use it standalone to indicate a local decision to pause, and it can tell any receiver that halting media delivery is deliberate, plus which RTP packet was the last one transmitted 🫶
Special considerations that apply when using TMMBR/TMMBN for pause and resume purposes are described in Section 5.6. This specification applies to both the new messages defined here and their TMMBR/TMMBN counterparts, except when explicitly stated otherwise 🌸 An obvious exception is any reference to message parameters that are only available in the messages defined here. For example, any reference to PAUSE in the text below is equally applicable to TMMBR 0, and any reference to PAUSED is equally applicable to TMMBN 0. Therefore, and for brevity, TMMBR/TMMBN won't be mentioned in the text unless there's a specific reason to 🙈
This section is meant to be explanatory, so it intentionally contains no mandatory statements. Those can be found in other parts of this specification 📖
5.1. Expressing Capability 🎀
An endpoint can use an extension to CCM SDP signaling to declare capability to understand the messages defined in this specification 📣 Capability to understand only a subset of messages is possible, to support partial implementation, which is specifically believed to be feasible for the 'RTP Mixer to Media Sender' use case (Section 3.2) 💅 In that use case, only the RTP Mixer has the capability to request the media sender to pause or resume. Consequently, in that same use case, only the media sender has the capability to pause and resume its sent streams based on requests from the RTP Mixer 🧚 Allowing partial implementation of this specification is not believed to hamper interoperability, as long as the subsets are well defined and describe a consistent functionality, including a description of how a more capable implementation must perform fallback 🫶
For the case when TMMBR/TMMBN are used for pause and resume purposes, it's possible to explicitly express joint support for TMMBR and TMMBN, but not for TMMBN only 🦋
5.2. PauseID 🎀
Okay bestie, every message in this spec (see Section 8) carries a PauseID, which is how we satisfy the sequence numbering design goal from Section 4.7 🔢✨ The PauseID is scoped to the targeted RTP stream (its SSRC), so it's a property of that stream and not just a counter for individual messages. It numbers an entire "pause and resume operation" for the stream 💅 That means the PauseID usually stays the same across all the related messages in one operation. The PauseID value in use during an operation is called the current PauseID 🦋
A brand new "pause and resume operation" starts when the RTP stream sender resumes a stream that was paused. At that point, the current PauseID goes up by one, using modulo arithmetic 🌈 Later on we sometimes need to talk about PauseID values that were already used as the current PauseID, and those are called past PauseIDs 🕰️ Because of modulo arithmetic, a past PauseID can actually have a bigger number than the current one. That's the wrap-around magic, periodt 💜
So we need a clear way to tell "past" apart from "future" 👀 Half of the whole PauseID range counts as past, and a quarter of the range counts as future. The last quarter is deliberately left undefined on purpose 🧚
5.3. Requesting to Pause 💖
An RTP stream receiver can send a PAUSE request whenever it wants, as long as it follows the AVPF timing rules ⏰
The PAUSE request carries the current PauseID (Section 5.2) 🎟️
When a sender that isn't paused gets a PAUSE request, it keeps sending the RTP stream 🌊 It then waits for a bit (a hold-off period) so that the other receivers in the same RTP session who saw the PAUSE can push back by sending a RESUME (Section 5.5) for the same stream with the same current PauseID as the PAUSE they're disagreeing with 🙅♀️ If a disagreeing RESUME shows up during that hold-off period, the pause just doesn't happen 🚫 In point-to-point setups the hold-off period can be set to zero 🍓 A hold-off of zero also fits when using TMMBR 0, which matches what that message is meant to do 💅
If the sender gets more PAUSE requests with the current PauseID while it's waiting like this, it ignores the extras 🙈
What if the PAUSE gets lost before it reaches the sender? Totally fine, the receiver will notice because it keeps getting the RTP stream, and it won't see a PAUSED indication (Section 5.4) for that stream either 🕵️♀️ The same thing can happen when the sender got a disagreeing RESUME from receiver A for a PAUSE sent by sender B. The difference is that B never got that RESUME from A, so B might just think its PAUSE got lost 😭 In both situations, the PAUSE can be sent again using the same current PauseID 🔁 If TMMBR 0 is in use, the request MAY be retransmitted when the requester doesn't get a TMMBN 0 confirmation 🫶
If a pending pause gets aborted because of a disagreeing RESUME, that pause and resume operation is over for that PauseID 🏁 The current PauseID gets updated, so any new PAUSE has to use the new current PauseID to actually work ✨
If a sender gets a PAUSE that doesn't use the current PauseID, it lets the receiver know by sending a REFUSED notification that carries the current PauseID 🙋♀️
This happens a lot when a brand-new receiver joins the session and wants to PAUSE a stream but doesn't know which PauseID to use yet 👶 The REFUSED gives it everything it needs to build a valid PAUSE. That extra round trip isn't a problem, because pausing a stream isn't time critical (Section 4.1) ⏳
Sometimes there are local reasons why the sender can't pause the stream, and then it can answer with a REFUSED 🛑 In that case, if the PauseID that was used would otherwise have worked, the REFUSED carries the same current PauseID as the PAUSE request 💜 Heads up on TMMBR 0, though: that request can't be refused (TMMBN > 0) because of the existing rule in Section 4.2.2.2 of [RFC5104] that TMMBN has to contain the current bounding set. A TMMBR 0 is always the most restrictive point in any bounding set, no matter what the bounding set overhead value is 😤
If the sender gets several identical PAUSE requests for a stream that it already answered at least once with REFUSED, and the thing causing the REFUSED is still true, the extra REFUSED notifications should be sent using regular RTCP timing 🕐 One REFUSED can answer a whole batch of identical PAUSE requests 📦
5.4. Media Sender Pausing 🌙
A sender can choose to pause its stream at any time ⏸️ This can happen because it got a PAUSE, or because of something local on the sender's side 🏠 When it pauses, it sends a PAUSED indication with the current PauseID 💌 If a PAUSED is sent without a PAUSE having been received first, the current PauseID in that unsolicited PAUSED is incremented compared to the last PAUSED that was sent 📈 The sender also keeps including the PAUSED indication in the next two regular RTCP reports, as long as the pause is still in effect 🔁
Nobody replies to a PAUSED indication. It's just a clear signal that the stream is paused 🧘♀️ That helps receivers quickly understand that the transmission was intentionally and temporarily suspended, so they don't need to do any specific fixing 🙌
The sender might want to think about exactly when it pauses, like finishing a media unit or a forward error correction block before it stops 🎬
The PAUSED indication also includes the RTP extended highest sequence number at the moment the pause took effect 🔢 That gives receivers firsthand info so they can tell whether they lost any packets right before the pause or right when the stream resumes 🕵️ Receivers can then safely take the pause into account in things like retransmission or congestion control algorithms 🧠✨
If the sender gets PAUSE requests with the current PauseID while the stream is already paused, it ignores them 🙈
While the stream is paused, the PAUSED indication MAY be sent together with any regular RTCP Sender Report (SR) or Receiver Report (RR) 📨 Doing it this way lets receivers join while the stream is paused and instantly learn that there's a paused stream, what the last sent extended RTP sequence number was, and what the current PauseID is 🪄 With that info they can build valid PAUSE and RESUME requests later on 🫶
When the sender finds out a new endpoint joined the session (for example, a new SSRC and a CNAME it hasn't seen before in the session), it should send PAUSED indications for all its paused streams as soon as it can ⚡ It should also keep including PAUSED indications in at least two regular RTCP reports after that 💖
5.5. Requesting to Resume 🌸
An RTP stream receiver can ask the sender to resume a stream with a RESUME request at any time, as long as it follows the AVPF timing rules ⏰ For a RESUME to work, the receiver must include the current PauseID in it 🎫
A paused sender that gets a RESUME with the current PauseID resumes the stream at the earliest chance it gets 🏃♀️ If it gets RESUME requests for a stream that isn't paused, no action is needed and it can just ignore them 🙃
Sometimes the sender has local reasons it can't resume right then, like the media device not being ready yet. In that case it can answer with a REFUSED that contains the current PauseID ⏳ When a receiver gets a REFUSED whose current PauseID matches the one in its RESUME, it should hold off on sending more RESUME requests for a reasonable amount of time so the condition can clear 🌷 A sender that has sent a REFUSED SHOULD resume the stream through local considerations (see below) once the condition behind that REFUSED is no longer true 🎉
If the sender gets several identical RESUME requests for a stream that it already answered at least once with REFUSED, and the condition causing it is still there, the extra REFUSED notifications should be sent using regular RTCP timing 🕐 One REFUSED can respond to a whole bunch of identical RESUME requests 📦
A paused sender can use local considerations and resume the paused stream whenever it wants 🌈 But if TMMBR 0 was used to pause the stream, resumption is blocked by the protocol itself, even if the sender would like to resume for local reasons 🔒 If TMMBR/TMMBN signaling is used, the stream was paused for local reasons (Section 5.4), and the sender owns the TMMBN bounding set, then the stream can be resumed for local reasons 💅
When resuming a paused stream, especially for media that relies on temporal redundancy between samples (like video), it might not be a good idea to use that temporal dependency between samples from before the pause and samples at the moment of resuming 🎞️ If the sender does use temporal dependency on samples from before the pause, the receiver has to have saved those samples from before the pause to keep decoding correctly after resuming 💾 Whether to use that dependency is up to the sender 🫡 If the sender doesn't use it, the first encoded sample after the pause won't depend on anything from before the pause (for video, that might be a so-called intra picture) 🖼️ If the sender does use that dependency when resuming, but the receiver didn't save any samples from before the pause, the receiver can send a FIR request [RFC5104] to explicitly ask for a sample with no temporal dependency (for video, a so-called intra picture), even at the same time as it sends the RESUME 🆘
5.6. TMMBR/TMMBN Considerations ✨
Okay so, as we said earlier, TMMBR/TMMBN can be used to give you pause and resume vibes for the point-to-point case 💖. If the topology is not point to point, TMMBR/TMMBN cannot safely be used for pause or resume 🙈. This use is mainly expected for interworking with implementations that don't support the messages defined in this specification (Section 8) but still use TMMBR/TMMBN to get a similar effect 🫶.
Here's the quick tea on what TMMBR/TMMBN actually gives you 🍓:
TMMBR 0: basically PAUSE, minus the hold-off period where you wait for RESUME before the RTP stream gets paused 🦋.TMMBR > 0: basically RESUME, but only when the RTP stream was paused withTMMBR 0first. Since there's only a single RTP stream receiver, the sender doesn't have to wait until after sendingTMMBN > 0to resume, and it doesn't have to apply the hold-off period from [RFC5104] before raising the bitrate from zero 💅. The bitrate used when resuming after aTMMBR 0pause is either based on known limitations or on starting the stream at the configured maximum for the stream or session, for example, given by the "b=" line in SDP 🌈.TMMBN 0: corresponds to PAUSED when the RTP stream was paused withTMMBR 0, but it can also be sent unsolicited, just like PAUSED ✨. An unsolicited pause based on local sender considerations uses the RTP stream's own SSRC as the TMMBR restriction owner in the TMMBN message bounding set. It also corresponds to a REFUSED notification when a stream is requested to be resumed withTMMBR > 0, which makes the stream sender the owner of the bounding set in the TMMBN message 👑.TMMBN > 0: cannot be used as a REFUSED notification when a stream is requested to be paused withTMMBR 0, for the reasons stated in Section 5.3 🙅.
6. Participant States 🌸
This document introduces three new states for a stream in an RTP sender, as shown in the figure and subsections below. Any references to PAUSE, PAUSED, RESUME, and REFUSED in this section SHALL be taken to apply to the extent possible also when TMMBR/TMMBN are used (Section 5.6) for this functionality 🧚.
+------------------------------------------------------+
| Received RESUME |
v |
+---------+ Received PAUSE +---------+ Hold-off period +--------+
| Playing |---------------->| Pausing |---------------->| Paused |
| |<----------------| | | |
+---------+ Received RESUME +---------+ +--------+
^ | | PAUSE decision |
| | v |
| | PAUSE decision +---------+ PAUSE decision |
| +------------------>| Local |<--------------------+
+-------------------------| Paused |
RESUME decision +---------+Figure 4: RTP Pause States in Sender
6.1. Playing State 🎀
This state isn't new. It's the normal media sending state from [RFC3550]. When entering the state, the current PauseID MUST be incremented by one in modulo arithmetic 🔢. The RTP sequence number for the first packet sent after a pause SHALL be incremented by one compared to the highest RTP sequence number sent before the pause. The first RTP timestamp for the first packet sent after a pause SHOULD be set according to capture times at the source, which means the RTP timestamp difference compared to before the pause reflects the time the RTP stream was paused ⏸️.
6.2. Pausing State 🦋
In this state, the RTP stream sender has received at least one PAUSE message for the stream in question. The RTP stream sender SHALL wait during a hold-off period for the possible reception of RESUME messages for the RTP stream being paused before it actually pauses RTP stream transmission 🤞. The hold-off period to wait SHALL be long enough to allow another RTP stream receiver to respond to the PAUSE with a RESUME, if it decides it would not like to see the stream paused. This hold-off period is determined by the formula:
2 * RTT + T_dither_max,where RTT is the longest round trip known to the RTP stream sender and T_dither_max is defined in Section 3.4 of [RFC4585]. The hold-off period MAY be set to 0 by some signaling (Section 9) means when it can be determined that there is only a single receiver, for example, in point to point or some unicast situations 💫.
If the RTP stream sender has set the hold-off period to 0 and receives information that it was an incorrect decision, and that there are in fact several receivers of the stream, it MUST change the hold-off period to be based on the above formula instead 🙃.
An RTP stream sender SHOULD use the following criteria to determine if there is only a single receiver, unless it has explicit and more reliable information:
- Observing only a single CNAME across all received SSRCs (CNAMEs for received CSRCs are insignificant), or
- If RTCP reporting groups [MULTI-STREAM-OPT] is used, observing only a single, endpoint external RTCP reporting group.
6.3. Paused State 🌙
An RTP stream is in paused state when the sender pauses its transmission after receiving at least one PAUSE message and the hold-off period has passed without receiving any RESUME message for that stream. Pausing transmission SHOULD only be done when reaching an appropriate place to pause in the stream, like a media boundary that avoids a media receiver triggering repair or concealment actions 🧸.
When entering the state, the RTP stream sender SHALL send a PAUSED indication to all known RTP stream receivers, and SHALL also repeat PAUSED in the next two regular RTCP reports, as long as it is still in paused state 📣.
Pausing an RTP stream MUST NOT affect the sending of RTP keepalive [RFC6263][RFC5245] applicable to that RTP stream. Keepalives keep going, no drama 🫶.
The following subsections discuss some potential issues when an RTP sender goes into paused state. These conditions are also valid if an RTP Translator is used in the communication. When an RTP Mixer implementing this specification is involved between the participants (which forwards the stream by marking the RTP data with its own SSRC), it SHALL be a responsibility of the Mixer to control sending PAUSE and RESUME requests to the sender. The conditions below also apply to the sender and receiver parts of the RTP Mixer, respectively 🎛️.
6.3.1. RTCP BYE Message 👋
When a participant leaves the RTP session, it sends an RTCP BYE message. In addition to the semantics described in Sections 6.3.4 and 6.3.7 of RTP [RFC3550], the following two conditions MUST also be considered when an RTP participant sends an RTCP BYE message:
- If a paused sender sends an RTCP BYE message, receivers observing this SHALL NOT send further PAUSE or RESUME requests to it 🚫.
- Since a sender pauses its transmission on receiving PAUSE requests from any receiver in a session, the sender MUST keep a record of which receiver caused the RTP stream to pause. If that receiver sends an RTCP BYE message observed by the sender, the sender SHALL resume the RTP stream. No receivers that were in the RTP session when the stream was paused objected to the pause. But if there were receivers added to the session during the pause that the sender hadn't detected yet, those may not know the paused stream exists, either because no PAUSED was sent for it or because those receivers don't support PAUSED. Resuming the stream when the pausing party leaves the RTP session lets those potentially undetected receivers learn that the stream exists 🕵️♀️.
6.3.2. SSRC Time-Out ⏰
Section 6.3.5 in RTP [RFC3550] describes the SSRC time-out of an RTP participant. Every RTP participant maintains a sender and receiver list in a session. If a participant does not get any RTP or RTCP packets from some other participant for the last five RTCP reporting intervals, it removes that participant from the receiver list. Any streams that were paused by that removed participant SSRC SHALL be resumed 🌷.
6.4. Local Paused State 🍑
This state can be entered at any time, based on a local decision from the RTP stream sender. Pausing transmission SHOULD only be done when reaching an appropriate place to pause in the stream, like a media boundary that avoids a media receiver triggering repair or concealment actions 🎯.
As with paused state (Section 6.3), the RTP stream sender SHALL send a PAUSED indication to all known RTP stream receivers when entering the state, unless the stream was already in paused state (Section 6.3). Such PAUSED indication SHALL be repeated a sufficient number of times to reach a high probability that the message is correctly delivered, and the repetition stops whenever the sender leaves the state 🔁.
When using TMMBN 0 as a PAUSED indication and when already in paused state, the actions when entering local paused state depend on the bounding set overhead value in the received TMMBR 0 that caused the paused state, and the bounding set overhead value used in (the RTP stream sender's own) TMMBN 0 💅:
TMMBN 0 overhead <= TMMBR 0 overhead: The RTP stream sender SHALL NOT send any new TMMBN 0 replacing that active (more restrictive) bounding set, even if entering local paused state 🛑.TMMBN 0 overhead > TMMBR 0 overhead: The RTP stream sender SHALL send TMMBN 0 with itself in the TMMBN bounding set when entering local paused state 📩.
The case above, when using TMMBN 0 as a PAUSED indication, being in local paused state, and having received a TMMBR 0 with a bounding set overhead value greater than the value the RTP stream sender would itself use in a TMMBN 0, needs further consideration. For clarity, it's referred to as "restricted local paused state" from here on 🧐.
As indicated in Figure 4, local paused state has higher precedence than paused state (Section 6.3), and RESUME messages alone cannot resume a paused RTP stream as long as the local decision still applies. An RTP stream sender in local paused state is responsible for leaving the state whenever the conditions that caused the decision to enter the state no longer apply 🌤️.
If the RTP stream sender is in restricted local paused state, it cannot leave that state until the TMMBR 0 limit causing the state is removed by a TMMBR > 0 (RESUME). If the RTP stream sender then needs to stay in local paused state due to local considerations, it MAY continue pausing the RTP stream by entering local paused state, and it MUST then act accordingly, including sending a TMMBN 0 with itself in the bounding set 🙌.
Pausing an RTP stream MUST NOT affect the sending of RTP keepalive [RFC6263][RFC5245] applicable to that RTP stream 💜.
When leaving the local paused state, the stream state SHALL become Playing, regardless of whether there were any RTP stream receivers that sent PAUSE for that stream during the local paused state, effectively clearing the RTP stream sender's memory for that stream 🧹✨.
7. Message Format ✨
okay so Section 6 of AVPF [RFC4585] gives us three flavours of low-delay RTCP feedback: transport-layer, payload-specific, and application-layer. this doc adds a brand-new transport-layer feedback message 💅 and it comes in four subtypes: a PAUSE request, a RESUME request, a PAUSED indication, or a REFUSED notification 🦋
transport-layer feedback messages are the ones with RTCP payload type RTPFB (205), as defined by AVPF [RFC4585]. this transport-layer message, holding one or more of those subtyped messages, is called the PAUSE-RESUME message from here on. the specific FCI format gets picked out by a Feedback Message Type (FMT) value in the common packet header for the feedback message, which is defined in Section 6.1 of AVPF [RFC4585]. the PAUSE-RESUME transport-layer feedback message FCI is identified by FMT value = 9 🎀
the Common Packet Format for feedback messages from AVPF [RFC4585] looks like this:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P| FMT | PT | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC of packet sender |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC of media source |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: Feedback Control Information (FCI) :
: :
Figure 5: AVPF Common Feedback Message Packet Formatfor the PAUSE-RESUME message in this memo, here's how the packet fields get read 👀
FMT: the FMT value identifying the PAUSE-RESUME FCI: 9PT: Payload Type = 205 (RTPFB)Length: as defined by AVPF, i.e., the length of this packet in 32-bit words minus one, including the header and any padding.SSRC of packet sender: the SSRC of the RTP session participant sending the messages in the FCI. note, for endpoints that have multiple SSRCs in an RTP session, any of its SSRCs MAY be used to send any of the pause message types 🌸SSRC of media source: not used, so it SHALL be set to 0. the FCI identifies the SSRC the message is targeted for.
the FCI field holds one or more PAUSE, RESUME, PAUSED, or REFUSED messages, or any future extension. these messages all share this FCI format:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Target SSRC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type | Res | Parameter Len | PauseID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: Type Specific :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 6: Syntax of FCI Entry in the PAUSE and RESUME Messagehere's the lowdown on each FCI field 🫶
- Target SSRC (32 bits): for a PAUSE-RESUME message, this is the SSRC the request is meant for. for PAUSED, it MUST be the SSRC being paused. if pausing happens because of a PAUSE request, the value in PAUSED is effectively the same as Target SSRC in a related PAUSE request. for REFUSED, it MUST be the Target SSRC of the PAUSE or RESUME request that can't change state. a CSRC MUST NOT be used as a target, because the interpretation of such a request is unclear ⚠️
- Type (4 bits): the pause feedback type. the values defined in this specification are:
- 0: PAUSE request message.
- 1: RESUME request message.
- 2: PAUSED indication message.
- 3: REFUSED notification message.
- 4-15: reserved for future use. FCI fields with these Type values SHALL be ignored on reception by receivers, and MUST NOT be used by senders implementing this specification 💀
- Res (4 bits): type-specific reserved bits. they SHALL be ignored by receivers implementing this specification, and MUST be set to 0 by senders implementing this specification.
- Parameter Len (8 bits): length of the Type Specific field in 32-bit words. it MAY be 0.
- PauseID (16 bits): message sequence identification, as described in Section 5.2. it SHALL be incremented by one modulo 2^16 for each new PAUSE message, unless the message is retransmitted. the initial value SHOULD be 0. the PauseID is scoped by the Target SSRC, which means PAUSE, RESUME, and PAUSED messages share the same PauseID space for a specific Target SSRC 🧚
- Type Specific (variable): defined per pause feedback type. it MAY be empty. a receiver implementing this specification MUST be able to skip and ignore any unknown Type Specific data, even for Type values defined in this specification 🌈
8. Message Details ✨
this section is where we get into the nitty-gritty of each message defined in this specification. all transmissions of requests and indications are governed by the transmission rules in Section 8.5 🎀
any references to PAUSE, PAUSED, RESUME, and REFUSED in this section SHALL be taken to apply to the extent possible, and also when TMMBR/TMMBN are used (Section 5.6) for this functionality. TMMBR/TMMBN MAY be used instead of the messages defined in this specification when the effective topology is point to point. this use is expected to be mainly for interworking with implementations that don't support the messages defined here but do use TMMBR/TMMBN to get a similar effect 🦋 if either the sender or the receiver learns that the topology is not point to point, TMMBR/TMMBN MUST NOT be used for pause/resume functionality. if the messages defined in this specification are supported in addition to TMMBR/TMMBN by all involved parties, pause/resume signaling MUST use messages from this specification. if the topology is not point to point and the messages defined in this specification are not supported, pause/resume functionality with TMMBR/TMMBN MUST NOT be used 🙈
for the scope of this specification, a past PauseID (Section 5.2) is defined as having a value between and including (PauseID - 2^15) MOD 2^16 and (PauseID - 1) MOD 2^16, where "MOD" is the modulo operator 🧮
similarly, a future PauseID is defined as having a value between and including (PauseID + 1) MOD 2^16 and (PauseID + 2^14) MOD 2^16. it's intentional that future PauseID is not defined as the entire range outside past PauseID. the remaining range of PauseID is simply "not current" 💜
8.1. PAUSE
an RTP stream receiver MAY schedule PAUSE for transmission at any time 🌸
PAUSE has no defined Type Specific parameters.
PauseID SHOULD be the current PauseID, as indicated by PAUSED (Section 8.2), REFUSED (Section 8.4), or implicitly determined by previously received PAUSE or RESUME (Section 8.3) requests. a randomly chosen PauseID MAY be used if it wasn't possible to retrieve current PauseID information. in that case the PAUSE will either succeed, or the current PauseID can be found in the returned REFUSED (Section 8.4) 🍓
it can be noted that, as a result of what is described in Section 6.1, PauseID is incremented by one, in modulo arithmetic, for each PAUSE request that is not a retransmission, compared to what was used in the last PAUSED indication sent by the media sender. PauseID in the message is supposed to match the current PauseID at the RTP stream sender 🧠
if an RTP stream receiver that sent a PAUSE with a certain PauseID for a Target SSRC receives a RESUME or a REFUSED with the same PauseID for the same Target SSRC, it is RECOMMENDED that it refrains from scheduling further PAUSE requests for some appropriate time. this is because the RESUME indicates that there are other receivers that still want the stream, and the REFUSED indicates that the RTP stream sender is currently unable to pause the stream. what counts as an appropriate time can vary from application to application, and it also depends on how important the bandwidth saving is. still, 2-5 regular RTCP intervals is expected to be appropriate ⏳
if the targeted RTP stream does not pause, if no PAUSED indication with a future PauseID compared to the one used in PAUSE is received, and if no REFUSED with the current or a future PauseID is received within 2 * RTT + T_dither_max, the PAUSE MAY be scheduled for retransmission, using the same current PauseID. RTT is the observed round trip to the RTP stream sender, and T_dither_max is defined in Section 3.4 of [RFC4585]. an RTP stream receiver in a bi-directional RTP communication will generally have an RTT estimate to the RTP stream sender, e.g., from RTCP SR/RR as described in Section 6.4 of [RFC3550]. however, RTP stream receivers that don't send any RTP streams will lack an RTT estimate unless they use additional mechanisms, such as the "Receiver Reference Time Report Block" part of RTCP XR [RFC3611]. RTP stream receivers that lack an RTT estimate to the sender SHOULD use 500 ms as the default value 🔁
when an RTP stream sender in playing state (Section 6.1) receives a PAUSE with the current PauseID, and unless local considerations currently make it impossible to pause the stream, it SHALL enter pausing state (Section 6.2) and act accordingly 🎬
if an RTP stream sender receives a PAUSE with the current PauseID while in pausing, paused (Section 6.3), or local paused (Section 6.4) states, the received PAUSE SHALL be ignored 🙅♀️
8.2. PAUSED 🦋
okay so PAUSED is the "hey, I'm officially on pause" message 💅 If you support it, it MUST be sent every single time you enter paused state (Section 6.3) or local paused state (Section 6.4). No skipping, bestie 🫶
The PauseID in the PAUSED message MUST contain the current PauseID, which is the one that can be used in a later RESUME (Section 8.3) 🎀 For local paused state, the PauseID in the message is the current PauseID, just as if the RTP stream sender had sent a PAUSE to itself. Iconic self-care, honestly 🧚
PAUSED SHALL start its Type Specific field with a fixed-length 32-bit parameter. That parameter is the extended RTP sequence number of the last RTP packet sent before the stream got paused, and it's in the same format as the extended highest sequence number received (Section 6.4.1 of [RFC3550]) 🔢✨
Once you've entered paused or local paused state and sent PAUSED once, PAUSED MUST also show up in (at least) the next two regular RTCP reports, as long as the pause condition is still going 💜
PAUSED indications MAY be retransmitted, subject to transmission rules (Section 8.5), so the message is more likely to reach the receiver on time 🚀 This is especially important when entering local paused state. The number of repeats could be tuned based on observed loss rate and desired loss probability, for example, based on RTCP reports received from the intended message target. Main character planning 📊
While you're still in paused or local paused state, PAUSED MAY be included in all compound RTCP reports, as long as the negotiated RTCP bandwidth isn't exceeded 🌸
Here's the highkey important part 👀 When you're in paused or local paused state and the RTP stream sender finds out that some endpoints never got the stream before (say, RTCP reports with an SSRC and a CNAME that weren't seen earlier in the RTP session), it is RECOMMENDED to send PAUSED at the earliest opportunity, and also to include it in (at least) the next two regular RTCP reports, as long as the pause condition is still going. The new people deserve to know the tea ☕
8.3. RESUME 🌈
An RTP stream receiver MAY schedule RESUME for transmission whenever it wants to resume a paused stream, or to disapprove of a stream being paused. Not me judging, but sometimes you just gotta 💅
PauseID SHOULD be the current PauseID. You can know it from PAUSED (Section 8.2), or it's implicitly determined from a previously received PAUSE (Section 8.1) or RESUME request. A randomly chosen PauseID MAY be used if it wasn't possible to retrieve the current PauseID information. In that case, the RESUME will either succeed, or the current PauseID can be found in a returned REFUSED (Section 8.4) 🎲
If an RTP stream receiver that sent a RESUME with a certain PauseID gets a REFUSED with that same PauseID, it is RECOMMENDED that it holds off on scheduling more RESUME requests for some appropriate time, because the REFUSE is basically saying it's not possible to resume the stream right now 😭 What counts as appropriate can vary from app to app, and it also depends on how important resuming the stream is. That said, 1-2 regular RTCP intervals is expected to be appropriate ⏳
RESUME requests MAY be retransmitted, subject to transmission rules (Section 8.5), so the message is more likely to reach the receiver on time 🫧 The number of repeats could be tuned based on observed loss rate and desired loss probability, for example, based on RTCP reports received from the intended message target. Such retransmission SHOULD stop as soon as RTP packets from the targeted stream arrive, or when a REFUSED with the current PauseID for the targeted RTP stream shows up. Know when to quit, babe 🙈
RESUME has no defined Type Specific parameters. Zero extras, zero drama ✨
When an RTP stream sender is in pausing (Section 6.2), paused (Section 6.3), or local paused state (Section 6.4) and gets a RESUME with the current PauseID, it SHALL enter playing state (Section 6.1) and act accordingly, unless local considerations make it impossible to resume the stream right now 🎬 If the RTP stream sender can't honor a RESUME request with the current PauseID, or if it gets a RESUME request with a PauseID that isn't the current PauseID while it's in paused or pausing state, the RTP stream sender SHALL schedule a REFUSED message for transmission as specified below. Rejected, but make it graceful 💅
If an RTP stream sender in playing state gets a RESUME containing either the current PauseID or a past PauseID, the received RESUME SHALL be ignored. Already vibing, no thank you 🙃
8.4. REFUSED 🚫
If an RTP stream sender gets a PAUSE (Section 8.1) or RESUME (Section 8.3) request containing the current PauseID, and the requested action can't be fulfilled because of some local consideration, it SHALL schedule transmission of a REFUSED notification containing the current PauseID from the rejected request 💌
REFUSED has no defined Type Specific parameters. Still no extras 🍓
If an RTP stream sender gets a PAUSE or RESUME request with a PauseID that isn't the current PauseID, it SHALL schedule a REFUSED notification containing the current PauseID. The only exception is when the RTP stream sender is in playing state and receives a RESUME with a past PauseID. In that case, the RESUME SHALL be ignored 🫠
If several PAUSE or RESUME requests that would produce identical REFUSED notifications come in before the scheduled REFUSED gets sent, duplicate REFUSED notifications MUST NOT be scheduled for transmission. One REFUSED can cover a whole pile of ineffective PAUSE or RESUME requests, which is very efficient of it 🧹
An RTP stream receiver that sent a PAUSE or RESUME request and gets a REFUSED with the same PauseID as in the request SHOULD hold off on sending an identical request for some appropriate time, so the condition that caused the REFUSED can clear 🕰️ For PAUSE, an appropriate time is suggested in Section 8.1. For RESUME, an appropriate time is suggested in Section 8.3.
An RTP stream receiver that sent a PAUSE or RESUME request and gets a REFUSED with a PauseID that's different from the request MAY schedule another request using the PauseID from the REFUSED notification. Plot twist, new ID, new attempt 🦋
8.5. Transmission Rules 🚦
The transmission of any RTCP feedback messages defined in this specification MUST follow the normal AVPF-defined timing rules and depend on the session's mode of operation. Rules are rules, babe 📜
All messages defined in this specification, as well as TMMBR/TMMBN used for pause/resume purposes (Section 5.6), can use Regular, Early, or Immediate timing. But they should make a trade-off between timely transmission (Section 4.1) and RTCP bandwidth consumption. Here's how to balance it 💖
- It is recommended that PAUSE use Early or Immediate timing, except for retransmissions, where RTCP bandwidth can motivate the use of Regular timing. 🏃♀️
- The first transmission of PAUSED for each (non-wrapped) PauseID is recommended to be sent with Immediate or Early timing, to stop unnecessary repetitions of PAUSE. It is recommended that subsequent transmissions of PAUSED for that PauseID use Regular timing, to avoid excessive PAUSED RTCP bandwidth caused by multiple PAUSE requests. 🌷
- It is recommended that unsolicited PAUSED (sent when entering local paused state (Section 6.4)) always use Immediate or Early timing, until PAUSED for that PauseID is considered delivered at least once to all receivers of the paused RTP stream. This stops RTP stream receivers from taking unnecessary corrective action when the RTP stream is no longer received. After that, it is recommended that PAUSE uses Regular timing (as for PAUSED triggered by PAUSE above). 🧠
- RESUME is often time critical, and it is recommended that it always uses Immediate or Early timing. Time-sensitive, no cap 🔥
- The first transmission of REFUSED for each (non-wrapped) PauseID is recommended to be sent with Immediate or Early timing, to stop unnecessary repetitions of PAUSE or RESUME. It is recommended that subsequent REFUSED notifications for that PauseID use Regular timing, to avoid excessive REFUSED RTCP bandwidth caused by multiple unreasonable requests. 🙅♀️✨
9. Signaling ✨
okay so, the ability to handle the messages in this spec MAY get shared at a higher layer, like SDP 💌 This doc extends the rtcp-fb attribute from Section 4 of AVPF [RFC4585] so it can ask for pause and resume 🎀 It also follows every rule in AVPF [RFC4585] and CCM [RFC5104] for an rtcp-fb attribute tied to a payload type in a session description. no cap, it's all lined up 💅
This spec adds a new pause parameter to the "ccm" feedback value from CCM [RFC5104] 🦋 That parameter means you can understand the RTCP feedback message and all the FCIs for PAUSE, RESUME, PAUSED, and REFUSED. it's giving full set of tools 🧰
Note: When TMMBR 0 / TMMBN 0 are used to implement pause and resume functionality (with the restrictions described in this specification), signaling the "rtcp-fb" attribute with the "ccm" and "tmmbr" parameters is sufficient and no further signaling is necessary. There is, however, no guarantee that TMMBR/TMMBN implementations predating this specification work exactly as described here when used with a bitrate value of 0. 🙈
The pause parameter has two optional attributes, nowait and config 🍓
nowaitmeans the hold-off period from Section 6.2 can be set to 0. That cuts down the latency before a stream can be paused after a PAUSE request comes in ⚡ This happens when there will only be a single receiver per direction in the RTP session, like in point-to-point sessions. It can also be used with unidirectional media. The conditions that letnowaitbe set (Section 6.2) also mean CCM TMMBR/TMMBN could work as pause/resume signaling 🌸configallows partial implementation of this spec based on the different roles in the use cases (Section 3). It takes a value that describes which subset is implemented 🧚1: Full implementation of this specification. This is the default configuration. A missingconfigpause attribute MUST be treated equivalent to providing aconfigvalue of 1. 👑2: The implementation intends to send PAUSE and RESUME requests for received RTP streams, so it is also capable of receiving PAUSED and REFUSED. It does not support receiving PAUSE and RESUME requests, but it may pause sent RTP streams due to local considerations and then intend to send PAUSED for them. 📨3: The implementation supports receiving PAUSE and RESUME requests targeted at RTP streams it sends. It will send PAUSED and REFUSED as needed. The node will not send any PAUSE and RESUME requests, but it supports and desires receiving PAUSED if received RTP streams are paused. 🎧4: The implementation intends to send PAUSE and RESUME requests for received RTP streams, so it is also capable of receiving PAUSED and REFUSED. It cannot pause any RTP streams it sends, so it does not support receiving PAUSE and RESUME requests, and it also does not support sending PAUSED indications. 🙅♀️5: The implementation supports receiving PAUSE and RESUME requests targeted at RTP streams it sends. It will send PAUSED and REFUSED as needed. It does not support sending PAUSE and RESUME requests to pause received RTP streams, and it also does not support receiving PAUSED indications. 🌷6: The implementation supports sent and received RTP streams being paused due to local considerations, so it supports sending and receiving PAUSED indications. 🫧7: The implementation supports and desires receiving PAUSED indications for received RTP streams, but it does not pause or send PAUSED indications for sent RTP streams. It does not support any other messages defined in this specification. 🌙8: The implementation supports pausing sent RTP streams and sending PAUSED indications for them, but it does not support receiving PAUSED indications for received RTP streams. It does not support any other messages defined in this specification. 🍒
All implementers of this specification are encouraged to include full support for all messages (config=1). It's recognized that this is sometimes not meaningful for implementations in an environment where only parts of the functionality are needed, and that's valid 🫶 The config subsets above are a trade-off between completeness and interoperability. They give at least the level of functionality the least-capable party wants when used as specified here. Implementing any functionality subsets other than those defined above is NOT RECOMMENDED. 🚫
When signaling a config value other than 1, an implementation MUST ignore non-supported messages on reception and SHOULD omit sending messages not supported by the remote peer. One reason you might send messages some receivers don't support is when there are multiple message receivers with different message support (different config values). That keeps the least-capable receiver from limiting what everyone else gets 💅 The table below sums up per-message send and receive support for each config value ("X" means support, "-" means no support) 📋
+---+-----------------------------+-----------------------------+
| # | Send | Receive |
| | PAUSE RESUME PAUSED REFUSED | PAUSE RESUME PAUSED REFUSED |
+---+-----------------------------+-----------------------------+
| 1 | X X X X | X X X X |
| 2 | X X X - | - - X X |
| 3 | - - X X | X X X - |
| 4 | X X - - | - - X X |
| 5 | - - X X | X X - - |
| 6 | - - X - | - - X - |
| 7 | - - - - | - - X - |
| 8 | - - X - | - - - - |
+---+-----------------------------+-----------------------------+ Figure 7: Supported Messages for Different "config" ValuesIn the partial implementation descriptions above, config values 2 and 4 match the RTP Mixer in the 'RTP Mixer to Media Sender' use case (Section 3.2). config values 3 and 5 match the media sender in that same use case. For that use case, it should be clear that an RTP Mixer implementing only config values 3 or 5 won't give you a working solution. Same deal the other way: for that use case, a media sender implementing only config values 2 or 4 won't give you a working solution 😭 Both the RTP Mixer and the media sender work fine when they implement the full set of messages, which is config=1 ✨
A partial implementation is not suitable for pause/resume support between cascaded RTP Mixers. Those mixers would need support matching config=1 🧩 That's because an RTP Mixer is also a media sender toward the other RTP Mixer, so it needs the union of config values 2 and 3, or config values 4 and 5, which effectively becomes config=1.
As you can see from Figure 7 above, config values 2 and 3 differ from config values 4 and 5 in just one way. In the latter, the PAUSE/RESUME message sender (for example, the RTP Mixer side) does not support local pause (Section 6.4) for any of its own streams, so it also doesn't support sending PAUSED 🦋
Partial implementations that only support local pause can declare that capability with config values 6-8 🌈
Viable fallback rules between different config values are described in Section 9.1 and Figure 9 🗺️
This is the resulting ABNF [RFC5234], which extends the existing ABNF in Section 7.1 of CCM [RFC5104] 📜
rtcp-fb-ccm-param =/ SP "pause" *(SP pause-attr)
pause-attr = pause-config ; partial message support
/ "nowait" ; no hold-off period
/ byte-string ; for future extensions
pause-config = "config=" pause-config-value
pause-config-value = 1*2DIGIT
; byte-string as defined in RFC 4566 Figure 8: ABNFAn endpoint that implements this spec and uses SDP to signal capability SHOULD indicate the new pause parameter with ccm signaling. It MAY instead use existing ccm tmmbr signaling [RFC5104] if the limitations on functionality when using TMMBR/TMMBN described in this specification (Section 5.6) are acceptable. In that case, no partial message support is possible. The messages from this specification (Section 8) SHOULD NOT be sent to receivers that did not declare capability to receive those messages 🚦
The pause functionality can normally be expected to work independently of the payload type. However, there might be situations where an endpoint needs to restrict, or at least configure, capabilities differently depending on the payload type carrying the media stream. Reasons for this might relate to being able to correctly handle media boundaries, and to avoiding any pause or resume operation that would leave a receiver or decoder with no choice but to attempt to repair or discard the media received just before or at the point of resuming 🎬
There MUST NOT be more than one a=rtcp-fb line with pause applicable to a single payload type in the SDP. The exception is when the additional line uses * as the payload type, in which case * SHALL be interpreted as applicable to all listed payload types that do not have an explicit pause specification. The config pause attribute MUST NOT appear more than once for each pause CCM parameter. The nowait pause attribute MUST NOT appear more than once for each pause CCM parameter 🔥
9.1. Offer/Answer Use ✨
okay so if you're an offerer and you want pause and resume to work, you have to include the "pause" CCM parameter with a suitable "config" attribute in your SDP. The config you pick depends on which messages you plan to send and which ones you want to receive in the session 💖
In SDP offer/answer, the "config" pause attribute and its message directions get read from the perspective of whoever wrote the SDP. The offerer describes itself in the offer, and the answerer describes itself in the answer 🎀
Now the answerer gets to play with the config a little 💅 If an answerer gets an offer with a "pause" CCM line and a "config" pause attribute describing what the offerer can send and receive, it MAY change that "config" value to a different one in the answer. The allowed swaps are in this table:
SDP Offer "config" value | Permitted SDP Answer "config" values
-------------------------+-------------------------------------
1 | 1, 2, 3, 4, 5, 6, 7, 8
2 | 3, 4, 5, 6, 7, 8
3 | 2, 4, 5, 6, 7, 8
4 | 5, 6, 7, 8
5 | 4, 6, 7, 8
6 | 6, 7, 8
7 | 8
8 | 7
Figure 9: "config" Values in Offer/AnswerIf an offer or answer doesn't include the "config" pause attribute, it MUST be treated as if it said "config=1" 🧚 Implementations of this spec MUST NOT use any "config" values beyond the ones defined above in an offer or answer. And when an offer has a "config" value you don't understand, you MUST remove the "pause" CCM line from your answer 😭 No matter what, the answerer MAY also wipe out any "pause" CCM line completely. That's a way of saying it doesn't understand, or doesn't want to use, pause for the affected payload types 🙈
Next up is the "nowait" pause attribute. If the offerer thinks it and the intended answerer are probably the only endpoints in the RTP session, it MAY put "nowait" on the "pause" line in the offer 🦋 If an answerer sees "nowait" on the "pause" line and has info saying the offerer and itself are not the only endpoints in the RTP session, it MUST NOT include any "nowait" pause attribute on its "pause" line in the SDP answer. The answerer also MUST NOT add "nowait" on the "pause" line in the answer unless it was already on the "pause" line in the offer. And if both the offer and the answer have a "nowait" pause attribute, then the hold-off period is set to 0 at both the offerer and the answerer 🌸
Unknown pause attributes MUST be ignored in the offer, and then they MUST be left out of the answer 🫶
Last thing for this bit: if both "pause" and "tmmbr" show up in the offer, both MAY also be included in the answer. In that case, TMMBR/TMMBN MUST NOT be used for pause/resume purposes (with a bitrate value of 0). This keeps the signaling from getting confusing 🔥
9.2. Declarative Use 🌈
okay so in declarative use, the SDP is used to configure the node that receives it. That changes how the SDP signaling extensions from this spec get read 👑
First, the "config" pause attribute and its message directions are read from the point of view of the node receiving the SDP, and it describes the RECOMMENDED level of operation. If the joining client doesn't support the "config" value that was indicated, some RTP session stream optimizations might not happen. For example, the joining client might not pause some RTP streams, and/or it might not be able to resume and receive the streams it wants because they're paused 😭
Second, the "nowait" pause attribute, if it's included, gets followed as specified. The declarative SDP sender is responsible for figuring out whether a configured node will be in a point-to-point session, based on how it's being used. For instance, a conference client set up for an any source multicast session using the Session Announcement Protocess (SAP) [RFC2974] won't be in a point-to-point session, so "nowait" can't be included there ✨ A Real-Time Streaming Protocol (RTSP) [RFC2326] client that gets a declarative SDP might well be in a point-to-point session, but it's highly doubtful an RTSP client would need to support this spec, since RTSP already has PAUSE support built in 💅
Unknown pause attributes MUST be ignored 💀
If both "pause" and "tmmbr" are in the SDP, TMMBR/TMMBN MUST NOT be used for pause/resume purposes (with a bitrate value of 0). Again, this is to avoid signaling ambiguity 🎀
10. Examples 🧚
okay the examples are where it all clicks 🦋 They show how PAUSE and RESUME messages get used, including offer/answer. Here's the lineup:
- Offer/Answer
- Point-to-Point Session
- Point to Multipoint using Mixer
- Point to Multipoint using Relay
10.1. Offer/Answer 💖
These figures show how to signal support for pausing and resuming streams, and whether the hold-off period can be set to 0 ✨
v=0
o=alice 3203093520 3203093520 IN IP4 alice.example.com
s=Pausing Media
t=0 0
c=IN IP4 alice.example.com
m=audio 49170 RTP/AVPF 98 99
a=rtpmap:98 G719/48000
a=rtpmap:99 PCMA/8000
a=rtcp-fb:* ccm pause nowait
Figure 10: SDP Offer with Pause and Resume CapabilityAlice supports all the messages in this spec, and she skips the optional "config" pause attribute 🫶 She also thinks she'll be the only receiver of Bob's stream, and Bob will be the only receiver of hers. So she includes the "nowait" pause attribute for the "pause" parameter 🌸
Here's Bob's SDP answer:
v=0
o=bob 293847192 293847192 IN IP4 bob.example.com
s=-
t=0 0
c=IN IP4 bob.example.com
m=audio 49202 RTP/AVPF 98
a=rtpmap:98 G719/48000
a=rtcp-fb:98 ccm pause config=2
Figure 11: SDP Answer with Pause and Resume CapabilityBob won't let his sent streams be paused or resumed, so he limits his answer to "config=2" 🙈 He also supports pausing his own RTP streams for local reasons, which is why he picked "config=2" and not "config=4". Bob somehow knows the RTP session won't be point-to-point, so he took "nowait" off the "pause" line. That means Alice has to use a non-zero hold-off period when she's asked to pause the stream 🍓
When you use TMMBR 0 / TMMBN 0 for pause and resume, the SDP doesn't change at all compared to CCM [RFC5104]. So there are no examples for that here 💜
10.2. Point-to-Point Session
okay so this is the most basic scenario, just two participants, and each one can be a sender, a receiver, or both 💖 Any RTP data receiver sends PAUSE or RESUME messages to the sender, and the sender pauses or resumes transmission to match. The hold-off period before pausing a stream is 0 ✨
+---------------+ +---------------+
| RTP Sender | | RTP Receiver |
+---------------+ +---------------+
: t1: RTP data :
| -------------------------------> |
| t2: PAUSE(3) |
| <------------------------------- |
| < RTP data paused > |
| t3: PAUSED(3) |
| -------------------------------> |
: < Some time passes > :
| t4: RESUME(3) |
| <------------------------------- |
| t5: RTP data |
| -------------------------------> |
: < Some time passes > :
| t6: PAUSE(4) |
| <------------------------------- |
| < RTP data paused > |
| t7: PAUSED(4) |
| -------------------------------> |
: :Figure 12: Pause and Resume Operation in Point to Point
Figure 12 is the basic pause and resume dance in a point-to-point setup 🦋 At t1, the RTP sender is sending data to the receiver, no drama. At t2, the receiver asks the sender to pause the stream using PauseID 3 (which it already knew about from earlier in this example). The sender pauses and replies with a PAUSED that has the same PauseID, iconic. Some time later (at t4), the receiver asks for a RESUME, and the sender gets its transmission going again 🌸 The next PAUSE, sent at t6, has a fresh PauseID (4), and the matching PAUSED shows up at t7. Main character energy for PauseIDs, honestly 👑
+---------------+ +---------------+
| RTP Sender | | RTP Receiver |
+---------------+ +---------------+
: t1: RTP data :
| -------------------------------> |
| t2: TMMBR 0 |
| <------------------------------- |
| < RTP data paused > |
| t3: TMMBN 0 |
| -------------------------------> |
: < Some time passes > :
| t4: TMMBR 150000 |
| <------------------------------- |
| t5: RTP data |
| -------------------------------> |
: < Some time passes > :
| t6: TMMBR 0 |
| <------------------------------- |
| < RTP data paused > |
| t7: TMMBN 0 |
| -------------------------------> |
: :Figure 13: TMMBR Pause and Resume in Point to Point
Figure 13 is the very same point-to-point story, just told with TMMBR/TMMBN signaling instead 🎀 Same vibes, different messages, periodt 💅
+---------------+ +----------------+
| RTP Sender A | | RTP Receiver B |
+---------------+ +----------------+
: t1: RTP data :
| -------------------------------> |
| < RTP data paused > |
| t2: TMMBN {A:0} |
| -------------------------------> |
: < Some time passes > :
| t3: TMMBR 0 |
| <------------------------------- |
| t4: TMMBN {A:0,B:0} |
| -------------------------------> |
: < Some time passes > :
| t5: TMMBN {B:0} |
| -------------------------------> |
: < Some time passes > :
| t6: TMMBR 80000 |
| <------------------------------- |
| t7: RTP data |
| -------------------------------> |
: :Figure 14: Unsolicited PAUSED Using TMMBN
Figure 14 is the plot twist 😭 This time the sender (A) pauses the stream all on its own, because of local considerations, and it's the only one doing the pausing. Both A and the receiver (B) use TMMBR/TMMBN for pause and resume. A pauses at t2 by sending TMMBN 0 to signal PAUSED, and it puts itself in the TMMBN bounding set. Then at t3, even though the stream is still paused, B decides it doesn't want the stream anymore and sends a TMMBR 0 to signal PAUSE. Now the bounding set has both A and B in it, so A sends out a new TMMBN that says exactly that 🫶 A little later, at t5, whatever local reason made A pause is gone, so A takes itself out of the bounding set and sends a new TMMBN to announce it. Then at t6, B changes its mind and wants the stream back, so it sends a TMMBR with a bitrate bigger than 0 to signal RESUME, and A starts sending RTP data again at t7. Iconic, a full redemption arc 🔥
+---------------+ +---------------+
| RTP Sender | | RTP Receiver |
+---------------+ +---------------+
: t1: RTP data :
| ------------------------------------> |
| t2: PAUSE(7), lost |
| <---X-------------- |
| |
| t3: RTP data |
| ------------------------------------> |
: :
| < Time-out, still receiving data > |
| t4: PAUSE(7) |
| <------------------------------------ |
| < RTP data paused > |
| t5: PAUSED(7) |
| ------------------------------------> |
: < Some time passes > :
| t6: RESUME(7), lost |
| <---X-------------- |
| t7: RESUME(7) |
| <------------------------------------ |
| t8: RTP data |
| ------------------------------------> |
| t9: RESUME(7) |
| <------------------------------------ |
: :Figure 15: Pause and Resume Operation with Messages Lost
Figure 15 is where things get messy 😬 It shows what happens when a PAUSE from the receiver never makes it to the sender. After the receiver sends a PAUSE, it waits for a time-out, so it can tell whether the sender actually paused the data or sent back a PAUSED, following the rules in Section 6.3 💜 Here the PAUSE gets lost at t2, so RTP data keeps flowing to the receiver. When the timer runs out, the receiver schedules a retransmission of the PAUSE, which goes out at t4. This time it arrives, so the sender pauses the stream and replies with PAUSED. Victory 🎉
At t6, the receiver wants the stream back and sends a RESUME, which gets lost too. Honestly not a disaster, since nothing says you have to wait before sending more RESUMEs, and another one goes out at t7 that actually lands. The sender resumes the stream at t8. The gap between t6 and t7 can vary, but it might be one RTCP feedback transmission interval, as set by the AVPF rules ⏱️
The receiver doesn't find out the stream resumed in time to cancel one more RESUME that was already scheduled for t9. Lowkey harmless though 🙈 That RESUME carries a past PauseID, so the sender ignores it. Even if some other receiver pauses the stream before the stray RESUME shows up, it still won't resume anything, because the current PauseID has moved on since the stray one was made. Old PauseIDs are literally just ignored, end of story 🧚
+---------------+ +---------------+
| RTP Sender | | RTP Receiver |
+---------------+ +---------------+
: t1: RTP data :
| ------------------------------> |
| t2: PAUSE(11) |
| <------------------------------ |
| |
| < Cannot pause RTP data > |
| t3: REFUSED(11) |
| ------------------------------> |
| |
| t4: RTP data |
| ------------------------------> |
: :Figure 16: Pause Request is Refused in Point to Point
In Figure 16, the receiver asks the sender to pause, but the sender says no because of something local to its side and answers with a REFUSED message 🙅♀️ Sometimes the sender just has its own reasons, and that's valid, bestie 🌈
10.3. Point to Multipoint Using Mixer ✨
okay so an RTP Mixer is an intermediate node that connects different transport-level clouds 🦋 It receives streams from different RTP sources, selects or combines them based on what the app needs, and forwards the generated stream(s) to the destination. The Mixer typically puts its own SSRC(s) in RTP data packets instead of the original source(s), which is giving main character energy 💅
The Mixer keeps track of all the streams delivered to it and how they're currently being used 👀 In this example, Mixer (M) picks the video stream to send to the RTP stream receiver (R) based on the voice activity of the RTP stream senders (S1 and S2). The video stream gets delivered to R using M's SSRC, with a CSRC showing the original source 🎀
Heads up: PauseID doesn't matter for this example, so it's left out of the description 🙈
+-----+ +-----+ +-----+ +-----+
| R | | M | | S1 | | S2 |
+-----+ +-----+ +-----+ +-----+
: : t1:RTP(S1) : :
| t2:RTP(M:S1) |<-----------------| |
|<-----------------| | |
| | t3:RTP(S2) | |
| |<------------------------------------|
| | t4: PAUSE(S2) | |
| |------------------------------------>|
| | | t5: PAUSED(S2) |
| |<------------------------------------|
| | | <S2:No RTP to M> |
| | t6: RESUME(S2) | |
| |------------------------------------>|
| | | t7: RTP to M |
| |<------------------------------------|
| t8:RTP(M:S2) | | |
|<-----------------| | |
| | t9:PAUSE(S1) | |
| |----------------->| |
| | t10:PAUSED(S1) | |
| |<-----------------| |
| | <S1:No RTP to M> | |
: : : :Figure 17: Pause and Resume Operation for a Voice-Activated Mixer
The session kicks off at t1 with S1 as the most active speaker, so S1 gets picked as the single video stream sent to R (t2) using M's SSRC, but with S1 as the CSRC (that's the bit after the colon in the figure) 🌸 Then S2 joins at t3 and starts delivering an RTP stream to M. Since S2 has less voice activity than S1, M decides to pause S2 at t4 by sending S2 a PAUSE request 😌 At t5, S2 replies with PAUSED and in that same instant stops delivering RTP to M.
At t6, the user at S2 starts talking and becomes the most active speaker, so M switches the video over to S2 and quickly sends S2 a RESUME request 🗣️ At t7, S2 has received the RESUME request and acts on it by resuming RTP stream delivery to M. When that RTP stream from t7 shows up at M, M switches it into its own SSRC at t8 and changes the CSRC to S2 ✨ S1 is now unused, so M sends S1 a PAUSE request at t9. S1 acknowledges with PAUSED at t10, and the RTP stream from S1 stops being delivered 🫶
10.4. Point to Multipoint Using Translator 🦋
okay so a transport Relay in an RTP session forwards messages from one peer to all the others 💖 Unlike a Mixer, the Relay doesn't mix the streams or change the SSRC of the messages or RTP media. These examples exist to show that the messages defined in this specification can be safely used in a transport Relay case too 🧚 The parentheses in the figures contain (Target SSRC, PauseID) information for the messages defined in this specification.
+-------------+ +-------------+ +-------------+
| Sender(S) | | Relay | | Receiver(R) |
+-------------+ +-------------+ +-------------+
: t1: RTP(S) : :
|------------------>| |
| | t2: RTP (S) |
| |------------------>|
| | t3: PAUSE(S,3) |
| |<------------------|
| t4:PAUSE(S,3) | |
|<------------------| |
: <Sender waiting for possible RESUME> :
| < RTP data paused > |
| t5: PAUSED(S,3) | |
|------------------>| |
| | t6: PAUSED(S,3) |
| |------------------>|
: : :
| | t7: RESUME(S,3) |
| |<------------------|
| t8: RESUME(S,3) | |
|<------------------| |
| t9: RTP (S) | |
|------------------>| |
| | t10: RTP (S) |
| |------------------>|
: : :Figure 18: Pause and Resume Operation between Two Participants Using a Relay
Figure 18 shows how a Relay can help the receiver (R) pause and resume the sender (S) 💅 S sends RTP data to R through the Relay, which just forwards it without modifying the SSRCs. R sends a PAUSE request to S. In this example, S knows there may be more receivers of the stream, so it waits a non-zero hold-off period to see if any other receiver wants the data. If no disapproving RESUME messages show up, S pauses itself and replies with PAUSED 😤 R resumes S in the same way, by sending a RESUME request through the Relay. Since this only covers a single pause and resume for one RTP stream sender, every message uses one PauseID, which here is three 🍓
+-----+ +-----+ +-----+ +-----+
| S | | Rel | | R1 | | R2 |
+-----+ +-----+ +-----+ +-----+
: t1:RTP(S) : : :
|----------------->| | |
| | t2:RTP(S) | |
| |----------------->------------------>|
| | t3:PAUSE(S,7) | |
| |<-----------------| |
| t4:PAUSE(S,7) | | |
|<-----------------|------------------------------------>|
| | | t5:RESUME(S,7) |
| |<------------------------------------|
| t6:RESUME(S,7) | | |
|<-----------------|----------------->| |
| | <RTP stream continues to R1 and R2> |
| | | t7: PAUSE(S,8) |
| |<------------------------------------|
| t8:PAUSE(S,8) | | |
|<-----------------|----------------->| |
: : : :
| < Pauses RTP stream > | |
| t9:PAUSED(S,8) | | |
|----------------->| | |
| | t10:PAUSED(S,8) | |
| |----------------->------------------>|
: : : :
| | t11:RESUME(S,8) | |
| |<-----------------| |
| t12:RESUME(S,8) | | |
|<-----------------|------------------------------------>|
| t13:RTP(S) | | |
|----------------->| | |
| | t14:RTP(S) | |
| |----------------->------------------>|
: : : :Figure 19: Pause and Resume Operation between One Sender and Two Receivers through Relay
Figure 19 walks through the pause and resume operations when a transport Relay (Rel) sits between a sender (S) and two receivers (R1 and R2) in an RTP session 🌈 Each message exchange is shown by the time it happens. At t1, S starts sending an RTP stream to Rel, which forwards it to R1 and R2. R1 and R2 receive RTP data from Rel at t2, and at this point both of them send RTCP Receiver Reports to S to say they got S's stream 📩
After some time (at t3), R1 decides to pause the stream. When S gets the PAUSE request from R1 at t4, it knows at least one receiver may still want the data, so it uses a non-zero hold-off period to wait for possible RESUME messages. R2 also received the PAUSE request at t4, and since it still wants the stream, it sends a RESUME at t5, which Rel forwards to S 🙌 S sees the RESUME at t6 and keeps sending data to Rel, which forwards it to both R1 and R2. At t7, R2 decides to pause the stream by sending a PAUSE request with an updated PauseID. S still knows more than one receiver (R1 and R2) may want the stream, so it waits another non-zero hold-off period. After that, with no disapproving RESUME messages received, it concludes the stream must be paused. S stops sending the stream and replies with PAUSED to R1 and R2 💜 If any receiver (R1 or R2) chooses to resume the stream from S, in this example R1, it sends a RESUME request to S, which R2 also sees, and S resumes the stream immediately ⭐
Now consider an RTP session that has one or more receivers, paused sender(s), and a Relay. Say a new participant joins the session who isn't aware of the paused sender(s). Once the paused sender(s) learn about the newly joined participant, e.g., from any RTP traffic or RTCP report (either SR or RR) it sends, they immediately send PAUSED indications for the paused streams 🔥 Why? Because there's now a receiver in the session that didn't pause the sender(s) and may want to receive the streams. That means the newly joined participant has the same chance as everyone else to resume the paused streams 🥹
11. IANA Considerations 📋
Okay, IANA gets a little sparkle moment here ✨ because the spec registers some new values:
- A new value for media stream pause/resume has been registered in the "FMT Values for RTPFB Payload Types" registry located at the time of publication at:
http://www.iana.org/assignments/rtp-parameters
Value: 9
Name: PAUSE-RESUME
Long Name: Media Pause/Resume
Reference: RFC 7728- A new value "pause" to be registered with IANA in the "Codec Control Messages" registry located at the time of publication at:
http://www.iana.org/assignments/sdp-parameters
Value Name: pause
Long Name: Media Pause/Resume
Usable with: ccm
Reference: RFC 772812. Security Considerations 🔐
Okay bestie, this part is serious, so let's get real 💀. This document extends CCM [RFC5104] and defines new messages, i.e., PAUSE, RESUME, PAUSED, and REFUSED. Sending these new messages has some security implications, and the user is the one who has to deal with them 🫠.
The messages defined here can seriously hurt the perceived media quality if somebody uses them maliciously 😤. First of all, there's a real risk of Denial of Service (DoS) on any RTP session that uses the PAUSE-RESUME functionality. If an attacker injects one or more PAUSE requests into the RTP session, they can potentially stop all media from flowing, especially when the hold-off period is zero. Injecting PAUSE messages is pretty easy, since it only takes knowing the SSRC and the PauseID. An on-path attacker can see this information unless RTCP messages are encrypted. Even off-path attacks are possible, because signaling messages often carry the SSRC value, and the 16-bit PauseID just has to be guessed or tried 🙈.
The way to protect the RTP session from these injections is to combine source authentication with message integrity, so that only the intended session participants can send these messages 🛡️. The security solution should provide replay protection. Otherwise, if a session lasts long enough for the PauseID value to wrap around, an attacker could replay old messages at the right moment and mess with the media sender's state 😱. There are several different ways to secure RTP sessions and stop this kind of attack. The Secure Real-time Transport Protocol (SRTP) is the most common one, but other methods exist too, as discussed in "Options for Securing RTP Sessions" [RFC7201] 🔑.
Most methods for securing RTP, however, don't give source authentication for each individual participant in a multiparty use case 😬. If one of the session participants is malicious, they can cause serious chaos in the RTP session and cause a DoS from the inside. They could even try to hide the damage by impersonating other endpoints in the session 👀. These attacks can be mitigated with a solution that gives true source authentication for all participants' RTCP packets. That comes with other implications, though. For multiparty sessions that include a middlebox, that middlebox is RECOMMENDED to check all forwarded RTCP packets, so that each participant only uses its own set of SSRCs and can't borrow another participant's SSRCs 🕵️. An attacker who can send a PAUSE request that doesn't reach any participants other than the media sender can pause a stream without giving anyone a chance to push back. This is mitigated in multiparty topologies where requests are seen by all or most of the RTP session participants, so those participants can send a RESUME 💪. In topologies with middleboxes that consume and process PAUSE requests, the middlebox can also help stop this behavior, since it will commonly not generate or forward a PAUSE message if it knows another participant still has use for the media stream 🧚.
The text above has focused on using the PAUSE message as the tool for malicious impact on the RTP session. That's because denying users access to RTP media streams has the bigger impact 💔. On the flip side, if an attacker tries to use RESUME for bad purposes, the media streams get delivered. However, that kind of attack basically blocks people from using the pause and resume functionality. So it can potentially force the media quality down because of limited available resources, like bandwidth that must be shared 📉.
Session establishment signaling is also a possible attack venue 🚨. Someone could stop pause and resume functionality from being enabled by modifying the signaling messages. The source-authentication mitigations above also require the signaling system to handle identities securely and to confirm that only the intended identities get into the RTP session and receive the relevant security contexts 🔒✨.
13. References 📚
13.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<http://www.rfc-editor.org/info/rfc2119>.
[RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model
with Session Description Protocol (SDP)", RFC 3264,
DOI 10.17487/RFC3264, June 2002,
<http://www.rfc-editor.org/info/rfc3264>.
[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550,
July 2003, <http://www.rfc-editor.org/info/rfc3550>.
[RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
Description Protocol", RFC 4566, DOI 10.17487/RFC4566,
July 2006, <http://www.rfc-editor.org/info/rfc4566>.
[RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey,
"Extended RTP Profile for Real-time Transport Control
Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585,
DOI 10.17487/RFC4585, July 2006,
<http://www.rfc-editor.org/info/rfc4585>.
[RFC5104] Wenger, S., Chandra, U., Westerlund, M., and B. Burman,
"Codec Control Messages in the RTP Audio-Visual Profile
with Feedback (AVPF)", RFC 5104, DOI 10.17487/RFC5104,
February 2008, <http://www.rfc-editor.org/info/rfc5104>.
[RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax
Specifications: ABNF", STD 68, RFC 5234,
DOI 10.17487/RFC5234, January 2008,
<http://www.rfc-editor.org/info/rfc5234>.
[RFC5245] Rosenberg, J., "Interactive Connectivity Establishment
(ICE): A Protocol for Network Address Translator (NAT)
Traversal for Offer/Answer Protocols", RFC 5245,
DOI 10.17487/RFC5245, April 2010,
<http://www.rfc-editor.org/info/rfc5245>.
[RFC6263] Marjou, X. and A. Sollaud, "Application Mechanism for
Keeping Alive the NAT Mappings Associated with RTP / RTP
Control Protocol (RTCP) Flows", RFC 6263,
DOI 10.17487/RFC6263, June 2011,
<http://www.rfc-editor.org/info/rfc6263>.13.2. Informative References
[MULTI-STREAM-OPT]
Lennox, J., Westerlund, M., Wu, W., and C. Perkins,
"Sending Multiple Media Streams in a Single RTP Session:
Grouping RTCP Reception Statistics and Other Feedback",
Work in Progress, draft-ietf-avtcore-rtp-multi-stream-
optimisation-11, December 2015.
[RFC2326] Schulzrinne, H., Rao, A., and R. Lanphier, "Real Time
Streaming Protocol (RTSP)", RFC 2326,
DOI 10.17487/RFC2326, April 1998,
<http://www.rfc-editor.org/info/rfc2326>.
[RFC2974] Handley, M., Perkins, C., and E. Whelan, "Session
Announcement Protocol", RFC 2974, DOI 10.17487/RFC2974,
October 2000, <http://www.rfc-editor.org/info/rfc2974>.
[RFC3261] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston,
A., Peterson, J., Sparks, R., Handley, M., and E.
Schooler, "SIP: Session Initiation Protocol", RFC 3261,
DOI 10.17487/RFC3261, June 2002,
<http://www.rfc-editor.org/info/rfc3261>.
[RFC3611] Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed.,
"RTP Control Protocol Extended Reports (RTCP XR)",
RFC 3611, DOI 10.17487/RFC3611, November 2003,
<http://www.rfc-editor.org/info/rfc3611>.
[RFC6190] Wenger, S., Wang, Y., Schierl, T., and A. Eleftheriadis,
"RTP Payload Format for Scalable Video Coding", RFC 6190,
DOI 10.17487/RFC6190, May 2011,
<http://www.rfc-editor.org/info/rfc6190>.
[RFC7201] Westerlund, M. and C. Perkins, "Options for Securing RTP
Sessions", RFC 7201, DOI 10.17487/RFC7201, April 2014,
<http://www.rfc-editor.org/info/rfc7201>.
[RFC7478] Holmberg, C., Hakansson, S., and G. Eriksson, "Web Real-
Time Communication Use Cases and Requirements", RFC 7478,
DOI 10.17487/RFC7478, March 2015,
<http://www.rfc-editor.org/info/rfc7478>.
[RFC7656] Lennox, J., Gross, K., Nandakumar, S., Salgueiro, G., and
B. Burman, Ed., "A Taxonomy of Semantics and Mechanisms
for Real-Time Transport Protocol (RTP) Sources", RFC 7656,
DOI 10.17487/RFC7656, November 2015,
<http://www.rfc-editor.org/info/rfc7656>.
[RFC7667] Westerlund, M. and S. Wenger, "RTP Topologies", RFC 7667,
DOI 10.17487/RFC7667, November 2015,
<http://www.rfc-editor.org/info/rfc7667>.
[SDP-SIMULCAST]
Burman, B., Westerlund, M., Nandakumar, S., and M. Zanaty,
"Using Simulcast in SDP and RTP Sessions", Work in
Progress, draft-ietf-mmusic-sdp-simulcast-04, February
2016.Acknowledgments 💖
Daniel Grondal made valuable contributions during the initial versions of this document. The authors would also like to thank Emil Ivov, Christian Groves, David Mandelberg, Meral Shirazipour, Spencer Dawkins, Bernard Aboba, and Ben Campbell, who provided valuable review comments 🫶.
Contributors 🌸
Daniel Grondal contributed in the creation and writing of early versions of this specification. Christian Groves contributed significantly to the SDP "config" pause attribute and its use in offer/answer 👑.
Authors' Addresses 🎀
Bo Burman
Ericsson
Kistavagen 25
SE - 164 80 Kista
Sweden
Email: bo.burman@ericsson.com
Azam Akram
Ericsson
Farogatan 6
SE - 164 80 Kista
Sweden
Phone: +46107142658
Email: akram.muhammadazam@gmail.com
URI: www.ericsson.com
Roni Even
Huawei Technologies
Tel Aviv
Israel
Email: roni.even@mail01.huawei.com
Magnus Westerlund
Ericsson
Farogatan 6
SE - 164 80 Kista
Sweden
Phone: +46107148287
Email: magnus.westerlund@ericsson.com📦 where this page comes from
- ietf: rfc7728 2016-02
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms