RFC 4585
Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)
Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF) 🎀
Okay bestie, this is RFC 4585, published July 2006 as a Standards Track document in the Network Working Group. It's written by J. Ott (Helsinki University of Technology), S. Wenger (Nokia), N. Sato (Oki), C. Burmeister, and J. Rey (Matsushita). Burmeister's affiliation isn't listed in the header. The header doesn't say it obsoletes or updates any other RFCs, so this one stands on its own. 💅
Status of This Memo ✨
This document specifies an Internet standards track protocol for the Internet community, and it asks for discussion and suggestions for improvements. For the standardization state and status of this protocol, check the current edition of the "Internet Official Protocol Standards" (STD 1). Distribution of this memo is unlimited. 📜
Copyright Notice 🔒
Copyright (C) The Internet Society (2006).
Abstract 🌸
Real-time media streams that use RTP are, to some degree, resilient against packet losses. Receivers can use the base mechanisms of RTCP to report packet reception statistics, which lets a sender adjust its transmission behavior over the mid-term. That's the only built-in way to get feedback and do feedback-based error repair, apart from a few codec-specific mechanisms 🙈
This document extends the Audio-visual Profile (AVP) so receivers can send feedback to senders more quickly. That means short-term adaptation and efficient feedback-based repair can actually happen. This early feedback profile (AVPF) keeps the AVP bandwidth constraints for RTCP and stays scalable to large groups. It's giving efficiency and it's giving scale 🦋
Table of Contents 🧚
1. Introduction ....................................................3
1.1. Definitions ................................................3
1.2. Terminology ................................................5
2. RTP and RTCP Packet Formats and Protocol Behavior ...............6
2.1. RTP ........................................................6
2.2. Underlying Transport Protocols .............................6
3. Rules for RTCP Feedback .........................................7
3.1. Compound RTCP Feedback Packets .............................7
3.2. Algorithm Outline ..........................................8
3.3. Modes of Operation .........................................9
3.4. Definitions and Algorithm Overview ........................11
3.5. AVPF RTCP Scheduling Algorithm ............................14
3.5.1. Initialization .....................................15
3.5.2. Early Feedback Transmission ........................15
3.5.3. Regular RTCP Transmission ..........................18
3.5.4. Other Considerations ...............................19
3.6. Considerations on the Group Size ..........................20
3.6.1. ACK Mode ...........................................20
3.6.2. NACK Mode ..........................................20
3.7. Summary of Decision Steps .................................22
3.7.1. General Hints ......................................22
3.7.2. Media Session Attributes ...........................22
4. SDP Definitions ................................................23
4.1. Profile Identification ....................................23
4.2. RTCP Feedback Capability Attribute ........................23
4.3. RTCP Bandwidth Modifiers ..................................27
4.4. Examples ..................................................27
5. Interworking and Coexistence of AVP and AVPF Entities ..........29
6. Format of RTCP Feedback Messages ...............................31
6.1. Common Packet Format for Feedback Messages ................32
6.2. Transport Layer Feedback Messages .........................34
6.2.1. Generic NACK .......................................34
6.3. Payload-Specific Feedback Messages ........................35
6.3.1. Picture Loss Indication (PLI) ......................36
6.3.2. Slice Loss Indication (SLI) ........................37
6.3.3. Reference Picture Selection Indication (RPSI) ......39
6.4. Application Layer Feedback Messages .......................41
7. Early Feedback and Congestion Control ..........................41
8. Security Considerations ........................................42
9. IANA Considerations ............................................43
10. Acknowledgements ..............................................47
11. References ....................................................48
11.1. Normative References .....................................48
11.2. Informative References ...................................481. Introduction ✨
Real-time media streams that use RTP are, to some degree, resilient against packet losses. RTP [1] has all the mechanisms needed to restore the ordering and timing from the sender, so a recipient can properly reproduce a media stream. RTP also gives continuous feedback about overall reception quality from all receivers. That lets senders adapt their coding scheme and transmission behavior over the mid-term (on the order of several seconds to minutes) to the quality of service (QoS) they're actually seeing on the network. 🧠
Here's the catch though: except for a few payload-specific mechanisms [6], RTP doesn't offer timely feedback that would let a sender repair the media right away. Think retransmissions, retroactive Forward Error Correction (FEC) control, or media-specific tricks for some video codecs, like reference picture selection. Not it 😭
Error resilience tools that already exist with RTP include audio redundancy coding [13], video redundancy coding [14], RTP-level FEC [11], and general advice on making media stream transmission more robust [12]. These can be applied proactively, which does increase the bandwidth of a given media stream. Or, in sufficiently small groups with small round-trip times (RTTs), senders can repair on-demand, using those mechanisms and/or approaches specific to the media encoding. Just note that "small group" and "sufficiently small RTT" both depend heavily on the application 🌈
This document specifies a modified RTP profile for audio and video conferences with minimal control, built on [1] and [2]. It makes two changes. First, to get timely feedback, it introduces Early RTCP messages, plus algorithms that allow low-delay feedback in small multicast groups while preventing feedback implosion in large ones. Special attention goes to point-to-point scenarios. Second, it defines a small set of general-purpose feedback messages, plus a format for codec- and application-specific feedback information, to be carried in RTCP payloads. Main character energy 👑
1.1. Definitions 📖
The definitions from RTP/RTCP [1] and the "RTP Profile for Audio and Video Conferences with Minimal Control" [2] apply. This document also uses the following definitions:
Early RTCP mode: A mode of operation in which a receiver of a media stream is often (but not always) able to report events of interest back to the sender close to when they happen. In Early RTCP mode, RTCP packets are transmitted according to the timing rules defined in this document.
Early RTCP packet: A packet sent earlier than the scheduling algorithm of [1] would allow, because a receiver observed an "event." Early RTCP packets may be sent in Immediate Feedback mode and in Early RTCP mode. Sending an Early RTCP packet is also called sending Early Feedback in this document.
Event: An observation made by the receiver of a media stream that is (potentially) of interest to the sender, such as a packet loss or packet reception, a frame loss, and so on. It's useful to report these back to the sender in a feedback message. 💅
Feedback (FB) message: An RTCP message, as defined in this document, that carries information about events observed at a receiver back to the sender of the media stream. It complements the long-term receiver status information carried in RTCP receiver reports (RRs). For clarity, this document refers to a feedback message as an FB message throughout.
Feedback (FB) threshold: The FB threshold marks the transition between Immediate Feedback mode and Early RTCP mode. In a multiparty scenario, it's the maximum group size at which, on average, each receiver can report each event back to the sender(s) right away, meaning with an Early RTCP packet and without waiting for its regularly scheduled RTCP interval. The threshold depends heavily on the type of feedback, network QoS (for example, packet loss probability and distribution), the codec and packetization scheme in use, the session bandwidth, and application requirements. The algorithms don't require all senders and receivers to agree on the same value. It's only there to give application designers some conceptual guidance, and it isn't used in any calculations. For clarity, this document refers to the feedback threshold as the FB threshold throughout. 🎀
Immediate Feedback mode: A mode of operation in which each receiver of a media stream is, statistically, able to report each event of interest back to the media stream sender right away. In Immediate Feedback mode, RTCP FB messages are transmitted according to the timing rules defined in this document.
Media packet: An RTP packet.
Regular RTCP mode: A mode of operation in which preferred transmission of FB messages is not allowed. Instead, RTCP messages are sent following the rules of [1]. Even so, those RTCP messages may still contain feedback information as defined in this document.
Regular RTCP packet: An RTCP packet that is not sent as an Early RTCP packet.
RTP sender: An RTP entity that transmits media packets and RTCP packets, and that receives both Regular and Early RTCP (that is, feedback) packets. The RTP sender is a logical role, so the same RTP entity can also act as an RTP receiver at the same time. 🦋
RTP receiver: An RTP entity that receives media packets and RTCP packets, and that transmits both Regular and Early RTCP (that is, feedback) packets. The RTP receiver is a logical role, so the same RTP entity can also act as an RTP sender at the same time.
1.2. Terminology 🍓
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [5]. Those are the official BCP 14 vibes, so read them literally, periodt 🫶
(Part 1 of 12. More coming up, bestie 💜)
2. RTP and RTCP Packet Formats and Protocol Behavior 🎀
2.1. RTP 💖
okay so the rules in [2] also apply to this profile ✨, EXCEPT for the ones listed below. no cap, read these carefully bestie 🧐
RTCP packet types 📦: two new RTCP packet types get registered, and the matching FB messages that carry feedback info are defined in Section 6 of this memo. so cute 🦋
RTCP report intervals ⏱️: this doc describes three modes that change the RTCP report intervals (see Section 3.2 of this memo) 🌸. In Regular RTCP mode, all the rules from [1] apply, except for the recommended minimal interval of five seconds between two RTCP reports from the same RTP entity. In both Immediate Feedback and Early RTCP modes, that five-second minimum gets dropped 💅. And ALSO, if RTCP packets with FB messages (defined in Section 4 of this memo) are going to be sent, the rules in Section 3 of this memo kick in too 🔥.
the rules in [1] can be overridden by session descriptions that specify different parameters 🫶 (like the bandwidth share given to RTCP for senders vs. receivers). For sessions set up with the Session Description Protocol (SDP) [3], the rules of [4] apply. it's giving organized 📋
Congestion control 🚦: the same basic rules from [2] apply. On top of that, Section 7 looks at how feedback affects things, and how a sender reacts to FB messages. ✨
2.2. Underlying Transport Protocols 🦋
RTP is meant to run on top of unreliable transport protocols, like UDP and the Datagram Congestion Control Protocol (DCCP) 🌈. This section gives a quick rundown of what changes beyond plain RTP once RTCP feedback (from this memo) is in the mix 💜.
UDP 📮: UDP gives best-effort delivery of datagrams for point-to-point AND multicast communications. It does NOT do congestion control or error repair 😭. RTCP-based feedback from this memo can give minimal support for limited error repair, but RTCP feedback isn't guaranteed to work on really small timescales (on the order of RTT) 💀, so it's not suitable for congestion control. This memo covers both unicast and multicast operation 🌸.
DCCP 🧚: DCCP [19] gives congestion-controlled but unreliable datagram flows for unicast communications. With TCP Friendly Rate Control (TFRC)-based [20] congestion control (CCID 3), DCCP is especially good for audio and video calls 🔥. DCCP's acknowledgement messages can give detailed feedback about which datagrams were received and which were missed (and so, about congestion) 👑.
When you run RTP over DCCP, congestion control happens at the DCCP layer, so the RTP layer doesn't need any extra mechanisms 🫶. Also, a sender that supports RTCP feedback can use the more frequent DCCP-based feedback, which means a receiver can skip sending (extra) Generic Feedback messages where that makes sense 🍓.
3. Rules for RTCP Feedback 💅
3.1. Compound RTCP Feedback Packets 🎀
Two parts make up RTCP-based feedback in this doc 💜:
- Status reports live inside sender report (SR)/received report (RR) packets. They get sent at regular intervals as part of compound RTCP packets (which also include source description (SDES) and maybe other messages). These status reports give an overall picture of how well a media stream is being received 📊.
- FB messages (defined in this doc) say which pieces of a media stream were lost or received (or give some other fairly immediate feedback on the data that came in) ⚡. The rules for sending FB messages are brand new in this doc 🌟.
RTCP FB messages are just another RTCP packet type (see Section 4) 🦋. So multiple FB messages MAY be combined in a single compound RTCP packet, and they MAY also be sent together with other RTCP packets ✨.
Compound RTCP packets that contain FB messages (as defined in this doc) MUST include RTCP packets in the order defined in [1] 📋:
- OPTIONAL encryption prefix, which MUST be present if the RTCP packet(s) are going to be encrypted according to Section 9.1 of [1] 🔐.
- mandatory SR or RR 💖.
- mandatory SDES, which MUST contain the CNAME item. All other SDES items are OPTIONAL 🌸.
- One or more FB messages 🔥.
The FB message(s) MUST be placed in the compound packet AFTER the RR and SDES RTCP packets defined in [1] 🧚. The ordering with respect to other RTCP extensions is not defined 🤷♀️.
There are two types of compound RTCP packets that carry feedback packets in this doc 👑:
a) Minimal compound RTCP feedback packet 🍓
A minimal compound RTCP feedback packet MUST contain only the mandatory info listed above: the encryption prefix if it's needed, exactly one RR or SR, exactly one SDES with only the CNAME item present, and the FB message(s) 🫶. The point is to keep the RTCP packet small, so feedback can go out as often as possible while still staying within the RTCP bandwidth limits 🌈.
This packet format SHOULD be used whenever an RTCP FB message is sent as part of an Early RTCP packet 🦋. In this doc, this packet type is called the minimal compound RTCP packet 💜.
b) (Full) compound RTCP feedback packet 🌙
A (full) compound RTCP feedback packet MAY contain any additional number of RTCP packets (more RRs, extra SDES items, etc.) ✨. The ordering rules above MUST still be followed 💅.
This packet format MUST be used whenever an RTCP FB message is sent as part of a Regular RTCP packet or in Regular RTCP mode 🔥. It MAY ALSO be used to send RTCP FB messages in Immediate Feedback or Early RTCP mode 🌸. In this doc, this packet type is called the full compound RTCP packet 🎀.
RTCP packets that DON'T contain FB messages are called non-FB RTCP packets 🙈. Those packets MUST follow the format rules in [1] 😭.
3.2. Algorithm Outline 🧚
FB messages are part of the RTCP control streams, so they're subject to the RTCP bandwidth constraints 💜. This means it may not always be possible to report an event seen at a receiver straight back to the sender ⏳. But feedback usually loses value over time 📉, both in terms of the media quality the user perceives at the receiving end and/or the cost of repairing the media stream 💀.
RTP [1] and the commonly used RTP profile [2] define the rules for when compound RTCP packets should be sent. This doc changes those rules so applications can report events (like loss or reception of RTP packets) in a timely way, and so algorithms that use FB messages can work 🦋.
Here's the modified RTCP transmission algorithm in a nutshell ✨: as long as no FB messages need to be conveyed, compound RTCP packets are sent following the rules of RTP [1], EXCEPT that the five-second minimum interval between RTCP reports isn't enforced 🌈. So the interval between RTCP reports is only derived from the average RTCP packet size and the RTCP bandwidth share available to the RTP/RTCP entity 🍓. optionally, a minimum interval between Regular RTCP packets may be enforced 💅.
If a receiver notices it needs to send an FB message, it may send it EARLIER than its next regular RTCP reporting interval (the one it would've been scheduled for under the regular RTCP algorithm above) 🔥. To avoid feedback implosion in multiparty sessions, feedback suppression is used 🫶: the receiver waits for a (short) random dithering interval to check whether it sees a matching FB message from any other receiver reporting the same event 🎲. Note that for point-to-point sessions there's NO such delay 🙅♀️.
If a matching FB message from another member shows up, this receiver holds off on sending its own FB message and keeps following the Regular RTCP transmission schedule 🌸. If the receiver hasn't seen a matching FB message from any other member yet, it checks whether it's allowed to send Early feedback 💖. If Early feedback is permitted, the receiver sends the FB message as part of a minimal compound RTCP packet 👑. Whether it's allowed depends on the type of the previous RTCP packet this receiver sent, and on when the previous Early feedback message went out ⏰.
FB messages can also be sent as part of full compound RTCP packets, which are transmitted as per [1] (except for the five-second lower bound) at regular intervals 🌈✨
3.3. Modes of Operation 🦋
okay so RTCP-based feedback can run in one of three modes ✨ (see Figure 1 below). The mode is basically just a vibe check on whether the receiver can, on average, tell the sender about every event in a timely way 💅. It does not change how the FB messages get scheduled, so don't mix those up bestie 🙈
And depending on reception quality and what each receiver locally monitors about the RTP session, individual receivers don't have to agree on one shared view of the current mode 🫶
a) Immediate Feedback mode: here the group size is below the FB threshold, which means every receiving party has enough bandwidth to send the RTCP feedback packets for their purpose 🍓 So each receiver has room to report every event with a basically instant RTCP feedback packet ⚡
The group size threshold depends on a bunch of stuff, including (but not necessarily limited to) the type of feedback in use (e.g., ACK vs. NACK), bandwidth, packet rate, packet loss probability and distribution, media type, codec, and the (worst case or observed) frequency of events to report (e.g., frame received, packet lost) 🧚
Here's a rough estimate 👀 Let N be the average number of events a receiver reports per interval T, B the RTCP bandwidth fraction for that particular receiver, and R the average RTCP packet size. Then the receiver stays in Immediate Feedback mode as long as N<=B*T/R 💜
b) Early RTCP mode: now the group size and other parameters no longer let each receiver react to every event that's worth reporting (or that needed reporting) 😭 But feedback can still be sent often enough that the sender can adapt the media stream transmission and boost the overall media playback quality 🌸
Using the notation above, Early RTCP mode can be roughly described by N > B*T/R as a "lower bound" 🎀 An estimate for the upper bound is trickier. If you set N=1, you get T = R/B for a given R and B, which is the average interval between events to be reported 🔥 That number can be used as a hint for figuring out whether early transmission of RTCP packets is useful ✨
c) Regular RTCP Mode: from some group size upwards, feedback for individual events from receivers just isn't useful anymore 💀 That's either because of the time scale the feedback could arrive on, and/or because in big groups the sender(s) have no real chance to react to individual feedback 🌈
No precise group size threshold can be pinned down for where this mode starts. Obviously though, that boundary lines up with the upper bound of Early RTCP mode from item b) above 👑
The feedback algorithm in this document scales smoothly, so the participants don't need to agree on the exact values of the respective FB thresholds within the group 🫶 That means the borders between all these modes are soft 🦋
ACK
feedback
V
:<- - - - NACK feedback - - - ->//
:
: Immediate ||
: Feedback mode ||Early RTCP mode Regular RTCP mode
:<=============>||<=============>//<=================>
: ||
-+---------------||---------------//------------------> group size
2 ||
Application-specific FB Threshold
= f(data rate, packet loss, codec, ...)
Figure 1: Modes of operationAs I said before, the respective FB thresholds depend on a number of technical parameters (of the codec, the transport, the type of feedback used, etc.) but also on the application scenarios themselves 🧠 Section 3.6 has some useful hints (but no precise calculations) on estimating these thresholds 💖
3.4. Definitions and Algorithm Overview
okay so this is the part where the RFC gets super organised about all the state each receiver has to keep track of 🎀 quick reminder: every one of these variables gets worked out independently at each receiver, so their local values can totally differ at any moment ✨ (except item h, which is the odd one out 👀)
a) Let "senders" be the number of active senders in the RTP session.
b) Let "members" be the current estimate of the number of receivers in the RTP session.
c) Let tn and tp be the time for the next (last) scheduled RTCP RR transmission calculated prior to timer reconsideration.
d) Let Tmin be the minimal interval between RTCP packets as per [1]. Unlike in [1], the initial Tmin is set to 1 second to allow for some group size sampling before sending the first RTCP packet. After the first RTCP packet is sent, Tmin is set to 0. 🌸
e) Let T_rr be the interval after which, having just sent a regularly scheduled RTCP packet, a receiver would schedule the transmission of its next Regular RTCP packet. This value is obtained following the rules of [1] but with Tmin as defined in this document: T_rr = T (the "calculated interval" as defined in [1]) with tn = tp + T. T_rr always refers to the last value of T that has been computed (because of reconsideration or to determine tn). T_rr is also referred to as Regular RTCP interval in this document. 💅
f) Let t0 be the time at which an event that is to be reported is detected by a receiver. 👀
g) Let T_dither_max be the maximum interval for which an RTCP feedback packet MAY be additionally delayed to prevent implosions in multiparty sessions; the value for T_dither_max is dynamically calculated based upon T_rr (or may be derived by means of another mechanism common across all RTP receivers to be specified in the future). For point-to-point sessions (i.e., sessions with exactly two members with no change in the group size expected, e.g., unicast streaming sessions), T_dither_max is set to 0. 🦋
h) Let T_max_fb_delay be the upper bound within which feedback to an event needs to be reported back to the sender to be useful at all. This value is application specific, and no values are defined in this document. 🧚
i) Let te be the time for which a feedback packet is scheduled. 🍓
j) Let T_fd be the actual (randomized) delay for the transmission of FB message in response to an event at time t0. 💜
k) Let allow_early be a Boolean variable that indicates whether the receiver currently may transmit FB messages prior to its next regularly scheduled RTCP interval tn. This variable is used to throttle the feedback sent by a single receiver. allow_early is set to FALSE after Early feedback transmission and is set to TRUE as soon as the next Regular RTCP transmission takes place. 🫶
l) Let avg_rtcp_size be the moving average on the RTCP packet size as defined in [1]. 🌈
m) Let T_rr_interval be an OPTIONAL minimal interval to be used between Regular RTCP packets. If T_rr_interval == 0, then this variable does not have any impact on the overall operation of the RTCP feedback algorithm. If T_rr_interval != 0, then the next Regular RTCP packet will not be scheduled T_rr after the last Regular RTCP transmission (i.e., at tp+T_rr). Instead, the next Regular RTCP packet will be delayed until at least T_rr_interval after the last Regular RTCP transmission, i.e., it will be scheduled at or later than tp+T_rr_interval. Note that T_rr_interval does not affect the calculation of T_rr and tp; instead, Regular RTCP packets scheduled for transmission before tp+T_rr_interval will be suppressed if, for example, they do not contain any FB messages. The T_rr_interval does not affect transmission scheduling of Early RTCP packets. 🎀
Note: Providing T_rr_interval as an independent variable is meant to minimize Regular RTCP feedback (and thus bandwidth consumption) as needed by the application while additionally allowing the use of more frequent Early RTCP packets to provide timely feedback. This goal could not be achieved by reducing the overall RTCP bandwidth as RTCP bandwidth reduction would also impact the frequency of Early feedback. 💖
n) Let t_rr_last be the point in time at which the last Regular RTCP packet has been scheduled and sent, i.e., has not been suppressed due to T_rr_interval. 🌸
o) Let T_retention be the time window for which past FB messages are stored by an AVPF entity. This is to ensure that feedback suppression also works for entities that have received FB messages from other entities prior to noticing the feedback event itself. T_retention MUST be set to at least 2 seconds. 🔥
p) Let M*Td be the timeout value for a receiver to be considered inactive (as defined in [1]). ✨
The feedback situation for an event to report at a receiver is depicted in Figure 2 below. At time t0, such an event (e.g., a packet loss) is detected at the receiver. The receiver decides -- based upon current bandwidth, group size, and other application-specific parameters -- that an FB message needs to be sent back to the sender. 🦋
To avoid an implosion of feedback packets in multiparty sessions, the receiver MUST delay the transmission of the RTCP feedback packet by a random amount of time T_fd (with the random number evenly distributed in the interval [0, T_dither_max]). Transmission of the compound RTCP packet MUST then be scheduled for te = t0 + T_fd. 💅
The T_dither_max parameter is derived from the Regular RTCP interval, T_rr, which, in turn, is based upon the group size. A future document may also specify other calculations for T_dither_max (e.g., based upon RTT) if it can be assured that all RTP receivers will use the same mechanism for calculating T_dither_max. 🌈
For a certain application scenario, a receiver may determine an upper bound for the acceptable local delay of FB messages: T_max_fb_delay. If an a priori estimation or the actual calculation of T_dither_max indicates that this upper bound MAY be violated (e.g., because T_dither_max > T_max_fb_delay), the receiver MAY decide not to send any feedback at all because the achievable gain is considered insufficient. 🙈
If an Early RTCP packet is scheduled, the time slot for the next Regular RTCP packet MUST be updated accordingly to have a new tn (tn=tp+2*T_rr) and a new tp (tp=tp+T_rr) afterwards. This is to ensure that the short-term average RTCP bandwidth used with Early feedback does not exceed the bandwidth used without Early feedback. 🫶
event to
report
detected
|
| RTCP feedback range
| (T_max_fb_delay)
vXXXXXXXXXXXXXXXXXXXXXXXXXXX ) )
|---+--------+-------------+-----+------------| |--------+--->
| | | | ( ( |
| t0 te |
tp tn
\_______ ________/
\/
T_dither_max
Figure 2: Event report and parameters for Early RTCP scheduling3.5. AVPF RTCP Scheduling Algorithm ✨
okay now we get to the actual scheduling algorithm, bestie 🧚 Let S0 be an active sender (out of S senders) and let N be the number of receivers with R being one of these receivers.
Assume that R has verified that using feedback mechanisms is reasonable at the current constellation (which is highly application specific and hence not specified in this document). 💖
Assume further that T_rr_interval is 0, if no minimal interval between Regular RTCP packets is to be enforced, or T_rr_interval is set to some meaningful value, as given by the application. This value then denotes the minimal interval between Regular RTCP packets. 🎀
With this, a receiver R MUST use the following rules for transmitting one or more FB messages as minimal or full compound RTCP packet. 👑
3.5.1. Initialization 🌸
Initially, R MUST set allow_early = TRUE and t_rr_last = NaN (Not-a-Number, i.e., some invalid value that can be distinguished from a valid time).
Furthermore, the initialization of the RTCP variables as per [1] applies except for the initial value for Tmin. For a point-to-point session, the initial Tmin is set to 0. For a multiparty session, Tmin is initialized to 1.0 seconds. 🦋
3.5.2. Early Feedback Transmission ✨
Okay bestie, set the scene 🎬 R already has its last Regular RTCP RR packet lined up for tp (sent or suppressed right then 💅) and the next one is booked for tn = tp + T_rr (possibly reconsidered first, per [1]) 📅. The last Regular RTCP packet went out at t_rr_last. 🦋
So here's the Early Feedback algorithm, step by step:
Step 1. At time t0, R notices it needs to send one or more FB messages. Maybe media "units" need ACKing or NACKing 🙏 and the sender would actually find the feedback useful. 🔥
Step 2. R checks first whether there's already a compound RTCP packet with FB messages scheduled (either an Early or a Regular one) 👀
2a) If there is, the new FB message MUST be included in that scheduled packet, and the scheduling of the waiting compound RTCP packet MUST remain unchanged 🤏 While doing that, the available feedback information SHOULD be merged so there are as few FB messages as possible ✨ That's the end of the immediate actions. 💖
2b) If no compound RTCP packet is scheduled yet, a new compound RTCP packet (minimal or full) MUST be created, and the minimal interval for T_dither_max MUST be chosen like this:
- i) If the session is a point-to-point session, then:
T_dither_max = 0.- ii) If the session is a multiparty session, then:
T_dither_max = l * T_rrwith l=0.5.
T_dither_max MAY be calculated some other way (for example based on RTT 🧠), but then that method MUST be specified in a future document. That future spec MUST make sure every RTP receiver uses the same mechanism to calculate T_dither_max. Iconic consistency energy 💅
The values above for T_dither_max are just minimum values. Application-specific feedback considerations might make it worth raising T_dither_max beyond that, and that's totally up to the implementer 🌸
Step 3. Then R MUST check whether its next Regular RTCP packet would land within the time bounds of the Early RTCP packet triggered at t0, meaning whether t0 + T_dither_max > tn 🕰️
3a) If it would, an Early RTCP packet MUST NOT be scheduled. Instead, the FB message(s) MUST be stored so they get included in the Regular RTCP packet scheduled for tn. And that's the end of the immediate actions. 🫶
3b) Otherwise, the following steps happen.
Step 4. R MUST check whether it's allowed to send an Early RTCP packet, i.e., whether allow_early == TRUE or not 🚦
4a) If allow_early == FALSE, then R MUST check when the next scheduled Regular RTCP packet is:
- 1. If
tn - t0 < T_max_fb_delay, the feedback could still be useful to the sender even though it's late 🥹 So R MAY create an RTCP FB message to go in the Regular RTCP packet sent attn. - 2. Otherwise, R MUST discard the RTCP FB message. 💀
That's the immediate course of action, done. ✨
4b) If allow_early == TRUE, then R MUST schedule an Early RTCP packet for te = t0 + RND * T_dither_max, where RND is a pseudo random function evenly distributed between 0 and 1 🎲
Step 5. R MUST detect overlaps between FB messages received from other RTP session members and the FB messages R wants to send 🕵️♀️ So while R is a member of the RTP session, it MUST keep watching for (minimal) compound RTCP packets and store every FB message in them for at least T_retention. When R schedules its own FB message after steps 1 through 4, it MUST check every stored and newly received FB message from RTCP packets received during the interval [t0 - T_retention ; te] and act like this:
5a) If R understands the semantics of the received FB message and the message contents is a superset of the feedback R wanted to send, then R MUST discard its own FB message and MUST re-schedule the next Regular RTCP packet transmission for tn (as calculated before). Bestie, that's a full suppression moment 🫠✨
5b) If R understands the semantics of the received FB message and the message contents is NOT a superset of the feedback R wanted to send, then R SHOULD transmit its own FB message as scheduled 📤 If there's overlap between the feedback to send and the feedback received, how much feedback gets sent is up to R. R MAY leave its feedback unchanged, and R MAY also eliminate any redundancy between its own feedback and what it's already received from other session members. 🧹
5c) If R does NOT understand the semantics of the received FB message, R MAY keep its own FB message scheduled as an Early RTCP packet, OR R MAY re-schedule the next Regular RTCP packet transmission for tn (as calculated before) and MAY append the FB message to the now regularly scheduled RTCP message. 🤷♀️
Note: With 5c), receiving unknown FB messages may not lead to feedback suppression at a particular receiver. As a consequence, a given event may cause M different types of FB messages (which are all appropriate but not mutually understood) to be scheduled, so that a "large" receiver group may effectively be partitioned into at most M groups. Among members of each of these M groups, feedback suppression will occur following 5a and 5b but no suppression will happen across groups. As a result, O(M) RTCP FB messages may be received by the sender. Hence, there is a chance for a very limited feedback implosion. However, as sender(s) and all receivers make up the same application using the same (set of) codecs in the same RTP session, only little divergence in semantics for FB messages can safely be assumed and, therefore, M is assumed to be small in the general case. 🧩
Given further that the O(M) FB messages are randomly distributed over a time interval of T_dither_max, we find that the resulting limited number of extra compound RTCP packets (a) is assumed not to overwhelm the sender and (b) should be conveyed as all contain complementary pieces of information. 🌈
Step 6. If R's FB message(s) wasn't suppressed by other receivers' FB messages under step 5, then when te is reached, R MUST transmit the (minimal) compound RTCP packet containing its FB message(s) 🚀 After that, R MUST set allow_early = FALSE, MUST recalculate tn = tp + 2*T_rr, and MUST set tp to the previous tn. As soon as the newly calculated tn is reached, whether R sends its next Regular RTCP packet or suppresses it because of T_rr_interval, it MUST set allow_early = TRUE again. Full circle, girl 🔄💖
3.5.3. Regular RTCP Transmission ✨
Full compound RTCP packets MUST be sent at regular intervals 📆 These packets MAY also carry one or more FB messages. Regular RTCP packets get scheduled like this:
If T_rr_interval == 0, then the transmission MUST follow the rules in Sections 3.2 and 3.4 of this document and MUST adhere to the adjustments of tn in Section 3.5.2 (aka skip one regular transmission if an Early RTCP packet transmission already happened 🙃). Timer reconsideration happens when tn is reached, per [1]. The Regular RTCP packet is sent after timer reconsideration. Whenever a Regular RTCP packet is sent or suppressed, allow_early MUST be set to TRUE, and tp and tn MUST be updated as per [1]. After the first transmission of a Regular RTCP packet, Tmin MUST be set to 0. 🎀
If T_rr_interval != 0, then the transmission times MUST be calculated following the rules in Sections 3.2 and 3.4 of this document and MUST adhere to the adjustments of tn in Section 3.5.2 (again, skip one regular transmission if an Early RTCP transmission happened). Timer reconsideration happens when tn is reached, per [1]. After timer reconsideration, these actions are taken:
- 1. If no Regular RTCP packet has been sent before (i.e., if
t_rr_last == NaN), then a Regular RTCP packet MUST be scheduled. Stored FB messages MAY be included in it. After the scheduled packet is sent,t_rr_lastMUST be set totn.TminMUST be set to 0. - 2. Otherwise, a temporary value
T_rr_current_intervalgets calculated like this:
T_rr_current_interval = RND*T_rr_intervalwith RND being a pseudo random function evenly distributed between 0.5 and 1.5. This dithered value decides which of the following alternatives happens 🎰
2a) If t_rr_last + T_rr_current_interval <= tn, then a Regular RTCP packet MUST be scheduled. Stored RTCP FB messages MAY be included in it. After the scheduled packet is sent, t_rr_last MUST be set to tn.
2b) If t_rr_last + T_rr_current_interval > tn and RTCP FB messages have been stored and are waiting to be sent, an RTCP packet MUST be scheduled for transmission at tn. This RTCP packet MAY be minimal or Regular (the implementer decides 🧑💻), and the compound RTCP packet MUST include the stored RTCP FB message(s). t_rr_last MUST remain unchanged.
2c) Otherwise (if t_rr_last + T_rr_current_interval > tn but no stored RTCP FB messages are waiting to be sent), the compound RTCP packet MUST be suppressed (i.e., it MUST NOT be scheduled). t_rr_last MUST remain unchanged. 😴
In all four cases above (1, 2a, 2b, and 2c), allow_early MUST be set to TRUE (possibly after sending the Regular RTCP packet), and tp and tn MUST be updated following the rules of [1] except for the five second minimum. ⏱️✨
3.5.4. Other Considerations 🧠
If T_rr_interval != 0, then the timeout calculation for RTP/AVPF entities (Section 6.3.5 of [1]) MUST be modified to use T_rr_interval instead of Tmin when computing Td (and therefore M*Td) for timing out RTP entities. 💅
Whenever a compound RTCP packet is sent or received (minimal or full compound, Early or Regular), the avg_rtcp_size variable MUST be updated accordingly (see [1]), and later computations of tn MUST use the new avg_rtcp_size. 📊
3.6. Considerations on the Group Size 👑
This section gives some guidelines on the group sizes at which the various feedback modes may be used. 🦋
3.6.1. ACK Mode 🦋
okay bestie, ACK mode is kinda intense 😭 the RTP session MUST have exactly two members and this group size MUST NOT grow, so it has to be point-to-point communications 💖 unicast addresses SHOULD be used in the session description ✨
for one-way AND two-way chats between two people, 2.5% of the RTP session bandwidth is available for RTCP traffic from the receivers, feedback included 🎀 for a 64-kbit/s stream that's 1,600 bit/s for RTCP. if we assume an average of 96 bytes (=768 bits) per RTCP packet, a receiver can report 2 events per second back to the sender 🌸 if acknowledgements for 10 events get collected in each FB message, then 20 events can be acknowledged per second, which is giving main character energy 👑 at 256 kbit/s, 8 events could be reported per second, so the ACKs may be sent in a finer granularity (e.g., only combining three ACKs per FB message) 🧚
from 1 Mbit/s upwards, a receiver could acknowledge each individual frame (not packet!) in a 30-fps video stream 😭🔥
ACK strategies MUST be defined to work properly with these bandwidth limitations 💅 an indication of whether or not ACKs are allowed for a session and, if so, which ACK strategy should be used, MAY be conveyed by out-of-band mechanisms, e.g., media-specific attributes in a session description using SDP 🫶
3.6.2. NACK Mode 💀
negative acknowledgements (and the other types of feedback with similar reporting characteristics) MUST be used for all sessions with a group size that may grow larger than two 🙈 of course, NACKs MAY be used for point-to-point communications as well ✨
whether or not Early RTCP packets should be considered depends on a bunch of parameters, including session bandwidth, codec, the special type of feedback, and the number of senders and receivers 🦋
the most important parameters when picking the mode of operation are the allowed minimal interval between two compound RTCP packets (T_rr) and the average number of events that probably need reporting per time interval (plus how they're spread out over time, obviously) 🍓 the minimum interval can be derived from the available RTCP bandwidth and the expected average size of an RTCP packet. the number of events to report (e.g., per second) may be derived from the packet loss rate and the sender's rate of sending packets. from these two values, the allowable group size for the Immediate Feedback mode can be calculated 🧮
as stated in Section 3.3:
Let N be the average number of events to be reported per interval T by a receiver, B the RTCP bandwidth fraction for this particular receiver, and R the average RTCP packet size, then the receiver operates in Immediate Feedback mode as long as N<=B*T/R.
the upper bound for Early RTCP mode then depends only on how much quality degradation is acceptable, i.e., how many events per time interval may go unreported 🥺
as stated in Section 3.3:
Using the above notation, Early RTCP mode can be roughly characterized byN > B*T/Ras "lower bound". An estimate for an upper bound is more difficult. SettingN=1, we obtain for a given R and B the intervalT = R/Bas average interval between events to be reported. This information can be used as a hint to determine whether or not early transmission of RTCP packets is useful.
Example 🎀: if a 256-kbit/s video at 30 fps gets sent through a network with an MTU of about 1,500 bytes, then in most cases each frame fits into one packet, which means a packet rate of 30 packets per second 😭 if 5% packet loss happens in the network (equally distributed, no inter-dependence between receivers), then each receiver will, on average, have to report 3 packets lost every two seconds 💀 assuming a single sender and more than three receivers, this gives 3.75% of the RTCP bandwidth allocated to the receivers, so 9.6 kbit/s 🍓 assuming further an average compound RTCP packet size of 120 bytes allows 10 RTCP packets to be sent per second, or 20 in two seconds ⭐ if every receiver needs to report three lost packets per two seconds, the maximum group size is 6-7 receivers if all loss events get reported. the rules for transmitting Early RTCP packets should give enough flexibility for most of this reporting to happen in a timely way 🫶
extending this example to find the upper bound for Early RTCP mode could lead to these thoughts 🧠: assume the underlying coding scheme and the application (plus the tolerant users) allow on the order of one loss without repair per two seconds. then the number of packets each receiver has to report drops to two per two seconds, and the group size goes up to 10 🌈 assuming further that some packet losses are correlated, feedback traffic drops even more, and group sizes of around 12 to 16 (maybe even 20) can be reasonably well supported using Early RTCP mode ✨ note that all of these considerations are based on statistics and will fail to hold in some cases 💅
3.7. Summary of Decision Steps 🧚
3.7.1. General Hints 💖
before even thinking about whether to send RTCP feedback information, an application has to figure out whether this mechanism is applicable at all 🔥
- An application has to decide whether, for the current ratio of packet rate to the associated (application-specific) maximum feedback delay and the currently observed round-trip time (if available), feedback mechanisms can be applied at all.
This decision may be based on (and dynamically revised after) RTCP reception statistics, as well as out-of-band mechanisms.
- The application has to decide, for a certain observed error rate, assigned bandwidth, frame/packet rate, and group size, whether (and which) feedback mechanisms can be applied.
Regular RTCP reception statistics give valuable input to this step too 🌸
- If the application decides to send feedback, the application has to follow the rules for transmitting Early RTCP packets or Regular RTCP packets containing FB messages 🦋
- The type of RTCP feedback sent should not duplicate information the sender already has from a lower layer transport protocol. That is, if the transport protocol provides negative or positive acknowledgements about packet reception (such as DCCP), the receiver should avoid repeating the same information at the RTCP layer (i.e., abstain from sending Generic NACKs) 🙈
3.7.2. Media Session Attributes 🎀
media sessions are usually described using out-of-band mechanisms to pass transport addresses, codec information, etc., between sender(s) and receiver(s) ✨ that mechanism is two-fold: a format used to describe a media session, and another mechanism for transporting that description 🧚
in the IETF, the Session Description Protocol (SDP) is the one currently used to describe media sessions, while protocols like SIP, Session Announcement Protocol (SAP), Real Time Streaming Protocol (RTSP), and HTTP (among others) are used to convey the descriptions 🌈
a media session description format MAY include parameters to indicate that RTCP feedback mechanisms are supported in this session, and which of the feedback mechanisms MAY be applied 💜
to do so, the profile "AVPF" MUST be indicated instead of "AVP" 👑 further attributes may be defined to show which type(s) of feedback are supported 💅
Section 4 contains the syntax specification for supporting RTCP feedback with SDP. Similar specifications for other media session description formats are outside the scope of this document 🫶
4. SDP Definitions 🌸
this section defines a number of additional SDP parameters used to describe a session. all of these are defined as media-level attributes ✨
4.1. Profile Identification 🦋
the AV profile defined in [4] is referred to as "AVP" in the context of, e.g., the Session Description Protocol (SDP) [3]. the profile specified in this document is referred to as "AVPF" 💖
feedback information following the modified timing rules in this document MUST NOT be sent for a particular media session unless the description for that session indicates the use of the "AVPF" profile (exclusively or jointly with other AV profiles) 🔥
4.2. RTCP Feedback Capability Attribute ✨
okay so there's a brand new SDP attribute made just for RTCP feedback, and it's called a=rtcp-fb 💖. The rtcp-fb attribute MUST only be used as an SDP media attribute, and it MUST NOT be provided at the session level 🙅♀️. It also MUST only be used in media sessions where "AVPF" is specified 🎀.
The rtcp-fb attribute SHOULD be used to show which RTCP FB messages MAY be used in this media session for the indicated payload type 📣. A wildcard payload type ("*") MAY be used too, which means the feedback applies to every payload type 🌈. If you support a few kinds of feedback, or want the same feedback on only some of the payload types, you MUST use several a=rtcp-fb lines 💅.
If there's no rtcp-fb attribute, RTP receivers MAY still send feedback using other suitable RTCP feedback packets for their media type 🦋. But receivers MUST NOT rely on the RTP senders reacting to any of the FB messages 💀. The RTP sender MAY choose to ignore some feedback messages, and that's just life 🙈.
When one or more rtcp-fb attributes show up in a media session description, the RTCP receivers for the media session(s) containing the rtcp-fb have some house rules 📋:
- MUST ignore all
rtcp-fbattributes whose semantics they don't fully understand (i.e., where they do not understand the meaning of all values in thea=rtcp-fbline); - SHOULD provide feedback information as specified in this document using any of the RTCP feedback packets from one of the
rtcp-fbattributes for this media session; and - MUST NOT use other FB messages than those listed in one of the
rtcp-fbattribute lines.
Now for the offer/answer model [8] 💃. When it's in play, the offerer MAY present a set of these AVPF attributes to its peer. The answerer MUST remove all attributes it does not understand, plus the ones it doesn't support in general or doesn't wish to use in this particular media session. The answerer MUST NOT add feedback parameters to the media description, and MUST NOT alter values of such parameters. The answer is binding for the media session, so both offerer and answerer MUST only use feedback mechanisms negotiated this way 🔒. Both offerer and answerer MAY independently decide to send RTCP FB messages of only a subset of the negotiated feedback mechanisms, but they SHOULD react properly to all types of the negotiated FB messages when received 🔥.
RTP senders MUST be prepared to receive any kind of RTCP FB messages, and MUST silently discard all those RTCP FB messages that they do not understand 🤫.
Here's the syntax for the rtcp-fb attribute. The feedback types and optional parameters are all case sensitive, so no sneaky capitals 👀.
(In the following ABNF, fmt, SP, and CRLF are used as defined in [3].)
rtcp-fb-syntax = "a=rtcp-fb:" rtcp-fb-pt SP rtcp-fb-val CRLF
rtcp-fb-pt = "*" ; wildcard: applies to all formats
/ fmt ; as defined in SDP spec
rtcp-fb-val = "ack" rtcp-fb-ack-param
/ "nack" rtcp-fb-nack-param
/ "trr-int" SP 1*DIGIT
/ rtcp-fb-id rtcp-fb-param
rtcp-fb-id = 1*(alpha-numeric / "-" / "_")
rtcp-fb-param = SP "app" [SP byte-string]
/ SP token [SP byte-string]
/ ; empty
rtcp-fb-ack-param = SP "rpsi"
/ SP "app" [SP byte-string]
/ SP token [SP byte-string]
/ ; empty
rtcp-fb-nack-param = SP "pli"
/ SP "sli"
/ SP "rpsi"
/ SP "app" [SP byte-string]
/ SP token [SP byte-string]
/ ; emptyThe literals of the above grammar have these semantics 🧠:
Feedback type "ack" 🌸
This feedback type says positive acknowledgements for feedback are supported.
The feedback type "ack" MUST only be used if the media session is allowed to operate in ACK mode as defined in Section 3.6.1.
Parameters MUST be provided to further distinguish different types of positive acknowledgement feedback.
The parameter "rpsi" indicates the use of Reference Picture Selection Indication feedback as defined in Section 6.3.3.
If the parameter "app" is specified, this indicates the use of application layer feedback. In this case, additional parameters following "app" MAY be used to further differentiate various types of application layer feedback. This document does not define any parameters specific to "app".
Further parameters for "ack" MAY be defined in other documents.
Feedback type "nack" 🙌
This feedback type indicates that negative acknowledgements for feedback are supported.
The feedback type "nack", without parameters, indicates use of the Generic NACK feedback format as defined in Section 6.2.1.
The following three parameters are defined in this document for use with "nack" in conjunction with the media type "video":
"pli"indicates the use of Picture Loss Indication feedback as defined in Section 6.3.1."sli"indicates the use of Slice Loss Indication feedback as defined in Section 6.3.2."rpsi"indicates the use of Reference Picture Selection Indication feedback as defined in Section 6.3.3.
"app" indicates the use of application layer feedback. Additional parameters after "app" MAY be provided to differentiate different types of application layer feedback. No parameters specific to "app" are defined in this document.
Further parameters for "nack" MAY be defined in other documents.
Other feedback types <rtcp-fb-id> 🧩
Other documents MAY define additional types of feedback. To keep the grammar extensible for those cases, the rtcp-fb-id is introduced as a placeholder. A new feedback scheme name MUST to be unique (and thus MUST be registered with IANA). Along with a new name, its semantics, packet formats (if necessary), and rules for its operation MUST be specified.
Regular RTCP minimum interval "trr-int" ⏱️
The attribute "trr-int" is used to specify the minimum interval T_rr_interval between two Regular (full compound) RTCP packets in milliseconds for this media session. If "trr-int" is not specified, a default value of 0 is assumed.
Note that it is assumed that more specific information about application layer feedback (as defined in Section 6.4) will be conveyed as feedback types and parameters defined elsewhere. Hence, no further provision for any types and parameters is made in this document.
Further types of feedback as well as further parameters may be defined in other documents.
It is up to the recipients whether or not they send feedback information, and up to the sender(s) (how) to make use of the feedback provided 🫶.
4.3. RTCP Bandwidth Modifiers 🎀
The standard RTCP bandwidth assignments as defined in [1] and [2] MAY be overridden by bandwidth modifiers that explicitly define the maximum RTCP bandwidth. For use with SDP, such modifiers are specified in [4]: b=RS:<bw> and b=RR:<bw> MAY be used to assign a different bandwidth (measured in bits per second) to RTP senders and receivers, respectively. The precedence rules of [4] apply to determine the actual bandwidth to be used by senders and receivers.
Applications operating knowingly over highly asymmetric links (such as satellite links) SHOULD use this mechanism to reduce the feedback rate for high bandwidth streams. That way they prevent deterministic congestion of the feedback path(s) 🛰️.
4.4. Examples 🍓
Example 1: The following session description indicates a session made up from audio and DTMF [18] for point-to-point communication, in which the DTMF stream uses Generic NACKs. This session description could be contained in a SIP INVITE, 200 OK, or ACK message to indicate that its sender is capable of and willing to receive feedback for the DTMF stream it transmits.
v=0
o=alice 3203093520 3203093520 IN IP4 host.example.com
s=Media with feedback
t=0 0
c=IN IP4 host.example.com
m=audio 49170 RTP/AVPF 0 96
a=rtpmap:0 PCMU/8000
a=rtpmap:96 telephone-event/8000
a=fmtp:96 0-16
a=rtcp-fb:96 nackThis allows sender and receiver to provide reliable transmission of DTMF events in an audio session 📞. Assuming a 64-kbit/s audio stream with one receiver, the receiver has 2.5% RTCP bandwidth available for the negative acknowledgement stream, i.e., 250 bytes per second or some 2 RTCP feedback messages every second. Hence, the receiver can individually communicate up to two missing DTMF audio packets per second.
Example 2: The following session description indicates a multicast video-only session (using either H.261 or H.263+), with the video source accepting Generic NACKs for both codecs and Reference Picture Selection for H.263. Such a description may have been conveyed using the Session Announcement Protocol (SAP) 📡.
v=0
o=alice 3203093520 3203093520 IN IP4 host.example.com
s=Multicast video with feedback
t=3203130148 3203137348
m=audio 49170 RTP/AVP 0
c=IN IP4 224.2.1.183
a=rtpmap:0 PCMU/8000
m=video 51372 RTP/AVPF 98 99
c=IN IP4 224.2.1.184
a=rtpmap:98 H263-1998/90000
a=rtpmap:99 H261/90000
a=rtcp-fb:* nack
a=rtcp-fb:98 nack rpsiThe sender may use an incoming Generic NACK as a hint to send a new intra-frame as soon as possible (congestion control permitting) 🏃♀️. Receipt of a Reference Picture Selection Indication (RPSI) message allows the sender to avoid sending a large intra-frame. Instead, it may continue to send inter-frames, however, choosing the indicated frame as the new encoding reference 🖼️.
Example 3: The following session description defines the same media session as Example 2, but allows for mixed-mode operation of AVP and AVPF RTP entities (see also the next section). Note that both media descriptions use the same addresses. However, two m= lines are needed to convey information about both applicable RTP profiles 👯.
v=0
o=alice 3203093520 3203093520 IN IP4 host.example.com
s=Multicast video with feedback
t=3203130148 3203137348
m=audio 49170 RTP/AVP 0
c=IN IP4 224.2.1.183
a=rtpmap:0 PCMU/8000
m=video 51372 RTP/AVP 98 99
c=IN IP4 224.2.1.184
a=rtpmap:98 H263-1998/90000
a=rtpmap:99 H261/90000
m=video 51372 RTP/AVPF 98 99
c=IN IP4 224.2.1.184
a=rtpmap:98 H263-1998/90000
a=rtpmap:99 H261/90000
a=rtcp-fb:* nack
a=rtcp-fb:98 nack rpsiNote that these two m= lines SHOULD be grouped by some appropriate mechanism to indicate that both are alternatives actually conveying the same contents. A sample framework by which this can be achieved is defined in [10] 🔗.
In this example, the RTCP feedback-enabled receivers will gain an occasional advantage: they can report events back to the sender earlier, which may benefit the entire group 🌟. On average, however, all RTP receivers will provide the same amount of feedback. The interworking between AVP and AVPF entities is discussed in depth in the next section 💜.
5. Interworking and Coexistence of AVP and AVPF Entities ✨
okay so the AVPF profile in this doc is basically an upgrade of the AVP profile from [2] 💅 Both profiles follow the same basic rules, including the upper bandwidth limit for RTCP and how bandwidth gets split between senders and receivers. That means AVP and AVPF senders and receivers can totally hang out in one session together (see Example 3 in Section 4.5) 🫶
AVP and AVPF are built so that, robustness-wise, RTP entities don't need to know about the other profile. They won't mess with each other's functioning 🦋 The catch is that the quality of the media might take a hit ngl 😭
Here's what to keep in mind when senders and receivers from both profiles share a session.
AVP entities (senders and receivers) 🌸
AVP senders will receive RTCP feedback packets from AVPF receivers, and they'll just ignore them 🙈 They'll also notice RTCP messages showing up a bit closer together sometimes (like breaking the five-second rule) because of AVPF entities. Both kinds of entities stick to the overall bandwidth limits, so AVP entities still get their fair slice of the RTCP bandwidth. BUT, while AVP entities are bound by the five-second rule, AVPF entities can send more frequent reports than AVP ones, depending on group size and session bandwidth. Also, overall reporting might dip a little, since AVPF entities can send bigger compound RTCP packets (because of the extra RTCP packets) 💀
If T_rr_interval is used as the lower bound between Regular RTCP packets, T_rr_interval is big enough (for example, T_rr_interval > M*Td as per Section 6.3.5 of [1]), and no Early RTCP packets are sent by AVPF entities, then AVP entities might accidentally time out those AVPF group members and end up underestimating the group size 😬 So if AVP entities might be involved in a media session, T_rr_interval SHOULD NOT be larger than five seconds.
AVPF entities (senders and receivers) 🧚
If the dynamically calculated T_rr is pretty small (for example, less than one second), AVPF entities might accidentally time out AVP group members and underestimate the group size. Soooo, if AVP entities might be involved in a media session, T_rr_interval SHOULD be used and SHOULD be set to five seconds 💖
To sum it all up: if AVP entities might be involved in a media session and T_rr_interval is going to be used, T_rr_interval SHOULD be set to five seconds 🎀
AVPF senders 👑
AVPF senders only get feedback from AVPF receivers. If they rely on that feedback to hit their target media quality, the quality for AVP receivers could end up being suboptimal 🥺
AVPF receivers 🍓
AVPF receivers SHOULD send Early RTCP feedback packets only if all sending entities in the media session support AVPF. AVPF receivers MAY send feedback information as part of regularly scheduled compound RTCP packets, following the timing rules of [1] and [2], even in media sessions running in mixed mode. Just remember that the receiver giving feedback MUST NOT rely on the sender reacting to that feedback at all 🫣
6. Format of RTCP Feedback Messages 🎀
This section defines the format of the low-delay RTCP feedback messages. These messages come in three categories:
- Transport layer FB messages
- Payload-specific FB messages
- Application layer FB messages
Transport layer FB messages are for general purpose feedback, meaning info that doesn't depend on the particular codec or the application being used. This info is expected to be generated and processed at the transport/RTP layer. Right now, only a generic negative acknowledgement (NACK) message is defined 🦋
Payload-specific FB messages carry info that's specific to a certain payload type, and they get generated and acted upon at the codec "layer". This document defines a common header to be used with all payload-specific FB messages. The definition of the specific messages is left either to RTP payload format specifications or to additional feedback format documents 🌈
Application layer FB messages are a way to transparently pass feedback from the receiver's application to the sender's application. The info in these messages isn't expected to be acted upon at the transport/RTP or codec layer. The data exchanged between two application instances is usually defined in the application protocol specification, so the application can identify it without any extra external info. That's why this document only defines a common header for use with all application layer FB messages. From a protocol point of view, an application layer FB message is treated as a special case of a payload-specific FB message 💜
Note: Proper processing of some FB messages at the media sender side may require the sender to know which payload type the FB message refers to. Most of the time, this can be derived from a media stream using only a single payload type. However, if several codecs are used at the same time (for example, with audio and DTMF) or when codec changes happen, the payload type information may need to be conveyed explicitly as part of the FB message. This applies to all payload-specific as well as application layer FB messages. It's up to the specification of an FB message to define how payload type information is transmitted 🙈
This document defines two transport layer FB messages and three (video) payload-specific FB messages, plus a single container for application layer FB messages. Additional transport layer and payload-specific FB messages MAY be defined in other documents and MUST be registered through IANA (see Section 9, "IANA Considerations") ✨
The general syntax and semantics for the above RTCP FB message types are described in the following subsections.
6.1. Common Packet Format for Feedback Messages 💅
All FB messages MUST use a common packet format, depicted in Figure 3:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P| FMT | PT | length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC of packet sender |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC of media source |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: Feedback Control Information (FCI) :
: :
Figure 3: Common Packet Format for Feedback MessagesThe fields V, P, SSRC, and length are defined in the RTP specification [2]. Here's a quick rundown of what they mean:
version (V): 2 bits 🌸 This field identifies the RTP version. The current version is 2.
padding (P): 1 bit 🌷 If set, the padding bit means the packet has extra padding octets at the end. Those aren't part of the control information, but they are included in the length field.
Feedback message type (FMT): 5 bits 🦋 This field identifies the type of the FB message and is read relative to the type of feedback (transport layer, payload-specific, or application layer). The values for each of the three feedback types are defined in the respective sections below.
Payload type (PT): 8 bits 🧚 This is the RTCP packet type that marks the packet as an RTCP FB message. Two values are defined by IANA:
Name | Value | Brief Description
----------+-------+------------------------------------
RTPFB | 205 | Transport layer FB message
PSFB | 206 | Payload-specific FB messageLength: 16 bits 📏 The length of this packet in 32-bit words minus one, including the header and any padding. This matches the definition of the length field used in RTCP sender and receiver reports [3].
SSRC of packet sender: 32 bits 🫶 The synchronization source identifier for the originator of this packet.
SSRC of media source: 32 bits 🎬 The synchronization source identifier of the media source that this piece of feedback information relates to.
Feedback Control Information (FCI): variable length 🍓 The following three sections define which additional information MAY be included in the FB message for each type of feedback: transport layer, payload-specific, or application layer. Further FCI contents MAY be specified in further documents too 💖
Each RTCP feedback packet MUST contain at least one FB message in the FCI field. Sections 6.2 and 6.3 define, for each FCI type, whether or not multiple FB messages MAY be compressed into a single FCI field. If that's the case, they MUST be of the same type, i.e., same FMT. If multiple types of feedback messages (several FMTs) need to be conveyed, then several RTCP FB messages MUST be generated and SHOULD be concatenated in the same compound RTCP packet 🔥
6.2. Transport Layer Feedback Messages 🌈
Transport layer FB messages are identified by the value RTPFB as the RTCP message type.
Only one general purpose transport layer FB message is defined in this document: Generic NACK. It's identified by the FMT parameter like this:
0: unassigned
1: Generic NACK
2-30: unassigned
31: reserved for future expansion of the identifier number spaceThe following subsection defines the formats of the FCI field for this type of FB message. Further generic feedback messages MAY be defined in the future 👑
6.2.1. Generic NACK ✨
okay so the Generic NACK message is identified by PT=RTPFB and FMT=1 🎀. the FCI field MUST contain at least one and MAY contain more than one Generic NACK 💅.
the Generic NACK is used to indicate the loss of one or more RTP packets 😭. the lost packets get identified with a packet identifier and a bit mask 🦋.
ngl Generic NACK feedback SHOULD NOT be used if the underlying transport protocol is capable of providing similar feedback information to the sender (as may be the case, e.g., with DCCP) 🙈. if DCCP already handles it, don't double up, bestie 💀.
the Feedback Control Information (FCI) field has the following syntax (Figure 4) 🌸:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PID | BLP |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 4: Syntax for the Generic NACK message 💜
Packet ID (PID): 16 bits 🍓 The PID field is used to specify a lost packet. The PID field refers to the RTP sequence number of the lost packet 🧚.
bitmask of following lost packets (BLP): 16 bits 🌈 the BLP lets you report losses of any of the 16 RTP packets immediately following the RTP packet indicated by the PID ✨. its definition is identical to the one given in [6] 💅. so if we call the BLP's least significant bit bit 1, and its most significant bit bit 16, then bit i of the bit mask is set to 1 if the receiver has not received RTP packet number (PID+i) (modulo 2^16) and indicates this packet is lost. bit i is set to 0 otherwise 🫶.
Note that the sender MUST NOT assume that a receiver has received a packet because its bit mask was set to 0. 🙈 this part is super important, bestie 🔥
for example, the least significant bit of the BLP would be set to 1 if the packet corresponding to the PID and the following packet have been lost 👑. however, the sender cannot infer that packets PID+2 through PID+16 have been received simply because bits 2 through 15 of the BLP are 0 😭. all the sender knows is that the receiver has not reported them as lost at this time 🦋.
The length of the FB message MUST be set to 2+n, with n being the number of Generic NACKs contained in the FCI field 🎀.
the Generic NACK message implicitly references the payload type through the sequence number(s) 🌸.
6.3. Payload-Specific Feedback Messages 💖
Payload-Specific FB messages are identified by the value PT=PSFB as RTCP message type 🧚.
there are three payload-specific FB messages defined so far, plus an application layer FB message 💅. they're identified by the FMT parameter like this 👑:
0: unassigned
1: Picture Loss Indication (PLI)
2: Slice Loss Indication (SLI)
3: Reference Picture Selection Indication (RPSI)
4-14: unassigned
15: Application layer FB (AFB) message
16-30: unassigned
31: reserved for future expansion of the sequence number spacethe following subsections define the FCI formats for the payload-specific FB messages, and Section 6.4 defines the FCI format for the application layer FB message ✨.
6.3.1. Picture Loss Indication (PLI) 📸
The PLI FB message is identified by PT=PSFB and FMT=1 🌸.
There MUST be exactly one PLI contained in the FCI field 🫶.
6.3.1.1. Semantics 💜
with the Picture Loss Indication message, a decoder tells the encoder about the loss of an undefined amount of coded video data belonging to one or more pictures 😭. when it's used with any video coding scheme based on inter-picture prediction, an encoder that gets a PLI finds out the prediction chain may be broken 💀. the sender MAY react to a PLI by sending an intra-picture to resync (which makes this message basically similar to the FIR message as defined in [6]) 🦋. BUT the sender MUST consider congestion control as outlined in Section 7, and that MAY restrict its ability to send an intra frame 🙈.
other RTP payload specs, like RFC 2032 [6], already define a feedback mechanism for certain codecs 🌈. an application supporting both schemes MUST use the feedback mechanism defined in this specification when sending feedback 🔥. for backward compatibility reasons, such an application SHOULD also be capable of receiving and reacting to the feedback scheme defined in the respective RTP payload format, if that payload format requires it 🧚.
6.3.1.2. Message Format 🎀
PLI does not require parameters. Therefore, the length field MUST be 2, and there MUST NOT be any Feedback Control Information 💅.
the semantics of this FB message are independent of the payload type ✨.
6.3.1.3. Timing Rules ⏰
the timing follows the rules outlined in Section 3. in systems that use both PLI and other types of feedback, it may be advisable to follow the Regular RTCP RR timing rules for PLI, since PLI is not as delay critical as other FB types 🌸.
6.3.1.4. Remarks 🍓
PLI messages typically trigger the sending of full intra-pictures 🫶. intra-pictures are several times larger than predicted (inter-) pictures 😭. their size is independent of the time they are generated 🦋. in most environments, especially with bandwidth-limited links, using an intra-picture means accepting a delay that's a significant multiple of the typical frame duration 💀.
for example: if the sending frame rate is 10 fps, and an intra-picture is assumed to be 10 times as big as an inter-picture, then a full second of latency has to be accepted 👑. in an environment like that, there's no need for a particularly short delay in sending the FB message 💖. so waiting for the next possible time slot allowed by RTCP timing rules as per [2] with Tmin=0 does not have a negative impact on system performance ✨.
6.3.2. Slice Loss Indication (SLI) 🧩
The SLI FB message is identified by PT=PSFB and FMT=2 🌈.
The FCI field MUST contain at least one and MAY contain more than one SLI 🎀.
6.3.2.1. Semantics 💜
with the Slice Loss Indication, a decoder can tell an encoder that it has detected the loss or corruption of one or several consecutive macroblock(s) in scan order (see below) 😭. this FB message MUST NOT be used for video codecs with non-uniform, dynamically changeable macroblock sizes, such as H.263 with Annex Q enabled 🙈. in that case, an encoder cannot always identify the corrupted spatial region 💀.
6.3.2.2. Format 🍓
The Slice Loss Indication uses one additional FCI field, whose content is depicted in Figure 6. The length of the FB message MUST be set to 2+n, with n being the number of SLIs contained in the FCI field 🧚.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| First | Number | PictureID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 6: Syntax of the Slice Loss Indication (SLI) 💜
First: 13 bits 🌸 the macroblock (MB) address of the first lost macroblock 🫶. MB numbering works like this: the macroblock in the upper left corner of the picture is macroblock number 1, and the number for each macroblock increases from left to right and then from top to bottom in raster-scan order. so if there's a total of N macroblocks in a picture, the bottom right macroblock is macroblock number N 🦋.
Number: 13 bits 🌈 the number of lost macroblocks, in scan order as discussed above 🎀.
PictureID: 6 bits 💅 the six least significant bits of the codec-specific identifier used to reference the picture in which the loss of the macroblock(s) occurred. for many video codecs, the PictureID is identical to the Temporal Reference ✨.
the applicability of this FB message is limited to a small set of video codecs, so no explicit payload type information is provided 👑.
6.3.2.3. Timing Rules ⏰
the efficiency of algorithms using the Slice Loss Indication drops a lot when the Indication isn't sent in a timely fashion 😭. motion compensation propagates corrupted pixels that aren't reported as corrupted 💀. so the use of the algorithm discussed in Section 3 is highly recommended 🙈.
6.3.2.4. Remarks 🧠
the term Slice is defined and used here in the sense of MPEG-1: a consecutive number of macroblocks in scan order 🍓. more recent video coding standards sometimes understand the term Slice differently. in H.263 (1998), for example, there's a concept known as "rectangular slice" 🌸. the loss of one rectangular slice may mean you need to send more than one SLI to precisely identify the region of lost/damaged MBs 😭.
the first field of the FCI defines the first macroblock of a picture as 1, and not, as one might suspect, as 0 🫶. this was done to align this specification with the comparable mechanism available in ITU-T Rec. H.245 [24] 🦋. the maximum number of macroblocks in a picture (2**13 or 8192) corresponds to the maximum picture sizes of most ITU-T and ISO/IEC video codecs 🌈. if future video codecs offer larger picture sizes and/or smaller macroblock sizes, then an additional FB message has to be defined 💜. the six least significant bits of the Temporal Reference field are deemed sufficient to indicate the picture in which the loss occurred ✨.
the reaction to an SLI is not part of this specification 🧚. one typical way of reacting to an SLI is to use intra refresh for the affected spatial region 🎀.
algorithms were reported that keep track of the regions affected by motion compensation, so that Intra macroblocks can be sent to all those areas regardless of the timing of the FB (see H.263 (2000) Appendix I [17] and [15]) 💅. even though the timing of the FB is less critical when those algorithms are used, those algorithms correct large parts of the picture, so they have to transmit a much higher data volume when FBs are delayed 😭🔥.
6.3.3. Reference Picture Selection Indication (RPSI) 🦋
The RPSI FB message is identified by PT=PSFB and FMT=3 🌸.
There MUST be exactly one RPSI contained in the FCI field 💖.
6.3.3.1. Semantics ✨
Okay so, modern video coding standards like MPEG-4 visual version 2 [16] or H.263 version 2 [17] let the encoder use an older reference picture instead of the most recent one for predictive coding 🎞️ Typically there's a first-in-first-out queue of reference pictures kept around. If an encoder finds out that encoder-decoder synchronicity has been lost 💔, it can grab a known-as-correct reference picture to use instead. Since that reference picture is temporally further away then usual, the predictively coded picture ends up using more bits 😤
Both MPEG-4 and H.263 define a binary format for the "payload" of an RPSI message 📦 It includes stuff like the temporal ID of the damaged picture and the size of the damaged region. This bit string is typically small (a couple of dozen bits), it's of variable length, and it's self-contained, meaning it has everything needed to do reference picture selection. Iconic 💖
Both MPEG-4 and H.263 also allow RPSI with positive feedback info 👀 That means reporting pictures (or Slices) that were decoded without error ✅ But here's the tea ☕: any form of positive feedback MUST NOT be used when in a multiparty session (reporting positive feedback about individual reference pictures at RTCP intervals is not expected to be of much use anyway). Not me, saying a MUST NOT out loud 🙈
6.3.3.2. Format 🎀
The FCI for the RPSI message follows the format shown in Figure 7 💅
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PB |0| Payload Type| Native RPSI bit string |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| defined per codec ... | Padding (0) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 7: Syntax of the Reference Picture Selection Indication (RPSI) 🦋
Let's go field by field 🧚
PB: 8 bits. This is the number of unused bits needed to pad the length of the RPSI message to a multiple of 32 bits.0: 1 bit. This MUST be set to zero upon transmission and ignored upon reception 🙏Payload Type: 7 bits. This shows the RTP payload type that the native RPSI bit string MUST be interpreted in the context of 🌈Native RPSI bit string: variable length. This is the RPSI info as natively defined by the video codec 🍓Padding: #PB bits. A bunch of bits set to zero to fill the RPSI message up to the next 32-bit boundary. The number of padding bits MUST be indicated by the PB field 💜
6.3.3.3. Timing Rules ⏰
RPSI is even more critical to delay than algorithms using SLI, no cap 😭 The reason is that the older the RPSI message is, the more bits the encoder has to spend to re-establish encoder-decoder synchronicity. See [15] for some info about the overhead of RPSI for certain bit rate/frame rate/loss rate scenarios 📊
So, RPSI messages should typically be sent as soon as possible, using the algorithm of Section 3 🚀
6.4. Application Layer Feedback Messages 🌸
Application layer FB messages are a special case of payload-specific messages, identified by PT=PSFB and FMT=15 👑 There MUST be exactly one application layer FB message contained in the FCI field, unless the application layer FB message structure itself allows for stacking (e.g., by means of a fixed size or explicit length indicator). Lowkey a strict rule, but it makes sense ✨
These messages carry application-defined data straight from the receiver's application to the sender's application 💌 The data that's transported isn't identified by the FB message itself. Therefore, the application MUST be able to identify the message payload 🔍
Usually, applications define their own set of messages, like NEWPRED messages in MPEG-4 [16] (carried in RTP packets according to RFC 3016 [23]) or FB messages in H.263/Annex N, U [17] (packetized as per RFC 2429 [14]). These messages don't need any extra info from the RTCP message, so the application message just gets dropped straight into the FCI field like this, and the length field is set to match 🫶
Application Message (FCI): variable length 🎁 This field holds the original application message that should be transported from the receiver to the source. The format is application dependent, and the length is variable. If the application data isn't 32-bit aligned, padding bits and bytes MUST be added to get to 32-bit alignment. How you identify the padding is up to the application layer and isn't defined in this spec 💅
The application layer FB message specification MUST define whether or not the message needs to be interpreted specifically in the context of a certain codec (identified by the RTP payload type). If a reference to the payload type is required for proper processing, the application layer FB message specification MUST define a way to communicate the payload type information as part of the application layer FB message itself 🧠
7. Early Feedback and Congestion Control 🔥
Earlier sections covered how FB messages are defined and the timing rules for sending them 📜 How you react to the feedback you get depends on the application using the feedback mechanisms, so that's beyond the scope of this document 🤷♀️
That said, across all applications there's a common requirement for (TCP-friendly) congestion control on the media stream, as defined in [1] and [2], when you're operating in a best-effort network environment 🌐
It should be noted that RTCP feedback by itself is insufficient for congestion control purposes 😬 It's likely to operate on much slower timescales than other transport layer feedback mechanisms (which usually operate on the order of RTT). Therefore, additional mechanisms are required to perform proper congestion control 🛠️
A congestion control algorithm that shares the available bandwidth reasonably fairly with competing TCP connections, like TFRC [7], MUST be used to determine the data rate for the media stream. It has to stay within the bounds of the RTP sender's and the media session's capabilities, if the RTP/AVPF session is transmitted in a best-effort environment 🌊
8. Security Considerations 🔐
Okay, this part is serious, bestie 😭 RTP packets carrying information with the proposed payload format are subject to the security considerations discussed in the RTP specification [1] and in the RTP/AVP profile specification [2]. This profile doesn't specify any additional security services, so those are the ones you've got 🛡️
This profile modifies the timing behavior of RTCP. It gets rid of the minimum RTCP interval of five seconds and lets receivers give feedback earlier. Group members of the associated RTP session (possibly pretending to represent a large number of entities) may disturb RTCP operation by sending large numbers of RTCP packets. This cuts into the RTCP bandwidth available for Regular RTCP reporting as well as for Early FB messages. (Note that an entity doesn't need to be a member of a multicast group to cause these effects 👻) Similarly, malicious members may send very large RTCP messages, which increases the avg_rtcp_size variable and reduces the effectively available RTCP bandwidth 📉
Feedback information may be suppressed if unknown RTCP feedback packets are received 🙈 That creates a risk of a malicious group member reducing Early feedback by simply sending payload-specific RTCP feedback packets with random contents that no receiver recognizes (so they suppress feedback) or that the sender doesn't recognize (so no repair actions get taken) 💀
A malicious group member can also report arbitrarily high loss rates in the feedback information. That can make the sender throttle the data transmission, increase the amount of redundancy information, or take other action to deal with the pretended packet loss (e.g., send fewer frames or decrease audio/video quality). This may degrade the quality of the reproduced media stream 😭
Finally, a malicious group member can act as a large number of group members and grab an artificially large share of the Early feedback bandwidth. That reduces the reactivity of the other group members, possibly even making them stop operating in Immediate or Early feedback mode, which undermines the whole purpose of this profile 🥀
Senders as well as receivers SHOULD behave conservatively when they see strange reporting behavior ⚠️ For excessive failure reporting from one or a few receivers, the sender MAY decide to stop considering that feedback when adapting its transmission behavior for the media stream. In any case, senders and receivers SHOULD still adhere to the maximum RTCP bandwidth, but make sure they can transmit at least regularly scheduled RTCP packets. Senders SHOULD carefully consider how to adjust their transmission bandwidth when they encounter strange reporting behavior. They MUST NOT increase their transmission bandwidth, even if they're ignoring suspicious feedback 🚫
Attacks using false RTCP packets (Regular as well as Early ones) can be avoided by authenticating all RTCP messages. This can be done by using the AVPF profile together with the Secure RTP profile defined in [22]. As a prerequisite, an appropriate combination of those two profiles (an "SAVPF") is being specified [21]. Note that, when employing group authentication (as opposed to source authentication), the aforementioned attacks may still be carried out by malicious or malfunctioning group members who have the right keying material 🔑
9. IANA Considerations 🎀
okay bestie, this is the paperwork part 📋 but it's SO important, because it's how the official registries get the new values. the contact info goes on every registration in here, so here it is:
Contact: Joerg Ott
mailto:jo@acm.org
tel:+358-9-451-2460the feedback profile is an extension of the audio-visual conferences with minimal control profile, and it got registered as the SDP "proto" value RTP/AVPF ✨
SDP Protocol ("proto"):
Name: RTP/AVPF
Long form: Extended RTP Profile with RTCP-based Feedback
Type of name: proto
Type of attribute: Media level only
Purpose: RFC 4585
Reference: RFC 4585then there's the SDP attribute, rtcp-fb, which is the RTCP Feedback parameter 💖
SDP Attribute ("att-field"):
Attribute name: rtcp-fb
Long form: RTCP Feedback parameter
Type of name: att-field
Type of attribute: Media level only
Subject to charset: No
Purpose: RFC 4585
Reference: RFC 4585
Values: See this document and registrations belowa brand new registry got made just for rtcp-fb 🦋 and it starts off with four entries from this document: ack, nack, trr-int, and app.
Initial value registration for the attribute "rtcp-fb"
Value name: ack
Long name: Positive acknowledgement
Reference: RFC 4585.
Value name: nack
Long name: Negative Acknowledgement
Reference: RFC 4585.
Value name: trr-int
Long name: Minimal receiver report interval
Reference: RFC 4585.
Value name: app
Long name: Application-defined parameter
Reference: RFC 4585.more entries can get added later on a first-come first-serve basis 🌸 every new one has to say the parameter name and the syntax of any extra arguments. and this part is a big deal: every new registration needs a document that's permanent, stable, and publicly accessible. that document has to spell out what the parameter means, how its parameters work, and the feedback packet formats if it needs them. the general registration procedures of [3] apply too.
next up is a joint sub-registry for values that work with both ack and nack 💅
Initial value registration for the attribute values "ack" and "nack":
Value name: sli
Long name: Slice Loss Indication
Usable with: nack
Reference: RFC 4585.
Value name: pli
Long name: Picture Loss Indication
Usable with: nack
Reference: RFC 4585.
Value name: rpsi
Long name: Reference Picture Selection Indication
Usable with: ack, nack
Reference: RFC 4585.
Value name: app
Long name: Application layer feedback
Usable with: ack, nack
Reference: RFC 4585.new entries here also go first-come first-serve 🔥 each one has to say its parameter name, the syntax of any extra arguments, and whether it's for ack, for nack, for both, or for some other rtcp-fb parameter. the same "permanent, stable, publicly accessible document" rule applies, and the general registration procedures of [3] still apply.
okay now the RTCP packet types 🌈 there are two classes of feedback messages: transport layer FB messages (RTPFB) and payload-specific FB messages (PSFB). per Section 6, RTPFB=205 and PSFB=206 got added to the RTCP registry.
RTP RTCP Control Packet types (PT):
Name: RTPFB
Long name: Generic RTP Feedback
Value: 205
Reference: RFC 4585.
Name: PSFB
Long name: Payload-specific
Value: 206
Reference: RFC 4585.since AVPF adds more RTCP payload types, the "reserved" RTP payload type space (72-76, as defined in [2]) got expanded to fit them 🦋
then there's a sub-registry for the FMT values, for both the RTPFB and PSFB payload types. the initial registrations look like this:
Within the RTPFB range, two format (FMT) values are registered first ✨
Name: Generic NACK
Long name: Generic negative acknowledgement
Value: 1
Reference: RFC 4585.
Name: Extension
Long name: Reserved for future extensions
Value: 31
Reference: RFC 4585.Within the PSFB range, five format (FMT) values are registered first 🍓
Name: PLI
Long name: Picture Loss Indication
Value: 1
Reference: RFC 4585.
Name: SLI
Long name: Slice Loss Indication
Value: 2
Reference: RFC 4585.
Name: RPSI
Long name: Reference Picture Selection Indication
Value: 3
Reference: RFC 4585.
Name: AFB
Long name: Application Layer Feedback
Value: 15
Reference: RFC 4585.
Name: Extension
Long name: Reserved for future extensions.
Value: 31
Reference: RFC 4585.more FMT entries can be registered following the "Specification Required" rules from RFC 2434 [9] 🫶 each registration has to give the FMT value, whether there's a specific FB message going into the FCI field, and whether multiple FB messages can be stacked in a single FCI field. same deal as always: a permanent, stable, publicly accessible document has to exist that spells out the semantics of the registered parameter, plus the syntax and semantics of the FB message (if there is one). the general registration procedures of [3] apply.
10. Acknowledgements 🌷
this doc came out of the Audio-Visual Transport (AVT) Working Group of the IETF 💜 the authors wanted to thank Steve Casner and Colin Perkins for their comments, suggestions, and for always answering questions. they also gave a special shoutout to Magnus Westerlund for his review and valuable suggestions, and to Shigeru Fukunaga for the contributions on FB message formats and semantics 👑
and they thanked Andreas Buesching and the people at Panasonic for running simulations and building the first independent implementations of the feedback profile 🎉
11. References 📚
11.1. Normative References
[1] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson,
"RTP: A Transport Protocol for Real-Time Applications", STD 64,
RFC 3550, July 2003.
[2] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and Video
Conferences with Minimal Control", STD 65, RFC 3551, July 2003.
[3] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
Description Protocol", RFC 4566, July 2006.
[4] Casner, S., "Session Description Protocol (SDP) Bandwidth
Modifiers for RTP Control Protocol (RTCP) Bandwidth", RFC 3556,
July 2003.
[5] Bradner, S., "Key words for use in RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119, March 1997.
[6] Turletti, T. and C. Huitema, "RTP Payload Format for H.261 Video
Streams", RFC 2032, October 1996.
[7] Handley, M., Floyd, S., Padhye, J., and J. Widmer, "TCP Friendly
Rate Control (TFRC): Protocol Specification", RFC 3448, January
2003.
[8] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with
Session Description Protocol (SDP)", RFC 3264, June 2002.
[9] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA
Considerations Section in RFCs", BCP 26, RFC 2434, October 1998.11.2. Informative References
[10] Camarillo, G., Eriksson, G., Holler, J., and H. Schulzrinne,
"Grouping of Media Lines in the Session Description Protocol
(SDP)", RFC 3388, December 2002.
[11] Perkins, C. and O. Hodson, "Options for Repair of Streaming
Media", RFC 2354, June 1998.
[12] Rosenberg, J. and H. Schulzrinne, "An RTP Payload Format for
Generic Forward Error Correction", RFC 2733, December 1999.
[13] Perkins, C., Kouvelas, I., Hodson, O., Hardman, V., Handley, M.,
Bolot, J., Vega-Garcia, A., and S. Fosse-Parisis, "RTP Payload
for Redundant Audio Data", RFC 2198, September 1997.
[14] Bormann, C., Cline, L., Deisher, G., Gardos, T., Maciocco, C.,
Newell, D., Ott, J., Sullivan, G., Wenger, S., and C. Zhu, "RTP
Payload Format for the 1998 Version of ITU-T Rec. H.263 Video
(H.263+)", RFC 2429, October 1998.
[15] B. Girod, N. Faerber, "Feedback-based error control for mobile
video transmission", Proceedings IEEE, Vol. 87, No. 10, pp.
1707 - 1723, October, 1999.
[16] ISO/IEC 14496-2:2001/Amd.1:2002, "Information technology -
Coding of audio-visual objects - Part2: Visual", 2001.
[17] ITU-T Recommendation H.263, "Video Coding for Low Bit Rate
Communication", November 2000.
[18] Schulzrinne, H. and S. Petrack, "RTP Payload for DTMF Digits,
Telephony Tones and Telephony Signals", RFC 2833, May 2000.
[19] Kohler, E., Handley, M., and S. Floyd, "Datagram Congestion
Control Protocol (DCCP)", RFC 4340, March 2006.
[20] Handley, M., Floyd, S., Padhye, J., and J. Widmer, "TCP Friendly
Rate Control (TFRC): Protocol Specification", RFC 3448, January
2003.
[21] Ott, J. and E. Carrara, "Extended Secure RTP Profile for RTCP-
based Feedback (RTP/SAVPF)", Work in Progress, December 2005.
[22] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K.
Norrman, "The Secure Real-time Transport Protocol (SRTP)", RFC
3711, March 2004.
[23] Kikuchi, Y., Nomura, T., Fukunaga, S., Matsui, Y., and H.
Kimata, "RTP Payload Format for MPEG-4 Audio/Visual Streams",
RFC 3016, November 2000.
[24] ITU-T Recommendation H.245, "Control protocol for multimedia
communication", May 2006.Authors' Addresses 💌
Joerg Ott
Helsinki University of Technology (TKK)
Networking Laboratory
PO Box 3000
FIN-02015 TKK
Finland
EMail: jo@acm.org
Stephan Wenger
Nokia Research Center
P.O. Box 100
33721 Tampere
Finland
EMail: stewe@stewe.org
Noriyuki Sato
Oki Electric Industry Co., Ltd.
1-16-8 Chuo, Warabi-city, Saitama 335-8510
Japan
Phone: +81 48 431 5932
Fax: +81 48 431 9115
EMail: sato652@oki.com
Carsten Burmeister
Panasonic R&D Center Germany GmbH
EMail: carsten.burmeister@eu.panasonic.com
Jose Rey
Panasonic R&D Center Germany GmbH
Monzastr. 4c
D-63225 Langen, Germany
EMail: jose.rey@eu.panasonic.comFull Copyright Statement 🎀
Okay bestie, the legal stuff, but make it cute. 💅 The copyright for this one belongs to The Internet Society, 2006 ✨. The document is covered by the rights, licenses, and restrictions in BCP 78, and the authors keep all their rights except where BCP 78 says otherwise 📜.
The document and everything in it is handed over "AS IS", and the contributor, the organization they represent or are sponsored by (if any), the Internet Society, and the Internet Engineering Task Force all say, very loudly, that there are no warranties 🙈. That includes any warranty that using the info won't infringe rights, and any implied warranty of merchantability or fitness for a particular purpose. The exact wording is below, because legal text is not the place for my vibes 🫶:
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.Intellectual Property 💜
Okay so the IETF is not picking a side on whether any patents or other proprietary rights cover the tech in this doc, or whether you'd be able to get a license for them 🧐. It also hasn't done a search to find any. If you want the rules on rights in RFC documents, look in BCP 78 and BCP 79 📖.
Copies of IPR disclosures sent to the IETF Secretariat live in the IETF's online IPR repository at http://www.ietf.org/ipr. You'll also find any licensing assurances there, plus the results of anyone trying to get a general license or permission to use the proprietary rights in this spec 🦋.
The IETF is also inviting anyone to flag copyrights, patents, patent applications, or other proprietary rights that might cover tech needed to implement this standard. Send that info to ietf-ipr@ietf.org 📬.
Acknowledgement 🌸
Huge thank-you moment 🙌: funding for the RFC Editor function comes from the IETF Administrative Support Activity (IASA). Without them, the RFC Editor wouldn't be able to keep these documents coming, and we'd all be lost 😭🫶.
📦 where this page comes from
- ietf: rfc4585 2006-07
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms