RFC 6354
Forward-Shifted RTP Redundancy Payload Support
RFC 6354: Forward-Shifted RTP Redundancy Payload Support 🎀
okay bestie, this one is from the IETF, and it's a Standards Track doc, so it's the real deal 💅 It was written by Q. Xie in August 2011, and it's RFC 6354. The ISSN is 2070-1721 ✨ It updates RFC 2198 and RFC 4102 (so those are the docs it's tweaking), and it's part of the IETF Internet Standards process 🌸
Abstract 🦋
okay so this doc adds a small upgrade so RTP sessions can use forward-shifted redundant encodings. that means the backup data gets sent before the real data it's protecting, not after 🧚 this helps cover up losses of a huge run of media frames, like seconds or even tens of seconds of audio/video, and it's giving resilience 💜
Status of This Memo ✨
this is an Internet Standards Track document 🎀
it's a product of the IETF, so it's got community consensus. it went through public review and got approved for publication by the IESG 🌈 if you want more on Internet Standards, check out Section 2 of RFC 5741 💅
for info on the current status, any errata, or how to send feedback, go to http://www.rfc-editor.org/info/rfc6354 🫶
Copyright Notice 🌸
the copyright is 2011, held by the IETF Trust and the document authors, and everything is reserved 💖
this doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they stood when it was published. read those carefully, bestie, because they explain your rights and restrictions 📜 any Code Components pulled from this doc have to include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come with no warranty, same as that license says 💅
some of the material here may come from IETF Documents or Contributions that were published or made public before November 10, 2008 😭 whoever controls the copyright on some of that stuff might not have let the IETF Trust allow changes outside the IETF Standards Process. so unless you get a proper license from them, this doc can't be modified outside that process, and you can't make derivative works outside it either. the only exceptions are formatting it for publication as an RFC or translating it into languages other than English 🍓
Table of Contents 🧚
1. Introduction ....................................................2
1.1. Sending Redundant Data Inband vs. Out-of-Band ..............3
2. Conventions .....................................................4
3. Allowing Forward-Shifted Redundant Data .........................4
4. Registration of Media Type "fwdred" .............................5
5. Mapping Media Type Parameters into SDP ..........................7
6. Usage in Offer/Answer ...........................................7
7. IANA Considerations .............................................7
8. Security Considerations .........................................8
9. Normative References ............................................8
Appendix A. Anti-Shadow Loss Concealment Using
Forward-Shifted Redundancy .............................9
A.1. Sender-Side Operations .....................................9
A.2. Receiver-Side Operations ..................................11
A.2.1. Normal-Mode Operation ................................11
A.2.2. Anti-Shadow-Mode Operation ...........................121. Introduction ✨
okay so this doc is a simple upgrade to RFC 2198 [RFC2198] that lets RTP sessions use forward-shifted redundant encodings. basically, redundant data gets sent before the primary data it's covering 💖
this matters a LOT for mobile wireless receivers 📱 the radio signal can get blocked by tall buildings, mountains, tunnels, and stuff like that. then the receiver is basically stuck in a "shadow" of coverage, and no new data shows up while that's happening 😭 sometimes the receiver can be stuck in a shadow for seconds or even tens of seconds. the old backward-shifted scheme from RFC 2198 [RFC2198] (where redundant data goes after the primary data) just doesn't handle those huge consecutive losses 💀
forward-shifted redundancy is the fix 🦋 it lets the receiver do effective anti-shadow loss management (Appendix A has the details), so the service doesn't drop when a mobile receiver walks into a shadow. iconic 👑
this stuff works great for streaming pre-recorded media 🎬 but for live media, the source has to add a delay at least as long as the forward-shift amount, because it needs time to generate that forward-shifted anti-shadow stream. so yeah, there's a tradeoff there 🙈
1.1. Sending Redundant Data Inband vs. Out-of-Band 🌈
no matter which way the time shift goes (forward, or backward like RFC 2198) and no matter the encoding scheme (FEC or non-FEC), you always get to choose: send the redundant data and primary data in the same RTP session (inband), or in separate RTP sessions (out-of-band) 🧩 both have pros and cons, so here's the breakdown.
Inband approach:
- Pro: a single RTP session is faster to set up and easier to manage ✨
- Pro: a single RTP session is a simpler problem for NAT/firewall traversal 🔥
- Pro: less overall overhead, since there's only one source of RTP/UDP/IP overhead 💅
- Con: less flexibility, because it's hard for middle boxes like gateways to add or remove the redundant data 🙃
- Con: needs more specification, since special payload formats have to be defined to carry the redundant data inband 📝
Out-of-band approach:
- Pro: flexibility, since a middle box like a gateway can add, remove, or replace the redundant data more easily 🦋
- Pro: little or no specification, because no new payload format is needed 🍓
- Con: multiple RTP sessions may take longer to set up and may be more complex to manage 😵💫
- Con: multiple RTP sessions are harder to handle for NAT/firewall traversal 🧱
- Con: bigger overall overhead, since there's more than one source of RTP/UDP/IP overhead 📈
the inband payload formats in this doc and in RFC 2198 don't stop a deployment from going out-of-band. they just give the deployment the choice of whichever approach works best in its situation 🫶
2. Conventions 💜
the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119] 📖
3. Allowing Forward-Shifted Redundant Data 🎀
okay so in RFC 2198, the timestamp offset in the extra header for a redundant block is a 14-bit unsigned offset from the timestamp in the RTP header 🧮 RFC 2198 says it like this:
The use of an unsigned offset implies that redundant data must be sent after the primary data, and is hence a time to be subtracted from the current timestamp to determine the timestamp of the data for which this block is the redundancy.
and that's exactly why RFC 2198 can't support forward-shifted redundant blocks 😭 it's just not built for it.
so to fix that, this doc introduces the media type fwdred 🌸 it has a parameter called forwardshift, which shows that you can use forward-shifted redundancy and also gives the base value for how much forward-shifting happens. the base value of forwardshift is an integer that's 0 or greater, measured in RTP timestamp units 🦋
in an RTP session that uses forward-shifted redundant encodings, the timestamp of a redundant block in a received RTP packet is worked out like this:
timestamp of redundant block = timestamp in RTP header
- timestamp offset in additional header
+ forward-shift base valuequick note: in a forward-shifted session, the timestamp offset in the additional header is usually set to '0' 💅
and one more very important rule 🚨 the sender MUST NOT change the contents of a packet that's in a forward-shifted stream when it's time to send that packet in the main stream. no edits, bestie. not even a little 🙅♀️
🦋 that's the end of Part 1 🦋
5. Mapping Media Type Parameters into SDP ✨
okay so the media type info has a super specific mapping to fields in SDP [RFC4566], which is what people usually use to describe RTP sessions 🎀 when you're using the forward-shifted redundant format, it goes like this:
- the media type ("audio") goes in SDP "m=" as the media name 🎧
- the media subtype ("fwdred") goes in SDP "a=rtpmap" as the encoding name 💅
- the required parameter "forwardshift" goes in the SDP "a=fmtp" attribute, copied straight from the media type string as
forwardshift=value🫶
here's an example that shows forward-shifted redundancy by 5.1 sec 🦋
m=audio 12345 RTP/AVP 121 0 5
a=rtpmap:121 fwdred/8000/1
a=fmtp:121 0/5 forwardshift=40800and this one is sending redundancy without forward-shifting (equivalent to RFC 2198) 🌸
m=audio 12345 RTP/AVP 121 0 5
a=rtpmap:121 fwdred/8000/1
a=fmtp:121 0/5 forwardshift=06. Usage in Offer/Answer 🌈
the "forwardshift" SDP parameter in this doc is declarative, and all reasonable values are expected to be supported 💖
7. IANA Considerations 👑
IANA made the assignments described below per this document, periodt 💅
- IANA added the following to the "Audio Media Types" registry:
fwdred [RFC6354]- IANA added the following to the "Text Media Types" registry:
fwdred [RFC6354]8. Security Considerations 🔥
okay this section is important, so listen up 💀 the security considerations from Section 6 of [RFC2198], Section 4 of [RFC4856], and Sections 9 and 14 of [RFC3550] all apply to this spec too. ALSO, to stop denial-of-service attacks, a receiver SHOULD be prepared to ignore a 'forwardshift' parameter declaration if it thinks the offset value in that declaration is excessive 😭 and in that case, the receiver SHOULD also ignore the redundant stream in the resulting RTP session 🙈
9. Normative References 📚
here's the reference list, copied exactly so nothing gets lost 🍓
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2198] Perkins, C., Kouvelas, I., Hodson, O., Hardman, V.,
Handley, M., Bolot, J., Vega-Garcia, A., and S. Fosse-
Parisis, "RTP Payload for Redundant Audio Data", RFC 2198,
September 1997.
[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, RFC 3550, July 2003.
[RFC4102] Jones, P., "Registration of the text/red MIME Sub-Type",
RFC 4102, June 2005.
[RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
Description Protocol", RFC 4566, July 2006.
[RFC4856] Casner, S., "Media Type Registration of Payload Formats in
the RTP Profile for Audio and Video Conferences",
RFC 4856, February 2007.
[RFC4867] Sjoberg, J., Westerlund, M., Lakaniemi, A., and Q. Xie,
"RTP Payload Format and File Storage Format for the
Adaptive Multi-Rate (AMR) and Adaptive Multi-Rate Wideband
(AMR-WB) Audio Codecs", RFC 4867, April 2007.Appendix A. Anti-Shadow Loss Concealment Using Forward-Shifted Redundancy 🧚
(this Appendix is included for Informational purposes ✨)
it's not unusual in a wireless mobile setup for the radio signal to a mobile receiver to get temporarily blocked by tall buildings, mountains, tunnels, etc. for a while 🏔️ so the receiver goes into a shadow of the radio coverage. when the receiver is in a shadow, no new data gets received, and in extreme cases it might spend seconds or even tens of seconds in there 😭
without special design to make up for data lost to shadowing, a mobile user can run into an unacceptable level of service interruptions 💀 traditional redundant encoding schemes (including RFC 2198 and most FEC schemes) are known to be ineffective at handling losses of consecutive frames like this 🙈
BUT forward-shifted redundancy, combined with the anti-shadow loss concealment at the receiver (as described here), can effectively prevent service interruptions caused by shadowing 🦋
A.1. Sender-Side Operations 🎀
for anti-shadow loss management, the RTP sender just adds a forward-shifted redundant stream (called anti-shadow or AS stream) while it transmits the primary media stream 💜 the amount of forward-shifting, which should stay constant for the whole session, decides the maximal length of shadows that can be completely concealed at the receiver, as explained below ✨
except for the fact that the redundant stream is forward-shifted relative to the primary stream (the redundant data is sent ahead of the corresponding primary data), the design decisions and trade-offs on quality, encoding, bandwidth overhead, etc. for the redundant stream are the same as the traditional RTP payload redundancy scheme 🌸
this diagram shows a chunk of the transmission sequence of a forward-shifted redundant RTP session, where the AS stream is forward-shifted by 155 frames 🍓 if we assume, for simplicity, that the timestamp goes up by 1 between two consecutive frames, this forward-shifted redundancy can be indicated with:
forwardshift=155and the timestamp offset is set to 0 in all the additional headers 💅 that can mean 3.1 seconds of forward-shifting if each frame represents 20 ms of original media ⏱️
Primary stream AS stream
... | |
v v
Pkt k+8 [ 111 ] [ 266 ]
| |
v v
Pkt k+7 [ 110 ] [ 265 ]
| |
v v
^ Pkt k+6 [ 109 ] [ 264 ]
| | |
| v v
Pkt k+5 [ 108 ] [ 263 ]
T | |
I v v
M Pkt k+4 [ 107 ] [ 262 ]
E | |
v v
Pkt k+3 [ 106 ] [ 261 ]
| |
v v
Pkt k+2 [ 105 ] [ 260 ]
| |
v v
Pkt k+1 [ 104 ] [ 259 ]
| |
v v
Pkt k [ 103 ] [ 258 ]
| |
v v
(Transmitted first)
Figure 1: An Example of Forward-Shifted Redundant RTP Packet
TransmissionA.2. Receiver-Side Operations 🦋
here's what the anti-shadow receiver looks like 💖
+---------+
normal mode sw1 | media | media
Primary stream ======================o___o==>| decoder |===> output
AS stream ---- +---------+ device
| AS mode o
| +---------+ |
| | anti- | |
------->| shadow |----
| buffer |
+---------+
|
V
expired frames
discarded
Figure 2: Anti-Shadow RTP Receiverthe anti-shadow receiver works between two modes, "normal mode" and "AS mode" ✨ when the receiver isn't in a shadow (aka it still gets new data), it runs in normal mode. otherwise, it runs in AS mode 🌈
A.2.1. Normal-Mode Operation 💜
in normal mode, after getting a new RTP packet with the primary data and the forward-shifted AS data, the receiver passes the primary data straight to the right media decoder for play-out (a de-jittering buffer might be used before play-out, but for simplicity we assume none is used here 🙈), while the received AS data gets stored in an anti-shadow buffer 🎧
the data in the anti-shadow buffer keeps getting checked for expiry. if any redundant data in the buffer has a timestamp older (aka smaller) than the last primary frame passed to the media decoder, it counts as expired and gets purged from the anti-shadow buffer 🧹
this example shows how the anti-shadow buffer works in normal mode 🍓 same assumptions as Figure 1, and the initial timestamp value is 103 when the session starts ⭐
Timestamp Timestamp
Time being of media in
(in ms) played out AS buffer Note
------------------------------------------------------------------
t < 0 -- (buffer empty)
...
t=0 103 258 (hold 1 AS frame)
t=20 104 258-259 (hold 2 AS frames)
t=40 105 258-260 (hold 3 AS frames)
...
t=3080 257 258-412 (full, hold 155 AS frames)
t=3100 258 259-413 (full, frame 258 purged)
t=3120 259 260-414 (full, frame 259 purged)
...
t=6240 415 416-570 (always holds 3.1 sec
worth of redundant data)
...
Figure 3: Example of Anti-Shadow Buffer Operation in Normal Modebefore the session starts (t<0), the anti-shadow buffer is empty 🫧 when the first primary frame arrives, play-out starts right away, and the first received AS frame gets stored in the anti-shadow buffer 💅 as more forward-shifted redundant frames show up, the anti-shadow buffer slowly fills up 🌊
for the example in Figure 3, after 3.08 seconds (the forward-shifting amount minus one frame) from the start of the session, the anti-shadow buffer is full 🎀 it holds exactly 3.1 seconds of redundant data, with the oldest frame at t=3.1 sec and the youngest at t=6.18 sec ✨
it's not hard to see that in normal mode, because expired frames keep getting purged and new frames keep getting added, the anti-shadow buffer is always full, holding the next 'forwardshift' amount of redundant frames 💖
A.2.2. Anti-Shadow-Mode Operation 🦋
okay so, when the receiver falls into a shadow (or any other situation that stops it from getting new media data), it switches into anti-shadow mode 🫣. In that mode it just keeps playing out from the forward-shifted redundant data sitting in the anti-shadow buffer. literally the whole point of the buffer 💖
Let's look at Figure 3 🎀. If the receiver enters a shadow at t=3120, it can keep the play-out going using the forward-shifted redundant frames (ts=260-414) from the anti-shadow buffer. As long as the receiver gets out of the shadow by t=6240, there's no service interruption ✨
When the shadow condition ends (meaning new data starts arriving again), the receiver switches right back to normal mode and plays out from the newly arrived primary frames 🌸. At the same time, the new AS frames start refilling the anti-shadow buffer, so it's ready for the next time things go sideways 💅
BUT (and this part is important, bestie 😭) if the shadow lasts longer than the amount of forward-shifting, the receiver runs out of media frames in its anti-shadow buffer. At that point, service interruption happens 💀
Author's Address 🧚
Qiaobing Xie
15901 Wetherburn Rd.
Chesterfield, MO 63017
US
Phone: +1-847-893-0222
EMail: Qiaobing.Xie@gmail.com📦 where this page comes from
- ietf: rfc6354 2011-08
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms