RFC 6015
RTP Payload Format for 1-D Interleaved Parity Forward Error Correction (FEC)
RFC 6015 Glow-Up, Part 1 of 6 🦋
Title block (the vitals) 💖
- RFC number: 6015 🎀
- Title: RTP Payload Format for 1-D Interleaved Parity Forward Error Correction (FEC) ✨
- Author: A. Begen, from Cisco 💅
- Category: Standards Track 👑
- Date: October 2010 🌸
- ISSN: 2070-1721 🧚
The header doesn't have an "Obsoletes" or "Updates" line, so this one doesn't list anything it replaces or changes. 🫶
Abstract
okay so this doc defines a brand-new RTP payload format for Forward Error Correction (FEC) 🔥 that's made by the 1-D interleaved parity code from some source media wrapped up in RTP. The vibe: it's a systematic code. You make a bunch of repair symbols from a set of source symbols, and then you send those repairs in their own repair flow, separate from the source flow carrying the originals. 🦋
The 1-D interleaved parity code is lowkey iconic because it protects you really well against bursty packet losses, and it doesn't cost a ton of complexity 💜. The new payload format should only be used (with some exceptions) as part of the Digital Video Broadcasting-IPTV (DVB-IPTV) Application-layer FEC specification. So it's not a free-for-all, bestie 🙈
Status of This Memo
This is an Internet Standards Track document. 👑
This doc is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community, it got public review, and the Internet Engineering Steering Group (IESG) approved it for publication. If you want more on Internet Standards, check out Section 2 of RFC 5741 🌈
For the current status, any errata, and how to send feedback, go to http://www.rfc-editor.org/info/rfc6015 🍓
Copyright Notice
Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved. 💜
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they stand on the day this document was published. Please read these carefully, bestie, because they spell out your rights and restrictions for this doc. 🙈 Code Components pulled out of this document must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as the Simplified BSD License describes. 💅
Table of Contents
1. Introduction ....................................................4
1.1. Use Cases ..................................................6
1.2. Overhead Computation .......................................8
1.3. Relation to Existing Specifications ........................8
1.3.1. RFCs 2733 and 3009 ..................................8
1.3.2. SMPTE 2022-1 ........................................8
1.3.3. ETSI TS 102 034 .....................................9
1.4. Scope of the Payload Format ...............................10
2. Requirements Notation ..........................................10
3. Definitions, Notations, and Abbreviations ......................10
3.1. Definitions ...............................................10
3.2. Notations .................................................11
4. Packet Formats .................................................11
4.1. Source Packets ............................................11
4.2. Repair Packets ............................................11
5. Payload Format Parameters ......................................15
5.1. Media Type Registration ...................................15
5.1.1. Registration of audio/1d-interleaved-parityfec .....15
5.1.2. Registration of video/1d-interleaved-parityfec .....16
5.1.3. Registration of text/1d-interleaved-parityfec ......18
5.1.4. Registration of
application/1d-interleaved-parityfec ...............19
5.2. Mapping to SDP Parameters .................................20
5.2.1. Offer-Answer Model Considerations ..................21
5.2.2. Declarative Considerations .........................22
6. Protection and Recovery Procedures .............................22
6.1. Overview ..................................................22
6.2. Repair Packet Construction ................................22
6.3. Source Packet Reconstruction ..............................24
6.3.1. Associating the Source and Repair Packets ..........25
6.3.2. Recovering the RTP Header and Payload ..............25
7. Session Description Protocol (SDP) Signaling ...................27
8. Congestion Control Considerations ..............................27
9. Security Considerations ........................................28
10. IANA Considerations ...........................................29
11. Acknowledgments ...............................................29
12. References ....................................................29
12.1. Normative References .....................................29
12.2. Informative References ...................................301. Introduction ✨
okay so this doc extends the Forward Error Correction (FEC) header defined in [RFC2733] 🎀 and then uses that new FEC header for the FEC generated by the 1-D interleaved parity code from source media encapsulated in RTP [RFC3550]. The resulting new RTP payload format is registered by this document. 💖
The source media protected by the 1-D interleaved parity code can be audio, video, text, or application 🌈 The FEC data get generated according to the media type parameters communicated through out-of-band means. The associations and relationships between the source and repair flows are also communicated through out-of-band means. 🧚
The 1-D interleaved parity FEC uses the exclusive OR (XOR) operation to make its repair symbols. In a nutshell, here's what happens 💅
- The sender figures out a set of source packets to protect together, based on the media type parameters.
- The sender applies the XOR operation to the source symbols to make the required number of repair symbols.
- The sender packetizes the repair symbols and sends the repair packet(s) along with the source packets to the receiver(s), in different flows. The repair packets may be sent proactively or on demand. 🔥
Note that source and repair packets live in different source and repair flows, and the sender needs to give receivers a way to demultiplex them 👑 That's still true even when they're sent in the same transport flow (same source/destination address/port with UDP). This is required to offer backward compatibility (see Section 4). 🙈 At the receiver side, if every source packet arrives successfully, there's no need for FEC recovery and the repair packets get discarded. But if some source packets are missing, the repair packets can be used to recover the missing info. 🦋 The block diagrams for the systematic parity FEC encoder and decoder are sketched in Figures 1 and 2, respectively. ✨
+------------+
+--+ +--+ +--+ +--+ --> | Systematic | --> +--+ +--+ +--+ +--+
+--+ +--+ +--+ +--+ | Parity FEC | +--+ +--+ +--+ +--+
| Encoder |
| (Sender) | --> +==+ +==+
+------------+ +==+ +==+
Source Packet: +--+ Repair Packet: +==+
+--+ +==+
Figure 1: Block diagram for systematic parity FEC encoder +------------+
+--+ X X +--+ --> | Systematic | --> +--+ +--+ +--+ +--+
+--+ +--+ | Parity FEC | +--+ +--+ +--+ +--+
| Decoder |
+==+ +==+ --> | (Receiver) |
+==+ +==+ +------------+
Source Packet: +--+ Repair Packet: +==+ Lost Packet: X
+--+ +==+
Figure 2: Block diagram for systematic parity FEC decoderOkay, here's the main idea 🧠 Suppose we have a group of D x L source packets, with sequence numbers starting at 1 and running up to D x L. If we apply the XOR operation to the source packets whose sequence numbers are L apart from each other, as sketched in Figure 3, we generate L repair packets. This process is called 1-D interleaved FEC protection, and the resulting L repair packets are called interleaved (or column) FEC packets. It's giving organized 🍓
+-------------+ +-------------+ +-------------+ +-------+
| S_1 | | S_2 | | S3 | ... | S_L |
| S_L+1 | | S_L+2 | | S_L+3 | ... | S_2xL |
| . | | . | | | | |
| . | | . | | | | |
| . | | . | | | | |
| S_(D-1)xL+1 | | S_(D-1)xL+2 | | S_(D-1)xL+3 | ... | S_DxL |
+-------------+ +-------------+ +-------------+ +-------+
+ + + +
------------- ------------- ------------- -------
| XOR | | XOR | | XOR | ... | XOR |
------------- ------------- ------------- -------
= = = =
+===+ +===+ +===+ +===+
|C_1| |C_2| |C_3| ... |C_L|
+===+ +===+ +===+ +===+
Figure 3: Generating interleaved (column) FEC packetsIn Figure 3, S_n and C_m stand for the source packet with sequence number n and the interleaved (column) FEC packet with sequence number m, respectively. 💜
to be continued, bestie 🫶
1.1. Use Cases
okay so, we generate one interleaved FEC packet out of D non-consecutive source packets 🧚 This repair packet can fully recover the missing info if only ONE packet is missing from those source packets. That's why 1-D interleaved FEC is so good against bursty loss, as long as L is big enough. L packet duration should not be shorter than the duration of the burst you're trying to repair. Main character energy for the burst-loss problem 💅
Check out Figure 4, where the sender makes interleaved FEC packets and a bursty loss hits the source packets. The number of columns is bigger than the number of packets lost to the burst, so the repair works and everything gets recovered 🎀
+---+
| 1 | X X X
+---+
+---+ +---+ +---+ +---+
| 5 | | 6 | | 7 | | 8 |
+---+ +---+ +---+ +---+
+---+ +---+ +---+ +---+
| 9 | | 10| | 11| | 12|
+---+ +---+ +---+ +---+
+===+ +===+ +===+ +===+
|C_1| |C_2| |C_3| |C_4|
+===+ +===+ +===+ +===+
Figure 4: Example scenario where 1-D interleaved FEC protection
succeeds error recoveryThe sender can make interleaved FEC packets to fight bursty packet losses 🦋 BUT two or more random packet losses might hit the source and repair packets in the same column, and then the repair fails 😭 That's shown in Figure 5. It's also possible for two or more bursty losses to happen in the same source block, and then interleaved FEC packets may still fail to recover the lost data 💀
+---+ +---+ +---+
| 1 | X | 3 | | 4 |
+---+ +---+ +---+
+---+ +---+ +---+
| 5 | X | 7 | | 8 |
+---+ +---+ +---+
+---+ +---+ +---+ +---+
| 9 | | 10| | 11| | 12|
+---+ +---+ +---+ +---+
+===+ +===+ +===+ +===+
|C_1| |C_2| |C_3| |C_4|
+===+ +===+ +===+ +===+
Figure 5: Example scenario where 1-D interleaved FEC protection fails
error recovery1.2. Overhead Computation
the overhead is the ratio of the bytes in the repair packets to the bytes in the protected source packets 🍓
Assume each repair packet carries an equal number of bytes to a source packet, and ignore the size of the FEC header. Then the overhead works out like this:
Overhead = 1/Dwhere D is the number of rows in the source block ✨
1.3. Relation to Existing Specifications
this section talks about how this spec connects to other existing specs 🌸
1.3.1. RFCs 2733 and 3009
this spec extends the FEC header defined in [RFC2733] and registers a new RTP payload format 🎀 The new payload format is NOT backward compatible with the payload format registered by [RFC3009]. Heads up, bestie 🙈
1.3.2. SMPTE 2022-1
okay so in 2007, the Society of Motion Picture and Television Engineers (SMPTE) - Technology Committee N26 on File Management and Networking Technology - decided to revise the Pro-MPEG Code of Practice (CoP) #3 Release 2 specification (first made by the Pro-MPEG Forum in 2004). That spec talked about several aspects of sending MPEG-2 transport streams over IP networks 📺 The new SMPTE spec is called [SMPTE2022-1].
The Pro-MPEG CoP #3 Release 2 doc was originally based on [RFC2733] 🦋 SMPTE revised it by extending the FEC header proposed in [RFC2733] (by setting the E bit). This extended header brings some upgrades ✨
For example, instead of using the bitmap field from [RFC2733], [SMPTE2022-1] adds separate fields to carry the number of rows (D) and columns (L) of the source block, plus the type of the repair packet (whether it's an interleaved FEC packet computed over a column, or a non-interleaved FEC packet computed over a row). These fields, together with the base sequence number, let the receiver connect source and repair packets 🧩 Note that even though the bitmap field isn't used, the FEC header of [SMPTE2022-1] still inherently carries over the bitmap field from [RFC2733]. Sneaky, right? 👀
On the other hand, some parts of [SMPTE2022-1] aren't compliant with RTP [RFC3550] 😤 For example, [SMPTE2022-1] sets the Synchronization Source (SSRC) field to zero and doesn't use the timestamp field in the RTP headers of the repair packets (receivers ignore the timestamps of the repair packets). It also sets the CSRC Count (CC) field in the RTP header to zero and doesn't allow any Contributing Source (CSRC) entry in the RTP header.
This document adopts the extended FEC header of [SMPTE2022-1] and registers a new RTP payload format. At the same time, it fixes the parts of [SMPTE2022-1] that aren't compliant with RTP [RFC3550], except the one discussed below 💅
The baseline header format first proposed in [RFC2733] doesn't have fields to protect the P and X bits and the CC fields of the source packets linked to a repair packet. Instead, the P bit, X bit, and CC field in the RTP header of the repair packet are used to protect those bits and fields. This can sometimes cause failures when doing the RTP header validity checks specified in [RFC3550]. While this behavior was fixed in [RFC5109], which obsoleted [RFC2733], the RTP payload format defined in this document still allows it for legacy purposes. Implementations following this specification must be aware of this potential issue when RTP header validity checks are applied ⚠️
1.3.3. ETSI TS 102 034
in 2009, the Digital Video Broadcasting (DVB) consortium published a technical specification [ETSI-TS-102-034] through the European Telecommunications Standards Institute (ETSI) 🇪🇺 It covers several areas related to sending MPEG-2 transport stream-based services over IP networks.
Annex E of [ETSI-TS-102-034] defines an optional protocol for Application-layer FEC (AL-FEC) protection of streaming media for DVB-IP services carried over RTP [RFC3550] transport 📡 The DVB-IPTV AL-FEC protocol uses two layers of protection: a base layer produced by a packet-based interleaved parity code, and an enhancement layer produced by a Raptor code [DVB-AL-FEC]. The enhancement layer is optional, but the base layer is mandatory wherever AL-FEC is used. The DVB-IPTV AL-FEC protocol is also described in [DVB-AL-FEC] 🌈
The interleaved parity code used in the base layer is a subset of [SMPTE2022-1]. Specifically, the AL-FEC base layer only uses the 1-D interleaved FEC protection from [SMPTE2022-1]. The new RTP payload format defined and registered in this document (with some exceptions listed in [DVB-AL-FEC]) is used as the AL-FEC base layer 💖
1.4. Scope of the Payload Format
the payload format in this document must only be used in legacy applications where the limitations explained in Section 1.3.2 are known not to impact any system components or other RTP elements 🕰️ Whenever possible, a payload format that is fully compliant with [RFC3550] must be used, such as [RFC5109] or other newer payload formats 🫶
2. Requirements Notation
the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119] 📖✨
3. Definitions, Notations, and Abbreviations
this section sums up the definitions and notations used in this document 💜
3.1. Definitions
this document uses the following definitions 🎀
- Source Flow: The packet flow(s) carrying the source data to which FEC protection is to be applied.
- Repair Flow: The packet flow(s) carrying the repair data.
- Symbol: A unit of data. Its size, in bytes, is referred to as the symbol size.
- Source Symbol: The smallest unit of data used during the encoding process.
- Repair Symbol: Repair symbols are generated from the source symbols.
- Source Packet: Data packets that contain only source symbols.
- Repair Packet: Data packets that contain only repair symbols.
- Source Block: A block of source symbols that are considered together in the encoding process.
3.2. Notations
- L: Number of columns of the source block.
- D: Number of rows of the source block.
4. Packet Formats 🎁
this section defines the formats of the source and repair packets 🧚
4.1. Source Packets
the source packets need to contain info that identifies the source block and the position within the source block the packet occupies 🗺️ Since the source packets carried within an RTP stream already contain unique sequence numbers in their RTP headers [RFC3550], we can identify the source packets in a straightforward way, so there's no need to append additional field(s). The big perk of not modifying the source packets in any way is backward compatibility for receivers that don't support FEC at all. In multicast scenarios this is super useful, because non-FEC-capable and FEC-capable receivers can both receive and interpret the same source packets sent in the same multicast session 🫶🌸
4.2. Repair Packets ✨
okay bestie, repair packets have to MUST contain info that says which source block they belong to and how their repair symbols relate to that original source block 🫶 so they borrow the RTP header plus another header tucked inside the RTP payload, called the FEC header. see Figure 6 💜
+------------------------------+
| IP Header |
+------------------------------+
| Transport Header |
+------------------------------+
| RTP Header | __
+------------------------------+ |
| FEC Header | \
+------------------------------+ > RTP Payload
| Repair Symbols | /
+------------------------------+ __|
Figure 6: Format of repair packetsthe RTP header follows [RFC3550] with some extra clarifications, so here's the tea 🍓
- Version: the version field is set to 2 ✨
- Padding (P) Bit: this bit is the XOR sum of the matching P bits from the RTP headers of the source packets this repair packet protects 🫧 BUT padding octets are never present in a repair packet, no matter what the P bit says.
- Extension (X) Bit: same energy! this bit is the XOR sum of the matching X bits from the source packets' RTP headers 💅 BUT an RTP header extension is never present in a repair packet, regardless of the X bit.
- CSRC Count (CC): this field is the XOR sum of the matching CC values from the source packets' RTP headers 🌸 BUT a CSRC list is never present in a repair packet, regardless of the CC field.
- Marker (M) Bit: this bit is the XOR sum of the matching M bits from the source packets' RTP headers 🦋
- Payload Type: the (dynamic) payload type for repair packets gets figured out out-of-band 🧚 this document registers a brand new payload format for repair packets (see Section 5 for the deets). according to [RFC3550], an RTP receiver that can't recognize a payload type must discard it. that's the backward compatibility moment 🌈 which means FEC can be used in a multicast group with a mix of FEC-capable and non-FEC-capable receivers. if a non-FEC-capable receiver gets a repair packet, it won't recognize the payload type and so it discards the repair packet. iconic, honestly 👑
- Sequence Number (SN): the sequence number has the standard definition 💖 it MUST be one higher than the sequence number in the previously transmitted repair packet. the initial value SHOULD be random (unpredictable) [RFC3550] 🎀
- Timestamp (TS): the timestamp SHALL be set to a time that matches the repair packet's transmission time ⭐ the timestamp value has no use in the actual FEC protection process, but it's usually handy for jitter calculations 💅
- Synchronization Source (SSRC): the SSRC value SHALL be randomly assigned, as suggested by [RFC3550] ✨ this lets the sender multiplex the source and repair flows on the same port, or multiplex multiple repair flows on a single port 🫶 the repair flows SHOULD use the RTP Control Protocol (RTCP) CNAME field to associate themselves with the source flow 💜
in some networks the RTP Source (which makes the source packets) and the FEC Source (which generates the repair packets from those source packets) might not be the same host 🌸 in that situation, using the same CNAME for the source and repair flows means the RTP Source and the FEC Source MUST share the same CNAME (for this specific source-repair flow association) 🔥 a common CNAME can be produced from an algorithm that both the RTP and FEC Source know. this usage is compliant with [RFC3550] ✨
and bestie, because the SSRC assignments are random, there's a possibility of an SSRC collision 😭 when that happens, the collisions MUST be resolved as described in [RFC3550] 🛠️
the P bit, X bit, CC field, and M bit of the source packets are protected by the matching bits and fields in the repair packet's RTP header. on the other hand, the payload of a repair packet protects the concatenation of (if present) the CSRC list, RTP extension, payload, and padding of the source RTP packets linked to this repair packet 🧚
the FEC header is 16 octets, and its format is shown in Figure 7 🦋
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SN base low | Length recovery |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|E| PT recovery | Mask |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| TS recovery |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|N|D|Type |Index| Offset | NA | SN base ext |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 7: Format of the FEC headerthe FEC header has these fields (it's a lot, but you've got this 💪):
- SN base low: tells you the lowest sequence number, taking wraparound into account, of the source packets this repair packet protects 🌈
- Length recovery: used to figure out the length of any recovered packets 🫧
- E bit: the extension flag introduced in [RFC2733], and it's used to extend the [RFC2733] FEC header 💜
- PT recovery: used to figure out the payload type of the recovered packets 🎀
- Mask: not used 🙈
- TS recovery: used to figure out the timestamp of the recovered packets ⭐
- N bit: the extension flag that's reserved for future use 🔮
- D bit: not used 🙈
- Type: tells you the type of error-correcting code being used. this document only defines one error-correcting code 👑
- Index: not used 🙈
- Offset and NA: these tell you the number of columns (L) and rows (D) of the source block, respectively 🍓
- SN base ext: not used 🙈
the details on setting these fields are in Section 6.2 ✨
okay so a Mask-based approach (like the one in [RFC2733]) might not be very efficient for showing which source packets in the current source block go with a given repair packet 😬 for apps that want big source block sizes, the Mask needed to describe those source-repair associations could get prohibitively large. a systematized approach is inherently more efficient, so that's the move 💅
5. Payload Format Parameters 🎀
this section has the media subtype registration for the 1-D interleaved parity FEC 🦋 it also defines the parameters you need to configure the FEC encoding and decoding operations, so it's the setup guide basically 🧚
5.1. Media Type Registration
this registration uses the template defined in [RFC4288] and follows the guidance in [RFC4855] 💜
5.1.1. Registration of audio/1d-interleaved-parityfec
Type name: audio
Subtype name: 1d-interleaved-parityfec
Required parameters:
- rate: the RTP timestamp (clock) rate in Hz 🌸 the (integer) rate SHALL be larger than 1000 so there's enough resolution for RTCP operations ✨ however, it is RECOMMENDED to pick the rate that matches the rate of the protected source RTP stream 💖
- L: the number of columns of the source block 🍓 L is a positive integer that's less than or equal to 255 🔢
- D: the number of rows of the source block 🍓 D is a positive integer that's less than or equal to 255 🔢
- repair-window: the time that spans the FEC block (so, the source packets plus the matching repair packets) 🫧 an FEC encoder processes a block of source packets and generates a bunch of repair packets, which then get sent within a duration not larger than the value of the repair window ⏳ on the receiver side, the FEC decoder should wait at least for the duration of the repair window after getting the first packet in an FEC block, so all the repair packets have time to arrive (the waiting time can be adjusted if there are missing packets at the beginning of the FEC block) 🌈 the FEC decoder can start decoding the packets it already has sooner, but it SHOULD NOT register an FEC decoding failure until it has waited at least for the repair-window duration 🙅♀️ the size of the repair window is specified in microseconds ⏱️
Optional parameters: None.
Encoding considerations: this media type is framed (see Section 4.8 in the template document [RFC4288]) and contains binary data 🫶
Security considerations: see Section 9 of [RFC6015] 🔐
Interoperability considerations: None.
Published specification: [RFC6015].
Applications that use this media type: multimedia apps that want to improve resiliency against packet loss by sending redundant data in addition to the source media 💪
Additional information: None.
Person & email address to contact for further information: Ali Begen <abegen@cisco.com> and the IETF Audio/Video Transport Working Group 💌
Intended usage: COMMON.
Restriction on usage: this media type depends on RTP framing, so it's only defined for transport via RTP [RFC3550] 🚫
Author: Ali Begen <abegen@cisco.com>.
Change controller: IETF Audio/Video Transport Working Group, delegated from the IESG 👑
5.1.2. Registration of video/1d-interleaved-parityfec ✨
okay bestie, now we're doing video 🎀 same registration vibe as audio, just with the video tag on it 📹
Type name: video
Subtype name: 1d-interleaved-parityfec
Required parameters:
rate: this is the RTP timestamp (clock) rate in Hz ⏱️ It's an integer and it SHALL be larger than 1000 so RTCP has enough resolution to do its job. However, it is RECOMMENDED to pick the rate that matches the protected source RTP stream 🌸L: the number of columns in the source block.Lis a positive integer that is less than or equal to 255 💖D: the number of rows in the source block.Dis a positive integer that is less than or equal to 255 💖repair-window: this is the time that spans the FEC block (so the source packets plus their matching repair packets) 🦋 The FEC encoder chews through a block of source packets and spits out a bunch of repair packets, and those get sent within a duration that's no bigger than the repair window. On the receiver side, the FEC decoder should wait at least the repair-window duration after the first packet of an FEC block shows up, so all the repair packets have time to arrive (you can adjust that wait if there are missing packets at the start of the block). The decoder can start decoding with what it already has, but it SHOULD NOT register an FEC decoding failure until it has waited at least the repair-window duration. The repair window is measured in microseconds ⏳
Optional parameters: None.
Encoding considerations: This media type is framed (see Section 4.8 in the template document [RFC4288]) and contains binary data 💾
Security considerations: See Section 9 of [RFC6015] 🔐
Interoperability considerations: None.
Published specification: [RFC6015] 📜
Applications that use this media type: Multimedia applications that want to level up their resiliency against packet loss by sending redundant data alongside the source media 🛡️✨
Additional information: None.
Person & email address to contact for further information: Ali Begen <abegen@cisco.com> and the IETF Audio/Video Transport Working Group 💌
Intended usage: COMMON.
Restriction on usage: This media type depends on RTP framing, so it's only defined for transport via RTP [RFC3550]. no RTP, no party 🚫🎉
Author: Ali Begen <abegen@cisco.com>.
Change controller: IETF Audio/Video Transport Working Group, delegated from the IESG 👑
5.1.3. Registration of text/1d-interleaved-parityfec
same energy, now with text 📝 the parameters are identical to the video one, so no need to reinvent the wheel 🙈
Type name: text
Subtype name: 1d-interleaved-parityfec
Required parameters:
rate: this is the RTP timestamp (clock) rate in Hz ⏱️ It's an integer and it SHALL be larger than 1000 so RTCP has enough resolution to do its job. However, it is RECOMMENDED to pick the rate that matches the protected source RTP stream 🌸L: the number of columns in the source block.Lis a positive integer that is less than or equal to 255 💖D: the number of rows in the source block.Dis a positive integer that is less than or equal to 255 💖repair-window: this is the time that spans the FEC block (so the source packets plus their matching repair packets) 🦋 The FEC encoder chews through a block of source packets and spits out a bunch of repair packets, and those get sent within a duration that's no bigger than the repair window. On the receiver side, the FEC decoder should wait at least the repair-window duration after the first packet of an FEC block shows up, so all the repair packets have time to arrive (you can adjust that wait if there are missing packets at the start of the block). The decoder can start decoding with what it already has, but it SHOULD NOT register an FEC decoding failure until it has waited at least the repair-window duration. The repair window is measured in microseconds ⏳
Optional parameters: None.
Encoding considerations: This media type is framed (see Section 4.8 in the template document [RFC4288]) and contains binary data 💾
Security considerations: See Section 9 of [RFC6015] 🔐
Interoperability considerations: None.
Published specification: [RFC6015] 📜
Applications that use this media type: Multimedia applications that want to level up their resiliency against packet loss by sending redundant data alongside the source media 🛡️✨
Additional information: None.
Person & email address to contact for further information: Ali Begen <abegen@cisco.com> and the IETF Audio/Video Transport Working Group 💌
Intended usage: COMMON.
Restriction on usage: This media type depends on RTP framing, so it's only defined for transport via RTP [RFC3550]. no RTP, no party 🚫🎉
Author: Ali Begen <abegen@cisco.com>.
Change controller: IETF Audio/Video Transport Working Group, delegated from the IESG 👑
5.1.4. Registration of application/1d-interleaved-parityfec
last one in this batch, now with application 💜 the template is the same again, so we're basically copy-pasting our homework (but make it aesthetic) 🧚
Type name: application
Subtype name: 1d-interleaved-parityfec
Required parameters:
rate: this is the RTP timestamp (clock) rate in Hz ⏱️ It's an integer and it SHALL be larger than 1000 so RTCP has enough resolution to do its job. However, it is RECOMMENDED to pick the rate that matches the protected source RTP stream 🌸L: the number of columns in the source block.Lis a positive integer that is less than or equal to 255 💖D: the number of rows in the source block.Dis a positive integer that is less than or equal to 255 💖repair-window: this is the time that spans the FEC block (so the source packets plus their matching repair packets) 🦋 The FEC encoder chews through a block of source packets and spits out a bunch of repair packets, and those get sent within a duration that's no bigger than the repair window. On the receiver side, the FEC decoder should wait at least the repair-window duration after the first packet of an FEC block shows up, so all the repair packets have time to arrive (you can adjust that wait if there are missing packets at the start of the block). The decoder can start decoding with what it already has, but it SHOULD NOT register an FEC decoding failure until it has waited at least the repair-window duration. The repair window is measured in microseconds ⏳
Optional parameters: None.
Encoding considerations: This media type is framed (see Section 4.8 in the template document [RFC4288]) and contains binary data 💾
Security considerations: See Section 9 of [RFC6015] 🔐
Interoperability considerations: None.
Published specification: [RFC6015] 📜
Applications that use this media type: Multimedia applications that want to level up their resiliency against packet loss by sending redundant data alongside the source media 🛡️✨
Additional information: None.
Person & email address to contact for further information: Ali Begen <abegen@cisco.com> and the IETF Audio/Video Transport Working Group 💌
Intended usage: COMMON.
Restriction on usage: This media type depends on RTP framing, so it's only defined for transport via RTP [RFC3550]. no RTP, no party 🚫🎉
Author: Ali Begen <abegen@cisco.com>.
Change controller: IETF Audio/Video Transport Working Group, delegated from the IESG 👑
5.2. Mapping to SDP Parameters
okay so now we connect this to SDP 🧃 apps using RTP usually describe their sessions with Session Description Protocol (SDP) [RFC4566], and the info that specifies the media types has specific spots in an SDP description. this section gives those mappings for the subtype this doc registers ("1d-interleaved-parityfec") 💅 And if an app doesn't use SDP to describe its RTP sessions, it MUST NOT be left guessing: an appropriate mapping must be defined and used for whatever control/description protocol that app runs, so the media types and their parameters still get specified 🫶
the mapping for "1d-interleaved-parity fec" and its parameters in SDP goes like this:
- the media type (e.g., "application") goes into the
m=line as the media name 🎯 - the media subtype ("1d-interleaved-parityfec") goes into the
a=rtpmapline as the encoding name. the RTP clock rate parameter ("rate") goes into that samea=rtpmapline as the clock rate ⏱️ - the remaining required payload-format-specific parameters go into the
a=fmtpline, copied straight from the media type string as a semicolon-separated list of parameter=value pairs 📋
SDP examples are provided in Section 7 🌈
5.2.1. Offer-Answer Model Considerations
when you're offering 1-D interleaved parity FEC over RTP using SDP in an Offer/Answer model [RFC3264], these considerations apply 👀
- each combination of the
LandDparameters produces different FEC data and isn't compatible with any other combination 🙅♀️ A sender app may want to offer multiple offers with different sets ofLandDvalues, as long as the parameter values are valid. The receiver SHOULD normally choose the offer that has a sufficient amount of interleaving. If multiple such offers exist, the receiver may choose the offer with the lowest overhead, or the one that needs the smallest amount of buffering. it depends on the app's requirements 💜 - the value for the
repair-windowparameter depends on theLandDvalues and can't be picked randomly 🎲 More specifically,LandDdecide the lower limit for the repair-window size. the upper limit of the repair-window size does NOT depend onLandD🧠 (lowercase "does not" is fine, I promise 😭 the original has it as plain text, so I'm keeping it plain!) - combinations with the same
LandDvalues but different repair-window sizes still produce the same FEC data, but they count as different offers anyway 🦋 The repair-window size is tied to the maximum delay between sending a source packet and sending its repair packet. That directly affects how much buffering the receiver needs, so the receiver has to think about that when choosing an offer 💭 - there are no optional format parameters defined for this payload. Any unknown option in the offer MUST be ignored and deleted from the answer. If the receiver doesn't want FEC, it can delete it from the answer 🗑️
5.2.2. Declarative Considerations
now for declarative usage, like SDP in the Real-time Streaming Protocol (RTSP) [RFC2326] or the Session Announcement Protocol (SAP) [RFC2974] 📣 these apply:
- the payload format configuration parameters are all declarative, and a participant MUST use the configuration that's provided for the session 🫡
- you can provide more than one configuration (if you want) by declaring multiple RTP payload types. In that case, receivers should choose the repair flow that's best for them 🌟
6. Protection and Recovery Procedures ✨
this is the main event 👑 This section gives a complete specification of the 1-D interleaved parity code and its RTP payload format 🧚♀️
6.1. Overview
the next sections spell out the steps for generating the repair packets and for rebuilding the missing source packets from those repair packets 🔥
6.2. Repair Packet Construction 🎀
okay so the RTP header of a repair packet gets built by following the guidelines in Section 4.2 ✨
The FEC header is 16 octets, bestie 💅 It's made by XORing together the bit strings generated from each of the source packets that this particular repair packet protects. You can work out which source packets belong to a given repair packet with the formula in Section 6.3.1.
Each source packet gets its bit string by sticking these fields together, in this exact order:
- Padding bit (1 bit) (This is the most significant bit of the bit string.)
- Extension bit (1 bit)
- CC field (4 bits)
- Marker bit (1 bit)
- PT field (7 bits)
- Timestamp (32 bits)
- Unsigned network-ordered 16-bit representation of the source packet length in bytes minus 12 (for the fixed RTP header), i.e., the sum of the lengths of all the following if present: the CSRC list, header extension, RTP payload, and RTP padding (16 bits).
- If CC is nonzero, the CSRC list (variable length)
- If X is 1, the header extension (variable length)
- Payload (variable length)
- Padding, if present (variable length)
Note that if the lengths of the source packets are not equal, each shorter packet MUST be padded to the length of the longest packet by adding octet(s) of 0 at the end. Because of this possible padding and the mandatory FEC header, a repair packet ends up bigger than the source packets it protects. That can be a problem 😭 if the resulting repair packet size goes over the Maximum Transmission Unit (MTU) size of the path the repair flow is sent over.
Once you've got the bit strings from the source packets, you apply the parity operation to produce the FEC bit string. Then some parts of the RTP header and FEC header of the repair packet get filled in from that FEC bit string, like this:
- The first (most significant) bit in the FEC bit string is written into the Padding bit in the RTP header of the repair packet.
- The next bit in the FEC bit string is written into the Extension bit in the RTP header of the repair packet.
- The next 4 bits of the FEC bit string are written into the CC field in the RTP header of the repair packet.
- The next bit of the FEC bit string is written into the Marker bit in the RTP header of the repair packet.
- The next 7 bits of the FEC bit string are written into the PT recovery field in the FEC header.
- The next 32 bits of the FEC bit string are written into the TS recovery field in the FEC header.
- The next 16 bits are written into the Length recovery field in the FEC header. This lets the FEC procedure work even when the lengths of the protected source packets aren't identical. 🧚
- The remaining bits are set to be the payload of the repair packet.
The rest of the FEC header is set up like so:
- The SN base low field MUST be set to the lowest sequence number, taking wraparound into account, of those source packets protected by this repair packet.
- The E bit MUST be set to 1 to extend the [RFC2733] FEC header.
- The Mask field SHALL be set to 0 and ignored by the receiver.
- The N bit SHALL be set to 0 and ignored by the receiver.
- The D bit SHALL be set to 0 and ignored by the receiver.
- The Type field MUST be set to 0 and ignored by the receiver.
- The Index field SHALL be set to 0 and ignored by the receiver.
- The Offset field MUST be set to the number of columns of the source block (L).
- The NA field MUST be set to the number of rows of the source block (D).
- The SN base ext field SHALL be set to 0 and ignored by the receiver.
6.3. Source Packet Reconstruction 🦋
This section covers the recovery procedures for rebuilding missing source packets. The recovery process has two steps. In step one, the FEC decoder figures out which source and repair packets it should use to recover a missing packet. In step two, the decoder actually recovers the missing packet, which is made of an RTP header and an RTP payload.
Below we describe the RECOMMENDED algorithms for the first and second steps. Depending on the implementation, different algorithms MAY be adopted. However, the end result MUST be identical to the one produced by the algorithms described below. 💜
6.3.1. Associating the Source and Repair Packets
Step one is associating the source and repair packets. The SN base low field in the FEC header shows the lowest sequence number of the source packets that form the particular column. Also, the media type parameters in the SDP description tell you how many source packets are available in each column and row. Together, that info uniquely identifies all of the source packets associated with a given repair packet. ✨
Mathematically, for any received repair packet, p*, we can determine the sequence numbers of the source packets that are protected by this repair packet like so:
p*_snb + i * L (modulo 65536)Here, p*_snb is the value in the SN base low field of the FEC header of p*, and L is the number of columns of the source block, and
0 <= i < Dwhere D is the number of rows of the source block.
We call the set of source packets associated with repair packet p* the set T(p*). In a source block that's L columns by D rows, set T includes D source packets. Here's the tea ☕: 1-D interleaved FEC protection can fully recover the missing information if there's only one source packet missing in set T. But if the repair packet protecting the source packets in set T is missing, or the repair packet is available but two or more source packets are missing, then 1-D interleaved FEC protection can't recover the missing source packets in set T. 😭
6.3.2. Recovering the RTP Header and Payload
For a given set T, here's how you recover the RTP header of the missing packet, the one whose sequence number is SEQNUM:
- For each of the source packets that are successfully received in set T, compute the bit string as described in Section 6.2.
- For the repair packet associated with set T, compute the bit string in the same fashion, except use the PT recovery field instead of the PT field, and the TS recovery field instead of the Timestamp field, and set the CSRC list, header extension and padding to null regardless of the values of the CC field, X bit, and P bit.
- If any of the bit strings generated from the source packets are shorter than the bit string generated from the repair packet, pad them to be the same length as the bit string generated from the repair packet. For padding, the padding of octet 0 MUST be added at the end of the bit string.
- Calculate the recovered bit string as the XOR of the bit strings generated from all source packets in set T and the FEC bit string generated from the repair packet associated with set T.
- Create a new packet with the standard 12-byte RTP header and no payload.
- Set the version of the new packet to 2.
- Set the Padding bit in the new packet to the first bit in the recovered bit string.
- Set the Extension bit in the new packet to the next bit in the recovered bit string.
- Set the CC field to the next 4 bits in the recovered bit string.
- Set the Marker bit in the new packet to the next bit in the recovered bit string.
- Set the Payload type in the new packet to the next 7 bits in the recovered bit string.
- Set the SN field in the new packet to SEQNUM.
- Set the TS field in the new packet to the next 32 bits in the recovered bit string.
- Take the next 16 bits of the recovered bit string and set the new variable Y to whatever unsigned integer this represents (assuming network order). Convert Y to host order and then take Y bytes from the recovered bit string and append them to the new packet. Y represents the length of the new packet in bytes minus 12 (for the fixed RTP header), i.e., the sum of the lengths of all the following if present: the CSRC list, header extension, RTP payload, and RTP padding.
- Set the SSRC of the new packet to the SSRC of the source RTP stream.
This procedure completely recovers both the header and payload of an RTP packet. 🎀
7. Session Description Protocol (SDP) Signaling 📞
This section has an SDP [RFC4566] example. The example uses the FEC grouping semantics [RFC5956].
In this example, we've got one source video stream (mid:S1) and one FEC repair stream (mid:R1). We form one FEC group with the "a=group: FEC-FR S1 R1" line. The source and repair streams are sent to the same port on different multicast groups. The repair window is set to 200 ms.
v=0
o=ali 1122334455 1122334466 IN IP4 fec.example.com
s=Interleaved Parity FEC Example
t=0 0
a=group:FEC-FR S1 R1
m=video 30000 RTP/AVP 100
c=IN IP4 233.252.0.1/127
a=rtpmap:100 MP2T/90000
a=mid:S1
m=application 30000 RTP/AVP 110
c=IN IP4 233.252.0.2/127
a=rtpmap:110 1d-interleaved-parityfec/90000
a=fmtp:110 L=5; D=10; repair-window=200000
a=mid:R18. Congestion Control Considerations 🌈
FEC is an effective way to give applications resiliency against packet losses. However, in networks where congestion is a major cause of packet loss, the potential impacts of using FEC SHOULD be considered carefully before injecting the repair flows into the network. Especially in bandwidth-limited networks, FEC repair flows can eat up most or all of the available bandwidth 💀 and may end up congesting the network. In those cases, the applications MUST NOT arbitrarily increase the amount of FEC protection, because doing so may lead to a congestion collapse. If desired, stronger FEC protection MAY be applied only after the source rate has been reduced.
In a network-friendly implementation, an application SHOULD NOT send/receive FEC repair flows if it knows that sending/receiving those FEC repair flows would not help at all in recovering the missing packets. This helps cut down on wasted bandwidth. It is RECOMMENDED that the amount of FEC protection is adjusted dynamically based on the packet loss rate observed by the applications. 🫶
In multicast scenarios, it may be difficult to optimize the FEC protection per receiver. If there's a large variation among the levels of FEC protection needed by different receivers, it is RECOMMENDED that the sender offers multiple repair flows with different levels of FEC protection, and the receivers join the corresponding multicast sessions to receive the repair flow(s) that is best for them. 👑
9. Security Considerations 🔐
Okay bestie, RTP packets using this payload format come with the same security considerations as the RTP spec [RFC3550] and any applicable RTP profile, so read those too 💅
The main security stuff for the RTP packet carrying this payload format is confidentiality, integrity, and source authenticity 🫶 Confidentiality comes from encrypting the RTP payload. Also, altering the FEC packets can have a huge impact on reconstruction 😭 An attack that flips some bits in the FEC packets can seriously mess with the calculation and recovery of the source packets. For example, changing the length recovery field can lead to recovering a packet that's too long 💀 Depending on the app, it may be helpful to sanity check the received source and FEC packets before recovery, and to check that recovered packets are valid before using them 🧐
Integrity of RTP packets is achieved through a suitable cryptographic integrity protection mechanism 🔒 That kind of system may also let you authenticate where the payload came from. A suitable security mechanism for this RTP payload format should provide source authentication that can determine whether an RTP packet is from a member of the RTP session 👑
The right mechanism to secure RTP and payloads following this memo can vary, depending on the application, transport, and signaling protocol being used 🦋 So one single mechanism isn't enough, although if it suits you, using the Secure Real-time Transport Protocol (SRTP) [RFC3711] is RECOMMENDED ✨ Other mechanisms that may be used are IPsec [RFC4301] and Transport Layer Security (TLS) [RFC5246], and other alternatives may exist 🌈
If FEC protection is applied on already encrypted source packets, there's no need for extra encryption 🍓 But if the source packets get encrypted after FEC protection is applied, the FEC packets should be cryptographically as secure as the source packets. Not giving the FEC packets an equal level of confidentiality, integrity, and authentication can compromise the source packets' confidentiality, integrity, or authentication, since the FEC packets are generated by applying an XOR operation across the source packets 🙈🔥
10. IANA Considerations 🌸
New media subtypes are subject to IANA registration. For the registration of the payload format and its parameters introduced in this document, go check out Section 5 🧚
11. Acknowledgments 🫶
A major part of this document is borrowed from [RFC2733], [RFC5109], and [SMPTE2022-1], so big thank yous to the authors and editors of those earlier specs 💖 The author also thanks Colin Perkins for his constructive suggestions for this document 👑
12. References 📚
12.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to
Indicate Requirement Levels", BCP 14, RFC 2119,
March 1997.
[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, RFC 3550, July 2003.
[RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP:
Session Description Protocol", RFC 4566,
July 2006.
[RFC5956] Begen, A., "Forward Error Correction Grouping
Semantics in Session Description Protocol",
RFC 5956, September 2010.
[RFC4288] Freed, N. and J. Klensin, "Media Type
Specifications and Registration Procedures",
BCP 13, RFC 4288, December 2005.
[RFC4855] Casner, S., "Media Type Registration of RTP
Payload Formats", RFC 4855, February 2007.
[RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer
Model with Session Description Protocol (SDP)",
RFC 3264, June 2002.12.2. Informative References
[DVB-AL-FEC] Begen, A. and T. Stockhammer, "Guidelines for
Implementing DVB-IPTV Application-Layer Hybrid FEC
Protection", Work in Progress, December 2009.
[RFC2733] Rosenberg, J. and H. Schulzrinne, "An RTP Payload
Format for Generic Forward Error Correction",
RFC 2733, December 1999.
[RFC3009] Rosenberg, J. and H. Schulzrinne, "Registration of
parityfec MIME types", RFC 3009, November 2000.
[RFC5109] Li, A., "RTP Payload Format for Generic Forward
Error Correction", RFC 5109, December 2007.
[ETSI-TS-102-034] ETSI TS 102 034 V1.4.1, "Transport of MPEG 2 TS
Based DVB Services over IP Based Networks",
August 2009.
[RFC2326] Schulzrinne, H., Rao, A., and R. Lanphier, "Real
Time Streaming Protocol (RTSP)", RFC 2326,
April 1998.
[RFC2974] Handley, M., Perkins, C., and E. Whelan, "Session
Announcement Protocol", RFC 2974, October 2000.
[SMPTE2022-1] SMPTE 2022-1-2007, "Forward Error Correction for
Real-Time Video/Audio Transport over IP Networks",
2007.
[RFC3711] Baugher, M., McGrew, D., Naslund, M., Carrara, E.,
and K. Norrman, "The Secure Real-time Transport
Protocol (SRTP)", RFC 3711, March 2004.
[RFC4301] Kent, S. and K. Seo, "Security Architecture for
the Internet Protocol", RFC 4301, December 2005.
[RFC5246] Dierks, T. and E. Rescorla, "The Transport Layer
Security (TLS) Protocol Version 1.2", RFC 5246,
August 2008.Author's Address 💁
Ali Begen
Cisco
181 Bay Street
Toronto, ON M5J 2T3
Canada
EMail: abegen@cisco.com📦 where this page comes from
- ietf: rfc6015 2010-10
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms