RFC 8854

WebRTC Forward Error Correction Requirements

📅 published January 2021, from ietf

RFC 8854: WebRTC Forward Error Correction Requirements 🎀

Okay bestie, this one is from the IETF (Internet Engineering Task Force) 💖. It was written by J. Uberti at Google ✨. It's a Standards Track document dated January 2021, with ISSN 2070-1721 📄. The title block doesn't list anything it obsoletes or updates, so there's no other RFC it replaces or changes 🦋.

Abstract ✨

This doc gives info and requirements for using Forward Error Correction (FEC) in WebRTC implementations 💅.

Status of This Memo 📋

This is an Internet Standards Track document 🌈. It's a product of the IETF, it represents the consensus of the IETF community, it got public review, and the IESG approved it for publication. For more on Internet Standards, check Section 2 of RFC 7841 🧚. The current status, any errata, and where to send feedback live at https://www.rfc-editor.org/info/rfc8854 💜.

Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved 🔥. This doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they stand on the publication date. Read those carefully, because they spell out your rights and restrictions for this doc 🙈. Code Components pulled out of this document have to include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come with no warranty, as the Simplified BSD License describes 🫶.

Table of Contents 🍓

1.  Introduction
2.  Terminology
3.  Types of FEC
  3.1.  Separate FEC Stream
  3.2.  Redundant Encoding
  3.3.  Codec-Specific In-Band FEC
4.  FEC for Audio Content
  4.1.  Recommended Mechanism
  4.2.  Negotiating Support
5.  FEC for Video Content
  5.1.  Recommended Mechanism
  5.2.  Negotiating Support
6.  FEC for Application Content
7.  Implementation Requirements
8.  Adaptive Use of FEC
9.  Security Considerations
10. IANA Considerations
11. References
  11.1.  Normative References
  11.2.  Informative References
Acknowledgements
Author's Address

1. Introduction 🌸

When packet loss is high, or perfect media quality really matters, FEC can proactively recover from packet losses 🛟. This spec gives guidance on which FEC mechanisms to use for WebRTC implementations, and how to use them 💅.

2. Terminology 📚

Okay so the key words in this doc are "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" 🫶. They're read as BCP 14 [RFC2119] [RFC8174] only when they appear in ALL CAPS, like shown here 🔥.

3. Types of FEC 🦋

FEC means sending redundant info in an outgoing packet stream, so the info can still be recovered even if packets get lost 💖. There are multiple ways to do this for RTP media streams [RFC3550], and this section goes through the mechanisms and their trade-offs ✨.

3.1. Separate FEC Stream 🎀

This approach, from [RFC5956], Section 4.3, sends FEC packets as their own RTP stream. That stream gets its own synchronization source (SSRC) [RFC3550] and payload type, and it's multiplexed with the primary encoding 🌈. One FEC packet can protect multiple primary packets, but each FEC packet has its own IP/UDP/RTP/FEC header. That overhead can get excessive, like when every primary packet gets its own FEC packet 🙈.

This approach can recover entire RTP packets, including the full RTP header 💜.

3.2. Redundant Encoding 🍓

This approach, from [RFC2198], piggybacks redundant data onto an existing primary encoding, all in one packet 🧚. The redundant data can be an exact copy of an earlier payload. For codecs that do variable-bitrate encodings, it could be a smaller, lower-quality version instead. Sometimes the redundant data includes encodings of multiple earlier audio frames 🦋.

Since there's only one set of packet headers, this is super efficient for primary and redundant data together 💅. The catch is that the savings only happen when all the data fits in a single packet (i.e. the size is less than a MTU) 😭. So this approach usually isn't useful for video content 💀.

As described in [RFC2198], Section 4, this approach can't recover certain parts of the RTP header, including the marker bit, contributing source (CSRC) information, and header extensions 🥺.

3.3. Codec-Specific In-Band FEC 🎶

Some audio codecs, like Opus [RFC6716] and Adaptive Multi-Rate (AMR) [RFC4867], have their own in-band FEC mechanism 💖. The redundant data goes inside the codec payload. It's similar to redundant encoding, but since it adds no extra framing, it can be a bit more efficient ✨.

For Opus, audio frames that matter get re-encoded at a lower bitrate and appended to the next payload. That allows partial recovery of a lost packet 🌸. This scheme is pretty efficient. Experiments showed that with Opus FEC, the overhead is only about 20-30%, depending on how much protection you need 🔥. Heads up though: this mechanism can only carry redundancy for the immediately preceding audio frame. So the decoder can't fully recover multiple consecutive lost packets, which can be a problem on wireless networks 💀. For more details, see [RFC6716], Section 2.1.7, and this Opus mailing list post [OpusFEC] 🧚.

For AMR and AMR-Wideband (AMR-WB), packets can contain copies or lower-quality encodings of multiple earlier audio frames 🌈. See [RFC4867], Section 3.7.1, for details 🎀.

In-band FEC mechanisms can't recover any of the RTP header 😭.

4. FEC for Audio Content 🎶

Here's guidance on how to best use FEC for sending audio 💅. As Section 8 below explains, FEC only turns on if network conditions call for it, or when the app explicitly asks for it 🫶.

When you're using variable-bitrate codecs that don't have an internal FEC, redundant encoding (from Section 3.2) with lower-fidelity versions of the previous packet(s) is RECOMMENDED 💖. It gives reasonable protection for the payload with only a moderate bitrate bump, since the redundant encodings can be a lot smaller than the primary encoding 🌸.

When you're using the Opus codec, the built-in Opus FEC mechanism is RECOMMENDED 🎀. It gives reasonable protection against individual losses, with minimal overhead 🦋. But as noted above, the built-in Opus FEC only gives single-frame redundancy. If you need multi-packet protection, the redundant encoding with reduced-bitrate Opus encodings SHOULD be used instead 🙈.

When you're using the AMR/AMR-WB codecs, their built-in FEC mechanism is RECOMMENDED 🌈. It protects the audio stream a bit more efficiently than redundant encoding does 💜.

When you're using constant-bitrate codecs, like PCMU [RFC5391], redundant encoding MAY be used 🧚. But it can cause a potentially significant bitrate increase, and suddenly cranking up the bitrate to handle congestion losses might actually make things worse 💀.

Audio encodings usually send one packet per frame, so the packet rate is lower. That makes a separate FEC stream come with more overhead than the other mechanisms, so it is NOT RECOMMENDED 😭.

Like we said above, the recommended mechanisms can't recover parts of the RTP header that matter in some audio apps, like CSRCs and RTP header extensions such as the ones in [RFC6464] and [RFC6465] 🥺. Implementations SHOULD account for this and try to approximate that info, using an approach similar to the ones in [RFC2198], Section 4, and [RFC6464], Section 5 🔥.

4.2. Negotiating Support 🤝

Support for redundant encoding of a given RTP stream SHOULD be indicated by including audio/red [RFC2198] as an extra supported media type for the matching m= section in the SDP offer [RFC3264] 💖. Answerers can reject redundant encoding by leaving audio/red out of the matching m= section in the SDP answer 🦋.

Support for codec-specific FEC mechanisms is usually indicated with a=fmtp parameters ✨.

For Opus, a receiver MUST indicate that it's ready to use incoming FEC data with the useinbandfec=1 parameter, as specified in [RFC7587] 🎀. This parameter is declarative, and it can be negotiated separately for each media direction 🌸.

For AMR/AMR-WB, support for redundant encoding and the maximum supported depth are controlled by the max-red parameter, as specified in [RFC4867], Section 8.1 🧚. Receivers MUST include this parameter and set it to an appropriate value, as specified in [TS.26114], Table 6.3 🔥.

5. FEC for Video Content 🎥

Here's guidance on how to best use FEC for sending video 💅. As Section 8 below explains, FEC only turns on if network conditions call for it, or when the app explicitly asks for it 🫶.

Video frames are big, so they often need multiple RTP packets 🦋. As discussed above, a separate FEC stream can protect multiple packets with one FEC packet 💖. The Flexible FEC mechanism from [RFC8627] can also protect multiple RTP streams with one FEC stream, including all the streams in a BUNDLE group [RFC8843] ✨. So for video content, a separate FEC stream using the Flexible FEC RTP payload format is RECOMMENDED 💜.

To process the incoming FEC stream, the receiver can demultiplex it by SSRC. Then it can link it to the right primary stream(s) using the CSRC(s) in the RTP header of Flexible FEC repair packets, or the SSRC field in the FEC header of Flexible FEC retransmission packets 🎀.

5.2. Negotiating Support 🤝

Support for an SSRC-multiplexed Flexible FEC stream protecting a given RTP stream SHOULD be indicated by including video/flexfec (described in [RFC8627], Section 5.1.2) as an extra supported media type for the matching m= section in the SDP offer [RFC3264] 🌸. As mentioned above, with BUNDLE, only one Flexible FEC repair stream gets created for each BUNDLE group, even if Flexible FEC is negotiated for each primary stream 🧚.

Answerers can reject SSRC-multiplexed FEC by leaving video/flexfec out of the matching m= section in the SDP answer 🦋.

Using FEC-only m= lines, and grouping them with the SDP group mechanism from [RFC5956], Section 4.1, isn't defined for WebRTC yet, so they SHOULD NOT be offered 🙈.

Answerers SHOULD reject any FEC-only m= lines, unless they specifically know how to handle that in a WebRTC context (maybe defined by a future version of the WebRTC specifications) 🔥.

6. FEC for Application Content ✨

Okay so WebRTC can also send generic application data 💅, and it has transport-level retransmission mechanisms that support both full and partial (like, timed) reliability 🦋. If you want the deets, go check [RFC8831].

Since the application gets to decide exactly what data gets sent, it can keep an eye on packet statistics and add its own application-level FEC if it needs to 🧚.

So this document makes no recommendations about FEC for the underlying data transport 🌸.

7. Implementation Requirements

Okay, to support the functionality I mentioned above 💖, implementations MUST be able to receive and use the relevant FEC formats for their supported audio codecs, and they MUST indicate that support, just like Section 4 explains 🫶. Using these formats when sending, like we talked about, is RECOMMENDED 🎀.

The general FEC mechanism in [RFC8627] SHOULD also be supported, same as Section 5 says 🌈.

Implementations MAY support additional FEC mechanisms if they want, e.g., [RFC5109] ✨.

8. Adaptive Use of FEC

Let's talk about the tradeoff 👑. Because FEC always means sending redundant data, and the total has to stay within whatever bandwidth limits congestion control and the receiver set, you end up with less bandwidth for the primary encoding, even when the redundant data never gets used 💀. That's different from RTX [RFC4588] or Flexible FEC's retransmission mode ([RFC8627], Section 1.1.7), which only send redundant data when it's actually needed 🦋. The catch is that those cost an extra round trip, so media latency goes up 😭.

So, WebRTC implementations SHOULD prefer RTX or Flexible FEC retransmissions over FEC when the connection RTT fits inside the application's latency budget 🫶. Otherwise, they SHOULD only send as much FEC as is needed to cover the packet loss they've actually observed 📊. You can figure out that loss by watching transmit packet loss data in RTP Control Protocol (RTCP) receiver reports [RFC3550] 🔍. The exception is when the application says it's cool with a quality penalty in exchange for proactively dodging losses 🙈.

Now, when you're probing bandwidth (which means speculatively sending extra data to see if there's more link capacity), FEC data SHOULD be used as that extra data 🚀. Since that extra data is getting sent anyway, it makes total sense for it to protect the primary payload 🍓. Plus, FEC can usually be applied in a way that only bumps up bandwidth a little, which is exactly what you want while probing 💜.

When you're using FEC with layered codecs, like [RFC6386], where only base layer frames are critical for decoding future frames, implementations SHOULD only apply FEC to those base layer frames 🌸.

Last but not least, add redundancy as much as you like 🎀, but if the loss is caused by network congestion, all that extra bandwidth from redundant data can actually make things worse and seriously degrade the network 💀🔥.

9. Security Considerations

Okay, security time 🔒. In the WebRTC context, FEC is specifically about recovering data from lost packets 🧩. Any corrupted packets get thrown out by the Secure Real-Time Transport Protocol (SRTP) [RFC3711] decryption process 🗑️. So, as described in [RFC3711], Section 10, the default processing when you use FEC with SRTP is to do FEC first and then SRTP at the sender, and SRTP first and then FEC at the receiver 🔁. This ordering is used for all the SRTP protection profiles used in DTLS-SRTP [RFC5763], which are listed in [RFC5764], Section 4.1.2 👑.

Every individual FEC mechanism has its own additional security considerations, and those live in their respective documents 📚.

10. IANA Considerations

Good news for IANA: this document requires no actions from you 🦋.

11. References

11.1. Normative References

These are the must-reads 📖:

[RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
           Requirement Levels", BCP 14, RFC 2119,
           DOI 10.17487/RFC2119, March 1997,
           <https://www.rfc-editor.org/info/rfc2119>.

[RFC2198]  Perkins, C., Kouvelas, I., Hodson, O., Hardman, V.,
           Handley, M., Bolot, J.C., Vega-Garcia, A., and S. Fosse-
           Parisis, "RTP Payload for Redundant Audio Data", RFC 2198,
           DOI 10.17487/RFC2198, September 1997,
           <https://www.rfc-editor.org/info/rfc2198>.

[RFC3264]  Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model
           with Session Description Protocol (SDP)", RFC 3264,
           DOI 10.17487/RFC3264, June 2002,
           <https://www.rfc-editor.org/info/rfc3264>.

[RFC4867]  Sjoberg, J., Westerlund, M., Lakaniemi, A., and Q. Xie,
           "RTP Payload Format and File Storage Format for the
           Adaptive Multi-Rate (AMR) and Adaptive Multi-Rate Wideband
           (AMR-WB) Audio Codecs", RFC 4867, DOI 10.17487/RFC4867,
           April 2007, <https://www.rfc-editor.org/info/rfc4867>.

[RFC5956]  Begen, A., "Forward Error Correction Grouping Semantics in
           the Session Description Protocol", RFC 5956,
           DOI 10.17487/RFC5956, September 2010,
           <https://www.rfc-editor.org/info/rfc5956>.

[RFC7587]  Spittka, J., Vos, K., and JM. Valin, "RTP Payload Format
           for the Opus Speech and Audio Codec", RFC 7587,
           DOI 10.17487/RFC7587, June 2015,
           <https://www.rfc-editor.org/info/rfc7587>.

[RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
           2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
           May 2017, <https://www.rfc-editor.org/info/rfc8174>.

[RFC8627]  Zanaty, M., Singh, V., Begen, A., and G. Mandyam, "RTP
           Payload Format for Flexible Forward Error Correction
           (FEC)", RFC 8627, DOI 10.17487/RFC8627, July 2019,
           <https://www.rfc-editor.org/info/rfc8627>.

[TS.26114] 3GPP, "IP Multimedia Subsystem (IMS); Multimedia
           telephony; Media handling and interaction", 3GPP TS 26.114
           15.0.0, 22 September 2017,
           <http://www.3gpp.org/ftp/Specs/html-info/26114.htm>.

11.2. Informative References

And these are the extra-credit reads 🤓🌈:

[OpusFEC]  Terriberry, T., "Subject: Opus FEC", message to the opus
           mailing list, 28 January 2013,
           <http://lists.xiph.org/pipermail/
           opus/2013-January/001904.html>.

[RFC3550]  Schulzrinne, H., Casner, S., Frederick, R., and V.
           Jacobson, "RTP: A Transport Protocol for Real-Time
           Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550,
           July 2003, <https://www.rfc-editor.org/info/rfc3550>.

[RFC3711]  Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K.
           Norrman, "The Secure Real-time Transport Protocol (SRTP)",
           RFC 3711, DOI 10.17487/RFC3711, March 2004,
           <https://www.rfc-editor.org/info/rfc3711>.

[RFC4588]  Rey, J., Leon, D., Miyazaki, A., Varsa, V., and R.
           Hakenberg, "RTP Retransmission Payload Format", RFC 4588,
           DOI 10.17487/RFC4588, July 2006,
           <https://www.rfc-editor.org/info/rfc4588>.

[RFC5109]  Li, A., Ed., "RTP Payload Format for Generic Forward Error
           Correction", RFC 5109, DOI 10.17487/RFC5109, December
           2007, <https://www.rfc-editor.org/info/rfc5109>.

[RFC5391]  Sollaud, A., "RTP Payload Format for ITU-T Recommendation
           G.711.1", RFC 5391, DOI 10.17487/RFC5391, November 2008,
           <https://www.rfc-editor.org/info/rfc5391>.

[RFC5763]  Fischl, J., Tschofenig, H., and E. Rescorla, "Framework
           for Establishing a Secure Real-time Transport Protocol
           (SRTP) Security Context Using Datagram Transport Layer
           Security (DTLS)", RFC 5763, DOI 10.17487/RFC5763, May
           2010, <https://www.rfc-editor.org/info/rfc5763>.

[RFC5764]  McGrew, D. and E. Rescorla, "Datagram Transport Layer
           Security (DTLS) Extension to Establish Keys for the Secure
           Real-time Transport Protocol (SRTP)", RFC 5764,
           DOI 10.17487/RFC5764, May 2010,
           <https://www.rfc-editor.org/info/rfc5764>.

[RFC6386]  Bankoski, J., Koleszar, J., Quillio, L., Salonen, J.,
           Wilkins, P., and Y. Xu, "VP8 Data Format and Decoding
           Guide", RFC 6386, DOI 10.17487/RFC6386, November 2011,
           <https://www.rfc-editor.org/info/rfc6386>.

[RFC6464]  Lennox, J., Ed., Ivov, E., and E. Marocco, "A Real-time
           Transport Protocol (RTP) Header Extension for Client-to-
           Mixer Audio Level Indication", RFC 6464,
           DOI 10.17487/RFC6464, December 2011,
           <https://www.rfc-editor.org/info/rfc6464>.

[RFC6465]  Ivov, E., Ed., Marocco, E., Ed., and J. Lennox, "A Real-
           time Transport Protocol (RTP) Header Extension for Mixer-
           to-Client Audio Level Indication", RFC 6465,
           DOI 10.17487/RFC6465, December 2011,
           <https://www.rfc-editor.org/info/rfc6465>.

[RFC6716]  Valin, JM., Vos, K., and T. Terriberry, "Definition of the
           Opus Audio Codec", RFC 6716, DOI 10.17487/RFC6716,
           September 2012, <https://www.rfc-editor.org/info/rfc6716>.

[RFC8831]  Jesup, R., Loreto, S., and M. Tüxen, "WebRTC Data
           Channels", RFC 8831, DOI 10.17487/RFC8831, January 2021,
           <https://www.rfc-editor.org/info/rfc8831>.

[RFC8843]  Holmberg, C., Alvestrand, H., and C. Jennings,
           "Negotiating Media Multiplexing Using the Session
           Description Protocol (SDP)", RFC 8843,
           DOI 10.17487/RFC8843, January 2021,
           <https://www.rfc-editor.org/info/rfc8843>.

Acknowledgements

Huge shoutout 📣 to the people who gave significant input to this document: Bernard Aboba, Jonathan Lennox, Giri Mandyam, Varun Singh, Tim Terriberry, Magnus Westerlund, and Mo Zanaty 🫶💜.

Author's Address

Justin Uberti
Google
747 6th St S
Kirkland, WA 98033
United States of America

Email: justin@uberti.name
📦 where this page comes from