RFC 2198
RTP Payload for Redundant Audio Data
RFC 2198: RTP Payload for Redundant Audio Data 🎀
okay bestie, this is RFC 2198 and it's giving audio resilience ✨. It's a Standards Track doc from September 1997, written by C. Perkins, I. Kouvelas, O. Hodson, and V. Hardman from University College London, M. Handley from ISI, and J.C. Bolot, A. Vega-Garcia, and S. Fosse-Parisis from INRIA Sophia Antipolis 🌈. The title block doesn't list anything it obsoletes or updates, so no drama there 💅.
Status of This Memo 📋
okay so this doc specifies an Internet standards track protocol for the Internet community, and it requests discussion and suggestions for improvements 🫶. For the standardization state and status of this protocol, check the current edition of the "Internet Official Protocol Standards" (STD 1). Distribution of this memo is unlimited 🦋.
Abstract 🌸
this doc describes a payload format for the real-time transport protocol (RTP), version 2, for encoding redundant audio data 🎧. The main motivation is building audio conferencing tools for lossy packet networks like the Internet Mbone, but this scheme isn't limited to those apps 🔥.
1. Introduction 💖
okay so if multimedia conferencing is going to be widely used by the Internet Mbone community, users have to think the quality is good enough for most apps 😭. We found a bunch of problems that wreck conference quality, and the biggest one is packet loss. It's a persistent problem, especially since the Internet keeps getting more popular and therefore more loaded 💀. Speech intelligibility gets disrupted even at low loss rates, and people are experiencing that right now, which could convince a whole generation that multimedia conferencing over the Internet isn't viable 🙈. Adding redundancy to the data stream is offered as a solution [1]. If a packet is lost, the missing info can be reconstructed at the receiver from the redundant data in the following packet(s), as long as the average number of consecutively lost packets is small ✨. Recent work [4,5] shows that packet loss patterns on the Internet mean this scheme typically works really well 🎀.
this doc describes an RTP payload format for sending audio data encoded this redundant way 🧚. Section 2 covers the requirements and motivation behind this payload format, and it isn't part of the payload format definition itself. Sections 3 onwards define the RTP payload format for redundant audio data 🍓.
2. Requirements/Motivation 🎀
so the requirements for a redundant encoding scheme under RTP are:
- Packets have to carry a primary encoding and one or more redundant encodings.
- As a multitude of encodings may be used for redundant information, each block of redundant encoding has to have an encoding type identifier.
- As the use of variable size encodings is desirable, each encoded block in the packet has to have a length indicator.
- The RTP header provides a timestamp field that corresponds to the time of creation of the encoded data. When redundant encodings are used, this timestamp field can refer to the time of creation of the primary encoding data. Redundant blocks of data will correspond to different time intervals than the primary data, so each block of redundant encoding will require its own timestamp. To reduce the number of bytes needed to carry the timestamp, it can be encoded as the difference of the timestamp for the redundant encoding and the timestamp of the primary. 🫶
there are two essential ways redundant audio can be added to the standard RTP specification: a header extension may hold the redundancy, or one or more additional payload types may be defined 👑.
putting all the redundancy info for a packet in a header extension would make it easy for apps that don't implement redundancy to discard it and just process the primary encoding data. But there are a bunch of disadvantages with this scheme, and honestly they're kind of a lot 😭:
- There is a large overhead from the number of bytes needed for the extension header (4) and the possible padding that is needed at the end of the extension to round up to a four byte boundary (up to 3 bytes). For many applications this overhead is unacceptable. 💀
- Use of the header extension limits applications to a single redundant encoding, unless further structure is introduced into the extension. This would result in further overhead. 🙄
so for these reasons, using an RTP header extension to hold redundant audio encodings is disregarded 🚫.
the RTP profile for audio and video conferences [3] lists a set of payload types and provides for a dynamic range of 32 encodings that may be defined through a conference control protocol 🦋. This leads to two possible schemes for assigning additional RTP payload types for redundant audio apps:
- A dynamic encoding scheme may be defined, for each combination of primary/redundant payload types, using the RTP dynamic payload type range.
- A single fixed payload type may be defined to represent a packet with redundancy. This may then be assigned to either a static RTP payload type, or the payload type for this may be assigned dynamically.
it's possible to define a set of payload types that signify a particular combination of primary and secondary encodings for each of the 32 dynamic payload types provided. This would be a slightly restrictive yet feasible solution for packets with a single block of redundancy, since the number of possible combinations isn't too large 🌈. However, the need for multiple blocks of redundancy greatly increases the number of encoding combinations and makes this solution not viable 😤.
a modified version of the above solution could be to decide, prior to the beginning of a conference, on a set a 32 encoding combinations that will be used for the duration of the conference ⭐. All tools in the conference can be initialized with this working set of encoding combinations. The working set could be communicated through an external, out of band, mechanism. Setup is complicated, though, since great care needs to be taken in starting tools with identical parameters 🙈. This scheme is more efficient because only one byte is used to identify combinations of encodings ✨.
it's felt that the complication inherent in distributing the mapping of payload types onto combinations of redundant data precludes the use of this mechanism 🚫.
a more flexible solution is to have a single payload type which signifies a packet with redundancy. That packet then becomes a container, encapsulating multiple payloads into a single RTP packet. Such a scheme is flexible, since any amount of redundancy may be encapsulated within a single packet. There is, however, a small overhead, since each encapsulated payload must be preceded by a header indicating the type of data enclosed. This is the preferred solution, since it is both flexible, extensible, and has a relatively low overhead 👑🫶.
the remainder of this document describes this solution 🧚💜.
3. Payload Format Specification ✨
okay bestie, this is where the actual rules live 💅 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be read the way RFC2119 [7] describes 🎀
Okay so, the RTP payload type for this new packet format is not assigned here. That's outside the scope of this doc 🙈 The idea is that the RTP profile for a particular class of applications will assign a payload type for this encoding. If that doesn't happen, then a payload type in the dynamic range shall be chosen 🦋
An RTP packet with redundant data shall have a standard RTP header, and the payload type in that header says "redundancy" 🌸 The other RTP header fields are about the primary data block of the redundant data. So the primary is the main character, and the redundant stuff is the supporting cast 👑
After the RTP header come a bunch of extra headers, shown in the figure below. These describe each of the encodings carried in the packet. After those headers come the data blocks, which hold the standard RTP payload data for each encoding. Quick note: all the headers are aligned to a 32 bit boundary, but the payload data will typically not be aligned 🧚 Also, if multiple redundant encodings are carried in a packet, they should correspond to different time intervals. There's no reason to include multiple copies of data for a single time interval within a packet. Pls don't be that person 💀
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|F| block PT | timestamp offset | block length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+The bits in the header work like this 🍓
F: 1 bit. The first bit in the header tells you whether another header block follows. If it's 1, further header blocks follow. If it's 0, this is the last header block.block PT: 7 bits. The RTP payload type for this block.timestamp offset: 14 bits. An unsigned offset of the timestamp of this block, relative to the timestamp given in the RTP header. Because the offset is unsigned, redundant data must be sent after the primary data. So it's a time to be subtracted from the current timestamp to find the timestamp of the data this block is the redundancy for. Wild, right? ✨block length: 10 bits. The length in bytes of the corresponding data block, excluding the header.
Now, the unsigned timestamp offset does limit redundant data a little bit 🥺 It's not possible to send redundancy before the primary encoding. This might affect schemes where a low bandwidth coding that's good for redundancy gets produced early in the encoding process, and so could feasibly be sent early. BUT adding a sign bit would unacceptably shrink the range of the timestamp offset. And making the field bigger than 14 bits would eat into the block length field. Honestly, limiting redundancy to being sent after the primary causes fewer problems than limiting the size of the other fields. Choices, choices 💅
The timestamp offset for a redundant block is measured in the same units as the timestamp of the primary encoding. That means audio samples, with the same clock rate as the primary. So the redundant encoding MUST be sampled at the same rate as the primary. No exceptions, bestie 🔥
Also, the block length and timestamp offset are 10 bits and 14 bits respectively, instead of the more obvious 8 and 16 bits. Yes, this makes parsing the header info a bit more complicated and adds some processing overhead 🙃 But the more obvious choice has problems. An 8 bit block length field is enough for most encodings, but not all. For example, 80ms PCM and DVI audio packets are more than 256 bytes, so they can't be encoded with a single byte length field. You could add extra structure to the block length field (for example, the high bit set could mean the lower 7 bits code a length in words, not bytes). However, those schemes are complex. So the 10 bit block length field keeps things simple and gives a bigger range, at the cost of a smaller range of timestamp values. Trade-offs era 🌈
The primary encoding block header goes last in the packet. That means you can leave the timestamp and block-length fields out of this block's header, because they can be worked out from the RTP header and the overall packet length 🧩 So the header for the primary (final) block is just a zero F bit plus the block payload type information. That's 8 bits total. Here's the picture:
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|0| Block PT |
+-+-+-+-+-+-+-+-+The final header is followed right away by the data blocks, stored in the same order as the headers. There's no padding or other delimiter between the data blocks, and they're typically not 32 bit aligned. Again, this was chosen to reduce bandwidth overhead, at the cost of extra decoding time 💖
The choice of encodings should reflect the bandwidth requirements of those encodings. The redundant encoding is expected to use significantly less bandwidth than the primary encoding. The exception is when the primary is very low-bandwidth and has high processing requirements. In that case, a copy of the primary MAY be used as the redundancy 🎀 But the redundant encoding MUST NOT be higher bandwidth than the primary. That's the golden rule 👑
Multiple levels of redundancy are rarely necessary. But when they're needed, each level of redundancy is expected to use significantly less bandwidth than the level before it 🦋
4. Limitations 🙈
Real talk, there are a few things this format just can't do. The RTP marker bit is not preserved for redundant data blocks. So if the primary (the one containing the marker) gets lost, the marker is lost too 😭 It's believed this won't cause undue problems. Even if the marker bit were sent with the redundant info, it could still get lost, so applications would still have to be written with that in mind anyway 🫶
Also, CSRC information is not preserved for redundant data. The CSRC data in the RTP header of a redundant audio packet only relates to the primary. Since CSRC data in an audio stream is expected to change pretty infrequently, it is recommended that applications which need this info assume the CSRC data in the RTP header may be applied to the reconstructed redundant data. Lowkey a practical workaround, iykyk ✨
5. Relation to SDP 🎀
When a redundant payload is used, it may need to be bound to an RTP dynamic payload type. This can be done with any out-of-band mechanism, but one common way is to communicate the binding using the Session Description Protocol (SDP) [6]. SDP has a mechanism for binding dynamic payload types to a particular codec, sample rate, and number of channels, using the "rtpmap" attribute. Here's an example of how it's used, with the RTP audio/video profile [3]:
m=audio 12345 RTP/AVP 121 0 5
a=rtpmap:121 red/8000/1So this says an audio stream using RTP is using payload types 121 (a dynamic payload type), 0 (PCM u-law), and 5 (DVI). The "rtpmap" attribute binds payload type 121 to the codec "red", which tells you this codec is actually a redundancy frame. It's 8KHz and monaural 🌸 When used with SDP, the term "red" means the redundancy format discussed in this document.
Here, the extra formats of PCM and DVI are specified, so the receiver must be prepared to use those formats. This means the sender will send redundancy by default, but may also send PCM or DVI 💅 With a redundant payload, we also take it to mean that no codec other than PCM or DVI will be used in the redundant encodings. Also, the extra payload formats in the "m=" field may themselves be dynamic payload types. If so, you may need a few more "a=" attributes to describe them 🧚
To receive a redundant stream, that's all you need ✨ But to send one, the sender needs to know which codecs are recommended for the primary and secondary (and tertiary, etc.) encodings. That info is specific to the redundancy format and is conveyed with an additional attribute called "fmtp", which carries format-specific information. A session directory doesn't parse the values in an fmtp attribute. It just hands them to the media tool unchanged 🙈 For redundancy, the format parameters are defined as a slash "/" separated list of RTP payload types.
So a complete example looks like this:
m=audio 12345 RTP/AVP 121 0 5
a=rtpmap:121 red/8000/1
a=fmtp:121 0/5This says the default format for senders is redundancy, with PCM as the primary encoding and DVI as the secondary encoding. Encodings can't be specified in the fmtp attribute unless they're also specified as valid encodings on the media ("m=") line. Rules are rules 💜
6. Security Considerations 💀
Okay, this section is serious, so buckle up 🔥 RTP packets containing redundant information are subject to the security considerations discussed in the RTP specification [2], and in any appropriate RTP profile (for example [3]). This means confidentiality of the media streams is achieved by encryption. Encryption of a redundant data stream can happen in two ways:
- The entire stream is to be secured, and all participants are expected to have keys to decode the entire stream. In this case, nothing special needs to be done, and encryption is performed in the usual manner.
- A portion of the stream is to be encrypted with a different key than the remainder. In this case, a redundant copy of the last packet of that portion cannot be sent, since there's no following packet encrypted with the correct key to send it in. Similar limitations may occur when enabling or disabling encryption 😭
The choice between these two is a matter for the encoder only. Decoders can decrypt either form without modification 🫶
Whilst adding low-bandwidth redundancy to an audio stream is an effective way to protect that stream against packet loss, application designers should be aware of something. Adding large amounts of redundancy will increase network congestion, and that means more packet loss. This leads to a worsening of the very problem redundancy was meant to solve 🚨 At its worst, this can lead to excessive network congestion, and it may constitute a denial of service attack. So please, be chill with the redundancy, bestie 🌈
7. Example Packet ✨
okay bestie, time for the picture 🎀 this is an RTP audio packet carrying a DVI4 (8KHz) primary, plus one block of redundancy encoded with 8KHz LPC (both are 20ms packets). The layout comes from the RTP audio/video profile [3], and it's giving diagram 🦋
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|X| CC=0 |M| PT | sequence number of primary |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| timestamp of primary encoding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| synchronization source (SSRC) identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1| block PT=7 | timestamp offset | block length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| block PT=5 | |
+-+-+-+-+-+-+-+-+ +
| |
+ LPC encoded redundant data (PT=7) +
| (14 bytes) |
+ +---------------+
| | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +
| |
+ +
| |
+ +
| |
+ +
| DVI4 encoded primary data (PT=5) |
+ (84 bytes, not to scale) +
/ /
+ +
| |
+ +
| |
+ +---------------+
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+8. Authors' Addresses 💌
the squad behind this one 🫶
Colin Perkins/Isidor Kouvelas/Orion Hodson/Vicky Hardman
Department of Computer Science
University College London
London WC1E 6BT
United Kingdom
EMail: {c.perkins|i.kouvelas|o.hodson|v.hardman}@cs.ucl.ac.uk
Mark Handley
USC Information Sciences Institute
c/o MIT Laboratory for Computer Science
545 Technology Square
Cambridge, MA 02139, USA
EMail: mjh@isi.edu
Jean-Chrysostome Bolot/Andres Vega-Garcia/Sacha Fosse-Parisis
INRIA Sophia Antipolis
2004 Route des Lucioles, BP 93
06902 Sophia Antipolis
France
EMail: {bolot|avega|sfosse}@sophia.inria.fr9. References 📚
[1] V.J. Hardman, M.A. Sasse, M. Handley and A. Watson; Reliable Audio for Use over the Internet; Proceedings INET'95, Honalulu, Oahu, Hawaii, September 1995. http://www.isoc.org/in95prc/
[2] Schulzrinne, H., Casner, S., Frederick R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", RFC 1889, January 1996.
[3] Schulzrinne, H., "RTP Profile for Audio and Video Conferences with Minimal Control", RFC 1890, January 1996.
[4] M. Yajnik, J. Kurose and D. Towsley; Packet loss correlation in the MBone multicast network; IEEE Globecom Internet workshop, London, November 1996
[5] J.-C. Bolot and A. Vega-Garcia; The case for FEC-based error control for packet audio in the Internet; ACM Multimedia Systems, 1997
[6] Handley, M., and V. Jacobson, "SDP: Session Description Protocol (draft 03.2)", Work in Progress.
[7] Bradner, S., "Key words for use in RFCs to indicate requirement levels", RFC 2119, March 1997.
📦 where this page comes from
- ietf: rfc2198 1997-09
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms