RFC 3551
RTP Profile for Audio and Video Conferences with Minimal Control
RFC 3551: RTP Profile for Audio and Video Conferences with Minimal Control 🎀
Okay bestie, this is RFC 3551 💖 It's an Internet Standards Track doc from July 2003, written by H. Schulzrinne (Columbia University) and S. Casner (Packet Design). It obsoletes RFC 1890, so that older one is out. 🦋
Status of this Memo
This document specifies an Internet standards track protocol for the Internet community, and requests discussion and suggestions for improvements. Please refer to the current edition of the "Internet Official Protocol Standards" (STD 1) for the standardization state and status of this protocol. Distribution of this memo is unlimited. 📜
Copyright Notice
Copyright (C) The Internet Society (2003). All Rights Reserved. ©️
Abstract ✨
So this doc describes a profile called "RTP/AVP" for using the real-time transport protocol (RTP), version 2, plus its buddy control protocol, RTCP, inside audio and video conferences with minimal control. It basically gives you interpretations of the generic RTP fields that actually fit audio and video calls 💅 It also sets a default mapping from payload type numbers to encodings, which is a big deal. 🔥
It also explains how audio and video data gets carried inside RTP. It defines a set of standard encodings and their names when they're used within RTP. The descriptions point you to reference implementations and the detailed standards, so you can go deeper. This memo is meant as a helper for people building audio, video, and other real-time multimedia apps. 🫶
Also, this memo obsoletes RFC 1890. It's mostly backwards-compatible, except for functions that got removed because two interoperable implementations were never found 😭 The additions codify existing practice for using payload formats under this profile, and they include new payload formats defined since RFC 1890 was published. Iconic glow-up 🌈
Table of Contents
1. Introduction ................................................. 3
1.1 Terminology ............................................. 3
2. RTP and RTCP Packet Forms and Protocol Behavior .............. 4
3. Registering Additional Encodings ............................. 6
4. Audio ........................................................ 8
4.1 Encoding-Independent Rules .............................. 8
4.2 Operating Recommendations ............................... 9
4.3 Guidelines for Sample-Based Audio Encodings ............. 10
4.4 Guidelines for Frame-Based Audio Encodings .............. 11
4.5 Audio Encodings ......................................... 12
4.5.1 DVI4 ............................................ 13
4.5.2 G722 ............................................ 14
4.5.3 G723 ............................................ 14
4.5.4 G726-40, G726-32, G726-24, and G726-16 .......... 18
4.5.5 G728 ............................................ 19
4.5.6 G729 ............................................ 20
4.5.7 G729D and G729E ................................. 22
4.5.8 GSM ............................................. 24
4.5.9 GSM-EFR ......................................... 27
4.5.10 L8 .............................................. 27
4.5.11 L16 ............................................. 27
4.5.12 LPC ............................................. 27
4.5.13 MPA ............................................. 28
4.5.14 PCMA and PCMU ................................... 28
4.5.15 QCELP ........................................... 28
4.5.16 RED ............................................. 29
4.5.17 VDVI ............................................ 29
5. Video ........................................................ 30
5.1 CelB .................................................... 30
5.2 JPEG .................................................... 30
5.3 H261 .................................................... 30
5.4 H263 .................................................... 31
5.5 H263-1998 ............................................... 31
5.6 MPV ..................................................... 31
5.7 MP2T .................................................... 31
5.8 nv ...................................................... 32
6. Payload Type Definitions ..................................... 32
7. RTP over TCP and Similar Byte Stream Protocols ............... 34
8. Port Assignment .............................................. 34
9. Changes from RFC 1890 ........................................ 35
10. Security Considerations ...................................... 38
11. IANA Considerations .......................................... 39
12. References ................................................... 39
12.1 Normative References .................................... 39
12.2 Informative References .................................. 39
13. Current Locations of Related Resources ....................... 41
14. Acknowledgments .............................................. 42
15. Intellectual Property Rights Statement ....................... 43
16. Authors' Addresses ........................................... 43
17. Full Copyright Statement ..................................... 441. Introduction 🌸
Okay so this profile fills in all the RTP stuff that the RTP Version 2 protocol spec left unspecified (that's RFC 3550 [1]). It's built for audio and video conferences with minimal session control, which means there's no support for parameter negotiation or membership control 🙈 It's giving "keep it simple," and it's expected to be super useful in sessions where neither negotiation nor membership control is used. Think static payload types plus the membership indications RTCP already gives you. It can also work alongside a higher-level control protocol if you want that extra layer. 💅
Using this profile might just be implicit in the apps you're running. There might be no explicit signal through a port number, a protocol identifier, or anything like that. Session directories can use the name for this profile that's specified in Section 11 🧚
Other profiles are allowed to make different choices for everything covered here. No judgment, just different vibes ✨
This doc also defines a set of encodings and payload formats for audio and video. They're included here purely as a matter of convenience, since they're too small to deserve their own documents 🍓 Using these payload formats is NOT REQUIRED to use this profile. The only binding that's normative is the one that ties some of the payload formats to the static payload type numbers in Tables 4 and 5. Everything else is just helpful context 🦋
1.1 Terminology 📖
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [2]. They're the requirement levels for implementations that are compliant with this RTP profile, so they're kind of a big deal 👑
This document defines the term media type as sorting encodings of audio and video content into three classes: audio, video, and audio/video (interleaved). Bestie, that's the whole family tree 🌈
2. RTP and RTCP Packet Forms and Protocol Behavior ✨
okay so bestie, this section is basically the checklist of all the stuff from RFC 3550 that a profile is allowed to tweak 🎀. The vibe is "we're doing the default stuff unless we say otherwise," and honestly that's so relatable 💅. Let's go through it! 🦋
RTP data header: the standard fixed RTP data header is used, with one marker bit. Easy peasy 🌸
Payload types: the static payload types are defined in Section 6. Go peek there when you're ready 👀
RTP data header additions: no extra fixed fields get tacked onto the RTP data header. Nothing extra, no cap 🧚
RTP data header extensions: no RTP header extensions are defined here, but apps running under this profile MAY still use them 🫶. So the rule is that apps SHOULD NOT assume the X bit is always zero, and they SHOULD be ready to ignore the header extension. Be flexible, bestie! 💖 If an extension gets defined later, its definition MUST spell out the first 16 bits so that different extensions can be told apart. Very important, very iconic 👑
RTCP packet types: this profile doesn't define any extra RTCP packet types 🍓
RTCP report interval: the suggested constants are used for the RTCP report interval calculation ✨. Sessions MAY set a separate parameter for RTCP traffic bandwidth instead of the default fraction of session bandwidth 💜. The RTCP traffic bandwidth MAY also be split into two session parameters, one for active data senders and one for everyone else 🌈. Following the RTP spec's suggestion that 1/4 of RTCP bandwidth goes to data senders, the RECOMMENDED defaults are 1.25% and 3.75%, respectively 🔥. For a particular session, the RTCP bandwidth for non-data-senders MAY be set to zero when you're on unidirectional links or in sessions that don't need reception quality feedback 🙈. The RTCP bandwidth for data senders SHOULD stay non-zero, so sender reports can still go out for inter-media synchronization and so sources can be identified by CNAME 🦋. How those one or two session parameters get specified is outside the scope of this memo, so that's a different story 💅
SR/RR extension: no extension section is defined for the RTCP SR or RR packet 🌸
SDES use: apps MAY use any of the SDES items from the RTP spec 🎀. Here's the tea though: CNAME MUST be sent every reporting interval, while the other items SHOULD only go out every third reporting interval 😭. Inside that slot, NAME is sent seven out of eight times, and the remaining SDES items take turns in the eighth slot, as defined in Section 6.2.2 of the RTP spec ✨. So NAME shows up in RTCP packets 1, 4, 7, 10, 13, 16, 19, while, say, EMAIL gets packet 22 💖
Security: the RTP default security services are also the defaults under this profile 🔒
String-to-key mapping: this profile doesn't specify any mapping 🧚
Congestion: RTP and this profile can be used with enhanced network service, like Integrated Services (RFC 1633) [4] or Differentiated Services (RFC 2475) [5] 🌈. Or they can be used with best-effort service, which is totally valid too 🌸
If enhanced service is in use, RTP receivers SHOULD monitor packet loss to make sure the service they asked for is actually being delivered 👀. If it isn't, they SHOULD assume they're getting best-effort service and act accordingly 🫶
If best-effort service is in use, RTP receivers SHOULD monitor packet loss to make sure the loss rate stays within acceptable parameters 💅. Loss counts as acceptable if a TCP flow on the same network path, under the same network conditions, would get an average throughput (measured on a reasonable timescale) that's not less than what the RTP flow is getting 🔥. You can meet this by adding congestion control to adapt the transmission rate (or the number of layers subscribed to in a layered multicast session) 💖, or by having the receiver leave the session if loss gets unacceptably high 😭
The TCP comparison can't be spelled out exactly, so it's meant as an "order-of-magnitude" comparison in timescale and throughput 🦋. The timescale for TCP throughput is the round-trip time of the connection ✨. Basically, this requirement says it's NOT okay to deploy an app (using RTP or any other transport protocol) on the best-effort Internet that hogs bandwidth whenever it wants and doesn't compete fairly with TCP within an order of magnitude 💀👑
Underlying protocol: the profile specifies RTP over unicast and multicast UDP, as well as TCP 🌈. (This doesn't stop these definitions from being used when RTP rides on other lower-layer protocols 🧚)
Transport mapping: the standard mapping of RTP and RTCP to transport-level addresses is used 🎀
Encapsulation: this profile leaves it up to applications to specify RTP encapsulation in protocols other than UDP 💜
3. Registering Additional Encodings 🎀
okay so this part is about how encodings get their names and numbers 🦋. An encoding is a particular media compression or representation plus a payload format for putting it inside RTP 🌸. Some payload formats live here, and others get their own RFCs 💅. More encodings are totally expected in the future, and they'll be specified in additional payload format RFCs ✨
This profile also gives each encoding a short name that MAY be used by higher-level control protocols, like the Session Description Protocol (SDP), RFC 2327 [6], to pick encodings for a particular RTP session 🫶
Sometimes it's helpful to refer to encodings as a MIME content-type 💖. To make that easy, RFC 3555 [7] registers all the encoding names listed here as MIME subtype names under the "audio" and "video" MIME types, using the MIME registration procedure in RFC 2048 [8] 🌈
Any extra encodings specified for this profile (or others) may also get names registered as MIME subtypes with the Internet Assigned Numbers Authority (IANA) 🔥. This registry keeps the names of additional encodings unique 😭. RFC 3555 spells out the info needed to register RTP encodings 👑
On top of names, this profile also assigns static RTP payload type numbers to some encodings 🍓. But the payload type number space is pretty small and can't fit every existing and future encoding 💀. Back in early RTP days, static payload types were needed because no other way to bind encodings to payload types had been specified yet 🙈. It was expected that non-RTP stuff outside this memo's scope (like directory services or invitation protocols) would set up a dynamic mapping between a payload type and an encoding 🧚. Now, dynamic payload type bindings are specified in SDP and in other protocols like ITU-T Recommendation H.323/H.245 🎀. These mechanisms link the registered name of the encoding/payload format, plus any extra required parameters (like the RTP timestamp clock rate and number of channels), to a payload type number 💜. That association only lasts for the duration of the RTP session where the dynamic binding is made 🦋. Since it only applies to that one session, the numbers can be reused for different encodings in different sessions, so the number space limit doesn't matter as much ✨
This profile reserves payload type numbers in the range 96-127 exclusively for dynamic assignment 💖. Apps SHOULD use values in this range first for dynamic payload types 🫶. Apps that need to define more than 32 dynamic payload types MAY bind codes below 96, and in that case it is RECOMMENDED that unassigned payload type numbers get used first 🌸. Still, the statically assigned payload types are default bindings and MAY be dynamically bound to new encodings if needed 🌈. Redefining payload types below 96 may cause incorrect operation if someone tries to join a session without getting the session description that defines the dynamic payload types 😭
Dynamic payload types SHOULD NOT be used without a well-defined mechanism to indicate the mapping 🙈. Systems that expect to interoperate with others under this profile SHOULD NOT make their own assignments of proprietary encodings to particular, fixed payload types 💅
This spec sets the policy that no additional static payload types will be assigned beyond the ones defined in this document 🔥. That avoids the mess of trying to make criteria for accepting static assignments, and it encourages implementing and deploying the dynamic payload type mechanisms 👑
The final set of static payload type assignments is provided in Tables 4 and 5 🦋
4. Audio 🌈
4.1. Encoding-Independent Rules ✨
okay so the whole reason we send voice in packets is to skip the silent bits 🤫 That's why the RTP header carries both a sequence number and a timestamp, so a receiver can tell "oh no, that packet got lost" apart from "nobody was talking, totally normal" 💅 Discontiguous transmission (silence suppression) MAY be used with any audio payload format. Receivers MUST assume that senders may suppress silence unless this is restricted by signaling specified elsewhere. (Even if the transmitter does not suppress silence, the receiver should be prepared to handle periods when no data is present since packets may be lost.) 🫶
Some payload formats (see Sections 4.5.3 and 4.5.6) define a "silence insertion descriptor" or "comfort noise" frame. These describe the parameters for fake background noise that can be generated during silence, so it sounds like the background noise at the source 🎧 For other payload formats, there's a generic Comfort Noise (CN) payload format specified in RFC 3389 [9]. When the CN payload format is used with another payload format, different values in the RTP payload type field tell comfort-noise packets apart from packets of the selected payload format 🌈
For apps that send either no packets or only the occasional comfort-noise packet during silence, the first packet of a talkspurt (that's the first packet after a silence period where packets weren't sent back-to-back) SHOULD be marked by setting the marker bit in the RTP data header to one 🦋 The marker bit in all other packets is zero. The start of a talkspurt MAY be used to adjust the playout delay so it keeps up with changing network delays ⏱️ Apps without silence suppression MUST set the marker bit to zero. No exceptions, bestie 💀
The RTP clock rate used to generate the RTP timestamp is independent of the number of channels and the encoding. It usually equals the number of sampling periods per second. For N-channel encodings, each sampling period (say, 1/8,000 of a second) generates N samples. (This terminology is standard but lowkey confusing, because the total number of samples generated per second is then the sampling rate times the channel count 🙈)
If multiple audio channels are used, they're numbered left-to-right, starting at one. In RTP audio packets, info from lower-numbered channels comes before info from higher-numbered channels 👑
For more than two channels, the convention followed by the AIFF-C audio interchange format SHOULD be followed [3], using the notation below, unless some other convention is specified for a particular encoding or payload format 🎀
l left
r right
c center
S surround
F front
R rear
channels description channel
1 2 3 4 5 6
_________________________________________________
2 stereo l r
3 l r c
4 l c r S
5 Fl Fr Fc Sl Sr
6 l lc c r rc SNote: RFC 1890 defined two conventions for ordering four audio channels, and since the ordering was only implied by the channel count, it was ambiguous 🤯 This revision removes the one called "quadrophonic" to clear that up. The reasoning is that quadrophonic consumer audio never really caught on, while surround sound totally did 🔥
Samples for all channels belonging to a single sampling instant MUST be within the same packet. How samples from different channels get interleaved depends on the encoding. General guidelines are in Section 4.3 and 4.4 ✨
The sampling frequency SHOULD be drawn from this set: 8,000, 11,025, 16,000, 22,050, 24,000, 32,000, 44,100 and 48,000 Hz 🎶 (Older Apple Macintosh computers had a native sample rate of 22,254.54 Hz, which can be converted to 22,050 with acceptable quality by dropping 4 samples in a 20 ms frame 🍎) Most audio encodings are only defined for a more restricted set of sampling frequencies, though 🥺 Receivers SHOULD be prepared to accept multi-channel audio, but MAY choose to only play a single channel 🔈
4.2. Operating Recommendations 🎀
These recommendations are default operating parameters. Apps SHOULD be prepared to handle other values. The ranges given are there to guide app writers, so that apps following these guidelines can interoperate without extra negotiation. These guidelines are not meant to restrict operating parameters for apps that can negotiate a set of interoperable parameters, e.g., through a conference control protocol 🤝
For packetized audio, the default packetization interval SHOULD have a duration of 20 ms or one frame, whichever is longer, unless otherwise noted in Table 1 (column "ms/packet") 💖 The packetization interval sets the minimum end-to-end delay. Longer packets mean less header overhead, but more delay and more noticeable packet loss 📦 For non-interactive stuff like lectures, or for links with severe bandwidth constraints, a higher packetization delay MAY be used. A receiver SHOULD accept packets representing between 0 and 200 ms of audio data. (For framed audio encodings, a receiver SHOULD accept packets with a number of frames equal to 200 ms divided by the frame duration, rounded up.) This restriction gives the receiver reasonable buffer sizing 🧠
4.3. Guidelines for Sample-Based Audio Encodings 🌸
In sample-based encodings, each audio sample is represented by a fixed number of bits. Inside the compressed audio data, codes for individual samples may span octet boundaries 🧩 An RTP audio packet can contain any number of audio samples, as long as the number of bits per sample times the number of samples per packet gives a whole octet count. Fractional encodings produce less than one octet per sample.
The duration of an audio packet is decided by how many samples are in it ⏳
For sample-based encodings that produce one or more octets per sample, samples from different channels taken at the same sampling instant SHOULD be packed into consecutive octets. For example, with a two-channel encoding, the octet sequence goes like this:
(left channel, first sample),
(right channel, first sample),
(left channel, second sample),
(right channel, second sample), ....For multi-octet encodings, octets SHOULD be sent in network byte order (that's most significant octet first) 🔥
The packing of sample-based encodings that produce less than one octet per sample is encoding-specific 🤷♀️
The RTP timestamp reflects the instant the first sample in the packet was sampled, which is the oldest info in the packet 🕰️
4.4. Guidelines for Frame-Based Audio Encodings 🧚
Frame-based encodings take a fixed-length block of audio and encode it into another block of compressed data, usually also fixed-length 🎞️ For frame-based encodings, the sender MAY choose to combine several of these frames into one RTP packet. The receiver can tell how many frames are in an RTP packet, if all the frames are the same length, by dividing the RTP payload length by the audio frame size (which is defined as part of the encoding) ✨ This doesn't work when frames of different sizes are carried, unless the frame sizes are relatively prime. If they aren't, the frames MUST indicate their size 📏
For frame-based codecs, the channel order is defined for the whole block. So for two-channel audio, right and left samples SHOULD be coded independently, with the encoded frame for the left channel coming before the one for the right channel 👈👉
All frame-oriented audio codecs SHOULD be able to encode and decode several consecutive frames within a single packet. Since the frame size is given for frame-oriented codecs, there's no need for a separate name for the same encoding with a different number of frames per packet 💅
RTP packets SHALL contain a whole number of frames, with frames inserted by age within a packet, so the oldest frame (the one to be played first) comes right after the RTP packet header 🥇 The RTP timestamp reflects the instant the first sample in the first frame was sampled, which is the oldest info in the packet 🕰️
4.5. Audio Encodings 🎵
Here's the big table of audio encodings, which I am obsessed with 😭
name of sampling default
encoding sample/frame bits/sample rate ms/frame ms/packet
__________________________________________________________________
DVI4 sample 4 var. 20
G722 sample 8 16,000 20
G723 frame N/A 8,000 30 30
G726-40 sample 5 8,000 20
G726-32 sample 4 8,000 20
G726-24 sample 3 8,000 20
G726-16 sample 2 8,000 20
G728 frame N/A 8,000 2.5 20
G729 frame N/A 8,000 10 20
G729D frame N/A 8,000 10 20
G729E frame N/A 8,000 10 20
GSM frame N/A 8,000 20 20
GSM-EFR frame N/A 8,000 20 20
L8 sample 8 var. 20
L16 sample 16 var. 20
LPC frame N/A 8,000 20 20
MPA frame N/A var. var.
PCMA sample 8 var. 20
PCMU sample 8 var. 20
QCELP frame N/A 8,000 20 20
VDVI sample var. var. 20
Table 1: Properties of Audio Encodings (N/A: not applicable; var.:
variable)The properties of the audio encodings in this document are in Table 1, and they're listed in order of their payload type in Table 4 📋 Most audio codecs are only specified for a fixed sampling rate, but some sample-based algorithms (marked with "var." in the sampling rate column of Table 1) can be used at different sampling rates, which gives different coded bit rates 🎚️ When you use one at a sampling rate other than the one a static payload type is defined for, non-RTP means beyond the scope of this memo MUST be used to define a dynamic payload type and MUST indicate the selected RTP timestamp clock rate. That's usually the same as the sampling rate for audio 🔥
4.5.1 DVI4 💖
okay bestie, DVI4 uses an adaptive delta pulse code modulation (ADPCM) encoding 🎀 that the Interactive Multimedia Association (IMA) specified as the "IMA ADPCM wave type." BUT the encoding defined here as DVI4 is lowkey its own thing, and it differs from the IMA spec in three ways 👀
- The RTP DVI4 header contains the predicted value, not the first sample value that the IMA ADPCM block header holds.
- IMA ADPCM blocks have an odd number of samples, since the first sample is just in the header (uncompressed), followed by an even number of compressed samples. DVI4 is even-only compressed samples 💅 and uses the `predict' word from the header to decode the first sample.
- For DVI4, the 4-bit samples are packed with the first sample in the four most significant bits and the second sample in the four least significant bits. IMA ADPCM packs them in the opposite order 🙃
Each packet contains a single DVI block. This profile only defines the 4-bit-per-sample version, while IMA also specified a 3-bit-per-sample encoding ✨
The "header" word for each channel has this structure:
int16 predict; /* predicted value of first sample
from the previous block (L16 format) */
u_int8 index; /* current index into stepsize table */
u_int8 reserved; /* set to zero by sender, ignored by receiver */Each octet after the header contains two 4-bit samples, so the number of samples per packet MUST be even, because there's no way to say "hey, this last octet is only half full" 🫠
Packing of samples for multiple channels is for further study. So that's a future-me problem 🧚
The IMA ADPCM algorithm is described in the document IMA Recommended Practices for Enhancing Digital Audio Compatibility in Multimedia Systems (version 3.0). The catch? The Interactive Multimedia Association ceased operations in 1997 😭 Resources for an archived copy of that document and a software implementation of the RTP DVI4 encoding are listed in Section 13 🌈
4.5.2 G722 🦋
G722 is specified in ITU-T Recommendation G.722, "7 kHz audio-coding within 64 kbit/s" 🎶 The G.722 encoder produces a stream of octets, and each of those octets SHALL be octet-aligned in an RTP packet. The first bit transmitted in the G.722 octet, which is the most significant bit of the higher sub-band sample, SHALL correspond to the most significant bit of the octet in the RTP packet. Literally so precise, I'm obsessed 😍
Even though the actual sampling rate for G.722 audio is 16,000 Hz, the RTP clock rate for the G722 payload format is 8,000 Hz. Why? Because that value was erroneously assigned in RFC 1890, and it must remain unchanged for backward compatibility 🙈 The octet rate or sample-pair rate is 8,000 Hz.
4.5.3 G723 🧚♀️
G723 is specified in ITU Recommendation G.723.1, "Dual-rate speech coder for multimedia communications transmitting at 5.3 and 6.3 kbit/s" 🎀 The G.723.1 5.3/6.3 kbit/s codec was defined by the ITU-T as a mandatory codec for ITU-T H.324 GSTN videophone terminal applications. The algorithm has a floating point specification in Annex B to G.723.1, a silence compression algorithm in Annex A to G.723.1, and a scalable channel coding scheme for wireless applications in G.723.1 Annex C. The full lineup, bestie 💜
This Recommendation specifies a coded representation for compressing the speech signal component of multi-media services at a very low bit rate. Audio is encoded in 30 ms frames, with an additional delay of 7.5 ms due to look-ahead. A G.723.1 frame can be one of three sizes: 24 octets (6.3 kb/s frame), 20 octets (5.3 kb/s frame), or 4 octets. Those 4-octet frames are called SID frames (Silence Insertion Descriptor) and are used to specify comfort noise parameters 🌸 There is no restriction on how 4, 20, and 24 octet frames are intermixed. The least significant two bits of the first octet in the frame determine the frame size and codec type:
bits content octets/frame
00 high-rate speech (6.3 kb/s) 24
01 low-rate speech (5.3 kb/s) 20
10 SID frame 4
11 reservedIt's possible to switch between the two rates at any 30 ms frame boundary. Both (5.3 kb/s and 6.3 kb/s) rates are a mandatory part of the encoder and decoder. Receivers MUST accept both data rates and MUST accept SID frames unless restriction of these capabilities has been signaled. The MIME registration for G723 in RFC 3555 [7] specifies parameters that MAY be used with MIME or SDP to restrict to a single data rate or to restrict the use of SID frames. This coder was optimized to represent speech with near-toll quality at the above rates using a limited amount of complexity 🔥
The packing of the encoded bit stream into octets and the transmission order of those octets is specified in Rec. G.723.1, and it's the same as what the G.723 C code reference implementation produces. For the 6.3 kb/s data rate, the packing looks like this, where the header (HDR) bits are always "0 0" as shown in Fig. 1 to indicate operation at 6.3 kb/s, and the Z bit is always set to zero. The diagrams show the bit packing in "network byte order," also known as big-endian order. The bits of each 32-bit word are numbered 0 to 31, with the most significant bit on the left and numbered 0. The octets (bytes) of each word are transmitted most significant octet first. The bits of each data field are numbered in the order of the bit stream representation of the encoding (least significant bit first). The vertical bars indicate the boundaries between field fragments 🧩
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| LPC |HDR| LPC | LPC | ACL0 |LPC|
| | | | | | |
|0 0 0 0 0 0|0 0|1 1 1 1 0 0 0 0|2 2 1 1 1 1 1 1|0 0 0 0 0 0|2 2|
|5 4 3 2 1 0| |3 2 1 0 9 8 7 6|1 0 9 8 7 6 5 4|5 4 3 2 1 0|3 2|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ACL2 |ACL|A| GAIN0 |ACL|ACL| GAIN0 | GAIN1 |
| | 1 |C| | 3 | 2 | | |
|0 0 0 0 0|0 0|0|0 0 0 0|0 0|0 0|1 1 0 0 0 0 0 0|0 0 0 0 0 0 0 0|
|4 3 2 1 0|1 0|6|3 2 1 0|1 0|6 5|1 0 9 8 7 6 5 4|7 6 5 4 3 2 1 0|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| GAIN2 | GAIN1 | GAIN2 | GAIN3 | GRID | GAIN3 |
| | | | | | |
|0 0 0 0|1 1 0 0|1 1 0 0 0 0 0 0|0 0 0 0 0 0 0 0|0 0 0 0|1 1 0 0|
|3 2 1 0|1 0 9 8|1 0 9 8 7 6 5 4|7 6 5 4 3 2 1 0|3 2 1 0|1 0 9 8|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MSBPOS |Z|POS| MSBPOS | POS0 |POS| POS0 |
| | | 0 | | | 1 | |
|0 0 0 0 0 0 0|0|0 0|1 1 1 0 0 0|0 0 0 0 0 0 0 0|0 0|1 1 1 1 1 1|
|6 5 4 3 2 1 0| |1 0|2 1 0 9 8 7|9 8 7 6 5 4 3 2|1 0|5 4 3 2 1 0|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| POS1 | POS2 | POS1 | POS2 | POS3 | POS2 |
| | | | | | |
|0 0 0 0 0 0 0 0|0 0 0 0|1 1 1 1|1 1 0 0 0 0 0 0|0 0 0 0|1 1 1 1|
|9 8 7 6 5 4 3 2|3 2 1 0|3 2 1 0|1 0 9 8 7 6 5 4|3 2 1 0|5 4 3 2|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| POS3 | PSIG0 |POS|PSIG2| PSIG1 | PSIG3 |PSIG2|
| | | 3 | | | | |
|1 1 0 0 0 0 0 0|0 0 0 0 0 0|1 1|0 0 0|0 0 0 0 0|0 0 0 0 0|0 0 0|
|1 0 9 8 7 6 5 4|5 4 3 2 1 0|3 2|2 1 0|4 3 2 1 0|4 3 2 1 0|5 4 3|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 1: G.723 (6.3 kb/s) bit packingFor the 5.3 kb/s data rate, the header (HDR) bits are always "0 1," as shown in Fig. 2, to indicate operation at 5.3 kb/s 🌷
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| LPC |HDR| LPC | LPC | ACL0 |LPC|
| | | | | | |
|0 0 0 0 0 0|0 1|1 1 1 1 0 0 0 0|2 2 1 1 1 1 1 1|0 0 0 0 0 0|2 2|
|5 4 3 2 1 0| |3 2 1 0 9 8 7 6|1 0 9 8 7 6 5 4|5 4 3 2 1 0|3 2|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ACL2 |ACL|A| GAIN0 |ACL|ACL| GAIN0 | GAIN1 |
| | 1 |C| | 3 | 2 | | |
|0 0 0 0 0|0 0|0|0 0 0 0|0 0|0 0|1 1 0 0 0 0 0 0|0 0 0 0 0 0 0 0|
|4 3 2 1 0|1 0|6|3 2 1 0|1 0|6 5|1 0 9 8 7 6 5 4|7 6 5 4 3 2 1 0|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| GAIN2 | GAIN1 | GAIN2 | GAIN3 | GRID | GAIN3 |
| | | | | | |
|0 0 0 0|1 1 0 0|1 1 0 0 0 0 0 0|0 0 0 0 0 0 0 0|0 0 0 0|1 1 0 0|
|3 2 1 0|1 0 9 8|1 0 9 8 7 6 5 4|7 6 5 4 3 2 1 0|4 3 2 1|1 0 9 8|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| POS0 | POS1 | POS0 | POS1 | POS2 |
| | | | | |
|0 0 0 0 0 0 0 0|0 0 0 0|1 1 0 0|1 1 0 0 0 0 0 0|0 0 0 0 0 0 0 0|
|7 6 5 4 3 2 1 0|3 2 1 0|1 0 9 8|1 0 9 8 7 6 5 4|7 6 5 4 3 2 1 0|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| POS3 | POS2 | POS3 | PSIG1 | PSIG0 | PSIG3 | PSIG2 |
| | | | | | | |
|0 0 0 0|1 1 0 0|1 1 0 0 0 0 0 0|0 0 0 0|0 0 0 0|0 0 0 0|0 0 0 0|
|3 2 1 0|1 0 9 8|1 0 9 8 7 6 5 4|3 2 1 0|3 2 1 0|3 2 1 0|3 2 1 0|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 2: G.723 (5.3 kb/s) bit packingThe packing of G.723.1 SID (silence) frames, which are indicated by the header (HDR) bits having the pattern "1 0," is shown in Fig. 3 🤫
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| LPC |HDR| LPC | LPC | GAIN |LPC|
| | | | | | |
|0 0 0 0 0 0|1 0|1 1 1 1 0 0 0 0|2 2 1 1 1 1 1 1|0 0 0 0 0 0|2 2|
|5 4 3 2 1 0| |3 2 1 0 9 8 7 6|1 0 9 8 7 6 5 4|5 4 3 2 1 0|3 2|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 3: G.723 SID mode bit packing4.5.4. G726-40, G726-32, G726-24, and G726-16 🎀
Okay bestie, so ITU-T Recommendation G.726 describes, among others, the algorithm recommended for converting a single 64 kbit/s A-law or mu-law PCM channel encoded at 8,000 samples/sec to and from a 40, 32, 24, or 16 kbit/s channel. ✨ The conversion is applied to the PCM stream using an Adaptive Differential Pulse Code Modulation (ADPCM) transcoding technique. The ADPCM representation is a series of codewords with a one-to-one correspondence to the samples in the PCM stream. The G726 data rates of 40, 32, 24, and 16 kbit/s have codewords of 5, 4, 3, and 2 bits, respectively. 💖
The 16 and 24 kbit/s encodings do not provide toll quality speech 🙈 They're designed for use in overloaded Digital Circuit Multiplication Equipment (DCME). ITU-T G.726 recommends that the 16 and 24 kbit/s encodings should be alternated with higher data rate encodings to provide an average sample size of between 3.5 and 3.7 bits per sample. Mixing them in like that is so smart 🧚
The encodings of G.726 are here denoted as G726-40, G726-32, G726-24, and G726-16. Before 1990, G721 described the 32 kbit/s ADPCM encoding, and G723 described the 40, 32, and 16 kbit/s encodings. So G726-32 is the same algorithm as G721 in RFC 1890. Main character energy, honestly 👑
A stream of G726 codewords has no info about which encoding is being used. That's why transitions between G726 encoding types are not permitted within a sequence of packed codewords. ⚠️ Applications MUST determine the encoding type of packed codewords from the RTP payload identifier. No cap, that's the only way to know what you're looking at 🫶
No payload-specific header information SHALL be included as part of the audio data. A stream of G726 codewords MUST be packed into octets as follows: the first codeword goes into the first octet so that its least significant bit lines up with the least significant bit in the octet. The second codeword is then packed so that its least significant bit sits in the least significant unoccupied bit in the octet. When a complete codeword can't fit into an octet, the bits that overlap the octet boundary go into the least significant bits of the next octet. Packing MUST end with a completely packed final octet. 💅 So the number of codewords packed will always be a multiple of 8, 2, 8, and 4 for G726-40, G726-32, G726-24, and G726-16, respectively.
Here's an example of the packing scheme for G726-32 codewords, where bit 7 is the least significant bit of the first octet, and bit A3 is the least significant bit of the first codeword:
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
|B B B B|A A A A|D D D D|C C C C| ...
|0 1 2 3|0 1 2 3|0 1 2 3|0 1 2 3|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-And here's the G726-24 version 🦋 Again, bit 7 is the least significant bit of the first octet, and bit A2 is the least significant bit of the first codeword:
0 1 2
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-
|C C|B B B|A A A|F|E E E|D D D|C|H H H|G G G|F F| ...
|1 2|0 1 2|0 1 2|2|0 1 2|0 1 2|0|0 1 2|0 1 2|0 1|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-Note that the "little-endian" direction, in which samples are packed into octets in the G726-16, -24, -32 and -40 payload formats specified here, matches ITU-T Recommendation X.420. It's the opposite of what ITU-T Recommendation I.366.2 Annex E does for ATM AAL2 transport 😭 A second set of RTP payload formats that matches the I.366.2 Annex E packetization, identified by the MIME subtypes AAL2-G726-16, -24, -32 and -40, will be specified in a separate document. Two different vibes, so keep your wires straight 🌈
4.5.5. G728 💜
G728 is specified in ITU-T Recommendation G.728, "Coding of speech at 16 kbit/s using low-delay code excited linear prediction". ✨
A G.278 encoder translates 5 consecutive audio samples into a 10-bit codebook index, which gives a bit rate of 16 kb/s for audio sampled at 8,000 samples per second. That group of five consecutive samples is called a vector. Four consecutive vectors, labeled V1 to V4 (where V1 is to be played first by the receiver), build one G.728 frame. The four vectors of 40 bits are packed into 5 octets, labeled B1 through B5. B1 SHALL be placed first in the RTP packet. 🔥
Look at the figure below, the principle for bit order is "maintenance of bit significance". Bits from an older vector are more significant than bits from newer vectors. The MSB of the frame goes to the MSB of B1, and the LSB of the frame goes to the LSB of B5. Okay, that's a whole vibe 🍓
1 2 3 3
0 0 0 0 9
++++++++++++++++++++++++++++++++++++++++
<---V1---><---V2---><---V3---><---V4---> vectors
<--B1--><--B2--><--B3--><--B4--><--B5--> octets
<------------- frame 1 ---------------->Specifically, B1 contains the eight most significant bits of V1, with the MSB of V1 being the MSB of B1. B2 contains the two least significant bits of V1, with the more significant of those two in its MSB, plus the six most significant bits of V2. B1 SHALL be placed first in the RTP packet and B5 last. Order matters, bestie, so don't skip the line 💖
4.5.6. G729 🌸
G729 is specified in ITU-T Recommendation G.729, "Coding of speech at 8 kbit/s using conjugate structure-algebraic code excited linear prediction (CS-ACELP)". A reduced-complexity version of the G.729 algorithm is specified in Annex A to Rec. G.729. The speech coding algorithms in the main body of G.729 and in G.729 Annex A are fully interoperable with each other, so there's no need to tell them apart. An implementation that signals or accepts use of the G729 payload format may implement either G.729 or G.729A, unless additional signaling specified elsewhere restricts it in a way that's about the encoding itself rather than the payload format. 🧚 The G.729 and G.729 Annex A codecs were optimized to represent speech with high quality, though G.729 Annex A trades some speech quality for an approximate 50% complexity reduction [10]. See the next Section (4.5.7) for other data rates added in later G.729 Annexes. For all data rates, the sampling frequency (and RTP timestamp clock rate) is 8,000 Hz. ⭐
A voice activity detector (VAD) and comfort noise generator (CNG) algorithm in Annex B of G.729 is RECOMMENDED for digital simultaneous voice and data applications, and it can be used together with G.729 or G.729 Annex A. A G.729 or G.729 Annex A frame contains 10 octets, while the G.729 Annex B comfort noise frame takes up 2 octets. Receivers MUST accept comfort noise frames if restriction of their use has not been signaled. The MIME registration for G729 in RFC 3555 [7] specifies a parameter that MAY be used with MIME or SDP to restrict the use of comfort noise frames. 🙈
A G729 RTP packet may consist of zero or more G.729 or G.729 Annex A frames, followed by zero or one G.729 Annex B frames. The presence of a comfort noise frame can be deduced from the length of the RTP payload. The default packetization interval is 20 ms (two frames), but in some situations it may be desirable to send 10 ms packets. An example would be a transition from speech to comfort noise in the first 10 ms of the packet. For some applications, a longer packetization interval may be required to reduce the packet rate. 💅
Here's the bit packing for G.729 and G.729A:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|L| L1 | L2 | L3 | P1 |P| C1 |
|0| | | | |0| |
| |0 1 2 3 4 5 6|0 1 2 3 4|0 1 2 3 4|0 1 2 3 4 5 6 7| |0 1 2 3 4|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| C1 | S1 | GA1 | GB1 | P2 | C2 |
| 1 1 1| | | | | |
|5 6 7 8 9 0 1 2|0 1 2 3|0 1 2|0 1 2 3|0 1 2 3 4|0 1 2 3 4 5 6 7|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| C2 | S2 | GA2 | GB2 |
| 1 1 1| | | |
|8 9 0 1 2|0 1 2 3|0 1 2|0 1 2 3|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 4: G.729 and G.729A bit packing 🦋
The transmitted parameters of a G.729/G.729A 10-ms frame, which consist of 80 bits, are defined in Recommendation G.729, Table 8/G.729. The mapping of these parameters is given below in Fig. 4. The diagrams show the bit packing in "network byte order", also known as big-endian order. The bits of each 32-bit word are numbered 0 to 31, with the most significant bit on the left and numbered 0. The octets (bytes) of each word are transmitted most significant octet first. The bits of each data field are numbered in the order produced by the G.729 C code reference implementation. 🎀
The packing of the G.729 Annex B comfort noise frame is shown in Fig. 5:
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|L| LSF1 | LSF2 | GAIN |R|
|S| | | |E|
|F| | | |S|
|0|0 1 2 3 4|0 1 2 3|0 1 2 3 4|V| RESV = Reserved (zero)
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 5: G.729 Annex B bit packing 🌈
4.5.7 G729D and G729E 🦋
okay so Annexes D and E of ITU-T Recommendation G.729 give you extra data rates, and they're giving main character energy 💅. The rate isn't signaled in the bitstream, so each one gets its own RTP encoding name, and those names map to separate payload type numbers. G729D is the 6.4 kbit/s coding mode (G.729 Annex D, for momentary reduction in channel capacity). G729E is the 11.8 kbit/s mode (G.729 Annex E, for improved performance with a wide range of narrow-band input signals, e.g., music and background noise) 🎶. Annex E has two operating modes, backward adaptive and forward adaptive, and you can tell which one you've got from the first two bits in each frame (the most significant two bits of the first octet) 🔥.
The VAD and CNG combo from Annex B of G.729 (voice activity detector plus comfort noise generator) can be used with Annex D and Annex E frames, not just with G.729 and G.729 Annex A frames. The details for running Annexes D and E with the Annex B CNG live in G.729 Annexes F and G. ngl, Annexes F and G don't add any new encodings 🙈. Receivers MUST accept comfort noise frames if restriction of their use has not been signaled. The MIME registrations for G729D and G729E in RFC 3555 [7] include a parameter that MAY be used with MIME or SDP to restrict the use of comfort noise frames 🫶.
For G729D, an RTP packet can hold zero or more G.729 Annex D frames, followed by zero or one G.729 Annex B frame. Same deal for G729E, except it's Annex E frames instead ✨. You can figure out whether a comfort noise frame is in there just by checking the length of the RTP payload 🧚.
A single RTP packet has to contain frames of only one data rate, optionally followed by one comfort noise frame. The data rate can switch from packet to packet by changing the payload type number 🌈. G.729 Annexes D, E and H spell out what the encoding and decoding algorithms must do to handle a change in data rate, so go read those if you're building this 💜.
For G729D, the bits of a G.729 Annex D frame are packed like this in Fig. 6 (cf. Table D.1/G.729). The frame length is 64 bits 🍓.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|L| L1 | L2 | L3 | P1 | C1 |
|0| | | | | |
| |0 1 2 3 4 5 6|0 1 2 3 4|0 1 2 3 4|0 1 2 3 4 5 6 7|0 1 2 3 4 5|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| C1 |S1 | GA1 | GB1 | P2 | C2 |S2 | GA2 | GB2 |
| | | | | | | | | |
|6 7 8|0 1|0 1 2|0 1 2|0 1 2 3|0 1 2 3 4 5 6 7 8|0 1|0 1 2|0 1 2|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 6: G.729 Annex D bit packingThe net bit rate for the G.729 Annex E algorithm is 11.8 kbit/s, and a total of 118 bits get used. Two bits are tacked on as "don't care" bits so the frame ends on a whole number of octets 🎀. For G729E, the bits of a data frame are packed as shown in the next two diagrams (cf. Table E.1/G.729). The fields for the G729E forward adaptive mode are packed as shown in Fig. 7 💖.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0 0|L| L1 | L2 | L3 | P1 |P| C0_1|
| |0| | | | |0| |
| | |0 1 2 3 4 5 6|0 1 2 3 4|0 1 2 3 4|0 1 2 3 4 5 6 7| |0 1 2|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | C1_1 | C2_1 | C3_1 | C4_1 |
| | | | | |
|3 4 5 6|0 1 2 3 4 5 6|0 1 2 3 4 5 6|0 1 2 3 4 5 6|0 1 2 3 4 5 6|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| GA1 | GB1 | P2 | C0_2 | C1_2 | C2_2 |
| | | | | | |
|0 1 2|0 1 2 3|0 1 2 3 4|0 1 2 3 4 5 6|0 1 2 3 4 5 6|0 1 2 3 4 5|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | C3_2 | C4_2 | GA2 | GB2 |DC |
| | | | | | |
|6|0 1 2 3 4 5 6|0 1 2 3 4 5 6|0 1 2|0 1 2 3|0 1|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 7: G.729 Annex E (forward adaptive mode) bit packingThe fields for the G729E backward adaptive mode are packed as shown in Fig. 8 🌸.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1 1| P1 |P| C0_1 | C1_1 |
| | |0| 1 1 1| |
| |0 1 2 3 4 5 6 7|0|0 1 2 3 4 5 6 7 8 9 0 1 2|0 1 2 3 4 5 6 7|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | C2_1 | C3_1 | C4_1 |GA1 | GB1 |P2 |
| | | | | | | |
|8 9|0 1 2 3 4 5 6|0 1 2 3 4 5 6|0 1 2 3 4 5 6|0 1 2|0 1 2 3|0 1|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | C0_2 | C1_2 | C2_2 |
| | 1 1 1| | |
|2 3 4|0 1 2 3 4 5 6 7 8 9 0 1 2|0 1 2 3 4 5 6 7 8 9|0 1 2 3 4 5|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | C3_2 | C4_2 | GA2 | GB2 |DC |
| | | | | | |
|6|0 1 2 3 4 5 6|0 1 2 3 4 5 6|0 1 2|0 1 2 3|0 1|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 8: G.729 Annex E (backward adaptive mode) bit packing4.5.8 GSM 🌈
GSM (Group Speciale Mobile) is the European GSM 06.10 standard for full-rate speech transcoding, ETS 300 961 💜. It's based on RPE/LTP (residual pulse excitation/long term prediction) coding at a rate of 13 kb/s [11,12,13] ✨. If you want the actual standard text, here's where to get it 📚:
ETSI (European Telecommunications Standards Institute)
ETSI Secretariat: B.P.152
F-06561 Valbonne Cedex
France
Phone: +33 92 94 42 00
Fax: +33 93 65 47 16Blocks of 160 audio samples get compressed into 33 octets, for an effective data rate of 13,200 b/s 🔥.
4.5.8.1 General Packaging Issues ✨
okay so the GSM standard (ETS 300 961) is super specific about the bit stream the codec spits out 💖 but it's lowkey silent on how those bits should get packed up for transmission 📦 the packetization we're describing here was later adopted into ETSI Technical Specification TS 101 318, so it's officially a thing now 🎀 fyi, some software versions of the GSM codec pack things differently than this, so keep that in mind 🙈
Here's the ordering of the GSM variables, field by field. Table 2 is the big list, so you can see exactly which bits come in which order 💅
field field name bits field field name bits
________________________________________________
1 LARc[0] 6 39 xmc[22] 3
2 LARc[1] 6 40 xmc[23] 3
3 LARc[2] 5 41 xmc[24] 3
4 LARc[3] 5 42 xmc[25] 3
5 LARc[4] 4 43 Nc[2] 7
6 LARc[5] 4 44 bc[2] 2
7 LARc[6] 3 45 Mc[2] 2
8 LARc[7] 3 46 xmaxc[2] 6
9 Nc[0] 7 47 xmc[26] 3
10 bc[0] 2 48 xmc[27] 3
11 Mc[0] 2 49 xmc[28] 3
12 xmaxc[0] 6 50 xmc[29] 3
13 xmc[0] 3 51 xmc[30] 3
14 xmc[1] 3 52 xmc[31] 3
15 xmc[2] 3 53 xmc[32] 3
16 xmc[3] 3 54 xmc[33] 3
17 xmc[4] 3 55 xmc[34] 3
18 xmc[5] 3 56 xmc[35] 3
19 xmc[6] 3 57 xmc[36] 3
20 xmc[7] 3 58 xmc[37] 3
21 xmc[8] 3 59 xmc[38] 3
22 xmc[9] 3 60 Nc[3] 7
23 xmc[10] 3 61 bc[3] 2
24 xmc[11] 3 62 Mc[3] 2
25 xmc[12] 3 63 xmaxc[3] 6
26 Nc[1] 7 64 xmc[39] 3
27 bc[1] 2 65 xmc[40] 3
28 Mc[1] 2 66 xmc[41] 3
29 xmaxc[1] 6 67 xmc[42] 3
30 xmc[13] 3 68 xmc[43] 3
31 xmc[14] 3 69 xmc[44] 3
32 xmc[15] 3 70 xmc[45] 3
33 xmc[16] 3 71 xmc[46] 3
34 xmc[17] 3 72 xmc[47] 3
35 xmc[18] 3 73 xmc[48] 3
36 xmc[19] 3 74 xmc[49] 3
37 xmc[20] 3 75 xmc[50] 3
38 xmc[21] 3 76 xmc[51] 3
Table 2: Ordering of GSM variablesNext up is Table 3, the payload format, which maps out every bit of the octets 🧚
Octet Bit 0 Bit 1 Bit 2 Bit 3 Bit 4 Bit 5 Bit 6 Bit 7
_____________________________________________________________________
0 1 1 0 1 LARc0.0 LARc0.1 LARc0.2 LARc0.3
1 LARc0.4 LARc0.5 LARc1.0 LARc1.1 LARc1.2 LARc1.3 LARc1.4 LARc1.5
2 LARc2.0 LARc2.1 LARc2.2 LARc2.3 LARc2.4 LARc3.0 LARc3.1 LARc3.2
3 LARc3.3 LARc3.4 LARc4.0 LARc4.1 LARc4.2 LARc4.3 LARc5.0 LARc5.1
4 LARc5.2 LARc5.3 LARc6.0 LARc6.1 LARc6.2 LARc7.0 LARc7.1 LARc7.2
5 Nc0.0 Nc0.1 Nc0.2 Nc0.3 Nc0.4 Nc0.5 Nc0.6 bc0.0
6 bc0.1 Mc0.0 Mc0.1 xmaxc00 xmaxc01 xmaxc02 xmaxc03 xmaxc04
7 xmaxc05 xmc0.0 xmc0.1 xmc0.2 xmc1.0 xmc1.1 xmc1.2 xmc2.0
8 xmc2.1 xmc2.2 xmc3.0 xmc3.1 xmc3.2 xmc4.0 xmc4.1 xmc4.2
9 xmc5.0 xmc5.1 xmc5.2 xmc6.0 xmc6.1 xmc6.2 xmc7.0 xmc7.1
10 xmc7.2 xmc8.0 xmc8.1 xmc8.2 xmc9.0 xmc9.1 xmc9.2 xmc10.0
11 xmc10.1 xmc10.2 xmc11.0 xmc11.1 xmc11.2 xmc12.0 xmc12.1 xcm12.2
12 Nc1.0 Nc1.1 Nc1.2 Nc1.3 Nc1.4 Nc1.5 Nc1.6 bc1.0
13 bc1.1 Mc1.0 Mc1.1 xmaxc10 xmaxc11 xmaxc12 xmaxc13 xmaxc14
14 xmax15 xmc13.0 xmc13.1 xmc13.2 xmc14.0 xmc14.1 xmc14.2 xmc15.0
15 xmc15.1 xmc15.2 xmc16.0 xmc16.1 xmc16.2 xmc17.0 xmc17.1 xmc17.2
16 xmc18.0 xmc18.1 xmc18.2 xmc19.0 xmc19.1 xmc19.2 xmc20.0 xmc20.1
17 xmc20.2 xmc21.0 xmc21.1 xmc21.2 xmc22.0 xmc22.1 xmc22.2 xmc23.0
18 xmc23.1 xmc23.2 xmc24.0 xmc24.1 xmc24.2 xmc25.0 xmc25.1 xmc25.2
19 Nc2.0 Nc2.1 Nc2.2 Nc2.3 Nc2.4 Nc2.5 Nc2.6 bc2.0
20 bc2.1 Mc2.0 Mc2.1 xmaxc20 xmaxc21 xmaxc22 xmaxc23 xmaxc24
21 xmaxc25 xmc26.0 xmc26.1 xmc26.2 xmc27.0 xmc27.1 xmc27.2 xmc28.0
22 xmc28.1 xmc28.2 xmc29.0 xmc29.1 xmc29.2 xmc30.0 xmc30.1 xmc30.2
23 xmc31.0 xmc31.1 xmc31.2 xmc32.0 xmc32.1 xmc32.2 xmc33.0 xmc33.1
24 xmc33.2 xmc34.0 xmc34.1 xmc34.2 xmc35.0 xmc35.1 xmc35.2 xmc36.0
25 Xmc36.1 xmc36.2 xmc37.0 xmc37.1 xmc37.2 xmc38.0 xmc38.1 xmc38.2
26 Nc3.0 Nc3.1 Nc3.2 Nc3.3 Nc3.4 Nc3.5 Nc3.6 bc3.0
27 bc3.1 Mc3.0 Mc3.1 xmaxc30 xmaxc31 xmaxc32 xmaxc33 xmaxc34
28 xmaxc35 xmc39.0 xmc39.1 xmc39.2 xmc40.0 xmc40.1 xmc40.2 xmc41.0
29 xmc41.1 xmc41.2 xmc42.0 xmc42.1 xmc42.2 xmc43.0 xmc43.1 xmc43.2
30 xmc44.0 xmc44.1 xmc44.2 xmc45.0 xmc45.1 xmc45.2 xmc46.0 xmc46.1
31 xmc46.2 xmc47.0 xmc47.1 xmc47.2 xmc48.0 xmc48.1 xmc48.2 xmc49.0
32 xmc49.1 xmc49.2 xmc50.0 xmc50.1 xmc50.2 xmc51.0 xmc51.1 xmc51.2
Table 3: GSM payload formatIn the GSM packing used by RTP, the bits SHALL be packed starting from the most significant bit 🔥 every 160 sample GSM frame gets coded into one 33 octet (264 bit) buffer 🌸 each buffer kicks off with a 4 bit signature (0xD), followed by the MSB encoding of the frame's fields 🫶 so the first octet is 1101 in its 4 most significant bits (0-3), with the 4 most significant bits of F1 (0-3) tucked into the 4 least significant bits (4-7) 😭 the second octet holds the 2 least significant bits of F1 in bits 0-1, and F2 in bits 2-7, and so on 🍓 the order of the fields in the frame is in Table 2 💅
4.5.8.2 GSM Variable Names and Numbers 🦋
in the RTP encoding we've got the bit pattern in Table 3, where F.i means the ith bit of the field F. bit 0 is the most significant bit, and the bits of every octet get numbered 0 to 7 from most to least significant 💜
4.5.9 GSM-EFR 👑
GSM-EFR is GSM 06.60 enhanced full rate speech transcoding, specified in ETS 300 726, which you can grab from ETSI at the address given in Section 4.5.8 🎀 this codec has a frame length of 244 bits. for transmission in RTP, each codec frame gets packed into a 31 octet (248 bit) buffer that starts with a 4-bit signature 0xC, kinda like the original GSM 06.10 codec ✨ the packing itself is specified in ETSI Technical Specification TS 101 318 💖
4.5.10 L8 🌈
L8 means linear audio data samples using 8 bits of precision with an offset of 128, so the most negative signal is encoded as zero 🧚
4.5.11 L16 ⭐
L16 is uncompressed audio data samples using 16-bit signed representation, with 65,535 equally divided steps between minimum and maximum signal level, ranging from -32,768 to 32,767 🔥 the value is in two's complement notation and gets sent in network byte order (most significant byte first) 💅
The MIME registration for L16 in RFC 3555 [7] specifies parameters that MAY be used with MIME or SDP to say that analog pre-emphasis was applied to the signal before quantization, or to say that a multiple-channel audio stream follows a different channel ordering convention than the one specified in Section 4.1 🦋
4.5.12 LPC 🍓
LPC is an experimental linear predictive encoding contributed by Ron Frederick 🌸 it's based on an implementation written by Ron Zuckerman and posted to the Usenet group comp.dsp on June 26, 1992 💜 the codec makes 14 octets for every frame, the framesize is set to 20 ms, and that works out to a bit rate of 5,600 b/s ✨
4.5.13 MPA 🎀
MPA is MPEG-1 or MPEG-2 audio encapsulated as elementary streams 🧚 the encoding is defined in ISO standards ISO/IEC 11172-3 and 13818-3, and the encapsulation is specified in RFC 2250 [14] 💖
The encoding can be at any of three levels of complexity, called Layer I, II and III. the selected layer, along with the sampling rate and channel count, is indicated in the payload. the RTP timestamp clock rate is always 90,000, independent of the sampling rate 🫶 MPEG-1 audio supports sampling rates of 32, 44.1, and 48 kHz (ISO/IEC 11172-3, section 1.1; "Scope"), and MPEG-2 supports 16, 22.05 and 24 kHz 🌈 the number of samples per frame is fixed, but the frame size varies with the sampling rate and bit rate 💅
The MIME registration for MPA in RFC 3555 [7] specifies parameters that MAY be used with MIME or SDP to restrict the selection of layer, channel count, sampling rate, and bit rate 🦋
4.5.14 PCMA and PCMU 💅
PCMA and PCMU are specified in ITU-T Recommendation G.711 🌸 audio data is encoded as eight bits per sample, after logarithmic scaling. PCMU is mu-law scaling and PCMA is A-law scaling. a detailed description is given by Jayant and Noll [15] 💜
Each G.711 octet SHALL be octet-aligned in an RTP packet. The sign bit of each G.711 octet SHALL correspond to the most significant bit of the octet in the RTP packet (i.e., assuming the G.711 samples are handled as octets on the host machine, the sign bit SHALL be the most significant bit of the octet as defined by the host machine format) 🔥 the 56 kb/s and 48 kb/s modes of G.711 are not applicable to RTP, since PCMA and PCMU MUST always be transmitted as 8-bit samples 😭
See Section 4.1 regarding silence suppression 🙈
4.5.15 QCELP 🧚
The Electronic Industries Association (EIA) & Telecommunications Industry Association (TIA) standard IS-733, "TR45: High Rate Speech Service Option for Wideband Spread Spectrum Communications Systems", defines the QCELP audio compression algorithm for wireless CDMA applications 📡✨ the QCELP CODEC compresses each 20 milliseconds of 8,000 Hz, 16-bit sampled input speech into one of four different size output frames: Rate 1 (266 bits), Rate 1/2 (124 bits), Rate 1/4 (54 bits) or Rate 1/8 (20 bits) 🎀 for typical speech patterns, that works out to an average output of 6.8 kb/s for normal mode and 4.7 kb/s for reduced rate mode 💖 the packetization of the QCELP audio codec is described in [16] 🫶
4.5.16 RED 🌈
The redundant audio payload format "RED" is specified by RFC 2198 [17] 👑 it defines a way to send multiple redundant copies of an audio packet in a single RTP stream 🌸 each packet in that stream contains, in addition to the audio data for that packetization interval, a (more heavily compressed) copy of the data from a previous packetization interval 🦋 this lets you approximate the data from lost packets and recover it when a later packet gets decoded, giving much improved sound quality compared with silence substitution for lost packets 😭⭐
4.5.17 VDVI 💅
Okay, VDVI is the variable-rate glow-up of DVI4 🦋, giving speech bit rates of between 10 and 25 kb/s ✨. It's only specified for single-channel operation, so no stereo drama here 🙈. Samples get packed into octets starting at the most-significant bit. If the last sample doesn't fill the last octet, that octet gets padded with 1 bits 🫶. The padding is distinct from the valid codewords, and the receiver needs to detect it, because there's no explicit count of samples in the packet. Iconic problem, honestly 💀.
It uses the following encoding:
DVI4 codeword VDVI bit pattern
_______________________________
0 00
1 010
2 1100
3 11100
4 111100
5 1111100
6 11111100
7 11111110
8 10
9 011
10 1101
11 11101
12 111101
13 1111101
14 11111101
15 111111115. Video 🎀
Okay, video time 📺🌈 The following sections describe the video encodings that are defined in this memo and give their abbreviated names used for identification. These video encodings and their payload types are listed in Table 5.
All of these video encodings use an RTP timestamp frequency of 90,000 Hz, the same as the MPEG presentation time stamp frequency 🕰️ This frequency yields exact integer timestamp increments for the typical 24 (HDTV), 25 (PAL), and 29.97 (NTSC) and 30 Hz (HDTV) frame rates and 50, 59.94 and 60 Hz field rates 😭 While 90 kHz is the RECOMMENDED rate for future video encodings used within this profile, other rates MAY be used. However, it is not sufficient to use the video frame rate (typically between 15 and 30 Hz) because that does not provide adequate resolution for typical synchronization requirements when calculating the RTP timestamp corresponding to the NTP timestamp in an RTCP SR packet. The timestamp resolution MUST also be sufficient for the jitter estimate contained in the receiver reports.
For most of these video encodings, the RTP timestamp encodes the sampling instant of the video image contained in the RTP data packet. If a video image occupies more than one packet, the timestamp is the same on all of those packets. Packets from different video images are distinguished by their different timestamps 🧚
Most of these video encodings also specify that the marker bit of the RTP header SHOULD be set to one in the last packet of a video frame and otherwise set to zero. Thus, it is not necessary to wait for a following packet with a different timestamp to detect that a new frame should be displayed 👑
5.1 CelB
The CELL-B encoding is a proprietary encoding proposed by Sun Microsystems. The byte stream format is described in RFC 2029 [18] 🍓
5.2 JPEG
The encoding is specified in ISO Standards 10918-1 and 10918-2. The RTP payload format is as specified in RFC 2435 [19] ✨
5.3 H261
The encoding is specified in ITU-T Recommendation H.261, "Video codec for audiovisual services at p x 64 kbit/s". The packetization and RTP-specific properties are described in RFC 2032 [20] 🌸
5.4 H263
The encoding is specified in the 1996 version of ITU-T Recommendation H.263, "Video coding for low bit rate communication". The packetization and RTP-specific properties are described in RFC 2190 [21]. The H263-1998 payload format is RECOMMENDED over this one for use by new implementations 🦋
5.5 H263-1998
The encoding is specified in the 1998 version of ITU-T Recommendation H.263, "Video coding for low bit rate communication". The packetization and RTP-specific properties are described in RFC 2429 [22]. Because the 1998 version of H.263 is a superset of the 1996 syntax, this payload format can also be used with the 1996 version of H.263, and is RECOMMENDED for this use by new implementations. This payload format does not replace RFC 2190, which continues to be used by existing implementations, and may be required for backward compatibility in new implementations. Implementations using the new features of the 1998 version of H.263 MUST use the payload format described in RFC 2429. Non-negotiable, bestie 🔥
5.6 MPV
MPV designates the use of MPEG-1 and MPEG-2 video encoding elementary streams as specified in ISO Standards ISO/IEC 11172 and 13818-2, respectively. The RTP payload format is as specified in RFC 2250 [14], Section 3 🎬
The MIME registration for MPV in RFC 3555 [7] specifies a parameter that MAY be used with MIME or SDP to restrict the selection of the type of MPEG video 💜
5.7 MP2T
MP2T designates the use of MPEG-2 transport streams, for either audio or video. The RTP payload format is described in RFC 2250 [14], Section 2 📺
5.8 nv
The encoding is implemented in the program `nv', version 4, developed at Xerox PARC by Ron Frederick. Further information is available from the author:
Ron Frederick
Blue Coat Systems Inc.
650 Almanor Avenue
Sunnyvale, CA 94085
United States
EMail: ronf@bluecoat.com6. Payload Type Definitions 🌈
Tables 4 and 5 define this profile's static payload type values for the PT field of the RTP data header. In addition, payload type values in the range 96-127 MAY be defined dynamically through a conference control protocol, which is beyond the scope of this document. For example, a session directory could specify that for a given session, payload type 96 indicates PCMU encoding, 8,000 Hz sampling rate, 2 channels 📻 Entries in Tables 4 and 5 with payload type "dyn" have no static payload type assigned and are only used with a dynamic payload type.
Payload type 2 was assigned to G721 in RFC 1890 and to its equivalent successor G726-32 in draft versions of this specification, but its use is now deprecated and that static payload type is marked reserved due to conflicting use for the payload formats G726-32 and AAL2-G726-32 (see Section 4.5.4). Payload type 13 indicates the Comfort Noise (CN) payload format specified in RFC 3389 [9] 🤫 Payload type 19 is marked "reserved" because some draft versions of this specification assigned that number to an earlier version of the comfort noise payload format. The payload type range 72-76 is marked "reserved" so that RTCP and RTP packets can be reliably distinguished (see Section "Summary of Protocol Constants" of the RTP protocol specification) 🔍
The payload types currently defined in this profile are assigned to exactly one of three categories or media types: audio only, video only and those combining audio and video. The media types are marked in Tables 4 and 5 as "A", "V" and "AV", respectively. Payload types of different media types SHALL NOT be interleaved or multiplexed within a single RTP session, but multiple RTP sessions MAY be used in parallel to send multiple media types. An RTP source MAY change payload types within the same media type during a session. See the section "Multiplexing RTP Sessions" of RFC 3550 for additional explanation 🧩
PT encoding media type clock rate channels
name (Hz)
___________________________________________________
0 PCMU A 8,000 1
1 reserved A
2 reserved A
3 GSM A 8,000 1
4 G723 A 8,000 1
5 DVI4 A 8,000 1
6 DVI4 A 16,000 1
7 LPC A 8,000 1
8 PCMA A 8,000 1
9 G722 A 8,000 1
10 L16 A 44,100 2
11 L16 A 44,100 1
12 QCELP A 8,000 1
13 CN A 8,000 1
14 MPA A 90,000 (see text)
15 G728 A 8,000 1
16 DVI4 A 11,025 1
17 DVI4 A 22,050 1
18 G729 A 8,000 1
19 reserved A
20 unassigned A
21 unassigned A
22 unassigned A
23 unassigned A
dyn G726-40 A 8,000 1
dyn G726-32 A 8,000 1
dyn G726-24 A 8,000 1
dyn G726-16 A 8,000 1
dyn G729D A 8,000 1
dyn G729E A 8,000 1
dyn GSM-EFR A 8,000 1
dyn L8 A var. var.
dyn RED A (see text)
dyn VDVI A var. 1
Table 4: Payload types (PT) for audio encodings
PT encoding media type clock rate
name (Hz)
_____________________________________________
24 unassigned V
25 CelB V 90,000
26 JPEG V 90,000
27 unassigned V
28 nv V 90,000
29 unassigned V
30 unassigned V
31 H261 V 90,000
32 MPV V 90,000
33 MP2T AV 90,000
34 H263 V 90,000
35-71 unassigned ?
72-76 reserved N/A N/A
77-95 unassigned ?
96-127 dynamic ?
dyn H263-1998 V 90,000
Table 5: Payload types (PT) for video and combined
encodingsSession participants agree through mechanisms beyond the scope of this specification on the set of payload types allowed in a given session. This set MAY, for example, be defined by the capabilities of the applications used, negotiated by a conference control protocol or established by agreement between the human participants 🫶
Audio applications operating under this profile SHOULD, at a minimum, be able to send and/or receive payload types 0 (PCMU) and 5 (DVI4). This allows interoperability without format negotiation and ensures successful negotiation with a conference control protocol 💖
7. RTP over TCP and Similar Byte Stream Protocols 🌊
Under special circumstances, it may be necessary to carry RTP in protocols offering a byte stream abstraction, such as TCP, possibly multiplexed with other data. The application MUST define its own method of delineating RTP and RTCP packets (RTSP [23] provides an example of such an encapsulation specification) ✨
8. Port Assignment ✨
Okay so, RTP data SHOULD be carried on an even UDP port number 🦋, and the matching RTCP packets SHOULD be carried on the next higher (odd) port number. Think of it like pairing up: even for the media, odd for the control stuff. 🎀
Apps using this profile MAY use any such UDP port pair. Like, the pair MAY be picked randomly by a session management program. A single fixed port pair can't be required, bestie, because lots of apps using this profile are likely to run on the same host 🏠. Some operating systems also don't let multiple processes share the same UDP port with different multicast addresses, so a fixed pair would be a mess 💀.
BUT, port numbers 5004 and 5005 have been registered for this profile for apps that want to use them as the default pair 💖. Apps that run under multiple profiles MAY use this port pair as a hint to pick this profile, as long as they aren't bound by the constraint in the paragraph above. Apps don't need a default, and they MAY require that the port pair be explicitly specified ✨. The port numbers were picked to sit above 5000 because of how some versions of Unix handle ports: numbers below 1024 can only be used by privileged processes, and numbers between 1024 and 5000 get auto-assigned by the OS. So 5004 and 5005 are the iconic choice that stays out of that chaos 👑.
9. Changes from RFC 1890 🌸
This RFC revises RFC 1890. It's mostly backwards-compatible with RFC 1890, except for features that got removed because two interoperable implementations weren't found 😭. The additions codify what people were already doing with payload formats under this profile. Since this profile can be used without any of the payload formats listed here, adding new payload formats in this revision doesn't affect backwards compatibility. Here's the tea, broken into functional and non-functional changes 🍓.
Functional changes:
- Section 11, "IANA Considerations", was added to specify the registration of the name for this profile. That section also references a new Section 3, "Registering Additional Encodings", which sets a policy: no more registration of static payload types for this profile will happen beyond the ones added in this revision and listed in Tables 4 and 5. Instead, extra encoding names may be registered as MIME subtypes for binding to dynamic payload types 💅. Non-normative references were added to RFC 3555 [7], where MIME subtypes for all the listed payload formats are registered, some with optional parameters for using the payload formats.
- Static payload types
4,16,17and34were added to cover IANA registrations made since RFC 1890 came out, along with the payload format descriptions forG723andH263✨. - After working group discussion, static payload types
12and18were added, along with the payload format descriptions forQCELPandG729. Static payload type13was assigned to the Comfort Noise (CN) payload format defined in RFC 3389. Payload type19was marked reserved, because it had been temporarily allocated to an earlier version of Comfort Noise that showed up in some draft revisions of this document 🙈. - The payload format for G721 was renamed to
G726-32after the ITU-T renumbering, and the G726 payload format description was expanded to include the-16,-24and-40data rates. Because of confusion around draft revisions, some implementations of these G726 payload formats packed samples into octets starting with the most significant bit instead of the least significant bit, which is what's specified here 😭. To partially fix that incompatibility, new payload formats namedAAL2-G726-16,-24,-32and-40will be specified in a separate document (see note in Section 4.5.4), and use of static payload type2is deprecated, as explained in Section 6 🥀. - Payload formats
G729DandG729Ewere added after the ITU-T added Annexes D and E to Recommendation G.729. Listings were added for the payload formatsGSM-EFR,RED, andH263-1998, which were published in other documents after RFC 1890. These extra payload formats are only referenced by dynamic payload type numbers 🧚. - The descriptions of the payload formats for G722, G728, GSM, and VDVI were expanded 💜.
- The payload format for 1016 audio was removed, and its static payload type assignment
1was marked "reserved", because two interoperable implementations weren't found 💀. - Requirements for congestion control were added in Section 2 🔥.
- This profile follows the suggestion in the revised RTP spec that RTCP bandwidth may be specified separately from the session bandwidth, and separately for active senders and passive receivers 👑.
- The mapping of a user pass-phrase string into an encryption key was deleted from Section 2, because two interoperable implementations weren't found 😭.
- The "quadrophonic" sample ordering convention for four-channel audio was removed, to get rid of an ambiguity noted in Section 4.1 🌈.
Non-functional changes:
- In Section 4.1, it's now explicitly stated that silence suppression is allowed for all audio payload formats. (This has always been the case and comes from a fundamental part of RTP's design and the motivations for packet audio, but it wasn't explicitly stated before.) The use of comfort noise is also explained 🎧.
- In Section 4.1, the requirement level for setting the marker bit on the first packet after silence for audio changed from "is" to "SHOULD be", and it's clarified that the marker bit is only set when packets are intentionally not sent 🫶.
- Similarly, text was added to specify that the marker bit SHOULD be set to one on the last packet of a video frame, and that video frames are told apart by their timestamps 🎬.
- RFC references are added for payload formats published after RFC 1890 📚.
- The security considerations and full copyright sections were added 🔒.
- According to Peter Hoddie of Apple, only pre-1994 Macintosh used the
22254.54rate and none used the11127.27rate, so the latter was dropped from the discussion of suggested sampling frequencies 🍎. - Table 1 was corrected to move some values from the "ms/packet" column to the "default ms/packet" column, where they belonged 🙌.
- Since the Interactive Multimedia Association ceased operations, an alternate resource was provided for a referenced IMA document 🌷.
- A note was added for G722 to clarify a discrepancy between the actual sampling rate and the RTP timestamp clock rate ⭐.
- Small clarifications were made in several places, some in response to reader questions. In particular:
- A definition for "media type" is given in Section 1.1, so the explanation of multiplexing RTP sessions in Section 6 is clearer about multiplexing multiple media 💖.
- The explanation of how to work out the number of audio frames in a packet from its length was expanded 🧮.
- More description was added on how bandwidth is allocated to SDES items 📝.
- A note was added that the convention for channel order in Section 4.1 may be overridden by a particular encoding or payload format specification 🎶.
- The terms MUST, SHOULD, MAY, etc. are used as defined in RFC 2119 💅.
- A second author for this document was added 👯.
10. Security Considerations 🔐
Implementations using the profile defined in this specification are subject to the security considerations discussed in the RTP specification [1]. This profile doesn't specify any different security services. Its main job is to list a set of data compression encodings for audio and video media 🎧.
Confidentiality of the media streams is achieved by encryption. Because the data compression used with the payload formats described in this profile is applied end-to-end, encryption may be performed after compression, so there's no conflict between the two operations 🔑.
A potential denial-of-service threat exists for data encodings that use compression techniques with non-uniform receiver-end computational load. An attacker can inject pathological datagrams into the stream that are complex to decode and overload the receiver. This is a real threat, so take it seriously 💀🚨.
As with any IP-based protocol, a receiver may in some circumstances be overloaded just by receiving too many packets, whether they're wanted or not. Network-layer authentication MAY be used to discard packets from undesired sources, but the processing cost of the authentication itself may be too high 🥵. In a multicast environment, source pruning is implemented in IGMPv3 (RFC 3376) [24] and in multicast routing protocols, so a receiver can choose which sources are allowed to reach it 🛡️.
11. IANA Considerations 📋
The RTP specification establishes a registry of profile names for use by higher-level control protocols, such as the Session Description Protocol (SDP), RFC 2327 [6], to refer to transport methods. This profile registers the name RTP/AVP 🏷️.
Section 3 establishes the policy that no more registration of static RTP payload types for this profile will happen beyond those added in this document revision and included in Tables 4 and 5. IANA may reference that section when declining any additional registration requests. In Tables 4 and 5, note that types 1 and 2 have been marked reserved, and the set of "dyn" payload types included has been updated. These changes are explained in Sections 6 and 9 ✨.
12. References 📚
12.1 Normative References
[1] Schulzrinne, H., Casner, S., Frederick, R. and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", RFC 3550, July 2003.
[2] Bradner, S., "Key Words for Use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
[3] Apple Computer, "Audio Interchange File Format AIFF-C", August 1991. (also ftp://ftp.sgi.com/sgi/aiff-c.9.26.91.ps.Z).
12.2. Informative References 📚✨
okay so these are the background reads, bestie. They're not required to implement RFC 3551, but they're where all the good context lives 🤓
[4] Braden, R., Clark, D. and S. Shenker, "Integrated Services in the Internet Architecture: an Overview", RFC 1633, June 1994.
[5] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z. and W. Weiss, "An Architecture for Differentiated Service", RFC 2475, December 1998.
[6] Handley, M. and V. Jacobson, "SDP: Session Description Protocol", RFC 2327, April 1998.
[7] Casner, S. and P. Hoschka, "MIME Type Registration of RTP Payload Types", RFC 3555, July 2003.
[8] Freed, N., Klensin, J. and J. Postel, "Multipurpose Internet Mail Extensions (MIME) Part Four: Registration Procedures", BCP 13, RFC 2048, November 1996.
[9] Zopf, R., "Real-time Transport Protocol (RTP) Payload for Comfort Noise (CN)", RFC 3389, September 2002.
[10] Deleam, D. and J.-P. Petit, "Real-time implementations of the recent ITU-T low bit rate speech coders on the TI TMS320C54X DSP: results, methodology, and applications", in Proc. of International Conference on Signal Processing, Technology, and Applications (ICSPAT) , (Boston, Massachusetts), pp. 1656--1660, October 1996.
[11] Mouly, M. and M.-B. Pautet, The GSM system for mobile communications Lassay-les-Chateaux, France: Europe Media Duplication, 1993.
[12] Degener, J., "Digital Speech Compression", Dr. Dobb's Journal, December 1994.
[13] Redl, S., Weber, M. and M. Oliphant, An Introduction to GSM Boston: Artech House, 1995.
[14] Hoffman, D., Fernando, G., Goyal, V. and M. Civanlar, "RTP Payload Format for MPEG1/MPEG2 Video", RFC 2250, January 1998.
[15] Jayant, N. and P. Noll, Digital Coding of Waveforms--Principles and Applications to Speech and Video Englewood Cliffs, New Jersey: Prentice-Hall, 1984.
[16] McKay, K., "RTP Payload Format for PureVoice(tm) Audio", RFC 2658, August 1999.
[17] Perkins, C., Kouvelas, I., Hodson, O., Hardman, V., Handley, M., Bolot, J.-C., Vega-Garcia, A. and S. Fosse-Parisis, "RTP Payload for Redundant Audio Data", RFC 2198, September 1997.
[18] Speer, M. and D. Hoffman, "RTP Payload Format of Sun's CellB Video Encoding", RFC 2029, October 1996.
[19] Berc, L., Fenner, W., Frederick, R., McCanne, S. and P. Stewart, "RTP Payload Format for JPEG-Compressed Video", RFC 2435, October 1998.
[20] Turletti, T. and C. Huitema, "RTP Payload Format for H.261 Video Streams", RFC 2032, October 1996.
[21] Zhu, C., "RTP Payload Format for H.263 Video Streams", RFC 2190, September 1997.
[22] Bormann, C., Cline, L., Deisher, G., Gardos, T., Maciocco, C., Newell, D., Ott, J., Sullivan, G., Wenger, S. and C. Zhu, "RTP Payload Format for the 1998 Version of ITU-T Rec. H.263 Video (H.263+)", RFC 2429, October 1998.
[23] Schulzrinne, H., Rao, A. and R. Lanphier, "Real Time Streaming Protocol (RTSP)", RFC 2326, April 1998.
[24] Cain, B., Deering, S., Kouvelas, I., Fenner, B. and A. Thyagarajan, "Internet Group Management Protocol, Version 3", RFC 3376, October 2002.
13. Current Locations of Related Resources 🗺️💖
the lil resource map! several sections point at the ITU-T Software Tool Library (STL), so here's where to find it. ngl this part is a scavenger hunt 🦋
Note: Several sections below refer to the ITU-T Software Tool Library (STL). It is available from the ITU Sales Service, Place des Nations, CH-1211 Geneve 20, Switzerland (also check http://www.itu.int). The ITU-T STL is covered by a license defined in ITU-T Recommendation G.191, "Software tools for speech and audio coding standardization".
DVI4
An archived copy of the document IMA Recommended Practices for Enhancing Digital Audio Compatibility in Multimedia Systems (version 3.0), which describes the IMA ADPCM algorithm, is available at:
http://www.cs.columbia.edu/~hgs/audio/dvi/
An implementation is available from Jack Jansen at
ftp://ftp.cwi.nl/local/pub/audio/adpcm.shar
G722
An implementation of the G.722 algorithm is available as part of the ITU-T STL, described above.
G723
The reference C code implementation defining the G.723.1 algorithm and its Annexes A, B, and C are available as an integral part of Recommendation G.723.1 from the ITU Sales Service, address listed above. Both the algorithm and C code are covered by a specific license. The ITU-T Secretariat should be contacted to obtain such licensing information.
G726
G726 is specified in the ITU-T Recommendation G.726, "40, 32, 24, and 16 kb/s Adaptive Differential Pulse Code Modulation (ADPCM)". An implementation of the G.726 algorithm is available as part of the ITU-T STL, described above.
G729
The reference C code implementation defining the G.729 algorithm and its Annexes A through I are available as an integral part of Recommendation G.729 from the ITU Sales Service, listed above. Annex I contains the integrated C source code for all G.729 operating modes. The G.729 algorithm and associated C code are covered by a specific license. The contact information for obtaining the license is available from the ITU-T Secretariat.
GSM
A reference implementation was written by Carsten Bormann and Jutta Degener (then at TU Berlin, Germany). It is available at
http://www.dmn.tzi.org/software/gsm/
Although the RPE-LTP algorithm is not an ITU-T standard, there is a C code implementation of the RPE-LTP algorithm available as part of the ITU-T STL. The STL implementation is an adaptation of the TU Berlin version.
LPC
An implementation is available at
ftp://parcftp.xerox.com/pub/net-research/lpc.tar.Z
PCMU, PCMA
An implementation of these algorithms is available as part of the ITU-T STL, described above.
14. Acknowledgments 🫶
big thank-yous to Simao Campos, Richard Cox and the AVT Working Group participants for their comments and careful review 💜 The GSM description was adopted from the IMTC Voice over IP Forum Service Interoperability Implementation Agreement (January 1997). Fred Burg and Terry Lyons helped with the G.729 description. literally the squad 👑
15. Intellectual Property Rights Statement ⚖️
the legal part, but make it cute 🎀 The IETF takes no position on the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document, or on whether any license under such rights might or might not be available. It also doesn't say it has tried to find any such rights. Info on the IETF's procedures for rights in standards-track and standards-related documentation can be found in BCP-11. Copies of claims of rights made available for publication, any assurances of licenses to be made available, or the result of an attempt to get a general license or permission for implementors or users to use such proprietary rights can be obtained from the IETF Secretariat.
The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights which may cover technology that may be required to practice this standard. Please address the information to the IETF Executive Director.
16. Authors' Addresses 💌
who wrote this! 🧚
Henning Schulzrinne Department of Computer Science Columbia University 1214 Amsterdam Avenue New York, NY 10027 United States
EMail: schulzrinne@cs.columbia.edu
Stephen L. Casner Packet Design 3400 Hillview Avenue, Building 3 Palo Alto, CA 94304 United States
EMail: casner@acm.org
17. Full Copyright Statement 🌈
the last boring-but-important bit, still complete 💅
Copyright (C) The Internet Society (2003). All Rights Reserved.
Anyone can copy and furnish this document and translations of it to others, and prepare, copy, publish and distribute derivative works that comment on or otherwise explain it or assist in its implementation, in whole or in part, without restriction of any kind. The only catch is that the above copyright notice and this paragraph must be included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations. The exceptions are when needed for developing Internet standards, in which case the procedures for copyrights defined in the Internet Standards process must be followed, or when required to translate it into languages other than English.
The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Acknowledgement
Funding for the RFC Editor function is currently provided by the Internet Society. 🦋✨
📦 where this page comes from
- ietf: rfc3551 2003-07
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms