RFC 5574
RTP Payload Format for the Speex Codec
RFC 5574: RTP Payload Format for the Speex Codec 💖🎀
Okay bestie, this is the doc that explains how to pack Speex voice audio into RTP packets, plus the SDP details you need to set it up 🦋
The authors are G. Herlein (Independent), J. Valin (Xiph.Org Foundation), A. Heggestad (Creytiv.com), and A. Moizard (Antisip) ✨. It's a Standards Track document from the Network Working Group, dated June 2009 📅. The title block doesn't list any RFCs it obsoletes or updates, so nothing gets replaced or amended here 🧚.
Status of This Memo ✨
This document is an Internet standards track protocol for the Internet community, and it's asking for discussion and suggestions for improvements 🫶. For the standardization state and status of this protocol, check the current edition of the "Internet Official Protocol Standards" (STD 1). Distribution of this memo is unlimited 🌈.
Copyright Notice 💅
Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved 🔥.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents, as in effect on the date of publication (http://trustee.ietf.org/license-info). Read these carefully, because they spell out your rights and restrictions for this document 📜.
Abstract 🎀
Speex is an open-source voice codec made for VoIP (Voice over IP) apps 🌸. This document explains the payload format for Speex bit streams inside an RTP packet, and it also covers what you need to use Speex with the Session Description Protocol (SDP) 🧚.
Table of Contents 📜
1. Introduction ....................................................3
2. Terminology .....................................................3
3. RTP Usage for Speex .............................................3
3.1. RTP Speex Header Fields ....................................3
3.2. RTP Payload Format for Speex ...............................4
3.3. Speex Payload ..............................................4
3.4. Example Speex Packet .......................................5
3.5. Multiple Speex Frames in an RTP Packet .....................5
4. IANA Considerations .............................................6
4.1. Media Type Registration ....................................6
4.1.1. Registration of Media Type Audio/Speex ..............6
5. SDP Usage of Speex ..............................................8
5.1. Example Supporting All Modes, Prefer Mode 4 ...............10
5.2. Example Supporting Only Modes 3 and 5 .....................10
5.3. Example with Variable Bit-Rate and Comfort Noise ..........10
5.4. Example with Voice Activity Detection .....................11
5.5. Example with Multiple Sampling Rates ......................11
5.6. Example with Ptime and Multiple Speex Frames ..............11
5.7. Example with Complete Offer/Answer Exchange ...............12
6. Implementation Guidelines ......................................12
7. Security Considerations ........................................12
8. Acknowledgments ................................................13
9. References .....................................................13
9.1. Normative References ......................................13
9.2. Informative References ....................................131. Introduction ✨
Speex is built on Code Excited Linear Prediction [CELP] encoding, and it supports narrowband (nominal 8 kHz), wideband (nominal 16 kHz), or ultra-wideband (nominal 32 kHz) audio 🎤. Here's the highlight reel:
- Free software/open-source 💖
- Integration of wideband and narrowband in the same bit-stream 🌈
- Wide range of bit-rates available 🍓
- Dynamic bit-rate switching and variable bit-rate (VBR) 🦋
- Voice Activity Detection (VAD, integrated with VBR) 🤫
- Variable complexity 💅
Speex covers bit-rates from 2.15 kbit/s all the way up to 44 kbit/s, which is a lot of range 😭. Some implementations can't support every rate, though (because of bandwidth, RAM, or CPU limits), so the bar is lower for them. To be compliant with this specification, implementations MUST support at least narrowband (8 kHz) encoding and decoding at 8 kbit/s (narrowband mode 3) 💀. Narrowband at 15 kbit/s (narrowband mode 5) is RECOMMENDED, and wideband at 27.8 kbit/s (wideband mode 8) is also RECOMMENDED 🫶. The sampling rate MUST be 8, 16 or 32 kHz 🔥. This specification only covers single channel audio (mono) 🎀.
2. Terminology 📖
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC2119 [RFC2119], and they indicate requirement levels for compliant RTP implementations 🧠.
3. RTP Usage for Speex 🎧
3.1. RTP Speex Header Fields 🌸
The RTP header is defined in the RTP specification [RFC3550]. This section explains how the fields in the RTP header get used 💖.
Payload Type (PT): Assigning an RTP payload type for this packet format is outside the scope of this document. The RTP profile this payload format is used under specifies it, or it gets signaled dynamically out-of-band (e.g., using SDP) 🧚.
Marker (M) bit: The M bit is set to one on the first packet sent after a silence period, during which packets haven't been transmitted contiguously 🤫.
Extension (X) bit: Defined by the RTP profile used 🦋.
Timestamp: A 32-bit word that matches the sampling instant for the first frame in the RTP packet ⏰.
3.2. RTP Payload Format for Speex 🍓
The RTP payload for Speex has the format shown in Figure 1. No additional header fields specific to this payload format are required. For RTP-based transport of Speex-encoded audio, the standard RTP header [RFC3550] is followed by one or more payload data blocks, and an optional padding terminator may also be used 🎀.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| RTP Header |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| one or more frames of Speex .... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| one or more frames of Speex .... | padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 1: RTP Payload for Speex 🧁
3.3. Speex Payload 🎶
If you're packetizing the bit stream in RTP, you only need to care about the sequence of bits the Speex encoder [SPEEX] outputs, and then you present that same sequence to the decoder. The payload format here keeps that sequence intact ✨.
A typical Speex frame at the maximum bit-rate is approximately 110 octets, and the total number of Speex frames SHOULD be kept less than the path MTU to prevent fragmentation 📏. Speex frames MUST NOT be fragmented across multiple RTP packets 🚫.
The Speex frames must be placed starting with the oldest frame, then continue consecutively in time ⏳.
An RTP packet MAY contain Speex frames of the same bit-rate or of varying bit-rates, since the bit-rate for each frame is conveyed in-band with the signal 🦋.
The encoding and decoding algorithm can change the bit-rate at any 20 msec frame boundary, and the bit-rate change notification comes in-band with the bit stream 🔥. Each frame carries both its sampling rate (narrowband, wideband, or ultra-wideband) and its "mode" (bit-rate) in the bit stream 💖. No out-of-band notification is required for the decoder to process bit-rate changes sent by the encoder 🙌.
The sampling rate MUST be either 8000 Hz, 16000 Hz, or 32000 Hz 🎀.
The RTP payload MUST be padded to provide an integer number of octets as the payload length. These padding bits are LSB-aligned (Least Significant Bit) in network octet order, and they consist of a 0 followed by all ones (until the end of the octet). This padding is only required for the last frame in the packet, and only to make sure the packet contents end on an octet boundary 🧩.
3.4. Example Speex Packet 👀
In the example below, there's a single Speex frame with 5 bits of padding, so the packet size lands on an octet boundary 🍓.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| RTP Header |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| ..speex data.. |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ..speex data.. |0 1 1 1 1|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+3.5. Multiple Speex Frames in an RTP Packet 👯
Here's an example of two Speex frames packed into one RTP packet. The Speex frame length in this example lands on an octet boundary, so there's no padding 💅.
The Speex decoder [SPEEX] can detect the bit-rate from the payload, and it's responsible for detecting the 20 msec boundaries between each frame 🧠.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| RTP Header |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| ..speex frame 1.. |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ..speex frame 1.. | ..speex frame 2.. |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ..speex frame 2.. |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+4. IANA Considerations 🗂️
This document defines the Speex media type 🌈.
4.1. Media Type Registration 📝
This section describes the media types and names tied to this payload format. It registers the media types as per RFC 4288 [RFC4288] 🦋.
To be continued... 🫶🧚
4.1.1. Registration of Media Type Audio/Speex ✨
okay so this is the official media type registration, aka the paperwork that says what audio/speex is 💅
Media type name: audio
Media subtype name: speex
Required parameters:
- rate: the RTP timestamp clock rate, which is the same as the sampling rate in Hz. The sampling rate MUST be either 8000, 16000, or 32000. 🎀
Optional parameters:
- ptime: SHOULD be a multiple of 20 msec [RFC4566] 🦋
- maxptime: SHOULD be a multiple of 20 msec [RFC4566] 🦋
- vbr: variable bit-rate, which can be
'on','off', or'vad'(defaults to'off'). If it's'on', variable bit-rate is turned on. If it's'off', it's turned off. If it's set to'vad', then constant bit-rate is used, but silence gets encoded with special short frames to show there's no voice during that stretch. This one is just a preference for the encoder, so it's not a must 🌸 - cng: comfort noise generation, which can be
'on'or'off'(defaults to'off'). If it's'off', silence frames stay silent. If it's'on', those frames get filled with comfort noise. Also just an encoder preference 🫶 - mode: a comma-separated list of the Speex decoding modes you support, in order of preference. The first one is the most preferred, and the rest go in decreasing order of preference. The valid modes are different for narrowband and wideband, and they're defined like this:
{1,2,3,4,5,6,7,8,any}for narrowband{0,1,2,3,4,5,6,7,8,9,10,any}for wideband and ultra-wideband
The mode parameter can hold multiple values. When that happens, the remote party SHOULD configure its encoder using the first supported mode that's given. When any shows up, the offerer is saying it supports all decoding modes. The mode parameter value MUST always be quoted. If the mode parameter isn't provided, the mode value counts as equivalent to mode="3,any" in narrowband and mode="8,any" in wideband and ultra-wideband. 🍓 Also, each Speex frame does contain the mode (or bit-rate) that should be used to decode it. So an application MUST be able to decode any Speex frame, unless the SDP clearly says some modes aren't supported (for example, by leaving out mode="any"). Saying you support a given set of decoding modes also means the implementation supports the same encoding modes 💜
Encoding considerations:
This media type is framed and binary, see Section 4.8 in [RFC4288]. 🧚
Security considerations: See Section 6. 🔒
Interoperability considerations: None. ✨
Published specification: RFC 5574. 📜
Applications that use this media type: Audio streaming and conferencing applications. 🎧
Additional information: none.
Person and e-mail address to contact for further information: Alfred E. Heggestad: aeh@db.org 💌
Intended usage: COMMON
Restrictions on usage:
This media type depends on RTP framing, so it's only defined for transfer via RTP [RFC3550]. Transport within other framing protocols is not defined at this time. ⚠️
Author: Alfred E. Heggestad 👑
Change controller:
IETF Audio/Video Transport working group delegated from the IESG. 🌈
5. SDP Usage of Speex 🎀
The info in the media type specification maps to specific fields in the Session Description Protocol (SDP) [RFC4566], which is what people usually use to describe RTP sessions. When SDP is used to set up sessions that use the Speex codec, the mapping goes like this:
- The media type (
audio) goes in SDPm=as the media name. 🦋 - The media subtype (
speex) goes in SDPa=rtpmapas the encoding name. The required parameterratealso goes ina=rtpmapas the clock rate. 💖 - The parameters
ptimeandmaxptimego in the SDPa=ptimeanda=maxptimeattributes, respectively. 🌸 - Any remaining parameters go in the SDP
a=fmtpattribute, copied straight from the media type string as a semicolon-separated list of parameter=value pairs. 🍓
The tables below show the mode-to-bit-rate equivalences for narrowband, wideband, and ultra-wideband. There's also a matching "Speex quality" setting (see SPEEX_SET_QUALITY in the Speex Codec Manual [SPEEX]) included as an indication. 📊
+------+---------------+-------------+
| mode | Speex quality | bit-rate |
+------+---------------+-------------+
| 1 | 0 | 2.15 kbit/s |
| 2 | 2 | 5.95 kbit/s |
| 3 | 3 or 4 | 8.00 kbit/s |
| 4 | 5 or 6 | 11.0 kbit/s |
| 5 | 7 or 8 | 15.0 kbit/s |
| 6 | 9 | 18.2 kbit/s |
| 7 | 10 | 24.6 kbit/s |
| 8 | 1 | 3.95 kbit/s |
+------+---------------+-------------+
Table 1: Mode vs. Bit-Rate for Narrowband+------+---------------+-------------------+------------------------+
| mode | Speex quality | wideband bit-rate | ultra wideband |
| | | | bit-rate |
+------+---------------+-------------------+------------------------+
| 0 | 0 | 3.95 kbit/s | 5.75 kbit/s |
| 1 | 1 | 5.75 kbit/s | 7.55 kbit/s |
| 2 | 2 | 7.75 kbit/s | 9.55 kbit/s |
| 3 | 3 | 9.80 kbit/s | 11.6 kbit/s |
| 4 | 4 | 12.8 kbit/s | 14.6 kbit/s |
| 5 | 5 | 16.8 kbit/s | 18.6 kbit/s |
| 6 | 6 | 20.6 kbit/s | 22.4 kbit/s |
| 7 | 7 | 23.8 kbit/s | 25.6 kbit/s |
| 8 | 8 | 27.8 kbit/s | 29.6 kbit/s |
| 9 | 9 | 34.2 kbit/s | 36.0 kbit/s |
| 10 | 10 | 42.2 kbit/s | 44.0 kbit/s |
+------+---------------+-------------------+------------------------+
Table 2: Mode vs. Bit-Rate for Wideband and Ultra-WidebandThe Speex parameters tell you the decoding capabilities of the agent, and what the agent would rather receive. 🧠
In an SDP Offer/Answer exchange, the Speex parameters are completely orthogonal, so there's no relationship between the SDP Offer and the Answer. 🙈
You can give several Speex-specific parameters in one a=fmtp line, as long as you separate them with a semicolon:
a=fmtp:97 mode="1,any";vbr=onHere are some example SDP session descriptions using Speex encodings. 💅
5.1. Example Supporting All Modes, Prefer Mode 4
The offerer wants to receive a Speex stream at 8000 Hz, and wants to receive Speex 'mode 4'. It's important to understand that the remote party might still send any other mode: the device might have bandwidth limitations, or it might only be able to send 'mode="3"'. So applications that support all decoding modes SHOULD include mode="any", as shown in the example below: 🌈
m=audio 8088 RTP/AVP 97
a=rtpmap:97 speex/8000
a=fmtp:97 mode="4,any"5.2. Example Supporting Only Modes 3 and 5
The offerer says which mode it wants to receive (Speex 'mode 3'). This offer says mode 3 and mode 5 are supported, and that no other modes are. The remote party MUST NOT configure its encoder using another Speex mode. 🚫
m=audio 8088 RTP/AVP 97
a=rtmap:97 speex/8000
a=fmtp:97 mode="3,5"5.3. Example with Variable Bit-Rate and Comfort Noise
The offerer wants to receive variable bit-rate frames with comfort noise: 🎶
m=audio 8088 RTP/AVP 97
a=rtmap:97 speex/8000
a=fmtp:97 vbr=on;cng=on5.4. Example with Voice Activity Detection
The offerer wants to use silence suppression. In this case, the vbr=vad parameter gets used: 🤫
m=audio 8088 RTP/AVP 97
a=rtmap:97 speex/8000
a=fmtp:97 vbr=vad5.5. Example with Multiple Sampling Rates
The offerer wants to receive Speex audio at 16000 Hz with mode 10 (42.2 kbit/s) or, alternatively, Speex audio at 8000 Hz with mode 7 (24.6 kbit/s). The offerer supports decoding all modes. 🧁
m=audio 8088 RTP/AVP 97 98
a=rtmap:97 speex/16000
a=fmtp:97 mode="10,any"
a=rtmap:98 speex/8000
a=fmtp:98 mode="7,any"5.6. Example with Ptime and Multiple Speex Frames
The "ptime" SDP attribute denotes the packetization interval, meaning how many milliseconds of audio are encoded in a single RTP packet. Since Speex uses 20 msec frames, ptime values that are multiples of 20 mean multiple Speex frames per packet. It is recommended to use ptime values that are a multiple of 20. 🧩
If ptime contains a value that is not a multiple of 20, the internal interpretation of it should be rounded up to the nearest multiple of 20 before the number of Speex frames is calculated. For example, if the "ptime" attribute is set to 30, the internal interpretation should be rounded up to 40, and then used to calculate two Speex frames per packet. 🧮
In the example below, the ptime value is set to 40, which means there are two frames in each packet: 🎀
m=audio 8088 RTP/AVP 97
a=rtpmap:97 speex/8000
a=ptime:40Note that the ptime parameter applies to all payloads listed in the media line, and it isn't used as part of an a=fmtp directive. 📝
Care must be taken when setting the value of ptime so that the RTP packet size does not exceed the path MTU. ⚠️
5.7. Example with Complete Offer/Answer Exchange
The offerer wants to receive Speex audio at 16000 Hz or, alternatively, Speex audio at 8000 Hz. The offerer does support ALL modes, because no mode is specified. 🥰
m=audio 8088 RTP/AVP 97 98
a=rtmap:97 speex/16000
a=rtmap:98 speex/8000The answerer wants to receive Speex audio at 8000 Hz, which is the only sampling rate it supports. The answerer does support ALL modes, because no mode is specified. 🫶
m=audio 8088 RTP/AVP 99
a=rtmap:99 speex/80006. Implementation Guidelines 🛠️
Implementations that support Speex are responsible for correctly decoding incoming Speex frames. 💪
Each Speex frame does contain all the information it needs to decode itself. In particular, the 'mode' and 'ptime' values proposed in the SDP contents MUST NOT be used for decoding, because those values aren't needed to properly decode an RTP Speex stream. 🔥
7. Security Considerations 🔒
okay bestie, the RTP packets using this payload format come with the same security considerations as the RTP specification [RFC3550] and any appropriate RTP profile 💖 So confidentiality of the media streams gets handled by encryption 🔐 And since the compression is applied end-to-end, you can encrypt after compressing, so the two never fight each other 💅✨
ngl there's one potential denial-of-service vibe 😬 Some compression techniques put a non-uniform computational load on the receiving end. An attacker could sneak pathological datagrams into the stream that are super complex to decode and overload the receiver 💀 BUT the good news is this encoding doesn't have any significant non-uniformity, so it doesn't have that problem 🙌🎀
And like with any IP-based protocol, a receiver can get overloaded just from receiving too many packets, whether they're wanted or not 🫠 Network-layer authentication can be used to toss packets from unwanted sources 🧚 Just know that the processing cost of the authentication itself might be too high 😭
8. Acknowledgments 🫶
the authors wanna thank Equivalence Pty Ltd of Australia 🇦🇺 for helping try to standardize Speex use in H.323 applications, and for implementing Speex in their open-source OpenH323 stack 👑 They also thank Brian C. Wiles <brian@streamcomm.com> of StreamComm for helping develop the proposed standard for Speex use in H.323 applications 💜
the authors also thank these members of the Speex and AVT communities for their input: Ross Finlayson, Federico Montesino Pouzols, Henning Schulzrinne, Magnus Westerlund, Colin Perkins, and Ivo Emanuel Goncalves 🌸🦋
and a big thanks to the former authors of this document: Simon Morlat, Roger Hardiman, and Phil Kerr 🍓✨
9. References 📚
9.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, RFC 3550, July 2003.
[RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
Description Protocol", RFC 4566, July 2006.9.2. Informative References
[CELP] Schroeder, M. and B. Atal, "Code-excited linear
prediction(CELP): High-quality speech at very low bit
rates", Proc. International Conference on Acoustics,
Speech, and Signal Processing (ICASSP), Vol 10, pp. 937-
940, 1985, <http://www.ntis.gov/>.
[RFC4288] Freed, N. and J. Klensin, "Media Type Specifications and
Registration Procedures", BCP 13, RFC 4288, December 2005.
[SPEEX] Valin, J., "The Speex Codec Manual",
<http://www.speex.org/docs/>.Authors' Addresses ✨
Greg Herlein
Independent
2034 Filbert Street
San Francisco, California 94123
United States
EMail: gherlein@herlein.com
Jean-Marc Valin
Xiph.Org Foundation
EMail: jean-marc.valin@usherbrooke.ca
Alfred E. Heggestad
Creytiv.com
Biskop J. Nilssonsgt. 20a
Oslo 0659
Norway
EMail: aeh@db.org
Aymeric Moizard
Antisip
5 Place Benoit Crepu
Lyon, 69005
France
EMail: jack@atosc.org📦 where this page comes from
- ietf: rfc5574 2009-06
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms