RFC 6464
A Real-time Transport Protocol (RTP) Header Extension for Client-to-Mixer Audio Level Indication
RFC 6464: Title Block 🎀
Okay bestie, the title block is the ID card of this doc, so here's the tea ☕
Internet Engineering Task Force (IETF), Request for Comments: 6464, ISSN: 2070-1721
Authors: J. Lennox, Ed. (Vidyo), E. Ivov (Jitsi), and E. Marocco (Telecom Italia). Three cuties, one spec 💜
Category: Standards Track, so this is an actual Internet Standard and not just a vibe 🔥
Date: December 2011 📅
The title block doesn't say it obsoletes or updates anything, so it's a brand new standard with no drama attached 🌸
A Real-time Transport Protocol (RTP) Header Extension for Client-to-Mixer Audio Level Indication 🦋
Abstract ✨
Okay so this doc defines a way for RTP audio packets to shout out how loud their audio is, using an RTP header extension. Picture a big conference: the mixer or other middlebox only wants to forward the few loudest streams, and without this, it has to decode and measure every single stream it gets. With this, it can skip all that heavy lifting 💖
Status of This Memo 📝
This is an Internet Standards Track document. Lowkey the most official label you can get 💅
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). If you want more info on Internet Standards, check Section 2 of RFC 5741 🫶
For the current status, any errata, and how to send feedback, go to http://www.rfc-editor.org/info/rfc6464 🌈
Copyright Notice ©️
Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. Protected, obviously 💜
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they were in effect on the date of publication. Please actually read these, because they spell out your rights and restrictions for this doc 🙈
If you pull Code Components out of this document, they must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come without warranty as described in the Simplified BSD License. So no, you can't just yeet the code and blame us 💀
Table of Contents 📑
1. Introduction ....................................................2
2. Terminology .....................................................3
3. Audio Levels ....................................................3
4. Signaling (Setup) Information ...................................5
5. Considerations on Use ...........................................6
6. Security Considerations .........................................6
7. IANA Considerations .............................................7
8. References ......................................................7
8.1. Normative References .......................................7
8.2. Informative References .....................................81. Introduction ✨
Okay so picture a centralized RTP [RFC3550] audio conference. There's an audio mixer or forwarder that gets audio streams from pretty much everyone in the call, and then it picks some of them to send back out to the other participants 🎧
In a big conference, that server could be getting a huge pile of streams where only a handful actually need to go out. To choose which ones, it has to decode the audio, measure the levels, and maybe run voice activity detection on every single stream. That's a LOT of work, and it's the thing that limits how big or how many conferences a server can handle 😭
So this doc introduces an alternative: an RTP header extension [RFC5285] that lets senders say how loud the audio in each packet is. That takes the processing load off the server 🦋
This header extension is different from the one in [RFC6465], but they're complementary. [RFC6465] lets audio mixers tell clients the levels of the contributing sources that went into the mixed audio. So they're besties who work well together 👯♀️
2. Terminology 📖
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. They're the requirement levels for compliant implementations, so these ones are serious 🔥
3. Audio Levels 🔊
The audio level header extension carries the level of the audio in the RTP [RFC3550] payload of the packet it's attached to. It's packaged as an RTP header extension element, following "A General Mechanism for RTP Header Extensions" [RFC5285] 💖
The payload of the audio level header extension element can use either the one-byte or the two-byte header format from [RFC5285]. Figures 1 and 2 show sample audio level encodings for each of those formats 🧚
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ID | len=0 |V| level |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 1: Sample Audio Level Encoding Using the
One-Byte Header Format
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ID | len=1 |V| level | 0 (pad) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 2: Sample Audio Level Encoding Using the
Two-Byte Header FormatHeads up on the length field, because it's a little different between the two formats. In [RFC5285]'s one-byte header format, a length value of 0 means 1 byte follows. In the two-byte header format, the length field takes the value of 1 instead 🙈
The magnitude of the audio level is packed into the seven least significant bits of the single byte in the header extension, as shown in Figures 1 and 2. The least significant bit of the magnitude goes into the least significant bit of the byte. The most significant bit of the byte is a separate flag bit called "V", which we'll get to below 👑
The audio level is expressed in -dBov, with values from 0 to 127 representing 0 to -127 dBov. dBov is the level in decibels relative to the overload point of the system, which means the highest-intensity signal the payload format can encode. (Fun fact: measuring relative to the overload point is super useful for digital implementations, because you don't need to know how the analog circuitry was calibrated 🤓) For example, with u-law (audio/pcmu) audio [ITU.G711], the 0 dBov reference is a square wave with values +/- 8031. (That works out to 6.18 dBm0, relative to u-law's dBm0 definition in Table 6 of [ITU.G711].)
Digital silence is a big one. For a muted audio source, for example, the audio level MUST be represented as 127 (-127 dBov), no matter what the dynamic range of the encoded audio format is. Non-negotiable, periodt 💀
The audio level header extension only carries the level of the audio in the RTP payload of its packet. There's no long-term averaging or smoothing applied, so it's just the raw, honest number for that packet ✨ For payload formats with extra error-correction bits or loss-concealment info, the level only matches the data from the payload's normal decoding process. It does NOT reflect what you'd get under error or packet loss concealment 🙅♀️ The level is measured as a root mean square of all the samples in the audio encoded by the packet.
To make encoding easier, Appendix A of [RFC6465] has a sample Java implementation of an audio level calculator. It helps you get these values from raw linear Pulse Code Modulation (PCM) audio samples 💻
Now the flag bit. The "V" bit optionally indicates whether the encoder believes the audio packet contains voice activity. If the V bit is in use, a value of 1 means the encoder believes there's voice activity, and a value of 0 means it believes there isn't. (The voice activity detection algorithm is unspecified and left up to the implementation, so no one's judging your secret sauce 🤫) If the V bit is NOT in use, its value is unspecified and it MUST be ignored by receivers. Whether the V bit is in use gets signaled with the extension attribute "vad", which Section 4 covers.
When this header extension is used with RTP data sent using the RTP Payload for Redundant Audio Data [RFC2198], the header's data describes the contents of the primary encoding 🧸
Note: This audio level is defined the same way as the audio noise level in the RTP Payload Comfort Noise specification [RFC3389]. In [RFC3389], the overall magnitude of the noise level in comfort noise is encoded into the first byte of the payload, and spectral information about the noise goes in the bytes after that. This spec's audio level parameter is defined to be identical to the comfort noise payload's noise-level byte. So they match, which is iconic 🦋
4. Signaling (Setup) Information 📣
The URI for declaring this header extension in an extmap attribute is urn:ietf:params:rtp-hdrext:ssrc-audio-level 🎀
It has a single extension attribute, named "vad", and it takes the form "vad=on" or "vad=off". If the header extension element is signaled with "vad=on", the V bit described in Section 3 is in use, and it MUST be set by senders. If it's signaled with "vad=off", the V bit is not in use, and its value MUST be ignored by receivers. If the vad extension attribute isn't specified at all, the default is "vad=on" ✨
Here's an example attribute line in the Session Description Protocol (SDP) for a conference:
a=extmap:6 urn:ietf:params:rtp-hdrext:ssrc-audio-level vad=onThe vad extension attribute only controls the semantics of this header extension. It doesn't say anything about whether the sender uses other voice activity detection features, like discontinuous transmission, comfort noise, or silence suppression. Those are a whole separate convo 💅
Using the mechanisms of [RFC5285], an endpoint MAY signal multiple instances of the header extension element, with different values of the vad attribute, as long as those instances use different values for the extension identifier. But again following the rules of [RFC5285], the semantics chosen for a header extension element (including its vad setting) for a particular extension identifier value MUST NOT be changed within an RTP session. Once you pick it, you're locked in for that session 🔒
Part 1 of 2 is done, bestie! 🌈💖
5. Considerations on Use 💅
okay so bestie, mixers and forwarders generally ought not base their forwarding choices directly on packet-by-packet audio level info 🙈 instead they ought to look at the audio levels and the trends over time. this applies whether the levels come from endpoints (like the ones this doc is about) or get calculated at a server because nobody sent them in the first place ✨ this section is design guidance only and is not normative, so no pressure 🫶
first of all, audio levels generally ought to be measured over longer stretches than one single audio packet 🎀 otherwise you get false positives from little bursts of sound, like a cough or someone bumping their microphone 😭 so it's often useful to require that a participant's level stays up for some period of time before you treat it as "real." basically some kind of low-pass filter ought to be applied to the audio levels 🌸 but be careful, because that filtering has to be balanced against not clipping the start of someone's speech 💀
also, different participants may have their audio input set up differently 🦋 so it may be useful to apply some sort of automatic gain control to the audio levels 💖 there are a bunch of possible approaches, for example measuring peak audio levels, measuring average audio levels during speech, or measuring background audio levels (average audio levels during non-speech) 🌈
6. Security Considerations 🔐
⚠️ this one is serious, bestie. read it carefully.
a malicious endpoint could choose to set the values in this header extension falsely, so that it claims audio or voice is or is not present 😈 it's not clear what anyone would gain by falsely claiming that audio is not present 🤔 but an endpoint falsely claiming audio is present, or falsely exaggerating its reported levels, could perform a denial-of-service attack on an audio conference 📵 it could send silence to suppress other members' audio, or it could dominate the conference by seizing its speaker-selection algorithm 👑 so, thus, if a device relies on audio level data from untrusted endpoints, it SHOULD periodically audit the level information transmitted, and take appropriate corrective action against endpoints that appear to be sending incorrect data 🫠 (however, since it's valid for an endpoint to measure audio levels before encoding, some degree of discrepancy could be present, and that alone would not mean an endpoint is malicious 🧚)
in the Secure Real-time Transport Protocol (SRTP) [RFC3711], RTP header extensions are authenticated but not encrypted 😱 so when this header extension is used, the audio levels are visible on a packet-by-packet basis to an attacker who is passively observing the audio stream 👀 as discussed in [SRTP-VBR-AUDIO], such an attacker might be able to infer information about the conversation, possibly with phoneme-level resolution 💀 in scenarios where this is a concern, additional mechanisms MUST be used to protect the confidentiality of the header extension 🔒 that mechanism could be header extension encryption [SRTP-ENCR-HDR], or a lower-level security and authentication mechanism such as IPsec [RFC4301] 🛡️
7. IANA Considerations 📋
this document defines a new extension URI in the RTP Compact Header Extensions subregistry of the Real-Time Transport Protocol (RTP) Parameters registry, using the following data 🎀
Extension URI: urn:ietf:params:rtp-hdrext:ssrc-audio-level
Description: Audio Level
Contact: jonathan@vidyo.com
Reference: RFC 64648. References 📚
8.1. Normative References 💜
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC2198] Perkins, C., Kouvelas, I., Hodson, O., Hardman, V.,
Handley, M., Bolot, J., Vega-Garcia, A., and S. Fosse-
Parisis, "RTP Payload for Redundant Audio Data", RFC 2198,
September 1997.
[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, RFC 3550, July 2003.
[RFC5285] Singer, D. and H. Desineni, "A General Mechanism for RTP
Header Extensions", RFC 5285, July 2008.8.2. Informative References 🌷
[ITU.G711] International Telecommunication Union, "Pulse Code
Modulation (PCM) of Voice Frequencies",
ITU-T Recommendation G.711, November 1988.
[RFC3389] Zopf, R., "Real-time Transport Protocol (RTP) Payload for
Comfort Noise (CN)", RFC 3389, September 2002.
[RFC3711] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K.
Norrman, "The Secure Real-time Transport Protocol (SRTP)",
RFC 3711, March 2004.
[RFC4301] Kent, S. and K. Seo, "Security Architecture for the
Internet Protocol", RFC 4301, December 2005.
[RFC6465] Ivov, E., Ed., Marocco, E., Ed., and J. Lennox,
"A Real-time Transport Protocol (RTP) Header Extension for
Mixer-to-Client Audio Level Indication", RFC 6465,
December 2011.
[SRTP-ENCR-HDR]
Lennox, J., "Encryption of Header Extensions in the Secure
Real-Time Transport Protocol (SRTP)", Work in Progress,
October 2011.
[SRTP-VBR-AUDIO]
Perkins, C. and JM. Valin, "Guidelines for the use of
Variable Bit Rate Audio with Secure RTP", Work
in Progress, July 2011.Authors' Addresses 🦋
Jonathan Lennox (editor)
Vidyo, Inc.
433 Hackensack Avenue
Seventh Floor
Hackensack, NJ 07601
US
EMail: jonathan@vidyo.com
Emil Ivov
Jitsi
Strasbourg 67000
France
EMail: emcho@jitsi.org
Enrico Marocco
Telecom Italia
Via G. Reiss Romoli, 274
Turin 10148
Italy
EMail: enrico.marocco@telecomitalia.it📦 where this page comes from
- ietf: rfc6464 2011-12
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms