RFC 6871

Session Description Protocol (SDP) Media Capabilities Negotiation

📅 published February 2013, from ietf

RFC 6871: Session Description Protocol (SDP) Media Capabilities Negotiation 🎀

Okay bestie, let's talk about this one! 💖 It's RFC 6871, a Standards Track doc from February 2013 (ISSN 2070-1721). The authors are R. Gilman (Independent), R. Even (Huawei Technologies), and F. Andreasen (Cisco Systems). It updates RFC 5939, which is the base SDP capability negotiation spec, and the header doesn't list anything it obsoletes. Super important context for everything that follows! ✨

Abstract

okay so SDP capability negotiation gives you a general framework for signalling and negotiating capabilities in SDP 🦋 The base framework only covers capabilities for transport protocols and attributes. This document extends that framework by adding media capabilities, so you can negotiate media types and their parameters. 💅

This document also updates the IANA Considerations of RFC 5939. 🍓

Status of This Memo

This is an Internet Standards Track document. 🌈

This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community, received public review, and was approved for publication by the Internet Engineering Steering Group (IESG). If you want more on Internet Standards, check Section 2 of RFC 5741. 🧚

Info about the document's current status, any errata, and how to send feedback lives at http://www.rfc-editor.org/info/rfc6871. ⭐

Copyright Notice

This document is covered by BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they stood on the publication date. Please read those carefully, because they spell out your rights and restrictions for this document. Code components pulled out of this document have to include the Simplified BSD License text described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as the Simplified BSD License describes. 💀

This document may include material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The people who control the copyright in some of that material may not have given the IETF Trust permission to allow modifications outside the IETF Standards Process. So unless you get an adequate license from those copyright holders, this document can't be modified outside the IETF Standards Process, and derivative works can't be made outside it either. The one exceptions are formatting it for publication as an RFC or translating it into languages other than English. 🙈

   1. Introduction ....................................................4
   2. Terminology .....................................................4
   3. SDP Media Capabilities ..........................................6
      3.1. Requirements ...............................................6
      3.2. Solution Overview ..........................................7
      3.3. New Capability Attributes .................................13
           3.3.1. The Media Format Capability Attributes .............13
           3.3.2. The Media Format Parameter Capability Attribute ....16
           3.3.3. The Media-Specific Capability Attribute ............19
           3.3.4. New Configuration Parameters .......................21
           3.3.5. The Latent Configuration Attribute .................23
           3.3.6. Enhanced Potential Configuration Attribute .........25
           3.3.7. Substitution of Media Payload Type Numbers
                  in Capability ......................................29
           3.3.8. The Session Capability Attribute ...................30
      3.4. Offer/Answer Model Extensions .............................35
           3.4.1. Generating the Initial Offer .......................35
           3.4.2. Generating the Answer ..............................39
           3.4.3. Offerer Processing of the Answer ...................43
           3.4.4. Modifying the Session ..............................43
   4. Examples .......................................................44
      4.1. Alternative Codecs ........................................44
      4.2. Alternative Combinations of Codecs (Session
           Configurations) ...........................................47
      4.3. Latent Media Streams ......................................47
   5. IANA Considerations ............................................49
      5.1. New SDP Attributes ........................................49
      5.2. New SDP Capability Negotiation Option Tag .................50
      5.3. SDP Capability Negotiation Configuration
           Parameters Registry .......................................50
      5.4. SDP Capability Negotiation Configuration Parameter
           Registrations .............................................52
   6. Security Considerations ........................................52
   7. Acknowledgements ...............................................53
   8. References .....................................................54
      8.1. Normative References ......................................54
      8.2. Informative References ....................................54

1. Introduction ✨

"Session Description Protocol (SDP) Capability Negotiation" [RFC5939] gives you a general framework for signalling and negotiating capabilities in SDP [RFC4566]. The base framework only covers capabilities for transport protocols and attributes. 🎀

RFC 5939 [RFC5939] lists some of the issues with the current SDP capability negotiation process. There's also a real-life problem: you want to offer one media stream (like audio) while listing that you can support another one (like video), but without actually offering it at the same time. 😭

So in this document we extend the framework by defining media capabilities that can be used to indicate and negotiate media types and their associated format parameters. We also add the ability to declare support for media streams that can be offered and negotiated later, plus the ability to specify session configurations as combinations of media stream configurations. 💅 The new attributes for media capability negotiation are designed so that translating them into "conventional" SDP [RFC4566] media attributes is as straightforward as possible, which keeps implementation simple. This is meant to cut processing in two ways. First, each proposed configuration in an offer can easily be translated into a conventional SDP media stream record for the receiver to process. Second, building an answer from a selected proposed configuration is straightforward. 🦋

This document updates RFC 5939 [RFC5939] by updating its IANA considerations. All the other extensions defined here are considered extensions above and beyond RFC 5939 [RFC5939]. 🌸

2. Terminology 🧚

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119] and indicate requirement levels for compliant implementations. Those are the requirement-level words, so pay attention to them! 👑

Actual Configuration: An actual configuration specifies which combinations of SDP session parameters and media stream components can be used in the current offer/answer exchange, and with what parameters. Using an actual configuration doesn't need any further negotiation in the offer/answer exchange. See RFC 5939 [RFC5939] for more details. ✨

Base Attributes: Conventional SDP attributes that show up in the base configuration of a media block. 🍓

Base Configuration: The media configuration represented by a media block, leaving out all the capability negotiation attributes defined in this document, in the base capability negotiation document [RFC5939], or in any other capability negotiation document. In an offer SDP, the base configuration corresponds to the actual configuration as defined in RFC 5939 [RFC5939]. 🫶

Conventional Attribute: Any SDP attribute other than the ones defined by the series of capability negotiation specifications. 🌈

Conventional SDP: An SDP record with no capability negotiation attributes at all. 💜

Media Format Capability: A media format, usually a media subtype such as PCMU, H263-1998, or T38, expressed in the form of a capability. 🎀

Media Format Parameter Capability: A media format parameter (a=fmtp in conventional SDP) expressed in the form of a capability. It's associated with a media format capability. 🦋

Media Capability: The combined set of capabilities used to express a media format and its relevant parameters, such as media format parameters and media-specific parameters. 💅

Potential Configuration: A potential configuration indicates which combinations of capabilities can be used for the session and its associated media stream components. Potential configurations aren't ready for use yet. They are, however, offered for potential use in the current offer/answer exchange, as an alternative that may be used instead of the actual configuration, subject to negotiation in that exchange. See RFC 5939 [RFC5939] for more details. 🌸

Latent Configuration: A latent configuration indicates which combinations of capabilities could be used in a future negotiation for the session and its associated media stream components. Latent configurations are neither ready for use nor offered for actual or potential use in the current offer/answer exchange. They just tell the other side about configurations the entity supports. Those latent configurations can guide later offer/answer exchanges, but they aren't offered as part of the current one. 🧚⭐

3. SDP Media Capabilities 👑

The SDP capability negotiation [RFC5939] discusses using any SDP [RFC4566] attribute (a=) under the attribute capability acap. The limits of using acap for fmtp and rtpmap in a potential configuration are described in RFC 5939 [RFC5939]. For example, they can only be used at the media level, since they're media-level attributes. RFC 5939 [RFC5939] doesn't give you a way to exchange media-level capabilities before the actual offer of the associated media stream, and that's the gap this section tackles. 🙈 It gives an overview of the extensions that provide a more robust SDP media capability negotiation solution. After that, it defines the new SDP attributes for the solution and the updated offer/answer procedures [RFC3264] that go with them. 💖

3.1. Requirements

okay bestie, before we build anything we need a checklist 📝 these are the requirements the capability negotiation extensions have to hit. there are also a few "maybe later" ones at the end 👀

REQ-01: be able to list alternative media formats (codecs), or combinations of them, in a single media block 🎀
REQ-02: each media format can carry alternative format parameters 🧚
REQ-03: stay backward compatible with conventional SDP. every offered configuration has to translate easily into a matching SDP media block written with normal SDP lines 💅
REQ-04: work within the offer/answer model so media formats and parameters can be agreed on in a single exchange 🔥
REQ-05: let the offerer receive media as soon as the offer is sent. (ngl there's a note here: the offerer might not be able to render that media until keying material is exchanged 🙈)
REQ-06: let you offer latent media configurations for future negotiation 🦋
REQ-07: be reasonably efficient when expressing alternative media formats and/or format parameters, especially when lots of combinations are offered 💜
REQ-08: keep the extensibility of the base capability negotiation mechanism 🌈
REQ-09: let you specify acceptable combinations of media streams and media formats. for example, offer a PCMU audio stream with an H264 video stream, or a G729 audio stream with an H263 video stream. this lets the offerer limit processing requirements for simultaneous streams. it also lets an offer include an audio/T38 stream or an image/T38 stream, but not both ✨

other extensions have been discussed but aren't covered in this document. they might show up in the future. three of them are:

FUT-01: mix or change media types within a single media block. conventional SDP doesn't support this explicitly, so the usual trick is to define a media subtype that represents the actual format inside the nominal media type. for example, T.38 FAX as an alternative to audio/PCMU within an audio stream is identified as audio/T38, and a separate FAX stream would use image/T38 📠
FUT-02: support multiple transport protocols within an active media stream without reconfiguration. conventional SDP doesn't explicitly support this either 🚀
FUT-03: provide capability negotiation attributes for all media-level SDP line types, the same way it's already done for the attribute type. the exception is the media line type itself, which gets special treatment so codec/format options can be expressed compactly. the line types involved are bandwidth (b=), information (i=), connection data (c=) and, possibly, the deprecated encryption key (k=) 🫶

3.2. Solution Overview

okay this is where the magic happens ✨ the solution is made of new capability attributes that line up with conventional SDP line types, new parameters for the "pcfg", "acfg", and the new "lcfg" attributes (which extend the base attributes from RFC 5939 [RFC5939]), and a use of the "pcfg" attribute to send capability info back in the SDP answer.

a few new attributes are defined so they can be related to the capabilities in a media line and its matching "rtpmap" and "fmtp" attributes:

a=rmcap defines RTP-based media format capabilities as a media subtype (like "PCMU") plus its encoding parameters (like "/8000/2"). each resulting type/subtype capability gets a handle called a media capability number. the encoding parameters are the same ones specified for the "rtpmap" attribute in SDP [RFC4566], just without the payload type number part 💖
a=omcap defines other (non-RTP-based) media format capabilities as a media subtype only (like "T38"). each resulting type/subtype capability also gets a media capability number 🌸
a=mfcap specifies media format parameters tied to one or more media format capabilities. it's mainly used to attach the parameters normally carried in the "fmtp" attribute. media format parameters can be used with both RTP and non-RTP-based media formats 🧚
a=mscap specifies media parameters tied to one or more media format capabilities. it's used for attributes other than "fmtp" or "rtpmap", like the "rtcp-fb" attribute defined in RFC 4585 [RFC4585] 🍓
a=lcfg specifies latent media stream configurations when no matching media line ("m=") is offered. an example is offering latent video configs even though no video is on the table right now. if the peer says it supports one or more of those latent configs, the matching media stream(s) can be added through a new offer/answer exchange 🦋
a=sescap specifies an acceptable combination of simultaneous media streams and their configurations, as a list of potential and/or latent configurations 👑

new parameters are defined for the potential configuration ("pcfg"), latent configuration ("lcfg"), and accepted configuration ("acfg") attributes so they can link up with the new attributes:

a new parameter type (m=) is added to the potential configuration (a=pcfg:) attribute and the actual configuration (a=acfg:) attribute from RFC 5939 [RFC5939], and to the new latent configuration (a=lcfg:) attribute. this lets you specify media capabilities (with their parameters) and combinations of them for the configuration. for example, the a=pcfg: line might say PCMU plus telephone events [RFC4733], or G.729B plus telephone events, as acceptable configurations. the a=acfg: line in the answer says which configuration got chosen 💅
a new parameter type (pt=) is added to the potential configuration, actual configuration, and latent configuration attributes. it links RTP payload type numbers to the referenced RTP-based media format capabilities, and it only makes sense when the transport protocol uses RTP 🎀
a new parameter type (mt=) is used to specify the media type for latent configurations 🌈

there are also special processing rules for capability attribute arguments, so you don't have to copy basically identical attribute lines for the base configuration and the potential configurations (huge time saver, no cap 😭):

a substitution rule applies to any capability attribute. it lets you replace the (escaped) media capability number with the media format identifier, like the payload type number in audio/video profiles 🔥
replacement rules are defined for the conventional SDP equivalents of the a=mfcap and a=mscap capability attributes. this cuts down on needing the deletion qualifier in the a=pcfg parameter just to ignore "rtpmap", "fmtp", and certain other attributes in the base configuration 💜
an argument concatenation rule is defined for a=mfcap attributes that refer to the same media capability number. this makes it easy to combine format options concisely by tying multiple mfcap lines to multiple media format capabilities 🫶

this document extends the base protocol so the offer/answer model can handle capabilities and potential configurations inside an offer. media capabilities are the things that can be used in potential and latent configurations. potential configurations are alternative offers the answerer may accept instead of the actual configuration(s) in the "m=" line(s) and their parameters. latent configurations just tell the other side about configurations the entity might support. those can guide later offer/answer exchanges, but they aren't part of the current one 🧚

here's the mechanism in action, with Alice sending an offer to Bob:

               Alice                            Bob
              | (1) Offer (SRTP and RTP)         |
              |--------------------------------->|
              |                                  |
              | (2) Answer (RTP)                 |
              |<---------------------------------|
              |                                  |

Alice's offer includes RTP and Secure Real-time Transport Protocol (SRTP) as alternatives. RTP is the default, but SRTP is the one she prefers (long lines are folded to fit the margins):

v=0
o=- 25678 753849 IN IP4 192.0.2.1
s=
c=IN IP4 192.0.2.1
t=0 0
a=creq:med-v0
m=audio 3456 RTP/AVP 0 18
a=tcap:1 RTP/SAVP RTP/AVP
a=rtpmap:0 PCMU/8000/1
a=rtpmap:18 G729/8000/1
a=fmtp:18 annexb=yes
a=rmcap:1,4 G729/8000/1
a=rmcap:2 PCMU/8000/1
a=rmcap:5 telephone-event/8000
a=mfcap:1 annexb=no
a=mfcap:4 annexb=yes
a=mfcap:5 0-11
a=acap:1 crypto:1 AES_CM_128_HMAC_SHA1_32 \
inline:NzB4d1BINUAvLEw6UzF3WSJ+PSdFcGdUJShpX1Zj|2^20|1:32
a=pcfg:1 m=4,5|1,5 t=1 a=1 pt=1:100,4:101,5:102
a=pcfg:2 m=2 t=1 a=1 pt=2:103
a=pcfg:3 m=4 t=2 pt=4:18

the required base and extensions come from the a=creq attribute in RFC 5939 [RFC5939], with the option tag "med-v0", which says the extension framework defined here has to be supported. the base-level capability negotiation support ("cap-v0" [RFC5939]) is implied, since the extensions need it 🌸

the "m=" line says Alice is offering plain RTP with PCMU or G.729B. the media line implicitly sets the default transport protocol (RTP/AVP in this case) and the default actual configuration 🎀

the a=tcap:1 line, from the SDP capability negotiation base protocol [RFC5939], defines transport protocol capabilities. here that's Secure RTP (SAVP profile) as the first option and RTP (AVP profile) as the second 💖

the a=rmcap:1,4 line defines two G.729 RTP-based media format capabilities, numbered 1 and 4, along with their encoding rate. both are media type "audio" and subtype G729. notice the subtype is spelled out here instead of RTP payload type numbers. that leaves room to assign payload type numbers in the media stream configuration spec. since two G.729 subtype capabilities are defined, you can declare two sets of formatting parameters for G.729 🧚

the a=rmcap:2 line defines a G.711 mu-law capability, numbered 2 🦋

the a=rmcap:5 line defines an audio telephone-event capability, numbered 5 🌈

The a=mfcap:1 line is giving "I will NOT take G.729 Annex B packets" energy 💅 It sets the fmtp formatting parameters for capability 1, so the offerer will not accept G.729 Annex B packets. 🙅‍♀️

The a=mfcap:4 line is the opposite 🫶 It sets the fmtp formatting parameters for capability 4, so the offerer will accept G.729 Annex B packets. ✨

The a=mfcap:5 line handles DTMF 🎀 It sets the fmtp formatting parameters for capability 5, which is the dual-tone multi-frequency (DTMF) touchtones 0-9,*,#. Those are the little keypad beeps, bestie 📞🌸

The a=acap:1 line comes from the base protocol, and it's the one with the crypto attribute 🔐 That's where the keying material for SRTP lives, using SDP security descriptions. Very secure, very main character 👑

The a=pcfg: attributes are the potential configurations the offer puts on the table 🍓 Each one points back to the media capabilities, transport capabilities, attribute capabilities, and payload type number mappings. There are three explicit alternatives, and the lowest-numbered one is the preferred one. Preference is everything, periodt 💖

The a=pcfg:1 ... line is the big one 🧚 It covers media capabilities 4 and 5, which is G.729B and DTMF (with their media format parameters), or media capabilities 1 and 5, which is G.729 and DTMF (with their media format parameters). It also specifies transport protocol capability 1, which is the RTP/SAVP profile (secure RTP) 🔒 It names attribute capability 1, which is the crypto attribute. And last, it gives a payload type number mapping for the RTP-based media capabilities 1, 4, and 5. That mapping lets the offerer tell encrypted media apart from unencrypted media it receives before the answer shows up. Iconic, honestly 🦋

Using unique payload type numbers in the alternative configurations is not required 🌈 Codecs like Adaptive Multi-Rate Wideband (AMR-WB) [RFC4867] have so many possible combinations of options that defining a unique payload type number for every supported combination can be impractical. If unique payload type numbers can't be specified, the offerer is obliged to wait for the SDP answer before rendering received media 😭 For SRTP using Security Descriptions (SDES) inline keying [RFC4568], the offerer still needs to receive the answer before it can decrypt the stream. Not me waiting for the answer 💀

The second alternative, a=pcfg:2 ..., specifies media capability 2, which is PCMU, under the RTP/SAVP profile, using the same SRTP key material 🎶

The third alternative, a=pcfg:3 ..., offers G.729B unsecured. Its only job in this example is to show a preference for G.729B over PCMU ⭐

Per RFC 5939 [RFC5939], the media line, along with any qualifying attributes like fmtp or rtpmap, counts as a valid configuration in its own right. That's the current actual configuration, and it has the lowest preference (again, per RFC 5939 [RFC5939]) 🥺

Now Bob gets the SDP offer from Alice ✨ Bob supports G.729B, PCMU, and telephone events over RTP, but not SRTP. So he accepts potential configuration 3 for RTP, which Alice provided. Bob generates this answer:

v=0
o=- 24351 621814 IN IP4 192.0.2.2
s=
c=IN IP4 192.0.2.2
t=0 0
a=csup:med-v0
m=audio 4567 RTP/AVP 18
a=rtpmap:18 G729/8000
a=fmtp:18 annexb=yes
a=acfg:3 m=4 t=2 pt=4:18

Bob includes the a=csup and a=acfg attributes so Alice knows he can handle the med-v0 level of capability negotiations 🫶 Note that in this example, the answerer supports the capability extensions defined here. Had he not, he would have simply processed the offer based on the offered PCMU and G.729 codecs under the RTP/AVP profile only. In that case, the answer would have left out the a=csup attribute line and picked one or both of PCMU and G.729 instead 🙈 The answer carries the accepted configuration in the m= line, along with the matching rtpmap and/or fmtp parameters, as appropriate.

Note that per the base protocol, after all of this, Alice MAY generate a new offer with an actual configuration (m= line, etc.) that matches the actual configuration referenced in Bob's answer. That isn't shown here 💜

3.3. New Capability Attributes ✨

In this section we introduce the new attributes used to indicate media capabilities for SDP capability negotiation 🎀 The vibe is to keep things similar to the existing media capabilities, which are defined by the existing media descriptions (m= lines) and their rtpmap and fmtp attributes. We use media subtypes and "media capability numbers" to link the relevant media capability parameters together. This lets capabilities be defined at the session level and reused across multiple streams, if you want that 🦋 For RTP-based media formats, payload types are specified at the media level (see Section 3.3.4.2).

A media capability just indicates possible support for a media type and its media format(s) and parameters. To actually use a media capability in an offer/answer exchange, it MUST be referenced in a potential configuration. No referencing, no vibes 🙅‍♀️

Media capabilities, meaning the attributes used to express media capability formats, parameters, and so on, can be provided at the session level and/or the media level. Media capabilities provided at the session level may be referenced in any pcfg or lcfg attribute at the media level (as long as it matches the media type). Media capabilities provided at the media level may only be referenced by the pcfg or lcfg attribute within that same media stream 🧚 Either way, the scope of the <med-cap-num> is the entire session description. That makes every media capability uniquely referenceable across the whole session description, like in a potential configuration 💖

3.3.1. The Media Format Capability Attributes 🍓

Media subtypes can be expressed as media format capabilities using the a=rmcap and a=omcap attributes. The a=rmcap attribute MUST be used for RTP-based media, while the a=omcap attribute MUST be used for non-RTP-based (other) media formats. The two attributes look like this:

a=rmcap:<media-cap-num-list> <encoding-name>/<clock-rate>
                             [/<encoding-parms>]

a=omcap:<media-cap-num-list> <format-name>

Here, <media-cap-num-list> is a (list of) media capability number(s) used to number a media format capability. The <encoding name> or <format-name> is the media subtype, like H263-1998, PCMU, or T38. <clock rate> is the encoding rate, and <encoding parms> are the media encoding parameters for the media subtype. All media format capabilities in the list are assigned to the same media type/subtype 🌸

Each occurrence of the rmcap and omcap attribute MUST use unique values in its <media-cap-num-list>. The media capability numbers are shared between the two attributes, and the numbers MUST be unique across the entire SDP session. In short, the rmcap and omcap attributes define media format capabilities and tie them to a media capability number, the same way the rtpmap attribute defines formats and ties them to a payload type number 🔥 The attributes also let you define multiple capability numbers for the same media format by specifying a range of media capability numbers. This lets the media format be associated with different media parameters in different configurations. When you specify a range, the first (leftmost) capability number MUST be strictly smaller than the second (rightmost). In other words, the range increases and covers at least two numbers 👑

In ABNF [RFC5234], we have:

media-capability-line = rtp-mcap / non-rtp-mcap

rtp-mcap           = "a=rmcap:" media-cap-num-list
                        1*WSP encoding-name "/" clock-rate
                        ["/" encoding-parms]
non-rtp-mcap       = "a=omcap:" media-cap-num-list 1*WSP format-name
media-cap-num-list = media-cap-num-element
                     *("," media-cap-num-element)
media-cap-num-element = media-cap-num
                             / media-cap-num-range
media-cap-num-range = media-cap-num "-" media-cap-num
media-cap-num      = NonZeroDigit *9(DIGIT)
encoding-name      = token ;defined in RFC 4566
clock-rate         = NonZeroDigit *9(DIGIT)
encoding-parms     = token
format-name        = token ;defined in RFC 4566
NonZeroDigit       = %x31-39    ; 1-9

The encoding-name, clock-rate, and encoding-params are defined exactly as they'd appear in an rtpmap attribute for each media type/subtype. So it's easy to turn an rmcap attribute line into one or more rtpmap attribute lines, once a payload type number is assigned to a media-cap-num (see Section 3.3.5) 🦋

The format-name is a media format description for non-RTP-based media, as defined for the <fmt> part of the media description (m= line) in SDP [RFC4566]. Simply put, it's the name of the media format, like "t38" 🧚 This form can also be used in cases like Binary Floor Control Protocol (BFCP) [RFC4585], where the fmt list in the m= line is effectively ignored (BFCP uses "*").

The rmcap and omcap attributes can be provided at the session level and/or the media level. You can have more than one rmcap and more than one omcap attribute at both levels, meaning more than one of each at the session level and more than one of each in every media description 💅 Media capability numbers cannot include leading zeroes. Each media-cap-num MUST be unique within the entire SDP record, because it identifies that media capability in potential, latent, and actual configurations, and in other attribute lines as explained below. The media-cap-num values are shared between the rmcap and omcap attributes, so the uniqueness requirement covers the union of both 😭 When media capabilities are used in a potential, latent, or actual configuration, the media formats referred to by those configurations apply at the media level. That's true no matter whether the media capabilities themselves were specified at the session or media level. In other words, the media capability applies to the specific media description associated with the configuration that invokes it 💜

For example:

v=0
o=- 24351 621814 IN IP4 192.0.2.2
s=
c=IN IP4 192.0.2.2
t=0 0
a=rmcap:1 L16/8000/1
a=rmcap:2 L16/16000/2
a=rmcap:3 H263-1998/90000
a=omcap:4 example
m=audio 54320 RTP/AVP 0
a=pcfg:1 m=1|2, pt=1:99,2:98
m=video 66544 RTP/AVP 100
a=rtpmap:100 H264/90000
a=pcfg:10 m=3 pt=3:101
a=tcap:1 TCP
a=pcfg:11 m=4 t=1

3.3.2. The Media Format Parameter Capability Attribute ✨

okay so this attribute is how we attach media-format-specific parameters to one or more media format capabilities 🎀 the form is:

a=mfcap:<media-caps> <list of parameters>

the <media-caps> bit is what lets one parameter list hang off a whole bunch of capabilities at once 🦋 and the format parameters are specific to whatever media format you're dealing with. each mfcap line maps to one traditional SDP "fmtp" attribute line (one per entry in <media-caps>) that looks like this:

a=fmtp:<fmt> <list of parameters>

where <fmt> is the media format parameter defined in RFC 4566 [RFC4566], picked to suit the particular media stream 💅 now the big rule: the "mfcap" attribute MUST be used to encode attributes for media capabilities that would normally show up in an "fmtp" attribute. and the existing "acap" attribute MUST NOT be used to encode "fmtp" attributes 🙅‍♀️ no sneaking fmtp stuff into acap, bestie.

the "mfcap" attribute follows the SDP [RFC4566] attribute production rules, with:

media-format-parameter-capability =
       "a=mfcap:" media-cap-num-list 1*WSP fmt-specific-param-list
fmt-specific-param-list = text ; defined in RFC 4566

and yes, media format parameters work for both RTP-based and non-RTP-based media formats 🌈 they're not picky at all.

3.3.2.1. Media Format Parameter Concatenation Rule 🧚

okay this part is so clever ngl 💜 some media subtypes have a huge pile of formatting options (think AMR-WB [RFC4867]) and the rule is that only a single "fmtp" attribute can show up per media format. so it makes total sense to have a combining rule for "mfcap" parameters that share the same media capability number 🙌

so different mfcap lines MAY include the same media-cap-num in their media-cap-num-list. when a particular media capability gets selected for processing, the parameters from each mfcap line that mentions that capability number in its media-cap-num-list get glued together with ";", in the order the "mfcap" attributes appear in the SDP record. the result is the equivalent of one single "fmtp" attribute line ✨ this lets you define a separate mfcap line for just one parameter and value, and have it applied to every media capability in the media-cap-num-list. it's a super compact way to spell out lots of format parameter combos when your codec has multiple options 🍓

one thing to watch: order-dependent parameters SHOULD be placed in a single mfcap line, so a middlebox doesn't rearrange the lines and mess things up 🙈

format parameters don't get parsed by SDP at all. their content depends on the media type/subtype 💀 when format parameters for a specific media capability come from several "a=mfcap" lines that reference it, they get concatenated and separated by ";" to build the matching format attribute ("a=fmtp"). the final thing will look something like this (imagine it all on one line):

a=fmtp:<fmt> <fmt-specific-param-list1>;
             <fmt-specific-param-list2>;
             ...

where <fmt> depends on the transport protocol, the way RFC 4566 [RFC4566] defines it. SDP can't check whether the final a=fmtp parameter list is legal, so the user has to make sure they're generating legal parameter lists 🫶

the "mfcap" attribute can show up at the session level AND the media level, and you can have more than one of them at either level. the unique media-cap-num is what links the parameters to a media capability 👑

here's a simple example. a G.729 capability is considered by default to support comfort noise as defined in Annex B. so G.729 with and without comfort noise can be defined like this:

a=rmcap:1,2 G729/8000
a=mfcap:2 annexb:no

media capability 1 supports G.729 with Annex B, and media capability 2 supports G.729 without Annex B. iconic 🦋

example for H.263 video:

a=rmcap:1 H263-1998/90000
a=rmcap:2 H263-2000/90000
a=mfcap:1 CIF=4;QCIF=2;F=1;K=1
a=mfcap:2 profile=2;level=2.2

and finally, six format combos of the Adaptive Multi-Rate codec 😭✨

a=rmcap:1-3 AMR/8000/1
a=rmcap:4-6 AMR-WB/16000/1
a=mfcap:1,2,3,4 mode-change-capability=1
a=mfcap:5,6 mode-change-capability=2
a=mfcap:1,2,3,5 max-red=220
a=mfcap:3,4,5,6 octet-align=1
a=mfcap:1,3,5 mode-set=0,2,4,7
a=mfcap:2,4,6 mode-set=0,3,5,6

so when AMR codec #1 gets specified in a "pcfg" attribute within an audio stream block (and assigned payload type number 98), like this:

a=pcfg:1 m=1 pt=1:98

it's basically the same as this:

m=audio 49170 RTP/AVP 98
a=rtpmap:98 AMR/8000/1
a=fmtp:98 mode-change-capability=1; \
max-red=220; mode-set=0,2,4,7

and AMR codec #4 with payload type number 99, shown by the potential configuration:

a=pcfg:4 m=4, pt=4:99

is equivalent to:

m=audio 49170 RTP/AVP 99
a=rtpmap:99 AMR-WB/16000/1
a=fmtp:99 mode-change-capability=1; octet-align=1; \
mode-set=0,3,5,6

and so on for the other four combinations 🌸 SDP can turn these media capability specs into one or more alternative media stream specs, and one of them can get picked for the answer. main character energy, honestly 👑

3.3.3. The Media-Specific Capability Attribute 🦋

okay so attributes and parameters tied to a media format usually get written with the "rtpmap" and "fmtp" attributes in SDP, plus the similar "rmcap" and "mfcap" attributes in SDP media capabilities 💅 but some SDP extensions define other attributes that also need to link up with media formats. the "rtcp-fb" attribute from RFC 4585 [RFC4585] is one example. these media-specific attributes (beyond "rtpmap" and "fmtp") can be tied to media capability numbers through a brand new media-specific attribute called "mscap", which looks like this:

a=mscap:<media caps star> <att field> <att value>

where <media caps star> is a (list of) media capability number(s), <att field> is the attribute name, and <att value> is the value for that named attribute 🎀 the media capability numbers point at media format capabilities defined elsewhere in the SDP ("rmcap" and/or "omcap"). if you specify a range of capability numbers, the first (leftmost) number MUST be strictly smaller than the second (rightmost) 🙅‍♀️ no backwards ranges, periodt.

the media capability numbers can include a wildcard ("*"), which gets used instead of any payload type mappings in the resulting SDP (see RFC 4585 [RFC4585] and the example below for the vibe) ✨

in ABNF, it looks like this:

media-specific-capability = "a=mscap:"
                             media-caps-star
                             1*WSP att-field ; from RFC 4566
                             1*WSP att-value ; from RFC 4566
media-caps-star           =  media-cap-star-element
                             *("," media-cap-star-element)
media-cap-star-element    = (media-cap-num [wildcard])
                          / (media-cap-num-range [wildcard])
wildcard                  = "*"

now, if you've got an association between a media capability and a payload type number (set by the "pt=" parameters in a "pcfg" attribute line), a mscap line can be translated pretty easily into a normal SDP attribute line of this form:

a=<att field>":"<fmt> <att value> ; <fmt> defined in SDP [RFC4566]

a resulting attribute that isn't a legal SDP attribute (as defined by RFC 4566) MUST be ignored by the receiver 🫶 no drama, just skip it.

if a media capability number (or range) has a wildcard character at the end, any payload type mapping set for that media-specific capability (or range of capabilities) will use the wildcard character in the resulting SDP, instead of the payload type from the "pt" parameter in the configuration attribute 🧚

a single mscap line can refer to several media capabilities using a capability number range. that's the same as writing multiple mscap lines, each with the same attribute values but a different media capability number, one line per capability 🌈

multiple mscap lines can point at the same media capability. but unlike the "mfcap" attribute, there's no concatenation operation defined for them. so multiple mscap lines applied to the same media capability are the same as multiple lines of that attribute in a normal media record 💖

here's an example with the "rtcp-fb" attribute, adapted from one in RFC 5104 [RFC5104] (session level and audio media left out). if the offer has a media block like this (look at that wildcard 👀):

m=video 51372 RTP/AVP 98
a=rtpmap:98 H263-1998/90000
a=tcap:1 RTP/AVPF
a=rmcap:1 H263-1998/90000
a=mscap:1 rtcp-fb ccm tstr
a=mscap:1 rtcp-fb ccm fir
a=mscap:1* rtcp-fb ccm tmmbr smaxpr=120
a=pcfg:1 t=1 m=1 pt=1:98

and the proposed configuration gets chosen, the equivalent media block ends up looking like this:

m=video 51372 RTPAVPF 98

wait, that's not right, I'm correcting it now. the correct equivalent is:

m=video 51372 RTP/AVPF 98
a=rtpmap:98 H263-1998/90000
a=rtcp-fb:98 ccm tstr
a=rtcp-fb:98 ccm fir
a=rtcp-fb:* ccm tmmbr smaxpr=120

3.3.4. New Configuration Parameters 💜

along with the new attributes for media capabilities, there are new extension parameters for use in the potential configuration, the actual configuration, and/or the new latent configuration defined in Section 3.3.5 🌸 the vibe is a whole ecosystem, bestie.

3.3.4.1. The Media Configuration Parameter (m=) 🍓

the media configuration parameter is used to specify the media format(s) and related parameters for a potential, actual, or latent configuration. it follows the ABNF for extension-config-list in RFC 5939 [RFC5939], with:

ext-cap-name = "m"
ext-cap-list = media-cap-num-list
               [*(BAR media-cap-num-list)]

so we get:

media-config-list = ["+"] "m=" media-cap-num-list
                               *(BAR media-cap-num-list)
                     ;BAR is defined in RFC 5939
                     ;media-cap-num-list is defined above

alternative media configurations are separated by a vertical bar ("|"). they're ordered by preference, most-preferred first 🥇 when media capabilities aren't included in a potential configuration at the media level, the media type and media format from the associated "m=" line get used instead. and the plus sign ("+") usage is described in RFC 5939 ✨

3.3.4.2. The Payload Type Number Mapping Parameter (pt=) ✨

okay so the payload type number mapping parameter is how you say which payload type number goes with each RTP-based media format 🎀. It works for potential, actual, and latent configurations 💅. We define payload-number-config-list to follow the extension-config-list format from [RFC5939]. Here's the ABNF:

payload-number-config-list = ["+"] "pt=" media-map-list
media-map-list      = media-map *("," media-map)
media-map           = media-cap-num ":" payload-type-number
                         ; media-cap-num is defined in Section 3.3.1
payload-type-number = NonZeroDigit *2(DIGIT) ; RTP payload
                                             ; type number

The example in Section 3.3.7 shows how the parameters from the rmcap line get mapped to payload type numbers in the "pcfg" "pt" parameter 🍓. The plus sign ("+") is explained in [RFC5939] 🦋.

A latent configuration is about a future capability 🔮, so "pt=" doesn't really mean anything in the "lcfg" attribute, because no actual media session is being offered or accepted. It's still allowed, though, so payload type number parameters inside attributes can be tied to the right media format 🧚. The main example is Redundant Audio Data (RED) [RFC2198], whose format parameters are payload type numbers 🎧. Payload type numbers used in a latent configuration MAY be read as suggestions for any future offer based on that latent configuration, but they're not binding 🙈. The offerer and/or answerer can use whatever payload type numbers each of them thinks are right. You can skip explicit payload type numbers in latent configurations by using the parameter substitution rule from Section 3.3.7 ✨. Future extensions are also allowed. And heads up: leading zeroes are not permitted 🙅‍♀️.

3.3.4.3. The Media Type Parameter 🎬

When a latent configuration is specified (always at the media level) to show it can support an extra media stream, you need to say the media type (audio, video, etc.), plus the format and transport type 🫶. The media type parameter is defined in ABNF like this:

media-type = ["+"] "mt=" media; media defined in RFC 4566

Right now, the media-type parameter is only used in the latent configuration attribute. So using the "+" prefix to say "ignore the whole attribute line if you don't understand mt=" isn't needed yet 🤷‍♀️. But if media-type ever gets added to an existing capability attribute like "pcfg", the "+" would be super useful 👑. The media format(s) go in the media configuration parameter ("+m=") defined above, and the transport type(s) go in the transport parameter ("t=") defined in [RFC5939] 🌈.

3.3.5. The Latent Configuration Attribute 🦋

One big goal here is letting parties swap supportable media configurations, not just the ones being offered or accepted right now 💖. These are called "latent configurations". For example, someone can offer a session with an audio stream and also announce that they could support a video stream in that same session 🎀. The offerer lists its video capabilities as one or more latent video configurations along with the audio stream. The responder can show it can and wants to do video too by sending back a matching latent configuration 🫶.

Latent configurations in SDP answers MUST match the offered latent configurations (or parameter subsets of them) 😭. So it makes sense for the offerer to announce most, if not all, of its capabilities in the initial offer ✨. That keeps the answer compact, since you don't need acap, rmcap, tcap, etc. lines in the answer 💅.

Latent configurations can be announced with the latent configuration attribute, which looks a lot like the potential configuration attribute 💜. It's basically a media line and a potential configuration rolled into one 🔥. A latent configuration MUST include a media type (mt=) and a transport protocol configuration parameter, since it doesn't depend on whatever media line is there 🌸. In most cases, the media configuration (m=) parameter needs to show up too (see Section 4 for examples) 🧚. The "lcfg" attribute is a media-level attribute.

The "lcfg" attribute is defined as a media-level attribute because it describes a possible future media stream 🔮. Even so, "lcfg" isn't necessarily tied to the media description it appears in. Session capability attributes ("a=sescap") can be used to show which media stream configurations are supported 🌈.

Each media line in an SDP description is an offered simultaneous media stream 🎧. Each latent configuration is an extra stream that might get negotiated in a future offer/answer exchange 🚀. Session capability attributes can help decide whether a latent configuration can be used to make an offer for an additional simultaneous stream, or to reconfigure an existing stream in a later offer/answer exchange 🍓.

The latent configuration attribute looks like this:

a=lcfg:<config-number> <latent-cfg-list>

It follows the SDP [RFC4566] "attribute" production, with att-field and att-value defined as:

att-field  = "lcfg"
att-value  = config-number 1*WSP lcfg-cfg-list
config-number = NonZeroDigit *9(DIGIT)  ;DIGIT defined in RFC 5234
lcfg-cfg-list = media-type 1*WSP pot-cfg-list
                            ; as defined in RFC 5939
                            ; and extended herein

The media-type (mt=) parameter says what kind of media (audio, video, etc.) goes with the latent stream, and it MUST be present 🎀. The pot-cfg-list MUST contain a transport-protocol-config-list (t=) parameter and a media-config-list (m=) parameter 🫶. The pot-cfg-list MUST NOT contain more than one instance of each type of parameter list 🙅‍♀️. As [RFC5939] says, putting a "+" in front of a parameter means the whole configuration MUST be ignored if that parameter isn't understood. Otherwise, the parameter itself may just get ignored 💜.

Latent configurations don't assign media stream payload numbers 🧚. Those get assigned when the matching stream is actually offered in an "m=" line in a later exchange 🦋. The payload-number-config-list is included as a parameter to the "lcfg" attribute, in case you need to tie payload numbers in attribute capabilities to specific media capabilities 🍓.

If an "lcfg" attribute invokes an "acap" attribute that shows up at the session level, then that attribute is expected to show up at the session level of a later offer, when and if a matching media stream gets offered 👑. Otherwise, "acap" attributes at the media level are media-level attributes 🌈. But heads up: "rmcap", omcap, "mfcap", "mscap", and "tcap" attributes can appear at the session level, because they always end up as media-level attributes or "m=" line parameters 🎧.

Configuration numbers for latent configurations don't imply a preference 🙈. The offerer shows a preference when it actually offers potential configurations derived from the latent configurations negotiated earlier 💖. That said, the offerer of latent configurations MAY specify preferences for combinations of potential and latent configurations using the "sescap" attribute from Section 3.3.8 ✨. For example, say an SDP offer has an audio stream with "pcfg:1" and two latent video configurations, "lcfg:2" and "lcfg:3". Then a session with one audio stream and one video stream can be specified by including a=sescap:1 1,2|3 🎬. And one audio stream with two video streams can be specified by including a=sescap:2 1,2,3 in the offer 🔥. To let latent and potential configurations be combined in session capabilities, latent configuration numbers MUST be different from the numbers used for potential configurations 😭. This matters a lot if the offerer doesn't require cmed-v0 capability and the offer's recipient doesn't support it 💀. If the "lcfg" attribute isn't recognized, the capability attributes meant for it might get mixed up with the ones for a potential configuration of some other media stream 🌸. Also, leading zeroes are not permitted in configuration numbers 🙅‍♀️.

If a cryptographic attribute, like the SDES a=crypto: attribute [RFC4568], is referenced by a latent configuration through an "acap" attribute, then any keying material the conventional attribute needs (like the SDES key/salt string) MUST be included so the attribute meets its formatting rules 🔐. The keying material is visible here but not actually used yet, since it's a latent configuration 🙈. So the value(s) of the keying material MUST NOT be a real value used for real exchange of media 🚫. And the receiver of the "lcfg" attribute MUST ignore those value(s) 🫶.

3.3.6. Enhanced Potential Configuration Attribute 🌈

This work needs new extensions (parameters) for the "pcfg" attribute, which is defined in the SDP capability negotiation base protocol [RFC5939] 💖. The parameters and their definitions are "borrowed" from the latent configuration attribute definitions in Section 3.3.5 🦋. The expanded ABNF for the "pcfg" attribute is:

a=pcfg: <config-number> [<pot-cfg-list>]

where:

config-number = 1*DIGIT ;defined in [RFC5234]
pot-cfg-list  = pot-config *(1*WSP pot-config)
pot-config    =  attribute-config-list / ;def in [RFC5939]
     transport-protocol-config-list / ;defined in [RFC5939]
     extension-config-list / ;[RFC5939]
     media-config-list / ; Section 3.3.4.1
     payload-number-config-list ; Section 3.3.4.2

Except for the extension-config-list, the pot-cfg-list MUST NOT contain more than one instance of each parameter list 🙅‍♀️💅

3.3.6.1. Returning Capabilities in the Answer ✨

Okay so potential and latent configuration attributes can totally come back inside an answer SDP 🦋. This is how the answerer says "hey, I can handle these other setups too!" An offer might pack several potential configurations for one media stream and/or latent configurations for extra streams. The answer then tells you, with an acfg attribute 💅, which configuration it accepted and used to build the base configuration for each active media stream. But the reply MAY also contain potential and/or latent configuration attributes, with parameters, to show which of the other offered configurations would be acceptable too. This is super handy if someone later wants to reconfigure a stream, like to reduce resource consumption 🌸.

When potential and/or latent configurations come back in an answer, all numbering MUST refer to the configuration and capability attribute numbering of the offer 🎀. The offered capability attributes don't need to be returned in the answer. The answer MAY include additional capability attributes and/or configurations, with distinct numbering. The parameter values of any returned pcfg or lcfg attributes MUST be a subset of the ones in the offered configurations and/or the ones the answerer added 🧚. Values MAY be omitted only if they were indicated as alternative sets, or optional, in the original offer. The parameter set in the returned acfg attribute doesn't have to be repeated in a returned pcfg attribute. The answerer MAY return more than one pcfg attribute with the same configuration number if it's needed to describe selected combinations of optional or alternative parameters 💜.

Similarly, one or more session capability attributes (a=sescap) MAY be returned to indicate which of the offered session capabilities are supportable by the answerer (see Section 3.3.8) 🌈.

Note that, even though the answerer MAY return capabilities beyond the ones the offerer included, these capabilities MUST NOT be used to form any base level media description in the answer 🚫. For this reason, it's advisable for the offerer to include most, if not all, of the potential and latent configurations it can support in the initial offer, unless the size of the resulting SDP is a concern 🍓. Either party MAY later announce additional capabilities by renegotiating the session in a second offer/answer exchange 🦋.

3.3.6.2. Payload Type Number Mapping 🌸

When media format capabilities defined in rmcap attributes are used in potential configuration lines, the transport protocol uses RTP, and it's necessary to assign payload type numbers. Sometimes it's desirable to assign different payload type numbers to the same media format capability when it's used in different potential configurations. One example is when configurations for AVP and SAVP are offered. The offerer would like the answerer to use different payload type numbers for encrypted and unencrypted media, so the offerer can decide whether or not to render early media that arrives before the answer is received 👀.

For example, if the answerer picked AVP, then the media the offerer receives isn't encrypted, so it can be played out before the answer arrives. Conversely, if SAVP was picked, the cryptographic parameters and keying material in the answer might be needed to decrypt the received media 🔐. If the offer configuration said AVP media uses one set of payload types and SAVP uses a different set, then the offerer will know whether early media is encrypted just by checking the RTP payload type number in the received packet ✨.

This association of distinct payload type numbers with different transport protocols requires a separate pcfg line for each protocol. Clearly, this technique can't be used if the number of potential configurations exceeds the number of possible payload type numbers 💀.

When media capabilities negotiation is employed, SDP records are likely to contain conventional attributes like rtpmap, fmtp, and other media-format-related lines, plus capability attributes like rmcap, omcap, mfcap, and mscap that map into those conventional attributes when a potential configuration invokes them. In those cases, it MAY be appropriate to use the delete-attributes option [RFC5939] in the attribute configuration list parameter, so you don't end up generating conflicting fmtp attributes for a particular configuration 🧹. Any media-specific attributes in the media block that refer to media formats not used by the potential configuration MUST be ignored 🙈.

For example:

v=0
o=- 25678 753849 IN IP4 192.0.2.1
s=
c=IN IP4 192.0.2.1
t=0 0
a=creq:med-v0
m=audio 3456 RTP/AVP 0 18 100
a=rtpmap:100 telephone-event
a=fmtp:100 0-11
a=rmcap:1 PCMU/8000
a=rmcap:2 G729/8000
a=rmcap:3 telephone-event/8000
a=mfcap:3 0-15
a=pcfg:1 m=2,3|1,3 a=-m pt=1:0,2:18,3:100
a=pcfg:2

In this example, PCMU is media capability 1, G729 is media capability 2, and telephone-event is media capability 3. The a=pcfg:1 line says the preferred configuration is G.729 with extended DTMF events, second is G.711 mu-law with extended DTMF events, and the base media-level attributes are to be deleted 🎶. Mixing G.729, G.711, and "commercial" DTMF events is least preferred, and that's the base configuration provided by the m= line, which is, by default, the least preferred configuration 🙃. The rtpmap and fmtp attributes of the base configuration are replaced by the rmcap and mfcap attributes when the proposed configuration invokes them 🦋.

If the preferred configuration is selected, the SDP answer will look like the following:

v=0
o=- 25678 753849 IN IP4 192.0.2.1
s=
c=IN IP4 192.0.2.1
t=0 0
a=csup:med-v0
m=audio 3456 RTP/AVP 18 100
a=rtpmap:100 telephone-event/8000
a=fmtp:100 0-15
a=acfg:1 m=2,3 pt=1:0,2:18,3:100

3.3.7. Substitution of Media Payload Type Numbers in Capability Attribute Parameters 🧚

In some cases, for example when an RFC 2198 [RFC2198] redundancy audio subtype (RED) capability is defined in an mfcap attribute, the parameters to an attribute might contain payload type numbers. There are two options for specifying those payload type numbers. They can be expressed explicitly, in which case they're bound to actual payload types by means of the payload type number parameter (pt=) in the appropriate potential or latent configuration 🎀. For example, the following SDP fragment defines a potential configuration with redundant G.711 mu-law:

m=audio 45678 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=rmcap:1 PCMU/8000
a=rmcap:2 RED/8000
a=mfcap:2 0/0
a=pcfg:1 m=2,1 pt=2:98,1:0

The potential configuration is then equivalent to:

m=audio 45678 RTP/AVP 98 0
a=rtpmap:0 PCMU/8000
a=rtpmap:98 RED/8000
a=fmtp:98 0/0

A more general mechanism is provided via the parameter substitution rule. When an mfcap, mscap, or acap attribute is processed, its arguments will be scanned for a payload type number escape sequence of the following form (in ABNF):

ptn-esc = "%m=" media-cap-num "%" ; defined in Section 3.3.1

If the sequence is found, it's replaced by the payload type number assigned to the media capability number, as specified by the pt= parameter in the selected potential configuration. Only actual payload type numbers are supported -- wildcards are excluded 🚫. The sequence %% (null digit string) is replaced by a single percent sign, and processing continues with the next character, if any 💖.

For example, the above offer sequence could have been written as:

m=audio 45678 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=rmcap:1 PCMU/8000
a=rmcap:2 RED/8000
a=mfcap:2 %m=1%/%m=1%
a=pcfg:1 m=2,1 pt=2:98,1:0

and the equivalent SDP is the same as above 🌟.

3.3.8. The Session Capability Attribute ✨

okay so potential and latent configurations let offerers and answerers show off a whole buffet of options for now and later 🍓 but ngl, not every combo is gonna vibe together in real life 💀

The session capability attribute is how the offerer and/or answerer says "these exact combos of media stream configs? I can totally handle these" 🫶 Each session capability in an offer or answer MAY be written as a list of required potential configurations, and it MAY also include a list of optional potential and/or latent configurations.

What counts as a good combo can depend on processing load, total bandwidth, or literally anything else the people talking care about 🧚 If the answerer supports media capabilities negotiation and session configurations are offered, it MUST accept one of the offered configurations, or it MUST refuse the session. So if the offer includes any session capabilities, the offerer SHOULD list every session capability it's willing to support 💖

The session capability attribute is a session-level attribute written as:

"a=sescap:" <session num> <list of configs>

It lines up with the standard value attribute definition like this:

att-field        = "sescap"
att-value        = session-num 1*WSP list-of-configs
                   [1*WSP optional-configs]
session-num      = NonZeroDigit *9(DIGIT)  ; DIGIT defined
                                           ; in RFC 5234
list-of-configs  = alt-config *("," alt-config)
optional-configs = "[" list-of-configs "]"
alt-config       = config-number *("|" config-number)

The session-num is the session's ID. A lower-numbered session gets preference over a higher-numbered one, and leading zeroes are not permitted 🔢 Each alt-config list holds alternative media configurations within that session, and the preference order there is based on config-num as specified in RFC 5939 [RFC5939]. Heads up: when a session preference order is present, it beats the individual media stream configuration preference order 👑

Using session capability attributes means configuration numbers for potential and latent configurations MUST be unique across the entire session 🔒 RFC 5939 [RFC5939] only asks that "pcfg" configuration numbers be unique within a media description. Also, leading zeroes are still not permitted.

Here's an example. Picture an endpoint that can handle an audio stream alongside either one H.264 video stream or two H.263 video streams plus a floor control stream. In the second case, the second video stream is optional 🎀 The SDP offer might look something like this (offering audio, an H.263 video stream, BFCP, and another optional H.263 video stream). The empty lines are just for readability and aren't part of valid SDP:

v=0
o=- 25678 753849 IN IP4 192.0.2.1
s=
c=IN IP4 192.0.2.1
t=0 0
a=creq:med-v0
a=sescap:2 1,2,5,[3]
a=sescap:1 1,4

m=audio 54322 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=pcfg:1
m=video 22344 RTP/AVP 102
a=rtpmap:102 H263-1998/90000
a=fmtp:102 CIF=4;QCIF=2;F=1;K=1
i=main video stream
a=label:11
a=pcfg:2
a=rmcap:1 H264/90000
a=mfcap:1 profile-level-id=42A01E; packetization-mode=2
a=acap:1 label:13
a=pcfg:4 m=1 a=1 pt=1:104

m=video 33444 RTP/AVP 103
a=rtpmap:103 H263-1998/90000
a=fmtp:103 CIF=4;QCIF=2;F=1;K=1
i=secondary video (slides)
a=label:12
a=pcfg:3

m=application 33002 TCP/BFCP *
a=setup:passive
a=connection:new
a=floorid:1 m-stream:11 12
a=floor-control:s-only
a=confid:4321
a=userid:1234
a=pcfg:5

Now say the answerer gets MediaCapNeg but can't do Binary Floor Control Protocol. It'd respond with this (the empty lines are still just for readability and not valid SDP) 🙈

v=0
o=- 25678 753849 IN IP4 192.0.2.1
s=
c=IN IP4 192.0.2.22
t=0 0
a=csup:med-v0
a=sescap:1 1,4

m=audio 23456 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=acfg:1

m=video 41234 RTP/AVP 104
a=rtpmap:104 H264/90000
a=fmtp:104 profile-level-id=42A01E; packetization-mode=2
a=acfg:4 m=1 a=1 pt=1:104
m=video 0 RTP/AVP 103
a=acfg:3

m=application 0 TCP/BFCP *
a=acfg:5

An endpoint that doesn't do media capabilities negotiation but does support H.263 video would answer with one or two H.263 video streams. In the two-stream case, the answerer may issue a second offer to reconfigure the session to one audio channel and one video channel using H.264 or H.263 🎬

Session capabilities can include latent capabilities too 🦋 Here's a similar example. The offerer wants to set up an audio stream first, then add two video streams with chair control later. If the answerer doesn't understand Media CapNeg, or can't handle the dual video streams or the floor control, it may support a single H.264 video stream instead. Heads up that getting to the most favored configuration takes two offer/answer exchanges.

v=0
o=- 25678 753849 IN IP4 192.0.2.1
s=
c=IN IP4 192.0.2.1
t=0 0
a=creq:med-v0
a=sescap:1 1,3,4,5
a=sescap:2 1,2
a=sescap:3 1

a=rmcap:1 H263-1998/90000
a=mfcap:1 CIF=4;QCIF=2;F=1;K=1
a=tcap:1 RTP/AVP TCP/BFCP
m=audio 54322 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=pcfg:1
m=video 22344 RTP/AVP 102
a=rtpmap:102 H264/90000
a=fmtp:102 profile-level-id=42A01E; packetization-mode=2
a=label:11
a=content:main
a=pcfg:2
a=lcfg:3 mt=video t=1 m=1 a=31,32
a=acap:31 label:12
a=acap:32 content:main
a=lcfg:4 mt=video t=1 m=1 a=41,42
a=acap:41 label:13
a=acap:42 content:slides
a=lcfg:5 mt=application m=51 t=51
a=tcap:51 TCP/BFCP
a=omcap:51 *
a=acap:51 setup:passive
a=acap:52 connection:new
a=acap:53 floorid:1 m-stream:12 13
a=acap:54 floor-control:s-only
a=acap:55 confid:4321
a=acap:56 userid:1234

So in this one, the default offer (what endpoints without capability negotiation see) is a PCMU audio stream plus an H.264 video stream 🎧 Notice the lcfg lines for the video streams don't have pt= parameters, since they aren't needed yet. Payload type numbers get assigned during the offer/answer exchange that actually sets up those streams. Also, the three rmcap, mfcap, and tcap attributes used by lcfg:3 and lcfg:4 live at the session level so both latent configs can reference them. As per Section 3.3, the media attributes generated from rmcap, mfcap, and tcap are always media-level attributes 📌 If the answerer supports Media CapNeg and goes for the most desired configuration, it'd send back this SDP:

v=0
o=- 25678 753849 IN IP4 192.0.2.1
s=
c=IN IP4 192.0.2.22
t=0 0
a=csup:med-v0
a=sescap:1 1,3,4,5
a=sescap:2 1,2
a=sescap:3 1
m=audio 23456 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=acfg:1
m=video 0 RTP/AVP 102
a=pcfg:2
a=lcfg:3 mt=video t=1 m=1 a=31,32
a=lcfg:4 mt=video t=1 m=1 a=41,42
a=lcfg:5 mt=application t=2

This exchange gets an audio stream up right away so you can have a preliminary chat 🗣️ Then, at the right moment, a "reconfiguration" offer/answer exchange would presumably add the video and chair control streams. Main character behavior, honestly 👑

3.4. Offer/Answer Model Extensions 🎀

In this section, we extend the offer/answer model from RFC 3264 [RFC3264] and RFC 5939 [RFC5939]. The goal is to let media format and associated parameter capabilities, latent configurations, and acceptable combinations of media stream configurations all work inside the SDP capability negotiation framework 🧩 Quick note: these procedures build on RFC 5939 [RFC5939] Section 6. Those procedures are the baseline set of capability negotiation offer/answer rules, and they MUST be followed, with the extensions defined here layered on top 💅

SDP capability negotiation [RFC5939] gives you a fairly compact way to offer the equivalent of an ordered list of alternative configurations for offered media streams. This is the kind of thing you'd otherwise spell out with separate "m=" lines and their attributes 🌸 The acap, mscap, mfcap, omcap, and rmcap attributes are built to map pretty straightforwardly onto those equivalent "m=" lines and conventional attributes, as long as they're invoked by a pcfg, lcfg, or acfg attribute with the right parameters. The a=pcfg: lines, together with the "m=" line itself, represent offered media configurations. The a=lcfg: lines represent alternative capabilities for future use 🔮

3.4.1. Generating the Initial Offer 🍓

The media capabilities negotiation extensions in this document cover these categories of features:

Media format capabilities and associated parameters ("rmcap", "omcap", "mfcap", and "mscap" attributes)
Potential configurations using those media format capabilities and associated parameters
Latent media streams ("lcfg" attribute)
Acceptable combinations of media stream configurations ("sescap" attribute).

Here's the high-level version of how it works 🧠 When an endpoint generates an initial offer and wants to use what this document describes, it SHOULD identify and define the media formats and associated parameters it can support, using the "rmcap", "omcap", "mfcap", and "mscap" attributes. The SDP media lines ("m=") should be filled in with the actual configuration to use if the other party doesn't understand capability negotiation. By default, that's the least preferred configuration. Usually the media line holds the bare-minimum config the offerer can live with.

Preferred configurations for each media stream come right after the media line. The offer may also include latent configuration ("lcfg") attributes at the media level, describing media streams and/or configurations the offerer isn't offering right now but is willing to support in a future offer/answer exchange. A simple example would be slipping a latent video config into an offer for an audio stream 🎶

Last but not least, if the offerer wants to put restrictions on which combos of potential configurations get used, it includes session capability ("sescap") attributes to say so 🔐

If the offerer needs the answerer to understand the media capability extensions, the offerer MUST include a "creq" attribute containing the value "med-v0". If media capability negotiation is only required for specific media descriptions, the "med-v0" value MUST be provided only in "creq" attributes within those media descriptions, as described in RFC 5939 [RFC5939] 🫶

Down below, we get into the nitty-gritty of building the offer SDP 🧁

3.4.1.1. Offer with Media Capabilities 💅

okay bestie, first up: for each RTP-based media format you want to include as a media format capability, the offer MUST include an rmcap attribute for that format, as defined in Section 3.3.1 ✨

non-RTP-based formats get the same treatment, just with omcap instead. the offer MUST include an omcap attribute for the media format as defined in Section 3.3.1 🦋

here's the tea: the media capability number space is shared between rmcap and omcap, so each media capability number provided (including ranges) MUST be unique in the entire SDP. no repeats allowed, periodt 🔥

if an fmtp parameter value is needed for a media format (RTP-based or not) in a media capability, then the offer MUST include one or more mfcap parameters with the relevant fmtp parameter values for that media format as defined in Section 3.3.2. when multiple mfcap parameters are provided for a given media capability, they MUST be provided in accordance with the concatenation rules in Section 3.3.2.1. 🎀

for each of those media format capabilities, the offer MAY include one or more mscap parameters with attributes needed for those specific media formats as defined in Section 3.3.3. these get instantiated at the media level, so session-level-only attributes MUST NOT be used in the mscap parameter. the mscap parameter MUST NOT include an rtpmap or fmtp attribute (rmcap and mfcap are used instead) 🌸

want a media capability or parameter to only apply to one particular media stream? then it MUST be provided within the corresponding media description. otherwise, the media format capabilities and parameters MUST be provided at the session level. heads up: whatever attribute or parameter is embedded this way will always be instantiated at the media level anyway 👑

This is due to those parameters being effectively media-level parameters. If session-level attributes are needed, the acap attribute defined in RFC 5939 [RFC5939] can be used; however, it does not provide for media-format-specific instantiation.

including all of the above is not an offer to actually use the capabilities. you still need a potential configuration for that 🧚 if the offerer wants to offer one or more of those media capabilities, they MUST be included as part of a potential configuration (pcfg) attribute as defined in Section 3.3.4. each potential configuration MUST include a config-number, and each config-number MUST be unique in the entire SDP (note this differs from RFC 5939 [RFC5939], which only requires uniqueness within a media description). also, the config-number MUST NOT overlap with any config-number used by a latent configuration in the SDP. as described in RFC 5939 [RFC5939], lower config-numbers indicate a higher preference; the ordering still applies within a given media description only though 🌈

for a media capability to be included in a potential configuration, there MUST be an m= parameter in the pcfg attribute referencing the media capability number in question. when one or more media capabilities are included in an offered potential configuration (pcfg), they completely replace the list of media formats offered in the actual configuration (m= line). any attributes included for those formats remain in the SDP though (e.g., rtpmap, fmtp, etc.). for non-RTP-based media formats, the format-name (from the omcap media capability) is simply added to the m= line as a media format (e.g., t38). for RTP-based media, payload type mappings MUST be provided by use of the pt parameter in the potential configuration (see Section 3.3.4.2); payload type escaping may be used in mfcap, mscap, and acap attributes as defined in Section 3.3.7 💖

note that the mt parameter MUST NOT be used with the pcfg attribute (since it is defined for the lcfg attribute only). the media type in a potential configuration cannot be changed from that of the encompassing media description 🙈

3.4.1.2. Offer with Latent Configuration 🦋

if the offerer wishes to offer one or more latent configurations for future use, the offer MUST include a latent configuration attribute (lcfg) for each one, as defined in Section 3.3.6 ✨

each lcfg attribute:

MUST be specified at the media level
MUST include a config-number that is unique in the entire SDP (including for any potential configuration attributes). note that config-numbers in latent configurations do not indicate any preference order
MUST include a media type (mt)
MUST reference a valid transport capability (t)

each lcfg attribute MAY include additional capability references, which may refer to capabilities anywhere in the session description, subject to any restrictions normally associated with such capabilities. for example, a media-level attribute capability must be present at the media level in some media description in the SDP. this differs from the potential configuration attribute, which cannot validly refer to media-level capabilities in another media description (per RFC 5939 [RFC5939], Section 3.5.1) 🌸

Potential configurations constitute an actual offer and may instantiate a referenced capability. Latent configurations are not actual offers; hence, they cannot instantiate a referenced capability. Therefore, it is safe for those to refer to capabilities in another media description.

3.4.1.3. Offer with Configuration Combination Restrictions 💜

if the offerer wants to indicate restrictions or preferences among combinations of potential and/or latent configurations, a session capability (sescap) attribute MUST be provided at the session level for each such combination, as described in Section 3.3.8 🎀 each sescap attribute MUST include a session-num that is unique in the entire SDP. the lower the session-num, the more preferred that combination is. furthermore, sescap preference order takes precedence over any order specified in individual pcfg attributes 👑

For example, if we have pcfg-1 and pcfg-2, and sescap-1 references pcfg-2, whereas sescap-2 references pcfg-1, then pcfg-2 will be the most preferred potential configuration. Without the sescap, pcfg-1 would be the most preferred.

3.4.2. Generating the Answer 🫶

when receiving an offer, the answerer MUST check the offer for creq attributes containing the value med-v0. answerers compliant with this specification will support this value in accordance with the procedures specified in RFC 5939 [RFC5939] 💅

the SDP MAY contain:

media format capabilities and associated parameters (rmcap, omcap, mfcap, and mscap attributes)
potential configurations using those media format capabilities and associated parameters
latent media streams (lcfg attribute)
acceptable combinations of media stream configurations (sescap attribute)

the high-level informative description of the operation is as follows 🧚

when the answering party receives the offer, if it supports the required capability negotiation extensions, it should select the most-preferred configuration it can support for each media stream, and build its answer accordingly. the configuration selected for each accepted media stream is placed into the answer as a media line with associated parameters and attributes. if a proposed configuration is chosen for a given media stream, the answer must contain an actual configuration (acfg) attribute for that media stream to indicate which offered pcfg attribute was used to build the answer. the answer should also include any potential or latent configurations the answerer can support, especially any configurations compatible with other potential or latent configurations received in the offer. the answerer should make note of those configurations it might wish to offer in the future ✨

below we provide a more detailed normative description of how the answerer processes the offer SDP and generates an answer SDP 🌈

3.4.2.1. Processing Media Capabilities and Potential Configurations ✨

okay bestie, first things first: the answerer MUST figure out if it even needs to do media capability negotiation 🧐 by scanning the SDP for valid and preferred potential configuration attributes that include media configuration parameters (aka an m parameter in the pcfg attribute) 💅.

So a potential configuration counts as valid if:

It is valid according to the rules defined in RFC 5939 [RFC5939].
It has a config-number that is unique in the entire SDP and doesn't overlap with any latent configuration config-numbers 🦋.
All media format capabilities (rmcap or omcap), media format parameter capabilities (mfcap), and media-specific capabilities (mscap) referenced by the potential configuration (its m parameter) are valid themselves (as defined in Sections 3.3.1, 3.3.2, and 3.3.3). Each one also has to be provided either at the session level or inside this particular media description 🌸.
All RTP-based media format capabilities (rmcap) have a matching payload type (pt) parameter in the potential configuration. That has to map to a valid payload type that's unique within the resulting SDP 🎀.
Any concatenation (see Section 3.3.2.1) and substitution (see Section 3.3.7) applied to any capability (mfcap, mscap, or acap) referenced by this potential configuration ends up with a valid SDP 🙌.

ngl, SDP doesn't interpret the value of fmtp parameters at all, so any resulting fmtp parameter value counts as valid 🤷‍♀️.

Next up, the answerer MUST work out the order to negotiate potential configurations in 👑. If there are no session capability (sescap) attributes, this just follows RFC 5939 [RFC5939]: a lower config-number within a media description is preferred over a higher one. But if a valid sescap attribute is there, the preference order it gives MUST take precedence 🫶. A sescap attribute is valid if:

It follows the rules in Section 3.3.8.
All the configurations it references are valid themselves (and yes, that can include actual, potential, AND latent configurations 🍓).

The answerer MUST now process the offer for each media stream based on the most preferred valid potential configuration, following the procedures in RFC 5939 [RFC5939], Section 3.6.2, plus the extra stuff below 💖:

If one or more media format capabilities are in the potential configuration, they replace ALL the media formats in the m= line for that media description 😭. For non-RTP-based media formats (omcap), the format-name gets added. For RTP-based media formats (rmcap), the payload-type from the payload-type mapping (pt) gets added, and a matching rtpmap attribute gets added to the media description 🧚.
If one or more media format parameter capabilities are in the potential configuration, the matching fmtp attributes get added to the media description. This happens indirectly, through the media format capability 🌈.
If one or more media-specific capabilities are in the potential configuration, the matching attributes get added to the media description. This also happens indirectly, through the media format capability ✨.
When the answerer checks whether it supports a potential configuration with one or more media format capabilities, it MUST support at least one of the media formats that were offered. If it doesn't, the answerer MUST move on to the next potential configuration in the preference order 🚫.
If session capability (sescap) preference ordering is included, the potential configuration selection process MUST stick to that ordering 💅. This might mean coordinating which potential configurations get picked across media descriptions 🦋. The answerer MUST accept one of the offered sescap combinations (meaning all the required potential configurations specified), or it MUST reject the entire session 💀.

Once the answerer has picked a valid and supported offered potential configuration for every media stream (or has fallen back to the actual configuration plus any added session attributes) 🎀, it MUST generate a valid answer SDP as described in RFC 5939 [RFC5939], Section 3.6.2, plus the extra stuff below:

Additional answer capabilities and potential configurations MAY be returned, following Section 3.3.6.1 🌸. Their capability numbers and configuration numbers MUST be different from the ones used in the offer SDP 🙈.
Latent configuration processing and answer generation MUST be performed, as described below 🔥.
Session capability specs for the potential and latent configurations in the answer MAY be included (see Section 3.3.8) 💜.

3.4.2.2. Latent Configuration Processing

Okay, so the answerer MUST check whether it needs to do any latent configuration processing by looking through the SDP for valid latent configuration attributes (lcfg). An lcfg attribute is valid if:

It follows what Section 3.3.5 describes.
It has a config-number that's unique in the entire SDP and doesn't overlap with any potential configuration config-number 🧩.
It includes a valid media type (mt=).
It references a valid transport capability (t=).
All the other capabilities it references are valid too 🫶.

For each valid latent configuration in the offer, the answerer checks whether it could support that latent configuration in a later offer/answer exchange 🔮. If it could, it includes the latent configuration in the answer with the same configuration number. It's handled a lot like potential configurations, with the one selected returned in an actual configuration attribute (see RFC 5939 [RFC5939]) 💖. If the answerer only supports a (non-mandatory) subset of the parameters in a latent configuration, the answer's latent configuration only includes the parameters it does support (kinda like acfg processing) 🍓. Just remember, latent configurations are NOT an actual offer at this point in time. They just show extra configurations that could be supported 🦋.

If a session capability (sescap) attribute is included and it references a latent configuration, then the answerer has to process that latent configuration inside the limits the session capability sets. Basically, it must be possible to support it at the same time as any required (aka non-optional) potential configurations in the session capability 🎀. The answerer may also add its own sescap indications to the answer 🌈.

3.4.3. Offerer Processing of the Answer

The offerer MUST process the answer following Section 3.6.3 of RFC 5939 [RFC5939], plus the further explanation below 💅.

When the offerer processes the answer SDP based on a valid actual configuration attribute in the answer, and that valid configuration includes one or more media capabilities, the offerer MUST also process it as if the offer had been sent using those media capabilities instead of the actual configuration 🪄. In particular, the media formats in the m= line, plus any payload type mappings (rtpmap), fmtp parameters (mfcap), and media-specific attributes (mscap) MUST be used 🙌. This may involve concatenation and substitution rules (see Sections 3.3.2.1 and 3.3.7) 🧠. The actual configuration attribute can also be used to infer that higher-preference configurations that weren't chosen aren't acceptable, subject to any constraints in a session capability (sescap) attribute in the offer 👀. Heads up: the SDP capability negotiation base spec [RFC5939] requires the answerer to pick the highest-preference configuration it can support, subject to local policies 👑.

When the offerer gets the answer, it SHOULD also make note of any capabilities and/or latent configurations included for future use, plus any constraints on how those can be combined 📝.

3.4.4. Modifying the Session

If, later on, one of the parties wants to change the operating parameters of a session, like adding a new media stream or changing the properties of an existing stream 🔄, they can do that through the offer/answer mechanisms defined in [RFC3264]. If the initiating party remembered the codecs, potential configurations, latent configurations, and session capabilities the other party gave in the earlier negotiation, it MAY use that knowledge to boost the odds of a successful modification 🧚. Or, the initiator MAY do a brand new capabilities exchange as part of the reconfiguration 💫.

In that case, the new capabilities replace the previously negotiated ones. This can be super useful if conditions change on the endpoint 🌸.

4. Examples ✨

In this section, we show examples of how to use media capabilities with SDP capability negotiation 💜.

4.1. Alternative Codecs 🎀

okay so this example gives you a choice of one of six variations of the Adaptive Multi-Rate codec 🎶 the default config from the media line is the same as the most preferred config, so no confusion there 💅 each config uses its own payload type number so the offerer can interpret early media 🦋

v=0
o=- 25678 753849 IN IP4 192.0.2.1
s=
c=IN IP4 192.0.2.1
t=0 0
a=creq:med-v0
m=audio 54322 RTP/AVP 96
a=rtpmap:96 AMR-WB/16000/1
a=fmtp:96 mode-change-capability=1; max-red=220; \
mode-set=0,2,4,7
a=rmcap:1,3,5 audio AMR-WB/16000/1
a=rmcap:2,4,6 audio AMR/8000/1
a=mfcap:1,2,3,4 mode-change-capability=1
a=mfcap:5,6 mode-change-capability=2
a=mfcap:1,2,3,5 max-red=220
a=mfcap:3,4,5,6 octet-align=1
a=mfcap:1,3,5 mode-set=0,2,4,7
a=mfcap:2,4,6 mode-set=0,3,5,6
a=pcfg:1 m=1 pt=1:96
a=pcfg:2 m=2 pt=2:97
a=pcfg:3 m=3 pt=3:98
a=pcfg:4 m=4 pt=4:99
a=pcfg:5 m=5 pt=5:100
a=pcfg:6 m=6 pt=6:101

in that one, media capability 1 could've been left out of the first "rmcap" line and the matching "mfcap" attributes, and the "pcfg:1" line could've just been "pcfg:1" 🙈 small wins add up ✨

the next example offers a video stream with three options of H.264 and four transports 🎥 it also has an audio stream with different audio qualities: four variations of AMR, or AC3 🔊 the offer looks something like this:

v=0
o=- 25678 753849 IN IP4 192.0.2.1
s=An SDP Media NEG example
c=IN IP4 192.0.2.1
t=0 0
a=creq:med-v0
a=ice-pwd:speEc3QGZiNWpVLFJhQX
m=video 49170 RTP/AVP 100
c=IN IP4 192.0.2.56
a=maxprate:1000
a=rtcp:51540
a=sendonly
a=candidate 12345 1 UDP 9 192.0.2.56 49170 host
a=candidate 23456 2 UDP 9 192.0.2.56 51540 host
a=candidate 34567 1 UDP 7 198.51.100.1 41345 srflx raddr \
192.0.2.56 rport 49170
a=candidate 45678 2 UDP 7 198.51.100.1 52567 srflx raddr \
192.0.2.56 rport 51540
a=candidate 56789 1 UDP 3 192.0.2.100 49000 relay raddr \
192.0.2.56 rport 49170
a=candidate 67890 2 UDP 3 192.0.2.100 49001 relay raddr \
192.0.2.56 rport 51540
b=AS:10000
b=TIAS:10000000
b=RR:4000
b=RS:3000
a=rtpmap:100 H264/90000
a=fmtp:100 profile-level-id=42A01E; packetization-mode=2; \
sprop-parameter-sets=Z0IACpZTBYmI,aMljiA==; \
sprop-interleaving-depth=45; sprop-deint-buf-req=64000; \
sprop-init-buf-time=102478; deint-buf-cap=128000
a=tcap:1 RTP/SAVPF RTP/SAVP RTP/AVPF
a=rmcap:1-3,7-9 H264/90000
a=rmcap:4-6 rtx/90000
a=mfcap:1-9 profile-level-id=42A01E
a=mfcap:1-9 aMljiA==
a=mfcap:1,4,7 packetization-mode=0
a=mfcap:2,5,8 packetization-mode=1
a=mfcap:3,6,9 packetization-mode=2
a=mfcap:1-9 sprop-parameter-sets=Z0IACpZTBYmI
a=mfcap:1,7 sprop-interleaving-depth=45; \
sprop-deint-buf-req=64000; sprop-init-buf-time=102478; \
deint-buf-cap=128000
a=mfcap:4 apt=100
a=mfcap:5 apt=99
a=mfcap:6 apt=98
a=mfcap:4-6 rtx-time=3000
a=mscap:1-6 rtcp-fb nack
a=acap:1 crypto:1 AES_CM_128_HMAC_SHA1_80 \
inline:d0RmdmcmVCspeEc3QGZiNWpVLFJhQX1cfHAwJSoj|220|1:32
a=pcfg:1 t=1 m=1,4 a=1 pt=1:100,4:97
a=pcfg:2 t=1 m=2,5 a=1 pt=2:99,4:96
a=pcfg:3 t=1 m=3,6 a=1 pt=3:98,6:95
a=pcfg:4 t=2 m=7 a=1 pt=7:100
a=pcfg:5 t=2 m=8 a=1 pt=8:99
a=pcfg:6 t=2 m=9 a=1 pt=9:98
a=pcfg:7 t=3 m=1,3 pt=1:100,4:97
a=pcfg:8 t=3 m=2,4 pt=2:99,4:96
a=pcfg:9 t=3 m=3,6 pt=3:98,6:95
m=audio 49176 RTP/AVP 101 100 99 98
c=IN IP4 192.0.2.56
a=ptime:60
a=maxptime:200
a=rtcp:51534
a=sendonly
a=candidate 12345 1 UDP 9 192.0.2.56 49176 host
a=candidate 23456 2 UDP 9 192.0.2.56 51534 host
a=candidate 34567 1 UDP 7 198.51.100.1 41348 srflx \
raddr 192.0.2.56 rport 49176
a=candidate 45678 2 UDP 7 198.51.100.1 52569 srflx \
raddr 192.0.2.56 rport 51534
a=candidate 56789 1 UDP 3 192.0.2.100 49002 relay \
raddr 192.0.2.56 rport 49176
a=candidate 67890 2 UDP 3 192.0.2.100 49003 relay \
raddr 192.0.2.56 rport 51534
b=AS:512
b=TIAS:512000
b=RR:4000
b=RS:3000
a=maxprate:120
a=rtpmap:98 AMR-WB/16000
a=fmtp:98 octet-align=1; mode-change-capability=2
a=rtpmap:99 AMR-WB/16000
a=fmtp:99 octet-align=1; crc=1; mode-change-capability=2
a=rtpmap:100 AMR-WB/16000/2
a=fmtp:100 octet-align=1; interleaving=30
a=rtpmap:101 AMR-WB+/72000/2
a=fmtp:101 interleaving=50; int-delay=160000;
a=rmcap:14 ac3/48000/6
a=acap:23 crypto:1 AES_CM_128_HMAC_SHA1_80 \
inline:d0RmdmcmVCspeEc3QGZiNWpVLFJhQX1cfHAwJSoj|220|1:32
a=tcap:4 RTP/SAVP
a=pcfg:10 t=4 a=23
a=pcfg:11 t=4 m=14 a=23 pt=14:102

this offer shows the compactness win you get when you don't have to delete the base config attributes and recreate them in "acap" attributes for the potential configs 🧠✨

4.2. Alternative Combinations of Codecs (Session Configurations) 🦋

so if an endpoint has limited signal processing capacity, it might be able to handle, say, a G.711 mu-law audio stream together with an H.264 video stream, or a G.729B audio stream with an H.263-1998 video stream 📞🎥 it might then send an offer like this:

v=0
o=- 25678 753849 IN IP4 192.0.2.1
s=
c=IN IP4 192.0.2.1
t=0 0
a=creq:med-v0
a=sescap:1 2,4
a=sescap:2 1,3
m=audio 54322 RTP/AVP 18
a=rtpmap:18 G729/8000
a=fmtp:18 annexb=yes
a=rmcap:1 PCMU/8000
a=pcfg:1 m=1 pt=1:0
a=pcfg:2
m=video 54344 RTP/AVP 100
a=rtpmap:100 H263-1998/90000
a=rmcap:2 H264/90000
a=mfcap:2 profile-level-id=42A01E; packetization-mode=2
a=pcfg:3 m=2 pt=2:101
a=pcfg:4

note that the preferred session config (and the default too!) is G.729B with H.263 👑 this overrides the individual media stream preferences of PCMU and H.264 by the potential configuration numbering rule 💅

4.3. Latent Media Streams 🌸

picture this: the offerer can support either G.711 mu-law or G.729B, along with DTMF telephony events for the 12 common touchtone signals, but is willing to fall back to simple G.711 mu-law audio as a last resort 📱 on top of that, the offerer wants to announce that it can do video and Message Session Relay Protocol (MSRP) in the future, but doesn't want to offer a video stream or an MSRP stream right now 🫶 the offer might look like this:

v=0
o=- 25678 753849 IN IP4 192.0.2.1
s=
c=IN IP4 192.0.2.1
t=0 0
a=creq:med-v0
m=audio 23456 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=rmcap:1 PCMU/8000
a=rmcap:2 G729/8000
a=rmcap:3 telephone-event/8000
a=mfcap:3 0-11
a=pcfg:1 m=1,3|2,3 pt=1:0,2:18,3:100
a=lcfg:2 mt=video t=1 m=10|11
a=rmcap:10 H263-1998/90000
a=rmcap:11 H264/90000
a=tcap:1 RTP/AVP
a=lcfg:3 mt=message t=2 m=20
a=tcap:2 TCP/MSRP
a=omcap:20 *

the first "lcfg" line ("lcfg:2") announces support for H.263 and H.264 video (H.263 preferred) for future negotiation 🎬 the second "lcfg" line ("lcfg:3") announces support for MSRP for future negotiation 💬 the "m=" line and the "rtpmap" attribute offer an audio stream and provide the lowest precedence config (PCMU with no DTMF encoding) 🍓 the rmcap lines define the RTP-based media format capabilities (PCMU, G729, telephone-event, H263-1998, and H264), and the omcap line defines the non-RTP-based media format capability (wildcard) 🧚 the "mfcap" attribute gives the format parameters for telephone-event, specifying the 12 commercial DTMF 'digits' 🔢 and the "pcfg" line sets the most-preferred media config as PCMU plus DTMF events, with the next-most-preferred config as G.729B plus DTMF events 👑

if the answerer can support all the potential configs, and also H.263 video (but not H.264), it would reply with an answer like this:

v=0
o=- 24351 621814 IN IP4 192.0.2.2
s=
c=IN IP4 192.0.2.2
t=0 0
a=csup:med-v0
m=audio 54322 RTP/AVP 0 100
a=rtpmap:0 PCMU/8000
a=rtpmap:100 telephone-event/8000
a=fmtp:100 0-11
a=acfg:1 m=1,3 pt=1:0,3:100
a=pcfg:1 m=2,3 pt=2:18,3:100
a=lcfg:2 mt=video t=1 m=10

the "lcfg" line announces the ability to support H.263 video at a later time ⏰ the media line and the following "rtpmap" and "fmtp" attributes present the selected config for the media stream 🎀 the "acfg" line identifies the potential config it was taken from, and the "pcfg" line announces the potential ability to do G.729 with DTMF events too 🦋 > and here's the fun part: if congestion becomes a problem on the network later on, either party may, with expectation of success, offer a reconfiguration of the media stream to use G.729 in order to reduce packet sizes 🚨

5. IANA Considerations 💜

5.1. New SDP Attributes ✨

IANA has registered the following new SDP attributes 🙌

Attribute name: rmcap
Long form name: RTP-based media format capability
Type of attribute: session-level and media-level
Subject to charset: no
Purpose: associate RTP-based media capability number(s) with
media subtype and encoding parameters
Appropriate Values: see Section 3.3.1
Contact name: Flemming Andreasen, fandres@cisco.com

Attribute name: omcap
Long form name: non-RTP-based media format capability
Type of attribute: session-level and media-level
Subject to charset: no
Purpose: associate non-RTP-based media capability number(s) with
media subtype and encoding parameters
Appropriate Values: see Section 3.3.1
Contact name: Flemming Andreasen, fandreas@cisco.com

Attribute name: mfcap
Long form name: media format parameter capability
Type of attribute: session-level and media-level
Subject to charset: no
Purpose: associate media format attributes and
parameters with media format capabilities
Appropriate Values: see Section 3.3.2
Contact name: Flemming Andreasen, fandreas@cisco.com

Attribute name: mscap
Long form name: media-specific capability
Type of attribute: session-level and media-level
Subject to charset: no
Purpose: associate media-specific attributes and
parameters with media capabilities
Appropriate Values: see Section 3.3.3
Contact name: Flemming Andreasen, fandreas@cisco.com

Attribute name: lcfg
Long form name: latent configuration
Type of attribute: media-level
Subject to charset: no
Purpose: to announce supportable media streams
without offering them for immediate use.
Appropriate Values: see Section 3.3.5
Contact name: Flemming Andreasen, fandreas@cisco.com

Attribute name: sescap
Long form name: session capability
Type of attribute: session-level
Subject to charset: no
Purpose: to specify and prioritize acceptable
combinations of media stream configurations.
Appropriate Values: see Section 3.3.8
Contact name: Flemming Andreasen, fandreas@cisco.com

5.2. New SDP Capability Negotiation Option Tag 🎀

okay so IANA added a brand new option tag, med-v0 🦋, which is defined in this document. It got added to the "SDP Capability Negotiation Option Capability Tags" registry that was created for RFC 5939 [RFC5939]. ✨

5.3. SDP Capability Negotiation Configuration Parameters Registry 💅

IANA also glowed up the "SDP Capability Negotiation Potential Configuration Parameters" registry, which is currently registered and defined by RFC 5939 [RFC5939]. Here's the new plan 👑

The registry name should be "SDP Capability Negotiation Configuration Parameters Registry", and it should contain a table with these column headings:

Encoding Name: the syntactical value used for the capability negotiation configuration parameter, as defined in RFC 5939 [RFC5939], Section 3.5.
Descriptive Name: the name people commonly use to refer to the capability negotiation configuration parameter.
Potential Configuration Definition: a reference to the RFC that defines the configuration parameter in the context of a potential configuration attribute. If the configuration parameter isn't defined for potential configurations, the string "N/A" (Not Applicable) MUST be present instead. 🙈
Actual Configuration Definition: a reference to the RFC that defines the configuration parameter in the context of an actual configuration attribute. If the configuration parameter isn't defined for actual configurations, the string "N/A" (Not Applicable) MUST be present instead.
Latent Configuration Definition: a reference to the RFC that defines the configuration parameter in the context of a latent configuration attribute. If the configuration parameter isn't defined for latent configurations, the string "N/A" (Not Applicable) MUST be present instead.

An IANA SDP Capability Negotiation Configuration registration MUST be documented in an RFC in accordance with the IETF Review policy [RFC5226]. Furthermore, these are the non-negotiable vibes 🫶

The RFC MUST define the syntax and semantics of each new potential configuration parameter.
The syntax MUST adhere to the syntax provided for extension configuration lists in RFC 5939 [RFC5939], Section 3.5.1, and the semantics MUST adhere to the semantics provided for extension configuration lists in RFC 5939 [RFC5939], Sections 3.5.1 and 3.5.2.
Configuration parameters that apply to latent configurations MUST furthermore adhere to the syntax provided in Section 3.3.5 and the semantics defined overall in this document.
Associated with each registration MUST be the encoding name for the parameter as well as a short descriptive name for it.
Each registration MUST specify if it applies to:
- Potential configurations
- Actual configurations
- Latent configurations

5.4. SDP Capability Negotiation Configuration Parameter Registrations 🌈

IANA registered the following capability negotiation configuration parameters, and they are giving main character energy ⭐

Encoding Name: a
Descriptive Name: Attribute Configuration
Potential Configuration Definition: [RFC5939]
Actual Configuration Definition: [RFC5939]
Latent Configuration Definition: [RFC6871]

Encoding Name: t
Descriptive Name: Transport Protocol Configuration
Potential Configuration Definition: [RFC5939]
Actual Configuration Definition: [RFC5939]
Latent Configuration Definition: [RFC6871]

Encoding Name: m
Descriptive Name: Media Configuration
Potential Configuration Definition: [RFC6871]
Actual Configuration Definition: [RFC6871]
Latent Configuration Definition: [RFC6871]

Encoding Name: pt
Descriptive Name: Payload Type Number Mapping
Potential Configuration Definition: [RFC6871]
Actual Configuration Definition: [RFC6871]
Latent Configuration Definition: [RFC6871]

Encoding Name: mt
Descriptive Name: Media Type
Potential Configuration Definition: N/A
Actual Configuration Definition: N/A
Latent Configuration Definition: [RFC6871]

6. Security Considerations 🔐

okay bestie, this part is serious, so listen up 🙈 The security considerations of RFC 5939 [RFC5939] apply to this document too.

RFC 5939 [RFC5939] already flagged that negotiating transport protocols (like secure vs. non-secure) and negotiating keying methods and material are potential security issues that need integrity protection to fix. Latent configuration support gives the other side hints about what it supports for later offer/answer exchanges, including transport protocols and attribute capabilities, like keying methods. So if an attacker can remove or change latent configuration info to suggest that only non-secure or less-secure options are supported, they might be able to force a less secure session than would otherwise have happened 💀 The attack is a bit different from the ones in RFC 5939 [RFC5939], but the considerations and mitigation strategies are similar to those described there.

Another version of this attack involves the session capability ("sescap") attribute defined in this document. The sescap lets you set a preference order for all the potential configurations, and that preference takes precedence over any preference indicated in individual potential configuration attributes. So an attacker who can insert or modify a sescap attribute might be able to force negotiation of an insecure or less secure alternative than would otherwise have occurred. Again, the considerations and mitigation strategies are similar to those described in RFC 5939 [RFC5939].

The negotiable media formats and their associated parameters defined in this specification can cause problems for middleboxes that try to control bandwidth use, media flows, or processing resource consumption as part of network policy, but don't understand media capability negotiation. Just like the original SDP capability negotiation work [RFC5939], the SDP answer is always built so it carries the selected media encoding for every media stream that was selected. Until a middlebox understands capability negotiation, it should examine the answer SDP to get the best picture of the media streams being set up. As always, middleboxes do their job best when they fully understand media capability negotiation 🧚

7. Acknowledgements 🫶

this doc was heavily influenced by the discussions and work of the SDP Capability Negotiation design team. These people gave super useful comments and suggestions, either on the document itself or on the overall direction of the solution: Cullen Jennings, Matt Lepinski, Joerg Ott, Colin Perkins, and Thomas Stach 💜

We thank Ingemar Johansson and Magnus Westerlund for the examples that inspired this work, and for their critical reading of the document. We also thank Cullen Jennings, Christer Holmberg, and Miguel Garcia for reviewing the document 🌸

8. References 📚

8.1. Normative References

[RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
           Requirement Levels", BCP 14, RFC 2119, March 1997.

[RFC3264]  Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model
           with Session Description Protocol (SDP)", RFC 3264, June
           2002.

[RFC4566]  Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
           Description Protocol", RFC 4566, July 2006.

[RFC5226]  Narten, T. and H. Alvestrand, "Guidelines for Writing an
           IANA Considerations Section in RFCs", BCP 26, RFC 5226,
           May 2008.

[RFC5234]  Crocker, D. and P. Overell, "Augmented BNF for Syntax
           Specifications: ABNF", STD 68, RFC 5234, January 2008.

[RFC5939]  Andreasen, F., "Session Description Protocol (SDP)
           Capability Negotiation", RFC 5939, September 2010.

8.2. Informative References

[RFC2198]  Perkins, C., Kouvelas, I., Hodson, O., Hardman, V.,
           Handley, M., Bolot, J., Vega-Garcia, A., and S. Fosse-
           Parisis, "RTP Payload for Redundant Audio Data", RFC 2198,
           September 1997.

[RFC4568]  Andreasen, F., Baugher, M., and D. Wing, "Session
           Description Protocol (SDP) Security Descriptions for Media
           Streams", RFC 4568, July 2006.

[RFC4585]  Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey,
           "Extended RTP Profile for Real-time Transport Control
           Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, July
           2006.

[RFC4733]  Schulzrinne, H. and T. Taylor, "RTP Payload for DTMF
           Digits, Telephony Tones, and Telephony Signals", RFC 4733,
           December 2006.

[RFC4867]  Sjoberg, J., Westerlund, M., Lakaniemi, A., and Q. Xie,
           "RTP Payload Format and File Storage Format for the
           Adaptive Multi-Rate (AMR) and Adaptive Multi-Rate Wideband
           (AMR-WB) Audio Codecs", RFC 4867, April 2007.

[RFC5104]  Wenger, S., Chandra, U., Westerlund, M., and B. Burman,
           "Codec Control Messages in the RTP Audio-Visual Profile
           with Feedback (AVPF)", RFC 5104, February 2008.

Authors' Addresses 🌟

Robert R Gilman
Independent
3243 W. 11th Ave. Dr.
Broomfield, CO 80020
USA

EMail: bob_gilman@comcast.net


Roni Even
Huawei Technologies
14 David Hamelech
Tel Aviv  64953
Israel

EMail: roni.even@mail01.huawei.com


Flemming Andreasen
Cisco Systems
Iselin, NJ
USA

EMail: fandreas@cisco.com

📦 where this page comes from

ietf: rfc6871 2013-02
💖 girlified by claude-haiku-5-5 on 2026-10-17: a language model's rewrite of the original, under its terms