RFC 8851

RTP Payload Format Restrictions

📅 published January 2021, from ietf

RFC 8851: RTP Payload Format Restrictions 🎀

Okay bestie, this one is RFC 8851 and it's giving RTP Payload Format Restrictions ✨. It was written by A.B. Roach, Ed., from Mozilla 💅. It's a Standards Track document, so it's the real deal 👑, and it's dated January 2021. It updates RFC 4855 (the one about format parameter names) 🌸. The header doesn't list any RFC it obsoletes. The ISSN is 2070-1721, and it's published by the Internet Engineering Task Force (IETF) 🦋.

Abstract ✨

So here's the vibe 💖: this spec defines a framework for putting restrictions on RTP streams in the Session Description Protocol (SDP). It adds a brand-new "rid" (restriction identifier) SDP attribute, which uniquely identifies the RTP streams inside an RTP session and restricts their payload format parameters in a codec-agnostic way. That goes beyond what regular payload types can already do 🔥.

It also updates RFC 4855 with extra guidance on Format Parameter (fmtp) names and how they connect to the restrictions in this document 🧚.

Status of This Memo 🌈

This is an Internet Standards Track document, so it's official and not just a vibe 💅.

It's a product of the IETF and represents the consensus of the IETF community. It went through public review and got approved for publication by the Internet Engineering Steering Group (IESG). If you want more on Internet Standards, check Section 2 of RFC 7841 📚.

For the current status, any errata, and how to give feedback, go to https://www.rfc-editor.org/info/rfc8851 💜.

Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved 🎀.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they're in effect on the date this document was published. Please actually read these, bestie, because they describe your rights and restrictions for this document 🙈. Any Code Components pulled out of this document need to include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions. They come without warranty, as the Simplified BSD License describes ⭐.

Table of Contents 🦋

1.  Terminology
2.  Introduction
3.  Key Words for Requirements
4.  SDP "a=rid" Media Level Attribute
5.  "a=rid" Restrictions
6.  SDP Offer/Answer Procedures
  6.1.  Generating the Initial SDP Offer
  6.2.  Answerer Processing the SDP Offer
    6.2.1.  "a=rid"-Unaware Answerer
    6.2.2.  "a=rid"-Aware Answerer
  6.3.  Generating the SDP Answer
  6.4.  Offerer Processing of the SDP Answer
  6.5.  Modifying the Session
7.  Use with Declarative SDP
8.  Interaction with Other Techniques
  8.1.  Interaction with VP8 Format Parameters
    8.1.1.  max-fr - Maximum Frame Rate
    8.1.2.  max-fs - Maximum Frame Size, in VP8 Macroblocks
  8.2.  Interaction with H.264 Format Parameters
    8.2.1.  profile-level-id and max-recv-level - Negotiated
            Subprofile
    8.2.2.  max-br / MaxBR - Maximum Video Bitrate
    8.2.3.  max-fs / MaxFS - Maximum Frame Size, in H.264
            Macroblocks
    8.2.4.  max-mbps / MaxMBPS - Maximum Macroblock Processing Rate
    8.2.5.  max-smbps - Maximum Decoded Picture Buffer
  8.3.  Redundancy Formats and Payload Type Restrictions
9.  Format Parameters for Future Payloads
10. Formal Grammar
11. SDP Examples
  11.1.  Many Bundled Streams Using Many Codecs
  11.2.  Scalable Layers
12. IANA Considerations
  12.1.  New SDP Media-Level Attribute
  12.2.  Registry for RID-Level Parameters
13. Security Considerations
14. References
  14.1.  Normative References
  14.2.  Informative References
Acknowledgements
Contributors
Author's Address

1. Terminology 🌸

The terms "source RTP stream", "endpoint", "RTP session", and "RTP stream" are used as defined in [RFC7656] 💅.

[RFC4566] and [RFC3264] terminology is also used where it makes sense ✨.

2. Introduction 💖

Okay, so here's the big picture. The payload type (PT) field in RTP is what links the RTP payload format to the matching SDP media description 🔗. For any given PT, the SDP rtpmap and/or fmtp attributes describe the properties of the media carried in that RTP payload 🎀.

Recent standards have made multimedia apps super rich 🤩. They need either multiple RTP streams inside one RTP session [RFC8843] [RFC8853], or a huge pile of codecs. And that brings some real challenges 😭:

  • The PT space is restricted, which is a problem when you're specifying all the different payload configurations.
  • Codec-specific constructs for payload formats in SDP are a mess.
  • Payload format parameters can be missing or underspecified.
  • PTs get overloaded to mean not just codec configurations, but also individual streams inside an RTP session.

Let me explain the PT thing 🧠. [RFC3550] gives the PT field 7 bits in the RTP header. But the static mapping of RTP payload type numbers to payload formats, plus multiplexing RTP with other protocols (like the RTP Control Protocol (RTCP)), can leave only a limited number of payload type numbers for application use 🙈. So when the number of possible RTP payload configurations is bigger than the PT space available in an RTP session, we need a way to express the extra restrictions on payload configurations and map each RTP stream to its matching restrictions 💜. This gets worse with techniques like simulcast and layered codecs, which add even more streams into RTP sessions 📈.

This specification defines a new SDP framework for restricting source RTP streams (Section 2.1.10 of [RFC7656]). It also adds the SDP attributes needed to restrict payload formats in a codec-agnostic way 🦋. You can think of this framework as a complementary extension to the way media format parameters are already specified in SDP through the a=fmtp attribute 🍓.

The extra restrictions on individual streams get indicated with a new a=rid ("restriction identifier") SDP attribute 🎀. Important note 🚨: the restrictions sent through this attribute can only further restrict the parameters established on a PT format. They do NOT relax any restrictions imposed by other mechanisms. (Okay, I said NOT in caps, but that's just for vibes. The key words in this document are only the ones defined in Section 3, so don't read anything extra into it 🙃.)

This specification also uses the RTP Stream Identifier Source Description (SDES) RTCP item, defined in [RFC8852], to correlate RTP packets with their format specification in the SDP 🔍.

As Section 6.2.1 explains, this mechanism gets backwards compatibility through the normal SDP processing rules, which require unknown a= lines to be ignored 🫶. So implementations need to be ready to handle successful offers and answers from other implementations that neither indicate nor honor the restrictions this mechanism asks for 🌈.

And as Section 6 and its subsections explain, this mechanism gets extensibility in two ways: (a) offerers include all supported restrictions in their offer, and (b) answerers ignore a=rid lines that specify unknown restrictions 👑.

3. Key Words for Requirements 🔥

Time for the key words, bestie 💅. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here ✨. So if you see them in caps, they're requirements. If you see them in lowercase, they're just regular words 🧚.

4. SDP "a=rid" Media Level Attribute ✨

okay so this section defines a brand new SDP media-level attribute [RFC4566] called "a=rid" 🎀 its whole job is to carry a set of restrictions that get applied to one specific RTP stream. the rough shape looks like this (Section 10 has the official formal definition, so don't panic if this looks a little loose):

a=rid:<rid-id> <direction> [pt=<fmt-list>;]<restriction>=<value>...

an "a=rid" attribute sets restrictions that define one unique RTP payload configuration, and that configuration is identified by the "rid-id" field 💅 the magic part is that this value binds the restriction to the RTP stream carrying the matching RTP Stream Identifier Source Description (SDES) item [RFC8852]. implementations that use the "a=rid" parameter in SDP MUST support the RtpStreamId SDES item described in [RFC8852] 😭 and they MUST send that SDES item for every stream in an SDP media description ("m=") that still has "a=rid" lines left after the rules in Section 6 and its subsections have been applied. bestie, this is non-negotiable 🔥

now the redundancy situation 🦋 if your implementation uses the "a=rid" parameter in SDP and also uses redundancy RTP streams [RFC7656], like RTP RTX [RFC4588] or Forward Error Correction (FEC) [RFC5109], then for any source RTP stream that still has "a=rid" lines after Section 6 and its subsections are done, you MUST support the RepairedRtpStreamId SDES item described in [RFC8852] for those redundancy streams. and RepairedRtpStreamId MUST be used for every redundancy RTP stream it can be applied to 🎀 it is NOT applicable for redundancy formats that tie streams together directly through shared synchronization sources (SSRCs), like [RFC8627], or for other cases RepairedRtpStreamId can't handle, such as pointing at multiple source streams.

RepairedRtpStreamId is how the redundancy stream gets bound to its source stream. you set the RepairedRtpStreamId value on the redundancy stream to match the RtpStreamId value of the source stream 🌸 the redundancy stream MAY (but need not) have an "a=rid" line of its own. if it does, its RtpStreamId SDES item value will be different from the one on the source stream.

this indirection has a catch, though 🙈 if the SSRC tied to an RtpStreamId or RepairedRtpStreamId changes dynamically during the RTP session, you might temporarily lose the ability to correctly match source data with redundancy data. you can dodge this if every RTP packet, both source and repair, includes its RtpStreamId or RepairedRtpStreamId, respectively, after the change. to give redundancy info the best shot at being received and actually useful after a change like that, all the source packets referenced by the first several repair packets SHOULD include that info 🧚 it is RECOMMENDED that the number of such packets be big enough to give a high probability of the association getting updated. Section 4.1.1 of [RFC8285] has relevant guidance on RTP header extension transmission considerations. or, if you'd rather skip the whole issue, you can use redundancy mechanisms that directly reference their source data, like [RFC8627] ✨

the "direction" field tells you which direction of RTP stream packets the restrictions apply to. it can be either "send" or "recv" 💜 these restriction directions are written independently of any "inactive", "sendonly", "recvonly", or "sendrecv" attributes on the media section. so it's totally valid to put "recv" restrictions on a "sendonly" stream! those restrictions would kick in if the stream later switched to "sendrecv" or "recvonly" 👑

the optional "pt=<fmt-list>" lists one or more PT values that can be used in the associated RTP stream. if the "a=rid" attribute doesn't have a "pt" at all, then any PT value from the corresponding "m=" line is fair game 🍓

then there's the list of zero or more codec-agnostic restrictions (Section 5), which describes the restrictions the corresponding RTP stream will conform to 🌈

this framework MAY be used together with the "a=fmtp" SDP attribute to describe the media format parameters for a given RTP payload type. in that case, the "a=rid" restrictions (Section 5) further restrict the equivalent "a=fmtp" attributes ✨

a given SDP media description MAY have zero or more "a=rid" lines, each describing a different possible RTP payload configuration. and a given "rid-id" MUST NOT be repeated within the same media description ("m=" section) 🙅‍♀️ (okay, that one is a hard no, bestie)

the "a=rid" media attribute MAY be used for any RTP-based media transport. it isn't defined for other transports, although other documents may extend its semantics to cover them 🦋

the "rid" restrictions follow a syntax that looks similar to session-level and media-level parameters, but they're defined independently 💅 and all "rid" restrictions MUST be registered with IANA, using the registry defined in Section 12.

Section 10 gives the formal Augmented Backus-Naur Form (ABNF) [RFC5234] grammar for the "rid" attribute. the "a=rid" media attribute doesn't depend on charset either 🫶

5. "a=rid" Restrictions 🎀

this section defines the "a=rid" restrictions you can use to restrict the RTP payload encoding format in a codec-agnostic way. also check out the previous section for how the "pt" parameter works 💖

the following restrictions are meant to apply to video codecs in a codec-independent way:

  • max-width, for spatial resolution in pixels. if stream-orientation signaling is used to change the intended display orientation, this attribute refers to the width of the stream when a rotation of zero degrees is encoded.
  • max-height, for spatial resolution in pixels. if stream-orientation signaling is used to change the intended display orientation, this attribute refers to the height of the stream when a rotation of zero degrees is encoded.
  • max-fps, for frame rate in frames per second. for encoders that don't use a fixed frame rate, this value is used to limit the minimum amount of time between frames: the time between any two consecutive frames SHOULD NOT be less than 1/max-fps seconds ⏱️
  • max-fs, for frame size in pixels per frame. for rectangular frames, this is the product of frame width and frame height, in pixels. 📐
  • max-br, for bitrate in bits per second. this restriction only applies to the media payload and doesn't count overhead added by other layers (like RTP, UDP, IP, or Ethernet). the exact way to stay under this limit is up to the implementation, and brief spikes above the limit are allowed. but for any given one-second sliding window, the total number of bits in the RTP payload portion SHOULD NOT exceed the value specified in "max-br." 📊
  • max-pps, for pixel rate in pixels per second. this value SHOULD be handled exactly like max-fps, after doing this conversion: max-fps = max-pps / (width * height). if the stream resolution changes, this value gets recalculated. because of that recalculation, going over the specified maximum is possible near resolution-change boundaries. 🌷
  • max-bpp, for maximum number of bits per pixel, calculated as an average over all samples of any given coded picture. it's a floating point value with an allowed range of 0.0001 to 48.0. these values MUST NOT have more than four digits to the right of the decimal point 🔢
  • depend, to identify other streams that this stream depends on. the value is a comma-separated list of rid-ids. these rid-ids point at the RTP streams this stream needs in order to be interpreted properly. the mechanism in this document only lets you express those dependencies when the streams are in the same media section 🧩

all the restrictions are optional, and they're subject to negotiation based on the SDP offer/answer rules in Section 6 🤝

this list is meant to be an initial set of restrictions. future documents may define additional restrictions, so see Section 12.2 👀 this document doesn't define restrictions for audio codecs or for any media types other than video, but there's no reason other documents can't define and register them 🎶

Section 10 provides the formal Augmented Backus-Naur Form (ABNF) [RFC5234] grammar for each of the "a=rid" restrictions defined in this section 📜

6. SDP Offer/Answer Procedures 💌

this section describes the SDP offer/answer procedures [RFC3264] when you're using this framework 🫶

quick heads up: "rid-id" values only need to be unique within a media section ("m=" line). they don't necessarily have to be unique across the whole RTP session 🌈 in typical usage, each media section gets sent on its own unique 5-tuple (a combo of sending address, sending port, receiving address, receiving port, and transport protocol), and that gives an unambiguous scope ✨ similarly, when you use BUNDLE [RFC8843], Media Identification (MID) values tie RTP streams to exactly one media description. when restriction identifier (RID) gets used with the BUNDLE mechanism, streams will be associated with both MID and RID SDES items 💜

6.1. Generating the Initial SDP Offer 🎀

for each RTP media description in the offer, the offerer MAY choose to include one or more "a=rid" lines to specify a configuration profile for the given set of RTP payload types 💖

to build a given "a=rid" line, the offerer has to follow these steps:

  1. it MUST generate a "rid-id" that is unique within a media description.
  2. it MUST set the direction for the "rid-id" to either "send" or "recv".
  3. it MAY include a list of SDP media formats (usually matching RTP payload types) that are allowed to show up in the RTP stream. any payload type chosen MUST be a valid payload type for the media section, meaning it has to be listed on the "m=" line. the order of the listed formats matters: the alternatives go from (left) most preferred to (right) least preferred 🥇 when using RID, this preference overrides the normal codec preference that format type ordering on the "m=" line expresses, using regular SDP rules.
  4. the offerer then chooses zero or more "a=rid" restrictions (Section 5) to apply to the RTP stream, and adds them to the "a=rid" line 🛠️
  5. if the offerer wants the answerer to be able to specify a restriction but doesn't want to set a value itself, it includes the name of the restriction in the "a=rid" line, just without any value attached 🙌

Note: if an "a=fmtp" attribute is also used to provide media-format-specific parameters, then the "a=rid" restrictions will further restrict the equivalent "a=fmtp" parameters for the given payload type for the specified RTP stream 🌸

if a given codec would need an "a=fmtp" line when used without "a=rid", then the offer MUST include a valid matching "a=fmtp" line even when "a=rid" is in use 🧚

6.2. Answerer Processing the SDP Offer ✨

6.2.1. "a=rid"-Unaware Answerer 🦋

okay so if the receiver doesn't support the framework in this spec, the whole a=rid line just gets ignored, following the normal offer/answer rules [RFC3264]. no drama 🫶

Section 6.1 says the offer has to include a valid a=fmtp line for any media formats that need one anyway. a=rid can't replace a=fmtp config, bestie 🙅‍♀️ So ignoring the a=rid line is always going to leave you with a valid session description ✨ iconic.

6.2.2. "a=rid"-Aware Answerer 💅

If the answerer supports the a=rid attribute, it runs these verification steps, in order, for each a=rid line in a received offer:

  1. The answerer ensures that the a=rid line is syntactically well formed. If there's a syntax error, the a=rid line is discarded. 🧐
  2. The answerer pulls the rid-id out of the a=rid line and checks that it's unique within a media section. If it's a duplicate, the entire a=rid line, plus all a=rid lines with rid-ids that duplicate this one, are discarded and MUST NOT be included in the SDP answer. 🚫
  3. If the a=rid line contains a pt=, the list of payload types gets checked against the valid ones for the media section (the ones listed on the m= line). Any PT missing from the m= line gets discarded from the pt= values. If nothing is left in the pt= parameter after that, the a=rid line is discarded. 🗑️
  4. If the direction field is recv, the answerer makes sure the specified a=rid restrictions are supported. If a restriction isn't supported, the a=rid line is discarded. 🙃
  5. If the depend restriction is included, the answerer MUST make sure the listed rid-ids unambiguously match the rid-ids in the media description. Any depend a=rid lines that don't match are discarded. 🔍
  6. The answerer verifies that the restrictions are consistent with at least one of the codecs used with the RTP stream. If the a=rid line contains a pt=, that's the list of such codecs; otherwise, the list comes from the associated m= line. See Section 8 for more detail. If the a=rid restrictions are incompatible with the other codec properties for all codecs, the a=rid line is discarded. 💀

Note that the answerer doesn't need to understand every restriction in a send line. If a stream sender restricts the stream in a way the receiver doesn't understand, there are no interoperability issues. 🌸

6.3. Generating the SDP Answer 🎀

Having verified the SDP offer as described in Section 6.2.2, the answerer shall perform the following steps to generate the SDP answer.

For each a=rid line that hasn't been discarded by previous processing:

  1. The value of the direction field gets flipped: send becomes recv, and recv becomes send. 🔄
  2. The answerer MAY choose to modify specific a=rid restriction values in the answer SDP. If it does, the modified value MUST be more restrictive than the one in the offer. The answer MUST NOT include any restrictions that weren't present in the offer. 🤏
  3. The answerer MUST NOT modify the rid-id present in the offer. 🔒
  4. If the a=rid line contains a pt=, the answerer is allowed to discard one or more media formats from a given a=rid line. If it discards all the media formats, the answerer MUST discard the entire a=rid line. If the offer didn't contain a pt= for a given a=rid line, the answer MUST NOT contain a pt= in the corresponding line. 🙅
  5. If the answerer can't support the payload configuration in a given a=rid line with a direction of recv in the offer, the answerer MUST discard that a=rid line. This includes cases where the answerer doesn't understand one or more of the restrictions in an a=rid line with a direction of recv. 🥲

Note: If the answerer uses different PT values to represent a codec than the offerer did, the a=rid values in the answer use the PT values that are in its answer. 📝

6.4. Offerer Processing of the SDP Answer 🍓

The offerer SHALL follow these steps when processing the answer:

  1. The offerer matches the a=rid line in the answer to the a=rid line in the offer using the rid-id. If no matching line can be found in the offer, the a=rid line is ignored. 🔎
  2. If the answer contains any restrictions that weren't in the offer, the offerer SHALL discard the a=rid line. ❌
  3. If the restrictions have changed between the offer and the answer, the offerer MUST ensure the modifications are more restrictive than they were in the original offer and that they can be supported. If not, the offerer SHALL discard the a=rid line. 🧾
  4. If the a=rid line in the answer contains a pt= but the offer didn't, the offerer SHALL discard the a=rid line. 🫠
  5. If the a=rid line in the answer contains a pt= and the offer did too, the offerer checks that the list of payload types is a subset of those sent in the matching a=rid line in the offer. This matching has to be done semantically rather than on literal PT values, since the remote end may not be using symmetric PTs. For this comparison, for each PT on the a=rid line in the answer, the offerer looks up the matching a=rtpmap and a=fmtp lines in the answer. Then it searches the pt= values in the offer and tries to find one with an equivalent set of a=rtpmap and a=fmtp lines in the offer. If every PT in the answer can be matched, the pt= values pass validation. Otherwise, validation fails. If it fails, the offerer SHALL discard the a=rid line. Heads up: this semantic comparison actually requires understanding what the codec parameters mean, not a rote byte-by-byte comparison of their values. 🧠
  6. If the a=rid line contains a pt=, the offerer verifies that the attribute values in the a=rid line are consistent with the corresponding codecs and their other parameters. See Section 8 for more detail. If the a=rid restrictions are incompatible with the other codec properties, the offerer SHALL discard the a=rid line. 🚧
  7. The offerer verifies that the restrictions are consistent with at least one of the codecs used with the RTP stream. If the a=rid line contains a pt=, that's the list of such codecs; otherwise, the list comes from the associated m= line. See Section 8 for more detail. If the a=rid restrictions are incompatible with the other codec properties for all codecs, the offerer SHALL discard the a=rid line. 🫥

Any a=rid line in the offer that wasn't matched by step 1 above has been discarded by the answerer and isn't part of the negotiated restrictions on an RTP stream. The offerer MAY still apply any restrictions it indicated in an a=rid line with a direction field of send, but it isn't required to. 💖

There are several ways an offer can contain a media section with a=rid lines while the matching media section in the response doesn't. That includes cases where the answerer doesn't support a=rid at all, or doesn't support the indicated restrictions. In those situations, the offerer MUST be prepared to receive a media stream with no restrictions applied. 🌈

6.5. Modifying the Session 🌷

Offers and answers inside an existing session follow the same rules as initial session negotiation. Such an offer MAY propose a change in the number of RIDs in use. To avoid race conditions with media, any RIDs with proposed changes SHOULD use a new ID rather than reusing one from the previous offer/answer exchange. RIDs without proposed changes SHOULD reuse the ID from the previous exchange. 🔁

7. Use with Declarative SDP 🧚

This document doesn't define the use of a RID in declarative SDP. If concrete use cases for RID in declarative SDP come up in the future, we expect additional specifications will address them. 🔮

8. Interaction with Other Techniques 💜

Historically, a bunch of other approaches have been defined for restricting media streams via SDP. These include:

  • Codec-specific configuration set via format parameters (a=fmtp) -- for example, the H.264 "max-fs" format parameter [RFC6184]
  • Size restrictions imposed by the a=imageattr attribute [RFC6236]

When the mechanism in this document is used alongside these other restricting mechanisms, it's meant to impose additional restrictions beyond the ones communicated through other techniques. ✨

In an offer, this means a=rid lines, combined with other restrictions on the media stream, are expected to result in a non-empty intersection. For example, if image attributes say a PT has a minimum width of 640, then specifying max-width=320 in an a=rid line applied to that PT is nonsensical. According to the rules in Section 6.2.2, the recipient will ignore that a=rid line. 🙈

In an answer, the a=rid lines, combined with the other restrictions on the media stream, are also expected to result in a non-empty intersection. If the implementation generating an answer wants to restrict a property of the stream below what other parameters allow (e.g., those in a=fmtp or a=imageattr), its only recourse is to discard the a=rid line altogether, as described in Section 6.3. If it instead tries to restrict the stream beyond what other mechanisms allow, the offerer will ignore the corresponding a=rid line, as described in Section 6.4. 🫶

The following subsections show these interactions using commonly used video codecs. These descriptions are illustrative of the interaction principles above and are not normative. 🎬

8.1. Interaction with VP8 Format Parameters 🦋

[RFC7741] defines two format parameters for the VP8 codec. Both correspond to restrictions on receiver capabilities and never indicate sending restrictions. 👀

8.1.1. max-fr - Maximum Frame Rate 🎞️

The VP8 max-fr format parameter corresponds to the max-fps restriction defined in this specification. If an RTP sender is generating a stream using a format defined with this format parameter, and the sending restrictions defined via a=rid include a max-fps parameter, then the sent stream will conform to the smaller of the two values. 👑

8.1.2. max-fs - Maximum Frame Size, in VP8 Macroblocks ✨

okay bestie, the VP8 "max-fs" format parameter is basically the twin of the "max-fs" restriction we defined in this doc 👯‍♀️ The conversion factor is pixels per macroblock, which is typically 256. 🧩 So if an RTP sender is making a stream with a format that has this fmtp param, and the "a=rid" sending restrictions also include "max-fs", the stream has to follow whichever number is smaller 💅 That means the pixels per frame can't go over:

     min(rid_max_fs, fmtp_max_fs * macroblock_size)

This fmtp param also affects max-height and max-width 👑 Section 6.1 of [RFC7741] says the width and height of the frame in macroblocks need to be less than int(sqrt(fmtp_max_fs * 8)). So the max width of a transmitted stream gets capped at:

     min(rid_max_width, int(sqrt(fmtp_max_fs * 8)) * macroblock_width)

And the stream's height gets capped the same way:

     min(rid_max_height, int(sqrt(fmtp_max_fs * 8)) * macroblock_height)

8.2. Interaction with H.264 Format Parameters 🎀

[RFC6184] defines format parameters for the H.264 video codec 🦋 Most of them are not codec-independent restrictions, though. Here's the list:

  • deint-buf-cap
  • in-band-parameter-sets
  • level-asymmetry-allowed
  • max-rcmd-nalu-size
  • max-cpb
  • max-dpb
  • packetization-mode
  • redundant-pic-cap
  • sar-supported
  • sar-understood
  • sprop-deint-buf-req
  • sprop-init-buf-time
  • sprop-interleaving-depth
  • sprop-level-parameter-sets
  • sprop-max-don-diff
  • sprop-parameter-sets
  • use-level-src-parameter-sets

Note that max-cpb and max-dpb do correspond to restrictions on the stream 🙈 but they're super specific to how the H.264 codec works, so they don't have codec-independent equivalents.

The [RFC6184] codec format parameters covered in the following sections correspond to restrictions on receiver capabilities 🧚 and they never indicate sending restrictions.

8.2.1. profile-level-id and max-recv-level - Negotiated Subprofile 🌸

These parameters include a "level" indicator, which acts as an index into Table A-1 of [H264]. That table has a bunch of parameters, several of which match restrictions in this document. [RFC6184] also defines format parameters for H.264 that can bump up the maximum values the negotiated level gives you. The next sections explain how these parameters interact with the restrictions in this document. In every case, the H.264 parameters being talked about are the maximum of what [H264] Table A-1 says and what the matching "a=fmtp" line says. 💜

8.2.2. max-br / MaxBR - Maximum Video Bitrate 🍓

The H.264 "MaxBR" parameter (and its equivalent "max-br" format parameter) lines up with the "max-bps" restriction defined in this specification, using a conversion factor of 1000 or 1200. See [RFC6184] for which factor gets used in which situations 🧐

If an RTP sender is generating a stream using a format defined with this format parameter, and the sending restrictions defined via "a=rid" include a "max-fps" parameter, then the sent stream will conform to the smaller of the two values -- that is:

     min(rid_max_br, h264_MaxBR * conversion_factor)

8.2.3. max-fs / MaxFS - Maximum Frame Size, in H.264 Macroblocks 🌈

The H.264 "MaxFs" parameter (and its equivalent "max-fs" format parameter) roughly lines up with the "max-fs" restriction defined in this document, using a conversion factor of 256 (the number of pixels per macroblock) 💖

If an RTP sender is generating a stream using a format defined with this format parameter, and the sending restrictions defined via "a=rid" include a "max-fs" parameter, then the sent stream will conform to the smaller of the two values -- that is:

     min(rid_max_fs, h264_MaxFs * 256)

8.2.4. max-mbps / MaxMBPS - Maximum Macroblock Processing Rate ⭐

The H.264 "MaxMBPS" parameter (and its equivalent "max-mbps" format parameter) roughly lines up with the "max-pps" restriction defined in this document, using a conversion factor of 256 (the number of pixels per macroblock) 🔥

If an RTP sender is generating a stream using a format defined with this format parameter, and the sending restrictions defined via "a=rid" include a "max-pps" parameter, then the sent stream will conform to the smaller of the two values -- that is:

     min(rid_max_pps, h264_MaxMBPS * 256)

8.2.5. max-smbps - Maximum Decoded Picture Buffer 🧠

The H.264 "max-smbps" format parameter works the same way as "max-mbps", just under the hypothetical assumption that all macroblocks are static macroblocks 🙃 It gets handled by applying the conversion factor described in Section 8.1 of [RFC6184], and then the result of that conversion gets applied as described in Section 8.2.4.

8.3. Redundancy Formats and Payload Type Restrictions 🦋

Section 4 says that redundancy formats using redundancy RTP streams bind the redundancy RTP stream to the source RTP stream, either with the RepairedRtpStreamId SDES item or with other mechanisms. BUT there are redundancy RTP payload formats where the redundancy ends up inside the source RTP stream 😭 An example is "RTP Payload for Redundant Audio Data" [RFC2198], which wraps one source stream together with one or more redundancy streams in the same RTP payload.

Formats that define the source and redundancy encodings as regular RTP payload types need a bit more thought about how the "a=rid" restrictions get defined. The "pt=" parameter on an "a=rid" line can say whether the redundancy RTP payload type and/or the individual source RTP payload type(s) are part of the restriction ✨

Here's an example (SDP excerpt):

      m=audio 49200 RTP/AVP 97 98 99 100 101 102
      a=mid:foo
      a=rtpmap:97 G711/8000
      a=rtpmap:98 LPC/8000
      a=rtpmap:99 OPUS/48000/1
      a=rtpmap:100 RED/8000/1
      a=rtpmap:101 CN/8000
      a=rtpmap:102 telephone-event/8000
      a=fmtp:99 useinbandfec=1; usedtx=0
      a=fmtp:100 97/98
      a=fmtp:102 0-15
      a=ptime:20
      a=maxptime:40
      a=rid:5 send pt=99,102;max-br=64000
      a=rid:6 send pt=100,97,101,102

The RID with ID=6 restricts its payload types to 100 (the redundancy format), 97 (G.711), 101 (Comfort Noise), and 102 (dual-tone multi-frequency (DTMF) tones) 💅 So RID 6 can carry the Redundant Audio Data (RED) format wrapping encodings of the source media stream using payload types 97 and 98, or it can carry 97 without RED encapsulation, Comfort Noise, or DTMF tones. Payload type 98 isn't in the RID, so it can only be sent as redundancy info inside RED encapsulation 🥹 And if 97 were removed from the pt parameter, that would mean payload types 97 and 98 are only allowed through RED encapsulation. Lowkey tricky, but so cool 🧚

9. Format Parameters for Future Payloads 🔮

Registrations of future RTP payload format specifications that define media types with parameters matching the RID restrictions in this memo SHOULD name those parameters in a way that matches the names of those RID restrictions. They also SHOULD explicitly state which media-type parameters are restricted by which RID restrictions 💜

10. Formal Grammar 📜

This section gives a formal Augmented Backus-Naur Form (ABNF) [RFC5234] grammar, using the case-sensitive extensions described in [RFC7405], for each of the new media and "a=rid" attributes defined in this document 🫶

   rid-syntax        = %s"a=rid:" rid-id SP rid-dir
                       [ rid-pt-param-list / rid-param-list ]

   rid-id            = 1*(alpha-numeric / "-" / "_")

   alpha-numeric     = < as defined in [RFC4566] >

   rid-dir           = %s"send" / %s"recv"

   rid-pt-param-list = SP rid-fmt-list *(";" rid-param)

   rid-param-list    = SP rid-param *(";" rid-param)

   rid-fmt-list      = %s"pt=" fmt *( "," fmt )

   fmt               = < as defined in [RFC4566] >

   rid-param         = rid-width-param
                       / rid-height-param
                       / rid-fps-param
                       / rid-fs-param
                       / rid-br-param
                       / rid-pps-param
                       / rid-bpp-param
                       / rid-depend-param
                       / rid-param-other

   rid-width-param   = %s"max-width" [ "=" int-param-val ]

   rid-height-param  = %s"max-height" [ "=" int-param-val ]

   rid-fps-param     = %s"max-fps" [ "=" int-param-val ]

   rid-fs-param      = %s"max-fs" [ "=" int-param-val ]

   rid-br-param      = %s"max-br" [ "=" int-param-val ]

   rid-pps-param     = %s"max-pps" [ "=" int-param-val ]

   rid-bpp-param     = %s"max-bpp" [ "=" float-param-val ]

   rid-depend-param  = %s"depend=" rid-list

   rid-param-other   = 1*(alpha-numeric / "-") [ "=" param-val ]

   rid-list          = rid-id *( "," rid-id )

   int-param-val     = 1*DIGIT

   float-param-val   = 1*DIGIT "." 1*DIGIT

   param-val         = *(%x20-3A / %x3C-7E)
                       ; Any printable character except semicolon

11. SDP Examples 🎀

Note: See [RFC8853] for examples of RID used in simulcast scenarios 🌟

11.1. Many Bundled Streams Using Many Codecs 🎀

okay so picture this: the offerer is ready for literally everything 💅 For audio it's Opus, G.722, G.711, and DTMF. For video it's VP8, VP9, H.264 (CBP/CHP, mode 0/1), H.264-SVC (SCBP/SCHP), and H.265 (MP/M10P). An 8-way video call (to a mixer) totally works, meaning you send 1 and receive 7 video streams, by offering 7 video media sections. That's 1 sendrecv at max resolution and 6 recvonly at smaller resolutions, all bundled on the same port, using 3 different resolutions. The resolutions look like this:

  • 1 receive stream at 720p for the active speaker 👑
  • 2 receive streams at 360p for the prior 2 active speakers 🌸
  • 4 receive streams at 180p for everyone else in the call 🧚
NOTE: the SDP below skips a few lines to keep the example short and focused, as shown by either the "..." or the comments inserted. Don't panic, it's on purpose 🙈

Here's the offer for this scenario 💖

...
m=audio 10000 RTP/SAVPF 96 9 8 0 123
a=rtpmap:96 OPUS/48000
a=rtpmap:9 G722/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:123 telephone-event/8000
a=mid:a1
...
m=video 10000 RTP/SAVPF 98 99 100 101 102 103 104 105 106 107
a=extmap 1 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=rtpmap:98 VP8/90000
a=fmtp:98 max-fs=3600; max-fr=30
a=rtpmap:99 VP9/90000
a=fmtp:99 max-fs=3600; max-fr=30
a=rtpmap:100 H264/90000
a=fmtp:100 profile-level-id=42401f; packetization-mode=0
a=rtpmap:101 H264/90000
a=fmtp:101 profile-level-id=42401f; packetization-mode=1
a=rtpmap:102 H264/90000
a=fmtp:102 profile-level-id=640c1f; packetization-mode=0
a=rtpmap:103 H264/90000
a=fmtp:103 profile-level-id=640c1f; packetization-mode=1
a=rtpmap:104 H264-SVC/90000
a=fmtp:104 profile-level-id=530c1f
a=rtpmap:105 H264-SVC/90000
a=fmtp:105 profile-level-id=560c1f
a=rtpmap:106 H265/90000
a=fmtp:106 profile-id=1; level-id=93
a=rtpmap:107 H265/90000
a=fmtp:107 profile-id=2; level-id=93
a=sendrecv
a=mid:v1 (max resolution)
a=rid:1 send max-width=1280;max-height=720;max-fps=30
a=rid:2 recv max-width=1280;max-height=720;max-fps=30
...
m=video 10000 RTP/SAVPF 98 99 100 101 102 103 104 105 106 107
a=extmap 1 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
...same rtpmap/fmtp as above...
a=recvonly
a=mid:v2 (medium resolution)
a=rid:3 recv max-width=640;max-height=360;max-fps=15
...
m=video 10000 RTP/SAVPF 98 99 100 101 102 103 104 105 106 107
a=extmap 1 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
...same rtpmap/fmtp as above...
a=recvonly
a=mid:v3 (medium resolution)
a=rid:3 recv max-width=640;max-height=360;max-fps=15
...
m=video 10000 RTP/SAVPF 98 99 100 101 102 103 104 105 106 107
a=extmap 1 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
...same rtpmap/fmtp as above...
a=recvonly
a=mid:v4 (small resolution)
a=rid:4 recv max-width=320;max-height=180;max-fps=15
...
m=video 10000 RTP/SAVPF 98 99 100 101 102 103 104 105 106 107
a=extmap 1 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
...same rtpmap/fmtp as above...
...same rid:4 as above for mid:v5,v6,v7 (small resolution)...
...

11.2. Scalable Layers 🦋

okay this one is so good ✨ adding scalable layers to a session inside a multiparty conference gives a selective forwarding unit (SFU) extra flexibility. It can selectively forward packets from a source that best matches the bandwidth and capabilities of each receiver. Scalable encodings have dependencies between layers, which is the big difference from independent simulcast streams. RIDs can express those dependencies using the "depend" restriction 🔥 In the example below, the highest resolution is offered to be sent as 2 scalable temporal layers, using Multiple RTP Streams on a Single Media Transport (MRST). For more on simulcast, check out [RFC8853] 🌈

Offer:

...
m=audio ...same as previous example ...
...
m=video ...same as previous example ...
...same rtpmap/fmtp as previous example ...
a=sendrecv
a=mid:v1 (max resolution)
a=rid:0 send max-width=1280;max-height=720;max-fps=15
a=rid:1 send max-width=1280;max-height=720;max-fps=30;depend=0
a=rid:2 recv max-width=1280;max-height=720;max-fps=30
a=rid:5 send max-width=640;max-height=360;max-fps=15
a=rid:6 send max-width=320;max-height=180;max-fps=15
a=simulcast: send rid=0;1;5;6 recv rid=2
...
...same m=video sections as previous example for mid:v2-v7...
...

12. IANA Considerations 📋

this one's the paperwork bit, but it matters! This specification updates [RFC4855] to give extra guidance on how to pick Format Parameter (fmtp) names and how they relate to RID restrictions 💜

12.1. New SDP Media-Level Attribute 🍓

this document defines "rid" as an SDP media-level attribute. IANA has registered it under "Session Description Protocol (SDP) Parameters" under "att-field (media level only)" ✨

The "rid" attribute identifies the properties of an RTP stream within an RTP session. Its format is defined in Section 10.

Here's the formal registration info:

  • Contact name, email address, and telephone number: IETF MMUSIC Working Group, mmusic@ietf.org, +1 510 492 4080
  • Attribute name (as it will appear in SDP): rid
  • Long-form attribute name in English: Restriction Identifier
  • Type of attribute (session level, media level, or both): Media Level
  • Whether the attribute value is subject to the charset attribute: The attribute is not dependent on charset.
  • A one-paragraph explanation of the purpose of the attribute: The "rid" SDP attribute is used to unambiguously identify the RTP streams within an RTP session and restrict the streams' payload format parameters in a codec-agnostic way beyond what is provided with the regular payload types.
  • A specification of appropriate attribute values for this attribute: Valid values are defined by the ABNF in RFC 8851
  • Multiplexing (Mux) Category: SPECIAL

12.2. Registry for RID-Level Parameters 💅

this spec creates a brand new IANA registry called "RID Attribute Parameters" inside the SDP parameters registry. The "a=rid" restrictions MUST be registered with IANA and documented under the same rules as SDP session-level and media-level attributes, as specified in [RFC4566] 🙌

Parameters for "a=rid" lines that change the nature of encoded media MUST be set up so that applying the change still gives a stream that complies with the other parameters affecting the media. Basically, restrictions always have to narrow the definition to a subset of what's allowed, and never expand it 🫶

New restriction registrations go through the "Specification Required" policy of [RFC8126]. The registration MUST contain the RID parameter name and a reference to the matching specification. The spec itself has to include the following info (not all of it shows up in the registry):

  • restriction name (as it will appear in SDP)
  • an explanation of the purpose of the restriction
  • a specification of appropriate attribute values for this restriction
  • an ABNF definition of the restriction

The initial set of "a=rid" restriction names, with definitions in Section 5 of this document, is below 👑

                 +====================+===========+
                 | RID Parameter Name | Reference |
                 +====================+===========+
                 | pt                 | RFC 8851  |
                 +--------------------+-----------+
                 | max-width          | RFC 8851  |
                 +--------------------+-----------+
                 | max-height         | RFC 8851  |
                 +--------------------+-----------+
                 | max-fps            | RFC 8851  |
                 +--------------------+-----------+
                 | max-fs             | RFC 8851  |
                 +--------------------+-----------+
                 | max-br             | RFC 8851  |
                 +--------------------+-----------+
                 | max-pps            | RFC 8851  |
                 +--------------------+-----------+
                 | max-bpp            | RFC 8851  |
                 +--------------------+-----------+
                 | depend             | RFC 8851  |
                 +--------------------+-----------+

                 Table 1: "a=rid" restriction names

It's conceivable that a future document will want RID-level restrictions with string values. Those extensions need to conform to the ABNF defined for rid-param-other. In particular, it means such extensions will need to define escaping mechanisms if they want to allow semicolons, unprintable characters, or byte values greater than 127 in the string 🧚

13. Security Considerations 🔒

⚠️ this section is serious, bestie, so read it carefully 🙈

As with most SDP parameters, failing to provide integrity protection over the "a=rid" attributes gives attackers a way to modify the session in potentially unwanted ways. This could make an implementation send more data than a recipient wants to receive. In general, though, the "a=rid" attribute can only restrict a stream to be a subset of what is otherwise allowable. So modifying the value can't produce a stream with higher bandwidth than what would be sent to an implementation that doesn't support this mechanism 💀

The actual identifiers used for RIDs are expected to be opaque. So they aren't expected to contain information that would be sensitive if third parties observed it 🫣

14. References 📚

14.1. Normative References

okay bestie, these are the core reads you need before anything else makes sense 📚✨

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, https://www.rfc-editor.org/info/rfc2119.

[RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with Session Description Protocol (SDP)", RFC 3264, DOI 10.17487/RFC3264, June 2002, https://www.rfc-editor.org/info/rfc3264.

[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550, July 2003, https://www.rfc-editor.org/info/rfc3550.

[RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session Description Protocol", RFC 4566, DOI 10.17487/RFC4566, July 2006, https://www.rfc-editor.org/info/rfc4566.

[RFC4855] Casner, S., "Media Type Registration of RTP Payload Formats", RFC 4855, DOI 10.17487/RFC4855, February 2007, https://www.rfc-editor.org/info/rfc4855.

[RFC5234] Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, DOI 10.17487/RFC5234, January 2008, https://www.rfc-editor.org/info/rfc5234.

[RFC7405] Kyzivat, P., "Case-Sensitive String Support in ABNF", RFC 7405, DOI 10.17487/RFC7405, December 2014, https://www.rfc-editor.org/info/rfc7405.

[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, https://www.rfc-editor.org/info/rfc8174.

[RFC8852] Roach, A.B., Nandakumar, S., and P. Thatcher, "RTP Stream Identifier Source Description (SDES)", RFC 8852, DOI 10.17487/RFC8852, January 2021, https://www.rfc-editor.org/info/rfc8852.

14.2. Informative References

these ones are the extra-credit reading, the deep-dive stuff that's still super useful 🦋💜

[H264] International Telecommunication Union, "Advanced video coding for generic audiovisual services", ITU-T Recommendation H.264, June 2019, https://www.itu.int/rec/T-REC-H.264.

[RFC2198] Perkins, C., Kouvelas, I., Hodson, O., Hardman, V., Handley, M., Bolot, J.C., Vega-Garcia, A., and S. Fosse-Parisis, "RTP Payload for Redundant Audio Data", RFC 2198, DOI 10.17487/RFC2198, September 1997, https://www.rfc-editor.org/info/rfc2198.

[RFC4588] Rey, J., Leon, D., Miyazaki, A., Varsa, V., and R. Hakenberg, "RTP Retransmission Payload Format", RFC 4588, DOI 10.17487/RFC4588, July 2006, https://www.rfc-editor.org/info/rfc4588.

[RFC5109] Li, A., Ed., "RTP Payload Format for Generic Forward Error Correction", RFC 5109, DOI 10.17487/RFC5109, December 2007, https://www.rfc-editor.org/info/rfc5109.

[RFC6184] Wang, Y.-K., Even, R., Kristensen, T., and R. Jesup, "RTP Payload Format for H.264 Video", RFC 6184, DOI 10.17487/RFC6184, May 2011, https://www.rfc-editor.org/info/rfc6184.

[RFC6236] Johansson, I. and K. Jung, "Negotiation of Generic Image Attributes in the Session Description Protocol (SDP)", RFC 6236, DOI 10.17487/RFC6236, May 2011, https://www.rfc-editor.org/info/rfc6236.

[RFC7656] Lennox, J., Gross, K., Nandakumar, S., Salgueiro, G., and B. Burman, Ed., "A Taxonomy of Semantics and Mechanisms for Real-Time Transport Protocol (RTP) Sources", RFC 7656, DOI 10.17487/RFC7656, November 2015, https://www.rfc-editor.org/info/rfc7656.

[RFC7741] Westin, P., Lundin, H., Glover, M., Uberti, J., and F. Galligan, "RTP Payload Format for VP8 Video", RFC 7741, DOI 10.17487/RFC7741, March 2016, https://www.rfc-editor.org/info/rfc7741.

[RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 8126, DOI 10.17487/RFC8126, June 2017, https://www.rfc-editor.org/info/rfc8126.

[RFC8285] Singer, D., Desineni, H., and R. Even, Ed., "A General Mechanism for RTP Header Extensions", RFC 8285, DOI 10.17487/RFC8285, October 2017, https://www.rfc-editor.org/info/rfc8285.

[RFC8627] Zanaty, M., Singh, V., Begen, A., and G. Mandyam, "RTP Payload Format for Flexible Forward Error Correction (FEC)", RFC 8627, DOI 10.17487/RFC8627, July 2019, https://www.rfc-editor.org/info/rfc8627.

[RFC8843] Holmberg, C., Alvestrand, H., and C. Jennings, "Negotiating Media Multiplexing Using the Session Description Protocol (SDP)", RFC 8843, DOI 10.17487/RFC8843, January 2021, https://www.rfc-editor.org/info/rfc8843.

[RFC8853] Burman, B., Westerlund, M., Nandakumar, S., and M. Zanaty, "Using Simulcast in Session Description Protocol (SDP) and RTP Sessions", RFC 8853, DOI 10.17487/RFC8853, January 2021, https://www.rfc-editor.org/info/rfc8853.

Acknowledgements 🎀

Many thanks to Cullen Jennings, Magnus Westerlund, and Paul Kyzivat for reviewing. Thanks to Colin Perkins for input on future payload type handling. 🫶

Contributors 💜

The following individuals have contributed significant text to this document. these people are literally the reason this spec slaps 👑

Peter Thatcher Google

Email: pthatcher@google.com

Mo Zanaty Cisco Systems

Email: mzanaty@cisco.com

Suhas Nandakumar Cisco Systems

Email: snandaku@cisco.com

Bo Burman Ericsson

Email: bo.burman@ericsson.com

Byron Campen Mozilla

Email: bcampen@mozilla.com

Author's Address 🦋

Adam Roach (editor) Mozilla

Email: adam@nostrum.com

📦 where this page comes from