RFC 6185
RTP Payload Format for H.264 Reduced-Complexity Decoding Operation (RCDO) Video
RTP Payload Format for H.264 Reduced-Complexity Decoding Operation (RCDO) Video 🎀
okay bestie, this is RFC 6185 🦋 It's a Standards Track document from May 2011 (ISSN 2070-1721), written by T. Kristensen and P. Luthi at TANDBERG 💜. The header doesn't list any RFCs it obsoletes or updates, so this one stands on its own ✨.
Abstract 🌸
okay so this doc gives us an RTP payload format for RCDO (Reduced-Complexity Decoding Operation) for H.264 Baseline profile bitstreams 💅. It's the stuff specced in ITU-T Recommendation H.241, and RCDO cuts down the decoding cost and resource use of video processing. The whole format is built on the H.264 RTP payload format 🎀.
Status of This Memo 🌈
This is an Internet Standards Track document 💖.
It's a product of the Internet Engineering Task Force (IETF), so it represents the consensus of the IETF community. It's been through public review and got approved for publication by the Internet Engineering Steering Group (IESG) 👑. If you want more on Internet Standards, check out Section 2 of RFC 5741 🧚.
For the current status, any errata, and how to send feedback, go to http://www.rfc-editor.org/info/rfc6185 🔥.
Copyright Notice 💜
Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved 🫶.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they're in effect on the publication date. Please actually read these, because they spell out your rights and restrictions 🙈. Any Code Components pulled out of this document have to include the Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions. They come with no warranty, as the Simplified BSD License describes 🍓.
This document may contain material from IETF Documents or IETF Contributions that were published or made publicly available before November 10, 2008. The people who control the copyright in some of that material may not have given the IETF Trust permission to allow modifications outside the IETF Standards Process 😭. Unless you get an adequate license from those copyright holders, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside that process. The one exceptions are formatting it for publication as an RFC or translating it into languages other than English ✨.
Table of Contents 🦋
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Conventions Used in This Document . . . . . . . . . . . . . . 3
3. Media Format Background . . . . . . . . . . . . . . . . . . . 3
4. Payload Format . . . . . . . . . . . . . . . . . . . . . . . . 3
5. Congestion Control Considerations . . . . . . . . . . . . . . 3
6. Payload Format Parameters . . . . . . . . . . . . . . . . . . 3
6.1. Media Type Definition . . . . . . . . . . . . . . . . . . 4
7. Mapping to SDP . . . . . . . . . . . . . . . . . . . . . . . . 19
7.1. Offer/Answer Considerations . . . . . . . . . . . . . . . 20
7.2. Declarative SDP Considerations . . . . . . . . . . . . . . 20
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 20
9. Security Considerations . . . . . . . . . . . . . . . . . . . 20
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 21
11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 21
11.1. Normative References . . . . . . . . . . . . . . . . . . . 21
11.2. Informative References . . . . . . . . . . . . . . . . . . 211. Introduction 🦋
so ITU-T Recommendation H.241 [3] specs a Reduced-Complexity Decoding Operation (RCDO) for use with H.264 [2] Baseline profile bitstreams 🌸. It also specs a bitstream constraint that goes with RCDO, plus a way to signal RCDO inside the bitstream. The RCDO signaling tells you the bitstream follows that constraint, and that the decoder shall apply the RCDO decoding process to it 💅.
RCDO for H.264 is how you get higher resolutions at the same high frame rates we already use in current implementations 🚀. The trick is that it cuts the processing requirements, which lowers the decoding cost and resource consumption of the video processing. It's giving efficiency 😭.
This document defines media type parameters and lets them be used in systems that signal with the Session Description Protocol (SDP) [8] 🎀.
2. Conventions Used in This Document 💖
the key words are "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL". we're keeping them in capitals here, and they're to be read as described in [4] 🫶.
3. Media Format Background 🧚
The Reduced-Complexity Decoding Operation (RCDO) for H.264 Baseline profile bitstreams is specified in Annex B of H.241 [3] 💜. RCDO is its own separate H.264 mode, and it's distinct from any profile defined in H.264. An RCDO bitstream obeys all the constraints of the Baseline profile, so it's not a sneaky new profile 👑.
The media format is based on the H.264 RTP payload format, which is specified in RFC 6184 [1]. RFC 6184 is the basis for this document, so it gets referenced a bunch 🌈.
To signal extra H.264 modes, Table 8-13 of H.241 [3] specifies an AdditionalModesSupported parameter. Right now, RCDO is the only additional mode defined 🍓.
Informative note: Other additional modes may be defined in the future. H.264 additional modes may or may not be distinct from the profiles in H.264.
A separate media subtype, named H264-RCDO, is defined to keep backward compatibility with deployed H.264 implementations 🌸.
4. Payload Format ✨
The payload format defined in Section 5 of RFC 6184 [1] SHALL be used. That covers the RTP header usage and the payload format from RFC 6184. You can find examples of typical RTP packets in RFC 6184 too 🎀.
5. Congestion Control Considerations 🌊
Congestion control for RTP SHALL be used in accordance with RFC 3550 [6] and with any applicable RTP profile, for example RFC 3551 [7] 💅. If best-effort service is being used, users of this payload format SHALL monitor packet loss to make sure the packet loss rate stays within acceptable parameters 😭.
6. Payload Format Parameters 💜
This RTP payload format is identified with the H264-RCDO media subtype. It's registered in accordance with RFC 4855 [10], and it uses the template from RFC 4288 [13] 🦋.
6.1. Media Type Definition 🎀
Informative note: The media subtype definition for H264-RCDO is based on the definition of the H264 media subtype in Section 8.1 of RFC 6184 [1]. Except for the profile-level-id parameter, which gets new semantics below, the optional parameters are copied from RFC 6184 [1] so the media subtype registration is complete and self-contained for IANA. The references are updated to match this document's numbering.
The media subtype for RCDO for H.264 has been allocated from the IETF tree ✨.
Type name: video
Subtype name: H264-RCDO
Required parameters:
rate: this one tells you the RTP timestamp clock rate. The rate value MUST be 90000 🌸.
Optional parameters:
profile-level-id: this is a base16 RFC 4648 [9] (hexadecimal) representation of three bytes from the sequence parameter set NAL unit, as specified in H.264 [2]. Those bytes are: 1) profile_idc, 2) a byte called profile-iop, which is made of constraint_set0_flag, constraint_set1_flag, constraint_set2_flag, constraint_set3_flag, constraint_set4_flag, constraint_set5_flag, and reserved_zero_2bits in bit-significance order starting from the most-significant bit, and 3) level_idc 💖. Heads up: reserved_zero_2bits is required to be equal to 0 in H.264 [2], but ITU-T or ISO/IEC might specify other values for it in the future 👀.
The profile-level-id parameter shows the default sub-profile (the subset of coding tools that may have been used to generate the stream, or that the receiver supports) and the default level of the stream or the receiver 🧚.
RCDO is distinct from any profile, which means the profile value 0 (no profile) and the profile_idc byte of the profile-level-id parameter are both equal to 0. An RCDO bitstream MUST obey all the constraints of the Baseline profile. So only constraint_set0_flag is equal to 1 in the profile-iop part of the profile-level-id parameter, and the remaining bits are set to 0 🌈.
If the profile-level-id parameter describes the properties of a NAL unit stream, it means that to decode the stream, the minimum subset of coding tools a decoder has to support is the default sub-profile, and the lowest level the decoder has to support is the default level 🍓.
If the profile-level-id parameter is used for capability exchange or session setup, it shows the subset of coding tools (which is equal to the default sub-profile) that the codec supports for both receiving and sending. If max-recv-level is not present, the default level from profile-level-id is the highest level the codec wishes to support. If max-recv-level is present, it's the highest level the codec supports for receiving. For either receiving or sending, all levels lower than the highest supported level MUST also be supported 🫶.
For example, if a codec supports level 1.3, the profile-level-id becomes 00800d. Here 00 means the "no profile" value, 80 means the Baseline profile constraints, and 0d means level 1.3. When level 2.1 is supported, the profile-level-id becomes 008015 💅.
If no profile-level-id is present, level 1 (equivalent to profile-level-id 00800a) MUST be implied 😭.
Informative note: The definitions of the remaining optional parameters below are copied verbatim from Section 8.1 of RFC 6184 [1]. Only the references are updated to match this document's numbering.
max-recv-level: this parameter MAY be used to show the highest level a receiver supports when that level is higher than the default level (the one indicated by profile-level-id) 🔥. Its value is a base16 (hexadecimal) representation of the two bytes right after the profile_idc syntax element in the sequence parameter set NAL unit specified in H.264 [2]. Those two bytes are profile-iop (as defined above) and level_idc. If the level_idc byte of max-recv-level is equal to 11 and bit 4 of the profile-iop byte of max-recv-level is equal to 1, OR if the level_idc byte of max-recv-level is equal to 9 and bit 4 of the profile-iop byte is equal to 0, then the highest level the receiver supports is Level 1b. Otherwise, the highest level the receiver supports is equal to the level_idc byte of max-recv-level divided by 10 ✨.
max-recv-level MUST NOT be present if the highest level the receiver supports is not higher than the default level 🌸.
max-mbps, max-smbps, max-fs, max-cpb, max-dpb, and max-br: okay bestie, these six are the receiver showing off what it can handle 💅✨ They MAY be used to signal the capabilities of a receiver implementation. They MUST NOT be used for any other purpose 🙅♀️ The highest level in the profile-level-id or max-recv-level parameter MUST be one the receiver is fully capable of supporting. max-mbps, max-smbps, max-fs, max-cpb, max-dpb, and max-br MAY be used to show capabilities that go beyond the required capabilities of the signaled highest level, as specified below 🎀
When more than one parameter from the set (max-mbps, max-smbps, max-fs, max-cpb, max-dpb, max-br) is present, the receiver MUST support all of the signaled capabilities at the same time 🔥 For example, if both max-mbps and max-br are there, the signaled highest level with the frame rate and bitrate both extended is supported. That means the receiver can decode NAL unit streams where the macroblock processing rate is up to max-mbps (inclusive), the bitrate is up to max-br (inclusive), the coded picture buffer size is derived as described in the semantics of max-br below, and everything else matches the highest level in the profile-level-id or max-recv-level parameter 🦋
If a receiver can support every property of Level A, the highest level in the profile-level-id or max-recv-level parameter MUST be Level A (i.e., MUST NOT be lower than Level A). In other words, a receiver MUST NOT signal values of max-mbps, max-fs, max-cpb, max-dpb, and max-br that together meet the requirements of a higher level than the highest level in the profile-level-id or max-recv-level parameter 🙈
Informative note: When the OPTIONAL media type parameters are used to signal the properties of a NAL unit stream,max-mbps,max-smbps,max-fs,max-cpb,max-dpb, andmax-brare not present, and the value ofprofile-level-idmust always be such that the NAL unit stream complies fully with the specified profile and level 🧚
max-mbps: the value of max-mbps is an integer for the maximum macroblock processing rate, in units of macroblocks per second 🍓 The max-mbps parameter says the receiver can decode video at a higher rate than the signaled highest level in the profile-level-id or max-recv-level parameter requires. When max-mbps is signaled, the receiver MUST be able to decode NAL unit streams that conform to the signaled highest level, except that the MaxMBPS value in Table A-1 of H.264 [2] for that signaled highest level is replaced with the value of max-mbps. The value of max-mbps MUST be greater than or equal to the value of MaxMBPS given in Table A-1 of H.264 [2] for the highest level. Senders MAY use this knowledge to send pictures of a given size at a higher picture rate than the signaled highest level indicates 💜
max-smbps: the value of max-smbps is an integer for the maximum static macroblock processing rate, in units of static macroblocks per second, under the hypothetical assumption that all macroblocks are static macroblocks 🌸 When max-smbps is signaled, the MaxMBPS value in Table A-1 of H.264 [2] should be replaced with the result of the following computation:
- If the parameter
max-mbpsis signaled, set a variable MaxMacroblocksPerSecond to the value ofmax-mbps. Otherwise, set MaxMacroblocksPerSecond equal to the value of MaxMBPS in Table A-1 of H.264 [2] for the signaled highest level conveyed in the value of theprofile-level-idparameter or themax-recv-levelparameter. - Set a variable P_non-static to the proportion of non-static macroblocks in picture n.
- Set a variable P_static to the proportion of static macroblocks in picture n.
- The value of MaxMBPS in Table A-1 of H.264 [2] should be considered by the encoder to be equal to:
MaxMacroblocksPerSecond * max-smbps / (P_non-static * max-smbps
+ P_static * MaxMacroblocksPerSecond)The encoder should recompute this value for each picture 🤓 The value of max-smbps MUST be greater than or equal to the value of MaxMBPS given explicitly as the value of the max-mbps parameter, or implicitly in Table A-1 of H.264 [2] for the signaled highest level. Senders MAY use this knowledge to send pictures of a given size at a higher picture rate than the signaled highest level indicates 🦋
max-fs: the value of max-fs is an integer for the maximum frame size, in units of macroblocks 👑 The max-fs parameter says the receiver can decode bigger picture sizes than the signaled highest level in the profile-level-id or max-recv-level parameter requires. When max-fs is signaled, the receiver MUST be able to decode NAL unit streams that conform to the signaled highest level, except that the MaxFS value in Table A-1 of H.264 [2] for that signaled highest level is replaced with the value of max-fs. The value of max-fs MUST be greater than or equal to the value of MaxFS given in Table A-1 of H.264 [2] for the highest level. Senders MAY use this knowledge to send larger pictures at a proportionally lower frame rate than the signaled highest level indicates 🌈
max-cpb: the value of max-cpb is an integer for the maximum coded picture buffer size, in units of 1000 bits for the VCL HRD parameters and in units of 1200 bits for the NAL HRD parameters 🫶 Note that this parameter does not use the units of cpbBrVclFactor and cpbBrNALFactor (see Table A-1 of H.H.264 [2]). The max-cpb parameter says the receiver has more memory than the minimum coded picture buffer memory the signaled highest level in the profile-level-id or max-recv-level parameter requires. When max-cpb is signaled, the receiver MUST be able to decode NAL unit streams that conform to the signaled highest level, except that the MaxCPB value in Table A-1 of H.264 [2] for that signaled highest level is replaced with the value of max-cpb (after taking cpbBrVclFactor and cpbBrNALFactor into consideration when needed). The value of max-cpb (after taking cpbBrVclFactor and cpbBrNALFactor into consideration when needed) MUST be greater than or equal to the value of MaxCPB given in Table A-1 of H.264 [2] for the highest level. Senders MAY use this knowledge to construct coded video streams with more bitrate variation than can be achieved with the MaxCPB value in Table A-1 of H.264 [2] 💅
Informative note: The coded picture buffer is used in the hypothetical reference decoder (Annex C of H.264). The use of the hypothetical reference decoder is recommended in H.264 encoders to verify that the produced bitstream conforms to the standard and to control the output bitrate. Thus, the coded picture buffer is conceptually independent of any other potential buffers in the receiver, including de-interleaving and de-jitter buffers. The coded picture buffer need not be implemented in decoders as specified in Annex C of H.264, but rather standard-compliant decoders can have any buffering arrangements provided that they can decode standard-compliant bitstreams. Thus, in practice, the input buffer for a video decoder can be integrated with de-interleaving and de-jitter buffers of the receiver 🧠
max-dpb: the value of max-dpb is an integer for the maximum decoded picture buffer size, in units of 8/3 macroblocks 🍬 The max-dpb parameter says the receiver has more memory than the minimum decoded picture buffer memory the signaled highest level in the profile-level-id or max-recv-level parameter requires. When max-dpb is signaled, the receiver MUST be able to decode NAL unit streams that conform to the signaled highest level, except that the MaxDpbMbs value in Table A-1 of H.264 [2] for that signaled highest level is replaced with the value of max-dpb * 3 / 8. As a result, a receiver that signals max-dpb MUST be capable of storing the following number of decoded frames, complementary field pairs, and non-paired fields in its decoded picture buffer:
Min(max-dpb * 3 / 8 / ( PicWidthInMbs * FrameHeightInMbs), 16)Here, PicWidthInMbs and FrameHeightInMbs are defined in H.264 [2].
The value of max-dpb MUST be greater than or equal to the value of MaxDpbMbs * 3 / 8, where the value of MaxDpbMbs is given in Table A-1 of H.264 [2] for the highest level. Senders MAY use this knowledge to construct coded video streams with improved compression 🎀
Informative note: This parameter was added mainly to complement a similar codepoint in the ITU-T Recommendation H.245, so as to make signaling gateway designs easier. The decoded picture buffer stores reconstructed samples. There is no relationship between the size of the decoded picture buffer and the buffers used in RTP, especially de-interleaving and de-jitter buffers.
Informative note: In RFC 3984, which is obsoleted by RFC 6184, the unit of this parameter was 1024 bytes. The unit has been changed to 8/3 macroblocks in this document. The reason for this change was the changes from the 2003 version of the H.264 specification referenced by RFC 3984 to the 2010 version of the H.264 specification referenced by this document, particularly the changes to Table A-1 in the H.264 specification due to the addition of color formats and bit depths not supported earlier. The changed semantics of this parameter keep backward compatibility with RFC 3984 and support all profiles defined in the 2010 version of the H.264 specification 🥹
max-br: the value of max-br is an integer for the maximum video bitrate, in units of 1000 bits per second for the VCL HRD parameters and in units of 1200 bits per second for the NAL HRD parameters 🌟 Note that this parameter does not use the units of cpbBrVclFactor and cpbBrNALFactor (see Table A-1 of H.264 [2]).
The max-br parameter says the video decoder of the receiver can decode video at a higher bitrate than the signaled highest level in the profile-level-id or max-recv-level parameter requires.
When max-br is signaled, the video codec of the receiver MUST be able to decode NAL unit streams that conform to the signaled highest level, with the following exception within the limits specified by the highest level:
- The value of
max-br(after taking cpbBrVclFactor and cpbBrNALFactor into consideration when needed) replaces the MaxBR value in Table A-1 of H.264 [2] for the highest level. - When the
max-cpbparameter is not present, the result of the following formula replaces the value of MaxCPB in Table A-1 of H.264 [2]:(MaxCPB of the signaled level) * max-br / (MaxBR of the signaled highest level). ✨
For example, if a receiver signals capability for Main profile Level 1.2 with max-br equal to 1550, this indicates a maximum video bitrate of 1550 kbits/sec for VCL HRD parameters, a maximum video bitrate of 1860 kbits/sec for NAL HRD parameters, and a CPB size of 4036458 bits (1550000 / 384000 * 1000 * 1000). 🧮💖
The value of max-br (after taking cpbBrVclFactor and cpbBrNALFactor into consideration when needed) MUST be greater than or equal to the value MaxBR given in Table A-1 of H.264 [2] for the signaled highest level. 🔥
Senders MAY use this knowledge to send higher bitrate video as allowed in the level definition of Annex A of H.264 to achieve improved video quality. 🎀
Informative note: This parameter was added primarily to complement a similar codepoint in the ITU-T Recommendation H.245, so as to facilitate signaling gateway designs. The assumption that the network is capable of handling such bitrates at any given time cannot be made from the value of this parameter. In particular, no conclusion can be drawn that the signaled bitrate is possible under congestion control constraints. 🙈
redundant-pic-cap: okay so this one signals what a receiver can do 🦋. When it's equal to 0, the receiver makes no attempt to use redundant coded pictures to fix primary coded pictures that were decoded incorrectly. When it's equal to 0, the receiver also can't use redundant slices, so a sender SHOULD avoid sending redundant slices to save bandwidth 💅. When it's equal to 1, the receiver can decode any such redundant slice that covers a corrupted area in a primary decoded picture (at least partly), so a sender MAY send redundant slices 🌸. When the parameter is not present, a value of 0 MUST be used for redundant-pic-cap. When it's present, the value of redundant-pic-cap MUST be either 0 or 1. 🫶
When the profile-level-id parameter is present in the same signaling as the redundant-pic-cap parameter, and the profile indicated in profile-level-id disallows the use of redundant coded pictures (e.g., Main profile), the value of redundant-pic-cap MUST be equal to 0. When a receiver indicates redundant-pic-cap equal to 0, the received stream SHOULD NOT contain redundant coded pictures. 🚫
Informative note: Even if redundant-pic-cap is equal to 0, the decoder is able to ignore redundant codec pictures provided that the decoder supports a profile (Baseline, Extended) in which redundant coded pictures are allowed. ✨
Informative note: Even if redundant-pic-cap is equal to 1, the receiver may also choose other error concealment strategies to replace or complement decoding of redundant slices. 🧚
sprop-parameter-sets: this one MAY be used to convey any sequence and picture parameter set NAL units (we're calling these the initial parameter set NAL units 🍓) that can be placed in the NAL unit stream to precede any other NAL units in decoding order. The parameter MUST NOT be used to indicate codec capability in any capability exchange procedure. The value is a comma-separated (',') list of base64 RFC 4648 [9] representations of parameter set NAL units, as specified in Sections 7.3.2.1 and 7.3.2.2 of H.264 [2]. Note that the number of bytes in a parameter set NAL unit is typically less than 10, but a picture parameter set NAL unit can contain several hundred bytes. 👑
Informative note: When several payload types are offered in the SDP Offer/Answer model, each with its ownsprop-parameter-setsparameter, the receiver cannot assume that those parameter sets do not use conflicting storage locations (i.e., identical values of parameter set identifiers). Therefore, a receiver should buffer allsprop-parameter-setsand make them available to the decoder instance that decodes a certain payload type. 😭
The sprop-parameter-sets parameter MUST only contain parameter sets that are conforming to the profile-level-id, i.e., the subset of coding tools indicated by any of the parameter sets MUST be equal to the default sub-profile, and the level indicated by any of the parameter sets MUST be equal to the default level. 💜
sprop-level-parameter-sets: this one MAY be used to convey any sequence and picture parameter set NAL units (the initial parameter set NAL units again 🌈) that can be placed in the NAL unit stream to precede any other NAL units in decoding order, and that are associated with one or more levels different than the default level. The parameter MUST NOT be used to indicate codec capability in any capability exchange procedure. 🙅♀️
The sprop-level-parameter-sets parameter contains parameter sets for one or more levels that are different than the default level. All parameter sets associated with one level are clustered and prefixed with a three-byte field that has the same syntax as profile-level-id. This lets the receiver install the parameter sets for one level and discard the rest, which is so iconic 🤌. The three-byte field is named PLId, and all parameter sets associated with one level are named PSL, which has the same syntax as sprop-parameter-sets. Parameter sets for each level are represented in the form of PLId:PSL, i.e., PLId followed by a colon (':') and the base64 RFC 4648 [9] representation of the initial parameter set NAL units for the level. Each pair of PLId:PSLs is also separated by a colon. Note that a PSL can contain multiple parameter sets for that level, separated with commas (','). 🦋
The subset of coding tools indicated by each PLId field MUST be equal to the default sub-profile, and the level indicated by each PLId field MUST be different than the default level. All sequence parameter sets contained in each PSL MUST have the three bytes from profile_idc to level_idc, inclusive, equal to the preceding PLId. ✨
Informative note: This parameter allows for efficient level downgrade or upgrade in SDP Offer/Answer and out-of-band transport of parameter sets simultaneously. 🔥
use-level-src-parameter-sets: this one MAY be used to indicate a receiver capability. The value MAY be equal to either 0 or 1. When the parameter is not present, the value MUST be inferred to be equal to 0. A value of 0 means the receiver does not understand the sprop-level-parameter-sets parameter, does not understand the "fmtp" source attribute as specified in Section 6.3 of RFC 5576 [14], will ignore sprop-level-parameter-sets when present, and will ignore sprop-parameter-sets when conveyed using the "fmtp" source attribute. A value of 1 means the receiver understands the sprop-level-parameter-sets parameter, understands the "fmtp" source attribute as specified in Section 6.3 of RFC 5576 [14], and is capable of using parameter sets contained in the sprop-level-parameter-sets or contained in the sprop-parameter-sets that is conveyed using the "fmtp" source attribute. 🎀
Informative note: An RFC 3984 receiver does not understandsprop-level-parameter-sets,use-level-src-parameter-sets, or the "fmtp" source attribute as specified in Section 6.3 of RFC 5576 [14]. Therefore, during SDP Offer/Answer, an RFC 3984 receiver as the answerer will simply ignoresprop-level-parameter-setswhen present in an offer andsprop-parameter-setsconveyed using the "fmtp" source attribute, as specified in Section 6.3 of RFC 5576 [14]. Assume that the offered payload type was accepted at a level lower than the default level. If the offered payload type includedsprop-level-parameter-setsor includedsprop-parameter-setsconveyed using the "fmtp" source attribute, and if the offerer sees that the answerer has not includeduse-level-src-parameter-setsequal to 1 in the answer, the offerer knows that in-band transport of parameter sets is needed. 🙈
in-band-parameter-sets: this one MAY be used to indicate a receiver capability. The value MAY be equal to either 0 or 1. A value of 1 means the receiver discards out-of-band parameter sets in sprop-parameter-sets and sprop-level-parameter-sets, so the sender MUST transmit all parameter sets in-band. 🍓 A value of 0 means the receiver uses the out-of-band parameter sets included in sprop-parameter-sets and/or sprop-level-parameter-sets. However, in this case, the sender MAY still choose to send parameter sets in-band. When in-band-parameter-sets is equal to 1, use-level-src-parameter-sets MUST NOT be present or MUST be equal to 0. When the parameter is not present, this receiver capability is not specified, and therefore the sender MAY send out-of-band parameter sets only, it MAY send in-band parameter sets only, or it MAY send both. 🌸
level-asymmetry-allowed: this one MAY be used in SDP Offer/Answer to indicate whether level asymmetry is allowed. Level asymmetry means sending media encoded at a different level in the offerer-to-answerer direction than the level in the answerer-to-offerer direction. The value MAY be equal to either 0 or 1. When the parameter is not present, the value MUST be inferred to be equal to 0. The value 1 in both the offer and the answer indicates that level asymmetry is allowed. The value 0 in either the offer or the answer indicates that level asymmetry is not allowed. 🧐
If level-asymmetry-allowed is equal to 0 (or not present) in either the offer or the answer, level asymmetry is not allowed. In this case, the level to use in the direction from the offerer to the answerer MUST be the same as the level to use in the opposite direction. 💅
packetization-mode: this one signals the properties of an RTP payload type or the capabilities of a receiver implementation. Only a single configuration point can be indicated, so when capabilities to support more than one packetization-mode are declared, multiple configuration points (RTP payload types) must be used. 🧩
When the value of packetization-mode is equal to 0 or packetization-mode is not present, the single NAL mode MUST be used. This mode is in use in standards using ITU-T Recommendation H.241 [3] (see Section 12.1). When the value of packetization-mode is equal to 1, the non-interleaved mode MUST be used. When the value of packetization-mode is equal to 2, the interleaved mode MUST be used. The value of packetization-mode MUST be an integer in the range of 0 to 2, inclusive. 👑
sprop-interleaving-depth: this one MUST NOT be present when packetization-mode is not present or the value of packetization-mode is equal to 0 or 1. It MUST be present when the value of packetization-mode is equal to 2. 🎀
This one's the interleaving and buffering part 🧚 so the receiver can put NAL units back in the right order, lowkey the most important bit of the whole payload format 💅
sprop-interleaving-depth signals the properties of an RTP packet stream. It gives the maximum number of VCL NAL units that can come before any VCL NAL unit in the stream in transmission order, but still come after it in decoding order. So if a receiver's buffer for recovering NAL unit decoding order is at least sprop-interleaving-depth + 1 in VCL NAL units, it's guaranteed to rebuild the order. Iconic. ✨
The value of sprop-interleaving-depth MUST be an integer in the range of 0 to 32767, inclusive. 🌸
sprop-deint-buf-req MUST NOT be present when packetization-mode isn't there, or when it's 0 or 1. It MUST be present when packetization-mode is 2. 🎀
This one signals how big the de-interleaving buffer needs to be for the RTP packet stream. The value MUST be greater than or equal to the maximum buffer occupancy (in bytes) that's needed in that kind of de-interleaving buffer, as specified in Section 7.2 of RFC 6184 [1]. If the receiver's de-interleaving buffer is at least sprop-deint-buf-req in bytes, it can put interleaved NAL units back into decoding order. Slay. 💖
The value of sprop-deint-buf-req MUST be an integer in the range of 0 to 4294967295, inclusive.
🧠 Informative note: sprop-deint-buf-req only tells you the size of the de-interleaving buffer. If network jitter can happen, you also need to provision a properly sized jitter buffer. Don't skip it, bestie!
deint-buf-cap signals what a receiver can handle. It's the amount of de-interleaving buffer space in bytes that the receiver has free for putting NAL units back into decoding order. A receiver can handle any stream where sprop-deint-buf-req is smaller than or equal to this value. 🦋
If this parameter isn't present, a value of 0 MUST be used for deint-buf-cap. The value of deint-buf-cap MUST be an integer in the range of 0 to 4294967295, inclusive.
🧠 Informative note: deint-buf-cap is only the maximum possible size of the receiver's de-interleaving buffer. If network jitter can occur, a properly sized jitter buffer still has to be provisioned too. Same energy as before, bestie 😭
sprop-init-buf-time MAY be used to signal the properties of an RTP packet stream. It MUST NOT be present if packetization-mode is 0 or 1. 🌈
This parameter signals the initial buffering time a receiver MUST wait before it starts decoding, so it can recover the NAL unit decoding order from the transmission order. It's the maximum value of (decoding time of the NAL unit minus transmission time of a NAL unit). That's assuming reliable and instantaneous transmission, the same timeline for transmission and decoding, and decoding starts when the first packet arrives. The math is giving everything. 🧚
Here's an example. A NAL unit stream gets sent in the interleaved order below, where each number is the decoding time, and transmission goes from left to right:
0 2 1 3 5 4 6 8 7 ...If NAL units are steadily transmitted, the transmission times are:
0 1 2 3 4 5 6 7 8 ...Now subtract the decoding time from the transmission time, column by column, and you get:
0 -1 1 0 -1 1 0 -1 1 ...So in terms of intervals of NAL unit transmission times, sprop-init-buf-time is 1 in this example. The parameter is coded as a non-negative base10 integer in clock ticks of a 90-kHz clock. If it isn't present, then no initial buffering time value is defined. Otherwise, the value of sprop-init-buf-time MUST be an integer in the range of 0 to 4294967295, inclusive. 🍓
On top of the signaled sprop-init-buf-time, receivers SHOULD also take into account the transmission delay jitter buffering. That includes buffering for jitter caused by mixers, translators, gateways, proxies, traffic-shapers, and other network elements. Be prepared, bestie 💜
sprop-max-don-diff MAY be used to signal the properties of an RTP packet stream. It MUST NOT be used to signal transmitter, receiver, or codec capabilities. It MUST NOT be present if packetization-mode is 0 or 1. It's an integer in the range of 0 to 32767, inclusive. If it isn't present, the value is unspecified. ✨
It gets calculated like this:
sprop-max-don-diff = max{AbsDON(i) - AbsDON(j)}, for any i and
any j>i,Here i and j are the indexes of NAL units in transmission order, and AbsDON is a decoding order number for the NAL unit that doesn't wrap around to 0 after 65535. To put it another way, let m and n be consecutive NAL units in transmission order. For the very first NAL unit in transmission order (index 0), AbsDON(0) = DON(0). For the other NAL units, AbsDON gets worked out with these rules:
If DON(m) == DON(n), AbsDON(n) = AbsDON(m)
If (DON(m) < DON(n) and DON(n) - DON(m) < 32768),
AbsDON(n) = AbsDON(m) + DON(n) - DON(m)
If (DON(m) > DON(n) and DON(m) - DON(n) >= 32768),
AbsDON(n) = AbsDON(m) + 65536 - DON(m) + DON(n)
If (DON(m) < DON(n) and DON(n) - DON(m) >= 32768),
AbsDON(n) = AbsDON(m) - (DON(m) + 65536 - DON(n))
If (DON(m) > DON(n) and DON(m) - DON(n) < 32768),
AbsDON(n) = AbsDON(m) - (DON(m) - DON(n))In these rules, DON(i) is the decoding order number of the NAL unit with index i in transmission order. The decoding order number is specified in Section 5.5 of RFC 6184 [1]. 🎀
🧠 Informative note: Receivers may use sprop-max-don-diff to decide which NAL units in the receiver buffer can be passed to the decoder. Main character energy 👑
max-rcmd-nalu-size MAY be used to signal the capabilities of a receiver. It MUST NOT be used for any other purpose. The value is the largest NALU size in bytes that the receiver can handle efficiently. It's a recommendation, not a strict upper boundary. The sender MAY create larger NALUs, but it must be aware that handling them may cost more than handling NALUs that stick to the limit. 🦋
The value of max-rcmd-nalu-size MUST be an integer in the range of 0 to 4294967295, inclusive. If this parameter isn't specified, there's no known limit on NALU size. Senders still have to consider the MTU size available between the sender and the receiver, and SHOULD run MTU discovery for this purpose. 🌸
This parameter is motivated by things like an IP to H.223 video telephony gateway, where NALUs smaller than the H.223 transport data unit are more efficient. A gateway may terminate IP, so MTU discovery normally won't work beyond the gateway. Real talk, that's a lot of gateway drama 😭
🧠 Informative note: Setting this parameter to a lower value than necessary may have a negative impact. Don't undersell yourself, bestie 💅
sar-understood MAY be used to indicate a receiver capability and nothing else. It gives the largest aspect_ratio_idc value (specified in H.264 [2]) that is smaller than 255 and that the receiver understands. Table E-1 of H.264 [2] says aspect_ratio_idc 0 is "unspecified", 1 to 16 inclusive are specific Sample Aspect Ratios (SARs), 17 to 254 inclusive are "reserved", and 255 is the Extended SAR, where the SAR width and SAR height are explicitly signaled. So a receiver with an H.264 [2] decoder understands aspect_ratio_idc 1 to 16 inclusive, plus 255, because it knows exactly what the SAR is. For that receiver, sar-understood is 16. If Table E-1 of H.264 [2] gets extended in the future, say so the SAR for aspect_ratio_idc 17 is specified, then a receiver whose decoder understands that extension gets sar-understood 17. For a receiver with a decoder from the 2003 version of H.264 [2], sar-understood is 13, since the minimum reserved aspect_ratio_idc there is 14. 🧚
When sar-understood isn't present, its value MUST be inferred to be equal to 13. 💖
sar-supported MAY be used to indicate a receiver capability and nothing else. Its value is an integer in the range of 1 to sar-understood, inclusive, or equal to 255. A value of N smaller than 255 means the receiver supports all the SARs for H.264 aspect_ratio_idc values (see Table E-1 of H.264 [2]) from 1 to N inclusive, without geometric distortion. A value of 255 means the receiver supports every sample aspect ratio that can be expressed with two 16-bit integers as numerator and denominator. That's the ones expressible with the H.264 aspect_ratio_idc value of 255 (Extended_SAR, see Table E-1 of H.264 [2]), again without geometric distortion. 🎀
H.264-compliant encoders SHOULD NOT send an aspect_ratio_idc equal to 0, or one larger than sar-understood and smaller than 255. H.264-compliant encoders SHOULD send an aspect_ratio_idc that the receiver can display without geometric distortion. However, H.264-compliant encoders MAY choose to send pictures using any SAR. 🦋
The actual sample aspect ratio, or extended sample aspect ratio when it's present, is carried in the Video Usability Information (VUI) part of the sequence parameter set. 👑
Now the registration bits, the boring-but-important bits for the media type 🫶
Encoding considerations: This type is only defined for transfer via RTP (RFC 3550), and it's framed and binary (see Section 4.8 in RFC 4288). 🌈
Security considerations: See Section 9 of RFC 6185. Go read that one, it's serious business 💀
Interoperability considerations: None
Published specification: RFC 6185 and its reference section
Applications that use this media type: Video streaming and conferencing applications 🎀
Additional information: None
Magic number(s):
File extension(s):
Macintosh file type code(s):
Person & email address to contact for further information:
Tom Kristensen <tom.kristensen@tandberg.com>, <tomkri@ifi.uio.no>
Intended usage: COMMON
Restrictions on usage: This type depends on RTP framing, so it's only defined for transfer via RTP (see RFC 3550). Transport within other framing protocols isn't defined at this time. 🍓
Author: Tom Kristensen
Change controller: IETF Audio/Video Transport Working Group, delegated from the IESG 🫶
7. Mapping to SDP ✨
okay bestie, so the payload format media subtype and its parameters SHALL be mapped to SDP following Section 3 of RFC 4855 [10] 🎀. Here's what the fmtp attribute looks like in the media representation of a level 2.2 bitstream 🦋:
a=fmtp:97 profile-level-id=0080167.1. Offer/Answer Considerations
okay so when H264-RCDO gets offered over RTP using SDP in an Offer/Answer model [5] for unicast and multicast usage, the limitations and rules from Section 8.2.2 of RFC 6184 [1] apply 💅. Lowkey important: the profile_idc byte of the H264-RCDO profile-level-id parameter can only take the value of 0 (no profile) 🙈.
For interop with systems that don't support H264-RCDO, it is RECOMMENDED to offer the H264 media subtype as well 🫶. Per RFC 3264 [5], putting the H264-RCDO payload number before H264 in the format list on the "m=" line signals that H264-RCDO is preferred over H264 👑. Here's an example of that setup 🌈:
m=video 5555 RTP/AVP 97 98
a=rtpmap:97 H264-RCDO/90000
a=fmtp:97 profile-level-id=008016;max-mbps=42000;max-smbps=323500
a=rtpmap:98 H264/90000
a=fmtp:98 profile-level-id=428016;max-mbps=35000;max-smbps=3235007.2. Declarative SDP Considerations
when H264-RCDO over RTP is offered with SDP in a declarative style, like in the Real Time Streaming Protocol (RTSP) [11] or the Session Announcement Protocol (SAP) [12], the considerations in Section 8.2.3 of RFC 6184 [1] apply 🧚. Same tea as before: the profile_idc byte of the H264-RCDO profile-level-id parameter can only take the value of 0 (no profile) ✨.
8. IANA Considerations 🌸
IANA has registered H264-RCDO as specified in Section 6.1 🎀. The media subtype has also been added to the IANA registry for "RTP Payload Format MIME types" (http://www.iana.org) 💜.
9. Security Considerations 💀
bestie, this part is serious 😭. RTP packets using the payload format defined in this specification are subject to the security considerations discussed in the RTP specification [6] and in any applicable RTP profile. Refer also to the security considerations of the RTP Payload Format for H.264 Video specification in RFC 6184 [1]. No additional security considerations are introduced by this specification ✨.
10. Acknowledgements 🫶
The authors would like to thank Gisle Bjoentegaard and Arild Fuldseth for their technical contribution to the specification 👑. In the final phases, Roni Even did a helpful review 💖.
11. References 📚
11.1. Normative References
[1] Wang, Y., Even, R., Kristensen, T., and R. Jesup, "RTP Payload
Format for H.264 Video", RFC 6184, May 2011.
[2] International Telecommunications Union, "Advanced video coding
for generic audiovisual services", ITU-T Recommendation H.264,
March 2010.
[3] International Telecommunications Union, "Extended video
procedures and control signals for H.300-series terminals",
ITU-T Recommendation H.241, May 2006.
[4] Bradner, S., "Key words for use in RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119, March 1997.
[5] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with
Session Description Protocol (SDP)", RFC 3264, June 2002.
[6] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson,
"RTP: A Transport Protocol for Real-Time Applications", STD 64,
RFC 3550, July 2003.
[7] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and Video
Conferences with Minimal Control", STD 65, RFC 3551, July 2003.
[8] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
Description Protocol", RFC 4566, July 2006.
[9] Josefsson, S., "The Base16, Base32, and Base64 Data Encodings",
RFC 4648, October 2006.
[10] Casner, S., "Media Type Registration of RTP Payload Formats",
RFC 4855, February 2007.11.2. Informative References
[11] Schulzrinne, H., Rao, A., and R. Lanphier, "Real Time Streaming
Protocol (RTSP)", RFC 2326, April 1998.
[12] Handley, M., Perkins, C., and E. Whelan, "Session Announcement
Protocol", RFC 2974, October 2000.
[13] Freed, N. and J. Klensin, "Media Type Specifications and
Registration Procedures", BCP 13, RFC 4288, December 2005.
[14] Lennox, J., Ott, J., and T. Schierl, "Source-Specific Media
Attributes in the Session Description Protocol (SDP)",
RFC 5576, June 2009.Authors' Addresses 🦋
Tom Kristensen
TANDBERG
Philip Pedersens vei 22
N-1366 Lysaker
Norway
Phone: +47 67125125
EMail: tom.kristensen@tandberg.com, tomkri@ifi.uio.no
URI: http://www.tandberg.com
Patrick Luthi
TANDBERG
Philip Pedersens vei 22
N-1366 Lysaker
Norway
EMail: patrick.luthi@tandberg.com
URI: http://www.tandberg.com📦 where this page comes from
- ietf: rfc6185 2011-05
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms