RFC 4425

RTP Payload Format for Video Codec 1 (VC-1)

📅 published February 2006, from ietf

RFC 4425: RTP Payload Format for Video Codec 1 (VC-1) 🎀

okay bestie, let's get into it 💖 RFC 4425 is titled RTP Payload Format for Video Codec 1 (VC-1). Its author is A. Klemets from Microsoft, and it was published in February 2006 under the Network Working Group. It's a Standards Track document, which means it's an Internet standards track protocol specification. The header doesn't list any RFCs it obsoletes or updates, so there's no backstory to catch up on 🧚

Status of This Memo ✨

This document specifies an Internet standards track protocol for the Internet community, and requests discussion and suggestions for improvements. For the standardization state and status of this protocol, please check the current edition of the "Internet Official Protocol Standards" (STD 1). Distribution of this memo is unlimited 🌸

Copyright (C) The Internet Society (2006).

Abstract 💜

This memo specifies an RTP payload format for encapsulating Video Codec 1 (VC-1) compressed bit streams. VC-1 is defined by the Society of Motion Picture and Television Engineers (SMPTE) in the SMPTE 421M standard. SMPTE is the main standardizing body in the motion imaging industry, and SMPTE 421M defines a compressed video bit stream format and decoding process for television 📺

Table of Contents 🌈

   1. Introduction ....................................................2
      1.1. Conventions Used in This Document ..........................3
   2. Definitions and Abbreviations ...................................3
   3. Overview of VC-1 ................................................5
      3.1. VC-1 Bit Stream Layering Model .............................6
      3.2. Bit-stream Data Units in Advanced Profile ..................7
      3.3. Decoder Initialization Parameters ..........................7
      3.4. Ordering of Frames .........................................8
   4. Encapsulation of VC-1 Format Bit Streams in RTP .................9
      4.1. Access Units ...............................................9
      4.2. Fragmentation of VC-1 frames ..............................10
      4.3. Time Stamp Considerations .................................11
      4.4. Random Access Points ......................................13
      4.5. Removal of HRD Parameters .................................14
      4.6. Repeating the Sequence Layer Header .......................14
      4.7. Signaling of Media Type Parameters ........................15
      4.8. The "mode=1" Media Type Parameter .........................16
      4.9. The "mode=3" Media Type Parameter .........................16
   5. RTP Payload Format Syntax ......................................17
      5.1. RTP Header Usage ..........................................17
      5.2. AU Header Syntax ..........................................18
      5.3. AU Control Field Syntax ...................................19
   6. RTP Payload Format Parameters ..................................20
      6.1. Media type Registration ...................................20
      6.2. Mapping of media type parameters to SDP ...................28
      6.3. Usage with the SDP Offer/Answer Model .....................29
      6.4. Usage in Declarative Session Descriptions .................31
   7. Security Considerations ........................................32
   8. Congestion Control .............................................33
   9. IANA Considerations ............................................34
   10. References ....................................................34
      10.1. Normative References .....................................34
      10.2. Informative References ...................................35

1. Introduction 🎀

okay so this memo is the RTP payload format for the video coding standard Video Codec 1, which everyone calls VC-1. The official spec for the VC-1 bit stream format and decoding process is SMPTE 421M [1] 📼

VC-1 is seriously versatile ✨ It works for low bit rate Internet streaming, all the way up to High Definition Television (HDTV) broadcasts and Digital Cinema with nearly lossless coding. It's giving everything. The overall performance is so good that bit rate savings of more than 50% are reported [9] compared with MPEG-2. For more on how VC-1 stacks up against other codecs like MPEG-4 and H.264/AVC, check [9]. (Fun fact: [9] calls it by its earlier name, VC-9 🙈)

VC-1 is widely used for downloading and streaming movies on the Internet, in the form of Windows Media Video 9 (WMV-9) [9]. That's because the WMV-9 codec is compliant with the VC-1 standard 💅 It was also recently adopted as a mandatory compression format for the high-definition DVD formats HD DVD and Blu-ray. Iconic 👑

SMPTE 421M defines the VC-1 bit stream syntax and specifies constraints that must be met by VC-1 conformant bit streams. It also specifies the complete process required to decode the bit stream. However, it does not specify the VC-1 compression algorithm, so encoder makers can implement a VC-1 encoder in different ways 🧚

The VC-1 bit stream syntax has three profiles 🌸 Each profile comes with its own bit stream syntax elements and algorithms. Which one is best depends on the application, so some profiles may be more suitable than others. For example, Simple profile is built for low bit rate Internet streaming and for playback on devices that can only handle low-complexity decoding. Advanced profile is built for broadcast applications, like digital TV, HD DVD, or HDTV. Advanced profile is also the only VC-1 profile that supports interlaced video frames and non-square pixels 🦋

Here's the roadmap ✨ Section 2 defines the abbreviations used in this document. Section 3 gives a more detailed overview of VC-1. Sections 4 and 5 define the RTP payload format for VC-1, and section 6 defines the media type and SDP parameters for VC-1. Section 7 covers security considerations, and section 8 covers congestion control requirements 💖

1.1. Conventions Used in This Document 🫶

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14, RFC 2119 [2] 💅

2. Definitions and Abbreviations 📚

this doc uses the definitions from SMPTE 421M [1], so the important terms are restated right here for convenience 🤓

B-picture: A picture that's coded using motion compensated prediction from past and/or future reference fields or frames. A B-picture cannot be used for predicting any other picture 🌙

BI-picture: A B-picture that's coded using information only from itself. A BI-picture cannot be used for predicting any other picture 🫧

Bit-stream data unit (BDU): A unit of the compressed data that can be parsed (aka syntax decoded) independently of other information at the same hierarchical level. A BDU can be, for example, a sequence layer header, an entry-point header, a frame, or a slice 🧩

Encapsulated BDU (EBDU): A BDU that has been encapsulated using the encapsulation mechanism described in Annex E of SMPTE 421M [1], to prevent emulation of the start code prefix in the bit stream 🔒

Entry-point: A point in the bit stream that offers random access 🚪

frame: A frame contains lines of spatial information of a video signal. For progressive video, these lines hold samples starting from one time instant and continuing through successive lines to the bottom of the frame. For interlaced video, a frame consists of two fields, a top field and a bottom field. One of these fields starts one field period later than the other 🎞️

interlace: The property of frames where alternating lines of the frame represent different instances in time. In an interlaced frame, one of the fields is meant to be displayed first 🔀

I-picture: A picture coded using information only from itself 🌟

level: A defined set of constraints on the values that the parameters can take (like bit rate and buffer size) within a particular profile. A profile may contain one or more levels 📶

P-picture: A picture coded using motion compensated prediction from past reference fields or frames 🔁

picture: For progressive video, a picture is identical to a frame. For interlaced video, a picture may refer to a frame, or to the top field or the bottom field of the frame, depending on the context 🖼️

profile: A defined subset of the VC-1 syntax, with a specific set of coding tools, algorithms, and syntax associated with it. There are three VC-1 profiles: Simple, Main, and Advanced 🎀

progressive: The property of frames where all the samples of the frame represent the same instance in time 🏃‍♀️

random access: A random access point in the bit stream comes with this guarantee: if decoding begins at this point, all frames needed for display after this point have no decoding dependency on any data preceding this point, and they are also present in the decoding sequence after this point. A random access point is also called an entry-point 🎯

sequence: A coded representation of a series of one or more pictures. In VC-1 Advanced profile, a sequence is made of a series of one or more entry-point segments, where each entry-point segment is made of a series of one or more pictures. The first picture in each entry-point segment provides random access. In VC-1 Simple and Main profiles, the first picture in each sequence is an I-picture 🎬

slice: A consecutive series of macroblock rows in a picture, which are encoded as a single unit 🍰

start codes (SC): Unique 32-bit codes that are embedded in the coded bit stream and identify the beginning of a BDU. Start codes consist of a unique three-byte Start Code Prefix (SCP) and a one-byte Start Code Suffix (SCS) 🚦

3. Overview of VC-1 ✨

okay so VC-1 has three profiles: Simple, Main, and Advanced 💖. Simple is built for low bit rates and low complexity, like playing media on a personal digital assistant 📱. Its max bit rate is 384 kbps. Main profile is the high bit rate one, for things like streaming and TV over IP 📺. It supports B-pictures, which give you better compression efficiency but cost more complexity 🧠.

Advanced profile gets the good stuff. Non-square pixels and support for interlaced pictures are only in Advanced, since those are the features that help with high compression efficiency 🔥. Its max bit rate is 135 Mbps, which makes it great for nearly lossless encoding of HDTV signals 🎀.

Only Advanced profile can carry user-data (meta-data) in-band with the compressed bit stream 🫶. That's how closed captioning can work, for example 💬.

Of the three, only Advanced profile lets codec configuration parameters, like the picture aspect ratio, change through in-band signaling in the compressed bit stream 💅.

Each profile also has a set of "levels" 🌈. A profile is about which features or syntax elements you get, but a level is a set of constraints on parameter values within a profile, like bit rate or buffer size. VC-1 Simple has two levels, Main has three, and Advanced has five. For the full list of profiles and levels, check Annex D of SMPTE 421M [1] 📚.

3.1. VC-1 Bit Stream Layering Model

The VC-1 bit stream is a hierarchy of layers, kind of like a networking protocol stack 🧚. The outermost one is the sequence layer. The others are entry-point, picture, slice, macroblock, and block 🍓.

In Simple and Main profiles, a sequence is a series of one or more coded pictures. In Advanced profile, a sequence is one or more entry-point segments, and each segment is a series of one or more pictures. The first picture in each entry-point segment provides random access 🦋. A picture is broken down into macroblocks, and a slice is one or more contiguous rows of macroblocks 🌸.

The entry-point and slice layers only exist in Advanced profile. There, the start of each entry-point layer segment marks a random access point ✨. In Simple and Main profiles, every I-picture is a random access point 🙈.

Each picture can be an I-picture, P-picture, skipped picture, BI-picture, or B-picture 💜. These terms are defined in section 2 of this document and in section 4.12 of SMPTE 421M [1].

3.2. Bit-stream Data Units in Advanced Profile

In Advanced profile, each picture and slice counts as a Bit-stream Data Unit (BDU) 🎀. A BDU is always byte-aligned, and it's a unit that can be parsed (syntax decoded) on its own, without needing other info from the same layer 💖.

A BDU starts with an identifier called a Start Code (SC) 🔥. Sequence layer headers and entry-point headers are BDUs too, so you can spot them by their Start Codes 👑. For a full list of Start Codes, see Annex E of SMPTE 421M [1]. Blocks and macroblocks aren't BDUs, so they have no Start Code and aren't necessarily byte-aligned 🙅‍♀️.

The Start Code is four bytes long 🌈. The first three are 0x00, 0x00 and 0x01. The fourth byte is the Start Code Suffix (SCS), and it tells you what type of BDU comes after the Start Code. For example, a sequence layer header has SCS 0x0F, while an entry-point header has SCS 0x0E, so they're different 🧁. The Start Code is always byte-aligned and sent in network byte order 🫶.

To stop the Start Code from accidentally showing up inside the coded bit stream, SMPTE 421M has an encapsulation mechanism that uses byte stuffing 🛡️. A BDU that's been wrapped this way is called an Encapsulated BDU, or EBDU ✨.

3.3. Decoder Initialization Parameters

In VC-1 Advanced profile, the sequence layer header holds the parameters the VC-1 decoder needs to initialize 🧚‍♀️.

Those parameters apply to all entry-point segments until the next sequence layer header shows up in the coded bit stream 💫.

The sequence layer header parameters include the Advanced profile level, the maximum dimensions of the coded frames, the aspect ratio, interlace information, the frame rate, and up to 31 leaky bucket parameter sets for the Hypothetical Reference Decoder (HRD) 📊.

Section 6.1 of SMPTE 421M [1] has the formal spec for the sequence layer header 📖.

A sequence layer header isn't defined for VC-1 Simple and Main profiles 🙃. For those, decoder initialization parameters MUST be conveyed out-of-band 🚨. Those parameters include the maximum dimensions of the coded frames and a leaky bucket parameter set for the HRD. Section 4.7 explains how this RTP payload format conveys them 📨.

Each leaky bucket parameter set for the HRD specifies a peak transmission bit rate and a decoder buffer capacity 🪣. The coded bit stream is restricted by these parameters. The HRD model doesn't require the decoder to do any buffering 🙅‍♀️. Its job is to limit the encoder's bit rate fluctuations using a basic buffering model, so the resources needed to decode the bit stream are predictable 🔮. The HRD has a constant-delay mode and a variable-delay mode. Constant-delay is for broadcast and streaming, and variable-delay is for video-conferencing 📞.

Annex C of SMPTE 421M [1] covers how to use the hypothetical reference decoder for VC-1 bit streams 🧠. You can find a general explanation of the HRD theory in [10] 📚.

For Simple and Main profiles, the current buffer fullness value for the HRD leaky bucket is signaled with the BF syntax element in the picture header of I-pictures and BI-pictures 🍓.

For Advanced profile, the entry-point header gives the current buffer fullness values for the leaky buckets in the HRD 💜. It also gives the coding control parameters that stay in effect until the next entry-point header in the bit stream 🎀. The entry-point layer concept only applies to VC-1 Advanced profile. See Section 6.2 of SMPTE 421M [1] for the formal spec of the entry-point header 🦋.

3.4. Ordering of Frames

Frames are sent in the same order they're captured, unless B-pictures or BI-pictures are in the coded bit stream 🌸. A BI-picture is a special kind of B-picture, so for the rest of this section, B-picture and B-frame also cover BI-pictures and BI-frames 💅.

When B-pictures are present, frames get sent so the frames the B-pictures depend on arrive first. This is called the coded order of the frames 🔁.

The rules for turning coded order into display order are in section 5.4 of SMPTE 421M [1] 📜. The short version is that if B-pictures might be in the stream, a hypothetical decoder needs to buffer one extra decoded frame 🧺. When an I-frame or P-frame arrives, it can be decoded right away, but it doesn't get displayed until the next I- or P-frame shows up 🕰️. B-frames, though, get displayed immediately ⚡.

Figure 1 shows how capture timing, coded order, and display order line up when B-pictures are present. Frame P4 is delayed until P7 is received, while B2 and B3 are displayed right away 👀.

Capture:        |I0  P1  B2  B3  P4  B5  B6  P7  B8  B9  ...
                |
Coded order:    |        I0  P1  P4  B2  B3  P7  B5  B6  ...
                |
Display order:  |            I0  P1  B2  B3  P4  B5  B6  ...
                |
                |+---+---+---+---+---+---+---+---+---+--> time
                 0   1   2   3   4   5   6   7   8   9

Figure 1. Frame reordering when B-pictures are present

If there are no B-pictures, the coded order and display order are identical, so frames can be displayed without the extra delay shown in Figure 1 🎉.

4. Encapsulation of VC-1 Format Bit Streams in RTP 🎀

4.1. Access Units

Each RTP packet holds a whole number of application data units (ADUs) 📦. For VC-1 format bit streams, an ADU is the same as one Access Unit (AU). An AU is the AU header (defined in section 5.2) followed by a variable-length payload, following the rules and constraints in sections 4.1 and 4.2 💖. Figure 2 shows what an RTP packet with multiple AUs looks like 🦋.

           +-+-+-+-+-+-+-+-+-+-+-+-+-+- .. +-+-+-+-+
           | RTP     | AU(1) | AU(2) |     | AU(n) |
           | Header  |       |       |     |       |
           +-+-+-+-+-+-+-+-+-+-+-+-+-+- .. +-+-+-+-+

                Figure 2.  RTP packet structure

Each Access Unit MUST start with the AU header defined in section 5.2 🚪. The AU payload MUST contain data from exactly one VC-1 frame ✨. That means data from different VC-1 frames always goes in different AUs. A single VC-1 frame can be split across multiple AUs, though (see section 4.2) 🧩.

For interlaced video, a VC-1 frame has two fields that might be coded as separate pictures. Both pictures still belong to the same VC-1 frame 🌈.

When VC-1 Advanced profile is used, the AU payload follows these rules 👑:

  • The AU payload MUST contain VC-1 bit stream data in EBDU format (i.e., the bit stream must use the byte-stuffing encapsulation mode defined in Annex E of SMPTE 421M [1].) 🛡️
  • The AU payload MAY contain multiple EBDUs, e.g., a sequence layer header, an entry-point header, a frame (picture) header, a field header, and multiple slices and the associated user-data. However, all slices and their corresponding macroblocks MUST belong to the same video frame. 🍓
  • The AU payload MUST start at an EBDU boundary, except when the AU payload contains a fragmented frame, in which case the rules in section 4.2 apply. 🫶

When VC-1 Simple or Main profiles are used, the AU payload MUST start at the beginning of a frame, except when the AU payload contains a fragmented frame. Section 4.2 explains how to handle fragmented frames 🧁.

Access Units MUST be byte-aligned 💅. If the data in an AU (EBDUs for Advanced profile, or a frame for Simple and Main) doesn't end at an octet boundary, up to 7 zero-valued padding bits MUST be added to get octet-alignment 🌸.

4.2. Fragmentation of VC-1 frames 💅

okay so each AU payload SHOULD contain a complete VC-1 frame 🎀 that's the dream. but if that would push the RTP packet past the MTU size, the frame SHOULD be fragmented into multiple AUs so we dodge IP-level fragmentation 🙈. when an AU holds a fragmented frame, that MUST be flagged by setting the FRAG field in the AU header, as defined in section 5.3 ✨.

AU payloads that don't contain a fragmented frame, or that contain the first fragment of a frame, MUST start at an EBDU boundary if Advanced profile is used. for Simple and Main profiles, the AU payload MUST start at the beginning of a frame 💖.

If Advanced profile is used, AU payloads with a fragment of a frame other than the first fragment SHOULD start at an EBDU boundary, like at the start of a slice 🧚.

BUT (lowkey annoying) slices are only defined for Advanced profile, and they aren't always used. blocks and macroblocks are not BDUs (they have no Start Code) and they aren't byte-aligned, so it might not always be possible to continue a fragmented frame at an EBDU boundary 😭. you can check whether an AU payload starts at an EBDU boundary by looking at its first three bytes. if they match the Start Code Prefix (i.e., 0x00, 0x00, 0x01), it does 🔍.

For Simple and Main profiles, since blocks and macroblocks aren't byte-aligned, the fragmentation boundary can be chosen arbitrarily 🦋.

If an RTP packet contains an AU with the last fragment of a frame, additional AUs SHOULD NOT be included in that RTP packet 💅.

If the PTS Delta field is present in the AU header, each fragment of a frame MUST have the same presentation time. same deal for DTS Delta: if that field is present, each fragment MUST have the same decode time 🫶.

4.3. Time Stamp Considerations ⏰

VC-1 video frames MUST be transmitted in the coded order 🎬. coded order means no frame depends on a later one (more on that in section 3.4). when a video frame is a single picture, the frame's presentation time is identical to the picture's. when the VC-1 interlace coding mode is used, a frame can hold two pictures, one per field, and then the frame's presentation time is the presentation time of the field that gets displayed first 📺.

The RTP timestamp field MUST be set to the presentation time of the video frame in the first AU of the RTP packet. this works because the presentation time only differs from the sampling instant by an arbitrary constant offset ✨.

If the video frame in an AU has a presentation time that differs from the RTP timestamp field, then the presentation time MUST be specified using the PTS Delta field in the AU header. since the RTP timestamp field has to match the presentation time of the first video frame, this can only happen if an RTP packet has multiple AUs. the syntax of PTS Delta is in section 5.2 📝.

The decode time of a VC-1 frame is always going up monotonically when the frames are sent in the coded order 📈. if there are neither B- nor BI-pictures in the coded bit stream, the decode time of a frame SHALL be equal to its presentation time. a BI-picture is a special kind of B-picture, and for the rest of this section, the terms B-picture and B-frame also cover BI-pictures and BI-frames 🌈.

If B-pictures may be in the coded bit stream, the decode times of frames get figured out like this:

  • B-frames: The decode time SHALL be equal to the presentation time of the B-frame.
  • First non-B frame in the coded order: The decode time SHALL be at least one frame period less than the decode time of the next frame in the coded order. A frame period is defined as the inverse of the frame rate used in the coded bit stream (e.g., 100 milliseconds if the frame rate is 10 frames per seconds.) For bit streams with a variable frame rate, the maximum frame rate SHALL determine the frame period. If the maximum frame is not specified, the maximum frame rate allowed by the profile and level SHALL be used.
  • Non-B frames (other than the first frame in the coded order): The decode time SHALL be equal to the presentation time of the previous non-B frame in the coded order.

okay, example time 🧁. look at Figure 1 in section 3.4. to figure out the decode time of the first frame, I0, you first need the decode time of the next frame, P1. P1 is a non-B frame, so its decode time equals the presentation time of I0, which is 3 time units. that means I0's decode time has to be at least one frame period less than 3. in this example the frame period is 1, because one frame is displayed every time unit, so I0's decode time is chosen as 2 time units 🎯. the decode time of the third frame in the coded order, P4, is 4, because it has to equal the presentation time of the previous non-B frame in the coded order, P1. and the decode time of B-frame B2 is 5 time units, which is identical to its presentation time 🫶.

If the decode time of a video frame differs from its presentation time, then the decode time MUST be specified using the DTS Delta field in the AU header. the syntax of DTS Delta is in section 5.2 📝.

Receivers are not required to use the DTS Delta field. however, possible uses include buffer management and pacing of frames before decoding 🧠. if RTP packets get lost, the DTS Delta field can help figure out whether the lost sequence of RTP packets had reference frames or only B-frames 🕵️‍♀️. you do this by comparing the decode and presentation times of the first frame received after the lost sequence against the presentation time of the last reference frame received before the lost sequence.

Knowing whether the stream will contain B-pictures can help the receiver allocate resources more efficiently and can reduce delay, because if there are no B-pictures, no reordering of frames is needed between the decoding process and the display of the decoded frames. this can be important for interactive applications 💬.

The receiver SHALL assume that the coded bit stream may contain B-pictures in the following cases:

  • Advanced profile: If the value of the "bpic" media type parameter defined in section 6.1 is 1, or if the "bpic" parameter is not specified.
  • Main profile: If the MAXBFRAMES field in the STRUCT_C decoder initialization parameter has a non-zero value. STRUCT_C is conveyed in the "config" media type parameter, which is defined in section 6.1.

Simple profile does not use B-pictures 🌸.

4.4. Random Access Points 🎯

The entry-point header has the info the decoder needs to decode the frames in that entry-point segment. that means if RTP packets get lost, the decoder might not be able to decode frames until the next entry-point header shows up 😵.

The first frame after an entry-point header is a random access point into the coded bit stream. Simple and Main profiles don't have entry-point headers, so for those profiles, each I-picture is a random access point 🔑.

To let the RTP receiver detect that a lost RTP packet contained a random access point, this RTP payload format defines a field called "RA Count". this field is present in every AU, and its value goes up (modulo 256) for every random access point. for more details, check the definition of "RA Count" in section 5.2 🔢.

To make it easy to tell whether an AU contains a random access point, this RTP payload format also defines a bit called the "RA" flag in the AU Control field. this bit is set to 1 only on those AU's that contain a random access point. the RA bit is defined in section 5.3 🚩.

4.5. Removal of HRD Parameters ✂️

The sequence layer header of Advanced profile may include up to 31 leaky bucket parameter sets for the Hypothetical Reference Decoder (HRD). each leaky bucket parameter set specifies a possible peak transmission bit rate (HRD_RATE) and a decoder buffer capacity (HRD_BUFFER). see section 3.3 for more about the HRD 🪣.

If the actual peak transmission rate is known by the RTP sender, the RTP sender MAY remove all leaky bucket parameter sets except the one that matches the actual peak transmission rate 🧹.

For each leaky bucket parameter set in the sequence layer header, there's also a parameter in the entry-point header that specifies the initial fullness (HRD_FULL) of the leaky bucket 💧.

If the RTP sender has removed any leaky bucket parameter sets from the sequence layer header, then for any removed leaky bucket parameter set, it MUST also remove the corresponding HRD_FULL parameter in the entry-point header 🙅‍♀️.

Removing leaky bucket parameter sets like this may significantly shrink the size of the sequence layer headers and the entry-point headers 📉.

4.6. Repeating the Sequence Layer Header 🔁

To improve robustness against loss of RTP packets, it is RECOMMENDED that if the sequence layer header changes, it should be repeated frequently in the bit stream 🛡️. in that case, it is RECOMMENDED that the number of leaky bucket parameters in the sequence layer header and the entry-point headers gets reduced to one, as described in section 4.5. that helps keep the overhead from repeating the sequence layer header down 💅.

Any data in the VC-1 bit stream, including repeated copies of the sequence header itself, must be accounted for when computing the leaky bucket parameter for the HRD. see section 3.3 for more about the HRD 🧮.

If the value of TFCNTRFLAG in the sequence layer header is 1, each picture header contains a frame counter field (TFCNTR). every time the sequence layer header is inserted in the bit stream, the value of this counter MUST be reset 🔄.

To let the RTP receiver detect that an RTP packet that was lost contained a new sequence layer header, the AU Control field defines a bit called the "SL" flag. this bit is toggled when a sequence layer header is transmitted, but only if that header is different from the most recently transmitted sequence layer header. the SL bit is defined in section 5.3 🏁.

4.7. Signaling of Media Type Parameters 🎀

Okay so, when this payload format is used with SDP, the decoder initialization parameters from section 3.3 MUST be signaled in SDP using the media type parameters from section 6.1. Section 6.2 explains how those media type parameters get mapped to SDP [5], section 6.3 covers the rules for the SDP Offer/Answer model, and section 6.4 covers the rules for when SDP is used in a declarative style. 💅

With Simple or Main profiles, you cannot change the decoder initialization parameters through the coded bit stream, bestie 😭 So any changes would have to happen out-of-band, like a SIP [14] re-invite or something similar that sends over an updated session description. 📨

With Advanced profile, the decoder initialization parameters MAY be changed by inserting a new sequence layer header or an entry-point header into the coded bit stream. ✨

The sequence layer header tells you the VC-1 level, the maximum size of the coded frames, and optionally the maximum frame rate too. The media type parameters level, width, height, and framerate set the upper limits for these. So the sequence layer header MAY specify values that are lower than those media type parameters, but it MUST NOT exceed any of them. It's giving "stay within the limits," honestly 🙈🌸

4.8. The "mode=1" Media Type Parameter 🦋

In certain applications using Advanced profile, the sequence layer header never changes. This MAY be signaled with the media type parameter mode=1. (The mode parameter is defined in section 6.1.) The mode=1 parameter acts as a "hint" to the RTP receiver that every sequence layer header in the bit stream will be identical. If mode=1 is signaled and a sequence layer header shows up in the coded bit stream, then it MUST be identical to the sequence layer header given by the config media type parameter. Iconic consistency 👑

Since the sequence layer header never changes under mode=1, the RTP sender MAY remove it from the bit stream. But heads up: if the TFCNTRFLAG value in the sequence layer header is 1, then each picture header has a frame counter field (TFCNTR). That field gets reset every time the sequence layer header occurs in the bit stream. So if the RTP sender decides to remove the sequence layer header, it MUST make sure the resulting bit stream is still compliant with the VC-1 specification (for example, by adjusting the TFCNTR field if needed). 🧚

4.9. The "mode=3" Media Type Parameter 🌈

In certain applications using Advanced profile, both the sequence layer header and the entry-point header never change. This MAY be signaled with the media type parameter mode=3. The same rules apply to mode=3 as they do for mode=1, which is described in section 4.8. On top of that, when mode=3 is signaled, the RTP sender MAY "compress" the coded bit stream by leaving sequence layer headers and entry-point headers out of the RTP packets. 🍓

The RTP receiver MUST "decompress" the coded bit stream by putting the entry-point headers back in before it hands the coded bit stream to the VC-1 decoder. The sequence layer header doesn't need to be decompressed by the receiver, because it never changes. ✨

If mode=3 is signaled and the RTP receiver gets a complete AU or the first fragment of an AU, and the RA bit is set to 1 but the AU does not begin with an entry-point header, then that means the entry-point header got "compressed" 🙈 In that case, the RTP receiver MUST insert an entry-point header at the beginning of the AU. When it does, the RTP receiver MUST use the one specified by the config media type parameter. Main character energy in that moment 💖

5. RTP Payload Format Syntax 💜

5.1. RTP Header Usage 🫶

The RTP header format is specified in RFC 3550 [3], and it's reprinted in Figure 3 so you don't have to go hunting for it 🙌

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |V=2|P|X|  CC   |M|     PT      |       sequence number         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           timestamp                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           synchronization source (SSRC) identifier            |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |            contributing source (CSRC) identifiers             |
   |                             ....                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Figure 3. RTP header according to RFC 3550

The fixed RTP header fields mean what they usually mean, as defined in RFC 3550 and by the RTP profile in use. A few extra notes for this format:

Marker bit (M): 1 bit This bit is set to 1 if the RTP packet contains an Access Unit containing a complete VC-1 frame or the last fragment of a VC-1 frame. 🎯

Payload type (PT): 7 bits This document doesn't assign an RTP payload type for this payload format. The assignment has to happen either through the RTP profile being used or dynamically. 🤷‍♀️

Sequence Number: 16 bits The RTP receiver can use the sequence number to recover the coded order of the VC-1 frames. A typical VC-1 decoder will need the VC-1 frames delivered in coded order. When VC-1 frames get fragmented across RTP packets, the RTP receiver can use the sequence number to make sure no fragment is missing. 🔍

Timestamp: 32 bits The RTP timestamp is set to the presentation time of the VC-1 frame in the first Access Unit. A clock rate of 90 kHz MUST be used. ⏰

5.2. AU Header Syntax 🎀

The Access Unit header is made up of a one-byte AU Control field, the RA Count field, and 3 optional fields. All fields MUST be written in network byte order. Here's what the AU header looks like, as shown in Figure 4 🧁

           +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
           |AU     | RA    |  AUP  | PTS   | DTS   |
           |Control| Count |  Len  | Delta | Delta |
           +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Figure 4. Structure of AU header

AU Control: 8 bits The AU Control field is defined in section 5.3. 🦋

RA Count: 8 bits Random Access Point Counter. This is a binary modulo 256 counter. Its value MUST be incremented by 1 each time an AU is transmitted where the RA bit in the AU Control field is set to 1. The initial value is undefined and MAY be chosen randomly. 🎲

AUP Len: 16 bits Access Unit Payload Length. This specifies the size, in bytes, of the payload of the Access Unit. The field does not include the size of the AU header itself. The field MUST be included in each AU header in an RTP packet, except for the last AU header in the packet. If this field is not included, the payload of the Access Unit SHALL be assumed to extend to the end of the RTP payload. 📏

PTS Delta: 32 bits Presentation time delta. This specifies the presentation time of the frame as a 2's complement offset (delta) from the timestamp field in this RTP packet's RTP header. The PTS Delta field MUST use the same clock rate as the timestamp field in the RTP header.

This field SHOULD NOT be included in the first AU header in the RTP packet, because the RTP timestamp field already specifies the presentation time of the frame in the first AU. If this field is not included, the presentation time of the frame SHALL be assumed to be specified by the timestamp field in the RTP header. ⏱️

DTS Delta: 32 bits Decode time delta. This specifies the decode time of the frame as a 2's complement offset (delta) between the presentation time and the decode time. Note that if the presentation time is larger than the decode time, the DTS Delta field ends up with a value greater than zero. The DTS Delta field MUST use the same clock rate as the timestamp field in the RTP header. If this field is not included, the decode time of the frame SHALL be assumed to be identical to the presentation time of the frame. 🕰️

5.3. AU Control Field Syntax 🌸

The 8-bit AU Control field is laid out as shown in Figure 5 🎀

  0    1    2    3    4    5    6    7
+----+----+----+----+----+----+----+----+
|  FRAG   | RA | SL | LP | PT | DT | R  |
+----+----+----+----+----+----+----+----+

Figure 5. Syntax of AU Control field.

FRAG: 2 bits Fragmentation Information. This field tells you whether the AU payload holds a complete frame or a fragment of a frame. It MUST be set as follows:

  • 0: The AU payload contains a fragment of a frame other than the first or last fragment.
  • 1: The AU payload contains the first fragment of a frame.
  • 2: The AU payload contains the last fragment of a frame.
  • 3: The AU payload contains a complete frame (not fragmented.)

RA: 1 bit Random Access Point indicator. This bit MUST be set to 1 if the AU contains a frame that is a random access point. For Simple and Main profiles, any I-picture is a random access point.

For Advanced profile, the first frame after an entry-point header is a random access point.

If entry-point headers are not transmitted at every random access point, this MUST be indicated using the media type parameter mode=3. 🚩

SL: 1 bit Sequence Layer Counter. This bit MUST be toggled, meaning changed from 0 to 1 or from 1 to 0, if the AU contains a sequence layer header and that header is different from the most recently transmitted sequence layer header. Otherwise, the value of this bit must be identical to the value of the SL bit in the previous AU.

The initial value of this bit is undefined and MAY be chosen randomly.

The bit MUST be 0 for Simple and Main profile bit streams, or if the sequence layer header never changes. 🔁

LP: 1 bit Length Present. This bit MUST be set to 1 if the AU header includes the AUP Len field. 📐

PT: 1 bit PTS Delta Present. This bit MUST be set to 1 if the AU header includes the PTS Delta field. ⏱️

DT: 1 bit DTS Delta Present. This bit MUST be set to 1 if the AU header includes the DTS Delta field. 🕰️

R: 1 bit Reserved. This bit MUST be set to 0 and MUST be ignored by receivers. 🙈

6. RTP Payload Format Parameters 🧚

6.1. Media type Registration 📝

This registration uses the template defined in RFC 4288 [7] and follows RFC 3555 [8]. ✨

Type name: video 🎥 Subtype name: vc1 💅 Required parameters:

  • profile: The value is an integer identifying the VC-1 profile. The following values are defined:
    • 0: Simple profile
    • 1: Main profile
    • 3: Advanced profile

Okay bestie, so when the profile parameter is describing a coded bit stream, it's telling you which VC-1 profile a decoder has to support to decode that stream 🎀

When profile shows up in capability exchange or session setup, it's saying which VC-1 profile the codec supports ✨

level

The value is an integer that tells you the level of the VC-1 profile 💖 For Advanced profile, valid values are 0 through 4, and those map to levels L0 through L4, respectively. For Simple and Main profiles, these values are defined:

  • 1: Low Level
  • 2: Medium Level
  • 3: High Level (only valid for Main profile)

If level is describing a coded bit stream, it's the highest level of the VC-1 profile that a decoder has to support when decoding that stream. Also, supporting a level means supporting every numerically lower level of that profile, so it's giving layered 🌈

If level is used for capability exchange or in a session setup procedure, it's the highest level of the VC-1 profile that the codec supports. See section 6.3 of RFC 4425 for specific rules on how this parameter is used with the SDP Offer/Answer model 🧚

Optional parameters

config

The value is a base16 [6] (hexadecimal) representation of an octet string that carries the decoder initialization parameters 🍓 Those parameters get mapped onto the base16 octet string on an MSB-first basis. The first bit of the decoder initialization parameters MUST be located at the MSB of the first octet. If the decoder initialization parameters aren't multiples of 8 bits, up to 7 zero-valued padding bits MUST be added in the last octet to achieve octet alignment. Octet alignment is the vibe 🫶

For Simple and Main profiles, the decoder initialization parameters are STRUCT_C, as defined in Annex J of SMPTE 421M [1].

For Advanced profile, the decoder initialization parameters are a sequence layer header directly followed by an entry-point header. The two headers MUST be in EBDU format, which means they must include their Start Codes and must use the encapsulation method defined in Annex E of SMPTE 421M [1] 💅

width

The value is an integer greater than zero, and it's the maximum horizontal size of the coded frames, in luma samples (pixels in the luma picture) 📏

For Simple and Main profiles, the value SHALL be identical to the actual horizontal size of the coded frames.

For Advanced profile, the value SHALL be greater than, or equal to, the largest horizontal size of the coded frames.

If this parameter is not specified, it defaults to the maximum horizontal size allowed by the specified profile and level.

height

The value is an integer greater than zero, and it's the maximum vertical size of the coded frames, in luma samples (pixels in a progressively coded luma picture) 🦋

For Simple and Main profiles, the value SHALL be identical to the actual vertical size of the coded frames.

For Advanced profile, the value SHALL be greater than, or equal to, the largest vertical size of the coded frames.

If this parameter is not specified, it defaults to the maximum vertical size allowed by the specified profile and level.

bitrate

The value is an integer greater than zero, and it's the peak transmission rate of the coded bit stream in bits per second. The number does NOT include RTP encapsulation overhead, meaning it doesn't count the AU headers or any of the RTP, UDP, or IP headers 💜

If this parameter is not specified, it defaults to the maximum bit rate allowed by the specified profile and level. See the values for "RMax" in Annex D of SMPTE 421M [1].

buffer

The value is an integer for the leaky bucket size, B, in milliseconds. That bucket has to be big enough to contain a stream transmitted at the rate given by the bitrate parameter. This parameter is defined in the hypothetical reference decoder model for VC-1, in Annex C of SMPTE 421M [1] 🌸

Heads up: this parameter only relates to the codec bit stream, and it doesn't account for any buffering time that might be needed to smooth out network jitter ⭐

If this parameter is not specified, it defaults to the maximum buffer size allowed by the specified profile and level. See the values for "BMax" and "RMax" in Annex D of SMPTE 421M [1].

framerate

The value is an integer greater than zero. It's the maximum number of frames per second in the coded bit stream, multiplied by 1000 and rounded to the nearest integer. For example, 30000/1001 (approximately 29.97) frames per second is represented as 29970 🎬

This parameter can be used to control resource allocation at the receiver. For example, a receiver may choose to do extra post-processing on decoded frames only if the frame rate is expected to be low. The parameter MUST NOT be used for pacing of the rendering process, since the actual frame rate may differ from the specified value. Don't use it for pacing, periodt 🙅‍♀️

If the parameter is not specified, it defaults to the maximum frame rate allowed by the specified profile and level.

bpic

This parameter signals that B- and BI-pictures may be present when Advanced profile is used. If this parameter is present, and B- or BI-pictures may be present in the coded bit stream, this parameter MUST be equal to 1.

A value of 0 means B- and BI-pictures SHALL NOT be present in the coded bit stream, even if the sequence layer header changes. Including this parameter with a value of 0 is RECOMMENDED if neither B- nor BI-pictures are included in the coded bit stream 👑

This parameter MUST NOT be used with Simple and Main profiles. For Main profile, the presence of B- and BI-pictures is indicated by the MAXBFRAMES field in the STRUCT_C decoder initialization parameter.

For Advanced profile, if this parameter is not specified, a value of 1 SHALL be assumed.

mode

The value is an integer that specifies how the sequence layer header and the entry-point header are used. This parameter is only defined for Advanced profile. The following values are defined:

  • 0: Both the sequence layer header and the entry-point header may change, and changed headers will be included in the RTP packets.
  • 1: The sequence layer header specified in the config parameter never changes. The rules in section 4.8 of RFC 4425 MUST be followed.
  • 3: The sequence layer header and the entry-point header specified in the config parameter never change. The rules in section 4.9 of RFC 4425 MUST be followed.

If the mode parameter is not specified, a value of 0 SHALL be assumed. The mode parameter SHOULD be specified if modes 1 or 3 apply to the VC-1 bit stream 🧁

max-width, max-height, max-bitrate, max-buffer, max-framerate

These parameters are for use in a capability exchange procedure. They don't signal properties of the coded bit stream. Instead, they set upper limits or preferred values for the width, height, bitrate, buffer, and framerate parameters. Section 6.3 of RFC 4425 provides specific rules for how these parameters are used with the SDP Offer/Answer model ✨

Receivers that signal support for a given profile and level MUST support the maximum values for these parameters for that profile and level. For example, a receiver that indicates support for Main profile, Low level, must support a width of 352 luma samples and a height of 288 luma samples, even if that means scaling the image to fit a smaller display device 📺

A receiver MAY use any of the max-width, max-height, max-bitrate, max-buffer, and max-framerate parameters to indicate preferred capabilities. For example, a receiver may set max-width and max-height to match the resolution of its display device, since a bit stream encoded with those values wouldn't need to be rescaled 🌈

If any of the max-width, max-height, max-bitrate, max-buffer, and max-framerate parameters signal a capability that is less than the required capabilities of the signaled profile and level, then the parameter SHALL be interpreted as a preferred value for that capability.

Any of these parameters MAY also be used to signal capabilities that exceed the required capabilities of the signaled profile and level. In that case, the parameter SHALL be interpreted as the maximum value that can be supported for that capability.

When more than one parameter from the set (max-width, max-height, max-bitrate, max-buffer, and max-framerate) is present, all signaled capabilities MUST be supported simultaneously.

A sender or receiver MUST NOT use these parameters to signal capabilities that meet the requirements of a higher level of the VC-1 profile than the one specified in the level parameter, even if the sender or receiver can support all the properties of the higher level. The exception is when specifying a higher level isn't allowed because of other restrictions. As an example of such a restriction, in the SDP Offer/Answer model, the value of the level parameter that can be used in an Answer is limited by what was specified in the Offer 🙈

max-width

The value is an integer greater than zero, and it's a horizontal size for the coded frames, in luma samples (pixels in the luma picture). If the value is less than the maximum horizontal size allowed by the profile and level, then the value specifies the preferred horizontal size. Otherwise, it specifies the maximum horizontal size that is supported 💖

If this parameter is not specified, it defaults to the maximum horizontal size allowed by the specified profile and level.

max-height: The value is an integer greater than zero, specifying a vertical size for the coded frames, in luma samples (pixels in a progressively coded luma picture). If the value is less than the maximum vertical size allowed by the profile and level, then the value specifies the preferred vertical size. Otherwise, it specifies the maximum vertical size that is supported.

If this parameter is not specified, it defaults to the maximum vertical size allowed by the specified profile and level.

max-bitrate: The value is an integer greater than zero, specifying a peak transmission rate for the coded bit stream in bits per second. The number does not include the overhead caused by RTP encapsulation, i.e., it does not include the AU headers, or any of the RTP, UDP, or IP headers.

If the value is less than the maximum bit rate allowed by the profile and level, then the value specifies the preferred bit rate. Otherwise, it specifies the maximum bit rate that is supported.

If this parameter is not specified, it defaults to the maximum bit rate allowed by the specified profile and level. See the values for "RMax" in Annex D of SMPTE 421M [1].

max-buffer: The value is an integer specifying a leaky bucket size, B, in milliseconds, required to contain a stream transmitted at the transmission rate specified by the max-bitrate parameter. This parameter is defined in the hypothetical reference decoder model for VC-1, in Annex C of SMPTE 421M [1].

Note that this parameter relates to the codec bit stream only and does not account for any buffering time that may be required to compensate for jitter in the network.

If the value is less than the maximum leaky bucket size allowed by the max-bitrate parameter and the profile and level, then the value specifies the preferred leaky bucket size. Otherwise, it specifies the maximum leaky bucket size that is supported for the bit rate specified by the max-bitrate parameter.

If this parameter is not specified, it defaults to the maximum buffer size allowed by the specified profile and level. See the values for "BMax" and "RMax" in Annex D of SMPTE 421M [1].

max-framerate: The value is an integer greater than zero, specifying a number of frames per second for the coded bit stream. The value is the frame rate multiplied by 1000 and rounded to the nearest integer value. For example, 30000/1001 (approximately 29.97) frames per second is represented as 29970.

If the value is less than the maximum frame rate allowed by the profile and level, then the value specifies the preferred frame rate. Otherwise, it specifies the maximum frame rate that is supported.

If the parameter is not specified, it defaults to the maximum frame rate allowed by the specified profile and level.

Encoding considerations: This media type is framed and contains binary data.

Security considerations: See Section 7 of RFC 4425.

Interoperability considerations: None.

Published specification: RFC 4425.

Applications that use this media type: Multimedia streaming and conferencing tools.

Additional Information: None.

Person & email address to contact for further information: Anders Klemets <anderskl@microsoft.com> IETF AVT working group.

Intended Usage: COMMON

Restrictions on usage: This media type depends on RTP framing; therefore, it is only defined for transfer via RTP [3].

Authors: Anders Klemets

Change controller: IETF Audio/Video Transport Working Group delegated from the IESG.

6.2. Mapping of media type parameters to SDP

The information carried in the media type specification has a specific mapping to fields in the Session Description Protocol (SDP) [4]. If SDP is used to specify sessions using this payload format, the mapping is done as follows:

  • The media name in the "m=" line of SDP MUST be video (the type name).
  • The encoding name in the "a=rtpmap" line of SDP MUST be vc1 (the subtype name).
  • The clock rate in the "a=rtpmap" line MUST be 90000.
  • The REQUIRED parameters "profile" and "level" MUST be included in the "a=fmtp" line of SDP.

These parameters are expressed in the form of a semicolon separated list of parameter=value pairs.

  • The OPTIONAL parameters "config", "width", "height", "bitrate", "buffer", "framerate", "bpic", "mode", "max-width", "max-height", "max-bitrate", "max-buffer", and "max-framerate", when present, MUST be included in the "a=fmtp" line of SDP.

These parameters are expressed in the form of a semicolon separated list of parameter=value pairs:

a=fmtp:<dynamic payload type> <parameter
name>=<value>[,<value>][; <parameter name>=<value>]
  • Any unknown parameters to the device that uses the SDP MUST be ignored. For example, parameters defined in later specifications MAY be copied into the SDP and MUST be ignored by receivers that do not understand them.

6.3. Usage with the SDP Offer/Answer Model

When VC-1 is offered over RTP using SDP in an Offer/Answer model [5] for negotiation for unicast usage, the following rules and limitations apply:

  • The "profile" parameter MUST be used symmetrically, i.e., the answerer MUST either maintain the parameter or remove the media format (payload type) completely if the offered VC-1 profile is not supported.
  • The "level" parameter specifies the highest level of the VC-1 profile supported by the codec.

The answerer MUST NOT specify a numerically higher level in the answer than that specified in the offer. The answerer MAY specify a level that is lower than that specified in the offer, i.e., the level parameter can be "downgraded".

If the offer specifies the sendrecv or sendonly direction attribute and the answer downgrades the level parameter, this may require a new offer to specify an updated "config" parameter. If the "config" parameter cannot be used with the level specified in the answer, then the offerer MUST initiate another Offer/Answer round or not use media format (payload type).

  • The parameters "config", "bpic", "width", "height", "framerate", "bitrate", "buffer", and "mode", describe the properties of the VC-1 bit stream that the offerer or answerer is sending for this media format configuration.

In the case of unicast usage and when the direction attribute in the offer or answer is recvonly, the interpretation of these parameters is undefined and they MUST NOT be used.

  • The parameters "config", "width", "height", "bitrate", and "buffer" MUST be specified when the direction attribute is sendrecv or sendonly.
  • The parameters "max-width", "max-height", "max-framerate", "max-bitrate", and "max-buffer" MAY be specified in an offer or an answer, and their interpretation is as follows:

When the direction attribute is sendonly, the parameters describe the limits of the VC-1 bit stream that the sender is capable of producing for the given profile and level, and for any lower level of the same profile.

When the direction attribute is recvonly or sendrecv, the parameters describe properties of the receiver implementation. If the value of a property is less than that allowed by the level of the VC-1 profile, then it SHALL be interpreted as a preferred value and the sender's VC-1 bit stream SHOULD NOT exceed it. If the value of a property is greater than what is allowed by the level of the VC-1 profile, then it SHALL be interpreted as the upper limit of the value that the receiver accepts for the given profile and level, and for any lower level of the same profile.

For example, if a recvonly or sendrecv offer specifies profile=0;level=1;max-bitrate=48000, then 48 kbps is merely a suggested bit rate, because all receiver implementations of Simple profile, Low level, are required to support bit rates of up to 96 kbps. Assuming that the offer is accepted, the answerer should specify bitrate=48000 in the answer, but any value up to 96000 is allowed. But if the offer specifies max-bitrate=200000, this means that the receiver implementation supports a maximum of 200 kbps for the given profile and level (or lower level). In this case, the answerer is allowed to answer with a bitrate parameter of up to 200000.

  • If an offerer wishes to have non-symmetrical capabilities between sending and receiving, e.g., use different levels in each direction, then the offerer has to offer different RTP sessions. This can be done by specifying different media lines declared as "recvonly" and "sendonly", respectively.

For streams being delivered over multicast, the following rules apply in addition:

  • The "level" parameter specifies the highest level of the VC-1 profile used by the participants in the multicast session. The value of this parameter MUST NOT be changed by the answerer. Thus, a payload type can be either accepted unaltered or removed.
  • The parameters "config", "bpic", "width", "height", "framerate", "bitrate", "buffer", and "mode", specify properties of the VC-1 bit stream that will be sent and/or received on the multicast session. The parameters MAY be specified, even if the direction attribute is recvonly.

The values of these parameters MUST NOT be changed by the answerer. Thus, a payload type can be either accepted unaltered or removed.

  • The values of the parameters "max-width", "max-height", "max-framerate", "max-bitrate", and "max-buffer" MUST be supported by the answerer for all streams declared as sendrecv or recvonly. Otherwise, one of the following actions MUST be performed: the media format is removed or the session is rejected.

6.4. Usage in Declarative Session Descriptions

When VC-1 is offered over RTP using SDP in a declarative style, as in RTSP [12] or SAP [13], the following rules and limitations apply:

  • The parameters "profile" and "level" indicate only the properties of the coded bit stream. They do not imply a limit on capabilities supported by the sender.
  • The parameters "config", "width", "height", "bitrate", and "buffer" MUST be specified.
  • The parameters "max-width", "max-height", "max-framerate", "max-bitrate", and "max-buffer" MUST NOT be used.

An example of media representation in SDP is as follows (Simple profile, Medium level):

m=video 49170 RTP/AVP 98
a=rtpmap:98 vc1/90000
a=fmtp:98 profile=0;level=2;width=352;height=288;framerate=15000;
bitrate=384000;buffer=2000;config=4e291800

7. Security Considerations 🔒

okay bestie, security is where things get serious 💀 but we're going to make it make sense 💅

RTP packets using this payload format come with the security stuff from the RTP spec [4] and whatever RTP profile you're using 🎀 So if you want your media streams to be private, encryption is the move ✨ SRTP [11] is a great example of how to do that 🫶

Now the scary part 😭 Some compression tricks make the receiver do way more work on some packets than others, which opens the door to a denial-of-service attack 🙈 A bad actor can slip pathological RTP packets into the stream that are super hard to decode and totally overload the receiver. VC-1 is especially vulnerable because one sneaky frame can mess with how a bunch of future frames get decoded 💀 So, data origin authentication and data integrity protection for at least the RTP packet is RECOMMENDED 🔥 SRTP [11] is one way to get that.

The right way to lock down confidentiality and integrity depends on your application and on the transport and signaling protocols you're using. SRTP is just an example up there. There are other options too 🌈

Next up, user-data 🎀 VC-1 bit streams can carry stuff like closed captioning info and content metadata. The VC-1 spec doesn't say how to interpret user-data, and identifiers for user-data are required to be registered with SMPTE. It's totally possible that some types of user-data could be defined to carry programmatic content, like scripts or commands the receiver would run 😳 Depending on the type, a sender could make non-compliant user-data that crashes the receiver or makes it unavailable for a while. Senders that transport VC-1 bit streams SHOULD make sure the user-data follows the spec registered with SMPTE (see Annex F of [1].) Receivers SHOULD prevent malfunctions when they get non-compliant user-data 🛡️

Another thing 👀 VC-1 streams can need a LOT of bandwidth, up to 135 Mbps for high-definition video 😱 If you blast that onto lots of Internet paths, it could cause denial of service. So users of this payload format MUST follow the congestion control requirements in section 8 💅

8. Congestion Control 🦋

Congestion control for RTP SHALL be used in accordance with RFC 3550 [3], and with any applicable RTP profile; e.g., RFC 3551 [15] ✨

If best-effort service is what you're using, you MUST keep an eye on packet loss so the loss rate stays within acceptable limits 👀 Loss counts as acceptable if a TCP flow on the same network path, under the same conditions, would get an average throughput (measured over a reasonable timescale) that's not less than what your RTP flow is getting. You can satisfy this by using congestion control to adjust your transmission rate, or by having a receiver leave the session if the loss rate gets unacceptably high 🙅‍♀️

Adjusting your bit rate to follow congestion control is super easy with real-time encoding 🧚 With pre-encoded content, though, bandwidth adaptation needs one or more of these:

  • Having more than one coded version of the same content at different bit rates. You can usually switch between these in the same RTP session by switching streams at random access point boundaries.
  • Having non-reference frames (like B-frames) in the bit stream. The transmitter can throw these away before putting them into RTP.

Only when non-downgradable parameters (like the VC-1 "profile" parameter) need to change does the media stream have to be ended and restarted 🍓 You can do that by using a different RTP payload type.

No matter how you adapt bandwidth, the resulting bit stream MUST comply with the VC-1 specification [1] 🌸 For example, if you discard non-reference frames, the FRMCNT syntax element (Simple and Main profile frames only) and the optional TFCNTR syntax element (Advanced profile frames only) must still increment as if no frames were dropped. TFCNTR counts frames in display order, which isn't the same as the coded (transmission) order, so the transmitter will have to "look ahead" or buffer some frames 🧠

Another example: when you switch between different representations of the same content, you might need to signal a discontinuity by changing the FRMCNT field or, if the Advanced profile is used, by setting the BROKEN_LINK flag in the entry-point header to 1 ✨

This payload format can also be used on networks that offer quality-of-service guarantees. If you're using enhanced service, receivers SHOULD keep an eye on packet loss to make sure the service they asked for is actually being delivered 👑 If it isn't, they SHOULD assume they're getting best-effort service and act accordingly 💜

9. IANA Considerations 🌈

IANA has registered the media type "video/vc1" and the associated RTP payload format in the Media Types registry and in the RTP Payload Format MIME types registry, as specified in section 6.1 🎀

10. References 📚

10.1. Normative References

  • [1] Society of Motion Picture and Television Engineers, "VC-1 Compressed Video Bitstream Format and Decoding Process", SMPTE 421M.
  • [2] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
  • [3] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, July 2003.
  • [4] Handley, M. and V. Jacobson, "SDP: Session Description Protocol", RFC 2327, April 1998.
  • [5] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with Session Description Protocol (SDP)", RFC 3264, June 2002.
  • [6] Josefsson, S., Ed., "The Base16, Base32, and Base64 Data Encodings", RFC 3548, July 2003.
  • [7] Freed, N. and J. Klensin, "Media Type Specifications and Registration Procedures", BCP 13, RFC 4288, December 2005.
  • [8] Casner, S. and P. Hoschka, "MIME Type Registration of RTP Payload Formats", RFC 3555, July 2003.

10.2. Informative References

  • [9] Srinivasan, S., Hsu, P., Holcomb, T., Mukerjee, K., Regunathan, S.L., Lin, B., Liang, J., Lee, M., and J. Ribas-Corbera, "Windows Media Video 9: overview and applications", Signal Processing: Image Communication, Volume 19, Issue 9, October 2004.
  • [10] Ribas-Corbera, J., Chou, P.A., and S.L. Regunathan, "A generalized hypothetical reference decoder for H.264/AVC", IEEE Transactions on Circuits and Systems for Video Technology, August 2003.
  • [11] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. Norrman, "The Secure Real-time Transport Protocol (SRTP)", RFC 3711, March 2004.
  • [12] Schulzrinne, H., Rao, A., and R. Lanphier, "Real Time Streaming Protocol (RTSP)", RFC 2326, April 1998.
  • [13] Handley, M., Perkins, C., and E. Whelan, "Session Announcement Protocol", RFC 2974, October 2000.
  • [14] Handley, M., Schulzrinne, H., Schooler, E., and J. Rosenberg, "SIP: Session Initiation Protocol", RFC 2543, March 1999.
  • [15] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and Video Conferences with Minimal Control", STD 65, RFC 3551, July 2003.

Acknowledgements 🫶

Huge thanks to Regis Crinon, Miska Hannuksela, Colin Perkins, Shankar Regunathan, Gary Sullivan, Stephan Wenger, and Magnus Westerlund for giving such detailed feedback on this document 💖

Author's Address 🧚

Anders Klemets
Microsoft Corp.
1 Microsoft Way
Redmond, WA 98052
USA

EMail: Anders.Klemets@microsoft.com

Copyright (C) The Internet Society (2006).

This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.

This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property 💼

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.

Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org.

Acknowledgement 🌸

Funding for the RFC Editor function is provided by the IETF Administrative Support Activity (IASA). ✨

📦 where this page comes from