RFC 8450

RTP Payload Format for VC-2 High Quality (HQ) Profile

📅 published October 2018, from ietf

RFC 8450 📜

okay so this RFC is about how to send VC-2 High Quality (HQ) profile video over RTP 🎀 It's titled RTP Payload Format for VC-2 High Quality (HQ) Profile, and it's a Standards Track document from the Internet Engineering Task Force (IETF), written by J. Weaver of the BBC 💅 It was published in October 2018, ISSN 2070-1721. The header doesn't list anything it obsoletes or updates, so it's a clean new spec ✨

Abstract

this memo describes an RTP payload format for the High Quality (HQ) profile of Society of Motion Picture and Television Engineers Standard ST 2042-1, aka VC-2 🦋 it covers how HQ Profile VC-2 gets carried inside RTP packets, and it's built for low-complexity, high-bandwidth streaming of both lossless and lossy compressed video 🎬

the HQ profile is made for low-latency video compression, where the latency can be on the order of lines of video, at high data rates (compression ratios around 2:1 or 4:1) 🔥

Status of This Memo

this is an Internet Standards Track document 🌸

it's a product of the IETF and represents the consensus of the IETF community. it got public review and was approved for publication by the Internet Engineering Steering Group (IESG). for more on Internet Standards, check Section 2 of RFC 7841 🧚

for info on the current status, any errata, and how to give feedback, go to https://www.rfc-editor.org/info/rfc8450 💜

Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved. 🌈

this document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as in effect on the publication date. please read these carefully, because they spell out your rights and restrictions for this document 🫶 code components pulled out of this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come with no warranty, as the Simplified BSD License describes 📜

Table of Contents

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
   2.  Conventions, Definitions, and Acronyms  . . . . . . . . . . .   3
   3.  Media Format Description  . . . . . . . . . . . . . . . . . .   3
   4.  Payload Format  . . . . . . . . . . . . . . . . . . . . . . .   4
     4.1.  RTP Header Usage  . . . . . . . . . . . . . . . . . . . .  10
     4.2.  Payload Header  . . . . . . . . . . . . . . . . . . . . .  11
     4.3.  The Choice of Parse Codes (Informative) . . . . . . . . .  13
     4.4.  Stream Constraints  . . . . . . . . . . . . . . . . . . .  14
     4.5.  Payload Data  . . . . . . . . . . . . . . . . . . . . . .  15
       4.5.1.  Reassembling the Data . . . . . . . . . . . . . . . .  16
   5.  Forward Error Correction (FEC) Considerations . . . . . . . .  18
   6.  Congestion Control Considerations . . . . . . . . . . . . . .  18
   7.  Payload Format Parameters . . . . . . . . . . . . . . . . . .  19
     7.1.  Media Type Definition . . . . . . . . . . . . . . . . . .  19
     7.2.  Mapping to the Session Description Protocol (SDP) . . . .  21
     7.3.  Offer/Answer Considerations . . . . . . . . . . . . . . .  21
   8.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  21
   9.  Security Considerations . . . . . . . . . . . . . . . . . . .  21
   10. References  . . . . . . . . . . . . . . . . . . . . . . . . .  22
     10.1.  Normative References . . . . . . . . . . . . . . . . . .  22
     10.2.  Informative References . . . . . . . . . . . . . . . . .  23
   Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  24

1. Introduction ✨

this memo specifies an RTP payload format for the video coding standard Society of Motion Picture and Television Engineers ST 2042-1:2017 [VC2], also known as VC-2 🎀

VC-2 is a wavelet-based codec made mainly for professional video with high bit-rates and only low levels of compression 💖 it was designed to be low in complexity and potentially very low latency through both the encoder and the decoder. with some parameter choices, that latency can be as low as a few lines of video 🦋

because the complexity is so low, VC-2 can do this low-latency thing, but it also means the codec doesn't have many of the fancier compression tricks other codecs use. so it's best for low compression ratios, where the coded data rate lands around half to a quarter of uncompressed video, with a similar visual quality 🍓

the main use for VC-2 is probably professional video production environments 🎬

2. Conventions, Definitions, and Acronyms 💅

the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be read as described in BCP 14 [RFC2119] [RFC8174], but only when they appear in all capitals, just like shown here 🔥

3. Media Format Description 🌸

the VC-2 spec says a VC-2 Stream is made of one or more Sequences. each Sequence can be decoded on its own, and it carries all the parameters and metadata the decoder needs to configure itself 🧚

each Sequence is a run of 13-octet Parse Info Headers and variable-length Data Units. the Sequence starts and ends with a Parse Info Header, and every Data Unit has a Parse Info Header in front of it. Data Units come in several types, and the type gets signaled in the Parse Info Header just before it 💜 the big ones are the Sequence Header, which holds the config data the decoder needs, and a few kinds of Coded Picture, which hold the coded data for the pictures themselves. each picture is either a frame in a progressively scanned video Sequence or a field in an interlaced one 🌈

the first Data Unit in a Sequence made by an encoder is always a Sequence Header. BUT Sequences can get joined in the middle, so you shouldn't assume the first Data Unit you receive will always be a Sequence Header 👀

the HQ profile for VC-2 limits which Parse Info Header types can show up in a Sequence (and so which Data Unit types too) to just these:

  • Sequence Headers (which are always followed by a Data Unit),
  • High Quality Pictures (which are always followed by a Data Unit),
  • High Quality Fragments (which are always followed by a Data Unit),
  • Auxiliary Data (which are always followed by a Data Unit),
  • Padding Data (which are always followed by a Data Unit), and
  • End of Sequence (which are never followed by a Data Unit).

at the time of writing, there's no definition for how Auxiliary Data gets used in VC-2, and Padding Data has to be ignored by all receivers 🙈

each High Quality Picture Data Unit has a set of parameters for the picture, followed by a series of Coded Slices. each slice is a rectangular region of the transformed picture. slices in one picture can differ in coded length, but they all share the same rectangle shape and size 🦋

each High Quality Fragment Data Unit holds either a set of parameters for a picture or a series of Coded Slices. fragments carry the same data as pictures, just chopped into smaller pieces so they can move over packet-based protocols like RTP 🍓

this payload format only uses Fragments, not pictures ✨

4. Payload Format ✨

in this spec, each RTP packet carries the data for one Parse Info Header and the Data Unit that follows it (if there is one). a single packet MAY NOT contain data from more than one Parse Info Header or Data Unit. a single Parse Info Header and Data Unit pair MUST NOT be split across more than one packet. the one exception is that an Auxiliary Data Unit MAY be split between multiple packets, and the B and E flags say where it starts and ends 🎀

this spec only covers transporting Sequence Headers (with their Data Unit), High Quality Fragments (with their Data Unit), Auxiliary Data (with its Data Unit), and (optionally) End Sequence Headers and Padding Data (which don't carry a Data Unit) 💖

High Quality Pictures can be sent by converting them into an equivalent set of High Quality Fragments. fragments should be sized so they fit within the MTU of the network in use 🫶

so this document defines six types of RTP packets in a VC-2 media stream 👑

  • a VC-2 Sequence Header (Figure 1) (see Section 11 of the VC-2 specification [VC2]),
  • a Picture Fragment containing the VC-2 Transform Parameters for a Picture (Figure 2) (see Section 14 of the VC-2 specification [VC2]),
  • a Picture Fragment containing VC-2 Coded Slices (Figure 3) for a picture (see Section 14 of the VC-2 specification [VC2]),
  • the end of a VC-2 Sequence (Figure 4) (see Section 10.5.2 of the VC-2 specification [VC2]),
  • the contents of an Auxiliary Data Unit (Figure 5) (see Section 10.4.4 of the VC-2 specification [VC2]), and
  • an indication of the presence of a padding Data Unit (Figure 6) (see Section 10.4.5 of the VC-2 specification [VC2]).

these six packet types can be told apart by their different codes in the Parse Code ("PC") field, except for the two types of Picture Fragment, which use the same PC value but have different values in the No. of Slices field 🌟

the options for PC codes are explained in more detail in Section 4.3 🧚

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | V |P|X|   CC  |M|    PT       |       Sequence Number         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           Timestamp                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           synchronization source (SSRC) identifier            |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |            contributing source (CSRC) identifiers             |
   |                             ....                              |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |                 Optional Extension Header                     |
   |                             ....                              |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |   Extended Sequence Number    |    Reserved   |   PC = 0x00   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
   .                                                               .
   .               Variable-Length Coded Sequence Header           .
   .                                                               .
   +---------------------------------------------------------------+

Figure 1: RTP Payload Format for Sequence Header 💜

okay bestie, that's the first chunk! the diagram above is Figure 1, the packet layout for a Sequence Header, and more of the spec comes next 🦋✨

Okay bestie, the figures are next, and they're giving a lot of bit-level drama 🎀✨

Figure 2 is the Transform Parameters fragment layout 🧚

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | V |P|X|   CC  |M|    PT       |       Sequence Number         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           Timestamp                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           synchronization source (SSRC) identifier            |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |            contributing source (CSRC) identifiers             |
   |                             ....                              |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |                 Optional Extension Header                     |
   |                             ....                              |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |   Extended Sequence Number    |  Reserved |I|F|   PC = 0xEC   |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |                       Picture Number                          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
   |       Slice Prefix Bytes      |        Slice Size Scaler      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
   |       Fragment Length         |         No. of Slices = 0     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
   .                                                               .
   .         Variable-Length Coded Transform Parameters            .
   .                                                               .
   +---------------------------------------------------------------+

Figure 2: RTP Payload Format for Transform Parameters Fragment

Figure 3 is the one for a fragment that carries slices 🦋

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | V |P|X|   CC  |M|    PT       |       Sequence Number         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           Timestamp                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           synchronization source (SSRC) identifier            |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |            contributing source (CSRC) identifiers             |
   |                             ....                              |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |                 Optional Extension Header                     |
   |                             ....                              |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |   Extended Sequence Number    |  Reserved |I|F|   PC = 0xEC   |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |                       Picture Number                          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
   |       Slice Prefix Bytes      |        Slice Size Scaler      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
   |       Fragment Length         |          No. of Slices        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
   |        Slice Offset X         |         Slice Offset Y        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
   .                                                               .
   .                          Coded Slices                         .
   .                                                               .
   +---------------------------------------------------------------+

Figure 3: RTP Payload Format for Fragment Containing Slices

Figure 4 is the End of Sequence layout 🌸

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | V |P|X|   CC  |M|    PT       |       Sequence Number         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           Timestamp                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           synchronization source (SSRC) identifier            |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |            contributing source (CSRC) identifiers             |
   |                             ....                              |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |                 Optional Extension Header                     |
   |                             ....                              |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |   Extended Sequence Number    |    Reserved   |   PC = 0x10   |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+

Figure 4: RTP Payload Format for End of Sequence

Figure 5 is the Auxiliary Data one 💜

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | V |P|X|   CC  |M|    PT       |       Sequence Number         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           Timestamp                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           synchronization source (SSRC) identifier            |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |            contributing source (CSRC) identifiers             |
   |                             ....                              |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |                 Optional Extension Header                     |
   |                             ....                              |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |   Extended Sequence Number    |B|E|  Reserved |   PC = 0x20   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           Data Length                         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   .                                                               .
   .                      Uncoded Payload Data                     .
   .                                                               .
   +---------------------------------------------------------------+

Figure 5: RTP Payload Format for Auxiliary Data

Figure 6 is the Padding Data one, last of the diagrams 🧚‍♀️

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   | V |P|X|   CC  |M|    PT       |       Sequence Number         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           Timestamp                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           synchronization source (SSRC) identifier            |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |            contributing source (CSRC) identifiers             |
   |                             ....                              |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |                 Optional Extension Header                     |
   |                             ....                              |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |   Extended Sequence Number    |B|E|  Reserved |   PC = 0x30   |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           Data Length                         |
   +---------------------------------------------------------------+

Figure 6: RTP Payload Format for Padding Data

Quick vibe check before the fields: every header field longer than one bit is read as an unsigned integer in network byte order. That's the math part, and it's non-negotiable 🔥

4.1. RTP Header Usage 💅

Okay, so the RTP header fields get a few extra notes on how to use them. Let's go 👑

Marker Bit (M): 1 bit. The marker bit MUST be set on any packet that contains the final slice in a coded picture and MUST NOT be set otherwise. 🚩

Payload Type (PT): 7 bits. This is a dynamically allocated payload type field that designates the payload as VC-2-coded video. Iconic 🎀

Sequence Number: 16 bits. VC-2-coded Streams can have a super high data rate, like gigabits per second instead of megabits, so the standard 16-bit RTP sequence number can cycle really fast 😭 That's why the sequence number is extended to 32 bits, and this field MUST hold the low-order 16 bits of this value. 🫶

Timestamp: 32 bits. If the packet has Transform Parameters or Coded Slice data for a coded picture, the timestamp matches the sampling instant of that coded picture. A 90kHz clock SHOULD be used. A single RTP packet MUST NOT contain coded data for more than one coded picture, so there's no ambiguity here, periodt ✨

A Sequence Header packet SHOULD have the same timestamp as the picture that will follow it in the Stream. An End of Sequence packet SHOULD have the same timestamp as the previous picture that appeared in the Stream. 🌈

The remaining RTP header fields are used as specified in RTP [RFC3550]. 💖

4.2. Payload Header 💖

okay so here's how the fields of the extended headers work 🎀

Extended Sequence Number: 16 bits. It MUST contain the high-order 16 bits of the 32-bit packet sequence number. This is giving real necessity 😭 because VC-2 Sequences run at such high data rates that the 16-bit sequence number is highly likely to roll over way too often to be useful for Stream synchronization.

B: 1 bit. It MUST be set to 1 if the packet contains the first byte of an Auxiliary Data Unit, and otherwise it MUST be 0. If the recommendations in Section 4.4 ("Stream Constraints") are followed, then every Auxiliary Data Unit will be small enough to fit in a single packet, so this bit (where present) will always be 1 ✨

E: 1 bit. It MUST be set to 1 if the packet contains the final byte of an Auxiliary Data Unit, and otherwise it MUST be 0. If the recommendations in Section 4.4 ("Stream Constraints") are followed, then every Auxiliary Data Unit will be small enough to fit in a single packet, so this bit (where present) will always be 1 🦋

I: 1 bit. It MUST be set to 1 if the packet contains coded picture parameters or slice data from a field in an interlaced frame. It MUST be set to 0 if the packet contains data from any part of a progressive frame.

F: 1 bit. It MUST be set to 1 if the packet contains coded picture parameters or slice data from the second field of an interlaced frame. It MUST be set to 0 if the packet contains data from the first field of an interlaced frame or any part of a progressive frame. 💅

Parse Code (PC): 8 bits. It contains a Parse Code that MUST be the value indicated for the type of data in the packet.

Data Length: 32 bits. For an auxiliary Data Unit, this contains the number of bytes of data contained in the payload section of this packet. If the recommendations in Section 4.4 ("Stream Constraints") are followed, then no Auxiliary Data Unit will be large enough to cause a packet to exceed the MTU of the network. 🌸

Picture Number: 32 bits. It MUST contain the Picture Number for the coded picture this packet contains data for, as described in Section 12.1 of the VC-2 specification [VC2].

Here's the vibe check for this one: the sender MUST send at least one transform-parameters packet for each coded picture, and it MAY include more than one as long as they contain identical data. The sender MUST NOT send a packet from a new picture until all the coded data from the current picture has been sent. Main character energy, no skipping ahead 👑

If the receiver gets Coded Slices packets for a picture but never gets a Transform Parameters packet for that picture, that's a sign of either packet loss, joining a Stream mid-picture, or a non-compliant transmitter 🙈 In that case, the receiver MAY assume that the parameters are unchanged since the last picture, or it MAY discard the picture. Choosing between those two is left up to the implementation, since it depends on intended use. The former may result in malformed pictures, while the latter will result in dropped frames 💀

Slice Prefix Bytes: 16 bits. It MUST contain the Slice Prefix Bytes value for the coded picture this packet contains data for, as described in Section 12.3.4 of the VC-2 specification [VC2].

Okay, quick side note: in the VC-2 specification, this value isn't restricted to 16 bits, but the constraints on Streams specified in this document (Section 4.4) do require it to be. 🧚

Slice Size Scaler: 16 bits. It MUST contain the Slice Size Scaler value for the coded picture this packet contains data for, as described in Section 12.3.4 of the VC-2 specification [VC2].

Same energy here: in the VC-2 specification, this value isn't restricted to 16 bits, but the constraints on Streams specified in this document (Section 4.4) do require it to be. 💜

Fragment Length: 16 bits. It MUST contain the number of bytes of data contained in the coded payload section of this packet.

No. of Slices: 16 bits. It MUST contain the number of Coded Slices contained in this packet, which MUST be 0 for a packet containing Transform Parameters. In a packet containing Coded Slices, this number MUST be the number of whole slices contained in the packet, and the packet MUST NOT contain any partial slices. No drama, only whole slices 🍓

Slice Offset X: 16 bits. It MUST contain the X coordinate of the first slice in this packet, in slices, starting from the top left corner of the picture.

Slice Offset Y: 16 bits. It MUST contain the Y coordinate of the first slice in this packet, in slices, starting from the top left corner of the picture.

4.3. The Choice of Parse Codes (Informative) 🧚

okay so the "PC" field in the packets carries the Parse Code, which tells you what kind of content is in the packet. This code matches the value of the Parse Code used to identify each Data Unit in a VC-2 Stream, as defined in the VC-2 specification, and each packet contains the entire Data Unit. 🎀

Figure 7 lists all of the Parse Codes currently allowed in a VC-2 Sequence. The final column tells you whether that code can show up in a Stream sent using this specification. Here's the lineup 💖

+----------+-----------+---------------------+---------------+
| PC (hex) | Binary    | Description         | Valid         |
+----------+-----------+---------------------+---------------+
| 0x00     | 0000 0000 | Sequence Header     | Yes           |
| 0x10     | 0001 0000 | End of Sequence     | Yes           |
| 0x20     | 0010 0000 | Auxiliary Data      | Yes           |
| 0x30     | 0011 0000 | Padding Data        | Yes           |
+----------+-----------+---------------------+---------------+
| 0xC8     | 1100 1000 | LD Picture          | No            |
| 0xE8     | 1110 1000 | HQ Picture          | No            |
| 0xEC     | 1110 1100 | HQ Picture Fragment | Yes           |
+----------+-----------+---------------------+---------------+

Figure 7: Parse Codes and Meanings

4.4. Stream Constraints ✨

A Sequence needs to conform to certain constraints in order to be transmissible with this specification. So basically, it has to fit the rules to get through 🫶

  • The Sequence MUST NOT contain Parse Info Headers with a Parse Code other than 0x00 (Sequence Header), 0x10 (End of Sequence), 0x20 (Auxiliary Data), 0x30 (Padding Data), or 0xEC (High Quality Picture Fragment). Some other Streams MAY be convertible to meet this restriction (see below).
  • Every High Quality Picture Fragment MUST be no longer than 65535 bytes. This can usually be ensured by splitting large Fragments into several smaller Fragments, except in the case where an individual slice is too large, in which case see the notes below on conversion. 😭
  • Informative note: this requirement ensures that every High Quality Picture Fragment will always contain no more than 65535 slices.
  • Every High Quality Picture Fragment SHOULD be small enough that the RTP packet carrying it will fit within the network MTU size. This can usually be ensured by splitting large Fragments into several smaller Fragments, except in the case where an individual slice is too large, in which case see the notes below on conversion.
  • Every High Quality Picture Fragment MUST be encoded using values for Slice Prefix Bytes and Slice Size Scaler no greater than 65535.

If a Sequence intended for transmission doesn't conform to these restrictions, then it MAY be possible to simply convert it into a form that does, by splitting pictures and/or large Fragments into suitably sized Fragments. That works provided the following (weaker) constraints are met 🦋

  • The Sequence does not contain Parse Info Headers with a Parse Code other than 0x00 (Sequence Header), 0x10 (End of Sequence), 0x20 (Auxiliary Data), 0x30 (Padding Data), 0xE8 (High Quality Picture), or 0xEC (High Quality Picture Fragment).
  • None of the High Quality Pictures or High Quality Picture Fragments contain slices that are individually longer than 65535 bytes. Note: When this is the case, the values of Slice Prefix Bytes and Slice Size Scaler will necessarily also be smaller than 65535.
  • None of the High Quality Pictures or High Quality Picture Fragments contain slices that are individually so large that an RTP packet carrying a Fragment containing that single slice will fit within the network MTU size.

It is not possible to send a Stream that doesn't meet the above requirements via this mechanism unless the Stream is re-encoded by a VC-2 Encoder so as to meet them. Re-encode it or it's not getting through 💀

In addition, every Auxiliary Data Unit SHOULD be small enough that a single RTP packet carrying it will fit within the network MTU size. Since there is currently no specification for the format of Auxiliary Data in VC-2, the mechanism for ensuring this with an encoder implementation that includes Auxiliary Data Units will be dependent upon the implementation's use for them. 🌈

When encoding VC-2 video intended to be transported via RTP, a VC-2 profile and level that ensures these requirements are met SHOULD be used. 👑

4.5. Payload Data 🎀

For the Sequence Header packet type (PC = 0x00), the payload data MUST be the coded Sequence Header exactly as it appears in the VC-2 Sequence. Exactly as it appears, no edits 💅

For the Transform Parameters packet type (PC = 0xEC and No. of Slices = 0), the payload data MUST be the variable-length coded Transform Parameters. This MUST NOT include the Fragment header, since all data in the picture header is already included in the packet header. No double-dipping on the headers 🙈

For the Auxiliary Data packet type (PC = 0x20), the payload data MUST be a portion of the auxiliary data bytes contained in the Auxiliary Data Unit being transmitted. The B flag MUST be set on the packet that contains the first byte, the E flag MUST be set on the packet that contains the last byte, the bytes MUST be included in order, and the packets MUST have contiguous sequence numbers. ✨

For the Picture Fragment packet type (PC = 0xEC and No. of Slices > 0), the payload data MUST be a specified number of Coded Slices in the same order that they appear in the VC-2 Stream. Which slices appear in the packet is identified using the Slice Offset X and Slice Offset Y fields in the payload header. 🦋

For the End of Sequence packet type (PC = 0x10), there is no payload data. 🌸

4.5.1. Reassembling the Data ✨

Okay bestie, this is where the receiver gets to put the whole puzzle back together 🧩 Here's the Parse Info Header layout they'll be building with:

     0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      0x42     |      0x42     |      0x43     |      0x44     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Parse Code   |                       Next Parse Offset
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |                       Prev Parse Offset
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                   |
   +-+-+-+-+-+-+-+-+

                     Figure 8: VC-2 Parse Info Header

To reassemble the data in the RTP packets into a valid VC-2 Sequence, the receiver follows these steps:

  • Okay first up, the receiver SHOULD take the data from each packet with a Parse Code of 0x00 and prepend a valid VC-2 Parse Info Header (Figure 8) with the same Parse Code (0x00) 💖 The resulting Sequence Header Parse Info Header and Data Unit MUST be included in the output stream before any coded pictures that followed the packet being processed in the RTP stream, unless an identical Sequence Header has already been included. They MAY be repeated (with appropriate modifications to the next and previous header offsets) at any point that results in a valid VC-2 Stream.
  • Next, the receiver SHOULD take the data from each packet with a Parse Code of 0xEC and No. of Slices set to 0 (which together indicate that this packet contains the Transform Parameters for a coded picture). Then it prepends, with the same Parse Code, a valid VC-2 Parse Info Header (Figure 8) followed by the picture number, Fragment data length, and slice count (0) 🎀
  • The receiver SHOULD take the data from each packet with a Parse Code of 0xEC and No. of Slices not set to 0 (which together indicate that this packet contains Coded Slices). It prepends, with the same Parse Code, a valid VC-2 Parse Info Header (Figure 8) followed by the picture number, Fragment data length, slice count, x offset and y offset taken from the packet header ✨
  • A receiver MAY combine all Fragment Data Units (with Parse Code 0xEC) and the same picture number into a single picture Data Unit with Parse Code 0xE8. If the Stream is required to comply with major versions 1 or 2 of the VC-2 specification, then this MUST be done 👑
  • The receiver SHOULD take the data from each packet with a Parse Code of 0x20 and the B bit set, and prepend a valid VC-2 Parse Info Header (Figure 8) with the Parse Code 0x20. Then it takes each subsequent packet with Parse Code 0x20 without the B bit set and appends its payload to the growing Data Unit. When all packets for a particular Data Unit have been received, it SHOULD be included in the output stream. The final packet for a Data Unit will have the E bit set 🦋
  • Once a Data Unit has been assembled, whether it's a Sequence Header, Coded Picture Fragment, Coded Picture, or Auxiliary Data Unit, the next parse offset and previous parse offset values in its Parse Info Header SHOULD be filled with the offset between the start of the header and the start of the next or previous header 🌸
  • An End of Sequence Parse Info Header MAY be inserted when a packet with Parse Code set to 0x10 is encountered, or at any other time that is allowed in a valid VC-2 Stream. After an End of Sequence Parse Info Header is included in the output stream, either the Stream must end, or it MUST be followed by a Sequence Header indicating the start of a new Sequence. The next parse offset of the End of Sequence header MUST be set to 0, and the previous parse offset SHOULD be filled with the offset from the start of the previous Parse Info Header in the Stream 🧚
  • A Padding Data Parse Info Header MAY be inserted when a packet with Parse Code set to 0x30 and the B bit set is encountered, or at any other time that is allowed in a valid VC-2 Stream. The length of the accompanying Data Unit MAY have any value, and its contents MUST be set to a series of zero bytes. The next parse offset and previous parse offset values in its Parse Info Header SHOULD be filled with the offset between the start of the header and the start of the next or previous header 💜

5. Forward Error Correction (FEC) Considerations ✨

Real talk, VC-2 doesn't protect itself against data loss at all 😭 So it may be useful to employ Forward Error Correction on the Stream. There's a mechanism for doing this to a generic RTP stream, specified in RFC 5109 [RFC5109]. If you're using that mechanism for multilevel protection, the packets SHOULD be assigned to layers based on their packet type. The packet types, in order of importance, are:

  1. Sequence Headers
  2. Fragments containing Transform Parameters
  3. Fragments containing Coded Slices
  4. Auxiliary Data and end of Sequence
  5. Padding

It is RECOMMENDED that, if multilevel protection is to be used, one layer protects all Sequence Header packets, and a second layer protects Sequence Headers and all Fragments. If desired, a third layer MAY protect Auxiliary Data and End of Sequence packets. Padding data SHOULD NOT be protected by FEC 🛡️

6. Congestion Control Considerations

Congestion control for RTP SHALL be used in accordance with RFC 3550 [RFC3550] and any applicable RTP profile, e.g., RFC 3551 [RFC3551]. If best-effort service is being used, there's an extra requirement: users of this payload format MUST monitor packet loss to make sure the packet loss rate stays within acceptable parameters. Circuit Breakers [RFC8083] are an update to RTP [RFC3550] that defines criteria for when you're required to stop sending RTP Packet Streams, and applications implementing this standard MUST comply with it. RFC 8085 [RFC8085] gives extra info on the best practices for applying congestion control to UDP streams 🌈

Also, it should be noted that the expected data rate for RTP sessions using this profile is likely to be in the range of gigabits per second. On a closed network that has been correctly provisioned for those rates, this might not be a problem, but there's always a risk of data getting out onto the open internet 😳

7. Payload Format Parameters

This RTP payload format is identified using the video/vc2 media type, which is registered in accordance with RFC 4855 [RFC4855], using the template of RFC 6838 [RFC6838] 🎀

7.1. Media Type Definition

Here's the registration, bestie 💅

  • Type name: video
  • Subtype name: vc2
  • Required parameters:
    • rate: The RTP timestamp clock rate. Applications using this payload format SHOULD use a value of 90000.
    • profile: The VC-2 profile in use. The only value currently allowed is "HQ".
  • Optional parameters:
    • version: The VC-2 specification version in use. The only currently allowed value is "3", since all Sequences transported using this mechanism will contain HQ Picture Fragment Data Units, which the VC-2 specification [VC2] defines as requiring version 3.
    • level: The VC-2 level in use. Any integer may be used.
  • Encoding considerations: This media type is framed and binary; see Section 4.8 in RFC 6838 [RFC6838].
  • Security considerations: Please see the security considerations in RFC 8450.
  • Interoperability considerations: N/A
  • Published specification: RFC 8450
  • Applications that use this media type: Video Communication.
  • Fragment identifier considerations: N/A
  • Additional information: N/A
  • Person & email address to contact for further information: James P. Weaver <james.barrett@bbc.co.uk>
  • Intended usage: COMMON
  • Restrictions on usage: This media type depends on RTP framing and hence is only defined for transfer via RTP [RFC3550]. Transport within other framing protocols is not defined at this time.
  • Author: James P. Weaver <james.barrett@bbc.co.uk>
  • Change controller: IETF PAYLOAD Working Group delegated from the IESG.
  • Provisional registration? (standards tree only): No

7.2. Mapping to the Session Description Protocol (SDP)

The mapping of the payload format media type defined above, and its parameters, SHALL be done according to Section 3 of RFC 4855 [RFC4855].

  • The type name (video) goes in SDP m= as the media name.
  • The subtype name (vc2) goes in SDP a=rtpmap as the encoding name, followed by a slash (/) and the rate parameter.
  • The required parameter profile and the optional parameters version and level, when present, are included in the a=fmtp attribute line of SDP as a semicolon-separated list of parameter=value pairs.

Version and level SHALL be specified in decimal when present.

Here's a sample SDP mapping for VC-2, bestie 🍓

             m=video 30000 RTP/AVP 112
             a=rtpmap:112 vc2/90000
             a=fmtp:112 profile=HQ;version=3;level=0

In this example, a dynamic payload type 112 is used for vc-2 data. The 90 kHz RTP timestamp rate is specified in the a=rtpmap line after the subtype. In the a=fmtp: line, profile HQ, version 3, and level 0 (unknown or non-standard level) are specified ✨

7.3. Offer/Answer Considerations

All parameters are declarative. 🫶

8. IANA Considerations

IANA has registered video/vc2 as specified in Section 7.1. The media type has been added to the IANA registry for "RTP Payload Format Media Types" at <https://www.iana.org/assignments/rtp-parameters>. 📋

9. Security Considerations

Okay, this part is serious, so listen up 🥺 RTP packets using the payload format defined in this specification are subject to the security considerations discussed in the RTP specification [RFC3550] and any applicable RTP profile, such as RTP/AVP [RFC3551], RTP/AVPF [RFC4585], RTP/SAVP [RFC3711], or RTP/SAVPF [RFC5124]. However, as "Securing the RTP Framework: Why RTP Does Not Mandate a Single Media Security Solution" [RFC7202] explains, it isn't an RTP payload format's job to discuss or mandate which solutions are used to meet the basic security goals of confidentiality, integrity, and source authenticity for RTP in general. That job belongs to whoever is using RTP in an application. They can find guidance on available security mechanisms and important considerations in "Options for Securing RTP Sessions" [RFC7201]. Applications SHOULD use one or more appropriate strong security mechanisms. The rest of this section covers the security-impacting properties of the payload format itself 🔐

This RTP payload format and its media decoder don't show any significant non-uniformity in receiver-side computational complexity for packet processing, so they're unlikely to pose a denial-of-service threat from pathological data. The RTP payload format also doesn't contain any active content 🙈

To avoid buffer overruns when processing these packets, the receiver MUST consider both the reported Fragment length and the actual received size of a packet containing slice data. 🚨

In some cases, the transmitter may need to decode variable-length coded headers to extract some data from the VC-2 bitstream before assembling packets. This process is potentially subject to buffer overruns if it isn't implemented carefully. Please be careful with this one, bestie 💀

10. References ✨

10.1. Normative References ✨

okay so these are the normative references 💅 the ones the spec really leans on, bestie

10.2. Informative References 🌈

these ones are the extra reading, the "ooh tell me more" section 🤭

Author's Address 🎀

James P. Weaver

BBC

Email: james.barrett@bbc.co.uk

📦 where this page comes from