RFC 10034

RTP Payload Format for Visual Volumetric Video-Based Coding (V3C)

πŸ“… published August 2026, from ietf

RFC 10034: RTP Payload Format for Visual Volumetric Video-Based Coding (V3C) πŸŽ€

Okay bestie, first the basics πŸ’– This is RFC 10034, and it's called RTP Payload Format for Visual Volumetric Video-Based Coding (V3C). It was written by L. Ilola and L. Kondrad, both from Nokia Technologies, and it's published under the Internet Engineering Task Force (IETF) with ISSN 2070-1721. It's on the Standards Track πŸ‘‘ It came out in August 2026. The title block doesn't list any RFCs that it obsoletes or updates, so there's no earlier document it replaces or amends πŸ¦‹


Abstract ✨

Okay so a V3C bitstream (that's the ISO/IEC 23090-5 one) is built out of V3C units. Those units hold V3C atlas sub-bitstreams, V3C video sub-bitstreams, and a V3C parameter set 🧚 This document is all about an RTP payload format for the V3C atlas sub-bitstreams. The video sub-bitstreams get their RTP payload format from whichever IETF RFC covers the video codec being used, so that part isn't on this doc's plate πŸ’…

Here's what the V3C RTP payload format can do. It can pack one or more V3C atlas Network Abstraction Layer (NAL) units into a single RTP packet payload, and it can also chop a single V3C atlas NAL unit up across several RTP packets (that's fragmentation) 🌸 The document then explains how to group the RTP streams of the V3C component sub-bitstreams together. Put it all together and you get a complete solution for streaming V3C-encoded content, which is honestly iconic πŸ”₯

Status of This Memo ✨

This is an Internet Standards Track document πŸ‘‘

This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community, and it went through public review before the Internet Engineering Steering Group (IESG) approved it for publication 🫢 If you want more info on Internet Standards, check out Section 2 of RFC 7841.

You can find the current status of this document, any errata, and how to send feedback at https://www.rfc-editor.org/info/rfc10034 πŸ’œ

Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved. 🌈

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they're in effect on the date this document was published. Please read these carefully, because they spell out your rights and restrictions for this document. Code Components pulled out of this document must include Revised BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as described in the Revised BSD License πŸŽ€

Table of Contents πŸ¦‹

1.  Introduction
2.  Conventions
3.  Definitions and Abbreviations
  3.1.  Abbreviations
  3.2.  Definitions
    3.2.1.  General
    3.2.2.  Definitions from the V3C Specification
4.  Media Format Description
  4.1.  Overview of the V3C Codec (Informative)
  4.2.  V3C Parameter Set (Informative)
  4.3.  V3C Atlas and Video Components (Informative)
    4.3.1.  General
    4.3.2.  Atlas NAL Units
  4.4.  Systems and Transport Interfaces (Informative)
5.  V3C Atlas RTP Payload Format
  5.1.  General
  5.2.  RTP Header
  5.3.  RTP Payload Header
  5.4.  Payload Structures
    5.4.1.  General
    5.4.2.  Single NAL Unit Packet
    5.4.3.  Aggregation Packet
    5.4.4.  Fragmentation Unit
    5.4.5.  Example of Fragmentation Unit (Informative)
  5.5.  Decoding Order Number
6.  Packetization and De-Packetization Rules
7.  Payload Format Parameters
  7.1.  Media Type Registration
  7.2.  Required Parameters Definition
  7.3.  Definitions of Optional Parameters
  7.4.  Mapping of Parameters to V3C Syntax
8.  Congestion Control Considerations
9.  Session Description Protocol
  9.1.  V3C Format Parameters "v3cfmtp" Attribute
  9.2.  Mapping of Payload Type Parameters to SDP
    9.2.1.  For V3C Atlas Components
    9.2.2.  For V3C Video Components
  9.3.  Grouping Framework
  9.4.  Offer and Answer Considerations
    9.4.1.  Unicast
    9.4.2.  Multicast
  9.5.  Declarative SDP Considerations
10. IANA Considerations
  10.1.  V3C Media Type Registration
  10.2.  V3C Format Parameters SDP Attribute
  10.3.  V3C Grouping Type Extension
11. Security Considerations
12. References
  12.1.  Normative References
  12.2.  Informative References
Authors' Addresses

1. Introduction 🌸

Okay so volumetric video is basically the same as regular 2D video in one way: uncompressed, it's a massive amount of data 😭 The payoff is huge though, because it lets you capture and play back a 3D object or scene, no matter where or how it was originally captured 🌈

The visual volumetric video-based coding (V3C) specification [ISO.IEC.23090-5] takes the compression wins from existing 2D video codecs and uses them to cut down the amount of data needed to store and send volumetric video. V3C is a generic mechanism for volumetric video coding, so lots of different apps can use it for volumetric content like point clouds, Video-based Point Cloud Compression (V-PCC) [ISO.IEC.23090-5], and immersive video with depth (MPEG Immersive Video (MIV)) [ISO.IEC.23090-12] ✨

Here's how the magic works. A V3C encoder takes volumetric frames (basically, 3D volumetric information) and turns them into a collection of 2D frames plus some associated data called atlas data 🧚 Those 2D frames then get coded with any video or image codec you like. For example, that could be ISO/IEC International Standard 14496-10 (Advanced Video Coding, AVC/H.264) [ISO.IEC.14496-10], ISO/IEC International Standard 23008-2 (High Efficiency Video Coding, HEVC/H.265) [ISO.IEC.23008-2], or ISO/IEC International Standard 23090-3 (Versatile Video Coding, VVC/H.266) [ISO.IEC.23090-3]. The atlas data itself is coded using the mechanisms in [ISO.IEC.23090-5] πŸ’œ

V3C uses a high-level syntax (HLS) design, which should feel super familiar if you've used conventional 2D video codecs, to represent the coded data (that's the atlas data). That coded atlas data is represented by Network Abstraction Layer (NAL) units. So the RTP payload format for V3C atlas data in this document shares its design philosophy, security, congestion control, and overall implementation complexity with the other NAL unit-based RTP payload formats, like the ones in [RFC6184], [RFC6190], and [RFC7798] 🫢

2. Conventions πŸ’…

Okay, this part is the important bit about the key words. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174], but only when they show up in all capitals, exactly like shown here πŸ”₯

Every field defined in this specification that's related to RTP payload structures SHALL be considered in network order πŸŽ€

3. Definitions and Abbreviations 🧚

3.1. Abbreviations ✨

ACL:    atlas coding layer

AP:     aggregation packet

AU:     aggregation unit

CVS:    coded V3C sequence

DON:    decoding order number

DOND:   decoding order number difference

DONL:   decoding order number least significant bits

IRAP:   intra random access point

MTU:    maximum transmission unit

NAL:    network abstraction layer

NALU:   NAL unit

RBSP:   raw byte sequence payload

V3C:    visual volumetric video-based coding

VPS:    V3C parameter set

3.2. Definitions πŸ’–

3.2.1. General 🌸

This document uses the definitions from [ISO.IEC.23090-5]. Section 3.2.2 lists the relevant definitions from [ISO.IEC.23090-5] for convenience πŸ’œ

3.2.2. Definitions from the V3C Specification πŸ¦‹

Here are the V3C definitions you'll be seeing all over this doc, bestie 🌈

  • atlas: a collection of 2D bounding boxes and their associated info, packed onto a rectangular frame, that corresponds to a volume in 3D space where volumetric data gets rendered 🧚
  • atlas bitstream: a sequence of bits that represents atlas frames and their associated data, which together form one or more coded atlas sequences πŸ’œ
  • atlas coding layer NAL unit: the umbrella term for the coded atlas tile layer NAL units, plus any NAL units with reserved nal_unit_type values that this document classifies as having a type class equal to ACL 🫢
  • atlas frame: a 2D rectangular array of atlas samples where patches get projected, along with extra info about those patches, matching one volumetric frame πŸŽ€
  • attribute: a scalar or vector property that can optionally be attached to each point in a volumetric frame, like color, reflectance, surface normal, timestamps, material ID, etc. ✨
  • coded atlas sequence: a run of coded atlas access units in decoding order. It starts with one IRAP-coded atlas access unit, then has zero or more access units that are not IRAP-coded after it, including every subsequent access unit up to, but not including, the next IRAP-coded atlas access unit 🌈
  • coded atlas access unit: a set of atlas NAL units that are linked to each other by a specified classification rule, sit consecutively in decoding order, and contain all the atlas NAL units for one particular output time ⭐
  • coded V3C sequence: a run of V3C atlas and video sub-bitstream(s) that are identified and separated by appropriate delimiters. It has to start with a VPS, and that VPS has to be included in at least one V3C unit or provided through external means πŸ‘‘
  • network abstraction layer unit: a syntax structure that says what type of data follows, plus the bytes holding that data in the form of an RBSP πŸ’…
  • patch: a rectangular region inside an atlas that's associated with volumetric information πŸ¦‹
  • raw byte sequence payload: a syntax structure holding a whole number of bytes, encapsulated in a NAL unit. It's either empty, or it's a string of data bits containing syntax elements, followed by an RBSP stop bit and then zero or more extra bits equal to 0 πŸ™ˆ
  • tile: an independently decodable rectangular region of an atlas frame πŸ“
  • V3C atlas sub-bitstream: a sub-bitstream extracted from the V3C bitstream that contains all or part of an atlas bitstream 🌸
  • V3C video sub-bitstream: a sub-bitstream extracted from the V3C bitstream that contains all or part of a video bitstream πŸ’–
  • V3C component: an atlas, occupancy, geometry, or attribute of a particular type, associated with a V3C volumetric content representation 🌈
  • V3C parameter set: a syntax structure with syntax elements that apply to zero or more entire CVSs, and that syntax elements in the V3C unit header can refer to πŸ‘‘
  • volumetric frame: a set of 3D points, each specified by Cartesian coordinates, plus zero or more matching sets of attributes at one particular time instance 🧚

4.1. Overview of the V3C Codec (Informative) ✨

okay so V3C encoding is basically a glow-up for a volumetric frame πŸ’… The frame gets converted from its 3D representation into multiple 2D representations, and then you generate associated data that documents all those conversions and transformations. That associated data is also called the atlas data, and it's the secret sauce that tells you how to reproject the 2D representations back into the 3D volumetric frame πŸ¦‹

The 2D representations are called V3C video components, and they're encoded with conventional 2D video codecs, so no cap, nothing exotic here 🌸 A V3C video component may include occupancy, geometry, or attribute data, for example. Occupancy data tells a V3C decoder which pixels in the other V3C video components contribute to the reconstructed 3D representation. Geometry data describes where the reconstructed voxels are positioned, and attribute data adds extra properties to the voxels, like color or material info πŸŽ€ A voxel is the smallest discrete addressable element in a 3D space, basically the 3D version of a pixel in 2D. iykyk 🧚

Next up is the atlas data, known as the V3C atlas component. It's what helps you interpret the V3C video components and rebuild a 3D representation of the volumetric frame from its 2D form ✨ The atlas data is made up of a collection of patches. Each patch identifies a region in the V3C video components and gives you the info you need to do the right inverse projection of that region back into 3D space. The shape of a patch region is set by a 2D bounding box attached to each patch, plus their coding order. And then the shape gets refined even further based on the occupancy data, which is giving precision πŸ‘‘

To enable parallelization, random access, and a bunch of other functionalities, an atlas frame can be split into one or more rectangular partitions called tiles 🌈 Tiles are not allowed to overlap, and they should be independently decodable. An atlas frame may also contain regions that aren't associated with any tile or patch, and that's totally fine 🫢

The binary forms of the V3C video components (the video bitstream) and the V3C atlas components (the atlas bitstream) can be grouped together and represented by a single V3C bitstream πŸ’œ That V3C bitstream is made up of a set of V3C units. Each V3C unit has a V3C unit header and a V3C unit payload. The header describes the V3C unit type for the payload, and the payload holds V3C video components, V3C atlas components, or a V3C parameter set. The V3C video components (occupancy, geometry, or attribute) line up with video data units, like the NAL units defined in [ISO.IEC.23008-2], which an appropriate video decoder can decode πŸ“ Here's an example of a V3C bitstream made of a V3C parameter set, an atlas bitstream, and three video component bitstreams (geometry, occupancy, attribute), shown in Figure 1:

    +-------------------+------------------+-------------------+
    | V3C Unit(V3C_VPS) | V3C Unit(V3C_AD) | V3C Unit(V3C_GVD) |
    +-------------------+------------------++-----------------++---
    | V3C Unit(V3C_OVD) | V3C Unit(V3C_AVD) | V3C Unit(V3C_AD)| ...
    +-------------------+-------------------+-----------------+----

                   Figure 1: Example of a V3C Bitstream

4.2. V3C Parameter Set (Informative) 🌸

This document specifies an encapsulation of V3C atlas data, and it also considers the aspects related to signaling the V3C parameter set, which is defined in [ISO.IEC.23090-5] πŸŽ€ The V3C parameter set gets encapsulated in its own V3C unit, which is a lowkey genius move because it decouples the transmission of the parameter set from the V3C video and atlas components πŸ¦‹ The parameter set can be sent by external means (for example, as a result of a capability exchange) or through a control protocol, which can be reliable or unreliable. Section 9 of this document explains how a V3C parameter set can be signaled using the Session Description Protocol (SDP) πŸ’…

Generally speaking, it's useful to signal a V3C parameter set out of band, because it describes the overall resources needed to decode and reconstruct the associated V3C bitstream ✨

Signaling the parameter set dynamically as part of an RTP stream might result in undefined behavior when the receiver doesn't have the capabilities needed to decode the received V3C video component sub-bitstreams, or when the reconstruction process relies on information the receiver doesn't support. Yikes, bestie 😭

4.3. V3C Atlas and Video Components (Informative) 🧚

4.3.1. General 🌈

In the V3C bitstream, the atlas component is identified by vuh_unit_type equal to V3C_AD, or by V3C_CAD in the case of common atlas data, inside the V3C unit header πŸ’– The V3C atlas component is made of atlas NAL units that define header and payload pairs (see Section 4.3.2) πŸ“ The V3C video components are identified by vuh_unit_type equal to V3C_OVD, V3C_GVD, V3C_AVD, and V3C_PVD. They can be further differentiated by other values in the V3C unit header, such as vuh_attribute_index, vuh_attribute_partition_index, vuh_map_index, and vuh_auxiliary_video_flag 🫢 By mapping the V3C parameter set information to vuh_attribute_index, a V3C decoder figures out which attribute a given V3C video component contains, like color 🎨

The information supplied by a V3C unit header should be provided in one form or another to a V3C decoder, for example as part of SDP as described in Section 9 πŸ¦‹ The four-byte V3C unit header syntax and semantics are copied below as defined in [ISO.IEC.23090-5], but the syntax is subject to change, so implementations should always refer to the latest specification of [ISO.IEC.23090-5] πŸ‘‘ The syntax of the four-byte V3C unit header is provided here for informative purposes only. The integers in the parentheses, like unsigned int(5), show how many bits the syntax element uses πŸ”₯

v3c_unit_header( ) {
 unsigned int(5) vuh_unit_type;
 if( vuh_unit_type == V3C_AVD || vuh_unit_type == V3C_GVD ||
   vuh_unit_type == V3C_OVD || vuh_unit_type == V3C_AD ||
   vuh_unit_type == V3C_CAD || vuh_unit_type == V3C_PVD ) {
   unsigned int(4) vuh_v3c_parameter_set_id;
  }
  if( vuh_unit_type == V3C_AVD || vuh_unit_type == V3C_GVD ||
    vuh_unit_type == V3C_OVD || vuh_unit_type == V3C_AD ||
    vuh_unit_type == V3C_PVD ) {
    unsigned int(6) vuh_atlas_id;
  }
  if( vuh_unit_type == V3C_AVD ) {
    unsigned int(7) vuh_attribute_index;
    unsigned int(5) vuh_attribute_partition_index;
    unsigned int(4) vuh_map_index;
    unsigned int(1) vuh_auxiliary_video_flag;
  }
  else if( vuh_unit_type == V3C_GVD ) {
    unsigned int(4) vuh_map_index;
    unsigned int(1) vuh_auxiliary_video_flag;
    bit(12) vuh_reserved_zero_12bits;
  }
  else if( vuh_unit_type == V3C_OVD || vuh_unit_type == V3C_AD ||
      vuh_unit_type == V3C_PVD) {
    bit(17) vuh_reserved_zero_17bits;
  }
  else if( vuh_unit_type == V3C_CAD ) {
    bit(23) vuh_reserved_zero_23bits;
  }
  else {
    bit(27) vuh_reserved_zero_27bits;
  }
}

vuh_unit_type indicates the V3C unit type for the V3C component, as specified in [ISO.IEC.23090-5] πŸ’œ For convenience, the mapping table from vuh_unit_type values to their semantics is shown in Table 1 πŸ’…

+===============+============+===========+======================+
| vuh_unit_type | Identifier | V3C unit  | Description          |
|               |            | type      |                      |
+===============+============+===========+======================+
| 0             | V3C_VPS    | V3C       | V3C level parameters |
|               |            | parameter |                      |
|               |            | set       |                      |
+---------------+------------+-----------+----------------------+
| 1             | V3C_AD     | Atlas     | Atlas information    |
|               |            | data      |                      |
+---------------+------------+-----------+----------------------+
| 2             | V3C_OVD    | Occupancy | Occupancy            |
|               |            | video     | information          |
|               |            | data      |                      |
+---------------+------------+-----------+----------------------+
| 3             | V3C_GVD    | Geometry  | Geometry information |
|               |            | video     |                      |
|               |            | data      |                      |
+---------------+------------+-----------+----------------------+
| 4             | V3C_AVD    | Attribute | Attribute            |
|               |            | video     | information          |
|               |            | data      |                      |
+---------------+------------+-----------+----------------------+
| 5             | V3C_PVD    | Packed    | Packing information  |
|               |            | video     |                      |
|               |            | data      |                      |
+---------------+------------+-----------+----------------------+
| 6             | V3C_CAD    | Common    | Information that is  |
|               |            | atlas     | common for atlases   |
|               |            | data      | in a CVS.  Specified |
|               |            |           | in ISO/IEC 23090-12. |
+---------------+------------+-----------+----------------------+
| 7...31        | V3C_RSVD   | Reserved  | -                    |
+---------------+------------+-----------+----------------------+

                 Table 1: V3C Unit Type Semantics

Here's the rest of the header breakdown, bestie πŸŽ€

  • vuh_v3c_parameter_set_id specifies the value of vps_v3c_parameter_set_id for the active V3C VPS. πŸ’–
  • vuh_atlas_id specifies the ID of the atlas that corresponds to the current V3C unit. 🌸
  • vuh_attribute_index indicates the index of the attribute data carried in the Attribute Video Data unit. πŸ¦‹
  • vuh_attribute_partition_index indicates the index of the attribute dimension group carried in the attribute video data unit. πŸ‘‘
  • vuh_map_index, when present, indicates the map index of the current geometry or attribute stream. When it's not present, the map index of the current geometry or attribute sub-bitstream is derived based on the type of the sub-bitstream. 🧚
  • vuh_auxiliary_video_flag equal to 1 indicates that the associated geometry or attribute video sub-bitstream contains only RAW and/or enhanced occupancy map (EOM) coded points. πŸ”₯

4.3.2. Atlas NAL Units ✨

okay so the atlas NAL unit (nal_unit(NumBytesInNalUnit)) is a byte-aligned syntax structure defined by [ISO.IEC.23090-5] to carry atlas data 🧚. Every atlas NAL unit has a 16-bit NAL unit header (nal_unit_header()), which tells you the type of the NAL unit (nal_unit_type) plus a few other things πŸ’…. The payload is basically the NAL unit minus that header. The atlas NAL unit syntax and semantics are copied here as defined in [ISO.IEC.23090-5], so you're getting the real deal 🫢.

   nal_unit_header(){
       bit(1) nal_forbidden_zero_bit;
       bit(6) nal_unit_type;
       bit(6) nal_layer_id;
       bit(3) nal_temporal_id_plus1;
   }
   nal_unit(NumBytesInNalUnit){
       nal_unit_header();
       NumBytesInRbsp = 0;
       for( i = 2; i < NumBytesInNalUnit; i++ )
         bit(8) rbsp_byte[ NumBytesInRbsp++ ];
   }

nal_forbidden_zero_bit is the one that gives us a way to indicate errors in NAL units 🚨. nal_unit_type tells you what kind of RBSP data structure lives in the NAL unit πŸŽ€. nal_layer_id says which layer an ACL NAL unit belongs to, or which layer a non-ACL NAL unit applies to 🌈. And nal_temporal_id_plus1 minus 1 gives you a temporal identifier for the NAL unit πŸ¦‹.

4.4. Systems and Transport Interfaces (Informative) 🌸

bestie, MPEG didn't stop at the V3C application specs [ISO.IEC.23090-5] and [ISO.IEC.23090-12] πŸ“š. They also went hard on systems-level work for file formats that wrap up compressed V3C content πŸ’–. The seventh edition of the ISO Base Media File Format (ISOBMFF) specification [ISO.IEC.14496-12] brings in a new media handler 'volv', which is made for volumetric visual media 🎬. It also adds other structures so derived specs can explain how different volumetric visual media get stored in ISOBMFF 🧚.

One of those derived specs is [ISO.IEC.23090-10] πŸ’œ. It defines how V3C content can be stored in a file and streamed over DASH [ISO.IEC.23009-1]. Mostly, ISO/IEC 23090-10 is about how ISOBMFF boxes and syntax elements can store volumetric media, but sometimes new boxes and syntax elements get added because this is a totally different kind of media πŸ™ˆ. It's not directly relevant for defining the RTP payload format for V3C atlas data, but it's a super useful resource, especially if you're designing ingestion of encoded V3C content into RTP streaming pipelines πŸ”₯.

5. V3C Atlas RTP Payload Format πŸŽ€

5.1. General

This section is all about the details of the V3C atlas RTP payload format 🌷. We're looking at the RTP header, the RTP payload header, and the general payload structure πŸ’…. Video components get their RTP payload format in their own specs, depending on which video codec is used πŸ¦‹.

5.2. RTP Header

The RTP header format is specified in [RFC3550] and copied into Figure 2 so you don't have to go hunting πŸ”. The V3C RTP payload format uses the RTP header fields in a way that's consistent with [RFC3550] 🫢. Unless it's contextualized below, the meaning of the fields in Figure 2 is the same as in Section 5.1 of [RFC3550] ✨.

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |V=2|P|X|  CC   |M|     PT      |       sequence number         |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                           timestamp                           |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |           synchronization source (SSRC) identifier            |
     +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
     |            contributing source (CSRC) identifiers             |
     |                             ....                              |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Figure 2: RTP Header

Marker bit (M): 1 bit πŸ’– Set for the last packet of the access unit and carried in the current RTP stream. This is in line with the normal use of the M bit in video formats, so playout buffers can be handled efficiently 🎬.

Payload Type (PT): 7 bits 🌈 The assignment of an RTP payload type for this new packet format is outside the scope of this document and will not be specified here πŸ™…. The assignment of a payload type MUST be performed either through the profile used or in a dynamic way.

Timestamp: 32 bits ⏰ The RTP timestamp is set to the sampling timestamp of the content. A 90 kHz clock rate MUST be used.

If the NAL unit has no timing properties of its own (like parameter sets and Supplemental Enhancement Information (SEI) NAL units), the RTP timestamp MUST be set to the RTP timestamp of the coded atlas of the access unit in which the NAL unit is included (according to Section 8.4.5.3 of [ISO.IEC.23090-5]) πŸ•°οΈ.

Receivers MUST use the RTP timestamp for the display process, even when the bitstream contains atlas frame timing SEI messages as specified in [ISO.IEC.23090-5] πŸ‘€.

Everything else in the RTP header just follows [RFC3550] πŸ’….

5.3. RTP Payload Header

The first two bytes of an RTP packet's payload are called the payload header πŸŽ€. It has the same fields (F, NUT, NLI, and TID) as the NAL unit header in Section 4.3.2, no matter what type of payload structure you've got πŸ’–. Here's what it looks like in Figure 3 🧚.

      0                   1
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |F|    NUT    |    NLI    | TID |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Figure 3: RTP Payload Header

F: The nal_forbidden_zero_bit as specified in [ISO.IEC.23090-5] is equal to 0. A value equal to 1 indicates that the payload may contain errors or syntax violations 🚩.

Media processing elements in the network that are capable of deep packet inspection SHOULD set the F bit to 1 to indicate detected bit errors in the NAL unit(s) πŸ”Ž. When a receiver sees an RTP payload header where the F bit is equal to 1, it discards that RTP packet and conceals the lost data in the discarded NAL unit(s) 🫣.

NUT: The nal_unit_type as specified in [ISO.IEC.23090-5] defines the type of the RBSP data structure contained in the NAL unit payload πŸ“¦. The NUT value could carry other meaning depending on the RTP packet type πŸ€”.

NLI: The nal_layer_id as specified in [ISO.IEC.23090-5] defines the identifier of the layer to which an ACL NAL unit belongs, or the identifier of a layer to which a non-ACL NAL unit applies 🌈.

TID: The nal_temporal_id_plus1 minus 1 as specified in [ISO.IEC.23090-5] defines a temporal identifier for the NAL unit πŸ¦‹. The value of nal_temporal_id_plus1 MUST NOT be equal to 0.

5.4. Payload Structures

5.4.1. General

Three different types of RTP packet payload structures are specified πŸ‘‘. A receiver can identify the payload structure from the first two bytes of the RTP packet payload, which double as the RTP payload header 🧐. Those two bytes are always structured as a NAL unit header, and the NAL unit type field tells you which structure is in the payload ✨.

The three different payload structures are as follows:

  • Single NAL Unit Packet: Contains a single NAL unit in the payload. This payload structure is specified in Section 5.4.2.
  • Aggregation Packet: Contains multiple NAL units in a single RTP payload. This payload structure is specified in Section 5.4.3.
  • Fragmentation Unit (FU): Contains a subset of a single NAL unit. This payload structure is specified in Section 5.4.4.
NOTE (informative): This document does not limit the size of NAL units encapsulated in NAL unit packets and fragmentation units. [ISO.IEC.23090-5] does not restrict the maximum size of a NAL unit directly, either. Instead, a NAL unit sample stream format may be used, which provides flexibility to signal NAL unit size up to UINT64_MAX bytes. NOTE (informative): Some of the fields described in the payload structures are conditional, and their presence is indicated through the relevant parameters as defined in Section 7.3. These parameters are assumed to be made available prior to sending any RTP packets.

5.4.2. Single NAL Unit Packet

A single NAL unit packet contains exactly one NAL unit 🎁. It's made of an RTP payload header plus these conditional fields: a 16-bit DONL and a 16-bit v3c-tile-id πŸ’œ. The rest of the payload data is the NAL unit payload data, which excludes the NAL unit header. A single NAL unit packet MUST only contain atlas NAL units of the types defined in Table 4 of [ISO.IEC.23090-5] 🚫. The structure is shown in Figure 4 πŸ‘‡.

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |      RTP payload header       |      DONL (conditional)       |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
     |      v3c-tile-id (cond)       |                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               |
     |                                                               |
     |                        NAL unit data                          |
     |                                                               |
     |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                               :...OPTIONAL RTP padding        |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Figure 4: Single NAL Unit Packet

The RTP payload header MUST be an exact copy of the NAL unit header of the contained NAL unit πŸͺž.

A NAL unit stream made by de-packetizing single NAL unit packets in RTP sequence number order MUST conform to the NAL unit decoding order when DONL is not present 🎢.

The DONL field, when present, specifies the value of the 16-bit decoding order number of the contained NAL unit πŸ”’. That number tells you the order the received NAL units should be reordered in so they form a bitstream that can be successfully decoded 🧩. If sprop-max-don-diff is greater than 0 for any of the RTP streams, the DONL field MUST be present; otherwise, the DONL field MUST NOT be present ⚠️.

The v3c-tile-id field, when present, specifies the 16-bit tile identifier for the NAL unit as signaled in the V3C atlas tile header defined in [ISO.IEC.23090-5] 🧱. If sprop-v3c-tile-id-pres is equal to 1 and the RTP payload header NUT is in range 0-35 inclusive, the v3c-tile-id field MUST be present. Otherwise, the v3c-tile-id field MUST NOT be present πŸ™….

NOTE (informative): Only values for NAL unit type (NUT) in range 0-35 inclusive are allocated for atlas tile layer data in [ISO.IEC.23090-5].

The presence of the "OPTIONAL RTP padding" is indicated by the padding (P) bit in the RTP header πŸŽ€. As defined in [RFC3550], the last octet of the padding contains a count of how many padding octets should be ignored, including itself 🫢.

5.4.3. Aggregation Packet πŸŽ€

okay so APs are basically the packing peanuts of RTP πŸ“¦ They cut down on packetization overhead for tiny NAL units, like most of the non-ACL NAL units, which are often only a few octets in size πŸ₯Ή

APs MAY be used to wrap multiple NAL units that belong to the same access unit in a single RTP payload 🎁 The first two bytes of an AP MUST contain the RTP payload header. The NAL unit type (NUT) for the NAL unit header inside that RTP payload header MUST be equal to 56, which lands in the unspecified range of NAL unit types defined in [ISO.IEC.23090-5] ✨ An AP MAY contain a conditional v3c-tile-id field. An AP MUST contain two or more AUs. Here's the layout, bestie πŸ‘‡

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |  RTP payload header (NUT=56)  |      v3c-tile-id (cond)       |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                                                               |
     |                  Two or more aggregation units                |
     |                                                               |
     |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                               :...OPTIONAL RTP padding        |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Figure 5: Aggregation Packet (AP)

The fields in the payload header are set like this πŸ’… The F bit MUST be equal to 0 if the F bit of each aggregated NAL unit is equal to zero; otherwise, it MUST be equal to 1. The NUT field MUST be equal to 56. The value of NLI MUST be equal to the lowest value of NLI of all the aggregated NAL units. The value of TID MUST be the lowest value of TID of all the aggregated NAL units.

All ACL NAL units in an aggregation packet have the same TID value, since they belong to the same access unit πŸ¦‹ However, the packet MAY contain non-ACL NAL units whose TID value in the NAL unit header MAY be different from the TID value of the ACL NAL units in the same AP. Not me being a little bit extra about TIDs πŸ™ˆ

The v3c-tile-id field, when present, specifies the 16-bit tile identifier for all ACL NAL units in the AP. If sprop-v3c-tile-id-pres is equal to 1, the v3c-tile-id field MUST be present. Otherwise, the v3c-tile-id field MUST NOT be present.

The presence of the "OPTIONAL RTP padding" is indicated by the padding (P) bit in the RTP header 🧚 As defined in [RFC3550], the last octet of the padding contains a count of how many padding octets should be ignored, including itself.

An AP MUST carry at least two AUs, and it can carry as many AUs as it needs 🌈 However, the total amount of data in an AP MUST fit into an IP packet, and the size SHOULD be chosen so that the resulting IP packet is smaller than the local MTU size, so you avoid IP layer fragmentation πŸ’– The structure of an AU depends on three things: whether the decoding order number is present, the sequence order of the AU in the AP, and whether the v3c-tile-id field is present. The structure of an AU is shown in Figure 6 🫢

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |  DOND (cond)  /  DONL (cond)  |      v3c-tile-id (cond)       |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
     |            NALU size          |                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               |
     |                                                               |
     |                            NAL unit                           |
     |                                                               |
     |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Figure 6: Aggregation Unit (AU)

If sprop-max-don-diff is greater than 0 for any of the RTP streams, an AU begins with the DOND/DONL field πŸ”₯ The first AU in the AP contains the DONL field, which specifies the 16-bit value of the decoding order number of the aggregated NAL unit. The variable DON for the aggregated NAL unit is derived as equal to the value of the DONL field. All subsequent AUs in the AP MUST contain an (8-bit) DOND field, which specifies the difference between the decoding order number values of the current aggregated NAL unit and the preceding aggregated NAL unit in the same AP. The variable DON for the aggregated NAL unit is derived as equal to the DON of the preceding aggregated NAL unit in the same AP, plus the value of the DOND field, plus 1, modulo 65536.

When sprop-max-don-diff is equal to 0 for all the RTP streams, the DOND/DONL fields MUST NOT be present in an aggregation unit ⭐ The aggregation units MUST be stored in the aggregation packet so that the decoding order of the containing NAL units is preserved. This means the first aggregation unit in the aggregation packet SHOULD contain the NAL unit that SHOULD be decoded first. Main character energy, literally πŸ‘‘

If sprop-v3c-tile-id-pres is equal to 2 and the AU NAL unit header type is in range 0-35 inclusive, the 16-bit v3c-tile-id field MUST be present in the aggregation unit after the conditional DOND/DONL field. Otherwise, the v3c-tile-id field MUST NOT be present in the aggregation unit.

The conditional fields of the aggregation unit are followed by a 16-bit NALU size field, which gives the size of the NAL unit (in bytes) in the aggregation unit πŸ“ The rest of the data in the aggregation unit SHOULD contain the NAL unit, including the unmodified NAL unit header ✨

5.4.4. Fragmentation Unit πŸ’œ

okay, FUs are the move when a single NAL unit is too chunky for one RTP packet πŸ₯Ί They let you split one NAL unit across multiple RTP packets, possibly without the encoder having to cooperate or even know about it. A fragment of a NAL unit is an integer number of consecutive octets of that NAL unit. Fragments of the same NAL unit MUST be sent in consecutive order with ascending RTP sequence numbers, with no other RTP packets within the same RTP stream being sent between the first and last fragment πŸ¦‹

When a NAL unit is fragmented and carried inside FUs, it's called a fragmented NAL unit 🌸 Aggregation packets MUST NOT be fragmented. FUs MUST NOT be nested, i.e., an FU MUST NOT contain a subset of another FU πŸ™…β€β™€οΈ The RTP header timestamp of an RTP packet carrying an FU is set to the NALU-time of the fragmented NAL unit.

An FU consists of an RTP payload header with NUT equal to 57, an 8-bit FU header, a conditional 16-bit DONL field, a conditional 16-bit v3c-tile-id field, and an FU payload πŸŽ€ The structure of an FU is illustrated below in Figure 7.

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |  RTP payload header (NUT=57)  |   FU header   |  DONL (cond)  |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
     |  DONL (cond)  |    v3c-tile-id (cond)         |               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+               |
     |                                                               |
     |                          FU payload                           |
     |                                                               |
     |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                               :...OPTIONAL RTP padding        |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Figure 7: Fragmentation Unit

The fields in the RTP payload header are set like this πŸ’… The NUT field MUST be equal to 57. The rest of the fields MUST be equal to the fragmented NAL unit.

The presence of the "OPTIONAL RTP padding" is indicated by the padding (P) bit in the RTP header 🧚 As defined in [RFC3550], the last octet of the padding contains a count of how many padding octets should be ignored, including itself.

The FU header is made of an S bit, an E bit, and a 6-bit fragmentation unit type (FUT) field. Its structure is illustrated in Figure 8 🌷

      0 1 2 3 4 5 6 7
     +-+-+-+-+-+-+-+-+
     |S|E|    FUT    |
     +-+-+-----------+

Figure 8: Fragmentation Unit Header

When set to 1, the S bit indicates the start of a fragmented NAL unit, i.e., the first byte of the FU payload is also the first byte of the payload of the fragmented NAL unit. When the FU payload is not the start of the fragmented NAL unit payload, the S bit MUST be set to 0.

When set to 1, the E bit indicates the end of a fragmented NAL unit, i.e., the last byte of the payload is also the last byte of the fragmented NAL unit. When the FU payload is not the last fragment of a fragmented NAL unit, the E bit MUST be set to 0.

The FUT field MUST be equal to the nal_unit_type of the fragmented NAL unit πŸ’

A non-fragmented NAL unit MUST NOT be transmitted in one FU, i.e., the Start bit and End bit MUST NOT both be set to 1 in the same FU header 🚫

The DONL field, when present, specifies the value of the 16-bit decoding order number of the fragmented NAL unit. If sprop-max-don-diff is greater than 0 for any of the RTP streams and the S bit is equal to 1, the DONL field MUST be present in the FU, and the variable DON for the fragmented NAL unit is derived as equal to the value of the DONL field. Otherwise (sprop-max-don-diff is equal to 0 for all the RTP streams, or the S bit is equal to 0), the DONL field MUST NOT be present in the FU.

The v3c-tile-id field, when present, specifies the 16-bit tile identifier for the fragmented NAL unit. If sprop-v3c-tile-id-pres is equal to 1, FUT is in range 0-35, and the S bit is equal to 1, the v3c-tile-id field MUST be present after the conditional DONL field. Otherwise, the v3c-tile-id field MUST NOT be present.

The FU payload consists of fragments of the payload of the fragmented NAL unit. If you concatenate the FU payloads of consecutive FUs, starting with an FU whose S bit is equal to 1 and ending with an FU whose E bit is equal to 1, the payload of the fragmented NAL unit can be reconstructed 🧩

The NAL unit header of the fragmented NAL unit is not included as such in the FU payload. Instead, its information is conveyed in the F, NLI, and TID fields of the RTP payload headers of the FUs, and in the FUT field of the FU header 🫧 An FU payload MUST NOT be empty.

If an FU is lost, the receiver SHOULD discard all following fragmentation units in transmission order that correspond to the same fragmented NAL unit, unless the decoder in the receiver is known to be prepared to gracefully handle incomplete NAL units πŸ₯²πŸ’”

5.4.5. Example of Fragmentation Unit (Informative) πŸŽ€

okay bestie, this example is literally so cute πŸ¦‹ It shows how a fragmentation unit can split one NAL unit into two RTP packets. Figure 9 is the first packet, carrying the first part of the chopped-up NAL unit πŸ’…

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |V=2|P|X|  CC   |M|     PT      |       sequence number         |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                           timestamp                           |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |           synchronization source (SSRC) identifier            |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |            contributing source (CSRC) identifiers             |
     |                             ....                              |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |  RTP payload header (NUT=57)  |1|0|    FUT    |               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+               |
     |                                                               |
     |                          FU payload                           |
     |                                                               |
     |                                                               |
     |                                                               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

               Figure 9: First Packet of Fragmented NAL Unit

Figure 10 is the second packet, which carries the rest of the fragmented NAL unit 🫢 Peep the flag bits in the payload header: they flip compared to the first packet. There's also an optional RTP padding bit at the end, shown in the diagram too 🌸

      0                   1                   2                   3
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |V=2|P|X|  CC   |M|     PT      |       sequence number         |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                           timestamp                           |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |           synchronization source (SSRC) identifier            |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |            contributing source (CSRC) identifiers             |
     |                             ....                              |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |  RTP payload header (NUT=57)  |0|1|    FUT    |               |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+               |
     |                                                               |
     |                          FU payload                           |
     |                                                               |
     |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |                               :...OPTIONAL RTP padding        |
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

              Figure 10: Second Packet of Fragmented NAL Unit

5.5. Decoding Order Number 🌈

okay so this part is all about deriving the variable AbsDon for each atlas NAL unit. It's the decoding order number, and it tells you the NAL unit decoding order πŸ’– Let NAL unit n be the n-th NAL unit in transmission order within an RTP stream.

If sprop-max-don-diff is equal to 0 for all the RTP streams carrying the atlas bitstream, AbsDon[n] (the value of AbsDon for NAL unit n) is just derived as equal to n. Easy peasy ✨

Otherwise (sprop-max-don-diff is greater than 0 for any of the RTP streams), AbsDon[n] is derived as follows, where DON[n] is the value of the variable DON for NAL unit n. Here's the math, served hot πŸ”₯

If (n == 0)
  AbsDon[n] = DON[0]
Else
  If (DON[n] == DON[n-1])
    AbsDon[n] = AbsDon[n-1]
  If (DON[n] > DON[n-1] and DON[n] - DON[n-1] < 32768)
    AbsDon[n] = AbsDon[n-1] + DON[n] - DON[n-1]
  If (DON[n] < DON[n-1] and DON[n-1] - DON[n] >= 32768)
    AbsDon[n] = AbsDon[n-1] + 65536 - DON[n-1] + DON[n]
  If (DON[n] > DON[n-1] and DON[n] - DON[n-1] >= 32768)
    AbsDon[n] = AbsDon[n-1] - (DON[n-1] + 65536 - DON[n])
  If (DON[n] < DON[n-1] and DON[n-1] - DON[n] < 32768)
    AbsDon[n] = AbsDon[n-1] - (DON[n-1] - DON[n])

For any two NAL units m and n, the following applies:

  • AbsDon[n] greater than AbsDon[m] means NAL unit n follows NAL unit m in NAL unit decoding order πŸ¦‹
  • When AbsDon[n] is equal to AbsDon[m], the NAL unit decoding order of the two NAL units can be in either order πŸ€·β€β™€οΈ
  • AbsDon[n] less than AbsDon[m] means NAL unit n precedes NAL unit m in decoding order 🧚

6. Packetization and De-Packetization Rules ✨

okay now the rules, bestie πŸ’… The following packetization rules apply for V3C atlas data:

  • If sprop-max-don-diff is greater than 0 for any of the RTP streams, the transmission order of NAL units carried in the RTP stream MAY be different than the NAL unit decoding order and the NAL unit output order. Otherwise (sprop-max-don-diff is equal to 0 for all the RTP streams), the transmission order of NAL units carried in the RTP stream MUST be the same as the NAL unit decoding order. 🌸
  • A NAL unit of a small size SHOULD be encapsulated in an aggregation packet together with one or more other NAL units, in order to avoid the unnecessary packetization overhead for small NAL units. For example, non-ACL NAL units such as access unit delimiters, parameter sets, or SEI NAL units are typically small and can often be aggregated with ACL NAL units without violating MTU size constraints. πŸŽ€
  • Each non-ACL NAL unit SHOULD, when possible and from an MTU size perspective, be encapsulated in an aggregation packet together with its associated ACL NAL unit, as typically a non-ACL NAL unit would be meaningless without the associated ACL NAL unit being available. 🫢
  • For carrying exactly one NAL unit in an RTP packet, a single NAL unit packet (Section 5.4.2) MUST be used. πŸ’–

The general concept behind de-packetization is to get the NAL units out of the RTP packets in an RTP stream, and out of all RTP streams the RTP stream depends on (if any), and pass them to the decoder in the NAL unit decoding order. 🧚

The de-packetization process is implementation dependent. Therefore, the following de-packetization rules SHOULD be taken as an example. πŸ“

  • All normal RTP mechanisms related to buffer management apply. In particular, duplicated or outdated RTP packets (as indicated by the RTP sequence number and the RTP timestamp) are removed. To determine the exact time for decoding, factors such as a possible intentional delay to allow for proper inter-stream synchronization must be factored in. 🌈
  • NAL units with NAL unit type values in the range of 0 to 55 inclusive may be passed to the decoder. NAL-unit-like structures with NAL unit type values in the range of 56 to 63 inclusive MUST NOT be passed to the decoder. πŸ™ˆ
  • When sprop-max-don-diff is equal to 0 for the received RTP stream, the NAL units carried in the RTP stream MAY be directly passed to the decoder in their transmission order, which is identical to their decoding order. πŸ¦‹
  • When sprop-max-don-diff is greater than 0 for any of the received RTP streams, the received NAL units need to be arranged into decoding order before handing them over to the decoder. πŸ’…
  • For further de-packetization examples, the reader is referred to Section 6 of [RFC7798]. πŸ“š

Regarding the packetization of V3C video component data, the respective RTP video payload specification(s) define how packetization and de-packetization should be handled. πŸ‘‘


7. Payload Format Parameters πŸ’œ

This section specifies the optional parameters. A mapping of the parameters into the Session Description Protocol (SDP) [RFC8866] is also provided for applications that use SDP. Equivalent parameters could be defined elsewhere for use with control protocols that do not use SDP. πŸŽ€

The receiver MUST ignore any parameter unspecified in this section. πŸ™…β€β™€οΈ

7.1. Media Type Registration πŸ¦‹

See Section 10.1 for information related to media type registration. 🌸

7.2. Required Parameters Definition πŸ“

sprop-v3c-parameter-set:

The sprop-v3c-parameter-set parameter provides V3C parameter set bytes as defined in [ISO.IEC.23090-5]. The value contains a base64-encoded [RFC4648] representation of the v3c_parameter_set() syntax element. 🧚

7.3. Definitions of Optional Parameters ✨

sprop-v3c-unit-header

sprop-v3c-unit-header gives you the bytes of a V3C unit header as defined in [ISO.IEC.23090-5] πŸŽ€ The value is a base64-encoded [RFC4648] version of those 4 bytes. The V3C unit header tells you which V3C component the media belongs to, so it's giving very specific energy πŸ’…

sprop-v3c-unit-header holds the same info as sprop-v3c-unit-type, sprop-v3c-vps-id, sprop-v3c-atlas-id, sprop-v3c-attr-idx, sprop-v3c-attr-part-idx, sprop-v3c-map-idx, and sprop-v3c-aux-video-flag all squished into one 🧚 To avoid signaling conflicting info, the separate parameters MUST NOT be present when sprop-v3c-unit-header is present. Bestie, do not send both πŸ’€

sprop-v3c-unit-type

sprop-v3c-unit-type gives a V3C unit type value that matches vuh_unit_type from [ISO.IEC.23090-5]. In other words, it names a V3C sub-bitstream type like geometry, occupancy, atlas data, or attribute 🌸

When sprop-v3c-unit-header is present, sprop-v3c-unit-type MUST NOT be present. When it is present, its value SHALL be in the range of 1 to 31 inclusive.

sprop-v3c-vps-id

sprop-v3c-vps-id gives a value that matches the active vuh_v3c_parameter_set_id from [ISO.IEC.23090-5], so it tells you the value of the active V3C parameter set id πŸ¦‹

When sprop-v3c-unit-header is present, sprop-v3c-vps-id MUST NOT be present. When it is present, its value SHALL be in the range of 0 to 15 inclusive.

sprop-v3c-atlas-id

sprop-v3c-atlas-id gives a value that matches vuh_atlas_id from [ISO.IEC.23090-5]. If a V3C bitstream has multiple atlases, this parameter tells you the atlas id for the media component 🧚

When sprop-v3c-unit-header is present, sprop-v3c-atlas-id MUST NOT be present. When it is present, its value SHALL be in the range of 0 to 63 inclusive.

sprop-v3c-attr-idx

sprop-v3c-attr-idx gives a value that matches vuh_attribute_index from [ISO.IEC.23090-5]. Attributes in V3C decide a feature of a reconstructed volumetric primitive, like texture (color), transparency, reflectance, or normal. The attribute index says which type of attribute the media is, and it's lowkey iconic πŸ‘‘

When sprop-v3c-unit-header is present, sprop-v3c-attr-idx MUST NOT be present. When it is present, its value SHALL be in the range of 0 to 127 inclusive.

sprop-v3c-attr-part-idx

sprop-v3c-attr-part-idx gives a value that matches vuh_attribute_partition_index from [ISO.IEC.23090-5]. In V3C, one attribute can be split into multiple components 🌈 This can be super handy when an attribute has four dimensions but the video codec can only code three channels of data.

When sprop-v3c-unit-header is present, sprop-v3c-attr-part-idx MUST NOT be present. When it is present, its value SHALL be in the range of 0 to 31 inclusive.

sprop-v3c-map-idx

sprop-v3c-map-idx gives a value that matches vuh_map_index from [ISO.IEC.23090-5]. Maps in V3C let you store multiple layers of projected volumetric data πŸ“

When sprop-v3c-unit-header is present, sprop-v3c-map-idx MUST NOT be present. When it is present, its value SHALL be in the range of 0 to 15 inclusive.

sprop-v3c-aux-video-flag

sprop-v3c-aux-video-flag gives a value that matches vuh_auxiliary_video_flag from [ISO.IEC.23090-5]. Auxiliary video in V3C can pack volumetric data straight into a video frame, no need to project it onto a 2D plane first 🫢

When sprop-v3c-unit-header is present, sprop-v3c-aux-video-flag MUST NOT be present. When it is present, its value SHALL be either 0 or 1.

sprop-v3c-tile-id

sprop-v3c-tile-id says the RTP stream only carries some of the tiles in an atlas πŸ’– Its value is a comma-separated list (',') of integer values, and those integers are the tile ids that show up in that RTP stream.

When sprop-v3c-tile-id is not present, the RTP stream is expected to contain all tiles, or only a single tile πŸŽ€

sprop-v3c-tile-id-pres

sprop-v3c-tile-id-pres tells you that the RTP packets include the v3c-tile-id field ✨

When it is present, its value SHALL be either 0 or 1. When it is not present, its default value is 0.

sprop-v3c-atlas-data

sprop-v3c-atlas-data MAY be used to carry any atlas data NAL units from the V3C atlas sub-bitstream for out-of-band transmission πŸ¦‹ Its value is a comma-separated list (',') of base64-encoded [RFC4648] versions of the atlas NAL units, as specified in [ISO.IEC.23090-5].

When it is present, the atlas NAL units stored in sprop-v3c-atlas-data shall be applied for the duration of the entire stream, until an in-band atlas NAL unit with the same NAL unit type overrides them πŸ”₯

sprop-v3c-common-atlas-data

sprop-v3c-common-atlas-data MAY be used to carry common atlas data NAL units from the V3C common atlas sub-bitstream for out-of-band transmission 🌸 Its value is a comma-separated list (',') of base64-encoded [RFC4648] versions of the common atlas NAL units (i.e., NAL_CASPS and NAL_CAF_IDR), as specified in [ISO.IEC.23090-5].

When it is present, the common atlas NAL units stored in sprop-v3c-common-atlas-data shall be applied for the duration of the entire stream, until an in-band common atlas NAL unit with the same NAL unit type overrides them πŸ’…

sprop-v3c-sei

sprop-v3c-sei MAY be used to carry SEI NAL units from the V3C atlas and common atlas sub-bitstreams for out-of-band transmission 🧚 Its value is a comma-separated list (',') of base64-encoded [RFC4648] versions of the SEI NAL units (i.e., NAL_PREFIX_NSEI and NAL_SUFFIX_NSEI, NAL_PREFIX_ESEI, NAL_SUFFIX_ESEI), as specified in [ISO.IEC.23090-5].

When it is present, the SEI NAL units stored in sprop-v3c-sei shall be applied for the duration of the entire stream, until an in-band SEI NAL unit with the same SEI payload type overrides them 🌈

v3c-ptl-level-idc

v3c-ptl-level-idc gives a value that matches ptl_level_idc from [ISO.IEC.23090-5]. It tells you the level that the V3C bitstream conforms to πŸ‘‘

When it is present, its value SHALL NOT conflict with the corresponding value in the sprop-v3c-parameter-set. Its value SHALL be in the range of 0 to 255 inclusive.

v3c-ptl-tier-flag

v3c-ptl-tier-flag gives a value that matches ptl_tier_flag from [ISO.IEC.23090-5]. It provides the tier context you need to interpret the value of v3c-ptl-level-idc πŸ“

When it is present, its value SHALL NOT conflict with the corresponding value in the sprop-v3c-parameter-set. Its value SHALL be either 0 or 1.

v3c-ptl-codec-idc

v3c-ptl-codec-idc gives a value that matches ptl_profile_codec_group_idc from [ISO.IEC.23090-5]. It tells you which codec group profile component the V3C bitstream conforms to 🫢

When it is present, its value SHALL NOT conflict with the corresponding value in the sprop-v3c-parameter-set. Its value SHALL be in the range of 0 to 127 inclusive.

v3c-ptl-toolset-idc

v3c-ptl-toolset-idc gives a value that matches ptl_profile_toolset_idc from [ISO.IEC.23090-5]. It tells you which toolset combination profile component the V3C bitstream conforms to πŸŽ€

When it is present, its value SHALL NOT conflict with the corresponding value in the sprop-v3c-parameter-set. Its value SHALL be in the range of 0 to 255 inclusive.

v3c-ptl-rec-idc

v3c-ptl-rec-idc gives a value that matches ptl_profile_reconstruction_idc as defined in [ISO.IEC.23090-5]. It tells you which reconstruction profile component the V3C bitstream is recommended to conform to πŸ¦‹

When it is present, its value SHALL NOT conflict with the corresponding value in the sprop-v3c-parameter-set. Its value SHALL be in the range of 0 to 255 inclusive.

sprop-max-don-diff

Okay, this one is a bit of a brain-teaser, so pay attention πŸ§ πŸ’œ If the transmission order of NAL units in the RTP stream(s) is the same as the decoding and NAL unit output order, this parameter must be equal to 0.

Otherwise, if the decoding order of the NAL units in the RTP stream(s) is the same as the NAL unit transmission order but not the same as the NAL unit output order, the value of this parameter MUST be equal to 1.

Otherwise, this parameter gives the maximum absolute difference between the decoding order number (i.e., AbsDon) values of any two NAL units, naluA and naluB, where naluA follows naluB in decoding order and precedes naluB in transmission order 🌸

The value of sprop-max-don-diff MUST be an integer in the range of 0 to 32767 inclusive.

When it is not present, the value of sprop-max-don-diff is inferred to be equal to 0 ✨

7.4. Mapping of Parameters to V3C Syntax πŸ—ΊοΈβœ¨

Okay bestie, this table is basically the cheat sheet that matches each SDP parameter to its V3C syntax buddy πŸ’…

   +============+========+================================+===========+
   | Parameter  |Required| V3C Syntax Counterpart         | ISO/IEC   |
   |            |        |                                | 23090-5   |
   |            |        |                                | Section   |
   |            |        |                                | Reference |
   +============+========+================================+===========+
   | sprop-v3c- |YES     | v3c_parameter_set()            | 8.3.4.1   |
   | parameter- |        |                                |           |
   | set        |        |                                |           |
   +------------+--------+--------------------------------+-----------+
   | sprop-v3c- |NO      | v3c_unit_header()              | 8.3.2.2   |
   | unit-      |        |                                |           |
   | header     |        |                                |           |
   +------------+--------+--------------------------------+-----------+
   | sprop-v3c- |NO      | vuh_unit_type                  | 8.4.2.2   |
   | unit-type  |        |                                |           |
   +------------+--------+--------------------------------+-----------+
   | sprop-v3c- |NO      | vuh_v3c_parameter_set_id       | 8.4.2.2   |
   | vps-id     |        |                                |           |
   +------------+--------+--------------------------------+-----------+
   | sprop-v3c- |NO      | vuh_atlas_id                   | 8.4.2.2   |
   | atlas-id   |        |                                |           |
   +------------+--------+--------------------------------+-----------+
   | sprop-v3c- |NO      | vuh_attribute_index            | 8.4.2.2   |
   | attr-idx   |        |                                |           |
   +------------+--------+--------------------------------+-----------+
   | sprop-v3c- |NO      | vuh_attribute_partition_index  | 8.4.2.2   |
   | attr-part- |        |                                |           |
   | idx        |        |                                |           |
   +------------+--------+--------------------------------+-----------+
   | sprop-v3c- |NO      | vuh_map_index                  | 8.4.2.2   |
   | map-idx    |        |                                |           |
   +------------+--------+--------------------------------+-----------+
   | sprop-v3c- |NO      | vuh_auxiliary_video_flag       | 8.4.2.2   |
   | aux-video- |        |                                |           |
   | flag       |        |                                |           |
   +------------+--------+--------------------------------+-----------+
   | sprop-v3c- |NO      | -                              | -         |
   | tile-id    |        |                                |           |
   +------------+--------+--------------------------------+-----------+
   | sprop-v3c- |NO      | -                              | -         |
   | tile-id-   |        |                                |           |
   | pres       |        |                                |           |
   +------------+--------+--------------------------------+-----------+
   | sprop-v3c- |NO      | nal_unit()                     | 8.3.5.1   |
   | atlas-data |        |                                |           |
   +------------+--------+--------------------------------+-----------+
   | sprop-v3c- |NO      | nal_unit()                     | 8.3.5.1   |
   | common-    |        |                                |           |
   | atlas-data |        |                                |           |
   +------------+--------+--------------------------------+-----------+
   | sprop-v3c- |NO      | nal_unit()                     | 8.3.5.1   |
   | sei        |        |                                |           |
   +------------+--------+--------------------------------+-----------+
   | v3c-ptl-   |NO      | ptl_level_idc                  | 8.4.4.2   |
   | level-idc  |        |                                |           |
   +------------+--------+--------------------------------+-----------+
   | v3c-ptl-   |NO      | ptl_tier_flag                  | 8.4.4.2   |
   | tier-flag  |        |                                |           |
   +------------+--------+--------------------------------+-----------+
   | v3c-ptl-   |NO      | ptl_profile_codec_group_idc    | 8.4.4.2   |
   | codec-idc  |        |                                |           |
   +------------+--------+--------------------------------+-----------+
   | v3c-ptl-   |NO      | ptl_profile_toolset_idc        | 8.4.4.2   |
   | toolset-   |        |                                |           |
   | idc        |        |                                |           |
   +------------+--------+--------------------------------+-----------+
   | v3c-ptl-   |NO      | ptl_profile_reconstruction_idc | 8.4.4.2   |
   | rec-idc    |        |                                |           |
   +------------+--------+--------------------------------+-----------+
   | sprop-max- |NO      | -                              | -         |
   | don-diff   |        |                                |           |
   +------------+--------+--------------------------------+-----------+

Table 2: Mapping of Parameters to V3C Syntax

8. Congestion Control Considerations 🚦

Congestion control for RTP SHALL be used in accordance with RTP [RFC3550] and with any applicable RTP profile, e.g., AVP [RFC3551]. This section only applies to unicast streaming, so considerations for multicast streaming are out of scope 🌸

Users of this payload format MUST monitor packet loss to make sure the loss rate stays in an acceptable range πŸ‘€ Packet loss counts as acceptable if a TCP flow on the same network path, dealing with the same network conditions, would get an average throughput, measured over a reasonable timescale, that is not less than the RTP flow's (see Section 10 of [RFC3550]). Main character energy, but make it responsible πŸ’…

You can meet this condition by implementing congestion-control mechanisms that adapt the transmission rate πŸ¦‹ A simple bitrate adaptation can happen when real-time coding is used for V3C video components, where quality parameters can be tuned on the fly. Video coding specifications MAY define further adaptation techniques ✨

An alternative is to have a receiver leave the session if the loss rate gets unacceptably high πŸ™ˆ For example, you could use a Circuit Breaker [RFC8083], which defines criteria for when one the RTP flow must stop sending RTP Packet Streams 😭

As an example, both the sender and the receiver may have their own definition of an acceptable packet loss rate 🌈 In that case, the receiver may decide to quit a stream when it finds the packet loss rate too high. Similarly, the sender may decide to drop a receiver when the reports it gets show packet loss rates that are too high from its point of view. Either side can make these decisions independently 🫢

Here's a bitrate adaptation technique: a sender or a receiver may adapt the bitrates of specific sub-streams. The adaptation should be done in a way that considers the effect on the experience as a whole, keeping the subjective quality of experience as high as possible πŸ‘‘ In practice that could mean dropping less important sub-streams entirely, or reducing the bitrates of the most important sub-streams throughout the session πŸ“

9. Session Description Protocol πŸ“»

A new attribute "v3cfmtp" is defined for carrying V3C format media type parameters in the matching fields of SDP [RFC8866] πŸ’œ The Grouping framework [RFC5888] is used to show which media lines (video and application) in the SDP make up a V3C representation 🧚

9.1. V3C Format Parameters "v3cfmtp" Attribute πŸŽ€

This document defines a new attribute for SDP, meant to carry V3C-specific media format parameters. It works a lot like "a=fmtp", except that it SHALL be used without the fmt-token, and it can also be used at the session level 🫧 The attribute lets V3C-specific media format parameters be associated with any media line in SDP. The detailed info on the new attribute (a=v3cfmtp) is in Section 10.2 πŸ“–

The value of the v3cfmtp attribute is a byte-string, as defined in [RFC8866], which contains at least one V3C-specific media format parameter as a "parameter=value" pair, as defined in this document πŸ”₯ You can store multiple semicolon-separated V3C media "parameter=value" pairs in the byte-string, and they get conveyed by SDP and handed over unchanged to the media tool that will use the format. Whitespace in the byte-string is ignored 🀍

Here's an example of the attribute in action πŸ‘‡ The first line is a session-level usage, signaling a V3C parameter set. The second line is a media-level attribute, signaling a V3C unit header and a profile tier level flag for the associated media line πŸ’–

  a=v3cfmtp:sprop-v3c-parameter-set=AUH/AAAP/zwAAAAAACgIAtEAgQLAIAAUQ
  BACWAM5QEDgQCAIAAAAABP8CzwAAAAAAAAAQAAAtAE/wLPAAAAAAAg=;

  a=v3cfmtp:sprop-v3c-unit-header=CAAAAA==;v3c-ptl-tier-flag=1;

9.2. Mapping of Payload Type Parameters to SDP πŸͺ„

9.2.1. For V3C Atlas Components 🌷

  • The media name in the "m=" line of SDP MUST be application.
  • The encoding name in the "a=rtpmap" line of SDP MUST be v3c.
  • The clock rate in the "a=rtpmap" line MUST be 90000.
  • The OPTIONAL parameters sprop-v3c-atlas-data, sprop-v3c-common-atlas-data, sprop-v3c-sei, sprop-v3c-tile-id, and sprop-v3c-tile-id-pres, when present, MUST be included in the "a=fmtp" line of SDP. This parameter is expressed as a media type string in the form of a semicolon-separated list of parameter=value pairs.
  • The OPTIONAL parameters sprop-v3c-unit-header, sprop-v3c-unit-type, sprop-v3c-vps-id, sprop-v3c-atlas-id, sprop-v3c-attr-idx, sprop-v3c-attr-part-idx, sprop-v3c-map-idx, sprop-v3c-aux-video-flag, sprop-max-don-diff, sprop-v3c-parameter-set, v3c-ptl-level-idc, v3c-ptl-tier-flag, v3c-ptl-codec-idc, v3c-ptl-toolset-idc, and v3c-ptl-rec-idc, when present, MUST be included in the "a=v3cfmtp" line of SDP. This parameter is expressed as a media type string in the form of a semicolon-separated list of parameter=value pairs.

The OPTIONAL parameters, when they're present in the V3C atlas component media line format parameters attribute, give values that stay valid for the coded V3C sequence until a new value arrives in-band 🧠 Some OPTIONAL parameters, like sprop-v3c-parameter-set or sprop-v3c-unit-header, can't be carried in-band in the atlas stream, so they may be considered static for the session 🫢 Carriage of V3C payload format parameters is split by logical context: "a=fmtp" holds atlas level media format parameters, and "a=v3cfmtp" holds V3C level media format parameters ✨

Here's an example of a media representation for the atlas data component (V3C_AD), where the static V3C parameter set and V3C unit header are carried out-of-band in SDP πŸ’…

  m=application 49170 RTP/AVP 98
  a=rtpmap:98 v3c/90000
  a=fmtp:98 sprop-v3c-tile-id=0,1
  a=v3cfmtp:sprop-v3c-unit-header=CAAAAA==;v3c-ptl-tier-flag=1;
    sprop-v3c-parameter-set=AQD/AAAP/zwAAAAAADwIAQ5BwAAOADjgQAADkA==

9.2.2. For V3C Video Components ✨

okay bestie, this part is all about the video components 🎬 (the ones carrying occupancy, geometry, and attribute stuff) πŸ’–

  • the media name in the m= line of SDP MUST be video πŸ’…
  • the encoding name in the a=rtpmap line can be any video subtype, e.g., H.264, H.265, H.266, etc. πŸ¦‹
  • the clock rate in the a=rtpmap line MUST be 90000 ⏰
  • the OPTIONAL parameters sprop-v3c-unit-header, sprop-v3c-unit-type, sprop-v3c-vps-id, sprop-v3c-atlas-id, sprop-v3c-attr-idx, sprop-v3c-attr-part-idx, sprop-v3c-map-idx, sprop-v3c-aux-video-flag, sprop-max-don-diff, sprop-v3c-parameter-set, sprop-v3c-atlas-data, sprop-v3c-common-atlas-data, sprop-v3c-sei, sprop-v3c-tile-id, sprop-v3c-tile-id-pres, v3c-ptl-level-idc, v3c-ptl-tier-flag, v3c-ptl-codec-idc, v3c-ptl-toolset-idc, and v3c-ptl-rec-idc, when present, MUST be included in the a=v3cfmtp line of SDP πŸŽ€ this parameter is expressed as a media type string in the form of a semicolon-separated list of parameter=value pairs 🧚

The OPTIONAL parameters, when present in the video media line V3C format parameters ("v3cfmtp") attribute, specify values that are considered static for the session. so they stay the same for the whole session, no drama 🌸

An example of media representation corresponding to occupancy video component (V3C_OVD) in SDP is as follows: πŸ‘€

m=video 49170 RTP/AVP 99
a=rtpmap:99 H265/90000
a=fmtp:99 sprop-max-don-diff=0;
a=v3cfmtp:sprop-v3c-unit-header=EAAAAA==

Below is an example of media representation corresponding to packed video component (V3C_PVD), where the static V3C parameter set, atlas data, and common atlas data are carried out-of-band in SDP. The values are considered static for the session, as they can't be signaled in-band in the video stream. the vibe is "it's all packed in the SDP, bestie" πŸ“¦

m=video 49170 RTP/AVP 99
a=rtpmap:99 H265/90000
a=v3cfmtp:sprop-v3c-unit-header=KAAAAA==;
  sprop-v3c-parameter-set=AUH/AAAP/zwAAAAAACgIAtEAgQLAIAAUQBACWAM5Q
  EDgQCAIAAAAABP8CzwAAAAAAAAAQAAAtAE/wLPAAAAAAAg=;
  sprop-v3c-atlas-data=SAGAFAQBaKjuXgABQEKA,SgHmIA==,LgFoDOAFAABaAA
  AAAAA+;
  sprop-v3c-common-atlas-data=YAEHgFA=,YgEAMAAAC/B0qcvv/Dbr/pTvb8oq
  fhC5JQVS9jn7kAQT/As9EFyrjRBcmxEQe+j5DuGbTT9mZmZAQAAAoA==

9.3. Grouping Framework 🧩

Different V3C components MAY be represented by their own respective RTP streams, whose payload formats are defined in the respective specifications. V3C atlas data RTP payload format is defined in this document, whereas the video component RTP payload formats are defined for example in [RFC6184] or [RFC7798]. A grouping tool, as defined in [RFC5888], is extended to indicate which media lines constitute a V3C representation. More details on the new grouping type are provided in Section 10.3. the grouping tool gets a glow-up so it can tell which media lines belong together πŸ’«

The group attribute with V3C type is provided to allow application to identify "m" lines that belong to the same V3C bitstream. Grouping type V3C MUST be used with the group attribute. The tokens that follow are mapped to 'mid'-values of individual media lines in the SDP. 🫢

a=group:V3C <tokens>

The following example shows an SDP including four media lines: three describing V3C video components (PT:96=occupancy, PT:97=geometry, and PT:98=attribute) and one describing a V3C atlas component (PT:100). All of these media lines are grouped under one V3C group. The V3C parameter set is provided via a session-level V3C media format parameter attribute. πŸ“

...
a=group:V3C 1 2 3 4
a=v3cfmtp:sprop-v3c-parameter-set=AQD/AAAP/zwAAAAAADwIAQ5BwAAOADjgQ
  AADkA==
m=video 40000 RTP/AVP 96
a=rtpmap:96 H264/90000
a=v3cfmtp:sprop-v3c-unit-header=EAAAAA==
a=mid:1
m=video 40002 RTP/AVP 97
a=rtpmap:97 H264/90000
a=v3cfmtp:sprop-v3c-unit-header=GAAAAA==
a=mid:2
m=video 40004 RTP/AVP 98
a=rtpmap:98 H264/90000
a=v3cfmtp:sprop-v3c-unit-header=IAAAAA==
a=mid:3
m=application 40008 RTP/AVP 100
a=rtpmap:100 v3c/90000
a=v3cfmtp:sprop-v3c-unit-header=CAAAAA==;
a=mid:4

The example below describes how content with two atlases can be signaled as separate streams. The V3C parameter set is carried in a session-level V3C media format parameter attribute and common atlas data are carried as part of the media-level V3C media format parameter attribute corresponding to atlas zero. PT values 96, 97, 98, and 100 correspond to the occupancy, geometry, and attribute video components, as well as the atlas data component, for atlas zero. PT values 101, 102, 103, and 104 correspond to the respective components for atlas one. two atlases, two little squads, zero chaos 🌈

...
a=group:V3C 1 2 3 4 5 6 7 8
a=v3cfmtp:sprop-v3c-parameter-set=AAUH/AAAP/zwAAABAADwIAWhBwAAOADjg
  QAADgAA8CAFoQcAADgA44EAAA6AkAgABRIA=;
m=video 40000 RTP/AVP 96
a=rtpmap:96 H264/90000
a=v3cfmtp:sprop-v3c-unit-header=EAAAAA==
a=mid:1
m=video 40002 RTP/AVP 97
a=rtpmap:97 H264/90000
a=v3cfmtp:sprop-v3c-unit-header=GAAAAA==
a=mid:2
m=video 40004 RTP/AVP 98
a=rtpmap:98 H264/90000
a=v3cfmtp:sprop-v3c-unit-header=IAAAAA==
a=mid:3
m=application 40008 RTP/AVP 100
a=rtpmap:100 v3c/90000
a=fmtp:100
  sprop-v3c-common-atlas-data=YAEHgFA=,YgEAMAAAa+96Z5v6VP1D+P7LzRsb
  WDJ/yz+ALzMZNfvCg2389Kjd+d6fZyM6QZBfhrDW3K0vaP2Rr8L+gLAq/ny3wAzs9
  veiXEjjS67MfH+H4xV/RgW4fkl/YkINe/OsWCOBwPAVLACCf4FnogwYZKIME6oiD9
  UCodqjLwCCf4FnogxqBiIMZNwiEBpJIduBUoCCf4FnogwOeSIMCaGiEA9VIdtGwwC
  Cf4FnogvB+aILvWIiEBB6IdqobKfmZmZoCmZmefmZmZoCmZmefmZmZoCmZmefmZmZ
  oCmZmdA=
a=v3cfmtp:sprop-v3c-unit-header=CAAAAA==;
a=mid:4
m=video 40010 RTP/AVP 101
a=rtpmap:101 H264/90000
a=v3cfmtp:sprop-v3c-unit-header=EAIAAA==
a=mid:5
m=video 40012 RTP/AVP 102
a=rtpmap:102 H264/90000
a=v3cfmtp:sprop-v3c-unit-header=GAIAAA==
a=mid:6
m=video 40014 RTP/AVP 103
a=rtpmap:103 H264/90000
a=v3cfmtp:sprop-v3c-unit-header=IAIAAA==
a=mid:7
m=application 40018 RTP/AVP 104
a=rtpmap:104 v3c/90000
a=v3cfmtp:sprop-v3c-unit-header=CAIAAA==
a=mid:8

9.4. Offer and Answer Considerations πŸŽ€

9.4.1. Unicast πŸ¦‹

Okay so this part is all about unicast streaming, and it negotiates through the offer/answer model from [RFC3264] πŸ’… V3C-coded content is an atlas bitstream plus one or more video coded bitstreams, and together those are called V3C components 🧚 The atlas and video bitstreams each get their own media line in the SDP, so they're separate but still besties 🫢

During session negotiation, the offerer lists every V3C component it has and tells the answerer which media lines SHOULD be consumed together πŸ‘‘ The answerer can go with the components the offerer suggested, or it can pick a subset by setting the port to zero on the media lines it doesn't want πŸŽ€ This is so clutch when the answerer is fully or partially clueless about the V3C coding scheme, since it can still consume a subset of the components anyway πŸ’–

Here are the rules and limits for the V3C atlas component media configuration 🌸

  • The parameters that identify the V3C atlas component media configuration are v3c-ptl-level-idc, v3c-ptl-tier-flag, v3c-ptl-codec-idc, and v3c-ptl-toolset-idc. These media configuration parameters, except level-id, MUST be used symmetrically.
  • Send only properties, identified by sprop-prefix, are considered declarative and SHOULD be omitted in the answers.

The answerer MUST structure its answer according to one of the following two options:

  • keep all configuration parameters at the same values the offer had for the media format (payload type), with one exception: the value of v3c-ptl-level-idc can change, as long as the highest level indicated by the answer is not higher than the one indicated by the offer, or
  • reject the media line in which one or more of the parameter values are not supported by setting the port to zero in the answer.

Here are the rules for the video-coded V3C component media configuration ✨

  • The parameters that identify a video-coded V3C component media configuration format follow the respective RTP video payload specification.

The answerer MUST structure its answer according to one of the following two options:

  • keep all configuration parameters at the same values the offer had for the media format (payload type), with the exceptions specified in the respective RTP video payload specification, or
  • reject the video coded V3C component media line completely when one or more of the parameter values are not supported by setting the port to zero in the answer.

To keep handling and matching of these configurations simple, the same RTP payload type number used in the offer SHOULD also be used in the answer, as specified in [RFC3264] 🧸

Below is an example of an offer that only sends V3C content πŸ“¦ It has video components in three different versions (H.264, H.265, and H.266). Any other differences between the alternatives get signaled as part of the media attribute parameters, the same way regular video streams do it πŸ“Ί

...
a=group:v3c 1 2 3 4
a=v3cfmtp:v3c-ptl-level-idc=60;
  sprop-v3c-parameter-set=AQD/AAAP/zwAAAAAADwIAQ5BwAAOADjgQAADkA==
m=video 40000 RTP/AVP 96 97 98
a=rtpmap:96 H264/90000
a=rtpmap:97 H265/90000
a=rtpmap:98 H266/90000
a=v3cfmtp:sprop-v3c-unit-type=2;sprop-v3c-vps-id=0;
  sprop-v3c-atlas-id=0
a=sendonly
a=mid:1
m=video 40002 RTP/AVP 99 100 101
a=rtpmap:99 H264/90000
a=rtpmap:100 H265/90000
a=rtpmap:101 H266/90000
a=v3cfmtp:sprop-v3c-unit-type=3;sprop-v3c-vps-id=0;
  sprop-v3c-atlas-id=0
a=mid:2
a=sendonly
m=video 40004 RTP/AVP 102 103 104
a=rtpmap:102 H264/90000
a=rtpmap:103 H265/90000
a=rtpmap:104 H266/90000
a=v3cfmtp:sprop-v3c-unit-type=4;sprop-v3c-vps-id=0;
  sprop-v3c-atlas-id=0
a=mid:3
a=sendonly
m=application 40006 RTP/AVP 105
a=rtpmap:105 v3c/90000
a=v3cfmtp:sprop-v3c-unit-type=1;sprop-v3c-vps-id=0;
  sprop-v3c-atlas-id=0;
a=mid:4
a=sendonly

And here's an answer that only receives V3C data with the selected versions 🌈

...
a=group:v3c 1 2 3 4
m=video 50000 RTP/AVP 96
a=rtpmap:96 H264/90000
a=recvonly
a=mid:1
m=video 50002 RTP/AVP 100
a=rtpmap:100 H265/90000
a=recvonly
a=mid:2
m=video 50004 RTP/AVP 104
a=rtpmap:104 H266/90000
a=recvonly
a=mid:3
m=application 50006 RTP/AVP 105
a=rtpmap:105 v3c/90000
a=recvonly
a=mid:4

This one is an offer that lets you bundle different V3C components into one stream, based on [RFC9143] πŸ“

...
a=group:BUNDLE 1 2 3 4
a=group:v3c 1 2 3 4
m=video 40000 RTP/AVP 96
a=rtpmap:96 H264/90000
a=v3cfmtp:sprop-v3c-unit-type=2;sprop-v3c-vps-id=0;
  sprop-v3c-atlas-id=0
a=mid:1
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
m=video 40002 RTP/AVP 97
a=rtpmap:97 H264/90000
a=v3cfmtp:sprop-v3c-unit-type=3;sprop-v3c-vps-id=0;
  sprop-v3c-atlas-id=0;
a=mid:2
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
m=video 40004 RTP/AVP 98
a=rtpmap:98 H264/90000
a=v3cfmtp:sprop-v3c-unit-type=4;sprop-v3c-vps-id=0;
  sprop-v3c-atlas-id=0
a=mid:3
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
m=application 40006 RTP/AVP 99
a=rtpmap:99 v3c/90000
a=v3cfmtp:sprop-v3c-unit-type=1;sprop-v3c-vps-id=0;
  sprop-v3c-atlas-id=0;
  sprop-v3c-parameter-set=AQD/AAAP/zwAAAAAADwIAQ5BwAAOADjgQAADkA==
a=mid:4
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid

And the answer that accepts the bundling of the different V3C components πŸŽ‰

a=group:BUNDLE 1 2 3 4
a=group:v3c 1 2 3 4
m=video 50000 RTP/AVP 96
a=rtpmap:96 H264/90000
a=mid:1
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
m=video 0 RTP/AVP 97
a=rtpmap:97 H264/90000
a=bundle-only
a=mid:2
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
m=video 0 RTP/AVP 98
a=rtpmap:98 H264/90000
a=bundle-only
a=mid:3
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
m=application 0 RTP/AVP 99
a=rtpmap:99 v3c/90000
a=bundle-only
a=mid:4
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid

9.4.2. Multicast πŸ“‘

When bitstreams are delivered over multicast, these rules apply πŸ’«

  • The atlas V3C component media configuration is identified by v3c-ptl-level-idc, v3c-ptl-tier-flag, v3c-ptl-codec-idc, and v3c-ptl-toolset-idc. These atlas format configuration parameters MUST be used symmetrically; that is, the answerer MUST either maintain all configuration parameters or reject the media line, including any associated video coded V3C component media lines. This implies that v3c-ptl-level-idc for offer/answer in multicast is not changeable.
  • The video-coded V3C component media configuration format follows the respective RTP video payload specification.
  • To simplify the handling and matching of these configurations, the same RTP payload type number used in the offer MUST also be used in the answer.
  • Parameter sets received MUST be associated with the originating source and MUST only be used in decoding the incoming bitstream from the same source.

9.5. Declarative SDP Considerations πŸŽ€

When V3C content over RTP is offered with SDP in a declarative style, the parameters that can indicate both bitstream properties and answerer capabilities are used to indicate only bitstream properties. For example, in this case, the parameters v3c-ptl-level-idc, v3c-ptl-tier-flag, v3c-ptl-codec-idc, v3c-ptl-toolset-idc, and v3c-ptl-rec-idc declare the values used by the bitstream, not the capabilities for receiving bitstreams 🧠

An answerer of the SDP is required to support all parameters and values of the parameters provided; otherwise, the answerer MUST reject or not participate in the session. It falls on the creator of the session to use values that are expected to be supported by the receiving application 🫑


10. IANA Considerations 🌸

This document contains three IANA considerations: a new media type, a new SDP attribute, and a new grouping type.

10.1. V3C Media Type Registration πŸ’œ

IANA has registered the following media type in the "Media Types" registry.

Type name:  application

Subtype name:  v3c

Required parameters:  sprop-v3c-parameter-set

Optional parameters:  sprop-v3c-unit-header, sprop-v3c-unit-type,
   sprop-v3c-vps-id, sprop-v3c-atlas-id, sprop-v3c-attr-idx, sprop-
   v3c-attr-part-idx, sprop-v3c-map-idx, sprop-v3c-aux-video-flag,
   sprop-v3c-tile-id, sprop-v3c-tile-id-pres, sprop-v3c-atlas-data,
   sprop-v3c-common-atlas-data, sprop-v3c-sei, v3c-ptl-level-idc,
   v3c-ptl-tier-flag, v3c-ptl-codec-idc, v3c-ptl-toolset-idc, v3c-
   ptl-rec-idc, and sprop-max-don-diff.

Encoding considerations:  framed

Security considerations:  See Section 11 of RFC 10034.

Interoperability considerations:  N/A

Published specification:  RFC 10034

Applications that use this media type:  Any application that relies
   on V3C-based media services over RTP.

Fragment identifier considerations:  N/A

Additional information:  N/A

Person & email address to contact for further information:  Lauri
   Ilola (lauri.ilola@nokia.com) or Lukasz Kondrad
   (lukasz.kondrad@nokia.com)

Intended usage:  COMMON

Restrictions on usage:  This media type depends on RTP framing and,
   hence, is only defined for transfer via RTP [RFC3550].  Transport
   within other framing protocols is not defined at this time.

Author:  See the Authors' Addresses section of RFC 10034.

Change controller:  IETF

10.2. V3C Format Parameters SDP Attribute πŸŽ€

okay so IANA registered a brand new SDP attribute, and it's giving main character πŸ’… It lives in the "attribute-name (formerly 'att-field')" registry, which sits inside the "Session Description Protocol (SDP) Parameters" registry group πŸ¦‹

Contact name: see the Authors' Addresses section of RFC 10034 πŸ’–

Contact email address: see the Authors' Addresses section of RFC 10034 πŸ’Œ

Attribute name: v3cfmtp ✨

Attribute syntax:

v3cfmtp-value = byte-string
  • The V3C format parameters are V3C media type parameters and need to reflect their syntax. 🌸
  • byte-string is as defined in [RFC8866]. 🧚
  • ABNF grammar is as defined in [RFC5234]. πŸ“

Attribute semantics: so basically, v3cfmtp-value is a byte-string (as defined in [RFC8866]) that holds at least one V3C-specific media format parameter written as a "parameter=value" pair, the way this document defines it πŸ’œ You can pack several V3C media "parameter=value" pairs into it, separated by semicolons. SDP just carries the whole byte-string along and hands it over unchanged to the media tool that's going to use the format. Whitespace in the byte-string is ignored, so don't stress about spacing ✨

Attribute value: v3cfmtp-value

Usage level: session, media

Charset dependent: No

Purpose: this attribute lets parameters that are specific to a V3C format get conveyed without SDP having to understand them at all πŸ™ˆ It can associate V3C-specific parameters with a session or with any media line. Parameters signaled as part of session-level attributes take effect when conflicting parameters are signaled as media-level attributes. πŸ¦‹

O/A procedures: the v3cfmtp attribute can be present in both offers and answers πŸ’•

Mux Category: NORMAL

Reference: RFC 10034

+===========+==========+================+==============+===========+
| Type      | SDP Name | Usage Level    | Mux Category | Reference |
+===========+==========+================+==============+===========+
| attribute | v3cfmtp  | session, media | NORMAL       | RFC 10034 |
+-----------+----------+----------------+--------------+-----------+

       Table 3: The attribute-name (formerly 'att-field') Registry

10.3. V3C Grouping Type Extension 🌈

the SDP Group attribute gets extended so it can establish relationships between sub-streams of a V3C representation, and it's honestly so iconic πŸ‘‘ IANA registered the following in the "Semantics for the 'group' SDP Attribute" registry, which lives under the "Session Description Protocol (SDP) Parameters" registry group:

            +==============+=======+==============+===========+
            | Semantics    | Token | Mux Category | Reference |
            +==============+=======+==============+===========+
            | V3C grouping | V3C   | NORMAL       | RFC 10034 |
            +--------------+-------+--------------+-----------+

                 Table 4: The Semantics for the 'group' SDP
                             Attribute Registry

11. Security Considerations πŸ”

okay, security time, and I'm not glossing over it at all 😭 RTP packets that use the payload format defined here are subject to the security considerations in the RTP spec [RFC3550], and in any RTP profile that applies, like RTP/AVP [RFC3551], RTP/AVPF [RFC4585], RTP/SAVP [RFC3711], or RTP/SAVPF [RFC5124]. ngl, as [RFC7202] points out, it isn't an RTP payload format's job to decide which solutions get used for the basic security goals (confidentiality, integrity, and source authenticity) for RTP in general. That responsibility belongs to whoever is using RTP in their application. They can find guidance on the available security mechanisms and the important considerations in [RFC7201]. πŸ›‘οΈ

This document does not mandate a specific security mechanism. Instead, applications are responsible for picking mechanisms that follow current best practices for confidentiality, integrity, and source authentication, and that reflect the evolving security landscape beyond what [RFC7201] covers. For modern best practices, applications can consider the following options:

  • (D)TLS-based protection: for guidance on using TLS 1.3 and DTLS, applications should refer to [BCP195], which provides up-to-date recommendations. πŸ”’
  • IPsec-based protection: relevant and current protocol specifications include [RFC4303] (ESP) and [RFC7296] (IKEv2). 🌸

The rest of the Security Considerations section discusses the security-impacting properties of the payload format itself 🧚

A V3C session can consist of multiple sub-streams carried over different RTP streams. Security considerations such as source authentication SHOULD be applied to all of its constituent sub-streams. All receivers of V3C data SHOULD exercise source caution and only receive data from senders they can trust. 🫢 On top of that, this RTP payload format supports multiple RTP streams for the different components needed to produce the decoded output, so it depends on all of those RTP streams and the signaling components (e.g., SDP and RTCP) being authentic to what the sender intended. It's a whole team effort, bestie πŸ’œ

This RTP payload format and its media decoder do not exhibit significant non-uniformity in the receiver-side computational complexity for packet processing, and thus are unlikely to pose a denial-of-service threat due to the receipt of pathological data. Furthermore, this payload format contains no active content πŸ“

Components of a system using this media type SHALL NOT construct RTP payloads that contain executable content. The implementer of the RTP payload format SHALL guarantee that the received content is properly de-packetized and fed to a V3C standard compliant decoder. What the receiver does with the decoded bitstream is unspecified. πŸ’€


12. References πŸ“š

12.1. Normative References πŸŽ€

these are the normative references, all listed with their full details πŸ’–

   [ISO.IEC.23090-5]
              ISO/IEC, "Information technology -- Coded representation
              of immersive media -- Part 5: Visual volumetric video-
              based coding (V3C) and video-based point cloud compression
              (V-PCC)", ISO/IEC 23090-5:2026, 2026,
              <https://www.iso.org/standard/91546.html>.

   [ISO.IEC.23090-12]
              ISO/IEC, "Information technology -- Coded representation
              of immersive media -- Part 12: MPEG Immersive video
              (MIV)", ISO/IEC 23090-12:2025, 2025,
              <https://www.iso.org/standard/87643.html>.

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC3264]  Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model
              with Session Description Protocol (SDP)", RFC 3264,
              DOI 10.17487/RFC3264, July 2002,
              <https://www.rfc-editor.org/info/rfc3264>.

   [RFC3550]  Schulzrinne, H., Casner, S., Frederick, R., and V.
              Jacobson, "RTP: A Transport Protocol for Real-Time
              Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550,
              July 2003, <https://www.rfc-editor.org/info/rfc3550>.

   [RFC4648]  Josefsson, S., "The Base16, Base32, and Base64 Data
              Encodings", RFC 4648, DOI 10.17487/RFC4648, October 2006,
              <https://www.rfc-editor.org/info/rfc4648>.

   [RFC5234]  Crocker, D., Ed. and P. Overell, "Augmented BNF for Syntax
              Specifications: ABNF", STD 68, RFC 5234,
              DOI 10.17487/RFC5234, January 2008,
              <https://www.rfc-editor.org/info/rfc5234>.

   [RFC5888]  Camarillo, G. and H. Schulzrinne, "The Session Description
              Protocol (SDP) Grouping Framework", RFC 5888,
              DOI 10.17487/RFC5888, June 2010,
              <https://www.rfc-editor.org/info/rfc5888>.

   [RFC8083]  Perkins, C. and V. Singh, "Multimedia Congestion Control:
              Circuit Breakers for Unicast RTP Sessions", RFC 8083,
              DOI 10.17487/RFC8083, March 2017,
              <https://www.rfc-editor.org/info/rfc8083>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

   [RFC8866]  Begen, A., Kyzivat, P., Perkins, C., and M. Handley, "SDP:
              Session Description Protocol", RFC 8866,
              DOI 10.17487/RFC8866, January 2021,
              <https://www.rfc-editor.org/info/rfc8866>.

   [RFC9143]  Holmberg, C., Alvestrand, H., and C. Jennings,
              "Negotiating Media Multiplexing Using the Session
              Description Protocol (SDP)", RFC 9143,
              DOI 10.17487/RFC9143, February 2022,
              <https://www.rfc-editor.org/info/rfc9143>.

12.2. Informative References

okay so these are the informative ones 🫢 not required reading, but they're the backstage pass to the stuff this doc leans on: RTP, SRTP, the video codec payload formats, and IKEv2 πŸŽ€


Authors' Addresses ✨

These are the people who wrote it, bestie 🫢 here's where you can find them:

Lauri Ilola
Nokia Technologies
Hatanpaeaen valtatie 30
FI-33100 Tampere
Finland
Email: lauri.ilola@nokia.com


Lukasz Kondrad
Nokia Technologies
Werinherstrasse 91
D-81541 Munich
Germany
Email: lukasz.kondrad@nokia.com
πŸ“¦ where this page comes from