RFC 9924
Advanced Professional Video
RFC 9924: Advanced Professional Video 🎀
okay bestie, let's get into this one 💖 it's an Independent Submission that's Informational, so it's not a standards-track thing. It was written by Y. Lim, M. Park, M. Budagavi, R. Joshi, and K. Choi, all from Samsung Electronics, and it's dated February 2026. It doesn't obsolete or update any other RFCs ✨ and it's giving "here's the codec spec, enjoy" energy 🦋
Abstract ✨
okay so this doc describes the bitstream format of Advanced Professional Video (APV) and how to decode it 🎬 APV is a professional video codec that gives you visually lossless compression, mostly for recording and post production. literally the pro-camera-footage codec, no cap 📹💅
Status of This Memo 🌸
this one is giving not a standard, just vibes 🫶 it's published for informational purposes only and isn't an Internet Standards Track spec. it's a contribution to the RFC Series, independent of any other RFC stream. the RFC Editor picked it to publish at their discretion and makes no claim about how useful it is for implementing or deploying. documents approved this way are not candidates for any level of Internet Standard (see Section 2 of RFC 7841 🧚). for the current status, errata, and feedback, check https://www.rfc-editor.org/info/rfc9924 💜
Copyright Notice 🌈
the copyright belongs to the IETF Trust and the document authors, all rights reserved, 2026 ✨ this doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they stand on the publication date. read those carefully bc they spell out your rights and restrictions 🙈
Table of Contents 🍓
1. Introduction
2. Terms
2.1. Terms and Definitions
2.2. Abbreviated Terms
3. Conventions Used in This Document
3.1. General
3.2. Operators
3.2.1. Arithmetic Operators
3.2.2. Bitwise Operators
3.3. Range Notation
3.3.1. Order of Operations Precedence
3.4. Variables, Syntax Elements, and Tables
3.5. Processes
4. Formats and Processes Used in This Document
4.1. Bitstream Formats
4.2. Source, Decoded, and Output Frame Formats
4.3. Partitioning of a Frame
4.3.1. Partitioning of a Frame into Tiles
4.3.2. Spatial or Component-Wise Partitioning
4.4. Scanning Processes
4.4.1. Zig-Zag Scan
4.4.2. Inverse Scan
5. Syntax and Semantics
5.1. Method of Specifying Syntax
5.2. Syntax Functions and Descriptors
5.2.1. byte_aligned()
5.2.2. more_data_in_tile()
5.2.3. next_bits(n)
5.2.4. read_bits(n)
5.2.5. Syntax Element Processing Functions
5.3. List of Syntax and Semantics
5.3.1. Access Unit
5.3.2. Primitive Bitstream Unit
5.3.3. Primitive Bitstream Unit Header
5.3.4. Frame
5.3.5. Frame Header
5.3.6. Frame Information
5.3.7. Quantization Matrix
5.3.8. Tile Info
5.3.9. Access Unit Information
5.3.10. Metadata
5.3.11. Filler
5.3.12. Tile
5.3.13. Tile Header
5.3.14. Tile Data
5.3.15. Macroblock Layer
5.3.16. AC Coefficient Coding
5.3.17. Byte Alignment
6. Decoding Process
6.1. MB Decoding Process
6.2. Block Reconstruction Process
6.3. Scaling and Transformation Process
6.3.1. Scaling Process for Transform Coefficients
6.3.2. Process for Scaled Transform Coefficients
7. Parsing Process
7.1. Process for Syntax Element Type h(v)
7.1.1. Process for abs_dc_coeff_diff
7.1.2. Process for coeff_zero_run
7.1.3. Process for abs_ac_coeff_minus1
7.1.4. Process for Variable-Length Codes
7.2. Codeword Generation Process for h(v) (Informative)
7.2.1. Process for abs_dc_coeff_diff
7.2.2. Process for coeff_zero_run
7.2.3. Process for abs_ac_coeff_minus1
7.2.4. Process for Variable-Length Codes
8. Metadata Information
8.1. Metadata Payload
8.2. List of Metadata Syntax and Semantics
8.2.1. Filler Metadata
8.2.2. Recommendation ITU-T T.35 Metadata
8.2.3. Mastering Display Color Volume Metadata
8.2.4. Content Light-Level Information Metadata
8.2.5. User-Defined Metadata
8.2.6. Undefined Metadata
9. Profiles, Levels, and Bands
9.1. Overview of Profiles, Levels, and Bands
9.2. Requirements on Video Decoder Capability
9.3. Profiles
9.3.1. General
9.3.2. 422-10 Profile
9.3.3. 422-12 Profile
9.3.4. 444-10 Profile
9.3.5. 444-12 Profile
9.3.6. 4444-10 Profile
9.3.7. 4444-12 Profile
9.3.8. 400-10 Profile
9.4. Levels and Bands
9.4.1. General
9.4.2. Limits of Levels and Bands
10. Security Considerations
11. IANA Considerations
12. References
12.1. Normative References
12.2. Informative References
Appendix A. Raw Bitstream Format
Appendix B. APV Implementations
B.1. OpenAPV Open Source Project
B.2. Android Open Source Project
B.3. FFmpeg Open Source Project
Authors' Addresses1. Introduction 🦋
okay so this section sets the scene ✨ it defines the bitstream format and decoding process for the Advanced Professional Video (APV) codec 🎬 APV was built for professional-level, high-quality video recording and post production, so its main job is pro video recording and editing workflows for all kinds of content 📹 and just so we're clear, this spec is neither the product of the IETF nor a consensus view of the community 🙈
the APV codec supports these features, and they are giving:
- perceptually lossless video quality, close to the original uncompressed stuff 🌸
- low complexity and high throughput intra frame only coding, no inter frame coding at all 🔥
- intra frame coding with no prediction between pixel values, but prediction between transformed values for low delay encoding ⚡
- high bit rates of up to a few Gbps for 2K, 4K, and 8K content, thanks to a lightweight entropy coding scheme 💜
- frame tiling for immersive content and for parallel encoding and decoding 🧚
- various chroma sampling formats from 4:0:0 to 4:4:4:4, and bit depths from 10 to 16 (note: only the profiles supporting 10 bits and 12 bits are defined so far 👀)
- the ability to decode and re-encode multiple times without severe visual quality loss 🫶
- various metadata, including HDR10/10+ and user-defined formats 🌈
that's the intro, bestie 💅 the next part picks up at Section 2, Terms 💖
2.1. Terms and Definitions ✨
- access unit (AU): okay so this is a bundle 📦 of primitive bitstream units (PBU) that holds various types of frames, metadata, filler, and access unit information, all tied to one specific time ⏰
- band: a defined set of constraints on the value of the maximum coded data rate of each level 🎚️
- block: an MxN (M-column by N-row) array of samples, or an MxN array of transform coefficients 🧱
- byte-aligned: a position in a bitstream that sits at an integer multiple of 8 bits from the position of the first bit in the bitstream 📏
- chroma: a sample array or single sample for one of the two color difference signals related to the primary colors, represented by the symbols
CbandCrin4:2:2or4:4:4color format 🌈 - coded frame: a coded representation of a frame that contains all macroblocks of the frame 🎞️
- coded representation: a data element as it looks in its coded form 💾
- component: an array or single sample from one of the three arrays (luma and two chroma) that make up a frame in
4:2:2or4:4:4color format, or an array or single sample from the array that makes up a frame in4:0:0color format, or an array or single sample from one of the four arrays that make up a frame in4:4:4:4color format 🧩 - decoded frame: a frame derived by decoding a coded frame 🖼️
- decoder: an embodiment of a decoding process 🤖
- decoding process: a specified process that reads a bitstream and derives decoded frames from it 📖
- encoder: an embodiment of an encoding process ✍️
- encoding process: a process that produces a bitstream conforming to this document 🏭
- flag: a variable or single-bit syntax element that can take one of the two possible values: 0 and 1 🚩
- frame: an array of luma samples and two corresponding arrays of chroma samples in
4:2:2and4:4:4color format, or an array of samples in4:0:0color format, or four arrays of samples in4:4:4:4color format 🎬 - level: a defined set of constraints on the values taken by the syntax elements and variables of this document, or the value of a transform coefficient prior to scaling 📊
- luma: a sample array or single sample representing the monochrome signal related to the primary colors, represented by the symbol or subscript
YorL🌙 - macroblock (MB): a square block of luma samples and two corresponding blocks of chroma samples of a frame in
4:2:2or4:4:4color format, or a square block of samples of a frame in4:0:0color format, or four square blocks of samples of a frame in4:4:4:4color format 🧱✨ - metadata: data that describes various characteristics related to a bitstream without directly affecting the decoding process of it 🏷️
- partitioning: a division of a set into subsets so that each element of the set is in exactly one of the subsets 🍕
- prediction: an embodiment of the prediction process 🔮
- prediction process: using a predictor to provide an estimate of the data element currently being decoded 🎯
- predictor: a combination of specified values or previously decoded data elements used in the decoding process of subsequent data elements 🧠
- primitive bitstream unit (PBU): a data structure used to construct an access unit with frame and metadata 🧃
- profile: a specified subset of the syntax of this document 🎀
- quantization parameter (QP): a variable used by the decoding process for the scaling value of transform coefficients 🎚️
- raster scan: a mapping of a rectangular two-dimensional pattern to a one-dimensional pattern, where the first entries of the one-dimensional pattern come from the top row of the two-dimensional pattern scanned from left to right, followed by the second, third, etc., rows of the pattern, each scanned from left to right 📜
- raw bitstream: an encapsulation of a sequence of access units where a field indicating the size of an access unit precedes each access unit, as defined in Appendix A 📦
- source: a term used to describe the video material, or some of its attributes, before the encoding process 🎥
- syntax element: an element of data represented in the bitstream 🔤
- syntax structure: zero or more syntax elements present together in a bitstream in a specified order 🧵
- tile: a rectangular region of MBs within a particular tile column and a particular tile row in a frame 🀄
- tile column: a rectangular region of MBs having a height equal to the height of the frame and a width specified by syntax elements in the frame header 🏛️
- tile row: a rectangular region of MBs having a height specified by syntax elements in the frame header and a width equal to the width of the frame 🛏️
- tile scan: a specific sequential ordering of MBs partitioning a frame, in which the MBs are ordered consecutively in MB raster scan within a tile, and the tiles in a frame are ordered consecutively in a raster scan of the tiles of the frame 🗺️
- transform coefficient: a scalar quantity, considered to be in a frequency domain, that is associated with a particular one-dimensional or two-dimensional index 🌊
2.2. Abbreviated Terms 💅
- LSB: least significant bit
- MSB: most significant bit
- RGB: Red, Green and Blue 🌈
3. Conventions Used in This Document 🎀
3.1. General 🫶
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" are the ones we're talking about here, and they get read the way BCP 14 [RFC2119] [RFC8174] describes, but only when they show up in all capitals, exactly like that. Otherwise they're just regular words, bestie 💜
3.2. Operators 🧮
The operators and their order of precedence match the ones used in the C programming language [ISO9899]. Ngl, there are a few exceptions for the operators in Section 3.2.1 and Section 3.2.2, and those follow widely used industry practices for video codecs ✨
3.2.1. Arithmetic Operators ➕
//: an integer division with rounding of the result toward zero. For example,7//4and-7//-4are rounded to 1, and-7//4and7//-4are rounded to -1 🔢/ordiv(x,y): a division in mathematical equations where no truncation or rounding is intended ➗min(x,y): the minimum value of the values x and y 📉max(x,y): the maximum value of the values x and y 📈ceil(x): the smallest integer value that is larger than or equal to x ⬆️clip(x,y,z): defined asclip(x,y,z)=max(x,min(z,y))✂️sum (i=x, y, f(i)): a summation of f(i) with i taking all integer values from x up to and including y 🧾log2(x): the base-2 logarithm of x 🪵
3.2.2. Bitwise Operators 🧠
&(bitwise "and"): when operating on integer arguments, it operates on a two's complement representation of the integer value. When operating on arguments with unequal bit depths, the bit depths are equalized by adding zeros in significant positions to the argument with lower bit depth 🫧|(bitwise "or"): when operating on integer arguments, it operates on a two's complement representation of the integer value. When operating on arguments with unequal bit depths, the bit depths are equalized by adding zeros in significant positions to the argument with lower bit depth 🫧x >> y: an arithmetic right shift of a two's complement integer representation of x by y binary digits. This function is defined only for non-negative integer values of y. Bits shifted into the most significant bits (MSBs) as a result of the right shift have a value equal to the MSB of x prior to the shift operation 👉x << y: an arithmetic left shift of a two's complement integer representation of x by y binary digits. This function is defined only for non-negative integer values of y. Bits shifted into the least significant bits (LSBs) as a result of the left shift have a value equal to 0 👈
3.3. Range Notation 🌸
x = y..z: x takes on integer values starting from y to z, inclusive, with x, y, and z being integer numbers and z being greater than y 🦋
3.3.1. Order of Operations Precedence 👑
When order of precedence is not indicated explicitly by use of parentheses, operations are evaluated in the following order.
- Operations of a higher precedence are evaluated before any operation of a lower precedence. Table 1 specifies the precedence of operations from highest to lowest, and operations closer to the top of the table indicate a higher precedence 📶
- Operations of the same precedence are evaluated sequentially from left to right ➡️
+=========================================+
| operations (with operands x, y, and z) |
+=========================================+
| "x++", "x--" |
+-----------------------------------------+
| "!x", "-x" (as a unary prefix operator) |
+-----------------------------------------+
| x^y (power) |
+-----------------------------------------+
| "x * y", "x / y", "x // y", "x % y" |
+-----------------------------------------+
| "x + y", "x - y", "sum (i=x, y, f(i))" |
+-----------------------------------------+
| "x << y", "x >> y" |
+-----------------------------------------+
| "x < y", "x <= y", "x > y", "x >= y" |
+-----------------------------------------+
| "x == y", "x != y" |
+-----------------------------------------+
| "x & y" |
+-----------------------------------------+
| "x | y" |
+-----------------------------------------+
| "x && y" |
+-----------------------------------------+
| "x || y" |
+-----------------------------------------+
| "x ? y : z" |
+-----------------------------------------+
| "x..y" |
+-----------------------------------------+
| "x = y", "x += y", "x -= y" |
+-----------------------------------------+
Table 1: Operation precedence from
highest (top of the table) to lowest
(bottom of the table)3.4. Variables, Syntax Elements, and Tables
Okay so bestie, this section is all about how the syntax elements and variables get named and written down 🎀. Every syntax element gets a name in all lowercase letters, and its type sits right next to the syntax code in each row. Every syntax element and every multi-byte integer is written in big endian format 💖. The decoding process acts differently depending on the value of the syntax element and on the values of syntax elements that were already decoded ✨.
Sometimes the syntax tables use variables that are derived from syntax element values 🦋. These show up in the tables or the text, and their names mix lowercase and uppercase letters with no underscore characters 💅. If a variable's name starts with an uppercase letter, it gets derived for decoding the current syntax structure and every dependent syntax structure 👑. Those uppercase variables can then be used in the decoding process for later syntax structures without mentioning where they came from 🌈. If a variable's name starts with a lowercase letter, it only gets used inside the section where it was derived 🧚.
Functions that describe the current position in the bitstream are called syntax functions 🌸. They're specified in Section 5.2 and they assume there's a bitstream pointer that marks the position of the next bit the decoding process will read from the bitstream 🫶.
A one-dimensional array is called a list 📝. A two-dimensional array is called a matrix 🍓. Arrays can be syntax elements or variables, and square brackets are used to index them 💜. If you picture a matrix visually, the first square bracket is the column (horizontal) index and the second square bracket is the row (vertical) index 🙈.
You can write out the values of the entries in rows and columns of an array with {{...}{...}}. Each inner pair of brackets holds the values for one row, in increasing column order, and the rows go in increasing row order 🎀. For example, setting a matrix s equal to {{1 6}{4 9}} means s[0][0] is 1, s[1][0] is 6, s[0][1] is 4, and s[1][1] is 9 💖. Yes, the indexes look backwards at first, but trust the process ✨.
Binary notation is written by putting the bit string inside single quote marks 🦋. For example, '0b01000001' is an eight-bit string where only the second bit and the last bit (counting from the most significant bit to the least significant bit) are 1 💅.
Hexadecimal notation works too, as long as you prefix the number with "0x" 👑. You can use it instead of binary when the number of bits is a multiple of 4 🌈. For example, 0x41 is that same eight-bit string, with only its second and last bits (counting from the most to the least significant bit) equal to 1 🧚.
A value of 0 counts as FALSE in a test statement 🌸. The value TRUE is any value that isn't zero 🫶. Easy peasy 💅✨.
3.5. Processes
Processes are how the decoding of syntax elements gets described 🎀. A process has its own separate specification and its own invoking 💖. When you invoke a process, the variables get assigned like this 🦋:
- If the variables at the invoking site and in the process specification don't have the same name, they get explicitly assigned to the lowercase input or output variables of the process specification 💅.
- Otherwise, if the variables at the invoking site and in the process specification have the same name, the assignment is implied 👑.
When a process is specified, a particular coding block is referred to by a variable whose value is the address of that coding block 🌈. Bestie, that's the whole trick 🧚✨.
- Formats and Processes Used in This Document 🌸
4.1. Bitstream Formats 🫶
This section specifies the bitstream format of the Advanced Professional Video (APV) codec 💅. A raw bitstream format is a sequence of AUs where each one is preceded by a field that indicates the size of the access unit 👑. The raw bitstream format is specified in Appendix A 🌈.
4.2. Source, Decoded, and Output Frame Formats 🧚
This section specifies how the source frames and the decoded frames relate to each other 🌸.
The video source represented by the bitstream is a sequence of frames 🫶. Source frames and decoded frames are each made up of one or more sample arrays 💅:
- Monochrome (for example, Luma only) 🎀
- Luma and two chroma (for example, YCbCr or YCgCo as specified in [H273]) 💖
- Green, blue, and red (GBR, also known as RGB) 🦋
- Arrays representing other unspecified tri-stimulus color samplings (for example, YZX, also known as XYZ as specified in [CIE15]) 💅
- Arrays representing other unspecified four color samplings 👑
For the convenience of notation and terminology in this document, the variables and terms tied to these arrays can be called luma and chroma, no matter what color representation method is actually in use 🌈.
The values of SubWidthC, SubHeightC, and NumComps depend on the chroma format sampling structure, as laid out in Table 2 🧚. The chroma format sampling structure is signaled through chroma_format_idc 🌸. Other values of chroma_format_idc, SubWidthC, SubHeightC, and NumComps may be specified in future versions of this document 🫶.
+===================+==========+===========+============+==========+
| chroma_format_idc | Chroma | SubWidthC | SubHeightC | NumComps |
| | format | | | |
+===================+==========+===========+============+==========+
| 0 | 4:0:0 | 1 | 1 | 1 |
+-------------------+----------+-----------+------------+----------+
| 1 | reserved | reserved | reserved | reserved |
+-------------------+----------+-----------+------------+----------+
| 2 | 4:2:2 | 2 | 1 | 3 |
+-------------------+----------+-----------+------------+----------+
| 3 | 4:4:4 | 1 | 1 | 3 |
+-------------------+----------+-----------+------------+----------+
| 4 | 4:4:4:4 | 1 | 1 | 4 |
+-------------------+----------+-----------+------------+----------+
| 5..7 | reserved | reserved | reserved | reserved |
+-------------------+----------+-----------+------------+----------+
Table 2: SubWidthC, SubHeightC, and NumComps values derived from
chroma_format_idcIn 4:0:0 sampling, there's only one sample array, and it can be treated as the luma array 🎀.
In 4:2:2 sampling, each of the two chroma arrays has the same height and half the width of the luma array 💖.
In 4:4:4 sampling and 4:4:4:4 sampling, all the sample arrays have the same height and width as the luma array 🦋.
The number of bits needed to represent each sample in the luma and chroma arrays of a video sequence is in the range of 10 to 16, inclusive 💅.
When chroma_format_idc is equal to 2, the chroma samples are co-sited with the corresponding luma samples 👑. The nominal locations in a frame are shown in Figure 1 🌈.
& * & * & * & * & * ...
& * & * & * & * & * ...
& * & * & * & * & * ...
& * & * & * & * & * ...
...
& - location where both luma and chroma sample exist
* - location where only luma sample exist
Figure 1: Nominal vertical and horizontal locations of 4:2:2 luma
and chroma samples in a frameFor each frame, when chroma_format_idc is equal to 3 or 4, all of the array samples are co-sited 🧚. The nominal locations in a frame are shown in Figure 2 🌸.
& & & & & & & & & & ...
& & & & & & & & & & ...
& & & & & & & & & & ...
& & & & & & & & & & ...
...
& - location where both luma and chroma sample exist
Figure 2: Nominal vertical and horizontal locations of 4:4:4 and
4:4:4:4 luma and chroma samples in a frameSamples are processed in units of MBs 🫶. The variables MbWidth and MbHeight, which give the width and height of the luma arrays for each MB, are defined as follows 💅:
- MbWidth = 16 👑
- MbHeight = 16 🌈
The variables MbWidthC and MbHeightC, which give the width and height of the chroma arrays for each MB, are derived as follows 🧚:
- MbWidthC = MbWidth // SubWidthC 🌸
- MbHeightC = MbHeight // SubHeightC 🫶
4.3. Partitioning of a Frame 💖
4.3.1. Partitioning of a Frame into Tiles 🦋
This section specifies how a frame is partitioned into tiles 💅.
A frame is divided into tiles 👑. A tile is a group of MBs that covers a rectangular region of a frame and is processed independently of other tiles 🌈. Every tile has the same width and height, except possibly tiles at the right or bottom frame boundary when the frame width or height isn't a multiple of the tile width or height, respectively 🧚. The tiles in a frame are scanned in raster order 🌸. Within a tile, the MBs are scanned in raster order 🫶. Each MB is made up of one (MbWidth) x (MbHeight) luma array and zero, two, or three corresponding chroma sample arrays 💅.
For example, a frame is divided into 6 tiles (3 tile columns and 2 tile rows) as shown in Figure 3 👑. In this example, the tile size is 4 column MBs and 4 row MBs 🌈. For the third and sixth tiles (in raster order), the tile size is 2 column MBs and 4 row MBs, since the frame width isn't a multiple of the tile width 🧚.
+===================+===================+=========+
# | | | # MB | MB | MB | MB # MB | MB #
+-------------------+-------------------+---------+
# | | | # MB | MB | MB | MB # MB | MB #
+----- tile -----+-------------------+---------+
# | | | # MB | MB | MB | MB # MB | MB #
+-------------------+-------------------+---------+
# | | | # MB | MB | MB | MB # MB | MB #
+===================+===================+=========+
# MB | MB | MB | MB # MB | MB | MB | MB # MB | MB #
+-------------------+-------------------+---------+
# MB | MB | MB | MB # MB | MB | MB | MB # MB | MB #
+-------------------+-------------------+---------+
# MB | MB | MB | MB # MB | MB | MB | MB # MB | MB #
+-------------------+-------------------+---------+
# MB | MB | MB | MB # MB | MB | MB | MB # MB | MB #
+===================+===================+=========+
#,= tile boundary
|,- MB boundary
Figure 3: Frame with 10 by 8 MBs that is partitioned into 6 tiles4.3.2. Spatial or Component-Wise Partitioning 🧚
The following divisions of processing elements count as spatial or component-wise partitioning 🌸:
- the division of each frame into components 🫶;
- the division of each frame into tile columns 💅;
- the division of each frame into tile rows 👑;
- the division of each tile column into tiles 🌈;
- the division of each tile row into tiles 🧚;
- the division of each tile into color components 🌸;
- the division of each tile into MBs 🫶;
- the division of each MB into blocks 💖.
4.4. Scanning Processes 🦋
4.4.1. Zig-Zag Scan ✨
okay so this is the zig-zag scan, and honestly it's giving main character energy 💅 it takes a 2D array and flattens it into a 1D one. you start at the top-left corner, then zoom diagonally, bouncing off the edges of the block like a little pinball 🎀 until you land on the bottom-right corner 🦋 figure 4 shows the order for a 4x4 block so you can see the pattern.
+===================+
# 00 | 01 | 05 | 06 #
+-------------------+
# 02 | 04 | 07 | 12 #
+-------------------+
# 03 | 08 | 11 | 13 #
+-------------------+
# 09 | 10 | 14 | 15 #
+===================+Figure 4: Example of zig-zag scanning order for 4x4 block
inputs to this process are:
- a variable
blkWidthspecifying the width of a block, and - a variable
blkHeightspecifying the height of a block.
the output is the array zigZagScan[sPos] 🌸
the array index sPos is the scan position, going from 0 to (blkWidth * blkHeight)-1. depending on blkWidth and blkHeight, the array zigZagScan gets derived like this:
pos = 0
zigZagScan[pos] = 0
pos++
for(line = 1; line < (blkWidth + blkHeight - 1); line++){
if(line % 2){
x = min(line, blkWidth - 1)
y = max(0, line - (blkWidth - 1))
while(x >=0 && y < blkHeight){
zigZagScan[pos] = y * blkWidth + x
pos++
x--
y++
}
}
else{
y = min(line, blkHeight - 1)
x = max(0, line - (blkHeight - 1))
while(y >= 0 && x < blkWidth){
zigZagScan[pos] = y * blkWidth + x
pos++
x++
y--
}
}
}Figure 5: Pseudocode for zig-zag scan
4.4.2. Inverse Scan 🔄
inputs to this process are:
- a variable
blkWidthspecifying the width of a block, and - a variable
blkHeightspecifying the height of a block.
the output is the array inverseScan[rPos] 💖
the array index rPos is the raster scan position, going from 0 to (blkWidth * blkHeight)-1. depending on blkWidth and blkHeight, the array inverseScan gets derived like this:
- the variable
forwardScanis derived by invoking the zig-zag scan order initialization process as specified in Section 4.4.1 with input parametersblkWidthandblkHeight. - the output variable
inverseScanis derived as follows:
for(pos = 0; pos < blkWidth * blkHeight; pos++){
inverseScan[forwardScan[pos]] = pos
}Figure 6: Pseudocode for inverse zig-zag scan
5. Syntax and Semantics 🧚
5.1. Method of Specifying Syntax 📜
the syntax tables specify a superset of the syntax of all allowed bitstreams. okay but here's the important bit: a decoder MUST implement some means for identifying entry points into the bitstream and some means to identify and handle non-conforming bitstreams. the methods for identifying and handling errors and other such situations are not specified in this document 😭
the APV bitstream is described using syntax code based on the C programming language [ISO9899], including use of if/else, while, and for, as well as functions defined within this document.
the syntax table in syntax code is presented in a two-column format, like in Figure 7. the type column gives a type that's referenced in that same line of syntax code, using the syntax element processing functions defined in Section 5.2.5.
syntax code | type
--------------------------------------------------------------|-----
ExampleSyntaxCode(){ |
operations |
syntax_element | u(n)
} |Figure 7: A depiction of type-labeled syntax code for syntax description in this document
5.2. Syntax Functions and Descriptors 🎀
the functions in this document are used in the syntactical description. they're expressed in terms of the value of a bitstream pointer, which marks the position of the next bit the decoding process is going to read from the bitstream.
5.2.1. byte_aligned()
- if the current position in the bitstream is on the last bit of a byte, i.e., the next bit in the bitstream is the first bit in a byte, the return value of
byte_aligned()is equal toTRUE. - otherwise, the return value of
byte_aligned()is equal toFALSE.
5.2.2. more_data_in_tile()
- if the current position in the i-th
tile()syntax structure is less thanTileSize[i]in bytes from the beginning of thetile_header()syntax structure of the i-th tile, the return value ofmore_data_in_tile()is equal toTRUE. - otherwise, the return value of
more_data_in_tile()is equal toFALSE.
5.2.3. next_bits(n)
this function gives you the next n bits in the bitstream for comparison purposes, without moving the bitstream pointer forward. peek, don't touch 👀
5.2.4. read_bits(n)
this function says the next n bits from the bitstream are to be read, and it moves the bitstream pointer forward by n bit positions. when n is equal to 0, read_bits(n) returns a value equal to 0 and doesn't move the bitstream pointer at all 🙈
5.2.5. Syntax Element Processing Functions 🍓
b(8): byte having any pattern of bit string (8 bits). the parsing process for this descriptor is specified by the return value of the functionread_bits(8).f(n): fixed-pattern bit string using n bits written (from left to right) with the left bit first, i.e., big endian format. the parsing process for this descriptor is specified by the return value of the functionread_bits(n).u(n): unsigned integer using n bits. the parsing process for this descriptor is specified by the return value of the functionread_bits(n)interpreted as a binary representation of an unsigned integer with the most significant bit written first.h(v): variable-length entropy coded syntax element with the left bit first, i.e., big endian format. the parsing process for this descriptor is specified in Section 7.1.
5.3. List of Syntax and Semantics 🦋
5.3.1. Access Unit 💜
syntax code | type
--------------------------------------------------------------|-----
access_unit(au_size){ |
signature | f(32)
currReadSize = 4 |
do(){ |
pbu_size | u(32)
currReadSize += 4 |
pbu() |
currReadSize += pbu_size |
} while (au_size > currReadSize) |
} |Figure 8: access unit syntax code
signature a four-character code that identifies the bitstream as an APV AU. the value MUST be 'aPv1' (0x61507631) 🫶
pbu_size the size of a primitive bitstream unit in bytes. a value of 0 is prohibited, and the value of 0xFFFFFFFF for pbu_size is reserved for future use.
Note: An AU consists of one primary frame, zero or more non-primary frames such as a frame for additional view, zero or more alpha frames, zero or more depth frames, zero or more preview frames such as a frame with smaller resolution, zero or more metadata, and zero or more fillers. ✨
5.3.2. Primitive Bitstream Unit 🌈
syntax code | type
--------------------------------------------------------------|-----
pbu(){ |
pbu_header() |
if((1 <= pbu_type && pbu_type <=2) || |
(25 <= pbu_type && pbu_type <= 27)) |
frame() |
else if(pbu_type == 65) |
au_info() |
else if(pbu_type == 66) |
metadata() |
else if (pbu_type == 67) |
filler() |
} |Figure 9: primitive bitstream unit syntax code
5.3.3. Primitive Bitstream Unit Header 🎀
okay so the PBU header is the tiny lil header at the front of every PBU, and it's only got three fields 🦋
syntax code | type
--------------------------------------------------------------|-----
pbu_header(){ |
pbu_type | u(8)
group_id | u(16)
reserved_zero_8bits | u(8)
} |Figure 10: primitive bitstream unit header syntax code
pbu_type tells you what kind of data is inside a PBU, and the options are listed in Table 3. Any other pbu_type values are reserved for future use 🌸
+==========+=========================+=======+
| pbu_type | meaning | notes |
+==========+=========================+=======+
| 0 | reserved | |
+----------+-------------------------+-------+
| 1 | primary frame | |
+----------+-------------------------+-------+
| 2 | non-primary frame | |
+----------+-------------------------+-------+
| 3...24 | reserved | |
+----------+-------------------------+-------+
| 25 | preview frame | |
+----------+-------------------------+-------+
| 26 | depth frame | |
+----------+-------------------------+-------+
| 27 | alpha frame | |
+----------+-------------------------+-------+
| 28...64 | reserved | |
+----------+-------------------------+-------+
| 65 | access unit information | |
+----------+-------------------------+-------+
| 66 | metadata | |
+----------+-------------------------+-------+
| 67 | filler | |
+----------+-------------------------+-------+
| 68...255 | reserved | |
+----------+-------------------------+-------+Table 3: List of PBU types
Note: A PBU with pbu_type equal to 65 (access unit information) may happen in an AU. If it exists, it MUST be the first PBU in an AU, and it can be ignored by a decoder. 👀
group_id is the identifier that links a coded frame up with its metadata 🔗. More than two frames can share the same group_id in a single AU. A primary frame and a non-primary frame MUST have different group_id values, and two non-primary frames MUST have different group_id values. When the value of group_id is equal to 0, the value of pbu_type MUST be greater than 64. The value of 0xFFFF for group_id is reserved for future use 💜
reserved_zero_8bits MUST be equal to 0 in bitstreams conforming to the profiles specified in Section 9. Values of reserved_zero_8bits greater than 0 are reserved for future use. Decoders conforming to the profiles specified in Section 9 MUST ignore PBU with values of reserved_zero_8bits greater than 0. Not me being strict about this, but it's a big deal 💀
5.3.4. Frame 🧚
a frame is basically a bundle of tiles, with a header up front and some filler at the end ✨
syntax code | type
--------------------------------------------------------------|-----
frame(){ |
frame_header() |
for(i = 0; i < NumTiles; i++){ |
tile_size[i] | u(32)
tile(i) |
} |
filler() |
} |Figure 11: frame() syntax code
tile_size[i] tells you how many bytes the i-th tile data (aka tile(i)) takes up, and the tiles go in raster order inside a frame 🌈 The value of 0 for tile_size[i] is reserved for future use.
The variable TileSize[i] is set equal to tile_size[i] 🫶
5.3.5. Frame Header 💅
this is the big one for frames. It holds the frame info, the colour stuff, the quantization matrix flag, tile info, and then byte alignment to wrap it up 🍓
syntax code | type
--------------------------------------------------------------|-----
frame_header(){ |
frame_info() |
reserved_zero_8bits | u(8)
color_description_present_flag | u(1)
if(color_description_present_flag){ |
color_primaries | u(8)
transfer_characteristics | u(8)
matrix_coefficients | u(8)
full_range_flag | u(1)
} |
use_q_matrix | u(1)
if(use_q_matrix){ |
quantization_matrix() |
} |
tile_info() |
reserved_zero_8bits | u(8)
byte_alignment() |
} |Figure 12: frame_header() syntax code
reserved_zero_8bits MUST be equal to 0 in bitstreams conforming to the profiles specified in Section 9. Values of reserved_zero_8bits greater than 0 are reserved for future use. Decoders conforming to the profiles specified in Section 9 MUST ignore PBU with values of reserved_zero_8bits greater than 0. 😭
color_description_present_flag equal to 1 specifies that color_primaries, transfer_characteristics, and matrix_coefficients are present. color_description_present_flag equal to 0 specifies that color_primaries, transfer_characteristics, and matrix_coefficients are not present.
color_primaries MUST have the semantics of ColourPrimaries as specified in [H273]. When the color_primaries syntax element is not present, the value of color_primaries is inferred to be equal to 2.
transfer_characteristics MUST have the semantics of TransferCharacteristics as specified in [H273]. When the transfer_characteristics syntax element is not present, the value of transfer_characteristics is inferred to be equal to 2.
matrix_coefficients MUST have the semantics of MatrixCoefficients as specified in [H273]. When the matrix_coefficients syntax element is not present, the value of matrix_coefficients is inferred to be equal to 2. 🎨
full_range_flag MUST have the semantics of VideoFullRangeFlag as specified in [H273]. When the full_range_flag syntax element is not present, the value of full_range_flag is inferred to be equal to 0.
use_q_matrix with a value of 1 specifies that the quantization matrices are present. A value of 0 specifies that the quantization matrices are not present.
reserved_zero_8bits MUST be equal to 0 in bitstreams conforming to the profiles specified in Section 9. Values of reserved_zero_8bits greater than 0 are reserved for future use. Decoders conforming to the profiles specified in Section 9 MUST ignore PBU with values of reserved_zero_8bits greater than 0. 🙈
5.3.6. Frame Information 🎀
okay so frame info is where the frame gets its ID card 🪪✨ it tells you the profile, level, bandwidth band, how big the frame is, the chroma format, bit depth, and capture timing. bestie, this one is lowkey the main character of the frame header 💅
syntax code | type
--------------------------------------------------------------|-----
frame_info(){ |
profile_idc | u(8)
level_idc | u(8)
band_idc | u(3)
reserved_zero_5bits | u(5)
frame_width | u(24)
frame_height | u(24)
chroma_format_idc | u(4)
bit_depth_minus8 | u(4)
capture_time_distance | u(8)
reserved_zero_8bits | u(8)
} |Figure 13: frame_info() syntax code
profile_idc 🌸 indicates a profile that the coded frame conforms to, as specified in Section 9. Bitstreams SHALL NOT contain values of profiles_idc other than those specified in Section 9. Other values of profile_idc are reserved for future use.
level_idc 🦋 indicates a level that the coded frame conforms to, as specified in Section 9. Bitstreams SHALL NOT contain values of level_idc other than those specified in Section 9. Other values of level_idc are reserved for future use.
band_idc 🍓 specifies a maximum coded data rate of level_idc, as specified in Section 9. Bitstreams SHALL NOT contain values of band_idc other than those specified in Section 9. The value of band_idc MUST be in the range of 0 to 3. Other values of band_idc are reserved for future use.
reserved_zero_5bits 🫶 MUST be equal to 0 in bitstreams conforming to the profiles specified in Section 9. Values of reserved_zero_8bits greater than 0 are reserved for future use. Decoders conforming to the profiles specified in Section 9 MUST ignore PBU with values of reserved_zero_8bits greater than 0.
frame_width 📐 specifies the width of the frame in units of luma samples. frame_width MUST be a multiple of 2 when chroma_format_idc has a value of 2. The value 0 is reserved for future use.
frame_height 📏 specifies the height of the frame in units of luma samples. The value 0 is reserved for future use.
The variables FrameWidthInMbsY, FrameHeightInMbsY, FrameWidthInSamplesY, FrameHeightInSamplesY, FrameWidthInSamplesC, FrameHeightInSamplesC, FrameSizeInMbsY, and FrameSizeInSamplesY are derived as follows:
FrameWidthInSamplesY = frame_widthFrameHeightInSamplesY = frame_heightFrameWidthInMbsY = ceil(FrameWidthInSamplesY / MbWidth)FrameHeightInMbsY = ceil(FrameHeightInSamplesY / MbHeight)FrameWidthInSamplesC = FrameWidthInSamplesY // SubWidthCFrameHeightInSamplesC = FrameHeightInSamplesY // SubHeightCFrameSizeInMbsY = FrameWidthInMbsY * FrameHeightInMbsYFrameSizeInSamplesY = FrameWidthInSamplesY * FrameHeightInSamplesY
chroma_format_idc 🌈 specifies the chroma sampling relative to the luma sampling, as specified in Table 2. The value of chroma_format_idc MUST be 0, 2, 3, or 4. Other values are reserved for future use.
bit_depth_minus8 💜 specifies the bit depth of the samples. The variables BitDepth and QpBdOffset are derived as follows:
BitDepth = bit_depth_minus8 + 8QpBdOffset = bit_depth_minus8 * 6
bit_depth_minus8 MUST be in the range of 2 to 8, inclusive. Other values are reserved for future use.
capture_time_distance ⏱️ indicates the time difference, in milliseconds, between the capture time of the frames in the previous access unit and the frames in the current access unit, if there has been any access unit preceding the access unit this frame belongs to.
reserved_zero_8bits 🙈 MUST be equal to 0 in bitstreams conforming to the profiles specified in Section 9. Values of reserved_zero_8bits greater than 0 are reserved for future use. Decoders conforming to the profiles specified in Section 9 MUST ignore PBU with values of reserved_zero_8bits greater than 0.
5.3.7. Quantization Matrix 🧁
okay the quantization matrix is basically a grid of scaling values, one 8x8 grid per colour component 🎨 it's giving organised. ngl I love a good grid
syntax code | type
--------------------------------------------------------------|-----
quantization_matrix(){ |
for(i = 0; i < NumComps; i++){ |
for(y = 0; y < 8; y++){ |
for(x = 0; x < 8; x++){ |
q_matrix[i][x][y] | u(8)
} |
} |
} |
} |Figure 14: quantization_matrix() syntax code
q_matrix[i][x][y] 🌷 specifies a scaling value in the quantization matrices. When q_matrix[i][x][y] is not present, it is inferred to be equal to 16. The array index i specifies an indicator for the color component. When chroma_format_idc is equal to 2 or 3, the value of the index i is equal to 0 for the Y component, 1 for Cb, and 2 for Cr. The value of 0 for q_matrix[i][x][y] is reserved for future use.
The quantization matrix, QMatrix[i][x][y], is derived as follows:
QMatrix[i][x][y] = q_matrix[i][x][y]
5.3.8. Tile Info 🧩
tile info is where you cut the frame into little puzzle pieces 🧩 tiles! you tell it how wide and how tall each tile is in macroblocks, and it works out where every column and row starts. it's giving very organised bestie 😭
syntax code | type
--------------------------------------------------------------|-----
tile_info(){ |
tile_width_in_mbs | u(20)
tile_height_in_mbs | u(20)
startMb = 0 |
for(i = 0; startMb < FrameWidthInMbsY; i++){ |
ColStarts[i] = startMb * MbWidth |
startMb += tile_width_in_mbs |
} |
ColStarts[i] = FrameWidthInMbsY*MbWidth |
TileCols = i |
startMb = 0 |
for(i = 0; startMb < FrameHeightInMbsY; i++){ |
RowStarts[i] = startMb * MbHeight |
startMb += tile_height_in_mbs |
} |
RowStarts[i] = FrameHeightInMbsY*MbHeight |
TileRows = i |
NumTiles = TileCols * TileRows |
tile_size_present_in_fh_flag | u(1)
if(tile_size_present_in_fh_flag){ |
for(i = 0; i < NumTiles; i++){ |
tile_size_in_fh[i] | u(32)
} |
} |
} |Figure 15: tile_info() syntax code
tile_width_in_mbs 🌼 specifies the width of a tile in units of MBs.
tile_height_in_mbs 🌿 specifies the height of a tile in units of MBs.
tile_size_present_in_fh_flag 🎟️ equal to 1 specifies that tile_size_in_fh[i] is present in the frame header. tile_size_present_in_fh_flag equal to 0 specifies that tile_size_in_fh[i] is not present in the frame header.
tile_size_in_fh[i] 📦 indicates the size in bytes of the i-th tile data in raster order in a frame. The value of tile_size_in_fh[i] MUST have the same value with tile_size[i]. When it is not present, the value of tile_size_in_fh[i] is inferred to be equal to tile_size[i]. The value of 0 for tile_size_in_fh[i] is reserved for future use.
5.3.9. Access Unit Information 📚
last one in this part, and it's the AU info, the table of contents for the whole access unit 📖 it lists how many frames are in there, and for each frame it gives you the pbu_type, group_id, and frame_info. slay, very organised, very tidy 💅
syntax code | type
--------------------------------------------------------------|-----
au_info(){ |
num_frames | u(16)
for(i = 0; i < num_frames; i++){ |
pbu_type | u(8)
group_id | u(16)
reserved_zero_8bits | u(8)
frame_info() |
} |
reserved_zero_8bits | u(8)
byte_alignment() |
filler() |
} |Figure 16: au_info() syntax code
num_frames 🎞️ indicates the number of frames contained in the current AU.
pbu_type 🎯 has the same semantics as pbu_type in the pbu_header() syntax.
Note: The value of pbu_type MUST be 1, 2, 25, 26, or 27 in bitstreams conforming to this document.
group_id 👯 has the same semantics as group_id in the pbu_header() syntax.
reserved_zero_8bits 🫧 MUST be equal to 0 in bitstreams conforming to the profiles specified in Section 9. Values of reserved_zero_8bits greater than 0 are reserved for future use. Decoders conforming to the profiles specified in Section 9 MUST ignore PBU with values of reserved_zero_8bits greater than 0.
5.3.10. Metadata 🧚
okay so metadata() is basically the bestie who carries a bunch of little notes around inside a PBU 💌 it's a loop that keeps reading payloads until the total size is reached, then it hits filler() to finish up. the bits are read exactly like this 👇
syntax code | type
--------------------------------------------------------------|-----
metadata(){ |
metadata_size | u(32)
currReadSize = 0 |
do{ |
payloadType = 0 |
while(next_bits(8) == 0xFF){ |
ff_byte | f(8)
payloadType += ff_byte |
currReadSize++ |
} |
metadata_payload_type | u(8)
payloadType += metadata_payload_type |
currReadSize++ |
|
payloadSize = 0 |
while(next_bits(8) == 0xFF){ |
ff_byte | f(8)
payloadSize += ff_byte |
currReadSize++ |
} |
metadata_payload_size | u(8)
payloadSize += metadata_payload_size |
currReadSize++ |
|
metadata_payload(payloadType, payloadSize) |
currReadSize += payloadSize |
} while(metadata_size > currReadSize) |
filler() |
} |Figure 17: metadata() syntax code
metadata_size tells you the size of the metadata that sits before filler() in the current PBU, so it's the total you're counting toward 💅
ff_byte is a byte equal to 0xFF. ✨
metadata_payload_type gives the last byte of the payload type of a metadata. 🌸
metadata_payload_size gives the last byte of the payload size of a metadata. 🦋
Syntax and semantics of metadata_payload() are specified in Section 8. so go check there for the juicy details 👑
5.3.11. Filler 🫶
filler() is the padding friend 🧸 it just keeps eating 0xFF bytes for as long as the next 8 bits are 0xFF. no drama, no cap.
syntax code | type
--------------------------------------------------------------|-----
filler(){ |
while(next_bits(8) == 0xFF) |
ff_byte | f(8)
} |Figure 18: filler() syntax code
ff_byte is a byte equal to 0xFF. 💖
5.3.12. Tile 🎀
okay this is the main character moment 👑 tile() is the whole package: it starts with the tile header, then grabs the tile data for every color component, and then it keeps skipping along with dummy bytes for as long as there's more data in the tile 💅
syntax code | type
--------------------------------------------------------------|-----
tile(tileIdx){ |
tile_header(tileIdx) |
for(i = 0; i < NumComps; i++){ |
tile_data(tileIdx, i) |
} |
while(more_data_in_tile()){ |
tile_dummy_byte | b(8)
} |
} |Figure 19: tile() syntax code
tile_dummy_byte has any pattern of 8-bit string. literally any 8 bits, it's giving "I don't care" energy 🙈
5.3.13. Tile Header 🌈
this is where the tile introduces itself 💅 it has the size, the index, how big each component's data is, and the QP for each component. then there's a reserved byte and a byte alignment to wrap it up. ✨
syntax code | type
--------------------------------------------------------------|-----
tile_header(tileIdx){ |
tile_header_size | u(16)
tile_index | u(16)
for(i = 0; i < NumComps; i++){ |
tile_data_size[i] | u(32)
} |
for(i = 0; i < NumComps; i++){ |
tile_qp[i] | u(8)
} |
reserved_zero_8bits | u(8)
byte_alignment() |
} |Figure 20: tile_header() syntax code
tile_header_size indicates the size of the tile header in bytes. 🌸
tile_index specifies the tile index in raster order in a frame. tile_index MUST have the same value as tileIdx. this one is a strict bestie rule, no exceptions 🫶
tile_data_size[i] indicates the size of the i-th color component data in a tile in bytes. the array index i is an indicator for the color component. when chroma_format_idc is equal to 2 or 3, the value of the index i is equal to 0 for Y component, 1 for Cb, and 2 for Cr. the value of 0 for tile_data_size[i] is reserved for future use, so don't use it yet 🦋
tile_qp[i] specifies the quantization parameter value for the i-th color component. the array index i is an indicator for the color component. when chroma_format_idc is equal to 2 or 3, the value of the index i is equal to 0 for Y component, 1 for Cb, and 2 for Cr. the Qp[i] used for the MBs in the tile is derived as follows: 🧮
Qp[i] = tile_qp[i] - QpBdOffsetQp[i]MUST be in the range of-QpBdOffsetto 51, inclusive.
reserved_zero_8bits MUST be equal to 0 in bitstreams conforming to the profiles specified in Section 9. values of reserved_zero_8bits greater than 0 are reserved for future use. decoders conforming to the profiles specified in Section 9 MUST ignore PBU with values of reserved_zero_8bits greater than 0. so if you see a reserved value, you skip that PBU, period 🙅♀️💀
5.3.14. Tile Data 💜
tile_data() is the calculator bestie 🧮 it works out where each macroblock in the tile lives, then collects them all in order.
syntax code | type
--------------------------------------------------------------|-----
tile_data(tileIdx, cIdx){ |
x0 = ColStarts[tileIdx % TileCols] |
y0 = RowStarts[tileIdx // TileCols] |
numMbColsInTile = (ColStarts[tileIdx % TileCols + 1] - |
ColStarts[tileIdx % TileCols]) // MbWidth |
numMbRowsInTile = (RowStarts[tileIdx // TileCols + 1] - |
RowStarts[tileIdx // TileCols]) // MbHeight |
numMbsInTile = numMbColsInTile * numMbRowsInTile |
PrevDC = 0 |
PrevDcDiff = 20 |
Prev1stAcLevel = 0 |
for(i = 0; i < numMbsInTile; i++){ |
xMb = x0 + ((i % numMbColsInTile) * MbWidth) |
yMb = y0 + ((i // numMbColsInTile) * MbHeight) |
macroblock_layer(xMb, yMb, cIdx) |
} |
byte_alignment() |
} |Figure 21: tile_data() syntax code
The tile_data() syntax calculates the location of the macroblocks belonging to each tile and collects them. ✨
5.3.15. Macroblock Layer 🦋
okay last one for this part and it's the one with all the coefficient drama 🔥 macroblock_layer() goes through each block in the macroblock, reads the DC difference, works out the transform coefficient, and then hands the rest over to ac_coeff_coding() 💅
syntax code | type
--------------------------------------------------------------|-----
macroblock_layer(xMb, yMb, cIdx){ |
subW = (cIdx == 0)? 1 : SubWidthC |
subH = (cIdx == 0)? 1 : SubHeightC |
blkWidth = (cIdx == 0)? MbWidth : MbWidthC |
blkHeight = (cIdx == 0)? MbHeight : MbHeightC |
TrSize = 8 |
for(y = 0; y < blkHeight; y += TrSize){ |
for(x = 0; x < blkWidth; x += TrSize){ |
abs_dc_coeff_diff | h(v)
if(abs_dc_coeff_diff) |
sign_dc_coeff_diff | u(1)
TransCoeff[cIdx][xMb // subW + x][yMb // subH + y] = |
PrevDC + abs_dc_coeff_diff * |
(1 - 2*sign_dc_coeff_diff) |
PrevDC = |
TransCoeff[cIdx][xMb // subW + x][yMb // subH + y] |
PrevDcDiff = abs_dc_coeff_diff |
ac_coeff_coding(xMb // subW + x, yMb // subH + y, |
log2(TrSize), log2(TrSize), cIdx) |
} |
} |
} |Figure 22: macroblock_layer() syntax code
abs_dc_coeff_diff specifies the absolute value of the difference between the current DC transform coefficient level and PrevDC. 🧠
sign_dc_coeff_diff specifies the sign of the difference between the current DC transform coefficient level and PrevDC. sign_dc_coeff_diff equal to 0 specifies that the difference has a positive value. sign_dc_coeff_diff equal to 1 specifies that the difference has a negative value. so 0 is the happy plus-vibes one and 1 is the minus one 🙃
The transform coefficients are represented by the arrays TransCoeff[cIdx][x0][y0]. the array indices x0 and y0 specify the location (x0, y0) relative to the top-left sample for each component of the frame. the array index cIdx is an indicator for the color component. when chroma_format_idc is equal to 2 or 3, the value of the index i is equal to 0 for Y component, 1 for Cb, and 2 for Cr. 🌈
The value of TransCoeff[cIdx][x0][y0] MUST be in the range of -32768 to 32767, inclusive. the coefficient has to stay in that range, no exceptions, periodt 💅✨
5.3.16. AC Coefficient Coding ✨
okay so this is where the AC coefficients get packed up 🎀 each one gets a little zero-run count and then its magnitude and sign, and it all walks through the block in zig-zag order 🌀
syntax code | type
--------------------------------------------------------------|-----
ac_coeff_coding(x0, y0, log2BlkWidth, log2BlkHeight, cIdx){ |
scanPos = 1 |
firstAC = 1 |
PrevLevel = Prev1stAcLevel |
PrevRun = 0 |
do{ |
coeff_zero_run | h(v)
for(i = 0; i < coeff_zero_run; i++){ |
blkPos = ScanOrder[scanPos] |
xC = blkPos & ((1 << log2BlkWidth) - 1) |
yC = blkPos >> log2BlkWidth |
TransCoeff[cIdx][x0+xC][y0 + yC] = 0 |
scanPos++ |
} |
PrevRun = coeff_zero_run |
if(scanPos < (1 << (log2BlkWidth + log2BlkHeight))){ |
abs_ac_coeff_minus1 | h(v)
sign_ac_coeff | u(1)
level = (abs_ac_coeff_minus1 + 1) * |
(1 - 2 * sign_ac_coeff) |
blkPos = ScanOrder[scanPos] |
xC = blkPos & ((1 << log2BlkWidth) - 1) |
yC = blkPos >> log2BlkWidth |
TransCoeff[cIdx][x0 + xC][y0 + yC] = level |
scanPos++ |
PrevLevel = abs_ac_coeff_minus1 + 1 |
if(firstAC == 1){ |
firstAC = 0 |
Prev1stAcLevel = PrevLevel |
} |
} |
} while(scanPos < (1 << (log2BlkWidth + log2BlkHeight))) |
} |Figure 23: ac_coeff_coding() syntax code
coeff_zero_run 🫧 This one counts the zero-valued transform coefficient levels that sit before the next non-zero transform coefficient level in a scan of transform coefficient levels. So basically it's the number of zeros you skip past before hitting a real value 💅
abs_ac_coeff_minus1 🌸 The "plus 1" part is the twist here. This field gives the absolute value of an AC transform coefficient level at the given scanning position, minus one, so you add 1 to get the actual magnitude ✨
sign_ac_coeff 🦋 This one tells you the sign of the AC transform coefficient level at the given scanning position. A value of 0 means the level is positive, and a value of 1 means it's negative. Very binary, very iconic 🙈
The array ScanOrder[sPos] maps the zig-zag scan position sPos, which runs from 0 to (1 << log2BlkWidth) * (1 << log2BlkHeight) - 1 inclusive, onto a raster scan position rPos. To build it, you invoke Section 4.4.1 with blkWidth equal to (1 << log2BlkWidth) and blkHeight equal to (1 << log2BlkHeight) as the input parameters. Main character energy for the lookup table 👑
5.3.17. Byte Alignment 🧚
this part is all about padding out to a byte boundary 🍓 you keep reading zero bits until you're byte aligned
syntax code | type
--------------------------------------------------------------|-----
byte_alignment(){ |
while(!byte_aligned()) |
alignment_bit_equal_to_zero | f(1)
} |Figure 24: byte_alignment() syntax code
alignment_bit_equal_to_zero 💜 This bit MUST be equal to 0. No exceptions, bestie 🙅♀️
6. Decoding Process 🌈
okay, now we're getting into the decoder side and this is where a bitstream of a coded frame gets turned back into a decoded frame 🦋 the input is one coded frame's bitstream and the output is the decoded frame, lowkey the whole point of the document ✨
The decoding process works like this for the current frame:
- The syntax structure for a coded frame gets parsed so you end up with the parsed syntax structures 🧩
- Sections 6.1, 6.2, and 6.3 spell out the decoding processes using the syntax elements from every syntax structure. For bitstreams that conform to this document, the coded tiles of the frame MUST contain tile data for every MB of the frame, so that dividing the frame into tiles and dividing the tiles into MBs makes a partitioning of the frame. Non-negotiable, periodt 💅
- Once every tile in the current frame has been decoded, the decoded frame gets cropped with the cropping rectangle, but only if
FrameWidthInSamplesYis not equal toFrameWidthInMbsY * MbWidthorFrameHeightInSamplesYis not equal toFrameHeightInMbsY * MbHeight🎀 - The cropping rectangle, which is the set of samples of a frame that actually get output, is derived like this:
- The cropping rectangle contains the luma samples with horizontal frame coordinates from 0 to
FrameWidthInSampleY - 1and vertical frame coordinates from 0 toFrameHeightInSamplesY - 1, inclusive 🌸 - The cropping rectangle also contains the two chroma arrays, which have frame coordinates
(x//SubWidthC, y//SubHeightC), where(x,y)are the frame coordinates of the specified luma samples 🫶
- The cropping rectangle contains the luma samples with horizontal frame coordinates from 0 to
6.1. MB Decoding Process ✨
This process is invoked for each MB 💖
The input is a luma location (xMb, yMb) that specifies the top-left sample of the current luma MB, relative to the top-left luma sample of the current frame. The outputs are the reconstructed samples of all color components. The total number of color components is whatever NumComps says for the current MB. For example, when chroma_format_idc is equal to 2 or 3, NumComps is equal to 3, and three components get reconstructed: the Y component, the Cb component, and the Cr component.
Here's the step-by-step vibe:
- Let
recSamples[0]be a(MbWidth)x(MbHeight)array of the reconstructed samples of the first color component (whenchroma_format_idcis equal to 2 or 3, that's Y) 🍓 - The block reconstruction process in Section 6.2 gets invoked with the luma location
(xMb, yMb),nBlkWset equal toMbWidth,nBlkHset equal toMbHeight,cIdxset equal to 0, and the(MbWidth)x(MbHeight)arrayrecSamples[0]as inputs. The output is a modified version ofrecSamples[0], which holds the reconstructed samples of the first color component for the current MB 🧚 - When
chroma_format_idcis not equal to 0, letrecSamples[1]be a(MbWidthC)x(MbHeightC)array of the reconstructed samples of the second color component. For example, whenchroma_format_idcis equal to 2 or 3,recSamples[1]is the Cb color component 🦋 - When
chroma_format_idcis not equal to 0, the block reconstruction process in Section 6.2 gets invoked with the luma location(xMb, yMb),nBlkWset equal toMbWidthC,nBlkHset equal toMbHeightC,cIdxset equal to 1, and the(MbWidthC)x(MbHeightC)arrayrecSamples[1]as inputs. The output is a modified version ofrecSamples[1], which holds the reconstructed samples of the second color component for the current MB 💜 - When
chroma_format_idcis not equal to 0, letrecSamples[2]be a(MbWidthC)x(MbHeightC)array of the reconstructed samples of the third color component. For example, whenchroma_format_idcis equal to 2 or 3,recSamples[2]is the Cr color component 🌈 - When
chroma_format_idcis not equal to 0, the block reconstruction process in Section 6.2 gets invoked with the luma location(xMb, yMb),nBlkWset equal toMbWidthC,nBlkHset equal toMbHeightC,cIdxset equal to 2, and the(MbWidthC)x(MbHeightC)arrayrecSamples[2]as inputs. The output is a modified version ofrecSamples[2], which holds the reconstructed samples of the third color component for the current MB ⭐ - When
chroma_format_idcis equal to 4, letrecSamples[3]be a(MbWidthC)x(MbHeightC)array of the reconstructed samples of the fourth color component 👑 - When
chroma_format_idcis equal to 4, the block reconstruction process in Section 6.2 gets invoked with the luma location(xMb, yMb),nBlkWset equal toMbWidthC,nBlkHset equal toMbHeightC,cIdxset equal to 3, and the(MbWidthC)x(MbHeightC)arrayrecSamples[3]as inputs. The output is a modified version ofrecSamples[3], which holds the reconstructed samples of the fourth color component for the current MB 🫧
6.2. Block Reconstruction Process 🎀
Inputs to this process are:
- a luma location
(xMb, yMb)that specifies the top-left sample of the current MB, relative to the top-left luma sample of the current frame 🌸 - two variables,
nBlkWandnBlkH, that specify the width and the height of the current block 💖 - a variable
cIdxthat specifies the color component of the current block 🦋 - an
(nBlkW)x(nBlkH)array ofrecSamplesof a reconstructed block ✨
The output of this process is a modified version of the (nBlkW)x(nBlkH) array recSamples of reconstructed samples. Slay 💅
The following applies:
- The variables
numBlkXandnumBlkYare derived as follows:numBlkX = nBlkW // TrSizenumBlkY = nBlkH // TrSize
- For
yIdx = 0..numBlkY - 1, the following applies:- For
xIdx = 0..numBlkX - 1, the following applies:- The variables
xBlkandyBlkare derived as follows:xBlk = xMb // (cIdx==0? 1: SubWidthC) + xIdx*TrSizeyBlk = yMb // (cIdx==0? 1: SubHeightC) + yIdx*TrSize
- The scaling and transformation process in Section 6.3 gets invoked with the location
(xBlk, yBlk), the variablecIdxset equal tocIdx, the transform widthnBlkWset equal toTrSize, and the transform heightnBlkHset equal toTrSizeas inputs. The output is a(TrSize)x(TrSize)arrayrof a reconstructed block 🧩 - The
(TrSize)x(TrSize)arrayrecSamplesgets modified like this:
- The variables
- For
recSamples[(xIdx * TrSize) + i, (yIdx * TrSize) + j] =
r[i,j], with i=0..TrSize-1, j=0..TrSize-16.3. Scaling and Transformation Process ✨
Okay bestie, this is where the reconstructed samples actually get cooked 🍳 Here's what goes in:
- a location
(xBlkY, yBlkY)of the current color component, which is the top-left sample of the current block relative to the top-left sample of the current frame, - a variable
nBlkWfor the width of the current block, - a variable
nBlkHfor the height of the current block, and - a variable
cIdxfor the color component of the current block.
What comes out is the (nBlkW)x(nBlkH) array of reconstructed samples r, where each element is r[x][y]. 💖
First up, the quantization parameter qP gets worked out like this:
qP = Qp[cIdx] + QpBdOffsetThen the (nBlKW)x(nBlkH) array of reconstructed samples r is built in three steps 🎀
- The scaling process for transform coefficients in Section 6.3.1 gets invoked with the block location
(xBlkY, yBlkY), the block widthnBlkW, the block heightnBlkH, the color component variablecIdx, and the quantization parameterqP. What you get back is an(nBlkW)x(nBlkH)array of scaled transform coefficientsd. - The transformation process for scaled transform coefficients in Section 6.3.2 gets invoked with the block location
(xBlkY, yBlkY), the block widthnBlkW, the block heightnBlkH, the color component variablecIdx, and the(nBlkW)x(nBlkH)array of scaled transform coefficientsd. The output is an(nBlkW)x(nBlkH)array of reconstructed samplesr. - The variable
bdShiftis derived like this:bdShift = 20 - BitDepth
Last step, the reconstructed sample values r[x][y] (with x = 0..nBlkW - 1 and y = 0..nBlkH - 1) get modified like so 🧚
r[x][y] = clip(0, (1 << BitDepth)-1, ((r[x][y]+(1 << (bdShift-
1)))>>bdShift) + (1 << (BitDepth-1)))6.3.1. Scaling Process for Transform Coefficients
This process takes in a bunch of stuff 💅 It needs:
- a location
(xBlkY, yBlkY)of the current color component, which is the top-left sample of the current block relative to the top-left sample of the current frame, - a variable
nBlkWfor the width of the current block, - a variable
nBlkHfor the height of the current block, - a variable
cIdxfor the color component of the current block, and - a variable
qPfor the quantization parameter.
Output is the (nBlkW)x(nBlkH) array d of scaled transform coefficients, with elements d[x][y]. 🦋
Here's how bdShift gets derived:
bdShift = BitDepth + ((log2(nBlkW) + log2(nBlkH)) // 2) - 5The list levelScale[] is just levelScale[k] = {40, 45, 51, 57, 64, 71} with k = 0..5. Iconic table, no notes 📝
For deriving the scaled transform coefficients d[x][y] (with x = 0..nBlkW - 1 and y = 0..nBlkH - 1), the following applies ✨
d[x][y] = clip(-32768, 32767, ((TransCoeff[cIdx][xBlkY][yBlkY]
* QMatrix[cIdx][x][y] * levelScale[qP % 6] << (qP//6)) + (1 <<
(bdShift-1)) >> bdShift))6.3.2. Process for Scaled Transform Coefficients
6.3.2.1. General
Okay so this one is the main event 👑 It takes in:
- a location
(xBlkY, yBlkY)of the current color component, which is the top-left sample of the current block relative to the top-left sample of the current frame, - a variable
nBlkWfor the width of the current block, - a variable
nBlkHfor the height of the current block, and - an
(nBlkW)x(nBlkH)arraydof scaled transform coefficients, with elementsd[x][y].
Output is the (nBlkW)x(nBlkH) array r of reconstructed samples, with elements r[x][y]. 🌈
The (nBlkW)x(nBlkH) array r gets derived in these steps:
- Each (vertical) column of scaled transform coefficients
d[x][y](withx = 0..nBlkW - 1andy = 0..nBlkH - 1) gets transformed intoe[x][y](same index ranges). This happens by invoking the one-dimensional transformation process from Section 6.3.2.2 for each columnx = 0..nBlkW - 1, with the size of the transform block set tonBlkH, and with the listd[x][y](y = 0..nBlkH - 1) as inputs. The output is the liste[x][y]withy = 0..nBlkH - 1. - The following applies:
g[x][y] = (e[x][y] + 64) >> 7
- Each (horizontal) row of the resulting array
g[x][y](withx = 0..nBlkW - 1andy = 0..nBlkH - 1) gets transformed intor[x][y](same index ranges). This happens by invoking the one-dimensional transformation process from Section 6.3.2.2 for each rowy = 0..nBlkH - 1, with the size of the transform block set tonBlkW, and with the listg[x][y](x = 0..nBlkW - 1) as inputs. The output is the listr[x][y]withx = 0..nBlkW - 1.
6.3.2.2. Transformation Process
Inputs to this one are:
- a variable
nTbSspecifying the sample size of scaled transform coefficients, and - a list of scaled transform coefficients
x, with elementsx[j], wherej = 0..(nTbS - 1).
Output is the list of transformed samples y, with elements y[i], where i = 0..(nTbS - 1). 🦋
The transformation matrix derivation process from Section 6.3.2.3 gets invoked with the transform size nTbS as input, and the transformation matrix transMatrix comes out as output.
The list of transformed samples y[i] (with i = 0..(nTbS - 1)) is derived like this: y[i] = sum(j = 0, nTbS - 1, transMatrix[i][j] * x[j]) 💅
6.3.2.3. Transformation Matrix Derivation Process
Input here is a variable nTbS, specifying the horizontal sample size of scaled transform coefficients. Output is the transformation matrix transMatrix. 🧚
The transformation matrix transMatrix gets derived based on nTbs like so:
- If
nTbSis equal to 8, the following applies:
transMatrix[m][n] =
{
{ 64, 64, 64, 64, 64, 64, 64, 64 }
{ 89, 75, 50, 18, -18, -50, -75, -89 }
{ 84, 35, -35, -84, -84, -35, 35, 84 }
{ 75, -18, -89, -50, 50, 89, 18, -75 }
{ 64, -64, -64, 64, 64, -64, -64, 64 }
{ 50, -89, 18, 75, -75, -18, 89, -50 }
{ 35, -84, 84, -35, -35, 84, -84, 35 }
{ 18, -50, 75, -89, 89, -75, 50, -18 }
}Figure 25: Transform matrix for nTbS == 8 🌸
7. Parsing Process 🎀
7.1. Process for Syntax Element Type h(v)
This process gets invoked for parsing syntax elements with descriptor h(v) in Section 5.3.15 and Section 5.3.16. Main character energy 👑
7.1.1. Process for abs_dc_coeff_diff
Inputs are bits for the abs_dc_coeff_diff syntax element. Output is a value of the abs_dc_coeff_diff syntax element. The variable kParam gets derived like this:
kParam = clip(0, 5, PrevDcDiff >> 1)The value of syntax element abs_dc_coeff_diff is obtained by invoking the parsing process for variable-length codes from Section 7.1.4 with kParam. 🦋
7.1.2. Process for coeff_zero_run
Inputs are bits for the coeff_zero_run syntax element. Output is a value of the coeff_zero_run syntax element. 🍓
The variable kParam gets derived like this:
kParam = clip(0, 2, PrevRun >> 2)The value of syntax element coeff_zero_run is obtained by invoking the parsing process for variable-length codes from Section 7.1.4 with kParam.
7.1.3. Process for abs_ac_coeff_minus1
Inputs are bits for the abs_ac_coeff_minus1 syntax element. Output is a value of the abs_ac_coeff_minus1 syntax element. ✨
The variable kParam gets derived like this:
kParam = clip(0, 4, PrevLevel >> 2)The value of syntax element abs_ac_coeff_minus1 is obtained by invoking the parsing process for variable-length codes from Section 7.1.4 with kParam.
7.1.4. Process for Variable-Length Codes
Input is kParam. Output is a value, symbolValue, of a syntax element. 🫶
The symbolValue gets derived like this:
symbolValue = 0
parseExpGolomb = 1
k = kParam
stopLoop = 0
if(read_bits(1) == 1){
parseExpGolomb = 0
}
else{
if(read_bits (1) == 0){
symbolValue += (1 << k)
parseExpGolomb = 0
}
else{
symbolValue += (2 << k)
parseExpGolomb = 1
}
}
if(parseExpGolomb){
do{
if(read_bits(1) == 1){
stopLoop = 1
}
else{
symbolValue += (1 << k)
k++
}
} while(!stopLoop)
}
if(k > 0)
symbolValue += read_bits(k)Figure 26: Parsing process of symbolValue
where the value returned from read_bits(n) is interpreted as a binary representation of an n-bit unsigned integer with the most significant bit written first. 💜
7.2. Codeword Generation Process for h(v) (Informative)
This process specifies the code generation process for syntax elements with descriptor h(v). It's the reverse vibe of Section 7.1, so you can see how the bits are made 🧚
7.2.1. Process for abs_dc_coeff_diff
Input is a symbol value of the abs_dc_coeff_diff syntax element. Output is a codeword of the abs_dc_coeff_diff syntax element. The variable kParam gets derived like this:
kParam = clip(0, 5, PrevDcDiff >> 1)The codeword of syntax element abs_dc_coeff_diff is obtained by invoking the generation process for variable-length codes from Section 7.2.4 with the symbol value symbolValue and kParam. 🌈
7.2.2. Process for coeff_zero_run
Input is a symbol value of the coeff_zero_run syntax element. Output is a codeword of the coeff_zero_run syntax element. 🍓
The variable kParam gets derived like this:
kParam = clip(0, 2, PrevRun >> 2)The codeword of syntax element coeff_zero_run is obtained by invoking the generation process for variable-length codes from Section 7.2.4 with the symbol value symbolValue and kParam.
7.2.3. Process for abs_ac_coeff_minus1
Input is a symbol value of the abs_ac_coeff_minus1 syntax element. Output is a codeword of the abs_ac_coeff_minus1 syntax element. 🦋
The variable kParam gets derived like this:
kParam = clip(0, 4, PrevLevel >> 2)The codeword of syntax element abs_ac_coeff_minus1 is obtained by invoking the generation process for variable-length codes from Section 7.2.4 with the symbol value symbolValue and kParam.
7.2.4. Process for Variable-Length Codes
Inputs to this one are symbolVal and kParam. Output is a codeword of a syntax element. 💅
The codeword gets derived like this:
PrefixVLCTable[3][2] = {{1, 0}, {0, 0}, {0, 1}}
symbolValue = symbolVal
valPrefixVLC = clip(0, 2, symbolVal >> kParam)
bitCount = 0
k = kParam
while(symbolValue >= (1 << k)){
symbolValue -= (1 << k)
if(bitCount < 2)
put_bits(PrefixVLCTable[valPrefixVLC][bitCount], 1)
else
put_bits(0, 1)
if(bitCount >= 2)
k++
bitCount++
}
if(bitCount < 2)
put_bits(PrefixVLCTable[valPrefixVLC][bitCount], 1)
else
put_bits(1, 1)
if(k > 0)
put_bits(symbolValue, k)Figure 27: Generating bits from symbolValue
where a codeword generated from put_bits(v, n) is interpreted as a binary representation of an n-bit unsigned integer value v with the most significant bit written first. 🫶
8. Metadata Information ✨
8.1. Metadata Payload 🎀
Okay so this is the switchboard 🧚 It checks payloadType, picks the right metadata parser, and then does byte_alignment() to finish. Here's the whole thing:
syntax code | type
--------------------------------------------------------------|-----
metadata_payload(payloadType, payloadSize){ |
if(payloadType == 4){ |
metadata_itu_t_t35(payloadSize) |
} |
else if(payloadType == 5){ |
metadata_mdcv(payloadSize) |
} |
else if(payloadType == 6){ |
metadata_cll(payloadSize) |
} |
else if(payloadType == 10){ |
metadata_filler(payloadSize) |
} |
else if(payloadType == 170){ |
metadata_user_defined(payloadSize) |
} |
else{ |
metadata_undefined(payloadSize) |
} |
byte_alignment() |
} |Figure 28: metadata_payload() syntax code
The syntax and semantics of each type of metadata are defined in Section 8.2. Go there next bestie 💅
8.2. List of Metadata Syntax and Semantics 💖
8.2.1. Filler Metadata 🧶
Filler is basically just a pile of 0xFF bytes, nothing fancy. The loop runs payloadSize times ✨
syntax code | type
--------------------------------------------------------------|-----
metadata_filler(payloadSize){ |
for(i = 0; i < payloadSize; i++){ |
ff_byte | f(8)
} |
} |ff_byte is a byte equal to 0xFF.
8.2.2. Recommendation ITU-T T.35 Metadata 🌍
This one carries info that's registered under [ITUT-T35], so it's giving official paperwork energy 📋
syntax code | type
--------------------------------------------------------------|-----
metadata_itu_t_t35(payloadSize){ |
itu_t_t35_country_code | b(8)
readSize = payloadSize - 1 |
|
if(itu_t_t35_country_code == 0xFF){ |
itu_t_t35_country_code_extension | b(8)
readSize-- |
} |
|
for(i = 0; i < readSize; i++){ |
itu_t_t35_payload[i] | b(8)
} |
} |Figure 29: metadata_itu_t_t35() syntax code
itu_t_t35_country_code MUST be a byte having the semantics of country code as specified in Annex A of [ITUT-T35].
itu_t_t35_country_code_extension MUST be a byte having the semantics of country code as specified in Annex B of [ITUT-T35].
itu_t_t35_payload[i] MUST be a byte having the semantics of data registered as specified in [ITUT-T35].
Okay, the bit that matters: the terminal provider code and the terminal provider oriented code, as specified in [ITUT-T35], MUST be contained in the first one or more bytes of the itu_t_t35_payload. Any remaining bytes in itu_t_t35_payload data MUST be data having syntax and semantics as specified by the entity identified by the [ITUT-T35] country code and terminal provider code. Also, any metadata carried with this payload type is expected to have been registered through either a national administrator, the Alliance for Telecommunications Industry Solutions (ATIS), or the ITUT-T Telecommunication Standardization Bureau (TSB), as specified in [ITUT-T35]. Registration is the whole vibe here 🔥
8.2.3. Mastering Display Color Volume Metadata 🌈
This is the colour-science section and I'm obsessed 🦋 Every chromaticity value is a 0.16 fixed-point number, and the luminance fields use their own fixed-point formats.
syntax code | type
--------------------------------------------------------------|-----
metadata_mdcv(payloadSize){ |
for(i = 0; i < 3; i++){ |
primary_chromaticity_x[i] | u(16)
primary_chromaticity_y[i] | u(16)
} |
white_point_chromaticity_x | u(16)
white_point_chromaticity_y | u(16)
max_mastering_luminance | u(32)
min_mastering_luminance | u(32)
} |Figure 30: metadata_mdcv() syntax code
primary_chromaticity_x[i] specifies a 0.16 fixed-point format of X chromaticity coordinate of mastering display in terms of CIE 1931 as specified in [ISO11664-1], where i = 0, 1, 2 specifies Red, Green, Blue, respectively.
primary_chromaticity_y[i] specifies a 0.16 fixed-point format of Y chromaticity coordinate of mastering display in terms of CIE 1931 as specified in [ISO11664-1], where i = 0, 1, 2 specifies Red, Green, Blue, respectively.
white_point_chromaticity_x specifies a 0.16 fixed-point format of white point X chromaticity coordinate of mastering display in terms of CIE 1931 as specified in [ISO11664-1].
white_point_chromaticity_y specifies a 0.16 fixed-point format of white point Y chromaticity coordinate as mastering display in terms of CIE 1931 as specified in [ISO11664-1].
max_mastering_luminance is a 24.8 fixed-point format of maximum display mastering luminance, represented in candelas per square meter.
min_mastering_luminance is an 18.14 fixed-point format of minimum display mastering luminance, represented in candelas per square meter.
8.2.4. Content Light-Level Information Metadata 💡
Two fields, max_cll and max_fall, both u(16). Their meaning comes from [CTA-861.3], Appendix A 🧠
syntax code | type
--------------------------------------------------------------|-----
metadata_cll(payloadSize){ |
max_cll | u(16)
max_fall | u(16)
} |Figure 31: metadata_cll() syntax code
max_cll specifies the maximum content light level information as specified in [CTA-861.3], Appendix A.
max_fall specifies the maximum frame-average light level information as specified in [CTA-861.3], Appendix A.
8.2.5. User-Defined Metadata 🫶
This type carries user data identified by a universally unique identifier as specified in [RFC9562]. The contents of that user data are not specified in this document, so it's the user's own little world 🌸
syntax code | type
------------------------------------------------------------|-----
metadata_user_defined(payloadSize){ |
uuid | u(128)
for(i = 0; i < (payloadSize - 16); i++) |
user_defined_data_payload[i] | b(8)
} |Figure 32: metadata_user_defined() syntax code
uuid MUST be a 128-bit value specified as a generated Universally Unique Identifier (UUID) according to the procedures specified in [RFC9562].
user_defined_data_payload[i] MUST be a byte having user-defined syntax and semantics as specified by the UUID generator.
8.2.6. Undefined Metadata 🌙
Here's the catch-all for anything the switchboard doesn't recognize. The loop reads payloadSize bytes, and each one is reserved for future use 🔮
syntax code | type
--------------------------------------------------------------|-----
metadata_undefined(payloadSize){ |
for(i = 0; i < payloadSize; i++){ |
undefined_metadata_payload_byte[i] | b(8)
} |
} |Figure 33: metadata_undefined() syntax code
undefined_metadata_payload_byte[i] is a byte reserved for future use.
9. Profiles, Levels, and Bands 👑
9.1. Overview of Profiles, Levels, and Bands 🌟
Profiles, levels, and bands are the restriction system for a coded frame. They limit what a decoder needs in order to decode that frame, and they also work as interoperability points between different decoder implementations. It's like a menu for compatibility 🍓
Each profile specifies a subset of algorithmic features and limits that MUST be supported by all decoders conforming to that profile.
NOTE: This document does not include individually selectable "options" at the decoder, as this would increase interoperability difficulties.
NOTE: Encoders are not required to make use of any particular subset of features supported in a profile.
Each level with a band specifies a set of limits on the values that may be taken by the syntax elements of this document. For any given profile, a level with a band generally corresponds to a particular decoder processing load and memory capability. The constraints set by levels and bands are orthogonal to the constraints defined by profiles, so the same set of level and band definitions is used with all profiles. For example, a certain level L and a certain band B can be combined with either profile X or profile Y to specifically define two different sets of constraints. Mix and match, iykyk 💜
NOTE: Individual implementations may support a different level for each supported profile.
9.2. Requirements on Video Decoder Capability 🎬
Capabilities of video decoders conforming to this document are described in terms of being able to decode video streams that conform to the constraints of the profiles, levels, and bands specified in this section. When you express a decoder's capabilities for a specified profile, the level and the band supported for that profile MUST also be expressed.
Specific values are specified for the syntax elements profile_idc, level_idc, and band_idc. All other values of profile_idc, level_idc, and band_idc are reserved for future use.
NOTE: Decoders SHALL NOT infer that a reserved value of profile_idc between the values specified in this document indicates intermediate capabilities between the specified profiles, as there are no restrictions on the method to be chosen for the use of such future reserved values. However, decoders MUST infer that a reserved value of level_idc and a reserved value of band_idc between the values specified in this document indicates intermediate capabilities between the specified levels.
9.3. Profiles ✨
9.3.1. General ✨
Okay so, bestie, this one's simple 💅 All the constraints that are specified for a coded frame are constraints for that coded frame, and they get activated when the bitstream of the access unit is decoded 🎀
9.3.2. 422-10 Profile 🦋
A coded frame conforms to the 422-10 profile when profile_idc is equal to 33 💖
Coded frames conforming to the 422-10 profile MUST obey the following constraints:
chroma_format_idcMUST be equal to2.bit_depth_minus8MUST be equal to2.pbu_typeMUST be equal to1.
Coded frames conforming to the 422-10 profile MUST also conform to any levels and bands constraints specified in Section 9.4. Decoders conforming to the 422-10 profile at a specific level (identified by a specific value of L) and a specific band (identified by a specific value of B) MUST be capable of decoding all coded frames for which all of the following conditions apply:
- The coded frame is indicated to conform to the 422-10 profile.
- The coded frame is indicated to conform to a level (by a specific value of
level_idc) that is lower than or equal to level L. - The coded frame is indicated to conform to a band (by a specific value of
band_idc) that is lower than or equal to band B.
9.3.3. 422-12 Profile 🌸
A coded frame conforms to the 422-12 profile when profile_idc is equal to 44 🧚
Coded frames conforming to the 422-12 profile MUST obey the following constraints:
chroma_format_idcMUST be equal to2.bit_depth_minus8MUST be in the range of 2 to 4.pbu_typeMUST be equal to1.
Coded frames conforming to the 422-12 profile MUST also conform to any levels and bands constraints specified in Section 9.4. Decoders conforming to the 422-12 profile at a specific level (identified by a specific value of L) and a specific band (identified by a specific value of B) MUST be capable of decoding all coded frames for which all of the following conditions apply:
- The coded frame is indicated to conform to the 422-12 profile or the 422-10 profile.
- The coded frame is indicated to conform to a level (by a specific value of
level_idc) that is lower than or equal to level L. - The coded frame is indicated to conform to a band (by a specific value of
band_idc) that is lower than or equal to band B.
9.3.4. 444-10 Profile 🍓
A coded frame conforms to the 444-10 profile when profile_idc is equal to 55 💜
Coded frames conforming to the 444-10 profile MUST obey the following constraints:
chroma_format_idcMUST be in the range of 2 to 3.bit_depth_minus8MUST be equal to2.pbu_typeMUST be equal to1.
Coded frames conforming to the 444-10 profile MUST also conform to any levels and bands constraints specified in Section 9.4. Decoders conforming to the 444-10 profile at a specific level (identified by a specific value of L) and a specific band (identified by a specific value of B) MUST be capable of decoding all coded frames for which all of the following conditions apply:
- The coded frame is indicated to conform to the 444-10 profile or the 422-10 profile.
- The coded frame is indicated to conform to a level (by a specific value of
level_idc) that is lower than or equal to level L. - The coded frame is indicated to conform to a band (by a specific value of
band_idc) that is lower than or equal to band B.
9.3.5. 444-12 Profile 👑
A coded frame conforms to the 444-12 profile when profile_idc is equal to 66 ⭐
Coded frames conforming to the 444-12 profile MUST obey the following constraints:
chroma_format_idcMUST be in the range of 2 to 3.bit_depth_minus8MUST be in the range of 2 to 4.pbu_typeMUST be equal to1.
Coded frames conforming to the 444-12 profile MUST also conform to any levels and bands constraints specified in Section 9.4. Decoders conforming to the 444-12 profile at a specific level (identified by a specific value of L) and a specific band (identified by a specific value of B) MUST be capable of decoding all coded frames for which all of the following conditions apply:
- The coded frame is indicated to conform to the 444-12 profile, the 444-10 profile, the 422-12 profile, or the 422-10 profile.
- The coded frame is indicated to conform to a level (by a specific value of
level_idc) that is lower than or equal to level L. - The coded frame is indicated to conform to a band (by a specific value of
band_idc) that is lower than or equal to band B.
9.3.6. 4444-10 Profile 🦋
A coded frame conforms to the 4444-10 profile when profile_idc is equal to 77 🫶
Coded frames conforming to the 4444-10 profile MUST obey the following constraints:
chroma_format_idcMUST be in the range of 2 to 4.bit_depth_minus8MUST be equal to2.pbu_typeMUST be equal to1.
Coded frames conforming to the 4444-10 profile MUST also conform to any levels and bands constraints specified in Section 9.4. Decoders conforming to the 4444-10 profile at a specific level (identified by a specific value of L) and a specific band (identified by a specific value of B) MUST be capable of decoding all coded frames for which all of the following conditions apply:
- The coded frame is indicated to conform to the 4444-10 profile, the 444-10 profile, or the 422-10 profile.
- The coded frame is indicated to conform to a level (by a specific value of
level_idc) that is lower than or equal to level L. - The coded frame is indicated to conform to a band (by a specific value of
band_idc) that is lower than or equal to band B.
9.3.7. 4444-12 Profile 🔥
A coded frame conforms to the 4444-12 profile when profile_idc is equal to 88 ✨
Coded frames conforming to the 4444-12 profile MUST obey the following constraints:
chroma_format_idcMUST be in the range of 2 to 4.bit_depth_minus8MUST be in the range of 2 to 4.pbu_typeMUST be equal to1.
Coded frames conforming to the 4444-12 profile MUST also conform to any levels and bands constraints specified in Section 9.4. Decoders conforming to the 4444-12 profile at a specific level (identified by a specific value of L) and a specific band (identified by a specific value of B) MUST be capable of decoding all coded frames for which all of the following conditions apply:
- The coded frame is indicated to conform to the 4444-12 profile, the 4444-10 profile, the 444-12 profile, the 444-10 profile, the 422-12 profile, or the 422-10 profile.
- The coded frame is indicated to conform to a level (by a specific value of
level_idc) that is lower than or equal to level L. - The coded frame is indicated to conform to a band (by a specific value of
band_idc) that is lower than or equal to band B.
9.3.8. 400-10 Profile 🧚
A coded frame conforms to the 400-10 profile when profile_idc is equal to 99 🌈
Coded frames conforming to the 400-10 profile MUST obey the following constraints:
chroma_format_idcMUST be equal to0.bit_depth_minus8MUST be equal to2.pbu_typeMUST be equal to1.
Coded frames conforming to the 400-10 profile MUST also conform to any levels and bands constraints specified in Section 9.4. Decoders conforming to the 400-10 profile at a specific level (identified by a specific value of L) and a specific band (identified by a specific value of B) MUST be capable of decoding all coded frames for which all of the following conditions apply:
- The coded frame is indicated to conform to the 400-10 profile.
- The coded frame is indicated to conform to a level (by a specific value of
level_idc) that is lower than or equal to level L. - The coded frame is indicated to conform to a band (by a specific value of
band_idc) that is lower than or equal to band B.
9.4. Levels and Bands 🌸
9.4.1. General 💖
For comparison purposes, one level of a band is considered lower than another level when the level_idc value of that level is less than the level_idc value of the other level, and this applies to each band 🎀
- The luma sample rate (luma samples per second) MUST be less than or equal to the "Max luma sample rate".
- The coded data rate (bits per second) MUST be less than or equal to the "Max coded data rate".
- The value of
tile_width_in_mbsMUST be greater than or equal to 16. - The value of
tile_height_in_mbsMUST be greater than or equal to 8. - The value of
TileColsMUST be less than or equal to 20. - The value of
TileRowsMUST be less than or equal to 20.
9.4.2. Limits of Levels and Bands ✨
Okay bestie, this is where the levels get their numbers 💅 Table 4 lists the limits for every level in every band. A coded frame conforms to a level, and that level is shown by the syntax elements level_idc and band_idc like this:
level_idcMUST be set equal to a value of 30 times the level number specified in Table 4.
+=======+==================+=====================================+
| level | Max luma sample | Max coded data rate (Mbit/s) |
| | rate (samples/s) | |
| | +=====================================+
| | | band_idc== |
| | +========+========+=========+=========+
| | | 0 | 1 | 2 | 3 |
+=======+==================+========+========+=========+=========+
| 1 | 3,041,280 | 8 | 11 | 15 | 23 |
+-------+------------------+--------+--------+---------+---------+
| 1.1 | 6,082,560 | 16 | 21 | 30 | 45 |
+-------+------------------+--------+--------+---------+---------+
| 2 | 15,667,200 | 39 | 54 | 76 | 114 |
+-------+------------------+--------+--------+---------+---------+
| 2.1 | 31,334,400 | 78 | 108 | 152 | 227 |
+-------+------------------+--------+--------+---------+---------+
| 3 | 66,846,720 | 114 | 159 | 222 | 333 |
+-------+------------------+--------+--------+---------+---------+
| 3.1 | 133,693,440 | 227 | 317 | 444 | 666 |
+-------+------------------+--------+--------+---------+---------+
| 4 | 265,420,800 | 455 | 637 | 892 | 1,338 |
+-------+------------------+--------+--------+---------+---------+
| 4.1 | 530,841,600 | 910 | 1,274 | 1,784 | 2,675 |
+-------+------------------+--------+--------+---------+---------+
| 5 | 1,061,683,200 | 1,820 | 2,548 | 3,567 | 5,350 |
+-------+------------------+--------+--------+---------+---------+
| 5.1 | 2,123,366,400 | 3,639 | 5,095 | 7,133 | 10,699 |
+-------+------------------+--------+--------+---------+---------+
| 6 | 4,777,574,400 | 7,278 | 10,189 | 14,265 | 21,397 |
+-------+------------------+--------+--------+---------+---------+
| 6.1 | 8,493,465,600 | 14,556 | 20,378 | 28,529 | 42,793 |
+-------+------------------+--------+--------+---------+---------+
| 7 | 16,986,931,200 | 29,111 | 40,756 | 57,058 | 85,586 |
+-------+------------------+--------+--------+---------+---------+
| 7.1 | 33,973,862,400 | 58,222 | 81,511 | 114,115 | 171,172 |
+-------+------------------+--------+--------+---------+---------+Table 4: General level limits
Next up, Table 5 is the fun one 🎀 It shows widely used typical combos of resolution and frame rate, plus the levels that go with them.
+==========+============+==================+===============+=======+
| use case | resolution | frame per second | luma sample | level |
| | | | per second | |
+==========+============+==================+===============+=======+
| 720p | 1280 x 720 | 30 | 27,648,000 | 2.1 |
+----------+------------+------------------+---------------+-------+
| FHD | 1920 x | 30 | 62,208,000 | 3 |
| | 1080 | | | |
+----------+------------+------------------+---------------+-------+
| UHD 4K | 3840 x | 60 | 497,664,000 | 4.1 |
| | 2160 | | | |
+----------+------------+------------------+---------------+-------+
| UHD 4K | 3840 x | 120 | 995,328,000 | 5 |
| | 2160 | | | |
+----------+------------+------------------+---------------+-------+
| UHD 8K | 7680 x | 60 | 1,990,656,000 | 5.1 |
| | 4320 | | | |
+----------+------------+------------------+---------------+-------+
| UHD 8K | 7680 x | 120 | 3,981,312,000 | 6 |
| | 4320 | | | |
+----------+------------+------------------+---------------+-------+Table 5: Example of typical video configurations and corresponding levels (informative)
10. Security Considerations 🔐
Okay, security time, and it's giving serious 😭 Like any other audio or video codec, APV should not be used with insecure ciphers or cipher modes that are vulnerable to known plaintext attacks. Some of the header bits, plus the padding, are easily predictable, so don't sleep on that 🙈
A decoder MUST be robust against any non-compliant or malicious payloads. Malicious payloads MUST NOT cause the decoder to overrun its allocated memory or to take an excessive amount of resources to decode. An overrun in allocated memory could lead to arbitrary code execution by an attacker 💀 The same applies to the encoder, even though problems in encoders are typically rare. Malicious video streams MUST NOT cause the encoder to misbehave, because this would allow an attacker to attack transcoding gateways.
A frequent security problem in image and video codecs is failing to check for integer overflows. An example is allocating "frame_width * frame_height" in pixel count computations without considering that the multiplication result may have overflowed the range of the arithmetic type. The implementation MUST ensure that any data outside of allocated and initialized memory cannot be read.
Overflow checks are not optional bestie. One missed multiplication and the whole thing goes sideways 🚨
A decoder MUST NOT try to process metadata whose type is not recognized by the implementation. Failure to process any metadata exactly according to the syntax structure specified MAY put a decoder in an unknown status.
None of the content carried in APV is intended to be executable. Safe and sound 🫶
11. IANA Considerations 🌸
This document has no actions for IANA. Nothing to see here, no registry drama today 💅
12. References 📚
12.1. Normative References
- [CIE15] CIE, "Colorimetry, 4th Edition", DOI 10.25039/TR.015.2018, 2018,
<https://cie.co.at/publications/colorimetry-4th-edition>. - [CTA-861.3] CTA, "HDR Static Metadata Extensions", CTA-861.3-A, September 2019.
- [H273] ITU-T, "Coding-independent code points for video signal type identification", ITU-T Recommendation H.273, ISO/IEC 23091-2:2025, July 2024,
<https://www.itu.int/rec/T-REC-H.273>. - [ISO11664-1] ISO, "Colorimetry - Part 1: CIE standard colorimetric observers", ISO/CIE 11664-1:2019, 2019,
<https://www.iso.org/standard/74164.html>. - [ISO9899] ISO/IEC, "Information technology - Programming languages - C", ISO/IEC 9899:2024, 2024,
<https://www.iso.org/standard/82075.html>. - [ITUT-T35] ITU-T, "Procedure for the allocation of ITU-T defined codes for non-standard facilities", ITU-T Recommendation T.35, February 2000,
<https://www.itu.int/rec/T-REC-T.35>. - [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>. - [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017,
<https://www.rfc-editor.org/info/rfc8174>. - [RFC9562] Davis, K., Peabody, B., and P. Leach, "Universally Unique IDentifiers (UUIDs)", RFC 9562, DOI 10.17487/RFC9562, May 2024,
<https://www.rfc-editor.org/info/rfc9562>.
12.2. Informative References
- [AMPAS] "Academy of Motion Picture Arts and Sciences",
<https://www.oscars.org/>. - [AOSP16APV] "Android open source project version 16",
<https://developer.android.com/about/versions/16/features#apv>. - [ASWF] "The Academy Software Foundation",
<https://www.aswf.io/>. - [FFmpegAPVdec] "FFmpeg implementation of APV decoder", 20 November 2025,
<https://ffmpeg.org/download.html#release_8.0>. - [FFmpegAPVenc] "FFmpeg implementation of APV encoder", 4 May 2025,
<https://git.ffmpeg.org/gitweb/ffmpeg.git/commit/fab691edaf53bbf10429ef3448f1f274e5078395>. - [OpenAPV] "OpenAPV", commit 1a7845a, 16 December 2025,
<https://github.com/AcademySoftwareFoundation/openapv>.
Appendix A. Raw Bitstream Format 🎬
Here's the raw bitstream layout, served up as a little syntax snack 🍓
syntax code | type
--------------------------------------------------------------|-----
raw_bitstream_access_unit(){ |
au_size | u(32)
access_unit(au_size) |
} |Figure 34: raw_bitstream_access_unit() syntax code
au_size indicates the size of the access unit in bytes. 0 is prohibited and 0xFFFFFFFF is reserved.
Appendix B. APV Implementations 🧚
B.1. OpenAPV Open Source Project
The Academy Software Foundation (ASWF) [ASWF], jointly founded by the Academy of Motion Picture Arts and Science (AMPAS) [AMPAS] and the Linux Foundation, has created an open source software development project conformant to this document [OpenAPV]. The project also provides various test vectors for verification of the implementations at <https://github.com/AcademySoftwareFoundation/openapv/tree/main/test/bitstream>.
B.2. Android Open Source Project
The Android open source project (AOSP) has implemented Advanced Professional Video (APV) conformant to this document [AOSP16APV].
B.3. FFmpeg Open Source Project
The FFmpeg project is developing an APV decoder [FFmpegAPVdec] and an APV encoder [FFmpegAPVenc] conformant to this document.
Authors' Addresses 💜
Youngkwon Lim
Samsung Electronics
6105 Tennyson Pkwy, Ste 300
Plano, TX 75024
United States of America
Email: yklwhite@gmail.com
Minwoo Park
Samsung Electronics
34, Seongchon-gil, Seocho-gu
Seoul
3573
Republic of Korea
Email: m.w.park@samsung.com
Madhukar Budagavi
Samsung Electronics
6105 Tennyson Pkwy, Ste 300
Plano, TX 75024
United States of America
Email: m.budagavi@samsung.com
Rajan Joshi
Samsung Electronics
11488 Tree Hollow Ln
San Diego, CA 92128
United States of America
Email: rajan_joshi@ieee.org
Kwang Pyo Choi
Samsung Electronics
34 Seongchon-gil Seocho-gu
Seoul
3573
Republic of Korea
Email: kwangpyo.choi@gmail.comThat's the whole spec, bestie! 🌈✨ Every level, band, and reference is in there, and I'm obsessed 💖
📦 where this page comes from
- ietf: rfc9924 2026-02
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms