RFC 9828

RTP Payload Format for JPEG 2000 Streaming with Sub-Codestream Latency

📅 published August 2025, from ietf

RTP Payload Format for JPEG 2000 Streaming with Sub-Codestream Latency 🎀

Okay bestie, let's get the paperwork out of the way first 📋✨ This is RFC 9828, published in August 2025 by the IETF (Internet Engineering Task Force). It's on the Internet Standards Track, so it's an actual standard and not just a suggestion 💅.

The lead author is P.-A. Lemieux, who is the editor (that's the "Ed." 🧚), from Sandflow Consulting LLC. The other author is D. Taubman from the University of New South Wales 🌸. The ISSN is 2070-1721. The header doesn't list anything this RFC obsoletes or updates, so there's no drama there 🙈.

Abstract ✨

So the big idea is this: it defines how to send a video signal over RTP when that video is encoded as a sequence of JPEG 2000 codestreams 🎞️. The lowkey iconic part is sub-codestream latency. The first RTP packet for an image can go out before the whole image is even available to, or encoded by, the sender 🚀.

Status of This Memo 📜

Okay so this is an Internet Standards Track document 🙌. It's a product of the IETF, which means it represents the consensus of the IETF community. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). If you want more on Internet Standards, check Section 2 of RFC 7841 🌈.

For the current status, any errata, and how to send feedback, head to https://www.rfc-editor.org/info/rfc9828 💌.

Copyright (c) 2025 IETF Trust and the persons identified as the document authors. All rights reserved 🦋.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they stand on the publication date. Please read these carefully, because they spell out your rights and restrictions for this document 🫶. Any Code Components pulled out of this document have to include the Revised BSD License text, as described in Section 4.e of the Trust Legal Provisions. They also come with no warranty, as described in the Revised BSD License ⚖️.

Table of Contents 🗺️

1.  Introduction
2.  Requirements Language
3.  Media Format Description
4.  Video Signal Description
5.  Payload Format
  5.1.  General
  5.2.  RTP Fixed Header Usage
  5.3.  Main Packet Payload Header
  5.4.  Body Packet Payload Header
6.  JPEG 2000 Codestream
7.  Sender
  7.1.  Main Packet
  7.2.  RTP Packet Filtering
  7.3.  Resync Point
  7.4.  PTSTAMP Field
  7.5.  RES Field
  7.6.  Extra Information
  7.7.  Unassigned Values
  7.8.  Extension Values
  7.9.  Code-Block Caching
8.  Receiver
  8.1.  PTSTAMP
  8.2.  QUAL
  8.3.  RES
  8.4.  Extra Information
  8.5.  Unassigned Values
  8.6.  Extension Values
  8.7.  Code-Block Caching
9.  Media Type
  9.1.  General
  9.2.  Definition
10. Mapping to the Session Description Protocol (SDP)
11. Congestion Control
12. IANA Considerations
13. Security Considerations
14. References
  14.1.  Normative References
  14.2.  Informative References
Appendix A.  Pixel Formats
Appendix B.  Signal Formats
Appendix C.  Sample Formats
Authors' Addresses

1. Introduction ✨

RTP, the Real-time Transport Protocol specified in [RFC3550], handles end-to-end network transport for real-time data 🚚. It doesn't define what the data itself (the payload) looks like. That part changes from app to app and gets defined in companion RTP payload format documents 🎀.

This RTP payload format is all about streaming a video signal that's encoded as a sequence of JPEG 2000 codestreams. (Section 3 has a primer on how JPEG 2000 codestreams are structured, so peek there if you're lost 🧚.) JPEG 2000 is a seriously flexible image codec. It supports resolution and quality scalability, lossy-to-lossless coding, non-iterative optimal rate control, and high dynamic range, multi-channel, and subsampled images 🌈. That's why it's a mainstay in high-performance stuff like medical, geospatial, archival, cinema, studio post-production, and TV production 📺.

The payload format works with a bunch of frame-scanning techniques (progressive, interlaced, and progressive segmented frame) and image characteristics 🎞️. It also supports real-time image transmission, aka live streaming, where the image gets encoded, sent, and decoded continuously as it's being produced, with minimal latency ⚡. Target apps include real-time TV production over IP [OV2110-0], remote presence, surveillance, and so on 👑. Here's the breakdown:

  • The payload format allows sub-codestream latency, so the first RTP packet of a given codestream can go out before the entire codestream is available 🚀. It doesn't lean on the JPEG 2000 PLM (Packet length, main header) and PLT (Packet length, tile-part header) marker segments for recovery after RTP packet loss. Those markers can only be written once the codestream is complete, so they clash with sub-codestream latency 🙅. Instead, the payload format adds payload header fields (ORDH, ORDB, POS, and PID) that say whether an RTP packet holds a resynchronization (resync) point, and how a recipient can restart codestream processing from that resync point 🔁. For contrast, [RFC5371] also specifies an RTP payload format for JPEG 2000, but it relies on codestream structures that can't be emitted until the whole codestream is available 😤.
  • Following [RFC4175], the payload format defines an extended sequence number. It extends the standard (16-bit) sequence number from the RTP Fixed Header by storing extra high-order bits in the payload header (ESEQ field) 🔢. This lets the format handle high data rates without ambiguity, since the standard sequence number rolls over super fast at those rates 🌀. For example, the standard sequence number rolls over in 0.5 seconds on a 1 Gbps video stream with RTP packet sizes of at least 1000 octets. That's a problem for spotting lost and out-of-order RTP packets, especially when the round-trip time is longer than the rollover period (0.5 seconds in this example) 😭.
  • The payload header can optionally carry a temporal offset (PTSTAMP) relative to the first RTP packet that has the same RTP timestamp field value (Section 5.2) ⏱️. PTSTAMP is higher resolution than the timestamp, so receivers can recover the sender's clock more quickly 💜.

On top of sub-codestream latency and high-precision sender clock recovery, the payload format improves on [RFC5371] by supporting:

  • code-block caching for screen content (see Section 7.9) 🖥️;
  • progressive segmented frame (PsF) video support (see Appendix B) 🎬; and
  • explicit colorspace signaling (see Section 5.3) 🌷.

Last but definitely not least, the payload format uses the unique scalability features of JPEG 2000. That means an intermediate system or recipient can discard resolution levels and/or quality layers just by inspecting RTP packet headers (QUAL and RES fields). It doesn't have to parse the underlying codestream to do it (see Section 7.2) 🦋.

2. Requirements Language 📖

Here's the key-word lowdown 💅. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be read as described in BCP 14 [RFC2119] [RFC8174]. That only applies when they show up in all capitals, exactly like this 🫶.

If a figure and the prose ever disagree, the prose wins 👑.

3. Media Format Description ✨

okay so bestie, this part is where we get to know the JPEG 2000 codestream itself 📸🦋 It's specified in full detail in [JPEG2000-1], and this section is the highlight reel.

Notes for you: as Section 6 explains, a JPEG 2000 codestream can use capabilities from any part of the JPEG 2000 family, including [JPEG2000-2] and [JPEG2000-15]. Iconic range 🌈

JPEG 2000 represents an image as one or more components, each uniformly sampled on a common rectangular reference grid. For example, an image can have the classic Y (luma), C_b (blue-difference chroma), and C_r (red-difference chroma) components, where C_b and C_r are subsampled by a factor of two compared to Y. The chroma is lowkey smaller, and nobody notices 🙈

An image can also be split into contiguous rectangular tiles, and each tile is independently coded and decoded. Separate little pieces, zero drama 🧩

JPEG 2000 codes each image as a standalone codestream. A codestream is made of (i) marker segments, which hold coding parameters and metadata, and (ii) coded data. Two parts, very clean 💅

For convenience, here are the abbreviated and full names of the marker segments mentioned in this specification. Several other marker segments are defined by JPEG 2000 and can show up in a codestream too, but these are the ones we care about 🎀

  • CAP: Extended Capabilities
  • COC: Coding style component
  • COD: Coding style default
  • COM: Comment
  • EOC: End of codestream
  • PLM: Packet length, main header
  • PLT: Packet length, tile-part header
  • SOC: Start of codestream
  • SOD: Start of data
  • SOT: Start of tile-part

The codestream starts with an SOC marker segment and ends with an EOC marker segment. The main header is the set of marker segments between SOC and the first SOT, and it holds info that applies to the whole codestream. It is generally impossible to decode a codestream without its main header, so don't skip it 😭

The rest of the codestream is additional marker segments (tile-part headers) interleaved with coded image data. Bestie, it's a whole layered outfit 👑

At the heart of JPEG 2000 coding is the wavelet transform. It breaks the image into successive resolution levels, and each level is related to the next by a spatial factor of two. In other words, each resolution level has half the horizontal and half the vertical resolution of the one before it. Zooming out, one step at a time 🔍✨

The coded image data ends up as code-blocks. Each code-block holds coded samples from a rectangular (spatial) region within one resolution level of one component. Code-blocks get collected into precincts, which represent the code-blocks covering a spatial region within one resolution level of one component. Groupies of groupies, honestly 🫶

The coded image data can be arranged in several progression orders. A progression order decides which aspect of the image shows up first in the codestream (in terms of byte offset). The progression orders are parameterized by:

  • Position (P): The first lines of the image come before the last lines of the image.
  • Component (C): The first component of the image comes before the last component of the image.
  • Resolution Level (R): The information needed to reconstruct the lower spatial resolutions of the image comes before the information needed to reconstruct the higher spatial resolutions.
  • Quality Layer (L): The information needed to reconstruct the most significant bits of each sample comes before the information needed to reconstruct the least significant bit of each sample.

Here's an example to make it click. In the PRCL progression order, the info needed to reconstruct the first lines of the image comes before the info for the last lines. Then, within a group of lines, the info for the lower spatial resolutions comes before the info for the higher ones. This progression order is especially useful for subframe latency operations, so it's giving practical and iconic 💜

4. Video Signal Description 🎬

This RTP payload format supports three distinct techniques for scanning video frames:

  • Progressive frame
  • Interlaced frame, where each frame consists of two fields. Field 1 occurs earlier in time than Field 2. The height in lines of each field is half the height of the image.
  • Progressive segmented frame (PsF), where each frame consists of two segments. Segment 1 contains the odd lines (1, 3, 5, 7, ...) of a frame, and Segment 2 contains the even lines (2, 4, 6, 8, ...) of the same frame. Lines from the top of the frame to the bottom of the frame are numbered sequentially starting at 1. Two halves, one frame, very cute 🧚

All frames are scanned left to right, top to bottom. Reading order, but make it video 📺

5. Payload Format 📦

5.1. General 💖

Here's the packetization picture. It's a whole flow chart and honestly a masterpiece 🖼️

   <--------------- Codestream (image) -------------->
   |                                                 |
   <----- Extended Header ----->                     |
   |                           |                     |
   +-----+-//-+-----+-//-+-----+--------//-----+-----+-----+---------
   | SOC | .. | SOT | .. | SOD | ............. | EOC |  P  | SOC  ...
   +-----+-//-+-----+-//-+-----+--------//-----+-----+-----+---------
   |          |                                            |
   <----------> Main header                                |
   |                                                       |
   +------------------------------+------+--//-+-----------+---------
   |             Main             | Body | ... |    Body   | Main ...
   +------------------------------+------+--//-+-----------+---------
   |                              |
   <--------- RTP Packet --------->

Figure 1: Packetization of a Sequence of JPEG 2000 Codestreams (Not to Scale)

In Figure 1, P denotes (optional) padding bytes. See Section 3 for expansions of SOC, SOD, SOT, and EOC. Bookmark that one 🔖

Each RTP packet, as specified in [RFC3550], is either a Main Packet or a Body Packet. Two flavours, same energy 🍓

A Main Packet is made of the following ordered sequence of structures, concatenated without gaps:

  • the RTP Fixed Header;
  • a Main Packet Payload Header, as specified in Section 5.3; and
  • the payload, which consists of a JPEG 2000 codestream fragment.

A Body Packet is made of the following ordered sequence of structures, concatenated without gaps:

  • the RTP Fixed Header;
  • a Body Packet Payload Header, as specified in Section 5.4; and
  • the payload, which consists of a JPEG 2000 codestream fragment.

When concatenated, the sequence of JPEG 2000 codestream fragments emitted by the sender MUST be a sequence of JPEG 2000 codestreams, where two successive JPEG 2000 codestreams MAY be separated by one or more padding bytes (see Figure 1). Gaps are allowed, but only as padding 🫧

The sender MUST set the value of each padding byte to zero. Zeros only, no funny business 🙅‍♀️

The receiver MUST ignore the values of the padding bytes. So if the bytes are weird, the receiver just vibes right past them 😌

The JPEG 2000 codestreams MUST conform to Section 6. Non-negotiable, bestie 📜

NOTE 1: Padding bytes can be used to achieve constant bit rate transmission. Ooh, that's a nice trick 🎯

A JPEG 2000 codestream fragment, and therefore an RTP packet, does not necessarily contain complete JPEG 2000 packets, as defined in [JPEG2000-1]. Fragments don't have to be whole packets, so no stress ✨

A JPEG 2000 codestream Extended Header consists of the bytes between, and including, the SOC marker and the first SOD marker. This is the big header wrapper 🎁

NOTE 2: The concept of the JPEG 2000 codestream Extended Header is specific to this document and is distinct from the JPEG 2000 codestream main header, which is defined in [JPEG2000-1]. The codestream main header consists of the bytes between, and including, the SOC marker and the first SOT marker. The codestream main header is a subset of the codestream Extended Header (see Figure 1). Two headers, one is inside the other, like a matryoshka doll 🪆

The payload of a Body Packet MUST NOT contain any bytes of the JPEG 2000 codestream Extended Header. Body packets stay out of the header section, period 🚫

The payload of a Main Packet MUST contain at least one byte of the JPEG 2000 codestream Extended Header and MAY contain bytes other than those of the JPEG 2000 codestream Extended Header. Main packets always bring at least one header byte to the party, and they can bring more stuff too 🎉

A payload MUST NOT contain bytes from more than one JPEG 2000 codestream. One codestream per payload, no mixing the drinks 🍸

5.2. RTP Fixed Header Usage 🧾

The following RTP header fields have a specific meaning in the context of this payload format:

marker

  • 1: The payload contains an EOC marker.
  • 0: Otherwise

timestamp

The timestamp is the presentation time of the image to which the payload belongs. Timing is everything, bestie ⏰

The timestamp clock rate is 90 kHz. 🎵

The timestamp of successive progressive frames MUST advance at regular increments based on the instantaneous video frame rate. Steady and consistent, like a good playlist 💿

The timestamp of Field 1 of successive interlaced frames MUST advance at regular increments based on the instantaneous video frame rate, and the Timestamp of Field 2 MUST be offset from the timestamp of Field 1 by one half of the instantaneous frame period. Field 2 shows up exactly halfway between, it's giving precision ⚖️

The timestamp of both segments of a progressive segmented frame MUST be equal. Both segments, same timestamp, twins 👯‍♀️

The timestamp of all RTP packets of a given image MUST be equal. Every packet of one image shares one timestamp. Matching fits, always 💅

sequence number

The low-order bits of the extended sequence number. These are the bits you see right away 👀

The high-order bits of the extended sequence number are contained in the ESEQ field, which is specified in Section 5.3. The big-picture part lives in ESEQ 🌍

The extended sequence number is calculated as follows:

<extended sequence number> = <ESEQ field> * 65536 + <sequence number field of the RTP Fixed Header>

5.3. Main Packet Payload Header ✨

okay so Figure 2 is the blueprint for the payload header 🎀 every field is an unsigned binary integer, read in network order (this part is literally so important bestie 💅)

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |MH | TP  |ORDH |P|XTRAC|        PTSTAMP        |     ESEQ      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |R|S|C| RSVD  |*|    PRIMS      |    TRANS      |      MAT      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              XTRAB                            |
   |                               ...                             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   * RANGE

Figure 2: Structure of the Main Packet Payload Header

MH (Codestream Main Header Presence): 2 bits 🦋

  • 0: The RTP packet is a Body Packet.
  • 1: The RTP packet is a Main Packet, and the codestream has more than one Main Packet. The next RTP packet is a Main Packet.
  • 2: The RTP packet is a Main Packet, and the codestream has more than one Main Packet. The next RTP packet is a Body Packet.
  • 3: The RTP packet is a Main Packet, and the codestream has exactly one Main Packet.

TP (Image Type): 3 bits 🌸

This one tells you the scanning structure of the image the payload belongs to, so it's giving video nerd energy 🤓

  • 0: Progressive frame.
  • 1: Field 1 of an interlaced frame, where the first line of the field is the first line of the frame.
  • 2: Field 2 of an interlaced frame, where the first line of the field is the second line of the frame.
  • 3: Field 1 of an interlaced frame, where the first line of the field is the second line of the frame.
  • 4: Field 2 of an interlaced frame, where the first line of the field is the first line of the frame.
  • 5: Segment 1 of a progressive segmented frame, where the first line of the image is the first line of the frame.
  • 6: Segment 2 of a progressive segmented frame, where the first line of the image is the second line of the frame.
  • 7: Extension value. See Sections 8.6 and 7.8.

ORDH (Progression Order Flag, Main Packet): 3 bits 👑

This one specifies the progression order the codestream uses and whether resync points get signaled. For the details on progression orders, check out Section 3 ✨

  • 0: Resync points are not necessarily signaled. The progression order can vary over the codestream.
  • 1: The progression order is LRCP for the entire codestream. The first resync point is specified in every Body Packet that contains one or more resync points.
  • 2: The progression order is RLCP for the entire codestream. The first resync point is specified in every Body Packet that contains one or more resync points.
  • 3: The progression order is RPCL for the entire codestream. The first resync point is specified in every Body Packet that contains one or more resync points.
  • 4: The progression order is PCRL for the entire codestream. The first resync point is specified in every Body Packet that contains one or more resync points.
  • 5: The progression order is CPRL for the entire codestream. The first resync point is specified in every Body Packet that contains one or more resync points.
  • 6: The progression order is PRCL for the entire codestream. The first resync point is specified in every Body Packet that contains one or more resync points.
  • 7: The progression order can vary over the codestream. The first resync point is specified in every Body Packet that contains one or more resync points.

ORDH MUST be 0 if the codestream consists of more than one tile. 🙈

NOTE 1: Only ORDH = 4 and ORDH = 6 allow sub-codestream latency streaming. 🧚

NOTE 2: Progression order PRCL is defined in [JPEG2000-2]. The other progression orders are specified in [JPEG2000-1].

P (Precision Timestamp Presence): 1 bit ⏰

  • 0: PTSTAMP is not used.
  • 1: PTSTAMP is used.

XTRAC (Extension Payload Length): 3 bits 🍓

Length, in multiples of 4 bytes, of the XTRAB field.

PTSTAMP (Precision Timestamp): 12 bits ✨

PTSTAMP = (timestamp + TOFF) mod 4096, if P = 1 in the Main Packet
of this codestream.

TOFF is the transmission time of this RTP packet, in the timebase of the timestamp clock and relative to the first RTP packet with the same timestamp value. So TOFF = 0 in the first RTP packet with the same timestamp value.

PTSTAMP = 0, if P = 0 in the Main Packet of this codestream.

NOTE 3: As described in Sections 7.4 and 8.1, PTSTAMP is intended to improve clock recovery at the receiver and only applies when the transmission time of two consecutive RTP packets with identical timestamp fields differ by no more than 45 ms (which is 4095/90,000). [RFC5450] addresses the general case when an RTP packet is transmitted at a time other than its nominal transmission time. 🕰️

ESEQ (Extended Sequence Number High-Order Bits): 8 bits 💜

The high-order bits of the extended sequence number.

NOTE 4: The low-order bits of the extended sequence number are stored in the sequence number field of the RTP Fixed Header.

Section 5.2 specifies the formula to compute the extended sequence number. 🧮

R (Codestream Main Header Reuse): 1 bit 🔁

This decides whether Main Packet and codestream main header information can be reused across codestreams.

  • 1: All Main Packets in this stream, as identified by the value of the SSRC field in the RTP Fixed Header:
    • MUST have identical Main Packet Payload Headers, with the exception of their TP, MH, ESEQ, and PTSTAMP fields;
    • MUST contain the same codestream main header information (with the exception of the SOT and COM marker segments and any pointer marker segments); and
    • MUST NOT contain bytes other than Extended Header bytes.
  • 0: Otherwise

S (Parameterized Colorspace Presence): 1 bit 🎨

  • 0: Component colorimetry is not specified; it is left to the session or the application.
    • PRIMS, TRANS, MAT, and RANGE MUST be zero.
  • 1: Component colorimetry is specified by the PRIMS, TRANS, MAT, and RANGE fields.
    • The codestream components MUST conform to one of the combinations in Table 1. 🌈
            +==================+====================+
            | Combination Name | Component Index    |
            |                  +====+=====+=====+===+
            |                  | 0  | 1   | 2   | 3 |
            +==================+====+=====+=====+===+
            | Y                | Y  |     |     |   |
            +==================+----+-----+-----+---+
            | YA               | Y  | A   |     |   |
            +==================+----+-----+-----+---+
            | RGB              | R  | G   | B   |   |
            +==================+----+-----+-----+---+
            | RGBA             | R  | G   | B   | A |
            +==================+----+-----+-----+---+
            | YCbCr            | Y  | C_B | C_R |   |
            +==================+----+-----+-----+---+
            | YCbCrA           | Y  | C_B | C_R | A |
            +==================+----+-----+-----+---+

                  Table 1: Mapping of Codestream
                   Components to Color Channels

Channel A is an opacity channel. The minimum sample value (0) indicates a completely transparent sample, and the maximum sample value (as determined by the bit depth of the codestream component) indicates a completely opaque sample. The opacity channel MUST map to a component with unsigned samples. 🫶

C (Code-Block Caching Usage): 1 bit 🧊

  • 0: Code-block caching is not in use.
  • 1: Code-block caching is in use.
    • R MUST be equal to 1.

RSVD (Reserved): 4 bits 🔒

Unassigned value. See Sections 8.5 and 7.7.

RANGE (Video Full Range Usage): 1 bit 🌗

Value of the VideoFullRangeFlag specified in [REC-ITU-T-H273].

PRIMS (Color Primaries): 8 bits 🎀

One of the ColourPrimaries values specified in [REC-ITU-T-H273].

TRANS (Transfer Characteristics): 8 bits 🍬

One of the TransferCharacteristics values specified in [REC-ITU-T-H273].

MAT (Color Matrix Coefficients): 8 bits 🧡

One of the MatrixCoefficients values specified in [REC-ITU-T-H273].

XTRAB (Extension Payload): variable 🧩

This allows the contents of the Main Packet Payload Header to be extended in the future. The size of the field is determined by Section 5.3. See also Sections 8.4 and 7.6.

5.4. Body Packet Payload Header ✨

Figure 3 shows the structure of the Body Packet Payload Header, and once again every field is an unsigned binary integer in network order 💖

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |MH | TP  |RES  |*|QUAL |       PTSTAMP         |     ESEQ      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |         POS           |                  PID                  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   * ORDB

Figure 3: Structure of the Body Packet Payload Header

MH: See Section 5.3. 🔗

TP: See Section 5.3. 🔗

RES (Resolution Levels): 3 bits 📐

  • 0: The payload can contribute to all resolution levels.
  • Otherwise: The payload contains at least one byte of one JPEG 2000 packet belonging to resolution level (N_L + RES - 7) but does not contain any byte of any JPEG 2000 packet belonging to lower resolution levels. N_L is the number of decomposition levels of the codestream.

ORDB (Progression Order Flag, Body Packet): 1 bit 🌟

  • 0: No resync point is specified for the payload.
  • 1: The payload contains a resync point.

ORDB MUST be 0 if the codestream consists of more than one tile. 💅

QUAL (Quality Layers): 3 bits 🎚️

  • 0: The payload can contribute to all quality layers.
  • Otherwise: The payload contributes only to quality layer index QUAL or above.

PTSTAMP: See Section 5.3. ⏱️

ESEQ: See Section 5.3. 💫

POS (Resync Point Offset): 12 bits 📍

Byte offset from the start of the payload to the first byte of the resync point belonging to the precinct identified by PID.

POS MUST be 0 if ORDB = 0. 🙃

PID (Precinct Identifier): 20 bits 🍭

Unique identifier of the precinct of the resync point.

PID = c + s * num_components

where:

  • c: Index (starting from 0) of the image component to which the precinct belongs.
  • s: Sequence number that identifies the precinct within its tile-component.
  • num_components: Number of components of the codestream.

If PID is present, the payload MUST NOT contain codestream bytes from more than one precinct. 🚫

PID MUST be 0 if ORDB = 0. 🌸

NOTE: PID is identical to precinct identifier I specified in [JPEG2000-9].

6. JPEG 2000 Codestream ✨

okay so a JPEG 2000 codestream is everything from the SOC marker to the EOC marker, including both of those markers, as defined in [JPEG2000-1]. 💖

the codestream MAY also carry capabilities beyond what [JPEG2000-1] covers, like the stuff in [JPEG2000-2] and [JPEG2000-15]. ✨

NOTE 1: the Rsiz parameter and CAP marker segments in every codestream hold the juicy details about what a decoder needs to be able to do. 🔍

NOTE 2: the caps media type parameter in Section 9.2 lets apps say which device capabilities they need. 📱

NOTE 3: the block coder in [JPEG2000-15] is faster and has less latency than the original arithmetic block coder in [JPEG2000-1]. 🚀

for interlaced or progressive segmented frames, the height in the JPEG 2000 main header MUST be the height in lines of the field or the segment, respectively. 🎀

if any decomposition level is horizontal-only, then no decomposition level MUST be vertical-only. and the other way around: if any level is vertical-only, then no level MUST be horizontal-only. it's giving balance 🫶

7. Sender 💅

7.1. Main Packet

a Main Packet MAY hold zero or more bytes of the JPEG 2000 codestream Extended Header. 🌸

the sender MUST emit either a single Main Packet with MH = 3, or one or more Main Packets with MH = 1 followed by a single Main Packet with MH = 2. no other combo, bestie 🙅‍♀️

the Main Packet Payload Headers fields MUST be identical in all Main Packets of a given codestream, except for:

  • MH;
  • ESEQ; and
  • PTSTAMP.

7.2. RTP Packet Filtering

an intermediate system MAY strip out RTP packets from a codestream that a particular client doesn't care about, e.g., based on a resolution or a spatial region of interest. ✂️🦋

7.3. Resync Point

a resync point is the first byte of JPEG 2000 packet header data for a precinct, and it has to satisfy PID < 2^24. 🧚

NOTE 1: resync points can't be specified if the codestream has more than one tile (ORDB and ORDH are both equal to zero). 🙈

NOTE 2: a receiver can use a resync point to process a codestream even if earlier RTP packets were corrupted, lost, or deliberately discarded by an intermediate system. so an intermediate system can use resync points to toss RTP packets that don't matter for a given rendering resolution or region of interest. resync points are kind of like pointer marker segments, just tailored for high-bandwidth, low-latency streaming. 🌊💜

7.4. PTSTAMP Field

a sender SHOULD set P = 1, but only if it can generate PTSTAMP accurately. ⏱️

PTSTAMP can come from the same clock that makes the 32-bit timestamp field in the RTP Fixed Header. basically, a sender keeps (at least conceptually) a 32-bit counter that ticks up on a 90 kHz clock. every time an RTP packet goes out, the counter gets sampled and the 12 least significant bits of that sample go into the PTSTAMP field. 🕰️✨

if P = 1, then the transmission time TOFF (as defined in Section 5.3) for two consecutive RTP packets with identical timestamp fields MUST NOT differ by more than 4095. 💀

7.5. RES Field

a sender SHOULD set RES > 0 whenever possible. 🍓

NOTE: a sender can always safely set RES = 0, but that makes it harder to discard RTP packets based on resolution, as described in Section 8.3. 😬

7.6. Extra Information

the sender MUST set XTRAC to 0. 🙅‍♀️

future updates to this RTP payload format can permit other values. 👀

7.7. Unassigned Values

the sender MUST set unassigned values to 0. 🫠

unassigned values are saved for future updates to this RTP payload format. as Section 8.5 says, receivers that follow this spec will ignore those future values. unlike extension values (see Section 7.8), this lets the format grow while staying compatible with receivers that follow this spec. 🧩🌈

7.8. Extension Values

a sender MUST NOT use an extension value. 🚫💅

7.9. Code-Block Caching

this section only applies if C = 1. 🎀

a sender can boost bandwidth efficiency by only occasionally sending code-blocks for the static parts of the video, and sending empty code-blocks the rest of the time. 🧠 when C = 1, a receiver keeps a simple cache of code-blocks it's already gotten, and uses that cache to replace empty code-blocks (see Section 8.7). ✨

only the sender decides which code-blocks get swapped for empty ones, and when that happens. 👑

the sender can't know the receiver's cache state for sure though. some code-blocks might've been lost in transit, the sender doesn't know exactly when the receiver started processing the stream, etc. 🤷‍♀️

a code-block is empty if:

  • it does not contribute code-bytes as specified in the parent JPEG 2000 packet header; or
  • it contains an HT cleanup segment and the first two bytes of the Magsgn byte-stream are between 0xFF80 and 0xFF8F (if the code-block conforms to [JPEG2000-15]).

NOTE: that last condition lets the encoder add padding bytes to keep a constant bit rate even when a code-block doesn't contribute any code-bytes, as suggested in Annex F.4 of [JPEG2000-15]. 🫶

8. Receiver 🦋

8.1. PTSTAMP

receivers can use PTSTAMP values to speed up sender clock recovery, since PTSTAMP usually updates more often than timestamp. ⚡

8.2. QUAL

a receiver can discard RTP packets where QUAL > N if it only wants to reconstruct an image that uses quality layers N and below. 🌟

8.3. RES 💅

okay so JPEG 2000 chops an image up using a bunch of discrete wavelet transform (DWT) stages 🧚

+===============+============+=============+===========+============+
| Decomposition | Resolution | Sub-bands   | Keep all  | ...to      |
| Level         | Level      |             | Body      | decode an  |
|               |            |             | Packets   | image with |
|               |            |             | with RES  | at most    |
|               |            |             | equal to  | these      |
|               |            |             | or less   | dimensions |
|               |            |             | than      |            |
|               |            |             | this      |            |
|               |            |             | value...  |            |
+===============+============+=============+===========+============+
| 1             | 5          | HL1,LH1,HH1 | 7         | W x H      |
+---------------+------------+-------------+-----------+------------+
| 2             | 4          | HL2,LH2,HH2 | 6         | (W/2) x    |
|               |            |             |           | (H/2)      |
+---------------+------------+-------------+-----------+------------+
| 3             | 3          | HL3,LH3,HH3 | 5         | (W/4) x    |
|               |            |             |           | (H/4)      |
+---------------+------------+-------------+-----------+------------+
| 4             | 2          | HL4,LH4,HH4 | 4         | (W/8) x    |
|               |            |             |           | (H/8)      |
+---------------+------------+-------------+-----------+------------+
| 5             | 1          | HL5,LH5,HH5 | 3         | (W/16) x   |
|               |            |             |           | (H/16)     |
+---------------+------------+-------------+-----------+------------+
| 5             | 0          | LL5         | 2         | (W/32) x   |
|               |            |             |           | (H/32)     |
+---------------+------------+-------------+-----------+------------+

   Table 2: Optional discarding of Body Packets based on the value
  of the RES field when decoding a reduced resolution image, in the
   case where N_L = 5 and all DWT stages consist of both horizontal
   and vertical transforms.  The image has nominal width and height
                              of W x H.

Table 2 shows the case where every DWT stage has both horizontal and vertical transforms, which is the only mode [JPEG2000-1] supports ✨. The first stage turns the image into (i) the half-resolution image (LL1 sub-bands) and (ii) leftover high-frequency data (HH1, LH1, HL1 sub-bands). The second stage takes that half-res image and makes a quarter-res image (LL2 sub-bands) plus more leftover high-frequency data (HH2, LH2, HL2 sub-bands). This keeps going N_L times, where N_L is the number of decomposition levels defined in the COD and COC marker segments of the codestream 🌸.

The decoding process rebuilds the image by running the whole coding process backwards, starting from the smallest version stored in the codestream (LL_(N_L)).

So you can get a smaller version of the image by stopping the decoding partway through 🦋. For example, to reconstruct the image at quarter resolution (LL2), only sub-bands with index greater than 2 (e.g., HL3, LH3, HH3, HL4, LH4, HH4, etc.) are needed. In other words, a receiver that wants a quarter-resolution image can discard all RTP packets where RES >= 6, since those packets can only contribute to the HL1, LH1, HH1, HL2, LH2, and HH2 sub-bands. It's giving efficiency 🙈

In the case where all DWT stages consist of both horizontal and vertical transforms, the maximum decodable resolution is reduced by a factor of 2^(7 - N) if all Body Packets where RES > N are discarded.

+===============+============+=============+===========+============+
| Decomposition | Resolution | Sub-bands   | Keep all  | ...to      |
| Level         | Level      |             | Body      | decode an  |
|               |            |             | Packets   | image with |
|               |            |             | with RES  | at most    |
|               |            |             | equal to  | these      |
|               |            |             | or less   | dimensions |
|               |            |             | than      |            |
|               |            |             | this      |            |
|               |            |             | value...  |            |
+===============+============+=============+===========+============+
| 1             | 5          | HL1,LH1,HH1 | 7         | W x H      |
+---------------+------------+-------------+-----------+------------+
| 2             | 4          | HL2,LH2,HH2 | 6         | (W/2) x    |
|               |            |             |           | (H/2)      |
+---------------+------------+-------------+-----------+------------+
| 3             | 3          | HX3         | 5         | (W/4) x    |
|               |            |             |           | (H/2)      |
+---------------+------------+-------------+-----------+------------+
| 4             | 2          | HX4         | 4         | (W/8) x    |
|               |            |             |           | (H/2)      |
+---------------+------------+-------------+-----------+------------+
| 5             | 1          | HX5         | 3         | (W/16) x   |
|               |            |             |           | (H/2)      |
+---------------+------------+-------------+-----------+------------+
| 5             | 0          | LX5         | 2         | (W/32) x   |
|               |            |             |           | (H/2)      |
+---------------+------------+-------------+-----------+------------+

   Table 3: Optional discarding of Body Packets based on the value
   of the RES field when decoding a reduced resolution image, in the
   case where N_L = 5 and some DWT stages consist of only horizontal
     transforms.  The image has nominal width and height of W x H.

Table 3 shows the case where some DWT stages consist of only horizontal transforms, as specified in Annex F of [JPEG2000-2] 🎀.

A receiver can therefore discard all Body Packets where RES is greater than some threshold value if it wants to decode an image with its resolution reduced by a factor determined by that threshold value, as illustrated in Tables 2 and 3 🌈.

8.4. Extra Information 🎀

The receiver MUST accept values of XTRAC other than 0 and MUST ignore the value of XTRAB, whose length is given by XTRAC 🫶.

Future updates to this RTP payload format can specify XTRAB contents in a way that lets receivers conforming to this specification ignore that content ✨.

8.5. Unassigned Values 🦋

The receiver MUST ignore unassigned values (see additional discussion in Section 7.7) 🙈.

8.6. Extension Values 💀

The receiver MUST discard an RTP packet that contains any extension value. no exceptions, bestie 😭.

8.7. Code-Block Caching 🧚

This section only applies if C = 1.

When C = 1, the sender can improve bandwidth efficiency by only occasionally transmitting code-blocks that correspond to static portions of the video, and otherwise transmitting empty code-blocks (see Section 7.9) 💜.

When decoding a codestream, the following procedures apply for each code-block in the codestream:

  • If the code-block in the codestream is empty, the receiver MUST replace it with a matching code-block from the cache, if one exists.
  • If the code-block in the codestream is not empty, the receiver MUST replace any matching code-block from the cache with the code-block in the codestream.

Two code-blocks are matching if the following characteristics are identical for both: spatial coordinates, resolution level, component, sub-band, and value of the TP field of the parent RTP packet. it's giving cache-matching main character energy 👑

9. Media Type ✨

9.1. General 🌸

This RTP payload format is identified using the media type defined in Section 9.2 🍓. This media type has been registered in accordance with [RFC4855] using the template in [RFC6838] 💅.

9.2. Definition 📝

okay bestie, this is the media type registration form, and it's giving very official 💅

  • Type name: video
  • Subtype name: jpeg2000-scl
  • Required parameters: N/A
  • Optional parameters:
    • pixel: Specifies the pixel format used by the video sequence. 🎨
      • The parameter MUST be a URI-reference as specified in [RFC3986].
      • If the parameter is a relative-ref as specified in [RFC3986], then it MUST be equal to one of the pixel formats specified in Table 4, and the RTP header and payload MUST conform to the characteristics of that pixel format.
      • If the parameter is not a relative-ref, the specification of the pixel format is left to the application that defined the URI.
      • If the parameter is not specified, the pixel format is unspecified.
    • sample: Specifies the format of the samples in each component of the codestream. 🧩
      • The parameter MUST be a URI-reference as specified in [RFC3986].
      • If the parameter is a relative-ref as specified in [RFC3986], then it MUST be equal to one of the formats specified in Appendix C, and the stream MUST conform to the characteristics of that format.
      • If the parameter is not a relative-ref, the specification of the sample format is left to the application that defined the URI.
      • If the parameter is not specified, the sample format is unspecified.
    • width: Maximum width in pixels of each image. Integer between 0 and 4,294,967,295. 📏
      • The parameter MUST be a sequence of 1 or more digits.
      • If the parameter is not specified, the maximum width is unspecified.
    • height: Maximum height in pixels of each image. Integer between 0 and 4,294,967,295. 📐
      • The parameter MUST be a sequence of 1 or more digits.
      • If the parameter is not specified, the maximum height is unspecified.
    • signal: Specifies the sequence of image types. 🖼️
      • The parameter MUST be a URI-reference as specified in [RFC3986].
      • If the parameter is a relative-ref as specified in [RFC3986], then it MUST be equal to one of the signal formats specified in Appendix B, and the image sequence MUST conform to that signal format.
      • If the parameter is not a relative-ref, the specification of the pixel format is left to the application that defined the URI.
      • If the parameter is not specified, the stream consists of an arbitrary sequence of image types.
    • caps: The parameter contains a list of sets of constraints to which the stream conforms, with each set of constraints identified using an absolute-URI defined by an application. 🧢
      • The parameter MUST conform to the uri-list syntax expressed as follows using ABNF [RFC5234]:
uri-list = absolute-URI *(";" absolute-URI)
  • The application that defines the absolute-URI MUST ensure that it does not contain any ";" character and MUST associate it with a set of constraints to which the stream conforms. Such constraints can, for example, include the maximum height and width of images.
  • If the parameter is not specified, constraints beyond those specified in this document are unspecified.
  • cache: The value of the parameter MUST be either false or true. 💾
    • If the parameter is true, the field C MAY be 0 or 1; otherwise, the field C MUST be 0.
    • If the parameter is not specified, then the parameter is equal to false.
  • Encoding considerations: This media type is framed and binary. See Section 4.8 of [RFC6838].
  • Security considerations: JPEG 2000 is a flexible image format. As a result, the size of the memory structures required to process JPEG 2000 images can vary greatly depending on the characteristics of the image and the encoding parameters. For example, the JPEG 2000 syntax allows image height and width up to 2^32 - 1 pixels, which is also captured in the syntax of the height and width parameters of this media type. Therefore, implementations SHOULD take care when processing input that influences the size of memory structures and SHOULD fail gracefully when resource constraints are exceeded. See also Section 13. 🛡️
  • Interoperability considerations: The RTP stream is a sequence of JPEG 2000 images. An implementation that conforms to the family of JPEG 2000 standards can decode and attempt to display each image.
  • Published specification: RFC 9828
  • Applications that use this media type: video streaming and communication
  • Fragment identifier considerations: N/A
  • Additional information: N/A
  • Person & email address to contact for further information: Pierre-Anthony Lemieux <pal@sandflow.com>
  • Intended usage: COMMON
  • Restrictions on Usage: This media type depends on RTP framing and hence is only defined for use with RTP as specified in [RFC3550]. Transport within other framing protocols is not defined at the time.
  • Author: Pierre-Anthony Lemieux <pal@sandflow.com>
  • Change controller: IETF

10. Mapping to the Session Description Protocol (SDP) 📡

The mapping of the payload format media type and its parameters to SDP, as specified in [RFC8866], MUST be done according to Section 3 of [RFC4855].

11. Congestion Control 🚦

okay so this RTP payload format gets used in a ton of different contexts, which means there's no single congestion control mechanism that works for all of them 🙈 Here's a non-exhaustive list of example mechanisms that can be used:

  • a sender can offer several alternative streams for a given video signal, each with a different data rate corresponding to a different compression ratio;
  • a sender can modulate the data rate within a stream by modulating the resolution, frame rate, or compression ratio of the underlying video signal; or
  • an intermediate system can lower the data rate of a stream by discarding resolution levels and/or quality layers, as specified in Section 7.2.

As suggested in Section 10 of [RFC3550], RTCP feedback can be used in the data rate adaptation process. ✨

12. IANA Considerations 🗂️

IANA has registered the media type specified in Section 9. 🎀

13. Security Considerations 🔐

bestie, this section is serious, so listen up 👑

RTP packets using the payload format specified in this document are subject to the security considerations discussed in [RFC3550] and in any applicable RTP profile such as [RFC3551], [RFC4585], [RFC3711], and [RFC5124]. However, as [RFC7202] discusses, it is not an RTP payload format's responsibility to discuss or mandate what solutions are used to meet the basic security goals like confidentiality, integrity, and source authenticity for RTP in general. This responsibility lies with anyone using RTP in an application. They can find guidance on available security mechanisms and important considerations in [RFC7201]. Applications SHOULD use one or more appropriate strong security mechanisms. The rest of this section discusses the security impacting properties of the payload format itself.

This RTP payload format and its media decoder do not exhibit any significant non-uniformity in the receiver-side computational complexity for RTP packet processing and thus are unlikely to pose a denial-of-service threat due to the receipt of pathological data. In addition, the RTP payload format does not contain any active content. 🧼

Security considerations related to the JPEG 2000 codestream contained in the payload are discussed in Section 3 of [RFC3745]. 💜

14. References 📚

14.1. Normative References

14.2. Informative References 📚

okay bestie, these are the extra reads that back up the spec. nothing here is required to implement it, but the references are all listed below exactly as written 🫶

[OV2110-0] SMPTE, "Professional Media Over Managed IP Networks,
           Roadmap for the 2110 Document Suite", 4 December 2018,
           <https://pub.smpte.org/doc/ov2110-0/20181204-pub/>.

[RFC3551]  Schulzrinne, H. and S. Casner, "RTP Profile for Audio and
           Video Conferences with Minimal Control", STD 65, RFC 3551,
           DOI 10.17487/RFC3551, July 2003,
           <https://www.rfc-editor.org/info/rfc3551>.

[RFC3711]  Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K.
           Norrman, "The Secure Real-time Transport Protocol (SRTP)",
           RFC 3711, DOI 10.17487/RFC3711, March 2004,
           <https://www.rfc-editor.org/info/rfc3711>.

[RFC3745]  Singer, D., Clark, R., and D. Lee, "MIME Type
           Registrations for JPEG 2000 (ISO/IEC 15444)", RFC 3745,
           DOI 10.17487/RFC3745, April 2004,
           <https://www.rfc-editor.org/info/rfc3745>.

[RFC4175]  Gharai, L. and C. Perkins, "RTP Payload Format for
           Uncompressed Video", RFC 4175, DOI 10.17487/RFC4175,
           September 2005, <https://www.rfc-editor.org/info/rfc4175>.

[RFC4585]  Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey,
           "Extended RTP Profile for Real-time Transport Control
           Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585,
           DOI 10.17487/RFC4585, July 2006,
           <https://www.rfc-editor.org/info/rfc4585>.

[RFC5124]  Ott, J. and E. Carrara, "Extended Secure RTP Profile for
           Real-time Transport Control Protocol (RTCP)-Based Feedback
           (RTP/SAVPF)", RFC 5124, DOI 10.17487/RFC5124, February
           2008, <https://www.rfc-editor.org/info/rfc5124>.

[RFC5371]  Futemma, S., Itakura, E., and A. Leung, "RTP Payload
           Format for JPEG 2000 Video Streams", RFC 5371,
           DOI 10.17487/RFC5371, October 2008,
           <https://www.rfc-editor.org/info/rfc5371>.

[RFC5450]  Singer, D. and H. Desineni, "Transmission Time Offsets in
           RTP Streams", RFC 5450, DOI 10.17487/RFC5450, March 2009,
           <https://www.rfc-editor.org/info/rfc5450>.

[RFC6838]  Freed, N., Klensin, J., and T. Hansen, "Media Type
           Specifications and Registration Procedures", BCP 13,
           RFC 6838, DOI 10.17487/RFC6838, January 2013,
           <https://www.rfc-editor.org/info/rfc6838>.

[RFC7201]  Westerlund, M. and C. Perkins, "Options for Securing RTP
           Sessions", RFC 7201, DOI 10.17487/RFC7201, April 2014,
           <https://www.rfc-editor.org/info/rfc7201>.

[RFC7202]  Perkins, C. and M. Westerlund, "Securing the RTP
           Framework: Why RTP Does Not Mandate a Single Media
           Security Solution", RFC 7202, DOI 10.17487/RFC7202, April
           2014, <https://www.rfc-editor.org/info/rfc7202>.

Appendix A. Pixel Formats 🌈

okay so Table 4 is where all the pixel formats live, and it's giving organized 💅

    +=============+=======+=======+=======+=======+===+=====+=========+
    | NAME        | SAMP  | COMPS | TRANS | PRIMS |MAT| VFR | Mapping |
    |             |       |       |       |       |   |     | in      |
    |             |       |       |       |       |   |     | Table 1 |
    +=============+=======+=======+=======+=======+===+=====+=========+
    | rgb444sdr   | 4:4:4 | RGB   | 1     | 1     |0  | 0,  | RGB     |
    |             |       |       |       |       |   | 1   |         |
    +-------------+-------+-------+-------+-------+---+-----+---------+
    | rgb444wcg   | 4:4:4 | RGB   | 1     | 9     |0  | 0,  | RGB     |
    |             |       |       |       |       |   | 1   |         |
    +-------------+-------+-------+-------+-------+---+-----+---------+
    | rgb444pq    | 4:4:4 | RGB   | 16    | 9     |0  | 0,  | RGB     |
    |             |       |       |       |       |   | 1   |         |
    +-------------+-------+-------+-------+-------+---+-----+---------+
    | rgb444hlg   | 4:4:4 | RGB   | 18    | 9     |0  | 0,  | RGB     |
    |             |       |       |       |       |   | 1   |         |
    +-------------+-------+-------+-------+-------+---+-----+---------+
    | ycbcr420sdr | 4:2:0 | YCbCr | 1     | 1     |1  | 0   | YCbCr   |
    +-------------+-------+-------+-------+-------+---+-----+---------+
    | ycbcr422sdr | 4:2:2 | YCbCr | 1     | 1     |1  | 0   | YCbCr   |
    +-------------+-------+-------+-------+-------+---+-----+---------+
    | ycbcr422wcg | 4:2:2 | YCbCr | 1     | 9     |9  | 0   | YCbCr   |
    +-------------+-------+-------+-------+-------+---+-----+---------+
    | ycbcr422pq  | 4:2:2 | YCbCr | 16    | 9     |9  | 0   | YCbCr   |
    +-------------+-------+-------+-------+-------+---+-----+---------+
    | ycbcr422hlg | 4:2:2 | YCbCr | 18    | 9     |9  | 0   | YCbCr   |
    +-------------+-------+-------+-------+-------+---+-----+---------+

                       Table 4: Defined Pixel Formats

Each pixel format is described by these fields:

  • NAME: identifies the pixel format 🏷️
  • SAMP: how the colour channels get subsampled
    • 4:2:0: the C_b and C_r colour channels are subsampled horizontally and vertically by 1/2.
    • 4:2:2: the C_b and C_r colour channels are subsampled horizontally by 1/2.
    • 4:4:4: no colour channels are subsampled.
  • COMPS: how the components are arranged
    • RGB: each codestream contains exactly three components, associated with the R, G, and B colour channels, in order.
    • YCbCr: each codestream contains exactly three components, associated with the Y, C_b, and C_r colour channels, in order.
  • TRANS: identifies the transfer characteristics allowed by the pixel format, as defined in [REC-ITU-T-H273]. 🎞️
  • PRIMS: identifies the colour primaries allowed by the pixel format, as defined in [REC-ITU-T-H273]. 🎨
  • MAT: identifies the matrix coefficients allowed by the pixel format, as defined in [REC-ITU-T-H273]. 🧮
  • VFR: allows values of the VideoFullRangeFlag defined in [REC-ITU-T-H273]. ✨

Appendix B. Signal Formats 🦋

these are the four signal formats, and the MUST lines are the ones you really can't skip 🙈

  • prog: The stream MUST only consist of a sequence of progressive frames.
  • psf: Progressive segmented frame (PsF) stream. The stream MUST only consist of an alternating sequence of first segment and second segment.
  • tff: Interlaced stream. The stream MUST only consist of an alternating sequence of first field and second field, where the first line of the first field is the first line of the frame.
  • bff: Interlaced stream. The stream MUST only consist of an alternating sequence of first field and second field, where the first line of the first field is the second line of the frame.

Appendix C. Sample Formats 🍓

last one, and it's all about how many bits each sample gets 💖

  • 8: All components consist of unsigned 8-bit integer samples.
  • 10: All components consist of unsigned 10-bit integer samples.
  • 12: All components consist of unsigned 12-bit integer samples.
  • 16: All components consist of unsigned 16-bit integer samples.

Authors' Addresses 🧚

Pierre-Anthony Lemieux (editor) Sandflow Consulting LLC San Mateo, CA United States of America Email: pal@sandflow.com

David Scott Taubman University of New South Wales Sydney Australia Email: d.taubman@unsw.edu.au

and that's the whole spec, bestie! 🌈👑

📦 where this page comes from