RFC 9134

RTP Payload Format for ISO/IEC 21122 (JPEG XS)

๐Ÿ“… published October 2021, from ietf

RFC 9134 ๐Ÿ’– Part 1 of 7

okay bestie, we're starting RFC 9134!! โœจ It's the spec for sending JPEG XS video over RTP, and yes, it's giving serious main character energy ๐Ÿ’…

Here's the title block tea ๐Ÿ“

  • Series: Internet Engineering Task Force (IETF) ๐ŸŒธ
  • Number and title: RFC 9134, RTP Payload Format for ISO/IEC 21122 (JPEG XS) ๐ŸŽ€
  • Category: Standards Track ๐Ÿ‘‘
  • Date: October 2021 ๐Ÿ“…
  • ISSN: 2070-1721
  • Authors: Thomas Bruylants (intoPIX), Alexandre Descampe (UCLouvain), Chris Damman (intoPIX), and Thomas Richter (Fraunhofer IIS) ๐Ÿซถ
  • Obsoletes and updates: the title block doesn't list anything this RFC obsoletes or updates ๐Ÿงš

Abstract โœจ

okay so this doc defines an RTP payload format for moving video that's been encoded with JPEG XS (ISO/IEC 21122) ๐Ÿฆ‹ JPEG XS is a low-latency, lightweight image coding system, and it's honestly iconic ๐Ÿ˜ญ Compared to uncompressed video, it lets you go for higher resolutions and frame rates, while still looking visually lossless, using less power, and keeping the encode-decode latency to a fraction of a video frame ๐Ÿ”ฅ

Status of This Memo ๐Ÿ“œ

this one's an Internet Standards Track document!! ๐Ÿ’œ

It's a product of the Internet Engineering Task Force (IETF), so it represents the consensus of the IETF community. It went through public review and got approved for publication by the Internet Engineering Steering Group (IESG). If you want more on Internet Standards, check out Section 2 of RFC 7841 ๐ŸŒˆ

For the current status, any errata, and how to send feedback, go to https://www.rfc-editor.org/info/rfc9134 ๐Ÿ”—

Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved. ๐ŸŽ€

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they stood on the publication date. Please read these carefully, bestie, because they spell out your rights and restrictions for this document. Code Components pulled out of this document must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come with no warranty, as described in the Simplified BSD License โš–๏ธ

Table of Contents ๐Ÿ—‚๏ธ

1.  Introduction
2.  Conventions, Definitions, and Abbreviations
3.  Media Format Description
  3.1.  Image Data Structures
  3.2.  Codestream
  3.3.  Video Support Box and Color Specification Box
  3.4.  JPEG XS Frame
4.  RTP Payload Format
  4.1.  RTP Packetization
  4.2.  RTP Header Usage
  4.3.  Payload Header Usage
  4.4.  Payload Data
5.  Traffic Shaping and Delivery Timing
6.  Congestion Control Considerations
7.  Payload Format Parameters
  7.1.  Media Type Registration
8.  SDP Parameters
  8.1.  Mapping of Payload Type Parameters to SDP
  8.2.  Usage with SDP Offer/Answer Model
9.  IANA Considerations
10. Security Considerations
11. References
  11.1.  Normative References
  11.2.  Informative References
Acknowledgments
Authors' Addresses

1. Introduction ๐Ÿฆ‹

okay so this doc is all about packetizing video that's been encoded with JPEG XS [ISO21122-1] so it can ride inside the Real-time Transport Protocol (RTP) [RFC3550] ๐Ÿš€

JPEG XS is seriously impressive. It compresses and recompresses image sequences using pretty modest computing resources, and it stays robust through multiple compression and decompression cycles, even when content from different sources gets mixed together (think subtitles, overlays, or logos) ๐Ÿ’… The target compression ratios that keep things visually lossless are in the range of 2:1 to 10:1, depending on what the source material is ๐Ÿ“ธ The latency added by encoding and decoding can be confined to a fraction of a video frame, usually anywhere from a small number of lines down to less than a single line โšก

2. Conventions, Definitions, and Abbreviations ๐Ÿ“š

bestie, listen to this one ๐Ÿซถ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are interpreted as described in BCP 14 [RFC2119] [RFC8174], but only when they appear in all capitals, exactly as shown here ๐Ÿ’–

Here are the terms you need to know ๐Ÿงš

Application Data Unit (ADU): the unit of source data handed to the transport layer as payload. In this RTP payload definition, one ADU is one single JPEG XS video frame ๐ŸŽฌ

Color Specification (CS) box: an ISO color specification box defined in [ISO21122-3] (JPEG XS Part 3). It holds color-related metadata needed to display JPEG XS video frames correctly, like color primaries, transfer characteristics, and matrix coefficients ๐ŸŒˆ

End of Codestream (EOC) marker: a marker made of the two bytes 0xff11, which marks the end of a JPEG XS codestream ๐Ÿ

JPEG XS codestream: a sequence of bytes representing a compressed image, formatted according to [ISO21122-1] (JPEG XS Part 1) ๐Ÿ–ผ๏ธ

JPEG XS codestream header: a sequence of bytes that starts with an SOC marker and sits at the beginning of each JPEG XS codestream. It's made of markers and marker segments that don't carry entropy coded data, just metadata like the video frame dimensions and component precision ๐Ÿ“‹

JPEG XS frame: for progressive video, a single JPEG XS picture segment. For interlaced video, it's the concatenation of two JPEG XS picture segments ๐ŸŽž๏ธ

JPEG XS header segment: the concatenation of a video support box [ISO21122-3], a color specification box [ISO21122-3], and a JPEG XS codestream header ๐Ÿง

JPEG XS picture segment: the concatenation of a video support box [ISO21122-3], a color specification box [ISO21122-3], and a JPEG XS codestream ๐Ÿ“ฆ

JPEG XS stream: a sequence of JPEG XS frames ๐ŸŒŠ

Marker: a two-byte functional sequence that's part of a JPEG XS codestream. It starts with a 0xff byte, followed by a byte that defines its function ๐Ÿ”‘

Marker segment: a marker, plus a 16-bit marker size, plus the payload data that follows the size ๐Ÿงฉ

Packetization unit: a portion of an ADU whose boundaries line up with the boundaries of RTP packet payloads (not counting the payload header). So the first (or respectively, last) byte of a packetization unit is the first (or respectively, last) byte of an RTP packet payload, excluding its payload header ๐Ÿ’Œ

SLH (SLice Header) marker: a marker that represents a slice header, as defined in [ISO21122-1] ๐Ÿฐ

Slice: the smallest independently decodable unit of a JPEG XS codestream. Keep in mind it decodes to wavelet coefficients, which still need inverse wavelet filtering before you get an image โœจ

Start of a Codestream (SOC) marker: a marker made of the two bytes 0xff10, which indicates the start of a JPEG XS codestream. The SOC marker counts as an integral part of the JPEG XS codestream header ๐ŸŒ…

Video Support (VS) box: an ISO video support box, as defined in [ISO21122-3], holding metadata needed to play back a JPEG XS stream. That metadata could include its maximum bit rate, subsampling structure, buffer model, and frame rate ๐Ÿ“บ

3. Media Format Description ๐Ÿ”

okay so this section explains the terminology and concepts that are specific to JPEG XS, as specified in [ISO21122-1], [ISO21122-2], and [ISO21122-3] ๐Ÿง 

3.1. Image Data Structures ๐Ÿงฑ

JPEG XS is a low-latency, lightweight image coding system for coding continuous-tone grayscale or continuous-tone color digital images ๐Ÿ–ค

It gives you an efficient way to represent image signals using wavelet analysis. The wavelet filter process splits each component into multiple bands. Each band is made of coefficients that describe the image signal of a given component within a frequency domain specific to the wavelet filter type, meaning the particular filter that goes with that band ๐ŸŒŠ

Wavelet coefficients get grouped into precincts. Each precinct includes all the coefficients across all bands that contribute to one spatial region of the image ๐Ÿ—บ๏ธ

One or more precincts get combined into slices, and each slice consists of an integer number of precincts. Precincts never cross slice boundaries, and wavelet coefficients in precincts from different slices can be decoded independently of each other. Note, though, that the wavelet transformation runs across slice boundaries ๐Ÿ‘€ A slice always extends over the full width of the image, but it may only cover parts of its height ๐Ÿชž

3.2. Codestream ๐ŸŽ€

A JPEG XS codestream is formed by these, in this order:

  • a JPEG XS codestream header, which starts with a Start of Codestream (SOC) marker,
  • one or more slices,
  • an EOC marker to signal the end of the codestream.

The JPEG XS codestream format, including the definition of all markers, is further defined in [ISO21122-1]. It represents sample values of a single image, without any interpretation relative to a color space ๐ŸŽจ

3.3. Video Support Box and Color Specification Box ๐ŸŽฌ

okay so the codestream has enough info to rebuild the sample values of one image, but it doesn't tell you how to interpret those samples. That job goes to the video support box and the color specification box, which hold the important stuff you need to play the JPEG XS stream properly. Their layout, syntax, and content are all defined in [ISO21122-3] ๐Ÿ’œ

The video support box gives you info on the maximum bit rate, the frame rate, the interlaced mode (progressive or interlaced), the subsampling image format, the informative timecode of the current JPEG XS frame, the profile, the level/sublevel used, and optionally the buffer model and the mastering display metadata ๐Ÿ“Š

Now the profile and level/sublevel, which are specified by the Ppih and Plev fields [ISO21122-2], set limits on what's needed to decode the codestream and handle the output ๐Ÿงฎ Profiles limit the required algorithmic features and parameter ranges used in the codestream. The combo of level and sublevel sets a lower bound on the required throughput for a decoder, in the image (or decoded) domain and in the codestream (or coded) domain, respectively. The actual defined profiles and levels/sublevels, along with the values for the Ppih and Plev fields, are defined in [ISO21122-2] ๐ŸŒŸ

Last one! The color specification box says which color primaries, transfer characteristics, matrix coefficients, and video full range flag are needed to specify the color space of the video stream ๐ŸŒˆ That's the whole picture so far, bestie, and there's more coming in the next part ๐Ÿซถโœจ

3.4. JPEG XS Frame โœจ

okay so bestie, a video support box, a color specification box, and a JPEG XS codestream all stuck together make ONE JPEG XS picture segment ๐ŸŽ€ it's literally a whole little package ๐Ÿ“ฆ

For a progressive video stream, each JPEG XS frame is just one picture segment. Simple and clean ๐Ÿฆ‹

For an interlaced video stream, though, each JPEG XS frame is two picture segments glued together ๐Ÿ‘ฏโ€โ™€๏ธ The codestream in each segment belongs to exactly one of the two fields in the interlaced frame. Both picture segments SHALL contain identical boxes, meaning the byte sequence made from the video support box plus the color specification box is exactly the same in both segments of the frame. no cap, that matching is non-negotiable ๐Ÿ’…

Also, the interlaced mode gets signaled by the frat field [ISO21122-3] in the video support box, and that field tells you whether it's progressive interlaced top-field-first or interlaced bottom-field-first mode ๐ŸŒˆ So for interlaced content, its value SHALL also be identical in both picture segments. It's giving consistency ๐Ÿ’–


4. RTP Payload Format ๐Ÿš€

this section is where we figure out how JPEG XS streams get shipped over the Real-time Transport Protocol (RTP) [RFC3550] ๐Ÿ“ก

to travel over RTP, each JPEG XS stream gets its own distinct RTP stream ๐ŸŽ€ and every one of those is identified by its own distinct synchronization source (SSRC) [RFC3550]. Main character energy for sure โœจ

A JPEG XS stream gets chopped up into Application Data Units (ADUs), and each ADU is exactly one JPEG XS frame ๐Ÿงš


4.1. RTP Packetization ๐Ÿ“ฆ

okay so an ADU is built out of several packetization units ๐Ÿงฉ If a packetization unit is bigger than the maximum size an RTP packet payload can hold, it gets split across multiple RTP packet payloads, as shown in Figure 1 ๐Ÿ‘€

Each packet SHALL contain (part of) one, and only one, packetization unit. A packetization unit can stretch over multiple packets ๐ŸŒธ The payload of every packet SHALL have the same size (based, e.g., on the Maximum Transfer Unit of the network), with the one possible exception being the last packet of a packetization unit. Also, the boundaries of a packetization unit SHALL line up with the boundaries of a packet's payload (not counting the payload header). So the first byte of the packetization unit SHALL be the first byte of the payload, and the last byte SHALL be the last byte of the payload, again not counting the header ๐Ÿ’…

RTP        +-----+------------------------+
Packet #1  | Hdr | Packetization unit #1  |
           +-----+------------------------+
RTP        +-----+--------------------------------------+
Packet #2  | Hdr | Packetization unit #2                |
           +-----+--------------------------------------+
RTP        +-----+--------------------------------------------------+
Packet #3  | Hdr | Packetization unit #3  (part 1/3)                |
           +-----+--------------------------------------------------+
RTP        +-----+--------------------------------------------------+
Packet #4  | Hdr | Packetization unit #3  (part 2/3)                |
           +-----+--------------------------------------------------+
RTP        +-----+----------------------------------------------+
Packet #5  | Hdr | Packetization unit #3  (part 3/3)            |
           +-----+----------------------------------------------+
             ...
RTP        +-----+-----------------------------------------+
Packet #P  | Hdr | Packetization unit #N  (part q/q)       |
           +-----+-----------------------------------------+

                Figure 1: Example of ADU Packetization

There are two packetization modes defined for this RTP payload format, and they're both lowkey iconic ๐Ÿ‘‘

Codestream packetization mode: In this mode, the packetization unit SHALL be the entire JPEG XS picture segment (so the codestream with the boxes in front of it). That means a progressive frame ends up with a single packetization unit, while an interlaced frame gets two. The progressive case is shown in Figure 2 ๐Ÿฆ‹

Slice packetization mode: In this mode, the packetization unit SHALL be the slice. In other words, there SHALL be data from no more than one slice per RTP packet ๐Ÿ“ The first packetization unit SHALL be made of the JPEG XS header segment (the VS box, the CS box, and the JPEG XS codestream header all combined). After that, successive units follow, and each one holds one and only one slice. The packetization unit that contains the last slice of a JPEG XS codestream SHALL also contain the EOC marker, sitting right after that last slice. This is shown in Figure 3. And for an interlaced frame, the JPEG XS header segment of the second field SHALL be in its own packetization unit ๐Ÿซถ

RTP        +-----+--------------------------------------------------+
Packet #1  | Hdr | VS box + CS box + JPEG XS codestream (part 1/q)  |
           +-----+--------------------------------------------------+
RTP        +-----+--------------------------------------------------+
Packet #2  | Hdr | JPEG XS codestream (part 2/q)                    |
           +-----+--------------------------------------------------+
             ...
RTP        +-----+--------------------------------------+
Packet #P  | Hdr | JPEG XS codestream (part q/q)        |
           +-----+--------------------------------------+

          Figure 2: Example of Codestream Packetization Mode
RTP        +-----+----------------------------+
Packet #1  | Hdr | JPEG XS header segment     |
           +-----+----------------------------+
RTP        +-----+--------------------------------------------------+
Packet #2  | Hdr | Slice #1  (part 1/2)                             |
           +-----+--------------------------------------------------+
RTP        +-----+-------------------------------------------+
Packet #3  | Hdr | Slice #1  (part 2/2)                      |
           +-----+-------------------------------------------+
RTP        +-----+--------------------------------------------------+
Packet #4  | Hdr | Slice #2  (part 1/3)                             |
           +-----+--------------------------------------------------+
             ...
RTP        +-----+---------------------------------------+
Packet #P  | Hdr | Slice #N  (part q/q) + EOC marker     |
           +-----+---------------------------------------+

            Figure 3: Example of Slice Packetization Mode

okay so here's the tea โ˜• In a constant bitrate (CBR) scenario of JPEG XS, the codestream packetization mode guarantees that a JPEG XS RTP stream produces both a constant number of bytes per video frame and a constant number of RTP packets per video frame. Slay ๐Ÿ’–

But to get similar guarantees with JPEG XS in a variable bitrate (VBR) mode, or when using the slice packetization mode (for either CBR or VBR), extra mechanisms are needed. These can include a constraint at the rate allocation stage in the JPEG XS encoder to impose a CBR at the slice level, the use of padding data, or inserting empty RTP packets (i.e., an RTP packet whose payload data is empty) ๐Ÿงธ Managing how many packets get produced per video frame is application dependent, though, and it's not a strict requirement of this RTP payload specification ๐ŸŒˆ


4.2. RTP Header Usage ๐ŸŽ€

the RTP header format is specified in [RFC3550] and reprinted in Figure 4 for convenience ๐Ÿซถ This RTP payload format uses the header fields in a way that's consistent with that spec, so no drama there โœจ

The RTP payload (and the settings for some RTP header bits) for packetization units are specified in Section 4.3 ๐Ÿงš

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |V=2|P|X|  CC   |M|     PT      |       sequence number         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           timestamp                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           synchronization source (SSRC) identifier            |
   +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
   |            contributing source (CSRC) identifiers             |
   |                             ....                              |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

              Figure 4: RTP Header According to RFC 3550

The version (V), padding (P), extension (X), CSRC count (CC), sequence number, synchronization source (SSRC), and contributing source (CSRC) fields all follow their definitions in [RFC3550], so those are already sorted ๐Ÿ’…

Here's how the rest of the RTP header gets set for this payload format ๐Ÿ‘‡

Marker (M) [1 bit]: If progressive scan video is being transmitted, the marker bit marks the end of a video frame. If interlaced video is being transmitted, it marks the end of the field. The marker bit SHALL be set to 1 for the last packet of the video frame/field, and it SHALL be set to 0 for all other packets ๐Ÿšฉ

Payload Type (PT) [7 bits]: This is a dynamically allocated payload type field that tells you the payload is JPEG XS video ๐ŸŽฌ

Timestamp [32 bits]: The RTP timestamp is set to the sampling timestamp of the content. A 90 kHz clock rate SHALL be used โฐ

As specified in [RFC3550] and [RFC4175], the RTP timestamp points to the sampling instant of the first octet of the video frame that the RTP packet belongs to. Packets SHALL NOT include data from multiple video frames, and all packets belonging to the same video frame SHALL have the same timestamp. So several successive RTP packets will have equal timestamps if they're in the same video frame (until the marker bit gets set to 1, marking the last packet of the frame). The timestamp only goes up when a new video frame begins ๐Ÿฆ‹

If the sampling instant doesn't correspond to an integer value of the clock, the value SHALL be truncated to the next lowest integer, with no ambiguity ๐ŸŒธ

4.3. Payload Header Usage ๐ŸŽ€

okay so the first four bytes of the payload of an RTP packet in this format are called the "payload header" ๐Ÿ’… Figure 5 shows exactly what it looks like:

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |T|K|L| I |F counter|     SEP counter     |     P counter       |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

                          Figure 5: Payload Header

The payload header has these fields, and bestie, they're all doing a lot ๐Ÿ˜ญ

Transmission mode (T) [1 bit]: the T bit gets set to say that packets are sent sequentially by the transmitter ๐Ÿš‚ This tells the receiver how to size its input buffer(s), which is so iconic. If T=0, nothing can be assumed about the transmission order, so packets may be sent out of order by the transmitter ๐ŸŒ€ If T=1, packets SHALL be sent sequentially by the transmitter. The T-bit value SHALL be identical for all packets of the RTP stream.

pacKetization mode (K) [1 bit]: the K bit tells you which packetization mode is being used โœจ K=0 is codestream packetization mode, and K=1 is slice packetization mode. When the Transmission mode (T) is set to 0 (out of order), the slice packetization mode SHALL be used and K SHALL be set to 1. This is required because only the slice packetization mode supports out-of-order packet transmission. The K-bit value SHALL be identical for all packets of the RTP stream.

Last (L) [1 bit]: the L bit is set to mark the last packet of a packetization unit ๐Ÿ The end of the video frame also ends the packet holding the last unit of that frame, so the L bit is set whenever the M bit is set. In codestream packetization mode, the L bit and M bit mean the same thing, so they SHALL have identical values in each packet. ๐Ÿซถ

Interlaced information (I) [2 bits]: these two I bits show how the JPEG XS frame is scanned, either progressive or interlaced ๐Ÿฆ‹ For an interlaced frame, they also tell you which JPEG XS picture segment the payload is part of (first or second).

  • 00: The payload is progressively scanned.
  • 01: This value is reserved for future use.
  • 10: The payload is part of the first JPEG XS picture segment of an interlaced video frame. The height specified in the included JPEG XS codestream header is half of the height of the entire displayed image.
  • 11: The payload is part of the second JPEG XS picture segment of an interlaced video frame. The height specified in the included JPEG XS codestream header is half of the height of the entire displayed image.

F counter [5 bits]: the Frame (F) counter tracks which video frame a packet belongs to, modulo 32 ๐ŸŽž๏ธ It goes up by 1 for each video frame transmitted. Together with the timestamp, the frame number can help the decoder manage its input buffer and put packets back in their natural order. Main character energy ๐Ÿ‘‘

Slice and Extended Packet (SEP) counter [11 bits]: this one changes its personality depending on the packetization mode ๐ŸŒˆ

  • In codestream packetization mode (K=0), this counter resets whenever the Packet counter resets (see Section 4.4), and it increments by 1 whenever the Packet counter overruns.
  • In slice packetization mode (K=1), this counter identifies the slice modulo 2047 that the packet contributes to. If the data belongs to the JPEG XS header segment, this field SHALL have its maximal value, namely 2047 = 0x07ff. Otherwise, it is the slice index modulo 2047. Slice indices are counted from 0, which is the top of the video frame.

P counter [11 bits]: the Packet (P) counter gives the packet number modulo 2048 within the current packetization unit ๐Ÿ”ข It is set to 0 at the start of the packetization unit and goes up by 1 for every subsequent packet (if any) in the same unit. Practically speaking, in codestream packetization mode this field counts the packets inside a JPEG XS picture segment, and the SEP counter extends it when it overruns. In slice packetization mode, it counts the packets within a slice or within the JPEG XS header segment. ๐Ÿ’–


4.4. Payload Data ๐Ÿ“

okay so the payload data of a JPEG XS RTP stream is basically a bunch of JPEG XS frames stuck together ๐Ÿงฉ Across the whole RTP stream, all of the video support boxes and all of the color specification boxes SHALL retain their respective layouts for each JPEG XS frame. That means each video support box in the RTP stream SHALL define the same sub boxes. The effective values in the boxes are allowed to change, as long as their relative byte offsets SHALL NOT change. It's giving consistency ๐Ÿ’…

Each JPEG XS frame is the concatenation of one or more packetization unit(s), as explained in Section 4.1. Figure 6 shows this layout for a progressive video frame in codestream packetization mode, Figure 7 shows it for an interlaced video frame in codestream packetization mode, Figure 8 shows it for a progressive video frame in slice packetization mode, and Figure 9 shows it for an interlaced video frame in slice packetization mode. The Frame counter value isn't indicated, because it stays constant for all packetization units of a given video frame โœจ

   +=====[ Packetization unit (PU) #1 ]====+
   |           Video support box           |  SEP counter=0
   |  +---------------------------------+  |  P counter=0
   |  :      Sub boxes of the VS box    :  |
   |  +---------------------------------+  |
   +- - - - - - - - - - - - - - - - - - - -+
   |        Color specification box        |
   |  +---------------------------------+  |
   |  :      Fields of the CS box       :  |
   |  +---------------------------------+  |
   +- - - - - - - - - - - - - - - - - - - -+
   |          JPEG XS codestream           |
   :             (part 1/q)                :  M=0, K=0, L=0, I=00
   +---------------------------------------+
   |          JPEG XS codestream           |  SEP counter=0
   |             (part 2/q)                |  P counter=1
   :                                       :  M=0, K=0, L=0, I=00
   +---------------------------------------+
   |          JPEG XS codestream           |  SEP counter=0
   |             (part 3/q)                |  P counter=2
   :                                       :  M=0, K=0, L=0, I=00
   +---------------------------------------+
   :                                       :
   +---------------------------------------+
   |          JPEG XS codestream           |  SEP counter=1
   |            (part 2049/q)              |  P counter=0
   :                                       :  M=0, K=0, L=0, I=00
   +---------------------------------------+
   :                                       :
   +---------------------------------------+
   |          JPEG XS codestream           |  SEP counter=(q-1) div 2048
   |             (part q/q)                |  P counter=(q-1) mod 2048
   :                                       :  M=1, K=0, L=1, I=00
   +=======================================+

           Figure 6: Example of JPEG XS Payload Data (Codestream
                Packetization Mode, Progressive Video Frame)
   +=====[ Packetization unit (PU) #1 ]====+
   |           Video support box           |  SEP counter=0
   +- - - - - - - - - - - - - - - - - - - -+  P counter=0
   |        Color specification box        |
   +- - - - - - - - - - - - - - - - - - - -+
   |     JPEG XS codestream (1st field)    |
   :             (part 1/q)                :  M=0, K=0, L=0, I=10
   +---------------------------------------+
   |     JPEG XS codestream (1st field)    |  SEP counter=0
   |             (part 2/q)                |  P counter=1
   :                                       :  M=0, K=0, L=0, I=10
   +---------------------------------------+
   :                                       :
   +---------------------------------------+
   |     JPEG XS codestream (1st field)    |  SEP counter=1
   |            (part 2049/q)              |  P counter=0
   :                                       :  M=0, K=0, L=0, I=10
   +---------------------------------------+
   :                                       :
   +---------------------------------------+
   |     JPEG XS codestream (1st field)    |  SEP counter=(q-1) div 2048
   |             (part q/q)                |  P counter=(q-1) mod 2048
   :                                       :  M=1, K=0, L=1, I=10
   +===============[ PU #2 ]===============+
   |           Video support box           |  SEP counter=0
   +- - - - - - - - - - - - - - - - - - - -+  P counter=0
   |        Color specification box        |
   +- - - - - - - - - - - - - - - - - - - -+
   |     JPEG XS codestream (2nd field)    |
   |             (part 1/q)                |
   :                                       :  M=0, K=0, L=0, I=11
   +---------------------------------------+
   |     JPEG XS codestream (2nd field)    |  SEP counter=0
   |             (part 2/q)                |  P counter=1
   :                                       :  M=0, K=0, L=0, I=11
   +---------------------------------------+
   :                                       :
   +---------------------------------------+
   |     JPEG XS codestream (2nd field)    |  SEP counter=(q-1) div 2048
   |             (part q/q)                |  P counter=(q-1) mod 2048
   :                                       :  M=1, K=0, L=1, I=11
   +=======================================+

           Figure 7: Example of JPEG XS Payload Data (Codestream
                Packetization Mode, Interlaced Video Frame)
   +===[ PU #1: JPEG XS Header segment ]===+
   |           Video support box           |  SEP counter=0x07FF
   +- - - - - - - - - - - - - - - - - - - -+  P counter=0
   |        Color specification box        |
   +- - - - - - - - - - - - - - - - - - - -+
   |      JPEG XS codestream header        |
   |  +---------------------------------+  |
   |  :  Markers and marker segments    :  |
   |  +---------------------------------+  |  M=0, T=0, K=1, L=1, I=00
   +==========[ PU #2: Slice #1 ]==========+
   |  +---------------------------------+  |  SEP counter=0
   |  |           SLH Marker            |  |  P counter=0
   |  +---------------------------------+  |
   |  :       Entropy Coded Data        :  |
   |  +---------------------------------+  |  M=0, T=0, K=1, L=1, I=00
   +==========[ PU #3: Slice #2 ]==========+
   |               Slice #2                |  SEP counter=1
   |              (part 1/q)               |  P counter=0
   :                                       :  M=0, T=0, K=1, L=0, I=00
   +---------------------------------------+
   |               Slice #2                |  SEP counter=1
   |              (part 2/q)               |  P counter=1
   :                                       :  M=0, T=0, K=1, L=0, I=00
   +---------------------------------------+
   :                                       :
   +---------------------------------------+
   |               Slice #2                |  SEP counter=1
   |              (part q/q)               |  P counter=q-1
   :                                       :  M=0, T=0, K=1, L=1, I=00
   +=======================================+
   :                                       :
   +========[ PU #N: Slice #(N-1) ]========+
   |             Slice #(N-1)              |  SEP counter=N-2
   |              (part 1/r)               |  P counter=0
   :                                       :  M=0, T=0, K=1, L=0, I=00
   +---------------------------------------+
   :                                       :
   +---------------------------------------+
   |             Slice #(N-1)              |  SEP counter=N-2
   |              (part r/r)               |  P counter=r-1
   :             + EOC marker              :  M=1, T=0, K=1, L=1, I=00
   +=======================================+
    Figure 8: Example of JPEG XS Payload Data (Slice Packetization Mode,
                          Progressive Video Frame)
   +====[ PU #1: JPEG XS Hdr segment 1 ]===+
   |           Video support box           |  SEP counter=0x07FF
   +- - - - - - - - - - - - - - - - - - - -+  P counter=0
   |        Color specification box        |
   +- - - - - - - - - - - - - - - - - - - -+
   |      JPEG XS codestream header 1      |
   |  +---------------------------------+  |
   |  :   Markers and marker segments   :  |
   |  +---------------------------------+  |  M=0, T=0, K=1, L=1, I=10
   +====[ PU #2: Slice #1 (1st field) ]====+
   |  +---------------------------------+  |  SEP counter=0
   |  |           SLH Marker            |  |  P counter=0
   |  +---------------------------------+  |
   |  :       Entropy Coded Data        :  |
   |  +---------------------------------+  |  M=0, T=0, K=1, L=1, I=10
   +====[ PU #3: Slice #2 (1st field) ]====+
   |              Slice #2                 |  SEP counter=1
   |             (part 1/q)                |  P counter=0
   :                                       :  M=0, T=0, K=1, L=0, I=10
   +---------------------------------------+
   |              Slice #2                 |  SEP counter=1
   |             (part 2/q)                |  P counter=1
   :                                       :  M=0, T=0, K=1, L=0, I=10
   +---------------------------------------+
   :                                       :
   +---------------------------------------+
   |              Slice #2                 |  SEP counter=1
   |             (part q/q)                |  P counter=q-1
   :                                       :  M=0, T=0, K=1, L=1, I=10
   +=======================================+
   :                                       :
   +==[ PU #N: Slice #(N-1) (1st field) ]==+
   |            Slice #(N-1)               |  SEP counter=N-2
   |             (part 1/r)                |  P counter=0
   :                                       :  M=0, T=0, K=1, L=0, I=10
   +---------------------------------------+
   :                                       :
   +---------------------------------------+
   |            Slice #(N-1)               |  SEP counter=N-2
   |             (part r/r)                |  P counter=r-1
   :            + EOC marker               :  M=1, T=0, K=1, L=1, I=10
   +=======================================+
   +===[ PU #N+1: JPEG XS Hdr segment 2 ]==+
   |           Video support box           |  SEP counter=0x07FF
   +- - - - - - - - - - - - - - - - - - - -+  P counter=0
   |        Color specification box        |
   +- - - - - - - - - - - - - - - - - - - -+
   |       JPEG XS codestream header 2     |
   |  +---------------------------------+  |
   |  :  Markers and marker segments    :  |
   |  +---------------------------------+  |  M=0, T=0, K=1, L=1, I=11
   +===[ PU #N+2: Slice #1 (2nd field) ]===+
   |  +---------------------------------+  |  SEP counter=0
   |  |           SLH Marker            |  |  P counter=0
   |  +---------------------------------+  |
   |  :      Entropy Coded Data         :  |
   |  +---------------------------------+  |  M=0, T=0, K=1, L=1, I=11
   +===[ PU #N+3: Slice #2 (2nd field) ]===+
   |               Slice #2                |  SEP counter=1
   |              (part 1/s)               |  P counter=0
   :                                       :  M=0, T=0, K=1, L=0, I=11
   +---------------------------------------+
   |               Slice #2                |  SEP counter=1
   |              (part 2/s)               |  P counter=1
   :                                       :  M=0, T=0, K=1, L=0, I=11
   +---------------------------------------+
   :                                       :
   +---------------------------------------+
   |               Slice #2                |  SEP counter=1
   |              (part s/s)               |  P counter=s-1
   :                                       :  M=0, T=0, K=1, L=1, I=11
   +=======================================+
   :                                       :
   +==[ PU #2N: Slice #(N-1) (2nd field) ]=+
   |             Slice #(N-1)              |  SEP counter=N-2
   |              (part 1/t)               |  P counter=0
   :                                       :  M=0, T=0, K=1, L=0, I=11
   +---------------------------------------+
   :                                       :
   +---------------------------------------+
   |             Slice #(N-1)              |  SEP counter=N-2
   |              (part t/t)               |  P counter=t-1
   :             + EOC marker              :  M=1, T=0, K=1, L=1, I=11
   +=======================================+
       Figure 9: Example of JPEG XS Payload Data (Slice Packetization
                       Mode, Interlaced Video Frame)

5. Traffic Shaping and Delivery Timing โœจ

Okay so, for senders and receivers to stay in sync ๐ŸŽ€, it is RECOMMENDED to do traffic shaping and delivery timing the way the Network Compatibility Model compliance definitions in [SMPTE2110-21] describe ๐Ÿ’…. If you go that route, the session description SHALL signal that compliance with the media type parameter TP ๐Ÿซถ. The actual shaping and timing delivery mechanism is outside the scope of this memo ๐Ÿ™ˆ, and it does not change how the payload gets packetized ๐Ÿฆ‹.

6. Congestion Control Considerations ๐Ÿšฆ

Congestion control for RTP SHALL be used following [RFC3550] and any applicable RTP profile, e.g., RTP/AVP [RFC3551] or RTP/AVPF [RFC4585] ๐Ÿ’–.

JPEG XS is mainly built for controlled network environments ๐Ÿข, but it can totally be employed on best-effort networks like the Internet ๐ŸŒ. Just know that in this case, the users of this payload format SHALL monitor packet loss so the packet loss rate stays within acceptable parameters ๐Ÿ“‰. One way to get there is, for example, RTP Control Protocol (RTCP) Feedback for Congestion Control [RFC8888] ๐Ÿงš.

Bestie, this part is important: packet loss monitoring is a must on best-effort networks. Don't skip it ๐Ÿ˜ญ.

Also, [RFC8083] is an update to [RFC3550] ๐Ÿ”„ that defines criteria for when you have to stop sending RTP Packet Streams, and for when applications implementing this standard SHALL comply with it ๐Ÿ™Œ.

[RFC8085] has extra info on best practices for applying congestion control to UDP streams ๐Ÿ“šโœจ.

7. Payload Format Parameters ๐Ÿ“‹

This section spells out the required and optional parameters for the payload format and/or the RTP stream. All parameters are declarative, which means the information the parameters signal is also present in the payload data itself ๐Ÿง . That lives in the payload header (see Section 4.3) or in the JPEG XS header segment [ISO21122-1] [ISO21122-3]. When parameters are provided, their respective values SHALL be consistent with the payload ๐Ÿ’œ.

7.1. Media Type Registration โœจ

Okay so this registration is done using the template in [RFC6838] and following [RFC4855]. This is where video/jxsv officially joins the media type club ๐Ÿ’…

The receiver SHALL ignore any unrecognized parameter. ๐Ÿ™ˆ

  • Type name: video
  • Subtype name: jxsv
  • Required parameters:
    • rate: The RTP timestamp clock rate. Applications using this payload format SHALL use a value of 90000. โฐ
    • packetmode: This is the configured packetization mode, as defined by the packetization mode (K) bit in the payload header of Section 4.3. This value SHALL be equal to the K-bit value configured in the RTP stream (so 0 for codestream or 1 for slice). ๐Ÿงฉ
  • Optional parameters:
    • transmode: This is the configured transmission mode, as defined by the Transmission mode (T) bit in the payload header of Section 4.3. If specified, this value SHALL be equal to the T-bit value configured in the RTP stream (so 0 for out-of-order-allowed or 1 for sequential-only). If not specified, a value 1 (sequential-only) SHALL be assumed and the T bit SHALL be set to 1. ๐Ÿ”
    • profile: The JPEG XS profile [ISO21122-2] in use. Any white space Unicode character in the profile name SHALL be omitted. Valid examples are 'Main444.12' or 'High444.12'. ๐ŸŽ€
    • level: The JPEG XS level [ISO21122-2] in use. Any white space Unicode character in the level name SHALL be omitted. Valid examples are '2k-1' or '4k-2'. ๐Ÿ“
    • sublevel: The JPEG XS sublevel [ISO21122-2] in use. Any white space Unicode character in the sublevel name SHALL be omitted. Valid examples are 'Sublev3bpp' or 'Sublev6bpp'. ๐ŸŒธ
    • depth: Determines the number of bits per sample. This is an integer, with typical values including 8, 10, 12, and 16. ๐ŸŒˆ
    • width: Determines the number of pixels per line. This is an integer between 1 and 32767, inclusive. ๐Ÿ“
    • height: Determines the number of lines per video frame. This is an integer between 1 and 32767, inclusive. ๐Ÿ“
    • exactframerate: Signals the video frame rate in frames per second. Integer frame rates SHALL be signaled as a single decimal number (for example, "25"), whilst non-integer frame rates SHALL be signaled as a ratio of two integer decimal numbers separated by a forward-slash character (for example, "30000/1001"), using the numerically smallest numerator value possible. ๐ŸŽฌ
    • interlace: If this parameter name is present, it indicates that the video is interlaced, or that the video is Progressive segmented Frame (PsF). If this parameter name is not present, the progressive video format SHALL be assumed. ๐ŸชŸ
    • segmented: If this parameter name is present, and the interlace parameter name is also present, then the video is a Progressive segmented Frame (PsF). Signaling this parameter without the interlace parameter is forbidden. ๐Ÿšซ
    • sampling: Signals the color difference signal sub-sampling structure.
      • Signals utilizing the non-constant luminance Y'C'B C'R signal format of [BT.601-7], [BT.709-6], [BT.2020-2], or [BT.2100-2] SHALL use the appropriate one of these values for the Media Type Parameter "sampling":
        • YCbCr-4:4:4 (4:4:4 sampling)
        • YCbCr-4:2:2 (4:2:2 sampling)
        • YCbCr-4:2:0 (4:2:0 sampling)
      • Signals utilizing the Constant Luminance Y'C C'BC C'RC signal format of [BT.2020-2] SHALL use the appropriate one of these values for the Media Type Parameter "sampling":
        • CLYCbCr-4:4:4 (4:4:4 sampling)
        • CLYCbCr-4:2:2 (4:2:2 sampling)
        • CLYCbCr-4:2:0 (4:2:0 sampling)
      • Signals utilizing the constant intensity I CT CP signal format of [BT.2100-2] SHALL use the appropriate one of these values for the Media Type Parameter "sampling":
        • ICtCp-4:4:4 (4:4:4 sampling)
        • ICtCp-4:2:2 (4:2:2 sampling)
        • ICtCp-4:2:0 (4:2:0 sampling)
      • Signals utilizing the 4:4:4 R' G' B' or RGB signal format (such as that of [BT.601-7], [BT.709-6], [BT.2020-2], [BT.2100-2], [SMPTE2065-1], or [SMPTE2065-3]) SHALL use this value for the Media Type Parameter "sampling":
        • RGB (RGB or R' G' B' samples)
      • Signals utilizing the 4:4:4 X' Y' Z' signal format (such as defined in [SMPTE428-1]) SHALL use this value for the Media Type Parameter "sampling":
        • XYZ (X' Y' Z' samples)
      • Key signals as defined in [SMPTE157] SHALL use the value key for the Media Type Parameter "sampling". The key signal is represented as a single component:
        • KEY (Samples of the key signal)
      • Signals utilizing a color sub-sampling other than what is defined here SHALL use this value for the Media Type Parameter "sampling":
        • UNSPECIFIED (Sampling signaled by the payload)
    • colorimetry: Specifies the system colorimetry used by the image samples. Valid values and their specifications are:
      • BT601-5: [BT.601-5].
      • BT709-2: [BT.709-2].
      • SMPTE240M: [SMPTE240M].
      • BT601: [BT.601-7].
      • BT709: [BT.709-6].
      • BT2020: [BT.2020-2].
      • BT2100: [BT.2100-2], Table 2 titled "System colorimetry".
      • ST2065-1: Academy Color Encoding Specification (ACES) [SMPTE2065-1].
      • ST2065-3: Academy Density Exchange Encoding (ADX) [SMPTE2065-3].
      • XYZ: [ISO11664-1], section titled "1931 Observer".
      • UNSPECIFIED: Colorimetry is signaled in the payload by the color specification box of [ISO21122-3], or it must be manually coordinated between sender and receiver.

Signals utilizing the [BT.2100-2] colorimetry SHOULD also signal the representational range using the optional parameter RANGE defined below. Signals utilizing the UNSPECIFIED colorimetry might require manual coordination between the sender and the receiver. ๐ŸŽจ

  • TCS: Transfer Characteristic System. This parameter specifies the transfer characteristic system of the image samples. Valid values and their specifications are:
    • SDR: Standard Dynamic Range video streams that utilize the Optical Electrical Transfer Function (OETF) of [BT.709-6] or [BT.2020-2]. Such streams SHALL be assumed to target the Electro-Optical Transfer Function (EOTF) specified in [BT.1886-0].
    • PQ: High dynamic range video streams that utilize the Perceptual Quantization system of [BT.2100-2].
    • HLG: High dynamic range video streams that utilize the Hybrid Log-Gamma system of [BT.2100-2].
    • UNSPECIFIED: Video streams whose transfer characteristics are signaled by the payload as specified in [ISO21122-3], or that must be manually coordinated between sender and receiver. โœจ
  • RANGE: This parameter SHOULD be used to signal the encoding range of the sample values within the stream. When paired with [BT.2100-2] colorimetry, this parameter has two allowed values, NARROW and FULL, corresponding to the ranges specified in TABLE 9 of [BT.2100-2]. In any other context, this parameter has three allowed values, NARROW, FULLPROTECT, and FULL, which correspond to the ranges specified in [SMPTE2077]. In the absence of this parameter, and for all but the UNSPECIFIED colorimetry, NARROW SHALL be the assumed value. When paired with the UNSPECIFIED colorimetry, FULL SHALL be the default assumed value. ๐ŸŽž๏ธ
  • Encoding considerations: This media type is framed in RTP and contains binary data; see Section 4.8 of [RFC6838].
  • Security considerations: See the Security Considerations section of RFC 9134. ๐Ÿ”
  • Interoperability considerations: None
  • Published specification: See the References section of RFC 9134.
  • Applications that use this media type: Any application that transmits video over RTP (like SMPTE ST 2110). ๐ŸŽฅ
  • Fragment identifier considerations: N/A
  • Additional information: None
  • Person & email address to contact for further information: T. Bruylants <rtp@intopix.com> and T. Richter <jpeg-xs-techsupport@iis.fraunhofer.de>. ๐Ÿ“ฌ
  • Intended usage: COMMON
  • Restrictions on usage: This media type depends on RTP framing; hence, it is only defined for transfer via RTP [RFC3550]. ๐Ÿšฆ
  • Author: See the Authors' Addresses section of RFC 9134.
  • Change controller: IETF Audio/Video Transport Working Group delegated from the IESG.

8. SDP Parameters ๐ŸŽ€

A mapping of the parameters into the Session Description Protocol (SDP) [RFC8866] is provided for applications that use SDP. ๐Ÿ’Œ

8.1. Mapping of Payload Type Parameters to SDP

The media type video/jxsv string is mapped to fields in the Session Description Protocol (SDP) [RFC8866] as follows:

  • The media type ("video") goes in SDP m= as the media name.
  • The media subtype ("jxsv") goes in SDP a=rtpmap as the encoding name, followed by a slash (/) and the required parameter "rate", corresponding to the RTP timestamp clock rate (which for the payload format defined in this document SHALL be 90000).
  • The required parameter "packetmode" and any of the additional optional parameters, as described in Section 7.1, go in the SDP media format description, being the a=fmtp attribute (Format Parameters), by copying them directly from the media type string as a semicolon-separated list of parameter=value pairs.

All parameters of the media format SHALL correspond to the parameters of the payload. In case of discrepancies between payload parameter values and SDP fields, the values from the payload data SHALL prevail. ๐Ÿง

The receiver SHALL ignore any parameter that is not defined in Section 7.1.

An example SDP mapping for JPEG XS video is as follows:

m=video 30000 RTP/AVP 112
a=rtpmap:112 jxsv/90000
a=fmtp:112 packetmode=0;sampling=YCbCr-4:2:2;
           width=1920;height=1080;depth=10;
           colorimetry=BT709;TCS=SDR;RANGE=FULL;TP=2110TPNL

In this example, a JPEG XS RTP stream is to be sent to UDP destination port 30000, with an RTP dynamic payload type of 112 and a media clock rate of 90000 Hz. Note that the a=fmtp: line has been wrapped to fit this page and will be a single long line in the SDP file. This example includes the TP parameter (as specified in Section 5). ๐Ÿฆ‹

8.2. Usage with SDP Offer/Answer Model

okay so, when you offer JPEG XS over RTP using SDP in the offer/answer model [RFC3264] for unicast negotiation, there are some limitations and rules you need to vibe with ๐Ÿ’…

  • The a=fmtp attribute SHALL be present specifying the required parameter packetmode, and it MAY specify any of the optional parameters, as described in Section 7.1. ๐ŸŽ€
  • All parameters in the a=fmtp attribute indicate sending capabilities (i.e., properties of the payload). ๐Ÿฆ‹
  • An answerer of the SDP is required to support all parameters and values of the parameters provided by the offerer; otherwise, the answerer SHALL reject the session. It falls on the offerer to use values that are expected to be supported by the answerer. If the answerer accepts the session, it SHALL reply with the exact same parameter values in the a=fmtp attribute as they were initially offered. ๐Ÿ˜ญ
  • The same RTP payload type number used in the offer SHOULD be used in the answer, as specified in [RFC3264]. ๐Ÿซถ

9. IANA Considerations ๐ŸŒˆ

IANA has registered the media type registration video/jxsv as specified in Section 7.1. โœจ The media type has also been added to the IANA registry for "RTP Payload Format Media Types" <https://www.iana.org/assignments/rtp-parameters>. ๐ŸŒธ

10. Security Considerations ๐Ÿ’€

RTP packets using this payload format are subject to the security considerations discussed in [RFC3550] and in any applicable RTP profile such as RTP/AVP [RFC3551], RTP/AVPF [RFC4585], RTP/SAVP [RFC3711], or RTP/SAVPF [RFC5124]. This implies that confidentiality of the media streams is achieved by encryption. ๐Ÿ”

However, as "Securing the RTP Framework: Why RTP Does Not Mandate a Single Media Security Solution" [RFC7202] discusses, it is not an RTP payload format's responsibility to discuss or mandate what solutions are used to meet the basic security goals like confidentiality, integrity, and source authenticity for RTP in general. That responsibility lies on anyone using RTP in an application. They can find guidance on available security mechanisms and important considerations in "Options for Securing RTP Sessions" [RFC7201]. Applications SHOULD use one or more appropriate strong security mechanisms. ๐Ÿ›ก๏ธ

Implementations of this RTP payload format need to take appropriate security considerations into account. It is important for the decoder to be robust against malicious or malformed payloads and to ensure that they do not cause the decoder to overrun its allocated memory or otherwise misbehave. An overrun in allocated memory could lead to arbitrary code execution by an attacker. ๐Ÿ˜ฑ The same applies to the encoder, even though problems in encoders are typically rarer.

This payload format and the JPEG XS encoding do not exhibit any substantial non-uniformity, either in output or in complexity to perform the decoding operation; thus, they are unlikely to pose a denial-of-service threat due to the receipt of pathological datagrams. ๐Ÿ™Œ

This payload format and the JPEG XS encoding do not contain code that is executable. ๐Ÿงš

It is important to note that high-definition (HD) or ultra-high-definition (UHD) video that is encoded with JPEG XS can have significant bandwidth requirements (typically more than 1 Gbps for UHD video, especially if using high framerate). This is sufficient to cause potential for denial of service if transmitted onto most currently available Internet paths. ๐Ÿšจ

Accordingly, if best-effort service is being used, users of this payload format SHALL monitor packet loss to ensure that the packet loss rate is within acceptable parameters. Packet loss is considered acceptable if a TCP flow across the same network path, and experiencing the same network conditions, would achieve an average throughput, measured on a reasonable timescale, that is not less than the RTP flow is achieving. This condition can be satisfied by implementing congestion control mechanisms to adapt the transmission rate (or the number of layers subscribed for a layered multicast session) or by arranging for a receiver to leave the session if the loss rate is unacceptably high. ๐Ÿ“‰

This payload format may also be used in networks that provide quality-of-service guarantees. If enhanced service is being used, receivers SHOULD monitor packet loss to ensure that the service that was requested is actually being delivered. If it is not, then they SHOULD assume that they are receiving best-effort service and behave accordingly. ๐Ÿ‘‘

11. References ๐Ÿ“š

11.1. Normative References

[ISO21122-1]
           ISO/IEC, "Information technology - JPEG XS low-latency
           lightweight image coding system - Part 1: Core coding
           system", ISO/IEC IS 21122-1.

[ISO21122-2]
           ISO/IEC, "Information technology - JPEG XS low-latency
           lightweight image coding system - Part 2: Profiles and
           buffer models", ISO/IEC IS 21122-2.

[ISO21122-3]
           ISO/IEC, "Information technology - JPEG XS low-latency
           lightweight image coding system - Part 3: Transport and
           container formats", ISO/IEC IS 21122-3.

[RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
           Requirement Levels", BCP 14, RFC 2119,
           DOI 10.17487/RFC2119, March 1997,
           <https://www.rfc-editor.org/info/rfc2119>.

[RFC3264]  Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model
           with Session Description Protocol (SDP)", RFC 3264,
           DOI 10.17487/RFC3264, June 2002,
           <https://www.rfc-editor.org/info/rfc3264>.

[RFC3550]  Schulzrinne, H., Casner, S., Frederick, R., and V.
           Jacobson, "RTP: A Transport Protocol for Real-Time
           Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550,
           July 2003, <https://www.rfc-editor.org/info/rfc3550>.

[RFC3551]  Schulzrinne, H. and S. Casner, "RTP Profile for Audio and
           Video Conferences with Minimal Control", STD 65, RFC 3551,
           DOI 10.17487/RFC3551, July 2003,
           <https://www.rfc-editor.org/info/rfc3551>.

[RFC4855]  Casner, S., "Media Type Registration of RTP Payload
           Formats", RFC 4855, DOI 10.17487/RFC4855, February 2007,
           <https://www.rfc-editor.org/info/rfc4855>.

[RFC6838]  Freed, N., Klensin, J., and T. Hansen, "Media Type
           Specifications and Registration Procedures", BCP 13,
           RFC 6838, DOI 10.17487/RFC6838, January 2013,
           <https://www.rfc-editor.org/info/rfc6838>.

[RFC8083]  Perkins, C. and V. Singh, "Multimedia Congestion Control:
           Circuit Breakers for Unicast RTP Sessions", RFC 8083,
           DOI 10.17487/RFC8083, March 2017,
           <https://www.rfc-editor.org/info/rfc8083>.

[RFC8085]  Eggert, L., Fairhurst, G., and G. Shepherd, "UDP Usage
           Guidelines", BCP 145, RFC 8085, DOI 10.17487/RFC8085,
           March 2017, <https://www.rfc-editor.org/info/rfc8085>.

[RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
           2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
           May 2017, <https://www.rfc-editor.org/info/rfc8174>.

[RFC8866]  Begen, A., Kyzivat, P., Perkins, C., and M. Handley, "SDP:
           Session Description Protocol", RFC 8866,
           DOI 10.17487/RFC8866, January 2021,
           <https://www.rfc-editor.org/info/rfc8866>.

11.2. Informative References ๐Ÿ“š

These ones aren't strictly required reading, but they're the receipts ๐Ÿ’… Here's the full list, exactly as it's written in the spec:

[BT.1886-0]
           ITU-R, "Reference electro-optical transfer function for
           flat panel displays used in HDTV studio production", ITU-R
           Recommendation BT.1886-0, March 2011,
           <https://www.itu.int/rec/R-REC-BT.1886-0-201103-I/en>.

[BT.2020-2]
           ITU-R, "Parameter values for ultra-high definition
           television systems for production and international
           programme exchange", ITU-R Recommendation BT.2020-2,
           October 2015,
           <https://www.itu.int/rec/R-REC-BT.2020-2-201510-I/en>.

[BT.2100-2]
           ITU-R, "Image parameter values for high dynamic range
           television for use in production and international
           programme exchange", ITU-R Recommendation BT.2100-2, July
           2018,
           <https://www.itu.int/rec/R-REC-BT.2100-2-201807-I/en>.

[BT.601-5] ITU-R, "Studio encoding parameters of digital television
           for standard 4:3 and wide screen 16:9 aspect ratios",
           ITU-R Recommendation BT.601-5, October 1995,
           <https://www.itu.int/rec/R-REC-BT.601-5-199510-S/en>.

[BT.601-7] ITU-R, "Studio encoding parameters of digital television
           for standard 4:3 and wide screen 16:9 aspect ratios",
           ITU-R Recommendation BT.601-7, March 2011,
           <https://www.itu.int/rec/R-REC-BT.601-7-201103-I/en>.

[BT.709-2] ITU-R, "Parameter values for the HDTV standards for
           production and international programme exchange", ITU-R
           Recommendation BT.709-2, October 1995,
           <https://www.itu.int/rec/R-REC-BT.709-2-199510-S/en>.

[BT.709-6] ITU-R, "Parameter values for the HDTV standards for
           production and international programme exchange", ITU-R
           Recommendation BT.709-6, June 2015,
           <https://www.itu.int/rec/R-REC-BT.709-6-201506-I/en>.

[ISO11664-1]
           ISO/CIE, "Colorimetry - Part 1: CIE standard colorimetric
           observers", ISO/CIE IS 11664-1:2019, June 2019,
           <https://www.iso.org/standard/74164.html>.

[RFC3711]  Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K.
           Norrman, "The Secure Real-time Transport Protocol (SRTP)",
           RFC 3711, DOI 10.17487/RFC3711, March 2004,
           <https://www.rfc-editor.org/info/rfc3711>.

[RFC4175]  Gharai, L. and C. Perkins, "RTP Payload Format for
           Uncompressed Video", RFC 4175, DOI 10.17487/RFC4175,
           September 2005, <https://www.rfc-editor.org/info/rfc4175>.

[RFC4585]  Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey,
           "Extended RTP Profile for Real-time Transport Control
           Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585,
           DOI 10.17487/RFC4585, July 2006,
           <https://www.rfc-editor.org/info/rfc4585>.

[RFC5124]  Ott, J. and E. Carrara, "Extended Secure RTP Profile for
           Real-time Transport Control Protocol (RTCP)-Based Feedback
           (RTP/SAVPF)", RFC 5124, DOI 10.17487/RFC5124, February
           2008, <https://www.rfc-editor.org/info/rfc5124>.

[RFC7201]  Westerlund, M. and C. Perkins, "Options for Securing RTP
           Sessions", RFC 7201, DOI 10.17487/RFC7201, April 2014,
           <https://www.rfc-editor.org/info/rfc7201>.

[RFC7202]  Perkins, C. and M. Westerlund, "Securing the RTP
           Framework: Why RTP Does Not Mandate a Single Media
           Security Solution", RFC 7202, DOI 10.17487/RFC7202, April
           2014, <https://www.rfc-editor.org/info/rfc7202>.

[RFC8888]  Sarker, Z., Perkins, C., Singh, V., and M. Ramalho, "RTP
           Control Protocol (RTCP) Feedback for Congestion Control",
           RFC 8888, DOI 10.17487/RFC8888, January 2021,
           <https://www.rfc-editor.org/info/rfc8888>.

[SMPTE157] SMPTE, "SMPTE Recommended Practice - Key and Alpha
           Signals", SMPTE RP 157:2012,
           DOI 10.1109/ICPST.1998.729044, November 2012,
           <https://ieeexplore.ieee.org/document/7290447>.

[SMPTE2065-1]
           SMPTE, "SMPTE Standard - Academy Color Encoding
           Specification (ACES)", SMPTE ST 2065-1:2021,
           DOI 10.5594/SMPTE.ST2065-1.2021, January 2021,
           <https://ieeexplore.ieee.org/document/9343931>.

[SMPTE2065-3]
           SMPTE, "SMPTE Standard - Academy Density Exchange Encoding
           (ADX) - Encoding Academy Printing Density (APD) Values",
           SMPTE ST 2065-3:2020, DOI 10.5594/SMPTE.ST2065-3.2020,
           November 2020,
           <https://ieeexplore.ieee.org/document/9286953>.

[SMPTE2077]
           SMPTE, "SMPTE Recommended Practice - Full-Range Image
           Mapping", SMPTE RP 2077:2013,
           DOI 10.5594/SMPTE.RP2077.2013, November 2013,
           <https://ieeexplore.ieee.org/document/7290588>.

[SMPTE2110-21]
           SMPTE, "SMPTE Standard - Professional Media Over Managed
           IP Networks: Traffic Shaping and Delivery Timing for
           Video", SMPTE ST 2110-21:2017,
           DOI 10.5594/SMPTE.ST2110-21.2017, November 2017,
           <https://ieeexplore.ieee.org/document/8165971>.

[SMPTE240M]
           SMPTE, "SMPTE Standard - For Television - 1125-Line High-
           Definition Production Systems - Signal Parameters",
           SMPTE ST 240M:1999, DOI 10.5594/SMPTE.ST240.1999, November
           1999, <https://ieeexplore.ieee.org/
           document/7291461?arnumber=7291461>.

[SMPTE428-1]
           SMPTE, "SMPTE Standard - D-Cinema Distribution Master -
           Image Characteristics", SMPTE ST 428-1:2019,
           DOI 10.5594/SMPTE.ST428-1.2019, March 2019,
           <https://ieeexplore.ieee.org/document/8709077>.

Acknowledgments ๐Ÿซถ

The authors wanted to give a huge shoutout to the people who made real contributions to this memo: Sรฉbastien Lugan, Arnaud Germain, Alexandre Willรจme, Gaรซl Rouvroy, Siegfried Foessel, and Jean-Baptise Lorent. Thank you, queens ๐Ÿ‘‘โœจ

Authors' Addresses ๐Ÿ’Œ

Tim Bruylants ๐ŸŒธ

Tim Bruylants
intoPIX S.A.
Rue Emile Francqui, 9
1435 Mont-Saint-Guibert
Belgium

Phone: +32 10 23 84 70
Email: t.bruylants@intopix.com
URI:   https://www.intopix.com/

Antonin Descampe ๐Ÿฆ‹

Antonin Descampe
Universitรฉ catholique de Louvain
bte L2.03.02
Ruelle de la Lanterne Magique, 14
1348 Louvain-la-Neuve
Belgium

Phone: +32 10 47 27 87
Email: antonin.descampe@uclouvain.be
URI:   https://uclouvain.be/antonin.descampe

Corentin Damman ๐ŸŽ€

Corentin Damman
intoPIX S.A.
Rue Emile Francqui, 9
1435 Mont-Saint-Guibert
Belgium

Phone: +32 10 23 84 70
Email: c.damman@intopix.com
URI:   https://www.intopix.com/

Thomas Richter ๐Ÿ’œ

Thomas Richter
Fraunhofer IIS
Am Wolfsmantel 33
91048 Erlangen
Germany

Phone: +49 9131 776 5126
Email: thomas.richter@iis.fraunhofer.de
URI:   https://www.iis.fraunhofer.de/
๐Ÿ“ฆ where this page comes from