RFC 2431
RTP Payload Format for BT.656 Video Encoding
RTP Payload Format for BT.656 Video Encoding 🎀
Okay bestie, let's start with the paperwork 💅 This is RFC 2431, from the Network Working Group, written by D. Tynan of Claddagh Films and dated October 1998. It's on the Standards Track. The title block doesn't list anything it obsoletes or updates, so it isn't replacing or patching any older RFC 🦋
Status of This Memo 📋
So this is an Internet standards track protocol for the whole Internet community, and they're literally asking for discussion and suggestions 🗣️ If you want the current standardization state and status, check the "Internet Official Protocol Standards" (STD 1) document 📚 And distribution of this memo is unlimited, so go wild 🌈
Copyright Notice ©
Copyright (C) The Internet Society (1998). All Rights Reserved.
Abstract ✨
This doc is all about the RTP payload format for wrapping up ITU Recommendation BT.656-3 video streams inside the Real-Time Transport Protocol (RTP). Each RTP packet holds all or part of one scan line, as defined by ITU Recommendation BT.601-5, plus fragmentation, decoding and positioning info. It's giving organized 💖
1. Introduction 🌸
Okay so this document describes a way to packetize uncompressed, studio-quality video streams (as defined by BT.656) so they can travel over RTP [1]. A BT.656 video stream is defined by ITU-R Recommendation BT.656-3 [2]. It's a way of connecting digital television equipment that runs on the 525-line or 625-line standards and follows the 4:2:2 encoding parameters from ITU-R Recommendation BT.601-5 (formerly CCIR-601) [3], Part A. Iconic 📺
RTP is defined by the Internet Engineering Task Force (IETF) to provide end-to-end network transport functions for applications that send real-time data over multicast or unicast network services. To fully specify RTP for a particular application, you need the RTP protocol document [1], a profile specification document [4], and a payload format specification. This document is meant to be the payload format specification for studio-quality video streams 🎬
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [5]. Read those carefully, they're the rules 🫶
2. Definitions 📖
For the purposes of this document, the following definitions apply:
- Y: An 8-bit or 10-bit coded "luminance" sample. In this context, "luminance" follows the BT.601-5 [3] definition, which is not the same as a true CIE luminance value. The value of "luminance" refers specifically to video luma. To avoid confusion with the BT.656 and BT.601 standards, the video luma value is called luminance in this document. Each value has 220 quantization levels, with the black level at level 16 and the peak white level at 235. 🖤🤍
- Cb, Cr: An 8-bit or 10-bit coded color-difference sample (as per BT.601-5). Each color-difference value has 225 quantization levels in the centre part of the quantization scale. A color-difference of zero has an encoded value of 128. 🌈
- True Black: BT.601-5 defines a true black level as the quad-sample sequence
0x80, 0x10, 0x80, 0x10, which represents color-difference values of 128 (0x80) and a luminance value of 16 (0x10). ⬛ - SAV, EAV: Video timing reference codes that appear at the start and end of a BT.656 scan line. ⏱️
3. Payload Design 🎀
ITU Recommendation BT.656-3 defines a schema for digitally interconnecting television video signals. It works together with BT.601-5, which defines the digital representation of the original analog signal. BT.601-5 covers images with or without color subsampling, but the interconnection standard (BT.656-3) specifically requires 4:2:2 subsampling. This specification also requires 4:2:2 subsampling, so the luminance stream gets twice the bandwidth of each of the two color-difference streams. For normal 4:3 aspect ratio images, that works out to 720 luminance samples per scan line and 360 samples for each of the two chrominance channels. The total number of samples per scan line in this case is 1440. This payload format can accommodate various image sizes and frame rates, but only the ones that follow BT.601-5 are currently supported. Slightly picky, but it matters 🙈
Because there's no video compression in the payload, and the sampling rate complies with BT.601-5, the resulting video stream can be considered "studio quality" ✨ The catch is that such a stream can need around 20 megabytes per second of network bandwidth 😭 To maximize packet size within a given MTU and to optimize scan line decoding, each video scan line gets encoded within one or more RTP packets. Smart 🧠
To allow for scan line synchronization, each packet includes certain flag bits (as defined in BT.656-3) and a unique scan line number. The SAV and EAV timing reference codes are removed. On top of that, no line blanking samples are included, so no ancillary data can ride in the line blanking period. The receiver is responsible for generating the timing reference codes and inserting the correct number of line blanking samples. Receiver's job, not the sender's 💅
Similarly, there's no requirement that the frame blanking samples be provided. However, you can include them if they carry relevant information, like a vertical-interlace time code (VITC) or teletext data. If there are no frame blanking samples, the receiver MUST generate true black levels, as defined above, to complete the correct number of scan lines per field. If frame blanking samples are provided, they MUST be copied without modification into the resulting BT.656-3 stream. No editing the blanking samples, that's the rule 🚫✏️
Scan lines MUST be sent in sequential order. Error concealment for missing scan lines or fragments of scan lines is up to the receiver's discretion 🫠
Both 8-bit and 10-bit quantization types, as defined by BT.601-5, are supported. 10-bit samples are treated as having two extra bits of fixed-point precision, so a binary value of 10111110.11 represents a sample value of 190.75. Using 8-bit quantization, that same sample would be 190. An application receiving 8-bit samples for a 10-bit device MUST consider the sample as reflecting the most-significant 8 bits. The two least-significant bits SHOULD be set to zero. In the other direction, an application sending 8-bit samples from a 10-bit device MUST drop the two least-significant bits. For a 10-bit quantization payload, each pair of samples MUST be encoded into a 40-bit word (five octets) before transmission, as specified in Section 6. Bit-level stuff, but it's so cute once it clicks 🧚
To allow for scan lines with octet lengths larger than the path maximum transmission unit (MTU), a scan offset field is included in the packet header. Applications SHOULD attempt path MTU discovery [6] and fragment scan lines into multiple packets that are no larger than the MTU. Big lines, small packets, no drama 📦
Fragmentation MUST occur on a sample-pair boundary, so the chrominance and luminance values never get split across packets. For 8-bit quantization this gives a four-octet alignment, and a five-octet alignment for 10-bit quantization. Because of that, the scan offset refers not to the byte offset within the payload, but to the sample-pair offset. Pairs stay together, always 🫶
4. Usage of RTP 🦋
Okay this part is so relatable, because RTP is unreliable and has no orderly delivery mechanism. So each packet has to carry enough info to form a single scan line without relying on prior scan lines or prior frames. On top of the RTP header, every packet includes a fixed-length payload header that is four octets long. Here's the layout:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| RTP Header |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Payload Header |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Payload Data |
| . |
| . |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+4.1. RTP Header usage 💜
Each RTP packet starts with a fixed RTP header. These are the fields of the RTP fixed header used for BT.656-3 encapsulation:
- Marker bit (M): The Marker bit of the RTP header is set to 1 for the last packet of a frame (or the last fragment of the last scan line, if fragmented). It's set to 0 on all other packets. Ending signal, very clear 🏁
- Payload Type (PT): The Payload Type indicates the use of the payload format defined in this document. A profile MAY assign a payload type value for this format either statically or dynamically, as described in RFC 1890 [4]. Flexible, which I love 🌷
- Timestamp: The RTP Timestamp encodes the sampling instant of the video frame currently being rendered. All scan line packets within the same frame have the same timestamp. The timestamp SHOULD refer to the 'Ov' field synchronization point of the first field. For the payload format defined by this document, the RTP timestamp is based on a 90kHz clock. Time is everything ⏰
That's the end of part 1 🎀 Part 2 picks up right after Section 4.1, so stay tuned, bestie 💖
5. Payload Header ✨
okay so the payload header is the main character 💅 it's a fixed four-octet header (that's four bytes, bestie) and it's laid out like this:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|F|V| Type |P| Z | Scan Line (SL) | Scan Offset (SO) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+F: 1 bit 🎀 When 0, it tells you this is the first field of a frame. For Type=0 or 2, that's lines 4 through 265 inclusive, and for Type=1 or 3, it's lines 1 through 312 inclusive. Any other scan line counts as part of the second field, and the F bit gets set to 1 for those. The transmitter copies this bit straight from the BT.656-compliant stream, and the receiver puts it back into the stream. 🦋
V: 1 bit 🌸 When 1, the scan line is part of the vertical interval. This should always be 0 unless frame blanking data is sent. If it is, the V bit SHOULD be set to 1 for scan lines that aren't an integral part of the image. The transmitter copies this bit straight from the BT.656-compliant stream, and the receiver inserts it. For receivers that don't receive scan lines during the vertical interval, BT.656 vertical interval data MUST be generated by repeating the quad-sample sequence 0x80, 0x10, 0x80, 0x10, which represents a true black level. 🖤
Type: 4 bits 🍓 This field tells you what type of frame encoding is inside the payload. It MUST remain unchanged for all scan lines within the same frame. Right now only four types are defined:
0- NTSC (13.5MHz sample rate; 720 samples per line; 60 fields per second; 525 lines per frame)1- PAL (13.5MHz sample rate; 720 samples per line; 50 fields per second; 625 lines per frame)2- High Definition NTSC (18MHz sample rate; 1144 samples per line; 60 fields per second; 525 lines per frame)3- High Definition PAL (18MHz sample rate; 1152 samples per line; 50 fields per second; 625 lines per frame)
Any further encodings can only be defined through the Internet Assigned Numbers Authority (IANA). For more info, check out Section 8, "IANA Considerations". 🧚
P: 1 bit 💜 This one sets the sample quantization size. When 0, the payload is made of 8-bit samples. Otherwise, it carries 10-bit samples. This bit MUST remain unchanged for all scan lines within the same frame. 🔥
Z: 2 bits 🌈 Reserved for future use. Must be set to zero by the transmitter and ignored by the receiver. Future-proofing, iconic 🫶
Scan Line (SL): 12 bits ⭐ This tells you which scan line is packed in the payload. Valid values run from 1 through 625 inclusive. If no frame blanking data is being transmitted, only scan lines 23 through 310 inclusive, and lines 336 through 623 inclusive, SHOULD be sent in the case of Type=1 or 3. For 525/60 encoding (Type=0 or 2), scan lines 10 through 263 inclusive and lines 273 through 525 SHOULD be transmitted. 👑
If a receiver is generating a BT.656-3 data stream directly from this packet, the F and V bits MUST be copied from the header rather than being generated implicitly from the scan line number. If there's ever a conflict, the F and V bits have precedence. Don't let the scan line number outvote the flags, bestie 🚫
Scan Offset (SO): 11 bits 🍭 This is the offset within the scan line for application-level fragmentation. After doing PMTU discovery, if the path MTU is smaller than what one complete scan line needs, the data SHOULD be fragmented so that a given RTP packet doesn't go over the allowable MTU. The offset for the first packet of a scan line MUST be set to zero. The scan offset counts sample pairs within the scan, so for a scan line width of 720, the maximum scan offset is 359. 📏
6. Payload Format 🎨
okay so BT.656 and BT.601 use 4:2:2 color subsampling, and this payload keeps that energy 💖 each pair of color-difference samples gets intermixed with two luminance samples. As per BT.656, the transmission format SHALL be Cb, Y, Cr, Y. Here's what a 720 sample packet with 8-bit quantization looks like:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Cb0 | Y0 | Cr0 | Y1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Cb1 | Y2 | Cr1 | Y3 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
.
.
.
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Cb359 | Y718 | Cr359 | Y719 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+1144 and 1152 sample packets SHOULD increase the packet size accordingly while maintaining the sample order. 📦
For 10-bit quantization, every group of four samples MUST be encoded into a 40-bit word (five octets) before transmission. The sample order is identical to 8-bit quantization. Here's a 720 sample packet with 10-bit quantization:
0 1 2 3
0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8
+---------+---------+---------+---------+
| Cb0 | Y0 | Cr0 | Y1 |
+---------+---------+---------+---------+
| Cb1 | Y2 | Cr1 | Y3 |
+---------+---------+---------+---------+
.
.
.
+---------+---------+---------+---------+
| Cb359 | Y718 | Cr359 | Y719 |
+---------+---------+---------+---------+
(Note that the word width is 40 bits)
+-------+-------+-------+-------+-------+
Octets: | 0 | 1 | 2 | 3 | 4 |
+-------+-------+-------+-------+-------+The octets in these diagrams are transmitted in network byte order, which is left-to-right as shown. 🧠
7. Security Considerations 🔐
Okay, this section is serious, so read it carefully 🚨
RTP packets using the payload format in this spec are subject to the security considerations in the RTP specification [1]. That means confidentiality of the media streams is achieved by encryption. Because the payload format is arranged end-to-end, encryption MAY be performed after encapsulation, so there's no conflict between the two operations. 🔒
This payload type doesn't show any significant non-uniformity in receiver-side computational complexity for packet processing, so it isn't a potential denial-of-service threat. 💅
8. IANA Considerations 📋
The four encoding types defined by this document relate to a specific schema defined by ITU-R Recommendation BT.656-3. Future revisions of the recommendation may create more encoding types that need to be supported over RTP. The "Type" field is four bits wide, which allows up to sixteen possible encodings, with twelve currently reserved for future use. Since there are only so many possible encodings, and it's very unlikely that future revisions of BT.656 will add any new schema, requests to extend the Type field MUST be vetted by the Internet Assigned Numbers Authority. Furthermore, implementors SHOULD check the IANA repository for new definitions of the Type field in order to comply with this document. 🌷
Applications for a new Type value MUST be submitted to the IANA and include the requestor's name and contact information, the reason for requesting a new Type, and references to appropriate standards, such as an updated version of ITU-R Recommendation BT.656. Furthermore, in the unlikely event that the new Type will lessen the security of a compliant implementation, that security risk MUST be detailed in the application. The application will be reviewed by a Designated Expert and, if appropriate, a new Type will be assigned. That type will then be listed in the IANA repository for future implementations. ✅
9. References 📚
[1] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", RFC 1889, January 1996.
[2] Interfaces for Digital Component Video Signals in 525-Line and 625-Line Television Systems operating at the 4:2:2 Level of Recommendation ITU-R BT.601 (Part A), ITU-R Recommendation BT.656-3, 1995.
[3] Studio Encoding Parameters of Digital Television for Standard 4:3 and Wide-Screen 16:9 Aspect Ratios, ITU-R Recommendation BT.601-5, 1995.
[4] Schulzrinne, H., "RTP Profile for Audio and Video Conference with Minimal Control", RFC 1890, January 1996.
[5] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
[6] Mogul, J., and S. Deering, "Path MTU Discovery", RFC 1191, November 1990.
10. Author's Address 🖊️
Dermot Tynan
Claddagh Films Limited
3 White Oaks
Clybaun Road
Galway
Ireland
EMail: dtynan@claddagh.ie
Phone: +353 91 52994411. Full Copyright Statement 🌟
Copyright (C) The Internet Society (1998). All Rights Reserved.
This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards, in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. 🌍
The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. 💜
This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 🫶
📦 where this page comes from
- ietf: rfc2431 1998-10
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms