RFC 6597
RTP Payload Format for Society of Motion Picture and Television Engineers (SMPTE) ST 336 Encoded Data
RFC 6597: RTP Payload Format for SMPTE ST 336 Encoded Data 🎀
okay bestie, this is RFC 6597 💖 It's a Standards Track doc from the IETF (Internet Engineering Task Force), dated April 2012. The title is RTP Payload Format for Society of Motion Picture and Television Engineers (SMPTE) ST 336 Encoded Data. J. Downs, Ed. wrote it at PAR Government Systems Corp., and J. Arbeiter, Ed. co-edited it. The ISSN is 2070-1721. The title block has no Obsoletes or Updates entries, so this RFC doesn't list any older RFCs it replaces or changes 🌸
Abstract ✨
This doc explains how to pack KLV (Key-Length-Value) Encoded Data into RTP. KLV is defined by SMPTE in SMPTE ST 336, and RTP is the Real-time Transport Protocol. So it's basically the instructions for getting KLV data into RTP packets 🦋
Status of This Memo 🌈
This is an Internet Standards Track document.
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 5741.
Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at http://www.rfc-editor.org/info/rfc6597.
Copyright Notice 💜
Copyright (c) 2012 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
Table of Contents 🍓
1. Introduction ....................................................2
2. Conventions, Definitions, and Acronyms ..........................3
3. Media Format Background .........................................3
4. Payload Format ..................................................4
4.1. RTP Header Usage ...........................................5
4.2. Payload Data ...............................................5
4.2.1. The KLVunit .........................................5
4.2.2. KLVunit Mapping to RTP Packet Payload ...............6
4.3. Implementation Considerations ..............................6
4.3.1. Loss of Data ........................................6
4.3.1.1. Damaged KLVunits ...........................7
4.3.1.2. Treatment of Damaged KLVunits ..............9
5. Congestion Control ..............................................9
6. Payload Format Parameters .......................................9
6.1. Media Type Definition ......................................9
6.2. Mapping to SDP ............................................10
6.2.1. Offer/Answer Model and Declarative Considerations ..10
7. IANA Considerations ............................................11
8. Security Considerations ........................................11
9. References .....................................................12
9.1. Normative References ......................................12
9.2. Informative References ....................................121. Introduction 🦋
okay so this doc is all about packing SMPTE ST 336 KLV data into RTP. It cites [SMPTE-ST336] for the KLV spec and [RFC3550] for RTP itself 💅
The payload format is built so that arbitrary KLV data can be carried. There are no restrictions on which KLV data keys can be used. no cap, it's super flexible ✨
The doc gives a quick description of SMPTE ST 336, "Data Encoding Protocol Using Key-Length-Value". The payload format itself, including how the RTP header fields get used, is in Section 4. Media type and IANA considerations are covered too, and the doc ends with the security considerations for this payload format 🔐
2. Conventions, Definitions, and Acronyms 🎀
The term "Universal Label Key" refers to a fixed-length, 16-byte SMPTE-administered Universal Label (see [SMPTE-ST298]). It's used as the identifying key in a KLV item 🔑
The term "KLV item" refers to one single Universal Label Key, length, and value triplet, encoded as described in [SMPTE-ST336] 💖
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. these are the official BCP 14 words, so read them carefully 🧚
3. Media Format Background 🌸
[SMPTE-ST336], "Data Encoding Protocol Using Key-Length-Value", defines a byte-level data encoding protocol for representing data items and data groups. This encoding protocol definition doesn't depend on the application or transport method being used 🌷
SMPTE ST 336 data encoding can be applied to a wide variety of binary data. It's been used to provide diverse and rich metadata sets that describe or enhance associated video presentations. Pairing SMPTE ST 336 metadata with video has improved multimedia presentations, content management and distribution, archival and retrieval, and production workflow. it's giving "main character energy" 🎬
The SMPTE ST 336 standard defines a KLV triplet as a data interchange protocol for data items or data groups. The Key identifies the data, the Length specifies the length of the data, and the Value is the data itself. The KLV protocol gives all compliant applications a common interchange point, no matter how they're implemented or what transport they use 🫶
The Key of a KLV triplet (a Universal Label Key) is coded using a fixed-length 16-byte SMPTE-administered Universal Label. [SMPTE-ST298] goes into more detail about how 16-byte SMPTE-administered Universal Labels are structured. Universal Label Keys are kept in registries published by SMPTE (see, for example, [SMPTE-ST335] and [SMPTE-RP210]) 📚
The standard also has ways to combine associated KLV triplets into data sets, where the set of KLV triplets is itself coded with the KLV data coding protocol. These sets can be coded in full form (Universal Sets) or in one of four increasingly bit-efficient forms (Global Sets, Local Sets, Variable Length Packs, and Defined Length Packs). The standard defines each of these data constructs 🧩
On top of that, the standard defines how KLV coding can carry information registered with a non-SMPTE external agency 👑
4. Payload Format 💅
The main goal of the payload format design for SMPTE ST 336 data is to carry SMPTE ST 336 data over RTP in a simple yet robust way. All forms of SMPTE ST 336 data can be carried by the payload format. It keeps things simple by using only the standard RTP headers and not defining any payload headers 🍰
SMPTE ST 336 KLV data gets broken into KLVunits. A KLVunit is basically a logical grouping of otherwise unframed KLV data, grouped by source data timing (see Section 4.2.1) ⏰ Each KLVunit then gets placed into one or more RTP packet payloads. The RTP header marker bit helps receivers find the boundaries of KLVunits 🚩
4.1. RTP Header Usage 🎀
This payload format uses the RTP packet header fields as described in the table below:
+-----------+-------------------------------------------------------+
| Field | Usage |
+-----------+-------------------------------------------------------+
| Timestamp | The RTP Timestamp encodes the instant along a |
| | presentation timeline that the entire KLVunit encoded |
| | in the packet payload is to be presented. When one |
| | KLVunit is placed in multiple RTP packets, the RTP |
| | timestamp of all packets comprising that KLVunit MUST |
| | be the same. The timestamp clock frequency is |
| | defined as a parameter to the payload format |
| | (Section 6). |
| | |
| M-bit | The RTP header marker bit (M) is used to demarcate |
| | KLVunits. Senders MUST set the marker bit to '1' for |
| | any RTP packet that contains the final byte of a |
| | KLVunit. For all other packets, senders MUST set the |
| | RTP header marker bit to '0'. This allows receivers |
| | to pass a KLVunit for parsing/decoding immediately |
| | upon receipt of the last RTP packet comprising the |
| | KLVunit. Without this, a receiver would need to wait |
| | for the next RTP packet with a different timestamp to |
| | arrive, thus signaling the end of one KLVunit and the |
| | start of another. |
+-----------+-------------------------------------------------------+The remaining RTP header fields are used as specified in [RFC3550] 🦋
The M-bit is the big one here. Senders have to set it to '1' on the packet holding the final byte of a KLVunit, and '0' on everything else, so receivers know when a KLVunit is done ✨
4.2. Payload Data 🌈
4.2.1. The KLVunit 🧁
A KLVunit is a logical collection of all the KLV items that are to be presented at a specific time. It's made of one or more KLV items. Compound items (sets, packs) are allowed as per [SMPTE-ST336], but the contents of a compound item MUST NOT be split across two KLVunits. Multiple KLV items in a KLVunit sit one after another, with no padding or stuffing between items 🍓
4.2.2. KLVunit Mapping to RTP Packet Payload 💖
An RTP packet payload SHALL contain one, and only one, KLVunit or a fragment of one 🫶 KLVunits small enough to fit in a single RTP packet (packet size is up to the implementation, but it should consider underlying transport and network factors such as MTU limitations) go straight into the payload of the RTP packet. The first byte of the KLVunit, which is the first byte of a KLV Universal Label Key, becomes the first byte of the RTP packet payload 🎀
KLVunits too large to fit into a single RTP packet payload MAY span multiple RTP packet payloads. When this happens, the KLVunit data MUST be sent in sequential byte order. That way, when all the RTP packets for the KLVunit are arranged in sequence number order, concatenating their payload data reproduces the original KLVunit exactly 🧩
Also, when a KLVunit is fragmented across multiple RTP packets, all the RTP packets carrying its fragments MUST have the same timestamp ⏰
KLVunits are bounded by changes in RTP packet timestamps. The marker (M) bit in the RTP packet headers marks the last RTP packet of a KLVunit (see Section 4.1) 🚩
4.3. Implementation Considerations 🔧
4.3.1. Loss of Data 🌧️
RTP is generally deployed in network environments where packet loss might happen. RTP header fields make it possible to detect lost packets, as described in [RFC3550]. When this payload format is used to transmit payload data, packet loss can take out whole KLVunits or parts of them 😭
4.3.1.1. Damaged KLVunits ✨
okay so a damaged KLVunit is any KLVunit that was carried in one or more RTP packets that got lost 💔. When a lost packet is detected (you spot it using the sequence number header field 🔍), the receiver has two big jobs:
- The receiver MUST consider the KLVunit partially received before a lost packet as damaged 🩹. This damaged KLVunit covers all the packets before the lost one (in sequence number order) back to, but not including, the most recent packet where the M-bit in the RTP header was set to '1'.
- The receiver MUST consider the first KLVunit received after a lost packet as damaged 💅. This one covers the first packet after the lost one (in sequence number order) and, if that first packet has its M-bit in the RTP header set to '0', every packet after it up to and including the next one with the M-bit in the RTP header set to '1'.
This applies no matter what the M-bit was in the lost packet itself, and that's the iconic part 🦋. It means even super basic receivers can just look at the M-bit to find the outer edges of damaged KLVunits. So if a packet with the M-bit set to '1' goes missing, the KLVunit that the lost packet would have ended is marked damaged, and so is the KLVunit made from the packets received after the lost one (up to and including the next received packet with the M-bit set to '1') 🌸.
Here's a picture of how a receiver handles a lost packet in another possible packet sequence 👀:
+---------+-------------+ +--------------+
| RTP Hdr | Data | | |
+---------+-------------+ +--------------+
.... | ts = 30 | KLV KLV ... | | | >---+
| M = 1 | | | | |
| seq = 5 | ... KLV KLV | | | |
+---------+-------------+ +--------------+ |
Last RTP pkt for time 30 Lost RTP Pkt |
(seq = 6) |
|
+--------------------------------------------------------+
|
| +---------+-------------+ +---------+-------------+
| | RTP Hdr | Data | | RTP Hdr | Data |
| +---------+-------------+ +---------+-------------+
+--> | ts = 45 | KLV KLV ... | | ts = 45 | ... KLV ... | >---+
| M = 0 | | | M = 1 | | |
| seq = 7 | ... KLV ... | | seq = 8 | ... KLV KLV | |
+---------+-------------+ +---------+-------------+ |
RTP pkt for time 45 Last RTP pkt for time 45 |
KLVunit carried in these two packets is "damaged" |
|
+----------------------------------------------------------------+
|
| +---------+-------------+
| | RTP Hdr | Data |
| +---------+-------------+
+--> | ts = 55 | KLV KLV ... | ....
| M = 1 | |
| seq = 9 | ... KLV ... |
+---------+-------------+
Last and only RTP pkt
for time 55In this example, packets 7 and 8 hold pieces of a KLVunit with a timestamp of 45 🎀. That KLVunit counts as "damaged" because packet 6 went missing, and packet 6 might have been part of it 😭. The KLVunit for timestamp 30 (which ended in packet 5) is totally unaffected by the missing packet 🌈. The KLVunit for timestamp 55, sitting in packet 9, is also unaffected, so it's complete and intact ✨🫶.
4.3.1.2. Treatment of Damaged KLVunits 🧚
okay this part is so cool: SMPTE ST 336 KLV data streams are built so you can partially recover from errors or missing data in a stream 🙌. How exactly you handle damaged KLVunits is left up to each implementation 💅, since different implementations have different abilities and robustness when they process KLV payloads downstream. Some implementations can be pretty limited when it comes to handling damaged KLVunits, so receivers MAY drop damaged KLVunits entirely 🗑️. No judgment, just vibes 💜.
5. Congestion Control 🌸
the general congestion control stuff for sending RTP data applies here, so go check out RTP [RFC3550] and any RTP profile that applies, like AVP [RFC3551] 📚.
Also, SMPTE ST 336 data can be encoded in different schemes that cut down the overhead of each data item in the overall stream 🍓. SMPTE ST 336 grouping constructs, like local sets and data packs, give you a way to lower bandwidth requirements 🦋.
6. Payload Format Parameters 💖
this RTP payload format is identified using the application/smpte336m media type 🎀. That media type is registered following [RFC4855], and it uses the template from [RFC4288] ✨.
6.1. Media Type Definition 📝
here's the registration template, copied exactly as-is 👑:
Type name: application
Subtype name: smpte336m
Required parameters:
rate: RTP timestamp clock rate. Typically chosen based on
sampling rate of metadata being transmitted, but other rates
can be specified.
Optional parameters: None
Encoding considerations: This media type is framed and binary; see
Section 4.8 of [RFC4288].
Security considerations: See Section 8 of RFC 6597.
Interoperability considerations: Data items in smpte336m can be very
diverse. Receivers might only be capable of interpreting a subset
of the possible data items; unrecognized items are skipped.
Agreement on data items to be used out of band, via application
profile or similar, is typical.
Published specification: RFC 6597
Applications that use this media type: Streaming of metadata
associated with simultaneously streamed video and transmission of
[SMPTE-ST336]-based media formats (e.g., Material Exchange Format
(MXF) [SMPTE-ST377]).
Additional Information: none
Person & email address to contact for further information: J. Downs
<jeff_downs@partech.com>; IETF Payload Working Group
<payload@ietf.org>
Intended usage: COMMON
Restrictions on usage: This media type depends on RTP framing, and
hence is only defined for transfer via RTP ([RFC3550]). Transport
within other framing protocols is not defined at this time.
Author:
J. Downs <jeff_downs@partech.com>
J. Arbeiter <jimsgti@gmail.com>
Change controller: IETF Payload working group delegated from the
IESG.6.2. Mapping to SDP 🗺️
the mapping of the media type above and its parameters SHALL be done according to Section 3 of [RFC4855] 💅.
6.2.1. Offer/Answer Model and Declarative Considerations 🫶
this payload format has no configuration or optional format parameters, so it's pretty chill 😌. When you offer SMPTE ST 336 Encoded Data over RTP using SDP in an Offer/Answer model [RFC3264], or in a declarative way (like SDP in RTSP [RFC2326] or SAP [RFC2974]), there are no specific considerations to worry about 🌈.
7. IANA Considerations 🏳️🌈
IANA has registered application/smpte336m as described in Section 6.1 🎉. The media type got added to the IANA registry for "RTP Payload Format media types" (http://www.iana.org/assignments/rtp-parameters) 🦋.
8. Security Considerations 🚨
okay bestie, this is the serious bit, so listen up 👀. RTP packets using the payload format in this specification are subject to the security considerations in the RTP specification [RFC3550] and in any RTP profile that applies. The main security concerns for the RTP packet carrying the RTP payload format defined in this memo are confidentiality, integrity, and source authenticity 🔐. Confidentiality comes from encrypting the RTP payload. Integrity of the RTP packets comes from a suitable cryptographic integrity protection mechanism. Cryptographic systems can also allow the authentication of the payload's source. A suitable security mechanism for this payload format should provide confidentiality, integrity protection, and at least source authentication that can tell whether or not an RTP packet is from a member of the RTP session 💜.
The right security mechanism for RTP and payloads following this memo can vary, though 🤔. It depends on the application, the transport, and the signaling protocol being used. So one single mechanism isn't enough, but if it's suitable, using the Secure Real-time Transport Protocol (SRTP) [RFC3711] is recommended 🛡️. Other mechanisms that may be used are IPsec [RFC4301] and Transport Layer Security (TLS) [RFC5246] (RTP over TCP), and other options might exist too ✨.
This RTP payload format can lead to a lot of non-uniformity in the receiver-side computational complexity when processing SMPTE ST 336 payload data 😳. Because the length of SMPTE ST 336 encoded data items is essentially unbounded, receivers must take care when allocating resources for processing 🧯. It's easy to build pathological data that would make a naive decoder allocate huge amounts of resources, which could lead to denial-of-service threats 💀. Receivers SHOULD place limits on resource allocation that stay within the bounds set by any application profile in use 🚧.
This RTP payload format doesn't contain any inherently active content 🙂. However, individual SMPTE ST 336 KLV items could be defined to convey active content in a particular application 👀. So receivers that can decode and interpret those data items should use appropriate caution and security practices 🔒. In particular, accepting active content from streams that lack authenticity or integrity protection puts a receiver at risk of attacks using spoofed packets 😱. Receivers that can't decode those data items aren't at risk, since unknown data items are skipped over and discarded according to SMPTE ST 336 processing rules 🧚.
9. References 📚
9.1. Normative References 💅
here are the normative references, copied in full 🎀:
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, RFC 3550, July 2003.
[RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio
and Video Conferences with Minimal Control", STD 65,
RFC 3551, July 2003.
[RFC4288] Freed, N. and J. Klensin, "Media Type Specifications
and Registration Procedures", BCP 13, RFC 4288,
December 2005.
[RFC4855] Casner, S., "Media Type Registration of RTP Payload
Formats", RFC 4855, February 2007.9.2. Informative References 📚
okay bestie, these are the extra-helpful reads. They aren't required to implement anything, but they're the receipts 🧾💖
[RFC2326]Schulzrinne, H., Rao, A., and R. Lanphier, "Real Time Streaming Protocol (RTSP)", RFC 2326, April 1998. 🎬[RFC2974]Handley, M., Perkins, C., and E. Whelan, "Session Announcement Protocol", RFC 2974, October 2000. 📣[RFC3264]Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with Session Description Protocol (SDP)", RFC 3264, June 2002. 🤝[RFC3711]Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. Norrman, "The Secure Real-time Transport Protocol (SRTP)", RFC 3711, March 2004. 🔐[RFC4301]Kent, S. and K. Seo, "Security Architecture for the Internet Protocol", RFC 4301, December 2005. 🛡️[RFC5246]Dierks, T. and E. Rescorla, "The Transport Layer Security (TLS) Protocol Version 1.2", RFC 5246, August 2008. 🔒[SMPTE-RP210]Society of Motion Picture and Television Engineers, "SMPTE RP 210v12:2010 Data Element Dictionary", 2010,<http://www.smpte-ra.org/mdd/>. 🎞️[SMPTE-ST298]Society of Motion Picture and Television Engineers, "SMPTE ST 298:2009 Universal Labels for Unique Identification of Digital Data", 2009,<http://www.smpte.org>. 🏷️[SMPTE-ST335]Society of Motion Picture and Television Engineers, "SMPTE ST 335:2012 Metadata Element Dictionary Structure", 2012,<http://www.smpte.org>. 🗂️[SMPTE-ST336]Society of Motion Picture and Television Engineers, "SMPTE ST 336:2007 Data Encoding Protocol Using Key-Length-Value", 2007,<http://www.smpte.org>. 🧩[SMPTE-ST377]Society of Motion Picture and Television Engineers, "SMPTE ST 377-1:2011 Material Exchange Format (MXF) - File Format Specification", 2011,<http://www.smpte.org>. 📼
Authors' Addresses 💌
J. Downs (editor) PAR Government Systems Corp. US
EMail: jeff_downs@partech.com
J. Arbeiter (editor) US
EMail: jimsgti@gmail.com
📦 where this page comes from
- ietf: rfc6597 2012-04
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms