RFC 5484
Associating Time-Codes with RTP Streams
RFC 5484: Associating Time-Codes with RTP Streams 🎀
Okay bestie, let's get into it! ✨ This is RFC 5484, written by D. Singer at Apple Computer Inc. 💻 It's a Standards Track document dated March 2009, and it lives in the Network Working Group. The header doesn't list anything it obsoletes or updates, so it's standing on its own 💅
Status of This Memo 🌸
So this one is an Internet standards track protocol for the whole Internet community 🌐 and they're literally asking for discussion and suggestions for improvements 🫶 If you want the current standardization state and status of this protocol, check the current edition of the "Internet Official Protocol Standards" (STD 1) 📚 Also, distribution of this memo is unlimited, so share away 🦋
Copyright Notice 💜
Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved. 🔒
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents, as they were in effect on the date of publication (http://trustee.ietf.org/license-info). Please review these carefully, because they spell out your rights and restrictions for this document. 📜
Abstract 🧚
This document describes a mechanism for attaching time-codes, as defined by the Society of Motion Picture and Television Engineers (SMPTE), to media streams. The best part is that it works no matter what RTP payload format the media stream uses 🎬
Table of Contents 📖
1. Introduction ....................................................2
2. Requirements Notation ...........................................3
3. Design Goals ....................................................3
4. Requirements and Constraints ....................................4
5. Signaling Information ...........................................4
6. In-Stream Information ...........................................6
6.1. Compact Format of the Time-Code ............................6
6.2. Full Format of the Time-Code ...............................7
6.3. Associations in RTCP .......................................8
6.4. Associations in RTP ........................................9
7. Implementation Note (Informative) ..............................10
8. Discussion (Informative) .......................................10
9. Security Considerations ........................................11
10. IANA Considerations ...........................................11
11. Acknowledgments ...............................................12
12. References ....................................................12
12.1. Normative References .....................................12
12.2. Informative References ...................................121. Introduction 🦋
Okay, first a little background on time-codes [SMPTE-12M].
The time-code system everyone uses is defined by SMPTE, and it counts frames. You'll usually see it displayed like a normal clock (hh:mm:ss.frame) 🕐 When the frame rate is truly integral, it really can be a normal clock value, because the seconds tick by at the same rate as the seconds we know and love ✨
NTSC video is the drama queen here 😭 It runs slightly slower than 30 frames per second (fps). Some people call it 29.97, which isn't quite right. To be accurate, a frame takes 1001 ticks of a 30000 tick/second clock. Be that as it may, SMPTE time-codes count 30 of these frames and call that a second 💀
This makes an SMPTE time-code display "run slow" compared to real-time. To fix that, people sometimes use a format called drop-frame. Some of the frame numbers get skipped, so the counter periodically "catches up." That means some time-code seconds actually only have 28 frames in them 🙈
It's worth knowing that the SMPTE time-code is never an accurate clock in either case. In the first case it runs slow. In the second, the adjustments are abrupt and periodic, and they still aren't quite accurate. That's why the rest of this document is careful to call a second in a time-code a "time-code second" 🤓
Still, SMPTE time-codes do run in real-time when they're used with systems that have integral fps (for example, film content at 24 fps or PAL video) 🎞️
This specification defines how to carry time-codes in RTP and in RTCP (RTP Control Protocol). It also covers how to associate them with a media stream and how to synchronize them with the RTP timestamps. It uses the general RTP header extension mechanism [RFC5285] 🎀
2. Requirements Notation 💅
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. 📝
3. Design Goals 👑
What we want is a system that lets us associate an SMPTE time-code with some media in an RTP [RFC3550] stream. Since RTP media already has a clock, we can often use that. If we treat the media as having "segments" of time where the time-code is just counting up, then you can calculate the time-code anywhere in a segment if you know:
- the RTP timestamp of the start of the segment;
- the time-code of the start of the segment;
- the counting rate and other parameters of the time-code;
- the RTP timestamp where you want to know the time-code.
There are two cases to think about:
- the time-codes are piece-wise continuous with only occasional discontinuities;
- the continuity of the time-codes is not certain (or not known).
The first case is handled by giving the details of the time-code axis, plus an initial mapping from RTP time to time-code time, plus periodic mappings in RTCP packets. That's defined in Section 6.3 🧚
The second case needs in-band signaling inside the RTP packets themselves. That's defined in Section 6.4 ✨
Some apps really need all 8 bytes of the SMPTE 12M time-code to get carried (for example, when the date of the time-code must be known, or when RTP is used as a transparent pipe). Other cases, like time-codes used with compressed audio, care a lot about bandwidth. To support both vibes, the spec makes room for a compact form and a full form of the time-code 🍓
4. Requirements and Constraints 💖
Receivers MUST support time-codes in both RTCP and RTP, and both forms (compact and full) of the time-code. Senders, of course, are free to choose 🫶
Heads up that the compact form can hold bigger frame numbers than the full form. The compact form has a 6-bit field, while the full form uses a full binary-coded decimal (BCD) digit and a 2-bit BCD digit. That gives the full form a maximum transmitted value of 29. Sometimes the color frame flag (bit 11) is used to "extend" the "tens of frames" field from 2 to 3 bits. Those practices are outside the scope of this specification, though 👀
In the case that a presentation has more than one stream, senders MUST keep sending the standard RTP synchronization information in RTCP. That's true even when the streams carry SMPTE time-codes that could be used for synchronization 🔥 When more than one stream carries time-codes, this document doesn't constrain them. At any given moment they may be the same, or they may differ. For example, they might carry the original time-codes of different source material that got edited together 🎬
5. Signaling Information 🌈
If the recipient ever has to calculate time-codes from the RTP times, it needs some setup information first. This MUST be sent out-of-band, for example in a SIP offer/answer exchange [RFC3264]. Since this specification is a general header extension [RFC5285], the 'extmap' attribute from that extension mechanism is also used when the Session Description Protocol (SDP) is in play 🎀
The setup information should include:
- the duration, in the RTP timescale, of a single frame-count in the 'frames' portion of the time-code (frame_duration)
- the number of those frames that make a time-code second (frames_per_tc_second); framecounter values may be between 0 and (frames_per_tc_second - 1)
- the drop-frame indication, is-NTSC-drop-frame, which indicates whether the usual drop-frame behavior should be applied or not
Other info we need for the math, like the clock rate of the RTP timestamp, is already supplied and assumed to be available ⏱️
Here's an example. On a video stream with the common timescale of 90000 ticks per second, a frame_duration of 3003 and a frames-per-tc-second of 30 gives you a "normal" SMPTE time-code for NTSC video. Values of 3750 and 24 give you a time-code for 24 fps film content, and so on 🎞️
We supply the frame duration and fps explicitly even though they're obviously closely related. That removes any ambiguity about what the counter values should be when drop-frame counting is in use. These three values MUST correspond with each other 🔒
When SDP is used, these three parameters are sent as extension attributes, as defined in the header extension specification [RFC5285]. They follow the ABNF syntax [RFC5234] below. The form of the extension attributes is "owned" by the extension name, and these parameters don't need any registration action beyond being documented here. The parameters are supplied as extension attributes, which makes them suitable for inline use in RTP, even if a given stream only uses the RTCP mapping 📝
digit = "0"/"1"/"2"/"3"/"4"/"5"/"6"/"7"/"8"/"9"
integer = 1*digit
frame-duration-length = integer
timestamp-rate = integer
frame-duration = frame-duration-length "@" timestamp-rate
frames-per-tc-second = integer
drop = "/drop"
extensionattributes = frame-duration "/" frames-per-tc-second [drop]The frame duration is written as a count of ticks of a clock that has timestamp-rate ticks per second. It is recommended that the timestamp-rate match the clock rate of the RTP stream the extension is embedded in, so you don't lose accuracy when converting timestamps 🧠 If the payload type changes during a stream, especially between payloads with different clock rates, it is strongly recommended that the header extension go on the first packet(s) of the new payload. That way the mapping for the new clock rate is set explicitly 🙌
If '/drop' is specified, the first two frame numbers are left out of the count for each minute, except for minutes 00, 10, 20, 30, 40, and 50. This is documented in Section 4.2.2 of the SMPTE specification [SMPTE-12M]. (This usually only applies to NTSC video.) 🍓
The URI used for the signaling is:
"urn:ietf:params:rtp-hdrext:smpte-tc".This URI signals the possible presence of associations in RTCP or RTP, as defined below 🦋
Here's an SDP example for film material on a stream with a timescale of 600:
a=extmap:4 urn:ietf:params:rtp-hdrext:smpte-tc 25@600/24And here's another one, for drop-frame NTSC, on a stream with a timescale of 600:
a=extmap:4 urn:ietf:params:rtp-hdrext:smpte-tc 20@600/30/drop6. In-Stream Information 🧚
Part 2 picks up right here with the Section 6 subsections 👀
6.1. Compact Format of the Time-Code 💖
okay so a compact binary SMPTE time-code in this design is only 24 bits, and it is not done in the BCD system. It uses binary fixed-width fields instead, so it's giving clean and efficient 💅 Here's the layout:
sign(1) -- 1 for negative, 0 for positive
hours (5 bits) -- 0 to 23; the values 24-31 are reserved
minutes (6 bits) -- 0 to 59; 60-63 are reserved
seconds (6 bits) -- 0 to 59; 60-63 are reserved
frames(6 bits) -- 0 to (frames-per-tc-second - 1)Also, these fields are bigger than what SMPTE 12M gives you (that one uses BCD, and only two bits for the tens digit of the frame count, so frame numbers above 39 literally can't be represented 💀). This design fixes that, bestie 🙌
6.2. Full Format of the Time-Code 🎀
A full SMPTE time-code takes up 64 bits ✨ It's formatted exactly as Sections 7 and 8 of SMPTE 12M [SMPTE-12M] describe, just without the 16-bit syncword. The value of the "drop frame flag" MUST agree with the use of the "drop" indicator in the signaling. Non-negotiable, iykyk 🫶
Here are the bit assignments from SMPTE 12M, just so you can see the whole picture:
0--3 Units of frames
4--7 First binary group
8--9 Tens of frames
10 Drop frame flag
11 Color frame flag
12--15 Second binary group
16--19 Units of seconds
20--23 Third binary group
24--26 Tens of seconds
27 Polarity correction
28--31 Fourth binary group
32--35 Units of minutes
36--39 Fifth binary group
40--42 Tens of minutes
43 Binary group flag BGF0
44--47 Sixth binary group
48--51 Units of hours
52--55 Seventh binary group
56--57 Tens of hours
58 Binary group flag BGF1
59 Binary group flag BGF2
60--63 Eighth binary group6.3. Associations in RTCP 🦋
So when the time-codes are piece-wise continuous, we send an RTP timestamp plus an SMPTE time-code in RTCP packets for the start of each run of calculable time-codes. That locks in the time-code for every RTP time greater than or equal to the one given, until a later RTCP packet resets the mapping. It's a whole vibe 🌈
Heads up though: the RTP timestamp in the RTCP mapping might not match the timestamp of any frame in the media stream. For video it usually would, but a timestamp transition can happen partway through a decoded audio frame. Since they share the same clock, the timing of that transition and the timing of the audio stream itself have the same accuracy ⏰
The RTCP packets don't need to use the same RTP timestamp as the sender report (or transmission time) in the same RTCP packet. They can be sent 'ahead of need' if possible (for example, with stored content, when the server can look ahead) or 'just-in-time'. For instance, packets sent 'just-in-time' may go out as early feedback packets, following the rules in [RFC4585], after a discontinuity in the time-code is detected. These packets give the media buffer in the client a chance to 'catch' the RTCP before the matching RTP packet is processed and displayed. Main character timing 🎬
The association is a new RTCP Control Packet Type, using the value 194 (see Section 10). This control packet comes in one of two forms, and you can tell them apart by their length 👀
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P| SC |PT=SMPTETC=194 | length=3 |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| SSRC of packet sender |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| RTP timestamp |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
|S| hours | minutes | seconds | frames | reserved=0 |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+Figure 1: RTCP Short Form Packet
The fields S (sign), hours, minutes, seconds, and frames are defined in Section 6.1. For this short form, the length takes the fixed value 3, which means a control packet of 4 32-bit words ✨
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P| SC |PT=SMPTETC=194 | length=4 |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| SSRC of packet sender |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| RTP timestamp |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| Full 8-byte |
| SMPTE 12M time-code |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+Figure 2: RTCP Full Form Packet
For this full time-code (long form), the length takes the fixed value 4, which means a control packet of 5 32-bit words 🧚
6.4. Associations in RTP 🌸
Okay so when the time-codes aren't known to be piece-wise continuous, or you want absolute surety of the mapping, the mapping can go into some or all of the RTP packets. Ngl this is the less desirable route 😬 It uses the RTP header extension [RFC5285], which some terminals may find problematic. And putting mapping info in every packet obviously uses more bandwidth.
In as many RTP packets as needed (possibly all of them), an RTP header extension is used [RFC5285] to associate an RTP time with an SMPTE time-code.
There are two forms of this header extension, again told apart by their length. The short form associates a compact time-code with the RTP timestamp of the packet. The long form associates a full time-code with a timestamp offset from the RTP timestamp of the packet.
The short form has a length of 3 bytes (24 bits). The long form has a length of 12 bytes (96 bits) and consists of a full SMPTE 12M time-code, followed by a signed 32-bit offset D from the RTP timestamp. If the packet has timestamp T, this establishes an RTP to time-code association for the RTP time T+D. 🎯
7. Implementation Note (Informative) 🧚
This section is a suggestion for how to calculate a time-code for a time T2, given an initial code at time T1, and the frame duration. It's giving math-girl energy 🧮
It might seem like drop-frame creates a 'fence post' problem: how many minutes in which frame numbers were dropped have passed since the initial time-code? But you can dodge this entirely if all calculations are 'zero-based', because then the number of 'fence posts' is known. ✨
framesSinceTCzero := TimeCodeToFrameCount( initialTimeCode );
framesSinceMapping := floor( (T2-T1)/frameDuration );
totalFrames := framesSinceTCzero + framesSinceMapping;
timeCode := FrameCountToTimeCode( totalFrames );The SMPTE engineering guideline [SMPTE-EG40] has all the appropriate equations, constants, and so on for these conversions and others 📖
8. Discussion (Informative) 💬
This design doesn't need new IP packets in the sessions or new data in the main data channel. It uses low bandwidth (vanishingly low for streams with no discontinuities), and it's independent of the design of the RTP packets themselves: the RTP profile (including possibly encryption) and the RTP payload format. SMPTE time-codes can be associated with any RTP stream, including ones with existing payload formats. Iconic flexibility 👑
It might be argued that we could also set the initial mapping in the SDP, since RTCP packets might get lost. But then the SDP would need to know the RTP random offset, which is nasty 🙈 Also, if you put this RTCP packet into all sender reports, that's probably good enough. And if you don't have time-codes, you don't have audio-video-sync either.
This specification associates the time-code with a particular media stream. An alternative would be to make it an RTP stream in its own right, but the data rate is so low that seems egregious. By packing it inline, we can stay backwards-compatible for gateways and similar devices that already handle dual-stream 🫶
There's no way described in this document to detect that an RTCP packet has been lost and that a mapping may be used outside its intended range. ⚠️
The design assumes that clients will hold mappings until they are superseded, and that a client may need to buffer some number of upcoming mappings.
9. Security Considerations 🔐
SMPTE time-codes are only informative, and there are no known security considerations from associating them with media streams.
10. IANA Considerations 📋
The RTCP packet type used for SMPTE time-codes has been registered, in accordance with Section 15 of [RFC3550]. IANA has added a new value to the RTCP Control Packet types sub-registry of the Real-Time Transport Protocol (RTP) Parameters registry, according to the following data:
abbrev. name value Reference
--------- ----------------------- ------ ---------
SMPTETC SMPTE time-code mapping 194 RFC 5484Additionally, IANA has registered a new extension URI to the RTP Compact Header Extensions sub-registry of the Real-Time Transport Protocol (RTP) Parameters registry, according to the following data:
Extension URI: urn:ietf:params:rtp-hdrext:smpte-tc
Description: SMPTE time-code mapping
Contact: singer@apple.com
Reference: RFC 548411. Acknowledgments 🫶
Both Brian Link and John Lazzaro provided helpful comments on an initial draft. Colin Perkins was helpful in reviewing and dealing with the details. Ladan Gharai provided a thoughtful final review. Shoutout to all of them 💖
12. References 📚
12.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer
Model with Session Description Protocol (SDP)",
RFC 3264, June 2002.
[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, RFC 3550, July 2003.
[RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J.
Rey, "Extended RTP Profile for Real-time Transport
Control Protocol (RTCP)-Based Feedback (RTP/AVPF)",
RFC 4585, July 2006.
[RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax
Specifications: ABNF", STD 68, RFC 5234, January 2008.
[RFC5285] Singer, D. and H. Desineni, "A General Mechanism for
RTP Header Extensions", RFC 5285, July 2008.12.2. Informative References
[SMPTE-12M] Society of Motion Picture and Television Engineers,
"SMPTE Standard for Television -- Time and Control
Code", SMPTE 12M-1-2008.
[SMPTE-EG40] SMPTE, "Conversion of Time Values Between SMPTE 12M
Time Code, MPEG-2 PCR Time Base and Absolute Time",
SMPTE EG40-2002, August 2002.Author's Address 🎀
David Singer
Apple Computer Inc.
1 Infinite Loop
Cupertino, CA 95014
US
Phone: +1 408 996 1010
EMail: singer@apple.com📦 where this page comes from
- ietf: rfc5484 2009-03
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms