RFC 2038

RTP Payload Format for MPEG1/MPEG2 Video

📅 published October 1996, from ietf

RFC 2038: RTP Payload Format for MPEG1/MPEG2 Video 🎬💖

Okay bestie, the title block is giving main character energy ✨ It's a Network Working Group doc from October 1996, and it's on the Standards Track, so it's a real spec people are supposed to implement 👑 The authors are D. Hoffman and G. Fernando (both from Sun Microsystems, Inc.) and V. Goyal (from Precept Software, Inc.) 🫶 Nothing in the title block says it obsoletes or updates another RFC, so it's standing on its own 🦋

Status of This Memo 🎀

This is an Internet standards track protocol for the Internet community, and the authors are asking for discussion and suggestions for improvements 💅 To find out where this protocol stands in the standardization process, check the current edition of "Internet Official Protocol Standards" (STD 1) 📚 Distribution of this memo is unlimited 🌈

Abstract 🌸

Okay so this memo describes a packetization scheme for MPEG video and audio streams 🎞️ The scheme can carry that video or audio flow over the transport protocols that RTP supports 🚀 There are two approaches. The first one is built for maximum interoperability with MPEG System environments 🤝 The second one is built for maximum compatibility with other RTP-encapsulated media streams and with future conference control work at the IETF 💜

1. Introduction 🦋

The ISO/IEC JTC1/SC29 WG11 committee (you might also see it called the MPEG committee) created the MPEG1 standard (ISO/IEC 11172)[1] and the MPEG2 standard (ISO/IEC 13818)[2] 🎥 This memo describes how to packetize MPEG video and audio streams so they can travel over the Real-time Transport Protocol (RTP), version 2 [3, 4] ✨

MPEG1 has three parts: System, Video and Audio 🧩 It was designed mainly for CD-ROM applications and is tuned for combined data rates of about 1.5 Mbits/sec 💿 The video and audio parts describe the basic format of each stream, and those formats define the Elementary Streams (ES) 🌊 The MPEG1 System part wraps up the ES so it carries Presentation Time Stamps (PTS), Decoding Time Stamps and System Clock references, and it multiplexes MPEG1 compressed video and audio ES's together with user data 🎁

MPEG2 is structured in a similar way, but it isn't limited to CD-ROM apps anymore 🙈 The MPEG2 System part defines two system stream formats: the MPEG2 Transport Stream (MTS) and the MPEG2 Program Stream (MPS) 🧃 The MTS is built for communicating or storing one or more programs of MPEG2 compressed data, plus other data, in environments that tend to have errors 🌧️ The MPS is built for environments that are pretty error-free ☀️

The goal is interoperability among 4 types of end-systems. Here they are 👇

Transmitting Interworking Unit (TIU)

Receives MPEG information from a native MTS system for distribution over packet networks using a native RTP-based system layer (such as an IP-based internetwork). Examples: real-time encoder, MTS satellite link to Internet, video server with MTS-encoded source material.

Receiving Interworking Unit (RIU)

Receives MPEG information in real time from an RTP-based network for forwarding to a native MTS environment. Examples: Internet-based video server to MTS-based cable distribution plant.

Transmitting Internet End-System (TAES)

Transmits MPEG information generated or stored within the internet end-system itself, or received from internet-based computer networks. Example: video server.

Receiving Internet End-System (RAES)

Receives MPEG information over an RTP-based internet for consumption at the internet end-system or forwarding to traditional computer network. Example: desktop PC or workstation viewing training video.

Every one of the 2 transmitter types has to work with every one of the 2 receiver types 🫶 The TAES is probably going to be built on existing and planned internet-connected computers, and the RAES almost certainly will be, so it's really desirable for the interoperable protocol to be based on RTP 💻✨

Since MPEG streams can show up in lots of different applications, the authors propose two payload formats 🌈

A lot of people in the MPEG community care about using one of the MPEG System encodings, so Section 2 proposes encapsulating MPEG1 System streams and MPEG2 Transport and Program Streams with RTP 🎀 This profile supports the full semantics of MPEG System and gives basic interoperability across all four end-system types 💖

When only internet-based end-systems (TAES and RAES) are talking to each other, a payload format with better compatibility with the Internet architecture is wanted 🌐 That format defers some system issues to other protocols being defined in the Internet community, such as the MMUSIC WG 🧚 Section 3 proposes encapsulating compressed video and audio data (called "Elementary Streams" (ES) in MPEG docs) that complies with either MPEG1 or MPEG2 🎞️ In this one, neither the MPEG1 nor the MPEG2 System standard is used, and the ES's get encapsulated directly with RTP 🦋

The memo uses a lot of MPEG terminology, so for the real definitions, go read the primary MPEG references 📖🙏

2. Encapsulation of MPEG System and Transport Streams 🎁

Each RTP packet has a timestamp that comes from the sender's 90KHz clock reference ⏰ That clock is synced to the system stream Program Clock Reference (PCR) or System Clock Reference (SCR), and it represents the target transmission time of the first byte of the packet payload 🫧 The RTP timestamp is NOT passed to the MPEG decoder, which is kinda different from normal RTP, because it isn't treated as the media display or presentation timestamp 🙃 Its main jobs are to estimate and reduce network-induced jitter and to keep the transmitter and receiver from drifting apart in relative time ⏳✨

For MPEG2 Transport Streams, the RTP payload holds an integral number of MPEG transport packets 📦 To avoid end-system inefficiencies, data from multiple small MTS packets (normally 188 bytes each) gets bundled into one RTP packet 💅 The number of transport packets inside is found by dividing the RTP payload length by the length of an MTS packet (188) 🧮

MPEG2 Program streams and MPEG1 system streams have no packetization restrictions. They're just treated as a packetized stream of bytes 🌸

2.1 RTP header usage 🧁

Here's how the RTP header fields get used. The field definitions are copied exactly from the original so the names and wording stay precise 🫶

   The RTP header fields are used as follows:

        Payload Type: Distinct payload types should be assigned for
          of MPEG1 System Streams, MPEG2 Program Streams and MPEG2
          Transport Streams.  See [4] for payload type assignments.

        M bit:  Set to 1 whenever the timestamp is discontinuous
          (such as might happen when a sender switches from one data
          source to another). This allows the receiver and any
          intervening RTP mixers or translators that are synchronizing
          to the flow to ignore the difference between this timestamp
          and any previous timestamp in their clock phase detectors.

        timestamp: 32 bit 90K Hz timestamp representing the target
          transmission time for the first byte of the packet.

3.1 MPEG Video elementary streams ✨

okay bestie, telling MPEG1 Video apart from MPEG2 Video is actually easy 💅 you just check the video sequence header. In MPEG2, a sequence_header() gets followed by a sequence_extension(). The profile and level (stuff like MAIN_Profile@MAIN_Level or HIGH_Profile@HIGH_Level) come from the profile_and_level_indicator field in that sequence_extension header 🎀

ngl, MPEG bit-streams were built for pretty clean environments 🙈 they depend on each other a LOT, both over time and across space, so losing some data can make the good data around it totally useless 😭 this encapsulation adds application layer framing plus extra info in the RTP stream-specific header so receivers can recover. Appendix 1 has a bunch of recovery strategies to try 🧚

MPEG pictures can be huge, so they usually get chopped into packets smaller than a typical LAN/WAN MTU. The fragmentation rules are:

The MPEG Video_Sequence_Header, when present, will always be at the beginning of an RTP payload.
An MPEG GOP_header, when present, will always be at the beginning of the RTP payload, or will follow a Video_Sequence_Header.
An MPEG Picture_Header, when present, will always be at the beginning of a RTP payload, or will follow a GOP_header.

Each ES header MUST be completely contained within the packet 🫶 so a minimum RTP payload size of 261 bytes MUST be supported, to fit the biggest single header in the ES (that's the extension_data() header holding the quant_matrix_extension()). Other than that, headers can show up anywhere in the payload.

In MPEG, each picture is made of one or more "slices," and a slice is the unit of recovery from data loss or corruption. A decoder will normally skip ahead to the start of the next slice whenever it hits an error. The MPEG slice begin and end bits live in the encapsulation header to make this easier 💖

The beginning of a slice MUST either be the first data in a packet (after any MPEG ES headers) or MUST follow some integral number of slices in a packet. This is what makes it possible to find the start of the next slice after a missing packet without the receiver having to dig through the packet contents 🔍 Slices can be fragmented across packets, as long as all the rules above are met.

An implementation based on this encapsulation assumes the Video_Sequence_Header is repeated periodically in the MPEG bit-stream. This isn't required by the MPEG standard, but in practice it's what lets people switch channels and start decoding a continuously relayed bit-stream at any point 📺 If you're playing back an MPEG stream from a file format (where the Video_Sequence_Header only shows up at the very start), it's suggested that the packetizer saves the first Video_Sequence_Header (the one right after an end-of-stream indicator) so it can be injected into the network stream periodically 🌸

3.2 MPEG Audio elementary streams

MPEG1 Audio and MPEG2 Audio can be told apart from the MPEG ancillary_data() header 🎶 For either one, distinct Presentation Time Stamps may show up for frames that correspond to either 384 samples (Layer-I) or 1152 samples (Layer-II or Layer-III). How many bytes it takes to represent that many samples depends on the encoder parameters.

Multiple audio frames may be encapsulated within one RTP packet. When that happens, an integral number of audio frames MUST be contained within the packet, and the fragmentation header defined in Section 3.5 SHALL be set to 0.

Also, if you're using relatively short packets, one frame might be so big that it straddles multiple RTP packets 😭 For example, Layer-II MPEG audio sampled at 44.1 KHz has frames that each cover a time slot of 26.1 msec. At that sampling rate, if the compressed bit-rate is 384 kbits/sec (about 48 kBytes/sec), the average audio frame size works out to 1.25 KBytes. If packets were 500 Bytes long, each audio frame would straddle 3 RTP packets. The audio fragmentation indicator header (see Section 3.5) SHALL be present for an MPEG1/2 Audio payload type so this fragmentation can be handled 🧩

Here's how the RTP header fields get used 💅

Payload Type: Distinct payload types SHOULD be assigned for video elementary streams and audio elementary streams. See [4] for payload type assignments.
M bit: For video, it's set to 1 on the packet containing the MPEG frame end code, and 0 otherwise. For audio, it's set to 1 on the first packet of a "talk-spurt," and 0 otherwise.
PT: The MPEG video or audio stream ID.
timestamp: A 32-bit 90K Hz timestamp representing the presentation time of the MPEG picture or audio frame. It's the same for all packets that make up a picture or audio frame. It may not be monotonically increasing in a video stream if B pictures are present. For packets that only contain a video sequence and/or GOP header, the timestamp is the one from the subsequent picture ⏰

3.4 MPEG Video-specific header ✨

This header SHALL be attached to each RTP packet, right after the RTP fixed header 🎀

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|    MBZ    |         TR        |MBZ|S|B|E|  P  | | BFC | | FFC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                                FBV     FFV

MBZ: Unused. It MUST be set to zero in the current specification. This space is reserved for future use.
TR: Temporal-Reference (10 bits). The temporal reference of the current picture within the current GOP. It ranges from 0-1023 and stays constant for all RTP packets of a given picture.
MBZ: Unused. It MUST be set to zero in the current specification. This space is reserved for future use.
S: Sequence-header-present (1 bit). Normally 0, and set to 1 at the occurrence of each MPEG sequence header. It's used to detect whether a sequence header is present in the RTP packet.
B: Beginning-of-slice (BS) (1 bit). Set when the start of the packet payload is a slice start code, or when a slice start code is preceded only by one or more of a Video_Sequence_Header, GOP_header and/or Picture_Header.
E: End-of-slice (ES) (1 bit). Set when the last byte of the payload is the end of an MPEG slice.
P: Picture-Type (3 bits). I (1), P (2), B (3) or D (4). This value is constant for each RTP packet of a given picture. The value 000B is forbidden, and 101B - 111B are reserved to support future extensions to the MPEG ES specification.
FBV: full_pel_backward_vector
BFC: backward_f_code
FFV: full_pel_forward_vector
FFC: forward_f_code

These values are obtained from the most recent picture header and stay constant for each RTP packet of a given picture 🧠 None of them are used for I frames, and they MUST be set to zero in the RTP header. For P frames, only the last two values are present, and FBV and BFC MUST be set to zero in the RTP header. For B frames, all four values are present.

3.5 MPEG Audio-specific header 🎶

This header SHALL be attached to each RTP packet at the start of the payload, and after any RTP headers, for an MPEG1/2 Audio payload type.

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|             MBZ               |          Frag_offset          |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Frag_offset: The byte offset into the audio frame for the data in this packet.

Appendix 1. Error Recovery and Resynchronization Strategies ✨

These error recovery and resynchronization strategies are just guidelines 🌷 A compliant receiver is totally free to use different strategies, or none at all.

When first decoding an RTP-encapsulated MPEG Elementary Stream, the receiver MAY discard all packets until the Sequence-header-present bit is set to 1. At that point, the stream contains enough state information for an MPEG decoder to process it 💅

Losing a packet that contains the GOP_header and/or Picture_Header is detected by an unexpected change in the Temporal-Reference and Picture-Type values. Here's an example GOP sequence:

In display order: 0B 1B 2I 3B 4B 5P 6B 7B 8P GOP_HDR 0B ...
In stream order:  2I 0B 1B 5P 3B 4B 8P 6B 7B GOP_HDR 2I ...

Consider also two counters:

ref_pic_temp (Reference Picture (I,P) Temporal Reference)
dep_pic_temp (Dependent Picture (B) Temporal Reference)

At each GOP beginning, set these counters to the temporal reference value of the corresponding picture type. For our example GOP sequence, ref_pic_temp = 2 and dep_pic_temp = 0. Keep incrementing BOTH counters by unity with each following picture. ref_pic_temp SHOULD match the temporal references of the I and P frames, and dep_pic_temp SHOULD match the temporal references of the B frames 🔢

    dep_pic_temp: -  0  1  2  3  4  5  6  7        8  9
In stream order:  2I 0B 1B 5P 3B 4B 8P 6B 7B GOP_H 2I 0B 1B ...
    ref_pic_temp: 2  3  4  5  6  7  8  9  10  ^    11
                  --------------------------  |    ^
                             Match            Drop |
                                                   Mismatch
                                                    in ref_pic_temp

The loss of a GOP header can be detected by matching the appropriate counter (based on picture type) to the temporal reference value. A mismatch indicates a lost GOP header. If you want, you can rebuild a GOP header using a "null" time_code, repeating the closed_gop flag from previous GOP headers, and setting the broken_link flag to 1 🛠️

The loss of a Picture_Header can also be detected by a mismatch between the Temporal Reference in the RTP packet and the appropriate dep_pic_temp or ref_pic_temp counter at the receiver 🦋

After scanning to the next Beginning-of-slice, the Picture_Header is reconstructed from the P, TR, FBV, BFC, FFV and FFC in that packet, plus stream-dependent default values.

Any time an RTP packet is lost (shown by a gap in the RTP sequence number), the receiver MAY discard all packets until the Beginning-of-slice bit is set. At that point, the stream has enough state information for an MPEG decoder to start processing at the next slice boundary (possibly after rebuilding the GOP_header and/or Picture_Header as described above) 👑

References ✨

Okay bestie, here are the receipts 🧾💅 Every source the spec leans on, all listed out:

[1] ISO/IEC International Standard 11172; "Coding of moving pictures and associated audio for digital storage media up to about 1,5 Mbits/s", November 1993. 🎬
[2] ISO/IEC International Standard 13818; "Generic coding of moving pictures and associated audio information", November 1994. 🎞️
[3] H. Schulzrinne, S. Casner, R. Frederick, V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", RFC 1889, January 1996. 🚚
[4] H. Schulzrinne, "RTP Profile for Audio and Video Conferences with Minimal Control", RFC 1890, January 1996. 🌸

Authors' Addresses 🎀

Meet the squad who wrote this! They're literally the reason we have MPEG over RTP, so give them some love 🫶💖

Gerard Fernando
Sun Microsystems, Inc.
Mail-stop UMPK14-305
2550 Garcia Avenue
Mountain View, California 94043-1100
USA

Phone: +1 415-786-6373
EMail: gerard.fernando@eng.sun.com


Vivek Goyal
Precept Software, Inc.
1072 Arastradero Rd,
Palo Alto, CA 94304
USA

Phone: +1 415-845-5200
EMail: goyal@precept.com


Don Hoffman
Sun Microsystems, Inc.
Mail-stop UMPK14-305
2550 Garcia Avenue
Mountain View, California 94043-1100
USA

Phone: +1 503-297-1580
EMail: don.hoffman@eng.sun.com

That's the whole document, bestie! 🦋✨ Three authors, two references and one very iconic specification. Slay forever 👑💅

📦 where this page comes from

ietf: rfc2038 1996-10
💖 girlified by claude-haiku-5-5 on 2026-10-15: a language model's rewrite of the original, under its terms