RFC 2429

RTP Payload Format for the 1998 Version of ITU-T Rec. H.263 Video (H.263+)

📅 published October 1998, from ietf

RFC 2429: RTP Payload Format for the 1998 Version of ITU-T Rec. H.263 Video (H.263+) 🎀

okay bestie, RFC 2429 is here and it is giving video streaming realness ✨ here's the title block tea:

  • Working group: Network Working Group 🌸
  • Category: Standards Track, so this is an actual Internet standard and not just a vibe 💅
  • Date: October 1998 🍓
  • Authors: C. Bormann (Univ. Bremen), L. Cline (Intel), G. Deisher (Intel), T. Gardos (PictureTel), C. Maciocco (Intel), D. Newell (Intel), J. Ott (Univ. Bremen), G. Sullivan (PictureTel), S. Wenger (TU Berlin), and C. Zhu (Intel) 🫶
  • Obsoletes / Updates: the title block doesn't list anything under either one, so there's nothing to obsolete or update here 🧚

Okay, the authors' affiliations are on the left side of the original header, and I didn't copy the column layout, but everyone is listed. Moving on!

Status of this Memo

okay so this part is the official "what is this doc" section 💖 It says this document specifies an Internet standards track protocol for the Internet community, and it's asking for discussion and suggestions for improvements 🔥 If you want the current standardization state and status of this protocol, check the current edition of the "Internet Official Protocol Standards" (STD 1) 📜 And the big one: distribution of this memo is unlimited 🦋

Copyright (C) The Internet Society (1998). All Rights Reserved. 🌈

1. Introduction ✨

okay bestie, this section is basically the whole pitch. This document lays out an RTP payload header format for video streams made from the 1998 version of ITU-T Recommendation H.263 [4]. 🎀

Here's the spicy part: the 1998 version of H.263 is a superset of the 1996 syntax, so this format can also be used with the 1996 version of H.263 [3], and it is RECOMMENDED for this use by new implementations 💅 This format does not replace RFC 2190, which continues to be used by existing implementations, and it may be required for backward compatibility in new implementations 🫶 And if you're implementing the new features of the 1998 version of H.263, you SHALL use the format described in this document 👑

The 1998 version of ITU-T Recommendation H.263 added a bunch of coding options to improve codec performance over the 1996 version. In this document, the 1998 version is called H.263+ 💜 The new options with the biggest impact on the RTP payload spec and on the error resilience of the video content are the slice structured mode, the independent segment decoding mode, the reference picture selection mode, and the scalability mode 😭 This section sums up how these new coding options affect packetization. For more info on coding options, refer to [4] 🧚

The slice structured mode was added to H.263+ for three purposes: enhanced error resilience, making the bitstream more amenable to use with an underlying packet transport such as RTP, and minimizing video delay ✨ It supports fragmentation at macroblock boundaries 🌸

With the independent segment decoding (ISD) option, a video picture frame is broken into segments and encoded so that each segment is independently decodable 🦋 Using ISD on a lossy network helps stop errors from spreading from one segment of the picture to the others 🔥

The reference picture selection mode lets you use an older reference picture instead of the one right before the current picture 💅 Usually the last transmitted frame is implicitly used as the reference picture for inter-frame prediction, but with this mode the data stream carries info on which reference frame to use, shown by the temporal reference as an ID for that frame 🍓 It can be used with or without a back channel, which tells the encoder about the decoder's internal status 🫶 However, this document makes no special provision for carrying back channel info 🙈

H.263+ also includes bitstream scalability as an optional coding mode 🌈 There are three kinds: temporal, signal-to-noise ratio (SNR), and spatial scalability 💖 Temporal scalability comes from the disposable nature of bi-directionally predicted frames, or B-frames 🧚 (A low-delay form called P-picture temporal scalability can also be achieved by using the reference picture selection mode from the previous paragraph.) SNR scalability lets you refine encoded video frames, which improves the quality (or SNR) ✨ Spatial scalability is similar to SNR scalability, except the refinement layer is twice the size of the base layer in the horizontal dimension, the vertical dimension, or both 👑

2. Usage of RTP

okay so this section is about how H.263+ streams actually ride on RTP 🦋 When you send H.263+ video streams over the Internet, the encoder's output can be packetized directly 💅 All the bits from the bitstream, including the fixed length codes and variable length codes, get included in the packet 🫶 The one exception: when a packet's payload begins with a Picture, GOB, Slice, EOS, or EOSBS start code, the first two (all-zero) bytes of that start code are removed and replaced by setting an indicator bit in the payload header 🙈

For H.263+ bitstreams coded with temporal, spatial, or SNR scalability, each layer can be transported to a different network address 🌸 More specifically, each layer can use a unique IP address and port number combo 🔥 The temporal relations between layers SHALL be expressed using the RTP timestamp, so they can be synchronized at the receiving ends in multicast or unicast applications 👑

The H.263+ video stream gets carried as payload data inside RTP packets 🎀 A new H.263+ payload header is defined in section 4, and this section covers how the RTP fixed header and the H.263+ video packet structure are used together 💜

2.1 RTP Header Usage

Each RTP packet starts with a fixed RTP header 🧚 These fields of the RTP fixed header are used for H.263+ video streams:

Marker bit (M bit): The Marker bit of the RTP header is set to 1 when the current packet carries the end of the current frame, and it's 0 otherwise 🍓

Payload Type (PT): The Payload Type SHALL specify the H.263+ video payload format 💅

Timestamp: The RTP Timestamp encodes the sampling instance of the first video frame data in the RTP data packet 😭 The RTP timestamp SHALL be the same on successive packets if a video frame takes up more than one packet 🫶 In a multilayer scenario, all pictures that correspond to the same temporal reference SHOULD use the same timestamp 🌈 If temporal scalability is used (if B-frames are present), the timestamp may not be monotonically increasing in the RTP stream 🦋 If B-frames are sent on a separate layer and address, they MUST be synchronized properly with the reference frames 👑 Refer to the 1998 ITU-T Recommendation H.263 [4] for info on the required transmission order to a decoder 🧚

For an H.263+ video stream, the RTP timestamp is based on a 90 kHz clock, the same as the RTP payload for H.261 stream [5] 💖 Since both the H.263+ data and the RTP header contain time info, it is REQUIRED that the timing info run synchronously 😭 That means both the RTP timestamp and the temporal reference (TR in the picture header of H.263) SHOULD carry the same relative timing info 🔥

Any H.263+ picture clock frequency can be expressed as 1800000/(cd*cf) source pictures per second, where cd is an integer from 1 to 127 and cf is either 1000 or 1001 🌸 Using the 90 kHz clock of the RTP timestamp, the time increment between each coded H.263+ picture SHOULD therefore be a integer multiple of (cd*cf)/20. This will always be an integer for any "reasonable" picture clock frequency. For example, it's 3003 for 29.97 Hz NTSC, 3600 for 25 Hz PAL, 3750 for 24 Hz film, and 1500, 1250 and 1200 for the computer display update rates of 60, 72 and 75 Hz, respectively 🎀 For RTP packetization of hypothetical H.263+ bitstreams using "unreasonable" custom picture clock frequencies, mathematical rounding could become necessary for generating the RTP timestamps 🙈

2.2 Video Packet Structure

A section of an H.263+ compressed bitstream is carried as a payload within each RTP packet ✨ For each RTP packet, the RTP header is followed by an H.263+ payload header, and then a number of bytes of a standard H.263+ compressed bitstream 💅 The size of the H.263+ payload header is variable, depending on the payload involved, as detailed in section 4 🍓 Here's the layout of the RTP H.263+ video packet:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |    RTP Header                                               ...
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |    H.263+ Payload Header                                    ...
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |    H.263+ Compressed Data Stream                            ...
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Any H.263+ start codes can be byte aligned by an encoder using the stuffing mechanisms of H.263+ 🦋 As specified in H.263+, picture, slice, and EOSBS start codes SHALL always be byte aligned, and GOB and EOS start codes MAY be byte aligned 🔥 For packetization purposes, GOB start codes SHOULD be byte aligned 🧚 However, since this is not required in H.263+, there may be some cases where GOB start codes aren't aligned, such as when transmitting existing content, or when using H.263 encoders that don't support GOB start code alignment 😭 In that case, follow-on packets (see section 5.2) SHOULD be used for packetization 💜

All H.263+ start codes (Picture, GOB, Slice, EOS, and EOSBS) begin with 16 zero-valued bits 👑 If a start code is byte aligned and it occurs at the beginning of a packet, these two bytes SHALL be removed from the H.263+ compressed data stream during packetization and SHALL instead be represented by setting a bit (the P bit) in the payload header 🌈✨

3. Design Considerations ✨

okay bestie, so the whole goal of this payload format is to pack an H.263+ standard compliant bitstream in an efficient way AND make it way more resilient to packet losses 🫶 H.263+ has sooo many possible coding schemes, so a copy of the picture header with configuration info gets tucked into the payload header when it makes sense. Then even if the packet with the original picture header gets lost, that copy plus the payload data can still let you decode the packet. no cap, that's a whole backup plan 🦋

There are a few assumptions and constraints tied to this payload header design. This section points out the design issues and talks through some H.263+ coding options that can seriously change how network-based H.263+ video performs 💅

  • The optional slice structured mode described in Annex K of H.263+ [4] gives you way more flexibility for packetization 🧚 Like a picture segment that starts with a GOB header, the motion vector predictors in a slice are restricted to stay inside its boundaries. But slices give you much more freedom in choosing the size and shape of the area that counts as a distinct decodable region. Slices can be sized dynamically so the data for each slice fits into a chosen packet size, and they can be rectangular, which is great for keeping errors and packet losses from messing up motion compensated prediction. For these reasons, the use of the slice structured mode is strongly recommended for any applications used in environments where significant packet loss occurs 💖
  • In non-rectangular slice structured mode, only complete slices SHOULD be included in a packet. Slices SHOULD NOT be fragmented across packet boundaries 🙈 The only real reason a slice would need splitting is when the encoder that made the H.263+ data stream couldn't know anything about the packetization process, like when the data goes through a network that isn't the one the encoder is attached to, such as in network gateway implementations. Ideally, each packet contains only one slice ✨
  • The independent segment decoding (ISD) described in Annex R of [4] stops any data dependency across slice or GOB boundaries in the reference picture. It can be used to boost resiliency even more in high loss conditions 🔥
  • If ISD is used together with the slice structure, the rectangular slice submode SHALL be enabled, and the dimensions and quantity of the slices in a frame SHALL remain the same between each two intra-coded frames (I-frames), as required in H.263+. The individual ISD segments MAY also be entirely intra coded from time to time, which gives you quick error recovery without the latency of sending complete INTRA-pictures 🌸
  • When the slice structure isn't applied, inserting a (preferably byte-aligned) GOB header can give you resync boundaries in the bitstream. A GOB header kills the motion vector prediction dependency across GOB boundaries, and those resync boundaries are natural spots for packet payload boundaries 🎀
  • H.263+ lets picture headers be sent in an abbreviated form so you don't keep repeating overhead info that doesn't change from picture to picture. For resiliency though, sending a complete picture header for every frame is often advisable. So (especially when there's a high packet loss probability and the picture header contents aren't expected to be super predictable), the sender MAY find it advisable to always set the subfield UFEP in PLUSPTYPE to '001' in the H.263+ video bitstream 🍓 (See [4] for the definition of the UFEP and PLUSPTYPE fields.)
  • In a multi-layer scenario, each layer MAY be transmitted to a different network address. The configuration of each layer, like the enhancement layer number (ELNUM), reference layer number (RLNUM), and scalability type, SHOULD be determined at the start of the session and SHOULD NOT change during the session 👑
  • All start codes can be byte aligned, and picture, slice, and EOSBS start codes are always byte aligned. Those boundaries are ideal spots for placing packet boundaries 🦋
  • We assume that a maximum Picture Header size of 504 bits is sufficient. The H.263+ syntax doesn't explicitly prohibit bigger picture headers, but extremely large ones aren't expected 💅

4. H.263+ Payload Header

For H.263+ video streams, each RTP packet carries only one H.263+ video packet. The H.263+ payload header is always present for each H.263+ video packet, and it's variable length 🌈 A 16 bit field of the basic payload header can be followed by an 8 bit field for Video Redundancy Coding (VRC) information, and/or by a variable length extra picture header as indicated by PLEN. When these optional fields show up, they appear in the order I just listed 🫶

If an extra picture header is included in the payload header, its length in bytes is specified by PLEN. The minimum length of the payload header is 16 bits, which matches PLEN equal to 0 with no VRC information present ✨

The rest of this section defines the components of the RTP payload header. Section five defines the packet types used to carry the different kinds of H.263+ coded data, and section six summarizes how to tell the packet types apart 🧚

4.1 General H.263+ payload header

The H.263+ payload header is structured like this 💜

      0                   1
      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
     |   RR    |P|V|   PLEN    |PEBIT|
     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  • RR: 5 bits. Reserved bits. These SHALL be zero.
  • P: 1 bit. This tells you the picture start, a picture segment (GOB/Slice) start, or a video sequence end (EOS or EOSBS). Two bytes of zero bits then have to be prefixed to the payload of such a packet to make a complete picture/GOB/slice/EOS/EOSBS start code. This bit lets you skip the first two bytes of the start codes, which improves the compression ratio 🌸
  • V: 1 bit. This shows whether an 8 bit field with Video Redundancy Coding (VRC) info is present. If it is, it comes right after the first 16 bits of the payload header. For the syntax and semantics of that 8 bit VRC field, see section 4.2 🎀
  • PLEN: 6 bits. The length in bytes of the extra picture header. If no extra picture header is attached, PLEN is 0. If PLEN>0, the extra picture header is attached right after the rest of the payload header. Heads up: the length reflects leaving out the first two bytes of the picture start code (PSC). See section 5.1 ⭐
  • PEBIT: 3 bits. This shows how many bits SHALL be ignored in the last byte of the picture header. If PLEN is not zero, the ignored bits SHALL be the least significant bits of that byte. If PLEN is zero, then PEBIT SHALL also be zero 🍓

4.2 Video Redundancy Coding Header Extension ✨

Okay bestie, so Video Redundancy Coding (VRC) is an optional mechanism 🦋 that exists to boost error resilience over packet networks 🛡️. To use VRC in H.263+, you'll need the Reference Picture Selection option from Annex N of [4]. The big idea is having multiple "threads" of pictures that are each inter-frame predicted on their own, so if one frame gets damaged, the mess stays inside its own thread and the other threads are totally unaffected 💅. Every so often, all the threads meet up at a so-called sync frame (an INTRA picture, or a non-INTRA picture that's redundantly represented across multiple threads) 💜. From that sync frame, the independent threads kick off again. For more on codec support for VRC, check out [7] 📚.

P-picture temporal scalability is another use of the reference picture selection mode, and honestly it's basically a special case of VRC where only one copy of each sync frame is sent 🌸. It gives you a thread-based way to do temporal scalability without the extra delay that comes with B pictures ⏱️. Here's the fun part: the sync frames in the first thread also get used to predict a second thread of pictures that land in between the sync frames, which bumps up the resulting frame rate 🚀. And those second-thread pictures can be dropped to reduce bit rate or decoding complexity, and later pictures can still be decoded just fine 🙌. You can even add a third or more threads, but each of those is only predicted from the sync frames (which are sent at least in thread 0) or from frames within that same thread 🧚.

Now, a VRC data stream is totally self-contained, just like all H.263+ data 💖. Still, it can be really useful for the transport hierarchy implementation to know the current damage status of each thread 👀. On the Internet, you can figure that out by looking at the marker bit, the sequence number of the RTP header, and the thread-id along with a circling "packet per thread" number 🎀. The last two numbers live in the VRC header extension.

The VRC header extension looks like this:

   0 1 2 3 4 5 6 7
  +-+-+-+-+-+-+-+-+
  | TID | Trun  |S|
  +-+-+-+-+-+-+-+-+

TID: 3 bits Thread ID. Up to 7 threads are allowed 🌈. Each frame of H.263+ VRC data will use as reference information only sync frames or frames within the same thread. By convention, thread 0 is expected to be the "canonical" thread, which is the thread the sync frame should ideally come from 👑. If thread 0's representation gets corrupted or lost, the decoder can use a representation of the sync frame with a higher thread number instead. Lower thread numbers are expected to contain equal or better representations of the sync frames than higher thread numbers when there's no data corruption or loss ✨. See [7] for a detailed discussion of VRC.

Trun: 4 bits A monotonically increasing (modulo 16) 4-bit number that counts the packet number within each thread 🔢.

S: 1 bit A bit that says the packet content is for a sync frame 🍓. An encoder using VRC may send several representations of the same "sync" picture. That way, no matter which thread of pictures gets hit by errors or packet losses, at least one representation of a particular picture is guaranteed to arrive (within at least one thread) 🫶. The sync picture can then be used to predict any thread. If no packet losses have happened, the sync frame contents of thread 0 can be used and those of the other threads can be discarded (and the same goes for the other threads) 💅. Thread 0 is considered the "canonical" thread, and using it is preferable to all the others 👑. The contents of packets with lower thread numbers shall be considered as having a higher processing and delivery priority than those with higher thread numbers. So packets with lower thread numbers for a given sync frame shall be delivered first to the decoder under loss-free and low-time-jitter conditions 🏃‍♀️💨. That leads to discarding the sync contents of the higher-numbered threads, as specified in Annex N of [4] 🦋.

5. Packetization schemes 🎀

5.1 Picture Segment Packets and Sequence Ending Packets (P=1) 🌸

A picture segment packet is defined as a packet that starts at the location of a Picture, GOB, or slice start code in the H.263+ data stream 🧁. This matches the definition of the start of a video picture segment in H.263+. For these packets, P=1 always.

Sometimes an extra picture header gets attached in the payload header of these packets 📦. Whenever an extra picture header is attached, as signified by PLEN>0, only the last six bits of its picture start code, '100000', are included in the payload header 🧩. A complete H.263+ picture header with a byte aligned picture start code can be conveniently put together on the receiving end by prepending the sixteen leading '0' bits 🪄.

When PLEN>0, the end bit position that matches the last byte of the picture header data is indicated by PEBIT 📍. The actual bitstream data shall begin on an 8-bit byte boundary following the payload header.

A sequence ending packet is defined as a packet that starts at the location of an EOS or EOSBS code in the H.263+ data stream 🔚. This marks the end of a sequence of H.263+ video data. (More H.263+ video data may still follow later, though, as specified in ITU-T Recommendation H.263 🌷.) For these packets, P=1 and PLEN=0 always.

The optional header extension for VRC may or may not be present, as indicated by the V bit flag 🎟️.

5.1.1 Packets that begin with a Picture Start Code 💜

Any packet that contains the whole or the start of a coded picture shall start at the location of the picture start code (PSC), and should normally be encapsulated with no extra copy of the picture header 🙅‍♀️. In other words, normally PLEN=0 in that case. However, if the coded picture contains an incomplete picture header (UFEP = "000"), then a representation of the complete (UFEP = "001") picture header may be attached during packetization for greater error resilience 🛡️. So, for packets that start at the location of a picture start code, PLEN shall be zero unless both of the following conditions apply:

  1. The picture header in the H.263+ bitstream payload is incomplete (PLUSPTYPE present and UFEP="000"), and
  2. The additional picture header which is attached is not incomplete (UFEP="001").

A packet which begins at the location of a Picture, GOB, slice, EOS, or EOSBS start code shall omit the first two (all zero) bytes from the H.263+ bitstream, and signify their presence by setting P=1 in the payload header 🫧.

Here's an example of encapsulating the first packet in a frame (without an attached redundant complete picture header) 🎬:

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |   RR    |1|V|0|0|0|0|0|0|0|0|0| bitstream data without the    |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  | first two 0 bytes of the PSC                                ...
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

5.1.2 Packets that begin with GBSC or SSC 🧚‍♀️

For a packet that begins at the location of a GOB or slice start code, PLEN may be zero or nonzero, depending on whether a redundant picture header is attached to the packet 🎀. In environments with very low packet loss rates, or when picture header contents are very seldom likely to change (except as can be detected from the GFID syntax of H.263+), a redundant copy of the picture header is not required 😌. However, in less ideal circumstances, a redundant picture header should be attached for enhanced error resilience, and its presence is indicated by PLEN>0 🔥.

Assuming a PLEN of 9 and P=1, here's an example of a packet that begins with a byte aligned GBSC or an SSC 🌈:

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |   RR    |1|V|0 0 1 0 0 1|PEBIT|1 0 0 0 0 0| picture header    |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  | starting with TR, PTYPE ...                                   |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  | ...                                           | bitstream     |
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  | data starting with GBSC/SSC without its first two 0 bytes   ...
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Notice that only the last six bits of the picture start code, '100000', are included in the payload header 👀. A complete H.263+ picture header with a byte aligned picture start code can be conveniently assembled, if needed, on the receiving end by prepending the sixteen leading '0' bits 💖.

5.1.3 Packets that Begin with an EOS or EOSBS Code 🎀

okay so for a packet that starts with an EOS or EOSBS code, PLEN SHALL be zero 🫶, and no Picture, GOB, or Slice start codes SHALL be included within the same packet 🙅‍♀️. just like with other packets that start with start codes, the two all-zero bytes at the front of the EOS or EOSBS code SHALL be left out 🫣, and their presence gets flagged by setting the P bit to 1 in the payload header 💜.

system designers SHOULD be aware 👀 that some decoders MAY read the loss of a packet with ONLY EOS or EOSBS info as losing essential video data 😭, and then they MAY stop displaying some of the video that comes after. the thing is, EOS and EOSBS codes don't actually affect how the pictures get decoded, so they're kinda unnecessary to send at all 🤷‍♀️. because of that whole misreading-a-lost-packet drama (you can spot the loss with the sequence number), encoders are generally to be discouraged from sending EOS and EOSBS 🚫.

here's an example of a packet with an EOS code 👇

   0                   1                   2
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |   RR    |1|V|0|0|0|0|0|0|0|0|0|1|1|1|1|1|1|0|0|
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

5.2 Encapsulating Follow-On Packet (P=0) 🦋

a Follow-on packet holds some bytes of coded H.263+ data that DON'T start at a synchronization point 🙃. basically it doesn't open with a Picture, GOB, Slice, EOS, or EOSBS header, and it MAY or MAY NOT start right at a macroblock boundary. since Follow-on packets don't start at sync points, the data at the very beginning of one isn't independently decodable 😵. for these packets, P=0 always 💯. if the packet right before a Follow-on packet got lost, the receiver MAY discard that Follow-on packet along with all the other Follow-on packets after it 🗑️. honestly, better behavior would be for the receiver to scan the inside of the packet payload for any start codes that can work as resync points 🔍. attaching a copy of a picture header to a Follow-on packet is only useful if the interior of the packet, or some later Follow-on packet, has a resync code like a GOB or slice start code 🧩. PLEN>0 is allowed, since it MAY allow resync in the interior of the packet ✨. the decoder MAY also be resynchronized at the next segment or picture packet 🔄.

here's an example of a Follow-on packet (with PLEN=0) 👇

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  |   RR    |0|V|0|0|0|0|0|0|0|0|0| bitstream data              ...
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

6. Use of this payload specification 💅

there's literally no syntactical difference between a picture segment packet and a Follow-on packet, except P=1 for picture segment or sequence ending packets and P=0 for Follow-on packets 🙌. here's the full breakdown of the packet types and how to tell them apart.

you can tell the packet types apart by checking the P bit and the first 6 bits of the payload, along with the header info 🕵️‍♀️. the table below shows the packet type for every combo of this info (also check the picture/GOB/Slice header descriptions in H.263+ for the details):

--------------+--------------+----------------------+-------------------
 First 6 bits | P-Bit | PLEN |  Packet              |  Remarks
 of Payload   |(payload hdr.)|                      |
--------------+--------------+----------------------+-------------------
 100000       |   1   |  0   |  Picture             |  Typical Picture
 100000       |   1   | > 0  |  Picture             |  Note UFEP
 1xxxxx       |   1   |  0   |  GOB/Slice/EOS/EOSBS |  See possible GNs
 1xxxxx       |   1   | > 0  |  GOB/Slice           |  See possible GNs
 Xxxxxx       |   0   |  0   |  Follow-on           |
 Xxxxxx       |   0   | > 0  |  Follow-on           |  Interior Resync
--------------+--------------+----------------------+-------------------

the details on the possible values of the five bit Group Number (GN) field (the one right after the initial "1" bit when the P-bit is "1" for a GOB, Slice, EOS, or EOSBS packet) are in section 5.2.3 of [4] 📖.

as defined here, every start of a coded frame (aka anytime there's a PSC) has to be wrapped up as a picture segment packet 🎁. if the whole coded picture fits into one packet of reasonable size (which depends on the connection), that's the only packet type that MAY need to be used 🎯. thanks to H.263+'s high compression ratio, this is often doable, especially for small spatial picture formats like QCIF and typical Internet packet sizes around 1500 bytes 🍓.

if the complete coded frame doesn't fit in a single packet, there are two different ways of packetizing that MAY be chosen 🤔. with very low or zero packet loss probability, one or more Follow-on packets MAY be used to code the rest of the picture. that gives minimal coding and packetization overhead plus optimal use of the maximal packet size, but it doesn't add any error resilience 🙅.

the alternative is to break the picture into reasonably small partitions called Segments (using the Slice or GOB mechanism) that DO offer synchronization points 🧩. by doing that and using the Picture Segment payload with PLEN>0, decoding of the transmitted packets is still possible even if the Picture packet with the picture header got lost (as long as any necessary reference picture is available) 🦸. Picture Segment packets can also be used together with Follow-on packets for large segment sizes 💖.

7. Security Considerations 🔐

RTP packets using this payload format are subject to the security considerations in the RTP specification [1], plus any appropriate RTP profile (for example [2]). that means confidentiality of the media streams is achieved by encryption 🔒. because the data compression used with this format is applied end-to-end, encryption MAY be performed after compression so there is no conflict between the two operations 💜.

⚠️ there's a potential denial-of-service threat for data encodings that use compression techniques with non-uniform receiver-end computational load 🚨. an attacker can inject pathological datagrams into the stream that are complex to decode and cause the receiver to get overloaded 😱. however, this encoding doesn't show any significant non-uniformity 🙌.

like with any IP-based protocol, a receiver MAY be overloaded just by receiving too many packets, desired or not 📦. network-layer authentication MAY be used to discard packets from undesired sources, but the processing cost of the authentication itself MAY be too high 🥴. in a multicast environment, pruning of specific sources MAY be implemented in future versions of IGMP [5] and in multicast routing protocols, so a receiver can pick which sources are allowed to reach it 🎯.

a security review of this payload format found no additional considerations beyond those in the RTP specification ✅.

8. Addresses of Authors 📇

meet the squad 💖

Carsten Bormann
Universitaet Bremen FB3 TZI      EMail: cabo@tzi.org
Postfach 330440                  Phone: +49.421.218-7024
D-28334 Bremen, GERMANY          Fax:   +49.421.218-7000


Linda Cline
Intel Corp. M/S JF3-206          EMail: lscline@jf.intel.com
2111 NE 25th Avenue              Phone: +1 503 264 3501
Hillsboro, OR 97124, USA         Fax:   +1 503 264 3483


Gim Deisher
Intel Corp. M/S JF2-78           EMail: gim.l.deisher@intel.com
2111 NE 25th Avenue              Phone: +1 503 264 3758
Hillsboro, OR 97124, USA         Fax:   +1 503 264 9372


Tom Gardos
Intel Corp. M/S JF2-78           EMail: thomas.r.gardos@intel.com
2111 NE 25th Avenue              Phone: +1 503 264 6459
Hillsboro, OR 97124, USA         Fax:   +1 503 264 9372


Christian Maciocco
Intel Corp. M/S JF3-206          EMail: christian.maciocco@intel.com
2111 NE 25th Avenue              Phone: +1 503 264 1770
Hillsboro, OR 97124, USA         Fax:   +1 503 264 9428


Donald Newell
Intel Corp. M/S JF3-206          EMail: donald.newell@intel.com
2111 NE 25th Avenue              Phone: +1 503 264 9234
Hillsboro, OR 97124, USA         Fax:   +1 503 264 9428

Joerg Ott
Universitaet Bremen FB3 TZI      EMail: jo@tzi.org
Postfach 330440                  Phone: +49.421.218-7024
D-28334 Bremen, GERMANY          Fax:   +49.421.218-7000


Gary Sullivan
PictureTel Corp. M/S 635         EMail: garys@pictel.com
100 Minuteman Road               Phone: +1 978 623 4324
Andover, MA 01810, USA           Fax:   +1 978 749 2804


Stephan Wenger
Technische Universitaet Berlin FB13
Sekr. FR 6-3                     EMail: stewe@cs.tu-berlin.de
Franklinstr. 28/29               Phone: +49.30.314-73160
D-10587 Berlin, GERMANY          Fax:   +49.30.314-25156


Chad Zhu
Intel Corp. M/S JF3-202          EMail: czhu@ix.netcom.com
2111 NE 25th Avenue              Phone: +1 503 264 6004
Hillsboro, OR 97124, USA         Fax:   +1 503 264 1805

9. References 📚

  • [1] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP : A Transport Protocol for Real-Time Applications", RFC 1889, January 1996.
  • [2] Schulzrinne, H., "RTP Profile for Audio and Video Conference with Minimal Control", RFC 1890, January 1996.
  • [3] "Video Coding for Low Bit Rate Communication," ITU-T Recommendation H.263, March 1996.
  • [4] "Video Coding for Low Bit Rate Communication," ITU-T Recommendation H.263, January 1998.
  • [5] Turletti, T. and C. Huitema, "RTP Payload Format for H.261 Video Streams", RFC 2032, October 1996.
  • [6] Zhu, C., "RTP Payload Format for H.263 Video Streams", RFC 2190, September 1997.
  • [7] S. Wenger, "Video Redundancy Coding in H.263+," Proc. Audio-Visual Services over Packet Networks, Aberdeen, U.K., September 1997.

okay bestie, this is the legal bit 🎀 but it's still important so we're keeping every single piece of it 💅

Copyright (C) The Internet Society (1998). All Rights Reserved.

so you can copy this document and hand it out to people, and you can write derivative works that comment on it, explain it, or help people implement it 🦋 you can publish and distribute all of that, in whole or in part, with no real restrictions 🌈 the one catch is that the copyright notice and this paragraph have to stay on every copy and every derivative work 🫶

the original document itself can't be modified in any way 🙈 that means you can't remove the copyright notice or the references to the Internet Society or other Internet organizations. the only exceptions are when you need to do it to develop Internet standards, in which case the procedures for copyrights defined in the Internet Standards process must be followed, or when you need to translate it into languages other than English 🌸

these permissions are perpetual 💜 the Internet Society and its successors or assigns won't revoke them, ever 🧚

and now the disclaimer, which is in all caps because it's serious 😭 this document and the information in it is provided on an "AS IS" basis, and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. no cap, they're not promising it'll work for everything 💀

📦 where this page comes from