RFC 8286

RTP/RTCP Extension for RTP Splicing Notification

📅 published October 2017, from ietf

RFC 8286: RTP/RTCP Extension for RTP Splicing Notification 🎀

  • Who wrote it: J. Xia, R. Even, R. Huang (Huawei), and L. Deng (China Mobile) 💜
  • Category: Standards Track ✨
  • Date: October 2017 📅
  • Series: IETF, Request for Comments 8286, ISSN 2070-1721 🌈
  • Obsoletes or updates: the title block doesn't list anything it obsoletes or updates 🙈

Abstract 🦋

Okay so, content splicing is when you swap out the content of a main multimedia stream for different multimedia content, then deliver that replacement to receivers for a stretch of time 🎬 The splicer handles RTP splicing, so it has to know exactly when to start and when to stop 🧚

This memo defines two RTP/RTCP extensions that tell the splicer about the splicing-related info. One is an RTP header extension that sends the info "in band" 🍓 The other is an RTP Control Protocol (RTCP) packet that sends it "out of band" 💅

Status of This Memo 📜

Bestie, this is an Internet Standards Track document 👑

It's a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community, received public review, and was approved for publication by the Internet Engineering Steering Group (IESG). More info on Internet Standards is in Section 2 of RFC 7841 🫶

Info about the current status of this document, any errata, and how to give feedback on it can be found at https://www.rfc-editor.org/info/rfc8286 🔥

Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved. 🌸

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they stood on the publication date. Please read these carefully, because they describe your rights and restrictions for this document 🙈 Code Components pulled out of this document must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty as described in the Simplified BSD License ✨

Table of Contents 📚

   1. Introduction ....................................................3
      1.1. Terminology ................................................3
   2. Overview ........................................................4
      2.1. Overview of RTP Splicing ...................................4
      2.2. Overview of Splicing Interval ..............................5
   3. Conveying Splicing Interval in RTP/RTCP Extensions ..............7
      3.1. RTP Header Extension .......................................7
      3.2. RTCP Splicing Notification Message .........................8
   4. Reducing Splicing Latency ......................................10
   5. Failure Cases ..................................................11
   6. Session Description Protocol (SDP) Signaling ...................12
      6.1. Declarative SDP ...........................................12
      6.2. Offer/Answer without BUNDLE ...............................13
      6.3. Offer/Answer with BUNDLE: All Media Are Spliced ...........14
      6.4. Offer/Answer with BUNDLE: A Subset of Media Are Spliced ...16
   7. Security Considerations ........................................18
   8. IANA Considerations ............................................19
      8.1. RTCP Control Packet Types .................................19
      8.2. RTP Compact Header Extensions .............................20
      8.3. SDP Grouping Semantic Extension ...........................20
   9. References .....................................................20
      9.1. Normative References ......................................20
      9.2. Informative References ....................................21
   Acknowledgements ..................................................22
   Authors' Addresses ................................................22

1. Introduction 🌷

Splicing is when you replace some multimedia content with other multimedia content and send that replacement to receivers for a while 📺 In predictable cases, like ad insertion, the splicing duration has to fit inside a specific pre-designated time slot ⏰ Before the splicer can start the splicing, it has to get the timing info about when to start and end it. This document calls that info the "Splicing Interval" 🎀

[SCTE35] has a method that wraps the Splicing Interval inside the MPEG2-TS (MPEG2 transport stream) layer in cable TV systems. When it's carried in RTP, you need a middlebox acting as the splicer to decode the RTP packets and dig through the payloads for the Splicing Interval. That extra work piles onto the middlebox and limits how many RTP sessions it can support 😭

This document defines an RTP header extension [RFC8285] that the main RTP sender uses to provide the Splicing Interval by including it in the RTP packets ✨

Here's the catch, though: the Splicing Interval in the RTP header extension might not reach the splicer successfully 💀 Any splicing-unaware middlebox on the path between the RTP sender and the splicer might strip this RTP header extension 🙈

To make it more robust for that case, this document also defines a new RTP Control Protocol (RTCP) packet type that carries the same Splicing Interval to the splicer 🧚 RTCP is also unreliable and may not be as "immediate" as the in-band technique, so it's only a complement to the RTP header extension 🫶

1.1. Terminology 💜

Okay, the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here 🔥

Besides that, we define the following terms 🌈

  • Main RTP Sender: The sender of RTP packets carrying the main RTP stream 🎬
  • Splicer: An intermediary node that inserts substitutive content into a main RTP stream. The splicer sends substitutive content to the RTP receiver instead of the main content during splicing. It's also responsible for processing RTCP traffic between the RTP sender and the RTP receiver 🦋
  • Splicing-In Point: A virtual point in the RTP stream, suitable for substitutive content entry, typically at the boundary between two independently decodable frames 🍓
  • Splicing-Out Point: A virtual point in the RTP stream, suitable for substitutive content exit, typically at the boundary between two independently decodable frames 🍓
  • Splicing Interval: The NTP timestamps, representing the main RTP sender wallclock time, for the splicing-in point and splicing-out point per [RFC6828], which let the splicer know when to start and end the RTP splicing 💅
  • Substitutive RTP Sender: The sender of RTP packets carrying the RTP stream that will replace the content in the main RTP stream 👑

2. Overview 🧩

2.1. Overview of RTP Splicing 🎀

RTP splicing swaps some multimedia content for certain substitutive multimedia content, then forwards it to the receivers for a period of time 💖 The main RTP sender authorizes this and offers a specific time window for inserting the substitutive content into the main content. A typical example is an IPTV service provider using its own regional advertising to replace national advertising, with the time window explicitly indicated by that provider 📺

The splicer is a middlebox handling RTP splicing. It gets the main content and substitutive content at the same time, but it only ever sends one of them to the receiver at any given moment 🦋 When RTP splicing begins, the splicer sends the substitutive content instead of the main content. When RTP splicing ends, it switches back to sending the main content ✨ That means the receiver is explicitly configured to receive traffic via the splicer, and it will return any RTCP feedback to the splicer while the splicer is in the picture 🫶

The splicer can be implemented as either an RTP mixer or an RTP translator 🌸 As an RTP mixer, the splicer uses its own synchronization source (SSRC), sequence number space, and timing model when generating the output stream to receivers. It uses the contributing source (CSRC) list to indicate whether the original or substitutive content is being delivered 🧚 The splicer, on behalf of the content provider, can omit the CSRC list from the RTP packets it generates. That simplifies the receivers, since they don't have to parse the CSRC list, but it makes it harder to tell when splicing is happening, because you'd need to inspect the RTP payload data instead of just the RTP headers 😭 A splicer working as an RTP mixer splits the flow between the sender and receiver into two, and it needs separate control loops for RTCP and congestion control. [RFC6828] gives an example of the RTP mixer approach 💜

A splicer implemented as an RTP translator [RFC3550] forwards the RTP packets from the original and substitutive senders with their SSRCs intact. It does need to rewrite RTCP Sender Report (SR) packets to account for the splicing 👑 In this case, the congestion control loops run between the original sender and receiver, and between the substitutive sender and receiver. The splicer needs to make sure the RTCP feedback messages from the receiver get passed to the right sender so that congestion control can do its thing 🔥

2.2. Overview of Splicing Interval ✨

okay so before the splicer can do anything at the reserved time slots the main RTP sender picked, it has to already know the Splicing Interval from that main sender. no Splicing Interval, no splice 💅

When a new splice is coming up, the main RTP sender needs to send the Splicing Interval over to the splicer. The Splicing Interval SHOULD be sent by the RTP header extension or an RTCP extension message more than once, so one lost packet doesn't ruin everything 🫶 And the splicer needs time to grab the substitutive content before the splice starts, so the main RTP sender MUST send the Splicing Interval well in advance 🕰️ For example, the main RTP sender can estimate when to send it based on the round-trip time (RTT), following the mechanisms in Section 6.4.1 of [RFC3550] when the splicer sends an RTCP Receiver Report (RR) to the main sender. main character timing, honestly 🎀

The substitutive sender also has to learn the Splicing Interval from the main RTP sender ahead of time, and then estimate when to transfer the substitutive content to the splicer 📦 The Splicing Interval could get passed from the main RTP sender to the substitutive content using some out-of-band mechanisms, like a proprietary way to exchange it. Or the substitutive sender could just be built into the same device as the main RTP sender, which is lowkey so convenient ✨ To make sure the Splicing Interval is valid for both the main RTP sender and the substitutive RTP sender, the two senders MUST share a common reference clock so the splicer can pull off accurate splicing ⏱️ How strict that clock needs to be (things like resolution and skew) depends on the codec used by the media content, so it's not one-size-fits-all 🌈

In this document, the main RTP sender uses a pair of NTP timestamps to say when to start and end the splicing to the splicer. The first one is the timestamp of the first substitutive RTP packet at the splicing-in point, and the second is the timestamp of the first main RTP packet at the splicing-out point 🦋

When the substitutive RTP sender gets the Splicing Interval, it must prepare the substitutive stream 🧁 The main content provider and the substitutive content provider MUST ensure that the RTP timestamp of the first substitutive RTP packet that would be presented to the receivers corresponds to the same time instant as the former NTP timestamp in the Splicing Interval 💜 To help the splicer figure out which substitutive packet it needs to send first, the substitutive RTP sender MUST send the substitutive RTP packet ahead of the splicing-in point. That way the splicer can find the timestamp of this first RTP packet in the substitutive RTP stream, for example by using a prior RTCP SR message 🔍

When it's time for the splice to end, the main content provider and the substitutive content provider MUST ensure that the RTP timestamp of the first main RTP packet that would be presented on the receivers corresponds to the same time instant as the latter NTP timestamp in the Splicing Interval 🎬

3. Conveying Splicing Interval in RTP/RTCP Extensions 🎀

okay so this memo defines two backward-compatible RTP extensions to convey the Splicing Interval to the splicer: an RTP header extension and an RTCP splicing notification message 💅✨

3.1. RTP Header Extension 🦋

The RTP header extension mechanism defined in [RFC8285] can be adapted to carry the Splicing Interval, which is a pair of NTP timestamps 🌸

This RTP header extension carries the 7 octets of the splicing-out NTP timestamp (the lower 24-bit part of the "Seconds" of an NTP timestamp and the 32 bits of the "Fraction" of an NTP timestamp, as defined in [RFC5905]), followed by the 8 octets of the splicing-in NTP timestamp (a 64-bit NTP timestamp as defined in [RFC5905]). Here's the wild part 🙈 The top 8 bits of the splicing-out NTP timestamp are inferred from the top 8 bits of the splicing-in NTP timestamp, assuming that (1) the splicing-out time is after the splicing-in time and (2) the Splicing Interval is less than 2^25 seconds. So if the value of the 7 octets of the splicing-out NTP timestamp is smaller than the value of the 7 lower octets of the splicing-in NTP timestamp, that means the 56-bit splicing-out NTP timestamp wrapped around 🌀 In that case, the top 8-bit value of the 64-bit splicing-out NTP timestamp equals the top 8-bit value of the splicing-in NTP timestamp plus 0x01. Otherwise, the top 8 bits of the splicing-out NTP timestamp are equal to the top 8 bits of the splicing-in NTP timestamp. the math is giving genius 🧠👑

This RTP header extension can be encoded using either the one-byte or two-byte header defined in [RFC8285]. Figures 1 and 2 show the Splicing Interval header extension in each of the two header formats 📐

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+E
   |   ID  | L=14  |    OUT NTP timestamp - Seconds (bit 8-31)     |x
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+t
   |          OUT NTP timestamp - Fraction (bit 0-31)              |e
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+n
   |           IN NTP timestamp - Seconds (bit 0-31)               |s
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+i
   |           IN NTP timestamp - Fraction (bit 0-31)              |o
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+n

       Figure 1: Splicing Interval Using the One-Byte Header Format

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+E
   |   ID          |    L=15       |  OUT NTP timestamp - Seconds  |x
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+t
   |OUT Secds(cont)|         OUT NTP timestamp - Fraction          |e
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+n
   |OUT Fract(cont)|          IN NTP timestamp - Seconds           |s
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+i
   | IN Secds(cont)|          IN NTP timestamp - Fraction          |o
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+n
   | IN Fract(cont)| 0 (pad)       |              ...
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

       Figure 2: Splicing Interval Using the Two-Byte Header Format

Since including an RTP header extension reduces the efficiency of RTP header compression 😭, it is RECOMMENDED that the main sender insert the RTP header extensions into a number of RTP packets, instead of all of them, prior to the splicing-in. Ration the extension, bestie 💅

After the splicer obtains the RTP header extension and derives the Splicing Interval, it generates its own stream and is not allowed to include the RTP header extension in outgoing packets. This keeps the header overhead down 🍓

3.2. RTCP Splicing Notification Message 💌

In addition to including the RTP header extension, the main RTP sender includes the Splicing Interval in an RTCP splicing notification message. Whether or not the timestamps are included in the RTP header extension, the main RTP sender MUST send the RTCP splicing notification message. This adds robustness in case a middlebox strips RTP header extensions, which is so real 🙄 The main RTP sender MUST make sure that the splicing information in the RTCP splicing notification message is consistent with the information in the RTP header extensions. no contradictions allowed, periodt 💅

The RTCP splicing notification message is a new RTCP packet type 🧚 It has a fixed header followed by a pair of NTP timestamps:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |V=2|P|reserved |    PT=213   |              length             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           SSRC                                |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             IN NTP timestamp (most significant word)          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             IN NTP timestamp (least significant word)         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             OUT NTP timestamp (most significant word)         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |             OUT NTP timestamp (least significant word)        |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

               Figure 3: RTCP Splicing Notification Message

The RTCP splicing notification message includes the following fields 📝

  • Length: 16 bits As defined in [RFC3550], the length of the RTCP packet in 32-bit words minus one, including the header and any padding. 📏
  • SSRC: 32 bits The SSRC of the main RTP sender. 🎤
  • Timestamp: 64 bits Indicates the wallclock time when this splicing starts and ends. The full-resolution NTP timestamp is used, which is a 64-bit unsigned fixed-point number with the integer part in the first 32 bits and the fractional part in the last 32 bits. This format is the same as the NTP timestamp field in the RTCP SR (Section 6.4.1 of [RFC3550]). ⏰

The RTCP splicing notification message can be included in the RTCP compound packet together with the RTCP SR generated at the main RTP sender. That means it follows the compound RTCP rules defined in Section 6.1 in [RFC3550] 🧩

If the use of non-compound RTCP [RFC5506] was previously negotiated between the sender and the splicer, the RTCP splicing notification messages may be sent as non-compound RTCP packets. In some cases, like when the mapping from the RTP timestamp to the NTP timestamp changes (for example, clock drift happens before the splicing event), sending an RTCP SR or even updated Splicing Interval information in a timely manner might be required to update the timestamp mapping for accurate splicing 🕰️✨

Since the RTCP splicing notification message is intentionally sent by the main RTP sender to the splicer, the splicer is not allowed to forward this message to the receivers. This avoids useless processing and extra RTCP bandwidth consumption downstream 🙅‍♀️💖

4. Reducing Splicing Latency 🎀

okay so when splicing starts or ends, the splicer is the one passing the multimedia content from another sender along to the receivers 🦋. The receivers need some info first before they can start processing the media ([RFC6285] calls this "Reference Information"), so the main RTP sender or the substitutive sender SHOULD send that Reference Information right alongside its multimedia content. That way the receivers don't have to wait around to grab it, which cuts down the delay ✨. How the Reference Information actually gets to the receivers is out of scope for this memo, so that's a different story 💅.

Another latency element is synchronization delay 🌸. The receivers have to get enough synchronization metadata before they can line up the separate components of the multimedia streams when splicing starts or ends. So the main RTP sender or the substitutive sender SHOULD send that metadata early enough for the receivers to play everything out in sync 🫶. To figure out when to send it, a sender can estimate based on things like the RTT, using the mechanisms in Section 6.4.1 of [RFC3550] when the splicer sends an RTCP RR to the main sender or the substitutive sender 🧚. The main RTP sender and the substitutive sender can also coordinate through some proprietary out-of-band mechanisms to decide when, and to whom, the metadata gets sent 🔥. If both of them send the information, the splicer SHOULD pick one depending on the current situation. For example, it could choose (1) the main RTP sender when it's synchronizing the main media content, or (2) the information from the substitutive sender when it's synchronizing the spliced content 💜. And if you want to cut down that possible synchronization delay even more, it is RECOMMENDED that the mechanisms defined in [RFC6051] be adopted 👑.

5. Failure Cases 💀

this section looks at what happens if the RTCP splicing notification messages get lost, like if the RTP header extension gets stripped on the path 🙈.

Since there might be a splicing-unaware middlebox sitting on the path between the main RTP sender and the splicer, the main and substitutive RTP senders can use one heuristic to check whether or not the Splicing Interval actually reaches the splicer 🔍.

The splicer can be implemented with its own SSRC so it can send RTCP reception reports to the senders of the main and substitutive RTP streams 🌈. This lets the senders spot problems on the path to the splicer, which is very iconic of them ✨. The other option is to implement the splicer with no SSRC and no RTCP reports at all 😭. But that means the senders can't keep an eye on the quality of the path to the splicer, and that's kind of a bummer 🥲.

If the splicer has an SSRC and sends its own RTCP reports, it can choose not to pass along the RTCP reports it gets from the receivers to the senders 🙅‍♀️. The catch is that the senders then can't monitor the quality of the paths from the splicer to the receivers.

A splicer that has an SSRC can choose to pass the RTCP reception reports from the receivers back to the senders, but only after making modifications to account for the splicing 💖. This lets the senders monitor the quality of the paths from the splicer to the receivers, which is lowkey really useful. A splicer that does NOT have its own SSRC has to forward and translate the RTCP reports from the receivers, or else the senders won't see any receivers in the RTP session at all 😳.

If the splicer is implemented as a mixer, it will have its own SSRC, send its own RTCP reports, and forward translated RTCP reports from the receivers 🍓.

When a failure is detected, the splicer can talk to the main sender and the substitutive sender through some out-of-band signaling technique, and then fall back to the payload-specific mechanisms it supports, like the MPEG2-TS splicing solution defined in [SCTE35]. Or it can just abandon the splicing altogether 🫠.

6. Session Description Protocol (SDP) Signaling 📝

this document defines the URI for declaring this header extension in an "extmap" attribute, and that URI is "urn:ietf:params:rtp-hdrext:splicing-interval" 🎀.

This document also extends the standard semantics from "The Session Description Protocol (SDP) Grouping Framework" [RFC5888] with a brand-new semantic called "SPLICE" 💅. It's used to represent the relationship between the main RTP stream and the substitutive RTP stream. Only two "m=" lines are allowed in the SPLICE group. The main RTP stream is the one with the extended "extmap" attribute, and the other one is the substitutive stream 🦋. A single "m=" line MUST NOT be included in different SPLICE groups at the same time 🚫. The main RTP sender is the one that provides the information about both the main and substitutive sources 🌸.

The extended SDP attribute from this document applies to offer/answer content [RFC3264], and it does not affect any of the rules for negotiating offers and answers ✨. When you're using multiple "m=" lines, substitutive RTP MUST be applied ONLY to the RTP packets whose SDP "m=" line is in the same group as the substitutive stream using SPLICE and has the extended splicing "extmap" attribute 🫶. This semantic also works for BUNDLE cases 🧚.

Here are some examples that show how SDP signaling could be used for splicing in different cases 💜.

6.1. Declarative SDP

v=0
o=xia 1122334455 1122334466 IN IP4 splicing.example.com
s=RTP Splicing Example
t=0 0
a=group:SPLICE 1 2
m=video 30000 RTP/AVP 100
i=Main RTP Stream
c=IN IP4 233.252.0.1/127
a=rtpmap:100 MP2T/90000
a=extmap:1 urn:ietf:params:rtp-hdrext:splicing-interval
a=mid:1
m=video 30002 RTP/AVP 100
i=Substitutive RTP Stream
c=IN IP4 233.252.0.2/127
a=sendonly
a=rtpmap:100 MP2T/90000
a=mid:2

Figure 4: Example SDP for a Single-Channel Splicing Scenario 🎬

okay so the splicer that gets the SDP message above receives one MPEG2-TS stream (payload 100) from the main RTP sender, which is sent to the multicast destination address 233.252.0.1 on port 30000 🌟. It can also receive another MPEG2-TS stream from the substitutive RTP sender, sent to 233.252.0.2 on port 30002 🍭. But at any particular point in time, the splicer only selects ONE stream and outputs the content from that chosen stream to the downstream receivers 🫧.

6.2. Offer/Answer without BUNDLE

SDP Offer - from the main RTP sender:

v=0
o=xia 1122334455 1122334466 IN IP4 splicing.example.com
s=RTP Splicing Example
t=0 0
a=group:SPLICE 1 2
m=video 30000 RTP/AVP 31 100
i=Main RTP Stream
c=IN IP4 splicing.example.com
a=rtpmap:31 H261/90000
a=rtpmap:100 MP2T/90000
a=extmap:1 urn:ietf:params:rtp-hdrext:splicing-interval
a=sendonly
a=mid:1
m=video 40000 RTP/AVP 31 100
i=Substitutive RTP Stream
c=IN IP4 substitutive.example.com
a=rtpmap:31 H261/90000
a=rtpmap:100 MP2T/90000
a=sendonly
a=mid:2

SDP Answer - from the splicer:

v=0
o=xia 1122334455 1122334466 IN IP4 splicer.example.com
s=RTP Splicing Example
t=0 0
a=group:SPLICE 1 2
m=video 30000 RTP/AVP 100
i=Main RTP Stream
c=IN IP4 splicer.example.com
a=rtpmap:100 MP2T/90000
a=extmap:1 urn:ietf:params:rtp-hdrext:splicing-interval
a=recvonly
a=mid:1
m=video 40000 RTP/AVP 100
i=Substitutive RTP Stream
c=IN IP4 splicer.example.com
a=rtpmap:100 MP2T/90000
a=recvonly
a=mid:2

6.3. Offer/Answer with BUNDLE: All Media Are Spliced

okay bestie, in this example the bundled audio and video media each get their very own substitutive media for splicing 🎧📺:

  1. An offer, in which the offerer assigns a unique address and a substitutive media to each bundled "m=" line for splicing within the BUNDLE group 💌.
  2. An answer, in which the answerer selects its own BUNDLE address and leaves the substitutive media untouched 🫶.

SDP Offer - from the main RTP sender:

v=0
o=alice 1122334455 1122334466 IN IP4 splicing.example.com
s=RTP Splicing Example
c=IN IP4 splicing.example.com
t=0 0
a=group:SPLICE foo 1
a=group:SPLICE bar 2
a=group:BUNDLE foo bar
m=audio 10000 RTP/AVP 0 8 97
a=mid:foo
b=AS:200
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 iLBC/8000
a=extmap:1 urn:ietf:params:rtp-hdrext:splicing-interval
a=sendonly
m=video 10002 RTP/AVP 31 32
a=mid:bar
b=AS:1000
a=rtpmap:31 H261/90000
a=rtpmap:32 MPV/90000
a=extmap:2 urn:ietf:params:rtp-hdrext:splicing-interval
a=sendonly
m=audio 20000 RTP/AVP 0 8 97
i=Substitutive audio RTP Stream
c=IN IP4 substitutive.example.com
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 iLBC/8000
a=sendonly
a=mid:1
m=video 20002 RTP/AVP 31 32
i=Substitutive video RTP Stream
c=IN IP4 substitutive.example.com
a=rtpmap:31 H261/90000
a=rtpmap:32 MPV/90000
a=mid:2
a=sendonly

SDP Answer - from the splicer:

v=0
o=bob 2808844564 2808844564 IN IP4 splicer.example.com
s=RTP Splicing Example
c=IN IP4 splicer.example.com
t=0 0
a=group:SPLICE foo 1
a=group:SPLICE bar 2
a=group:BUNDLE foo bar
m=audio 30000 RTP/AVP 0
a=mid:foo
b=AS:200
a=rtpmap:0 PCMU/8000
a=extmap:1 urn:ietf:params:rtp-hdrext:splicing-interval
a=recvonly
m=video 30000 RTP/AVP 32
a=mid:bar
b=AS:1000
a=rtpmap:32 MPV/90000
a=extmap:2 urn:ietf:params:rtp-hdrext:splicing-interval
a=recvonly
m=audio 30002 RTP/AVP 0
i=Substitutive audio RTP Stream
c=IN IP4 splicer.example.com
a=rtpmap:0 PCMU/8000
a=recvonly
a=mid:1
m=video 30004 RTP/AVP 32
i=Substitutive video RTP Stream
c=IN IP4 splicer.example.com
a=rtpmap:32 MPV/90000
a=mid:2
a=recvonly

6.4. Offer/Answer with BUNDLE: A Subset of Media Are Spliced ✨

okay so this one is the only video gets swapped scenario, bestie 🎬 the substitutive media only applies to video when splicing happens, so audio just chills 😌

the flow goes like this:

  1. an offer, where the offerer gives every bundled "m=" line in the BUNDLE group its own unique address 📍 and hands the bundled video "m=" line a substitutive media for splicing 🦋
  2. an answer, where the answerer picks its own BUNDLE address 🎀 and leaves the substitutive media totally alone 🫶

SDP Offer, from the main RTP sender:

v=0
o=alice 1122334455 1122334466 IN IP4 splicing.example.com
s=RTP Splicing Example
c=IN IP4 splicing.example.com
t=0 0
a=group:SPLICE bar 2
a=group:BUNDLE foo bar
m=audio 10000 RTP/AVP 0 8 97
a=mid:foo
b=AS:200
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 iLBC/8000
a=sendonly
m=video 10002 RTP/AVP 31 32
a=mid:bar
b=AS:1000
a=rtpmap:31 H261/90000
a=rtpmap:32 MPV/90000
a=extmap:2 urn:ietf:params:rtp-hdrext:splicing-interval
a=sendonly
m=video 20000 RTP/AVP 31 32
i=Substitutive video RTP Stream
c=IN IP4 substitutive.example.com
a=rtpmap:31 H261/90000
a=rtpmap:32 MPV/90000
a=mid:2
a=sendonly

SDP Answer, from the splicer:

v=0
o=bob 2808844564 2808844564 IN IP4 splicer.example.com
s=RTP Splicing Example
c=IN IP4 splicer.example.com
t=0 0
a=group:SPLICE bar 2
a=group:BUNDLE foo bar
m=audio 30000 RTP/AVP 0
a=mid:foo
b=AS:200
a=rtpmap:0 PCMU/8000
a=recvonly
m=video 30000 RTP/AVP 32
a=mid:bar
b=AS:1000
a=rtpmap:32 MPV/90000
a=extmap:2 urn:ietf:params:rtp-hdrext:splicing-interval
a=recvonly
m=video 30004 RTP/AVP 32
i=Substitutive video RTP Stream
c=IN IP4 splicer.example.com
a=rtpmap:32 MPV/90000
a=mid:2
a=recvonly

7. Security Considerations 🔒

okay security time, and this part is SERIOUS bestie 💀 the security considerations of the RTP specification [RFC3550] and the general mechanism for RTP header extensions [RFC8285] both apply here. the splicer can be a mixer or a translator, and all the security considerations of topologies [RFC7667] [RFC7201] for those two kinds of RTP intermediaries apply to the splicer too.

the splicer swaps some content for other content inside RTP packets 😬 which breaks any RTP-level end-to-end security, like source authentication and integrity protection. end-to-end source authentication is not possible with any known existing splicing solution. a new solution could theoretically be developed that identifies the participating entities and what each one provides, meaning the different media sources (main and substitutive) and the splicer, which does the RTP-level integration of the media payloads onto a common timeline and synchronization context. 🧩

since the splicer breaks RTP-level end-to-end security, it has to be part of the signaling context, and the necessary security associations (for example, Secure Real-time Transport Protocol (SRTP) [RFC3711] crypto contexts) have to be set up for the RTP session participants. when using SRTP, the splicer would have to be provisioned with the same security association as the main RTP sender. 🔐

if you're worried about the confidentiality of the splicing time information, the header extension defined in this document MUST also be protected 🛡️ header extension encryption [RFC6904] can be used for that. but a malicious endpoint may still get the splicing time information by other means, for example by inferring it from the communication between the main and substitutive content sources 👀. to avoid inserting invalid substitutive content, the splicer MUST have some mechanisms to authenticate the substitutive stream source.

when a malicious endpoint changes the splicing time information, the splicing might fail, since the substitutive media won't be available at the right time to arrive 😭. another case is where an attacker stops receivers from getting content from the main sender by inserting extra splicing time information. to avoid these scenarios, authentication of the RTP header extension for splicing time information SHOULD be considered.

when a splicer implemented as a mixer sends the stream to receivers, the CSRC list might get removed to make things easier for receivers that can't handle multiple sources in the RTP stream 🙃. the CSRC list is what can be used to detect RTP-level forwarding loops, as defined in Section 8.2 of [RFC3550]. without it, loops can happen, with packets looping back to a point upstream of the splicer, and that could turn into a serious denial-of-service threat 🚨. in that case, non-RTP means, for example signaling among all the participants, MUST be used to detect and resolve loops.

8. IANA Considerations 📋

8.1. RTCP Control Packet Types

based on the guidelines in [RFC8126] 📝 a new RTCP packet format got registered in the "RTCP Control Packet types (PT)" registry:

Name: SNM

Long name: Splicing Notification Message

Value: 213

Reference: This document

8.2. RTP Compact Header Extensions

IANA registered a new RTP Compact Header Extension [RFC8285] with these details 🎀:

Extension URI: urn:ietf:params:rtp-hdrext:splicing-interval

Description: Splicing Interval

Contact: Jinwei Xia <xiajinwei@huawei.com>

Reference: This document

8.3. SDP Grouping Semantic Extension

IANA also registered the new SDP grouping semantic extension called "SPLICE" in the "Semantics for the 'group' SDP Attribute" subregistry of the "Session Description Protocol (SDP) Parameters" registry ✨:

Semantics: Splice

Token: SPLICE

Reference: This document

9. References 📚

9.1. Normative References

9.2. Informative References

Acknowledgements 🫶

the authors would like to thank the following people, who reviewed this document and left super valuable comments: Colin Perkins, Bo Burman, Stephen Botzko, and Ben Campbell. legends 👑

Authors' Addresses 🦋

Jinwei Xia Huawei

Email: xiajinwei@huawei.com

Roni Even Huawei

Email: roni.even@huawei.com

Rachel Huang Huawei

Email: rachel.huang@huawei.com

Lingli Deng China Mobile

Email: denglingli@chinamobile.com

📦 where this page comes from