RFC 8829
JavaScript Session Establishment Protocol (JSEP)
Part 1 of 23 🎀
RFC 8829: JavaScript Session Establishment Protocol (JSEP) 💖
Okay bestie, here's the title block, glowed up ✨
- Number and title: RFC 8829, JavaScript Session Establishment Protocol (JSEP) 🌸
- Authors: J. Uberti (Google), C. Jennings (Cisco), and E. Rescorla, Ed. (Mozilla) 💅
- Category: Standards Track 🦋
- Date: January 2021 🍓
- ISSN:
2070-1721🔥
Abstract
This doc explains how a JavaScript app can control the signaling plane of a multimedia session through the interface in the W3C RTCPeerConnection API 🧚. It also talks about how all of that fits with the signaling protocols that already exist 💜.
Status of This Memo
This is an Internet Standards Track doc 👑.
It's a product of the Internet Engineering Task Force (IETF), so it represents the consensus of the IETF community. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). If you want more on Internet Standards, check out Section 2 of RFC 7841 🌈.
For the current status, any errata, and how to send feedback, go to https://www.rfc-editor.org/info/rfc8829 ✨.
Copyright Notice
Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved 🎀.
This doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they're in effect on the date of publication. Please read these carefully, because they spell out your rights and restrictions for this doc 🙈. Code Components pulled out of this doc must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as described in the Simplified BSD License 💅.
Table of Contents
1. Introduction
1.1. General Design of JSEP
1.2. Other Approaches Considered
1.3. Contradiction regarding bundle-only "m=" sections
2. Terminology
3. Semantics and Syntax
3.1. Signaling Model
3.2. Session Descriptions and State Machine
3.3. Session Description Format
3.4. Session Description Control
3.4.1. RtpTransceivers
3.4.2. RtpSenders
3.4.3. RtpReceivers
3.5. ICE
3.5.1. ICE Gathering Overview
3.5.2. ICE Candidate Trickling
3.5.2.1. ICE Candidate Format
3.5.3. ICE Candidate Policy
3.5.4. ICE Candidate Pool
3.5.5. ICE Versions
3.6. Video Size Negotiation
3.6.1. Creating an imageattr Attribute
3.6.2. Interpreting imageattr Attributes
3.7. Simulcast
3.8. Interactions with Forking
3.8.1. Sequential Forking
3.8.2. Parallel Forking
4. Interface
4.1. PeerConnection
4.1.1. Constructor
4.1.2. addTrack
4.1.3. removeTrack
4.1.4. addTransceiver
4.1.5. onaddtrack Event
4.1.6. createDataChannel
4.1.7. ondatachannel Event
4.1.8. createOffer
4.1.9. createAnswer
4.1.10. SessionDescriptionType
4.1.10.1. Use of Provisional Answers
4.1.10.2. Rollback
4.1.11. setLocalDescription
4.1.12. setRemoteDescription
4.1.13. currentLocalDescription
4.1.14. pendingLocalDescription
4.1.15. currentRemoteDescription
4.1.16. pendingRemoteDescription
4.1.17. canTrickleIceCandidates
4.1.18. setConfiguration
4.1.19. addIceCandidate
4.1.20. onicecandidate Event
4.2. RtpTransceiver
4.2.1. stop
4.2.2. stopped
4.2.3. setDirection
4.2.4. direction
4.2.5. currentDirection
4.2.6. setCodecPreferences
5. SDP Interaction Procedures
5.1. Requirements Overview
5.1.1. Usage Requirements
5.1.2. Profile Names and Interoperability
5.2. Constructing an Offer
5.2.1. Initial Offers
5.2.2. Subsequent Offers
5.2.3. Options Handling
5.2.3.1. IceRestart
5.2.3.2. VoiceActivityDetection
5.3. Generating an Answer
5.3.1. Initial Answers
5.3.2. Subsequent Answers
5.3.3. Options Handling
5.3.3.1. VoiceActivityDetection
5.4. Modifying an Offer or Answer
5.5. Processing a Local Description
5.6. Processing a Remote Description
5.7. Processing a Rollback
5.8. Parsing a Session Description
5.8.1. Session-Level Parsing
5.8.2. Media Section Parsing
5.8.3. Semantics Verification
5.9. Applying a Local Description
5.10. Applying a Remote Description
5.11. Applying an Answer
6. Processing RTP/RTCP
7. Examples
7.1. Simple Example
7.2. Detailed Example
7.3. Early Transport Warmup Example
8. Security Considerations
9. IANA Considerations
10. References
10.1. Normative References
10.2. Informative References
Appendix A. SDP ABNF Syntax
Acknowledgements
Authors' Addresses1. Introduction ✨
This doc describes how the W3C Web Real-Time Communication (WebRTC) RTCPeerConnection interface [W3C.webrtc] gets used to control the setup, management, and teardown of a multimedia session 🦋.
1.1. General Design of JSEP 🌈
WebRTC call setup was designed to focus on controlling the media plane, and it leaves signaling-plane behavior up to the app as much as possible 💖. The reason is that different apps might want different protocols, like the existing SIP call signaling protocol, or something custom for a particular app, maybe for a brand-new use case 🧚. In this approach, the key info that needs to be swapped is the multimedia session description, which carries the transport and media configuration needed to set up the media plane 🎀.
With all that in mind, this doc describes the JavaScript Session Establishment Protocol (JSEP), which gives you full control of the signaling state machine from JavaScript 👑. As we said, JSEP assumes a JavaScript app runs inside a runtime that has WebRTC APIs (the "JSEP implementation") ✨. The JSEP implementation is almost totally divorced from the core signaling flow. That flow is handled by the JavaScript, using two interfaces: (1) passing in local and remote session descriptions, and (2) interacting with the Interactive Connectivity Establishment (ICE) state machine [RFC8445] 💅. The combo of the JSEP implementation and the JavaScript app is called a "JSEP endpoint" throughout this doc 💜.
This doc talks about JSEP as if it always happens between two JSEP endpoints. But heads up, in lots of cases it'll really be between a JSEP endpoint and some kind of server, like a gateway or a Multipoint Control Unit (MCU) 🫶. The JSEP endpoint doesn't see the difference at all. It just follows whatever the API tells it to do 🌸.
JSEP's handling of session descriptions is simple and straightforward 🍓. Whenever an offer/answer exchange is needed, the initiating side makes an offer by calling a createOffer API. The app then uses that offer to set up its local config through the setLocalDescription API. The offer then gets sent to the remote side over whatever signaling mechanism the app likes (e.g., WebSockets) 🔥. When the offer arrives, the remote party installs it with the setRemoteDescription API 🙈.
To finish the offer/answer exchange, the remote party uses the createAnswer API to generate a proper answer, applies it with the setLocalDescription API, and sends the answer back to the initiator over the signaling channel 💫. When the initiator gets that answer, it installs it with the setRemoteDescription API, and initial setup is complete 🎉. This process can be repeated for more offer/answer exchanges 🔁.
Now, about ICE [RFC8445]: JSEP decouples the ICE state machine from the overall signaling state machine 🧠. The ICE state machine has to stay in the JSEP implementation, because only the implementation has the knowledge it needs about candidates and other transport info 🔒. Splitting things up this way gives extra flexibility to protocols that decouple session descriptions from transport 🦋. For example, in traditional SIP, every offer or answer is self-contained, with both the session descriptions and the transport info 📦. But [RFC8840] lets SIP be used with Trickle ICE [RFC8838], where the session description can be sent right away and the transport info can be sent whenever it's available ⚡. Sending transport info separately can mean faster ICE and DTLS startup, since ICE checks can kick off as soon as any transport info shows up, instead of waiting for all of it 🏃♀️. JSEP's decoupling of the ICE and signaling state machines means it can handle either model 💖.
Even though JSEP abstracts signaling, the app still has to know about the signaling process 👀. The app doesn't need to understand what's inside session descriptions to set up a call, but it does have to call the right APIs at the right times 🕰️. It also has to convert the session descriptions and ICE info into the defined messages of its chosen signaling protocol, and do the reverse on messages it gets from the other side 🔄.
One way to make life easier for the app is a JavaScript library that hides all this complexity from the developer 🧚. That library would implement a given signaling protocol along with its state machine and serialization code, and give the app developer a higher-level, call-oriented interface 🎀. For example, libraries exist that implement the SIP [RFC3261] and Extensible Messaging and Presence Protocol (XMPP) [RFC6120] signaling protocols on top of the JSEP API 👑. So JSEP gives experienced developers greater control, without piling extra complexity onto novice developers 🌈✨.
1.2. Other Approaches Considered 🦋
Okay so before JSEP won, the IETF had a few other ideas on the table, and honestly the reasons they were passed up are kind of iconic. 💅
First up, one idea was a lightweight signaling protocol. Instead of handing session descriptions to the API, the API would make and consume messages from that protocol. It sounds more high-level and cute, but it would have shoved a lot more signaling control into the JSEP implementation, which would then have to understand and handle things like signaling glare (see [RFC3264], Section 4). That's a lot of drama for one implementation 😭
Second, someone suggested splitting the media control objects away from session descriptions and giving the app APIs to control each component directly. This got rejected for two reasons. First, even a simple example would need a huge pile of code just to orchestrate everything. Second, it would create a massive API surface that everyone would have to agree on and document. Plus, those API points could be called in any order, which means way messier interactions with the media subsystem than the JSEP approach, where session descriptions get evaluated and applied in a set way. Chaotic energy, not it 🙈
Third, there was a variation that kept the session-description-oriented API but moved the offer and answer generation out of the JSEP implementation. Instead of createOffer/createAnswer methods, this would expose a getCapabilities API so the app could build its own session descriptions. The problem? The app would have to do way more work. It would need to know how to generate session descriptions from capabilities, and especially how to build the correct answer from any arbitrary offer plus the supported capabilities. A library could help with that, sure, but then you'd basically be forced to use that library even for a simple example. Providing createOffer/createAnswer avoids all of that, and that's why it's the move ✨
1.3. Contradiction regarding bundle-only "m=" sections 🌸
Okay, this part is a little awkward, but it's important. Since the WebRTC specification documents were approved, the IETF noticed an inconsistency between the document specifying JSEP and the document specifying BUNDLE (this RFC and [RFC8843], respectively). Instead of delaying publication even more to sort it out, the documents are being published exactly as they were originally approved. The IETF plans to restart work on these technologies, and revised versions will be published as soon as a resolution is available. Bestie, this is a "stay tuned" moment 🎀
The specific problem is how "m=" sections designated as bundle-only get handled, which is covered in Section 4.1.1 of this RFC. Right now, the specs disagree with each other and with existing browsers:
- JSEP says those "m=" sections should use port zero and add an "a=bundle-only" attribute in initial offers, but not in answers or subsequent offers.
- BUNDLE says those "m=" sections should be marked the way described above, but in all offers and answers.
- Most current browsers don't mark any "m=" sections with port zero. They use the same port for all bundled "m=" sections instead, and some others follow the JSEP behavior.
So yeah, the vibes are split right now, but the plan is to fix it 💜
2. Terminology ✨
Okay, the key words are the main characters of this whole document, so pay attention 👑 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. So if you see them in caps, they mean the official BCP 14 thing. Lowercase is just regular words, no cap 🌈
3. Semantics and Syntax 🧚
Now we get into the actual meat of JSEP.
3.1. Signaling Model 🦋
JSEP does not specify a particular signaling model or state machine. The only general requirement is that session descriptions get exchanged in the way described by [RFC3264] (offer/answer), so both sides of the session know how to run the session. JSEP gives you mechanisms to create offers and answers and to apply them to a session. However, the JSEP implementation is totally decoupled from how those offers and answers actually get communicated to the remote side, including addressing, retransmission, forking, and glare handling. All of that is left entirely up to the application. The app has complete control over which offers and answers get handed to the implementation, and when 💅
Here's the signaling picture:
+-----------+ +-----------+
| Web App |<--- App-Specific Signaling -->| Web App |
+-----------+ +-----------+
^ ^
| SDP | SDP
V V
+-----------+ +-----------+
| JSEP |<----------- Media ------------>| JSEP |
| Impl. | | Impl. |
+-----------+ +-----------+
Figure 1: JSEP Signaling Model3.2. Session Descriptions and State Machine 🎀
okay so, to actually get the media plane up, the JSEP implementation needs specific parameters telling it what to send to the other side and how to handle the media coming in 💖 Those parameters come from swapping session descriptions in offers and answers, and there are some sneaky details in that process that the JSEP APIs have to handle.
Whether a session description is about the local side or the remote side changes what it means 👀 For example, the list of codecs you send to a remote party says what you're willing to receive. When that gets intersected with the codecs they support, it decides what they should send. BUT not every parameter works like that 🙈 Some are declarative, so the remote side has to either accept them or reject them altogether. The TLS fingerprints [RFC8122] used with DTLS [RFC6347] are a good example. They're calculated from the local certificate(s) that were offered, so they aren't up for negotiation 🔐
Also, different RFCs put different rules on offers versus answers. For example, an offer can propose any number of "m=" sections (media descriptions, see [RFC4566], Section 5.14), but an answer has to contain the exact same number as the offer 🧚
Lastly, the exact media parameters aren't known until an offer and an answer have both been exchanged. Still, the offerer might get ICE checks, and maybe even media (like after a re-offer once the connection is already up), before the answer shows up 😭 So the offerer's media handler needs to know the details of the offer before the answer arrives, or it can't process that incoming media properly.
So, to handle session descriptions properly, the JSEP implementation needs to:
- Know whether a session description is about the local or the remote side.
- Know whether a session description is an offer or an answer.
- Allow the offer to be specified independently of the answer.
JSEP handles this by adding both setLocalDescription and setRemoteDescription methods, and by having session description objects carry a type field that says what kind of description they are 💅 That covers all three needs for both roles. The offerer calls setLocalDescription(sdp [offer]) first and then setRemoteDescription(sdp [answer]) later. The answerer calls setRemoteDescription(sdp [offer]) first and then setLocalDescription(sdp [answer]) later. Iconic, honestly ✨
While the offer/answer exchange is happening, the outstanding offer is "pending" on both the offerer and the answerer, since it could still get accepted or rejected 🦋 If this is a re-offer, each side also has "current" local and remote descriptions, which show the result of the last offer/answer exchange. Sections 4.1.14, 4.1.16, 4.1.13, and 4.1.15 go deeper on pending and current descriptions 🌸
JSEP also lets the application treat an answer as provisional. Provisional answers let an answerer send initial session parameters back to the offerer so the session can start, while a final answer can still be sent later 🎀 The final answer is a big deal in the offer/answer model. Once the caller gets it, any extra resources it grabbed can be released, because the exact session configuration is now known. Those resources can include things like extra ICE components, Traversal Using Relays around NAT (TURN) candidates, or video decoders 🍓 Provisional answers don't do any of that deallocation, so multiple different provisional answers, each with its own codec choices, transport parameters, and so on, can get received and applied during call setup 🫶 And the final answer doesn't have to match any of the provisional ones it followed.
In [RFC3264], the signaling-level rule is that only one offer can be outstanding for a given session. At the JSEP level, though, a new offer can be generated at any point 💜 For example, with SIP signaling, if one offer gets sent and then canceled with a SIP CANCEL, another offer can be generated even though no answer ever came back for the first one. To make that work, the JSEP media layer can provide an offer through the createOffer method whenever the JavaScript application needs one for signaling. The answerer can send back zero or more provisional answers, then wrap up the offer/answer exchange with a final answer. Here's the state machine for all of that 👑
setRemote(OFFER) setLocal(PRANSWER)
/-----\ /-----\
| | | |
v | v |
+---------------+ | +---------------+ |
| |----/ | |----/
| have- | setLocal(PRANSWER) | have- |
| remote-offer |------------------- >| local-pranswer|
| | | |
| | | |
+---------------+ +---------------+
^ | |
| | setLocal(ANSWER) |
setRemote(OFFER) | |
| V setLocal(ANSWER) |
+---------------+ |
| | |
| |<---------------------------+
| stable |
| |<---------------------------+
| | |
+---------------+ setRemote(ANSWER) |
^ | |
| | setLocal(OFFER) |
setRemote(ANSWER) | |
| V |
+---------------+ +---------------+
| | | |
| have- | setRemote(PRANSWER) |have- |
| local-offer |------------------- >|remote-pranswer|
| | | |
| |----\ | |----\
+---------------+ | +---------------+ |
^ | ^ |
| | | |
\-----/ \-----/
setLocal(OFFER) setRemote(PRANSWER)
Figure 2: JSEP State MachineApart from these state transitions, there's no other difference in how provisional ("pranswer") and final ("answer") answers are handled. Same vibe, different label 🌈
3.3. Session Description Format 🧁
JSEP's session descriptions use Session Description Protocol (SDP) syntax as their internal representation. Honestly, SDP isn't the most fun to manipulate from JavaScript 🙃 BUT it's widely accepted and keeps getting new features, so any other encoding would have to keep up with every SDP change, at least until it becomes as popular as SDP is ✨
To future-proof things, the SDP syntax gets wrapped in a SessionDescription object. That object can be built from SDP and serialized back out to SDP. If future specs agree on a JSON format for session descriptions, this object could easily be extended to generate and consume that JSON too 🦋
As detailed below, most applications should be able to treat the SessionDescriptions produced and consumed by these API calls as opaque blobs. In other words, the application won't need to read or change them 💅
3.4. Session Description Control 🎀
To give the application control over common session parameters, JSEP provides control surfaces that tell the JSEP implementation how to generate session descriptions. That way, JavaScript usually doesn't have to modify session descriptions directly 💖
Changes to these objects show up in the session descriptions generated by later createOffer/createAnswer calls. Cute and powerful 🌸
3.4.1. RtpTransceivers ✨
RtpTransceivers let the application control the RTP media associated with one "m=" section. Each RtpTransceiver has an RtpSender and an RtpReceiver, which the application can use to control sending and receiving RTP media 📡 The application can also modify the RtpTransceiver directly, for example by stopping it.
RtpTransceivers generally have a 1:1 mapping with "m=" sections. There can be more RtpTransceivers than "m=" sections, though, when RtpTransceivers have been created but not yet associated with an "m=" section, or when they've been stopped and disassociated from "m=" sections 🙈 An RtpTransceiver counts as associated with an "m=" section if its media identification (mid) property is non-null. Otherwise it's disassociated. The associated "m=" section is worked out from a mapping between transceivers and "m=" section indices, which gets formed when an offer is created or a remote offer is applied 🧚
An RtpTransceiver is never associated with more than one "m=" section. Once a session description is applied, every "m=" section is associated with exactly one RtpTransceiver. The exception is when an "m=" section has been rejected, as discussed in Section 5.2.2 below. In that case the "m=" section gets "recycled" and associated with a new RtpTransceiver that has a new MID value 🔁
RtpTransceivers can be created explicitly by the application, or implicitly by calling setRemoteDescription with an offer that adds new "m=" sections 👑
3.4.2. RtpSenders 🚀
RtpSenders let the application control how RTP media gets sent 💌 An RtpSender is conceptually responsible for the outgoing RTP stream(s) described by an "m=" section. That includes encoding the attached MediaStreamTrack, sending RTP media packets, and generating/processing the RTP Control Protocol (RTCP) for the outgoing RTP stream(s) 🎧
3.4.3. RtpReceivers 🌷
RtpReceivers let the application inspect how RTP media is received 👀 An RtpReceiver is conceptually responsible for the incoming RTP stream(s) described by an "m=" section. That includes processing received RTP media packets, decoding the incoming stream(s) to produce a remote MediaStreamTrack, and generating/processing RTCP for the incoming RTP stream(s) 🍭
3.5. ICE 🧊
3.5.1. ICE Gathering Overview ✨
okay so JSEP gathers ICE candidates only when the app actually needs them 🫶 That collecting step is called a gathering phase, and it gets triggered either when a new or recycled m= section shows up in the local session description, or when the description has new ICE credentials in it (that's an ICE restart). The app can ask for new credentials on purpose, or JSEP can do it on its own when the ICE configuration changes 🦋
When the ICE config changes in a way that needs a fresh gathering phase, a needs-ice-restart bit gets set 🚩 While that bit is set, calls to the createOffer API will generate brand-new ICE credentials. The bit gets cleared when setLocalDescription is called with new ICE credentials from either an offer or an answer, so it works for both a locally and a remotely initiated ICE restart 💅
Once a new gathering phase starts, the ICE agent lets the app know gathering is happening through a state change event. Then each new candidate gets handed to the app through an onicecandidate event, and those candidates also get automatically added to the current and/or pending local session description 🌸 Finally, when everything has been gathered, one last onicecandidate event fires to say gathering is complete 🎀
Quick note: gathering phases only collect candidates for new, recycled, or restarting m= sections. Other m= sections keep using the candidates they already have 🧚 Also, if an m= section is bundled (either through a successful bundle negotiation or by being marked as bundle-only), candidates only get gathered and exchanged for that section if and only if its MID item is a BUNDLE-tag, as described in [RFC8843]. Lowkey the most important detail in this whole subsection 👑
3.5.2. ICE Candidate Trickling 📬
Candidate trickling is when a caller sends candidates to the callee little by little, after the initial offer has already gone out. The semantics of "Trickle ICE" are defined in [RFC8838]. Why it's so iconic: the callee can start acting on the call and setting up the ICE (and maybe DTLS) connections right away, without waiting for the caller to gather every possible candidate ⚡ That means faster media setup in cases where gathering didn't happen before the call started.
JSEP supports optional candidate trickling through the APIs we covered above, which give control and feedback over the candidate gathering process 💖 Apps that support trickling can send the initial offer immediately and then send individual candidates as they get notified of them. Apps that don't support it can just wait for the indication that gathering is complete, and then create and send their offer with all the candidates at that point 🍓
Upon receipt of trickled candidates, the receiving app supplies them to its ICE agent. That triggers the ICE agent to start using the new remote candidates for connectivity checks 🔥
3.5.2.1. ICE Candidate Format 🎀
In JSEP, ICE candidates are abstracted by an IceCandidate object, and just like session descriptions, SDP syntax is used for the internal representation 🧩
The candidate details go in an IceCandidate field, using the same SDP syntax as the "candidate-attribute" field defined in [RFC8839], Section 5.1. Heads up: this field does not contain an "a=" prefix, as shown in this example:
candidate:1 1 UDP 1694498815 192.0.2.33 10000 typ hostThe IceCandidate object has a field that says which ICE username fragment (ufrag) it's associated with, as defined in [RFC8839], Section 5.4. That value tells you which session description (and therefore which gathering phase) the IceCandidate belongs to, which helps resolve ambiguities during ICE restarts 🫧 If this field is absent in a received IceCandidate (perhaps when talking to a non-JSEP endpoint), the most recently received session description is assumed.
The IceCandidate object also has fields that indicate which "m=" section it's associated with. That can be identified in one of two ways: either by an "m=" section index or by a MID. The "m=" section index is zero-based, with index N referring to the N+1th "m=" section in the session description referenced by this IceCandidate. The MID is a "media stream identification" value, as defined in [RFC5888], Section 4. It's a more robust way to identify the "m=" section in the session description, using the MID of the associated RtpTransceiver object. (That MID may have been generated locally by the answerer when it's interacting with a non-JSEP endpoint that doesn't support the MID attribute, as discussed in Section 5.10 below.) If the MID field is present in a received IceCandidate, it MUST be used for identification; otherwise, the "m=" section index is used instead 💜
Implementations MUST be prepared to receive objects with some fields missing, as mentioned above. 🙈
3.5.3. ICE Candidate Policy 🔒
Typically, when gathering ICE candidates, the JSEP implementation will gather every possible form of initial candidate: host, server-reflexive, and relay 🌈 But sometimes apps want more specific control over gathering because of privacy or related concerns. For example, someone might only want relay candidates so they leak as little location information as possible (keeping in mind that this choice comes with operational costs 💸). So JSEP lets the application restrict which ICE candidates get used in a session. This filtering is applied on top of any restrictions the implementation chooses to enforce about which IP addresses are permitted for the application, as discussed in [RFC8828].
There are also cases where the app wants to change which types of candidates get used while the session is active 🔄 A prime example: a callee may initially want to use only relay candidates, to avoid leaking location information to an arbitrary caller, and then switch to using all candidates (for lower operational cost) once the user has said they want to take the call 📞 For this scenario, the JSEP implementation MUST allow the candidate policy to be changed mid-session, subject to the interactions with local policy mentioned above.
To administer the ICE candidate policy, the JSEP implementation determines the current setting at the start of each gathering phase. Then, during the gathering phase, the implementation MUST NOT expose candidates disallowed by the current policy to the application, use them as the source of connectivity checks, or indirectly expose them via other fields, such as the raddr/rport attributes for other ICE candidates. Later, if a different policy is specified by the application, the application can apply it by kicking off a new gathering phase via an ICE restart 🎯
3.5.4. ICE Candidate Pool 🫧
JSEP apps typically tell the JSEP implementation to begin ICE gathering through the information supplied to setLocalDescription, since the local description indicates how many ICE components will be needed and which candidates must be gathered 🧠 But to speed things up when the app already knows how many ICE components it'll use, it can ask the implementation to gather a pool of potential ICE candidates for rapid media setup 🚀
When setLocalDescription is eventually called and the JSEP implementation prepares to gather the needed ICE candidates, it SHOULD start by checking whether any candidates are available in the pool. If there are candidates in the pool, they SHOULD be handed to the application immediately via the ICE candidate event. If the pool becomes depleted, either because a larger-than-expected number of ICE components are used or because the pool hasn't had enough time to gather candidates, the remaining candidates are gathered as usual. This only occurs for the first offer/answer exchange, after which the candidate pool is emptied and no longer used.
One example where this is useful: an app that expects an incoming call at some point in the future and wants to minimize the time it takes to establish connectivity, to avoid clipping the start of the media 😭 By pre-gathering candidates into the pool, it can exchange them and start sending connectivity checks from these candidates almost immediately upon receiving a call. Note, though, that by holding on to these pre-gathered candidates (which are kept alive as long as they may be needed), the app will consume resources on the STUN/TURN servers it's using. ("STUN" stands for "Session Traversal Utilities for NAT".) 🌸
3.5.5. ICE Versions 🦋
While this specification formally relies on [RFC8445], at the time of its publication, the majority of WebRTC implementations support the version of ICE described in [RFC5245]. The "ice2" attribute defined in [RFC8445] can be used to detect the version in use by a remote endpoint and to provide a smooth transition from the older specification to the newer one ✨ Implementations MUST be able to accept remote descriptions that do not have the "ice2" attribute. 💅
3.6. Video Size Negotiation 🎞️
Video size negotiation is the process where a receiver uses the a=imageattr SDP attribute [RFC6236] to indicate what video frame sizes it's capable of receiving 📐 A receiver may have hard limits on what its video decoder can process, or it may have some maximum set by policy. By specifying these limits in an a=imageattr attribute, JSEP endpoints can try to make sure the remote sender transmits video at an acceptable resolution. However, when communicating with a non-JSEP endpoint that doesn't understand this attribute, any signaled limits may be exceeded, and the JSEP implementation MUST handle this gracefully, e.g., by discarding the video 🥺
Note that certain codecs support transmission of samples with aspect ratios other than 1.0 (i.e., non-square pixels). JSEP implementations will not transmit non-square pixels, but they SHOULD receive and render such video with the correct aspect ratio. However, sample aspect ratio has no impact on the size negotiation described below; all dimensions are measured in pixels, whether square or not 📏
3.6.1. Creating an imageattr Attribute ✨
okay so the receiver first mashes together any local limits it knows about (like what its hardware decoder can handle, or local policy) to figure out the absolute smallest and biggest sizes it can take in 💅. If there are no known local limits, the a=imageattr attribute SHOULD be omitted. If those local limits mean it can't receive any video at all (the degenerate case of no permitted resolutions, not me being dramatic 💀), then the a=imageattr attribute MUST be omitted, and the m= section MUST be marked as sendonly/inactive, as appropriate.
Otherwise, an a=imageattr attribute gets made with a recv direction, and the resolution space from that intersection gets used to set its minimum and maximum x= and y= values 🦋.
The rules here are one single set of preferences, so the a=imageattr q= value isn't important. It SHOULD be set to 1.0 🌸.
The a=imageattr field is payload type specific. When all the video codecs supported have the same capabilities, using a single attribute with the wildcard payload type (*) is RECOMMENDED ✨. However, when the supported video codecs have different limitations, specific a=imageattr attributes MUST be inserted for each payload type 🙈.
As an example, consider a system with a multiformat video decoder, which is capable of decoding any resolution from 48x48 to 720p. In this case, the implementation would generate this attribute:
a=imageattr:* recv [x=[48:1280],y=[48:720],q=1.0]This declaration indicates that the receiver is capable of decoding any image resolution from 48x48 up to 1280x720 pixels 🎀.
3.6.2. Interpreting imageattr Attributes 🧚
[RFC6236] defines a=imageattr as an advisory field, which means it doesn't absolutely constrain the video formats the sender can use. It just gives an indication of the preferred values 💭.
This specification prescribes behavior that is more specific 👑. When a MediaStreamTrack, which is producing video of a certain resolution (the "track resolution"), is attached to an RtpSender, which is encoding the track video at the same or lower resolution(s) (the "encoder resolutions"), and a remote description is applied that references the sender and contains valid a=imageattr recv attributes, it MUST follow the rules below to ensure that the sender does not transmit a resolution that would exceed the size criteria specified in the attributes. These rules MUST be followed as long as the attributes remain present in the remote description, including cases in which the track changes its resolution or is replaced with a different track 🔥.
Depending on how the RtpSender is configured, it may be producing a single encoding at a certain resolution or, if simulcast (Section 3.7) has been negotiated, multiple encodings, each at their own specific resolution. In addition, depending on the configuration, each encoding may have the flexibility to reduce resolution when needed or may be locked to a specific output resolution 🌈.
For each encoding being produced by the RtpSender, the set of a=imageattr recv attributes in the corresponding m= section of the remote description is processed to determine what should be transmitted. Only attributes that reference the media format selected for the encoding are considered. Each such attribute is evaluated individually, starting with the attribute with the highest q= value. If multiple attributes have the same q= value, they are evaluated in the order they appear in their containing m= section. Note that while JSEP endpoints will include at most one a=imageattr recv attribute per media format, JSEP endpoints may receive session descriptions from non-JSEP endpoints with m= sections that contain multiple such attributes 🫶.
For each a=imageattr recv attribute, the following rules are applied. If this processing is successful, the encoding is transmitted accordingly, and no further attributes are considered for that encoding. Otherwise, the next attribute is evaluated, in the aforementioned order. If none of the supplied attributes can be processed successfully, the encoding MUST NOT be transmitted, and an error SHOULD be raised to the application 😭.
- The limits from the attribute are compared to the encoder resolution. Only the specific limits mentioned below are considered; any other values, such as picture aspect ratio, MUST be ignored. When considering a MediaStreamTrack that is producing rotated video, the unrotated resolution MUST be used for the checks. This is required regardless of whether the receiver supports performing receive-side rotation (e.g., through Coordination of Video Orientation (CVO) [TS26.114]), as it significantly simplifies the matching logic 🧠.
- If the attribute includes a
sar=(sample aspect ratio) value set to something other than1.0, indicating that the receiver wants to receive non-square pixels, this cannot be satisfied and the attribute MUST NOT be used 🙅♀️. - If the encoder resolution exceeds the maximum size permitted by the attribute and the encoder is allowed to adjust its resolution, the encoder SHOULD apply downscaling in order to satisfy the limits. Downscaling MUST NOT change the picture aspect ratio of the encoding, ignoring any trivial differences due to rounding. For example, if the encoder resolution is 1280x720 and the attribute specified a maximum of 640x480, the expected output resolution would be 640x360. If downscaling cannot be applied, the attribute MUST NOT be used 📐.
- If the encoder resolution is less than the minimum size permitted by the attribute, the attribute MUST NOT be used; the encoder MUST NOT apply upscaling. JSEP implementations SHOULD avoid this situation by allowing receipt of arbitrarily small resolutions, perhaps via fallback to a software decoder 🍓.
- If the encoder resolution is within the maximum and minimum sizes, no action is needed 🥳.
3.7. Simulcast 🌈
JSEP supports simulcast transmission of a MediaStreamTrack, where multiple encodings of the source media can be transmitted within the context of a single m= section. The current JSEP API is designed to let applications send simulcasted media but only receive a single encoding 💖. This allows for multi-user scenarios where each sending client sends multiple encodings to a server, which then, for each receiving client, chooses the appropriate encoding to forward 🧚♀️.
Applications request support for simulcast by configuring multiple encodings on an RtpSender. Upon generation of an offer or answer, these encodings are indicated via SDP markings on the corresponding m= section, as described below. Receivers that understand simulcast and are willing to receive it will also include SDP markings to indicate their support, and JSEP endpoints will use these markings to determine whether simulcast is permitted for a given RtpSender. If simulcast support is not negotiated, the RtpSender will only use the first configured encoding ✨.
Note that the exact simulcast parameters are up to the sending application. While the aforementioned SDP markings are provided to ensure that the remote side can receive and demux multiple simulcast encodings, the specific resolutions and bitrates to be used for each encoding are purely a send-side decision in JSEP 🦋.
JSEP currently does not provide a mechanism to configure receipt of simulcast. This means that if simulcast is offered by the remote endpoint, the answer generated by a JSEP endpoint will not indicate support for receipt of simulcast, and as such the remote endpoint will only send a single encoding per m= section 🙈.
In addition, JSEP does not provide a mechanism to handle an incoming offer requesting simulcast from the JSEP endpoint. This means that setting up simulcast in the case where the JSEP endpoint receives the initial offer requires out-of-band signaling or SDP inspection. However, in the case where the JSEP endpoint sets up simulcast in its initial offer, any established simulcast streams will continue to work upon receipt of an incoming re-offer. Future versions of this specification may add additional APIs to handle the incoming initial offer scenario 🔮.
When using JSEP to transmit multiple encodings from an RtpSender, the techniques from [RFC8853] and [RFC8851] are used. Specifically, when multiple encodings have been configured for an RtpSender, the m= section for the RtpSender will include an a=simulcast attribute, as defined in [RFC8853], Section 5.1, with a send simulcast stream description that lists each desired encoding, and no recv simulcast stream description. The m= section will also include an a=rid attribute for each encoding, as specified in [RFC8851], Section 4. The use of Restriction Identifiers (RIDs, also called rid-ids or RtpStreamIds) allows the individual encodings to be disambiguated even though they are all part of the same m= section 💅.
3.8. Interactions with Forking 🍓
Some call signaling systems allow various types of forking where an SDP Offer may be provided to more than one device. For example, SIP [RFC3261] defines both a "parallel search" and "sequential search" 📞. Although these are primarily signaling-level issues that are outside the scope of JSEP, they do have some impact on the configuration of the media plane that is relevant. When forking happens at the signaling layer, the JavaScript application responsible for the signaling needs to make the decisions about what media should be sent or received at any point in time, as well as which remote endpoint it should communicate with. JSEP is used to make sure the media engine can make the RTP and media perform as required by the application 🫶. The basic operations that the applications can have the media engine do are as follows:
- Start exchanging media with a given remote peer, but keep all the resources reserved in the offer.
- Start exchanging media with a given remote peer, and free any resources in the offer that are not being used.
3.8.1. Sequential Forking 💜
Sequential forking involves a call being dispatched to multiple remote callees, where each callee can accept the call, but only one active session ever exists at a time. No mixing of received media is performed 🎧.
JSEP handles sequential forking well, allowing the application to easily control the policy for selecting the desired remote endpoint 👑. When an answer arrives from one of the callees, the application can choose to apply it as either (1) a provisional answer, leaving open the possibility of using a different answer in the future or (2) a final answer, ending the setup flow.
In a "first-one-wins" situation, the first answer will be applied as a final answer, and the application will reject any subsequent answers. In SIP parlance, this would be ACK + BYE 💀.
In a "last-one-wins" situation, all answers would be applied as provisional answers, and any previous call leg will be terminated. At some point, the application will end the setup process, perhaps with a timer. At this point, the application could reapply the pending remote description as a final answer 🌸.
3.8.2. Parallel Forking 🍓
okay so parallel forking is when one call gets sent out to multiple remote callees at the same time, and any of them can pick up, and you can end up with multiple active signaling sessions at once 📞✨ bestie, it's giving chaos but make it organized. 💅 If several callees send media at the same time, the ways to handle that are described in [RFC3960], Section 3.1. Most SIP devices today only swap media with one device at a time and don't try to mix multiple early media audio sources, because that's a total mess 🙈 like, imagine mixing a European ringback tone with the North American ringback tone. The result wouldn't sound like either one and the user would be so confused 😭 If the signaling application only wants to swap media with one remote endpoint at a time, then from the media engine's point of view, this is exactly like the sequential forking case 🦋
Now the parallel forking case where the JavaScript application wants to chat with multiple peers at once is a little more complicated, but the JavaScript app can follow the strategy [RFC3960] describes, using UPDATE 🎀 The UPDATE approach lets the signaling set up a separate media flow for each peer the app wants to exchange media with. In JSEP, the offer used in the UPDATE gets made by simply creating a new PeerConnection (see Section 4.1) and making sure the same local media streams have been added to this new PeerConnection. Then the new PeerConnection object produces an SDP offer that the signaling can use to do the UPDATE strategy discussed in [RFC3960] 💖
So as a result of sharing the media streams, the app ends up with N parallel PeerConnection sessions, each with its own local and remote description and its own local and remote addresses 🌈 You can manage the media flowing from these sessions using setDirection (see Section 4.2.3), or the app can just play out the media from all the sessions mixed together 🧚 And if the app only wants to keep one session, it can simply terminate the sessions it doesn't need anymore 👑 bestie, that's the whole plot twist.
4. Interface ✨
This section details the basic operations that must be present to implement JSEP functionality. The actual API exposed in the W3C API may have somewhat different syntax but should map easily to these concepts. 🫶
4.1. PeerConnection 🎀
4.1.1. Constructor 💅
The PeerConnection constructor lets the application specify global parameters for the media session, like the STUN/TURN servers and credentials to use when gathering candidates, plus the initial ICE candidate policy and pool size, and the bundle policy to use 🌸
If an ICE candidate policy is specified, it works as described in Section 3.5.3, so the JSEP implementation only surfaces the permitted candidates (including any implementation-internal filtering) to the application and only uses those candidates for connectivity checks. The available policies are as follows:
all: All candidates permitted by implementation policy will be gathered and used.
relay: All candidates except relay candidates will be filtered out. This obfuscates the location information that might be ascertained by the remote peer from the received candidates. Depending on how the application deploys and chooses relay servers, this could obfuscate location to a metro or possibly even global level. 🗺️
The default ICE candidate policy MUST be set to "all", as this is generally the desired policy and also typically reduces the use of application TURN server resources significantly. 🦋
If a size is specified for the ICE candidate pool, this indicates the number of ICE components to pre-gather candidates for. Because pre-gathering results in utilizing STUN/TURN server resources for potentially long periods of time, this MUST only occur upon application request, and therefore the default candidate pool size MUST be zero. 🙅♀️
The application can specify its preferred policy regarding use of bundle, the multiplexing mechanism defined in [RFC8843]. Regardless of policy, the application will always try to negotiate bundle onto a single transport and will offer a single bundle group across all "m=" sections; use of this single transport is contingent upon the answerer accepting bundle. However, by specifying a policy from the list below, the application can control exactly how aggressively it will try to bundle media streams together, which affects how it will interoperate with a non-bundle-aware endpoint. When negotiating with a non-bundle-aware endpoint, only the streams not marked as bundle-only streams will be established. 🧸
The available policies are as follows:
balanced: The first "m=" section of each type (audio, video, or application) will contain transport parameters, which will allow an answerer to unbundle that section. The second and any subsequent "m=" sections of each type will be marked bundle-only. The result is that if there are N distinct media types, then candidates will be gathered for N media streams. This policy balances desire to multiplex with the need to ensure that basic audio and video can still be negotiated in legacy cases. When acting as answerer, if there is no bundle group in the offer, the implementation will reject all but the first "m=" section of each type. ⚖️
max-compat: All "m=" sections will contain transport parameters; none will be marked as bundle-only. This policy will allow all streams to be received by non-bundle-aware endpoints but will require separate candidates to be gathered for each media stream. 🛡️
max-bundle: Only the first "m=" section will contain transport parameters; all streams other than the first will be marked as bundle-only. This policy aims to minimize candidate gathering and maximize multiplexing, at the cost of less compatibility with legacy endpoints. When acting as answerer, the implementation will reject any "m=" sections other than the first "m=" section, unless they are in the same bundle group as that "m=" section. 🚀
As it provides the best trade-off between performance and compatibility with legacy endpoints, the default bundle policy MUST be set to "balanced". 💜
The application can specify its preferred policy regarding use of RTP/RTCP multiplexing [RFC5761] using one of the following policies:
negotiate: The JSEP implementation will gather both RTP and RTCP candidates but also will offer "a=rtcp-mux", thus allowing for compatibility with either multiplexing or non-multiplexing endpoints.
require: The JSEP implementation will only gather RTP candidates and will insert an "a=rtcp-mux-only" indication into any new "m=" sections in offers it generates. This halves the number of candidates that the offerer needs to gather. Applying a description with an "m=" section that does not contain an "a=rtcp-mux" attribute will cause an error to be returned. 🌟
The default multiplexing policy MUST be set to "require". Implementations MAY choose to reject attempts by the application to set the multiplexing policy to "negotiate". 🫧
4.1.2. addTrack 🎵
The addTrack method adds a MediaStreamTrack to the PeerConnection, using the MediaStream argument to associate the track with other tracks in the same MediaStream, so that they can be added to the same "LS" (Lip Synchronization) group when creating an offer or answer. Adding tracks to the same "LS" group indicates that the playback of these tracks should be synchronized for proper lip sync, as described in [RFC5888], Section 7. addTrack attempts to minimize the number of transceivers as follows: if the PeerConnection is in the "have-remote-offer" state, the track will be attached to the first compatible transceiver that was created by the most recent call to setRemoteDescription and does not have a local track. Otherwise, a new transceiver will be created, as described in Section 4.1.4. 🎬
4.1.3. removeTrack 🧹
The removeTrack method removes a MediaStreamTrack from the PeerConnection, using the RtpSender argument to indicate which sender should have its track removed. The sender's track is cleared, and the sender stops sending. Future calls to createOffer will mark the "m=" section associated with the sender as recvonly (if transceiver.direction is sendrecv) or as inactive (if transceiver.direction is sendonly). 👋
4.1.4. addTransceiver 🦩
The addTransceiver method adds a new RtpTransceiver to the PeerConnection. If a MediaStreamTrack argument is provided, then the transceiver will be configured with that media type and the track will be attached to the transceiver. Otherwise, the application MUST explicitly specify the type; this mode is useful for creating recvonly transceivers as well as for creating transceivers to which a track can be attached at some later point. 🎯
At the time of creation, the application can also specify a transceiver direction attribute, a set of MediaStreams that the transceiver is associated with (allowing "LS" group assignments), and a set of encodings for the media (used for simulcast as described in Section 3.7). 🌈
4.1.5. onaddtrack Event 🔔
The onaddtrack event is dispatched to the application when a new remote track has been signaled as a result of a setRemoteDescription call. The new track is supplied as a MediaStreamTrack object in the event, along with the MediaStream(s) the track is part of. 🎉
4.1.6. createDataChannel 📬
The createDataChannel method creates a new data channel and attaches it to the PeerConnection. If no data channel currently exists for this PeerConnection, then a new offer/answer exchange is required. All data channels on a given PeerConnection share the same SCTP/DTLS association ("SCTP" stands for "Stream Control Transmission Protocol") and therefore the same "m=" section, so subsequent creation of data channels does not have any impact on the JSEP state. 🧁
The createDataChannel method also includes a number of arguments that are used by the PeerConnection (e.g., maxPacketLifetime) but are not reflected in the SDP and do not affect the JSEP state. 💫
4.1.7. ondatachannel Event 🥳
The ondatachannel event is dispatched to the application when a new data channel has been negotiated by the remote side, which can occur at any time after the underlying SCTP/DTLS association has been established. The new data channel object is supplied in the event. 🌷
4.1.8. createOffer ✨
okay so createOffer is the method that spits out a blob of SDP holding an offer per [RFC3264] 💌 It packs in everything this session supports: the media added to this PeerConnection, the codec, RTP, and RTCP options this implementation can handle, and any candidates the ICE agent has gathered. It also takes an options parameter for extra control, and that's how an app can trigger an ICE restart to get connectivity back 🔁
In the initial offer, the SDP has all the functionality you actually want for the session (stuff that's supported but not wanted by default can get left out 🙈). For each SDP line, the generation follows whatever the spec defining that line says about generating an initial offer. The full details are in Section 5.2.1 below.
If createOffer runs after the session is already established, it makes an offer that modifies the current session based on what changed, like adding or stopping RtpTransceivers or asking for an ICE restart 🦋 For each existing stream, every SDP line MUST be generated using the process the RFC that specifies that line defines for an updated offer. For each new stream, the SDP MUST follow the initial offer process mentioned above. If nothing changed, or the lines aren't touched by the changes, the offer only includes the parameters negotiated in the last offer/answer exchange. The full details on subsequent offers are in Section 5.2.2 below.
Session descriptions from createOffer MUST be immediately usable by setLocalDescription. If a system has limited resources (like a finite number of decoders), createOffer SHOULD return an offer that reflects the current state of the system, so setLocalDescription succeeds when it goes to grab those resources 💅
Calling this can do things like generate new ICE credentials, but it doesn't change the PeerConnection state, kick off candidate gathering, or make media start or stop flowing. Specifically, the offer isn't applied, and it doesn't become the pending local description, until setLocalDescription is called 🫶
4.1.9. createAnswer 💜
createAnswer generates a blob of SDP with an SDP answer per [RFC3264], using the supported configuration that matches the parameters from the most recent call to setRemoteDescription. setRemoteDescription MUST have been called before createAnswer, no exceptions, bestie 🙅 Like createOffer, the blob includes the media on this PeerConnection, the codec/RTP/RTCP options negotiated for the session, and any candidates the ICE agent gathered. An options parameter can be passed in too, for extra control over the generated answer.
As an answer, the SDP has one specific configuration that sets up how the media plane gets established. For each SDP line, the generation MUST follow the process the spec defining that line gives for generating an answer. The full answer handling is in Section 5.3 below.
Session descriptions from createAnswer MUST be immediately usable by setLocalDescription. Like createOffer, the returned description SHOULD reflect the current state of the system ✨
Calling this can generate new ICE credentials, but it doesn't change the PeerConnection state, trigger candidate gathering, or cause a media state change. Specifically, the answer isn't applied and doesn't become the current local description until setLocalDescription is called.
4.1.10. SessionDescriptionType 🎀
Session description objects (RTCSessionDescription) can be of type "offer", "pranswer", "answer", or "rollback". These types tell you how the description parameter should be parsed and how the media state should change. Very important stuff, periodt 💖
"offer" means the description MUST be parsed as an offer, and it can include lots of possible media configurations. A description used as an "offer" may be applied any time the PeerConnection is in a "stable" state, or applied as an update to a previously supplied but unanswered "offer".
"pranswer" means the description MUST be parsed as an answer, but not a final one, so it MUST NOT lead to freeing up allocated resources. It may kick off media transmission if the answer doesn't specify an inactive media direction. A description used as a "pranswer" may be applied as a response to an "offer" or as an update to a previously sent "pranswer".
"answer" means the description MUST be parsed as an answer, the offer/answer exchange MUST be considered complete, and any resources (decoders, candidates) that are no longer needed SHOULD be released. A description used as an "answer" may be applied as a response to an "offer" or as an update to a previously sent "pranswer" 🔥
The only real difference between a provisional and a final answer is that the final one frees any unused resources that were allocated because of the offer. So the app gets to use some discretion on whether an answer is applied as provisional or final, and can change the session description type as needed. For example, in a serial forking scenario, an app might get multiple "final" answers, one from each remote endpoint. It could accept the early ones as provisional answers, and only apply an answer as final once it gets one that meets its criteria (like a live user instead of voicemail) 📞
"rollback" is a special session description type that means the state machine MUST be rolled back to the previous "stable" state, as described in Section 4.1.10.2. The contents MUST be empty.
4.1.10.1. Use of Provisional Answers 🌸
Most apps won't need to create answers with the "pranswer" type. Sending an immediate response to an offer is good practice, since it warms up the session transport and prevents media clipping. But the preferred JSEP handling is to create and send a "sendonly" final answer with a null MediaStreamTrack right after receiving the offer. That stops the caller from sending media, while letting media go out immediately once the callee answers. Later, when the callee actually accepts the call, the app plugs in the real MediaStreamTrack and creates a new "sendrecv" offer to update the previous offer/answer pair and start bidirectional media flow. This could also be done with a "sendonly" pranswer followed by a "sendrecv" answer, but the initial pranswer leaves the offer/answer exchange open, which means the caller can't send an updated offer during that time 🙈
Let's make it concrete. Picture a typical JSEP app that wants audio and video set up as fast as possible. When the callee gets an offer with audio and video MediaStreamTracks, it sends an immediate answer accepting those tracks as "sendonly". So the caller won't send the callee any media yet, and because the callee hasn't added its own MediaStreamTracks, the callee won't send any media either. Then it asks the user to accept the call and grabs the local tracks it needs. Once the user accepts, the app plugs in the tracks it acquired. Since ICE and DTLS handshaking have likely finished by now, those can start transmitting right away. The app also sends a new offer to the remote side saying the call was accepted and switching audio and video to two-way media. A detailed example flow along these lines is in Section 7.3 🎬
Of course, some apps can't do this double offer/answer exchange, especially ones trying to gateway to legacy signaling protocols. In those cases, pranswer can still give the app a way to warm up the transport.
4.1.10.2. Rollback ⏪
Sometimes you want to "undo" a change made to setLocalDescription or setRemoteDescription. Say a call is ongoing and one side wants to change some session parameters. That side generates an updated offer and calls setLocalDescription. But the remote side, either before or after setRemoteDescription, decides it doesn't want the new parameters and sends a reject message back to the offerer. Now the offerer, and possibly the answerer too, needs to return to a "stable" state and the previous local/remote description. To handle this, we introduce the concept of "rollback," which throws out any proposed changes to the session and returns the state machine to the "stable" state. A rollback is performed by supplying a session description of type "rollback" with empty contents to either setLocalDescription or setRemoteDescription 🧚
4.1.11. setLocalDescription 💅
The setLocalDescription method tells the PeerConnection to apply the supplied session description as its local configuration. The type field says whether the description should be processed as an offer, provisional answer, final answer, or rollback. Offers and answers are checked differently, using the various rules that exist for each SDP line.
This API changes the local media state. Among other things, it sets up local resources for receiving and decoding media. To handle scenarios where the app wants to offer a change from one media format to a different, incompatible format, the PeerConnection MUST be able to simultaneously support use of both the current and pending local descriptions (for example, support the codecs that exist in either description). This dual processing begins when the PeerConnection enters the "have-local-offer" state, and it keeps going until setRemoteDescription is called with either (1) a final answer, at which point the PeerConnection can fully adopt the pending local description, or (2) a rollback, which reverts to the current local description 🔄
This API indirectly controls the candidate gathering process. When a local description is supplied and the number of transports currently in use doesn't match the number of transports the local description needs, the PeerConnection will create transports as needed and begin gathering candidates for each transport, using ones from the candidate pool if available 🌈
If (1) setRemoteDescription was previously called with an offer, (2) setLocalDescription is called with an answer (provisional or final), (3) the media directions are compatible, and (4) media is available to send, this will result in the starting of media transmission. Main character energy 👑
4.1.12. setRemoteDescription 📥✨
okay so setRemoteDescription is how you tell the PeerConnection "hey, apply this as the remote config, pls" 💖. It works just like setLocalDescription: the type field on the description tells it how to process the thing 🎀.
This one changes your local media state, bestie 🎬. Among other things, it sets up local resources for sending and encoding media, so it's giving real setup energy 💅.
If (1) setLocalDescription was previously called with an offer, (2) setRemoteDescription is called with an answer (provisional or final), (3) the media directions are compatible, and (4) media is available to send, then media transmission starts 🚀🌈. Main character moment!
4.1.13. currentLocalDescription 🗂️
The currentLocalDescription method gives you the current negotiated local description -- aka the local description from the last successful offer/answer exchange -- plus any local candidates the ICE agent has generated since the local description was set 🧚.
If an offer/answer exchange hasn't been completed yet, you get a null object back 🙈.
4.1.14. pendingLocalDescription ⏳
The pendingLocalDescription method returns a copy of the local description that's currently in negotiation -- so a local offer that's been set but doesn't have a matching remote answer yet 💜. It also includes any local candidates the ICE agent has generated since the local description was set ✨.
If the PeerConnection state is "stable" or "have-remote-offer", you'll get a null object 🫶.
4.1.15. currentRemoteDescription 📬
The currentRemoteDescription method returns a copy of the current negotiated remote description -- the remote description from the last successful offer/answer exchange 🦋. It also includes any remote candidates supplied through processIceMessage since the remote description was set 💅.
No completed offer/answer exchange yet? Then it's a null object 🙈.
4.1.16. pendingRemoteDescription 🕰️
The pendingRemoteDescription method returns a copy of the remote description that's still in negotiation -- a remote offer that's been set without a corresponding local answer 🌸. It also includes any remote candidates supplied through processIceMessage since the remote description was set 🎀.
If the PeerConnection state is "stable" or "have-local-offer", it's a null object 🙈.
4.1.17. canTrickleIceCandidates 🌊
The canTrickleIceCandidates property tells you whether the remote side supports receiving trickled candidates 🌊. There are three possible values, and they're giving distinct vibes:
null: No SDP has been received from the other side, so it's not known whether it can handle trickle. This is the initial value beforesetRemoteDescriptionis called.true: SDP has been received from the other side indicating that it can support trickle ✅.false: SDP has been received from the other side indicating that it cannot support trickle ❌.
As Section 3.5.2 explains, JSEP implementations always hand candidates to the application individually, which is what Trickle ICE needs 🧁. But the application can check canTrickleIceCandidates to see whether its peer can actually do Trickle ICE. That's the real question, bestie: is it safe to send an initial offer or answer, and then send candidates later as they're gathered? 🤔
Only "true" definitively indicates remote Trickle ICE support. So an application that compares canTrickleIceCandidates against "true" will, by default, try Half Trickle on initial offers, and Full Trickle on later exchanges with a Trickle ICE-compatible agent 🍓✨.
4.1.18. setConfiguration ⚙️
The setConfiguration method lets you change the global configuration of the PeerConnection, which was first set by constructor parameters, during the session 🎛️. What happens depends on when you call it, and it can differ depending on which parameters you change:
- Changes to the STUN/TURN servers to use affect the next gathering phase. If an ICE gathering phase has already started or completed, the
'needs-ice-restart'bit mentioned in Section 3.5.1 gets set. That makes the next call tocreateOffergenerate new ICE credentials, which forces an ICE restart and kicks off a new gathering phase where the new servers get used 🔁. If the ICE candidate pool has a nonzero size and a local description hasn't been applied yet, existing candidates get discarded, and new candidates get gathered from the new servers 🧹. - Any change to the ICE candidate policy affects the next gathering phase. If an ICE gathering phase has already started or completed, the
'needs-ice-restart'bit gets set. Either way, policy changes have no effect on the candidate pool. Pooled candidates aren't made available to the application until a gathering phase happens, so any necessary filtering can still be done on them 🪄. - The ICE candidate pool size MUST NOT be changed after applying a local description. If a local description hasn't been applied yet, any changes to the pool size take effect immediately. If it's increased, additional candidates get pre-gathered; if it's decreased, the now-superfluous candidates get discarded 🫧.
- The bundle and RTCP-multiplexing policies MUST NOT be changed after the construction of the PeerConnection 🚫.
Calling this method might change the state of the ICE agent, so keep an eye on that 👀.
4.1.19. addIceCandidate 🧊
The addIceCandidate method gives the ICE agent an update through an IceCandidate object (Section 3.5.2.1) 💌. If the IceCandidate's candidate field is non-null, the IceCandidate is treated as a brand new remote ICE candidate. It gets added to the current and/or pending remote description, following the rules for Trickle ICE 🌟. Otherwise, the IceCandidate is treated as an end-of-candidates indication, as defined in [RFC8838], Section 14 🏁.
Either way, the "m=" section index, MID, and ufrag fields from the IceCandidate are used to figure out which "m=" section and which ICE candidate generation the IceCandidate belongs to, as described in Section 3.5.2.1 above 🔍. For an end-of-candidates indication, null values in the "m=" section index and MID fields mean the indication applies to all "m=" sections in that ICE candidate generation 🫶. But if both fields are null for a new remote candidate, this MUST be treated as an invalid condition, as specified below 🙅♀️.
If any IceCandidate fields have invalid values, or an error happens while processing the IceCandidate object, the supplied IceCandidate MUST be ignored and an error MUST be returned. No second chances, babe 💀.
Otherwise, the new remote candidate or end-of-candidates indication gets supplied to the ICE agent 🚀. For a new remote candidate, connectivity checks will be sent to it, assuming setLocalDescription has already been called to initialize the ICE gathering process 🧚♀️.
4.1.20. onicecandidate Event 🎉
The onicecandidate event gets dispatched to the application in two situations 📣. One: when the ICE agent has discovered a new allowed local ICE candidate during ICE gathering, as outlined in Section 3.5.1 and subject to the restrictions in Section 3.5.3. Two: when an ICE gathering phase completes. The event contains a single IceCandidate object, as defined in Section 3.5.2.1 💖.
In the first case, the newly discovered candidate is reflected in the IceCandidate object, and all of its fields MUST be non-null. That candidate also gets added to the current and/or pending local description, following the Trickle ICE rules ✨.
In the second case, the event's IceCandidate object MUST have its candidate field set to null. That signals the current gathering phase is complete, meaning no more onicecandidate events are coming in this phase 🏳️. Still, the IceCandidate's ufrag field MUST be specified, so everyone knows which ICE candidate generation is ending 🧾. The IceCandidate's "m=" section index and MID fields MAY be specified, to say the event applies to one particular "m=" section. Or they can be set to null, meaning it applies to all "m=" sections in the current ICE candidate generation 🌈. The application can use this event to generate an end-of-candidates indication, as defined in [RFC8838], Section 13 🦋.
4.2. RtpTransceiver 🦋💫
4.2.1. stop 🛑
The stop method stops an RtpTransceiver 🙅♀️. After that, future calls to createOffer generate a zero port for the associated "m=" section. There's more detail below, so keep reading 👀.
4.2.2. stopped 🧊
The stopped property tells you whether the transceiver has been stopped. That can happen because you called stop, or because you applied an answer that rejects the associated "m=" section 🚫. In either case it's set to "true". Otherwise it's "false" 💜.
A stopped RtpTransceiver doesn't send any outgoing RTP or RTCP, and it doesn't process any incoming RTP or RTCP. And it can't be restarted, so that's the end of that era 💀.
4.2.3. setDirection 🧭
The setDirection method sets the direction of a transceiver 🧭. That affects the direction property of the associated "m=" section on future calls to createOffer and createAnswer. The permitted values are "recvonly", "sendrecv", "sendonly", and "inactive", mirroring the identically named direction attributes defined in [RFC4566], Section 6 🎀.
When creating offers, the transceiver direction is directly reflected in the output, even for re-offers 🔁. When creating answers, the transceiver direction gets intersected with the offered direction, as explained in Section 5.3 below 🤝.
Here's the tea: while setDirection sets the direction property of the transceiver immediately (Section 4.2.4), that property doesn't instantly change whether the transceiver's RtpSender sends or its RtpReceiver receives 🫣. The direction that's actually in effect is shown by the currentDirection property, which only updates when an answer is applied ⏰.
4.2.4. direction 🎯
The direction property shows the last value passed into setDirection 💅. If setDirection has never been called, it's set to whatever direction the transceiver was initialized with 🌸.
4.2.5. currentDirection ✅
The currentDirection property shows the last negotiated direction for the transceiver's associated "m=" section 🤝. More specifically, it shows the direction attribute [RFC3264] of the associated "m=" section in the last applied answer, which includes provisional answers 📜. The "send" and "recv" directions get reversed if it was a remote answer 🔄. For example, if the direction attribute for the associated "m=" section in a remote answer is "recvonly", then currentDirection is set to "sendonly" 🙃.
If an answer that references this transceiver hasn't been applied yet, or if the transceiver is stopped, currentDirection is set to "null" 🙈.
4.2.6. setCodecPreferences 🎀
okay bestie, so setCodecPreferences sets the codec preferences on a transceiver 💅. Those prefs then decide which codecs show up in the associated "m=" section, and in what order, the next time createOffer or createAnswer runs ✨. BUT here's the tea 🙈: setCodecPreferences does NOT directly control which codec the implementation actually sends. It only changes which codecs the implementation says it prefers to receive, through the offer or answer. And even if a codec got excluded by setCodecPreferences, it can still be used to send until the next offer/answer exchange throws it out 💀.
The codec preferences of an RtpTransceiver can get codecs excluded from later createOffer and createAnswer calls. When that happens, the matching media formats in the associated "m=" section get excluded too 🧹. The prefs can't add media formats that wouldn't otherwise be there, so they're strictly a filter, no glow-ups allowed 🚫.
The codec preferences of an RtpTransceiver can also set the order of codecs in later createOffer and createAnswer calls. In that case, the media formats in the associated "m=" section follow whatever order was specified 📋✨.
5. SDP Interaction Procedures 📜
okay this section is the step-by-step manual for how to make and parse SDP objects 🧁. Read it like a recipe, bestie.
5.1. Requirements Overview 🌸
JSEP implementations MUST comply with the specs listed below, since those are the ones that govern how offers and answers get created and processed 🫶.
5.1.1. Usage Requirements 🔥
Every session description that a JSEP implementation handles, whether it's local or remote, MUST indicate support for the following specs 👑. If any of them are missing, that omission MUST be treated as an error 😭.
- ICE, as specified in [RFC8445], MUST be used. Note that the remote endpoint may use a lite implementation, so implementations MUST properly handle remote endpoints that use ICE-lite 🦋. The remote endpoint may also be running an older version of ICE, so implementations MUST properly handle remote endpoints that use ICE as specified in [RFC5245] 🕰️.
- DTLS [RFC6347] or DTLS-SRTP [RFC5763] MUST be used, depending on what fits the media type, as specified in [RFC8827] 🔐.
The SDP security descriptions mechanism for SRTP keying [RFC4568] MUST NOT be used, as discussed in [RFC8827] 🙅♀️.
5.1.2. Profile Names and Interoperability 🎭
For media "m=" sections, JSEP implementations MUST support the "UDP/TLS/RTP/SAVPF" profile specified in [RFC5764] AND the "TCP/DTLS/RTP/SAVPF" profile specified in [RFC7850]. They MUST indicate one of these profiles on every media "m=" line they produce in an offer 🎬. For data "m=" sections, implementations MUST support the "UDP/DTLS/SCTP" profile AND the "TCP/DTLS/SCTP" profile, and MUST indicate one of these profiles on every data "m=" line they produce in an offer 💬. The exact profile to use gets decided by the protocol attached to the current default or selected ICE candidate, as described in [RFC8839], Section 4.2.1.2 🧭.
Unfortunately, some endpoints generate other profile strings in an attempt at compatibility, even when they actually mean to support one of these profiles 😩. For example, an endpoint might generate "RTP/AVP" but also supply a=fingerprint and a=rtcp-fb attributes, which signals that it's willing to support "UDP/TLS/RTP/SAVPF" or "TCP/DTLS/RTP/SAVPF" 🤝. To keep things smooth with those endpoints, JSEP implementations MUST follow these rules when processing the media "m=" sections in a received offer 🧠:
- Any profile in the offer that matches one of the following MUST be accepted:
"RTP/AVP"(defined in [RFC4566], Section 8.2.2)"RTP/AVPF"(defined in [RFC4585], Section 9)"RTP/SAVP"(defined in [RFC3711], Section 12)"RTP/SAVPF"(defined in [RFC5124], Section 6)"TCP/DTLS/RTP/SAVP"(defined in [RFC7850], Section 3.4)"TCP/DTLS/RTP/SAVPF"(defined in [RFC7850], Section 3.5)"UDP/TLS/RTP/SAVP"(defined in [RFC5764], Section 9)"UDP/TLS/RTP/SAVPF"(defined in [RFC5764], Section 9)
- The profile on any "m=" line in any generated answer MUST exactly match the profile provided in the offer 🪞.
- Because DTLS-SRTP is REQUIRED, choosing SAVP or AVP makes no difference. Support for DTLS-SRTP is determined by whether one or more
a=fingerprintattributes are present. Note that a missinga=fingerprintattribute will lead to negotiation failure 💔. - Using AVPF or AVP only controls the timing rules for RTCP feedback. If AVPF is provided, or an
a=rtcp-fbattribute is present, assume AVPF timing, which means a default value oftrr-int=0. Otherwise, assume AVPF is being used in an AVP-compatible mode and use a value oftrr-int=4000⏱️. - For data "m=" sections, implementations MUST support receiving the
"UDP/DTLS/SCTP","TCP/DTLS/SCTP", or"DTLS/SCTP"(for backwards compatibility) profiles 🕹️.
Note that re-offers from JSEP implementations MUST use the correct profile strings, even if the initial offer/answer exchange used an (incorrect) older profile string 🔁. This keeps JSEP behavior simple with minimal downside, since any remote endpoint that can't handle such a re-offer also can't handle a JSEP endpoint's initial offer 🙌.
5.2. Constructing an Offer 🍓
When createOffer is called, a new SDP description MUST be created that includes the functionality specified in [RFC8834] ✨. The exact details of how that happens are explained below 📝.
5.2.1. Initial Offers 🐣
When createOffer is called for the first time, the result is known as the initial offer 🥇.
The first step in generating an initial offer is generating the session-level attributes, as specified in [RFC4566], Section 5. Specifically 🧩:
- The first SDP line MUST be
v=0, as defined in [RFC4566], Section 5.1 📍. - The second SDP line MUST be an
o=line, as defined in [RFC4566], Section 5.2 🎯. The value of the<username>field SHOULD be-. The<sess-id>MUST be representable by a 64-bit signed integer, and its value MUST be less than2^(63)-1. It is RECOMMENDED that the<sess-id>be built by generating a 64-bit quantity with the highest bit set to zero and the remaining 63 bits cryptographically random 🎲. The value of the<nettype> <addrtype> <unicast-address>tuple SHOULD be set to a non-meaningful address, such asIN IP4 0.0.0.0, so a local IP address doesn't leak through this field 🙈. This problem is discussed in [RFC8828]. As mentioned in [RFC4566], the entireo=line needs to be unique, but picking a random number for<sess-id>is enough to accomplish that 🔑. - The third SDP line MUST be an
s=line, as defined in [RFC4566], Section 5.3. To match theo=line, a single dash SHOULD be used as the session name, e.g.,s=-. Note that this differs from the advice in [RFC4566], which suggests a single space. But since botho=ands=are meaningless in JSEP, having the same meaningless value seems clearer 🫧. - Session Information (
i=), URI (u=), Email Address (e=), Phone Number (p=), Repeat Times (r=), and Time Zones (z=) lines aren't useful in this context and SHOULD NOT be included 🚫. - Encryption Keys (
k=) lines don't provide sufficient security and MUST NOT be included 🔒. - A
t=line MUST be added, as specified in [RFC4566], Section 5.9. Both<start-time>and<stop-time>SHOULD be set to zero, e.g.,t=0 0⏰. - An
a=ice-optionsline with thetrickleandice2options MUST be added, as specified in [RFC8840], Section 4.1.1 and [RFC8445], Section 10 🌊. - If WebRTC identity is being used, an
a=identityline MUST be added, as described in [RFC8827], Section 5 🪪.
The next step is generating "m=" sections, as specified in [RFC4566], Section 5.14 🎀. An "m=" section gets generated for each RtpTransceiver that has been added to the PeerConnection, excluding any stopped RtpTransceivers. This happens in the order the RtpTransceivers were added to the PeerConnection 📚. If there are no such RtpTransceivers, no "m=" sections are generated, but more can be added later, as discussed in [RFC3264], Section 5 🛠️.
For each "m=" section generated for an RtpTransceiver, establish a mapping between the transceiver and the index of that generated "m=" section 🗺️.
Each "m=" section, as long as it isn't marked as bundle-only, MUST contain a unique set of ICE credentials and a unique set of ICE candidates 🧑🤝🧑. Bundle-only "m=" sections MUST NOT contain any ICE credentials, and MUST NOT gather any candidates 🙅♀️.
For DTLS, all "m=" sections MUST use any and all certificates that have been specified for the PeerConnection. As a result, they MUST all have the same fingerprint value or values [RFC8122], or these values MUST be session-level attributes 🪪✨.
Each "m=" section MUST be generated as specified in [RFC4566], Section 5.14. For the "m=" line itself, the following rules MUST be followed 📏:
- If the "m=" section is marked as bundle-only, the
<port>value MUST be set to zero. Otherwise, the<port>value is set to the port of the default ICE candidate for this "m=" section. But since no candidates are available yet, the default port value of 9 (Discard) MUST be used, as indicated in [RFC8840], Section 4.1.1 🔌. - To properly indicate the use of DTLS, the
<proto>field MUST be set to"UDP/TLS/RTP/SAVPF", as specified in [RFC5764], Section 8 🔐. - If codec preferences have been set for the associated transceiver, media formats MUST be generated in the corresponding order, and MUST exclude any codecs that aren't present in the codec preferences 🎶.
- Unless they're excluded by the restrictions above, the media formats MUST include the mandatory audio/video codecs as specified in [RFC7874], Section 3 and [RFC7742], Section 5 🎥.
The "m=" line MUST be followed immediately by a "c=" line, as specified in [RFC4566], Section 5.7. Again, since no candidates are available yet, the "c=" line MUST contain the default value IN IP4 0.0.0.0, as defined in [RFC8840], Section 4.1.1 🧸.
[RFC8859] groups SDP attributes into different categories. To avoid unnecessary duplication when bundling, attributes of category IDENTICAL or TRANSPORT MUST NOT be repeated in bundled "m=" sections. This repeats the guidance from [RFC8843], Section 7.1.3. It covers "m=" sections where bundling has been negotiated and is still desired, as well as "m=" sections marked as bundle-only 📦.
The following attributes, which are of a category other than IDENTICAL or TRANSPORT, MUST be included in each "m=" section 🌈:
- An
a=midline, as specified in [RFC5888], Section 4. All MID values MUST be generated in a way that doesn't leak user information, e.g., randomly or with a per-PeerConnection counter. They SHOULD be 3 bytes or less, so they can fit efficiently into the RTP header extension defined in [RFC8843], Section 15.2 🏷️. Note that this does NOT set the RtpTransceiver mid property, since that only happens when the description is applied. The generated MID value can be considered a "proposed" MID at this point 🫶. - A direction attribute that is the same as the direction of the associated transceiver ➡️.
- For each media format on the "m=" line,
a=rtpmapanda=fmtplines, as specified in [RFC4566], Section 6 and [RFC3264], Section 5.1 🎞️. - okay bestie, for every primary codec that should be using RTP retransmission 🔁, you add a matching
a=rtpmapline that saysrtxwith the primary codec's clock rate, plus ana=fmtpline that references the primary codec's payload type, as specified in [RFC4588], Section 8.1 ✨ - for each supported Forward Error Correction (FEC) mechanism 🛡️, you add
a=rtpmapanda=fmtplines, as specified in [RFC4566], Section 6. The FEC mechanisms that MUST be supported are specified in [RFC8854], Section 7, and the specific usage for each media type is outlined in Sections 4 and 5 💅 - if this
m=section is for media with configurable durations per packet (think audio 🎧), add ana=maxptimeline. It shows the maximum amount of media, specified in milliseconds, that can be packed into each packet, as specified in [RFC4566], Section 6. This value is the smallest of the maximum duration values across all the codecs in them=section, so nobody gets overstuffed 🫶 - if this
m=section is for video and there are known limits on the size of images that can be decoded, add ana=imageattrline, as specified in Section 3.6 🖼️ - for each supported RTP header extension, add an
a=extmapline, as specified in [RFC5285], Section 5. The list of header extensions that SHOULD/MUST be supported is specified in [RFC8834], Section 5.2. Any header extensions that require encryption MUST be specified as indicated in [RFC6904], Section 4 🔐 - for each supported RTCP feedback mechanism, add an
a=rtcp-fbline, as specified in [RFC4585], Section 4.2. The list of RTCP feedback mechanisms that SHOULD/MUST be supported is specified in [RFC8834], Section 5.1 📣 - if the RtpTransceiver has a
sendrecvorsendonlydirection:- for each MediaStream that was associated with the transceiver when it was created via addTrack or addTransceiver, add an
a=msidline, as specified in [RFC8830], Section 2, but leave off theappdatafield 🎀
- for each MediaStream that was associated with the transceiver when it was created via addTrack or addTransceiver, add an
- if the RtpTransceiver has a
sendrecvorsendonlydirection, and the application has specified a rid-id for an encoding, or has specified more than one encoding in the RtpSenders's parameters, add ana=ridline for each encoding specified. Thea=ridline is specified in [RFC8851], and its direction MUST be"send". If the application has chosen a rid-id, it MUST be used; otherwise, a rid-id MUST be generated by the implementation. rid-ids MUST be generated in a way that doesn't leak user information, e.g., randomly or with a per-PeerConnection counter (see the guidance at the end of [RFC8852], Section 3.3). They SHOULD be 3 bytes or less, so they fit efficiently into the RTP header extensions defined in [RFC8852], Section 3.3 🌸. If no encodings have been specified, or only one encoding is specified but without a rid-id, then noa=ridlines get generated 🙈 - if the RtpTransceiver has a
sendrecvorsendonlydirection and more than onea=ridline has been generated, add ana=simulcastline, with direction"send", as defined in [RFC8853], Section 5.1. The associated set of rid-ids MUST include all of the rid-ids used in thea=ridlines for thism=section 👑 - if (1) the bundle policy for this PeerConnection is set to
"max-bundle"and this is not the firstm=section, or (2) the bundle policy is set to"balanced"and this is not the firstm=section for this media type, add ana=bundle-onlyline 🧚
The following attributes, which are of category IDENTICAL or TRANSPORT, MUST appear only in m= sections that either have a unique address or are associated with the BUNDLE-tag. (In initial offers, this means the m= sections that do not contain an a=bundle-only attribute.) This is the main-character rule for these attributes, so don't skip it 💜
a=ice-ufraganda=ice-pwdlines, as specified in [RFC8839], Section 5.4 🧊- for each desired digest algorithm, one or more
a=fingerprintlines for each of the endpoint's certificates, as specified in [RFC8122], Section 5 🔏 - an
a=setupline, as specified in [RFC4145], Section 4 and clarified for use in DTLS-SRTP scenarios in [RFC5763], Section 5. The role value in the offer MUST be"actpass"🎭 - an
a=tls-idline, as specified in [RFC8842], Section 5.2 🪪 - an
a=rtcpline, as specified in [RFC3605], Section 2.1, containing the default value9 IN IP4 0.0.0.0, because no candidates have been gathered yet 🫥 - an
a=rtcp-muxline, as specified in [RFC5761], Section 5.1.3 🍓 - if the RTP/RTCP multiplexing policy is
"require", ana=rtcp-mux-onlyline, as specified in [RFC8858], Section 4 🔒 - an
a=rtcp-rsizeline, as specified in [RFC5506], Section 5 🌈
Lastly, if a data channel has been created, an m= section MUST be generated for data. The <media> field MUST be set to "application", and the <proto> field MUST be set to "UDP/DTLS/SCTP" [RFC8841]. The <fmt> value MUST be set to "webrtc-datachannel", as specified in [RFC8841], Section 4.2.2. Data channels, the group chat of the Internet 💬✨
Within the data m= section, an a=mid line MUST be generated and included as described above. Also add an a=sctp-port line referencing the SCTP port number, as defined in [RFC8841], Section 5.1, and, if appropriate, an a=max-message-size line, as defined in [RFC8841], Section 6.1 🦋
As discussed above, the following attributes of category IDENTICAL or TRANSPORT are included only if the data m= section either has a unique address or is associated with the BUNDLE-tag (for example, if it's the only m= section):
a=ice-ufraga=ice-pwda=fingerprinta=setupa=tls-id
Once all m= sections have been generated, a session-level a=group attribute MUST be added as specified in [RFC5888]. This attribute MUST have semantics "BUNDLE" and MUST include the mid identifiers of each m= section. The effect is that the JSEP implementation offers all m= sections as one bundle group 🎀. Whether the m= sections are bundle-only or not depends on the bundle policy, so keep an eye on that one 👀
The next step is to generate session-level lip sync groups as defined in [RFC5888], Section 7. For each MediaStream referenced by more than one RtpTransceiver (by passing those MediaStreams as arguments to the addTrack and addTransceiver methods), a group of type "LS" MUST be added that contains the MID values for each RtpTransceiver 🎬💖
Attributes that SDP allows at either the session level or the media level SHOULD generally go at the media level, even if they're identical. This helps development and debugging, because it's way easier to understand individual media sections, especially if one of a set of initially identical attributes gets changed later 🧁. However, implementations MAY choose to aggregate attributes at the session level, and JSEP implementations MUST be prepared to receive attributes in either location ✨
Attributes other than the ones specified above MAY be included, except for the following attributes, which are specifically incompatible with the requirements of [RFC8834] and MUST NOT be included:
a=cryptoa=key-mgmta=ice-lite
Note that when bundle is used, any additional attributes that are added MUST follow the advice in [RFC8859] on how those attributes interact with bundle. Don't freestyle this one, bestie 🚨
Note that these requirements are in some cases stricter than those of SDP. Implementations MUST be prepared to accept compliant SDP even if it wouldn't conform to the requirements for generating SDP in this specification. Being generous on the receiving end is so iconic 🫶🌷
5.2.2. Subsequent Offers ✨
okay so bestie, this is where things get a little more involved 💅 createOffer can get called a second time (or later), or after a local description is already installed, and the processing is kinda different from the initial offer 🦋
If the previous offer was not applied using setLocalDescription, meaning the PeerConnection is still in the "stable" state, the steps for generating an initial offer MUST be followed, subject to the following restriction:
- The fields of the
"o="line MUST stay the same except for the<session-version>field, which MUST increment by one on each call to createOffer if the offer might differ from the output of the previous call to createOffer; implementations MAY opt to increment<session-version>on every call. The value of the generated<session-version>is independent of the<session-version>of the current local description; in particular, in the case where the current version is N, an offer is created and applied with version N+1, and then that offer is rolled back so that the current version is again N, the next generated offer will still have version N+2. 🌸
Note that if the application creates an offer by reading currentLocalDescription instead of calling createOffer, the returned SDP may be different than when setLocalDescription was originally called, due to the addition of gathered ICE candidates, but the <session-version> will not have changed. There are no known scenarios in which this causes problems, but if this is a concern, the solution is simply to use createOffer to ensure a unique <session-version>. 🫶
If the previous offer was applied using setLocalDescription, but a corresponding answer from the remote side has not yet been applied, meaning the PeerConnection is still in the "have-local-offer" state, an offer is generated by following the steps in the "stable" state above, along with these exceptions:
- The
"s="and"t="lines MUST stay the same. - If any RtpTransceiver has been added and there exists an
"m="section with a zero port in the current local description or the current remote description, that"m="section MUST be recycled by generating an"m="section for the added RtpTransceiver as if the"m="section were being added to the session description (including a new MID value) and placing it at the same index as the"m="section with a zero port. 🎀 - If an RtpTransceiver is stopped and is not associated with an
"m="section, an"m="section MUST NOT be generated for it. This prevents adding back RtpTransceivers whose"m="sections were recycled and used for a new RtpTransceiver in a previous offer/answer exchange, as described above. 🙈 - If an RtpTransceiver has been stopped and is associated with an
"m="section, and the"m="section is not being recycled as described above, an"m="section MUST be generated for it with the port set to zero and all"a=msid"lines removed. - For RtpTransceivers that are not stopped, the
"a=msid"line or lines MUST stay the same if they are present in the current description, regardless of changes to the transceiver's direction or track. If no"a=msid"line is present in the current description,"a=msid"line(s) MUST be generated according to the same rules as for an initial offer. 🍓 - Each
"m="and"c="line MUST be filled in with the port, relevant RTP profile, and address of the default candidate for the"m="section, as described in [RFC8839], Section 4.2.1.2 and clarified in Section 5.1.2. If no RTP candidates have yet been gathered, default values MUST still be used, as described above. 🧚 - Each
"a=mid"line MUST stay the same. - Each
"a=ice-ufrag"and"a=ice-pwd"line MUST stay the same, unless the ICE configuration has changed (e.g., changes to either the supported STUN/TURN servers or the ICE candidate policy) or the IceRestart option (Section 5.2.3.1) was specified. If the"m="section is bundled into another"m="section, it still MUST NOT contain any ICE credentials. 💜 - If the
"m="section is not bundled into another"m="section, its"a=rtcp"attribute line MUST be filled in with the port and address of the default RTCP candidate, as indicated in [RFC5761], Section 5.1.3. If no RTCP candidates have yet been gathered, default values MUST be used, as described in Section 5.2.1 above. - If the
"m="section is not bundled into another"m="section, for each candidate that has been gathered during the most recent gathering phase (see Section 3.5.1), an"a=candidate"line MUST be added, as defined in [RFC8839], Section 5.1. If candidate gathering for the section has completed, an"a=end-of-candidates"attribute MUST be added, as described in [RFC8840], Section 8.2. If the"m="section is bundled into another"m="section, both"a=candidate"and"a=end-of-candidates"MUST be omitted. 🔥 - For RtpTransceivers that are still present, the
"a=rid"lines MUST stay the same. - For RtpTransceivers that are still present, any
"a=simulcast"line MUST stay the same.
If the previous offer was applied using setLocalDescription, and a corresponding answer from the remote side has been applied using setRemoteDescription, meaning the PeerConnection is in the "have-remote-pranswer" state or the "stable" state, an offer is generated based on the negotiated session descriptions by following the steps mentioned for the "have-local-offer" state above. ✨
In addition, for each existing, non-recycled, non-rejected "m=" section in the new offer, the following adjustments are made based on the contents of the corresponding "m=" section in the current local or remote description, as appropriate:
- The
"m="line and corresponding"a=rtpmap"and"a=fmtp"lines MUST only include media formats that have not been excluded by the codec preferences of the associated transceiver and also MUST include all currently available formats. Media formats that were previously offered but are no longer available (e.g., a shared hardware codec) MAY be excluded. 👑 - Unless codec preferences have been set for the associated transceiver, the media formats on the
"m="line MUST be generated in the same order as in the most recent answer. Any media formats that were not present in the most recent answer MUST be added after all existing formats. 🌈 - The RTP header extensions MUST only include those that are present in the most recent answer. ⭐
- The RTCP feedback mechanisms MUST only include those that are present in the most recent answer, except for the case of format-specific mechanisms that are referencing a newly added media format. 💅
- The
"a=rtcp"line MUST NOT be added if the most recent answer included an"a=rtcp-mux"line. 🙈 - The
"a=rtcp-mux"line MUST be the same as that in the most recent answer. 🦋 - The
"a=rtcp-mux-only"line MUST NOT be added. 💀 - The
"a=rtcp-rsize"line MUST NOT be added unless present in the most recent answer. 🌸 - An
"a=bundle-only"line, as defined in [RFC8843], Section 6, MUST NOT be added. Instead, JSEP implementations MUST simply omit parameters in the IDENTICAL and TRANSPORT categories for bundled"m="sections, as described in [RFC8843], Section 7.1.3. 🧚 - Note that if media
"m="sections are bundled into a data"m="section, then certain TRANSPORT and IDENTICAL attributes may appear in the data"m="section even if they would otherwise only be appropriate for a media"m="section (e.g.,"a=rtcp-mux"). This cannot happen in initial offers because in the initial offer JSEP implementations always list media"m="sections (if any) before the data"m="section (if any), and at least one of those media"m="sections will not have the"a=bundle-only"attribute. Therefore, in initial offers, any"a=bundle-only""m="sections will be bundled into a preceding non-bundle-only media"m="section. 💖
The "a=group:BUNDLE" attribute MUST include the MID identifiers specified in the bundle group in the most recent answer, minus any "m=" sections that have been marked as rejected, plus any newly added or re-enabled "m=" sections. In other words, the bundle attribute MUST contain all "m=" sections that were previously bundled, as long as they are still alive, as well as any new "m=" sections. 🍓
"a=group:LS" attributes are generated in the same way as for initial offers, with the additional stipulation that any lip sync groups that were present in the most recent answer MUST continue to exist and MUST contain any previously existing MID identifiers, as long as the identified "m=" sections still exist and are not rejected, and the group still contains at least two MID identifiers. This ensures that any synchronized "recvonly" "m=" sections continue to be synchronized in the new offer. 🫶
5.2.3. Options Handling 🎀
The createOffer method takes as a parameter an RTCOfferOptions object. Special processing is performed when generating an SDP description if the following options are present. 💜
5.2.3.1. IceRestart 🔥
If the IceRestart option is specified, with a value of "true", the offer MUST indicate an ICE restart by generating new ICE ufrag and pwd attributes, as specified in [RFC8839], Section 4.4.3.1.1. If this option is specified on an initial offer, it has no effect (since a new ICE ufrag and pwd are already generated). Similarly, if the ICE configuration has changed, this option has no effect, since new ufrag and pwd attributes will be generated automatically. This option is primarily useful for reestablishing connectivity in cases where failures are detected by the application. 🌈
5.2.3.2. VoiceActivityDetection ✨
okay so silence suppression (aka DTX, "discontinuous transmission" 🎧) is lowkey a bandwidth saver. When no voice is detected, the audio switches to a special encoding, but you give up a little fidelity. it's a trade-off, bestie 💅
If the "VoiceActivityDetection" option is specified, with a value of "true", the offer MUST indicate support for silence suppression in the audio it receives by including comfort noise ("CN") codecs for each offered audio codec, as specified in [RFC3389], Section 5.1, except for codecs that have their own internal silence suppression support. For codecs that have their own internal silence suppression support, the appropriate fmtp parameters for that codec MUST be specified to indicate that silence suppression for received audio is desired. For example, when using the Opus codec [RFC6716], the "usedtx=1" parameter, specified in [RFC7587], would be used in the offer. 🎀
If the "VoiceActivityDetection" option is specified, with a value of "false", the JSEP implementation MUST NOT emit "CN" codecs. For codecs that have their own internal silence suppression support, the appropriate fmtp parameters for that codec MUST be specified to indicate that silence suppression for received audio is not desired. For example, when using the Opus codec, the "usedtx=0" parameter would be specified in the offer. In addition, the implementation MUST NOT use silence suppression for media it generates, regardless of whether the "CN" codecs or related fmtp parameters appear in the peer's description. The impact of these rules is that silence suppression in JSEP depends on mutual agreement of both sides, which ensures consistent handling regardless of which codec is used. 🫶
The "VoiceActivityDetection" option does not have any impact on the setting of the "vad" value in the signaling of the client-to-mixer audio level header extension described in [RFC6464], Section 4. 🌸
5.3. Generating an Answer 🧚
When createAnswer is called, a new SDP description MUST be created that is compatible with the supplied remote description as well as the requirements specified in [RFC8834]. The exact details of this process are explained below. 💖
5.3.1. Initial Answers 🦋
When createAnswer is called for the first time after a remote description has been provided, the result is known as the initial answer. If no remote description has been installed, an answer cannot be generated, and an error MUST be returned. 🙈
Note that the remote description SDP may not have been created by a JSEP endpoint and may not conform to all the requirements listed in Section 5.2. For many cases, this is not a problem. However, if any mandatory SDP attributes are missing or functionality listed as mandatory-to-use above is not present, this MUST be treated as an error and MUST cause the affected "m=" sections to be marked as rejected. 😭
The first step in generating an initial answer is to generate session-level attributes. The process here is identical to that indicated in Section 5.2.1 above, except that the a=ice-options line, with the "trickle" option as specified in [RFC8840], Section 4.1.3 and the "ice2" option as specified in [RFC8445], Section 10, is only included if such an option was present in the offer. ✨
The next step is to generate session-level lip sync groups, as defined in [RFC5888], Section 7. For each group of type "LS" present in the offer, select the local RtpTransceivers that are referenced by the MID values in the specified group, and determine which of them either reference a common local MediaStream (specified in the calls to addTrack/addTransceiver used to create them) or have no MediaStream to reference because they were not created by addTrack/addTransceiver. If at least two such RtpTransceivers exist, a group of type "LS" with the MID values of these RtpTransceivers MUST be added. Otherwise, the offered "LS" group MUST be ignored and no corresponding group generated in the answer. 🔥
Here's a simple example, bestie. Picture an offer with a single audio track and a single video track in the same MediaStream. SDP lines not relevant to this example have been removed for clarity. As explained in Section 5.2, a group of type "LS" has been added that references each track's RtpTransceiver. 🍓
a=group:LS a1 v1
m=audio 10000 UDP/TLS/RTP/SAVPF 0
a=mid:a1
a=msid:ms1
m=video 10001 UDP/TLS/RTP/SAVPF 96
a=mid:v1
a=msid:ms1If the answerer uses a single MediaStream when it adds its tracks, both of its transceivers will reference this stream, and so the subsequent answer will contain a "LS" group identical to that in the offer, as shown below: 👑
a=group:LS a1 v1
m=audio 20000 UDP/TLS/RTP/SAVPF 0
a=mid:a1
a=msid:ms2
m=video 20001 UDP/TLS/RTP/SAVPF 96
a=mid:v1
a=msid:ms2However, if the answerer groups its tracks into separate MediaStreams, its transceivers will reference different streams, and so the subsequent answer will not contain a "LS" group. 💀
m=audio 20000 UDP/TLS/RTP/SAVPF 0
a=mid:a1
a=msid:ms2a
m=video 20001 UDP/TLS/RTP/SAVPF 96
a=mid:v1
a=msid:ms2bFinally, if the answerer does not add any tracks, its transceivers will not reference any MediaStreams, causing the preferences of the offerer to be maintained, and so the subsequent answer will contain an identical "LS" group. 🌈
a=group:LS a1 v1
m=audio 20000 UDP/TLS/RTP/SAVPF 0
a=mid:a1
a=recvonly
m=video 20001 UDP/TLS/RTP/SAVPF 96
a=mid:v1
a=recvonlyThe example in Section 7.2 shows a more involved case of "LS" group generation. 🧁
The next step is to generate an "m=" section for each "m=" section that is present in the remote offer, as specified in [RFC3264], Section 6. For the purposes of this discussion, any session-level attributes in the offer that are also valid as media-level attributes are considered to be present in each "m=" section. Each offered "m=" section will have an associated RtpTransceiver, as described in Section 5.10. If there are more RtpTransceivers than there are "m=" sections, the unmatched RtpTransceivers will need to be associated in a subsequent offer. 🦋
For each offered "m=" section, if any of the following conditions are true, the corresponding "m=" section in the answer MUST be marked as rejected by setting the <port> in the "m=" line to zero, as indicated in [RFC3264], Section 6, and further processing for this "m=" section can be skipped:
- The associated RtpTransceiver has been stopped.
- There is no offered media format that is both supported and, if applicable, allowed by codec preferences.
- The bundle policy is "max-bundle", and this is not the first "m=" section or in the same bundle group as the first "m=" section.
- The bundle policy is "balanced", and this is not the first "m=" section for this media type or in the same bundle group as the first "m=" section for this media type.
- This "m=" section is in a bundle group, and the group's offerer tagged "m=" section is being rejected due to one of the above reasons. This requires all "m=" sections in the bundle group to be rejected, as specified in [RFC8843], Section 7.3.3.
Otherwise, each "m=" section in the answer MUST then be generated as specified in [RFC3264], Section 6.1. For the "m=" line itself, the following rules MUST be followed: 🎀
- The
<port>value would normally be set to the port of the default ICE candidate for this "m=" section, but given that no candidates are available yet, the default<port>value of 9 (Discard) MUST be used, as indicated in [RFC8840], Section 4.1.1. - The
<proto>field MUST be set to exactly match the<proto>field for the corresponding "m=" line in the offer. - If codec preferences have been set for the associated transceiver, media formats MUST be generated in the corresponding order, regardless of what was offered, and MUST exclude any codecs not present in the codec preferences.
- Otherwise, the media formats on the "m=" line MUST be generated in the same order as those offered in the current remote description, excluding any currently unsupported formats. Any currently available media formats that are not present in the current remote description MUST be added after all existing formats.
- In either case, the media formats in the answer MUST include at least one format that is present in the offer but MAY include formats that are locally supported but not present in the offer, as mentioned in [RFC3264], Section 6.1. If no common format exists, the "m=" section is rejected as described above.
The "m=" line MUST be followed immediately by a "c=" line, as specified in [RFC4566], Section 5.7. Again, as no candidates are available yet, the "c=" line MUST contain the default value IN IP4 0.0.0.0, as defined in [RFC8840], Section 4.1.3. 🫧
If the offer supports bundle, all "m=" sections to be bundled MUST use the same ICE credentials and candidates; all "m=" sections not being bundled MUST use unique ICE credentials and candidates. Each "m=" section MUST contain the following attributes (which are of attribute types other than IDENTICAL or TRANSPORT): 🍭
- If and only if present in the offer, an
a=midline, as specified in [RFC5888], Section 9.1. The MID value MUST match that specified in the offer. - A direction attribute, determined by applying the rules regarding the offered direction specified in [RFC3264], Section 6.1, and then intersecting with the direction of the associated RtpTransceiver. For example, in the case where an "m=" section is offered as "sendonly" and the local transceiver is set to "sendrecv", the result in the answer is a "recvonly" direction.
- For each media format on the "m=" line,
a=rtpmapanda=fmtplines, as specified in [RFC4566], Section 6 and [RFC3264], Section 6.1. - If "rtx" is present in the offer, for each primary codec where RTP retransmission should be used, a corresponding
a=rtpmapline indicating "rtx" with the clock rate of the primary codec and ana=fmtpline that references the payload type of the primary codec, as specified in [RFC4588], Section 8.1. - For each supported FEC mechanism,
a=rtpmapanda=fmtplines, as specified in [RFC4566], Section 6. The FEC mechanisms that MUST be supported are specified in [RFC8854], Section 7, and specific usage for each media type is outlined in Sections 4 and 5. - If this "m=" section is for media with configurable durations of media per packet, e.g., audio, an
a=maxptimeline, as described in Section 5.2. - If this "m=" section is for video media and there are known limitations on the size of images that can be decoded, an
a=imageattrline, as specified in Section 3.6. - For each supported RTP header extension that is present in the offer, an
a=extmapline, as specified in [RFC5285], Section 5. The list of header extensions that SHOULD/MUST be supported is specified in [RFC8834], Section 5.2. Any header extensions that require encryption MUST be specified as indicated in [RFC6904], Section 4. 💜
For each supported RTCP feedback mechanism that's in the offer, you add an a=rtcp-fb line, as specified in [RFC4585], Section 4.2. The list of RTCP feedback mechanisms that SHOULD/MUST be supported lives in [RFC8834], Section 5.1 🫶
- If the RtpTransceiver has a sendrecv or sendonly direction 🎬
- For each MediaStream that was tied to the transceiver when it got made via addTrack or addTransceiver, you add an
a=msidline, as specified in [RFC8830], Section 2, but you leave off theappdatafield 💅
- For each MediaStream that was tied to the transceiver when it got made via addTrack or addTransceiver, you add an
Each "m=" section that isn't bundled into another "m=" section MUST contain the following attributes (which are of category IDENTICAL or TRANSPORT) 🌸
a=ice-ufraganda=ice-pwdlines, as specified in [RFC8839], Section 5.4 ✨- For each desired digest algorithm, one or more
a=fingerprintlines for each of the endpoint's certificates, as specified in [RFC8122], Section 5 🔐 - An
a=setupline, as specified in [RFC4145], Section 4 and clarified for DTLS-SRTP scenarios in [RFC5763], Section 5. The role value in the answer MUST be"active"or"passive". When the offer has the"actpass"value (which is always the case with JSEP endpoints, bestie), the answerer SHOULD use the"active"role. Offers from non-JSEP endpoints MAY send other values for"a=setup", and in that case the answer MUST use a value that matches what the offer said 🦋 - An
a=tls-idline, as specified in [RFC8842], Section 5.3 🎀 - If it's in the offer, an
a=rtcp-muxline, as specified in [RFC5761], Section 5.1.3. Otherwise, ana=rtcpline, as specified in [RFC3605], Section 2.1, containing the default value9 IN IP4 0.0.0.0(because no candidates have been gathered yet, so it's giving placeholder energy) 🙈 - If it's in the offer, an
a=rtcp-rsizeline, as specified in [RFC5506], Section 5 ✨
If a data channel "m=" section has been offered, an "m=" section MUST also be generated for data. The <media> field MUST be set to application, and the <proto> and <fmt> fields MUST be set to exactly match the fields in the offer. Literally no wiggle room here 💀
Inside the data "m=" section, an a=mid line MUST be generated and included as described above, along with an a=sctp-port line that references the SCTP port number, as defined in [RFC8841], Section 5.1; and, if it makes sense, an a=max-message-size line, as defined in [RFC8841], Section 6.1 🧚
As discussed above, the following attributes of category IDENTICAL or TRANSPORT are only included if the data "m=" section is not bundled into another "m=" section 👑
a=ice-ufraga=ice-pwda=fingerprinta=setupa=tls-id
Note that if media "m=" sections are bundled into a data "m=" section, then certain TRANSPORT and IDENTICAL attributes might also show up in the data "m=" section, even though they'd normally only fit a media "m=" section (e.g., a=rtcp-mux). It's a whole vibe shift, iykyk 🌈
If a=group attributes with semantics of BUNDLE are offered, corresponding session-level a=group attributes MUST be added as specified in [RFC5888]. These attributes MUST have semantics BUNDLE and MUST include all mid identifiers from the offered bundle groups that haven't been rejected. Note that no matter whether a=bundle-only is in the offer, all "m=" sections in the answer MUST NOT have an a=bundle-only line. Not me saying "bundle-only" in an answer 🙅♀️
Attributes that are common across all "m=" sections MAY be moved up to the session level, but only if they're explicitly defined to be valid at the session level 🌸
The attributes that are prohibited when creating offers are also prohibited when creating answers. No loopholes, bestie 🔥
5.3.2. Subsequent Answers 💅
When createAnswer is called a second (or later) time, or is called after a local description has already been installed, the processing is a little different than for an initial answer 🍓
If the previous answer wasn't applied using setLocalDescription, meaning the PeerConnection is still in the "have-remote-offer" state, the steps for generating an initial answer MUST be followed, subject to the following restriction:
- The fields of the
o=line MUST stay the same, except for the<session-version>field, which MUST increment if the session description changes in any way from the previously generated answer 📝
If any session description was previously supplied to setLocalDescription, an answer is generated by following the steps in the "have-remote-offer" state above, along with these exceptions:
- The
s=andt=lines MUST stay the same 🌷 - Each
m=andc=line MUST be filled in with the port and address of the default candidate for the "m=" section, as described in [RFC8839], Section 4.2.1.2. Note that in certain cases, them=line protocol may not match that of the default candidate, because them=line protocol value MUST match what was supplied in the offer, as described above 🧭 - Each
a=ice-ufraganda=ice-pwdline MUST stay the same, unless the "m=" section is restarting, in which case new ICE credentials MUST be created as specified in [RFC8839], Section 4.4.1.1.1. If the "m=" section is bundled into another "m=" section, it still MUST NOT contain any ICE credentials 🔑 - Each
a=tls-idline MUST stay the same, unless the offerer'sa=tls-idline changed, in which case a new tls-id value MUST be created, as described in [RFC8842], Section 5.2 🦋 - Each
a=setupline MUST use an"active"or"passive"role value that's consistent with the existing DTLS association, if the association is being continued by the offerer 🎀 - RTCP multiplexing MUST be used, and an
a=rtcp-muxline gets inserted if and only if the "m=" section previously used RTCP multiplexing ✨ - If the "m=" section isn't bundled into another "m=" section and RTCP multiplexing isn't active, an
a=rtcpattribute line MUST be filled in with the port and address of the default RTCP candidate. If no RTCP candidates have been gathered yet, default values MUST be used, as described in Section 5.3.1 above 🌈 - If the "m=" section isn't bundled into another "m=" section, then for each candidate that was gathered during the most recent gathering phase (see Section 3.5.1), an
a=candidateline MUST be added, as defined in [RFC8839], Section 5.1. If candidate gathering for the section has completed, ana=end-of-candidatesattribute MUST be added, as described in [RFC8840], Section 8.2. If the "m=" section is bundled into another "m=" section, botha=candidateanda=end-of-candidatesMUST be omitted 🧚 - For RtpTransceivers that aren't stopped, the
a=msidline(s) MUST stay the same, regardless of changes to the transceiver's direction or track. If noa=msidline is present in the current description,a=msidline(s) MUST be generated according to the same rules as for an initial answer 💖
5.3.3. Options Handling 🎀
The createAnswer method takes an RTCAnswerOptions object as a parameter. The set of parameters for RTCAnswerOptions is different from the ones supported in RTCOfferOptions. The IceRestart option is unnecessary here, because ICE credentials will automatically change for all "m=" sections where the offerer chose to do an ICE restart 🔄
The following options are supported in RTCAnswerOptions 💅
5.3.3.1. VoiceActivityDetection ✨
Silence suppression in the answer is handled as described in Section 5.2.3.2, with one exception: if support for silence suppression wasn't indicated in the offer, the VoiceActivityDetection parameter has no effect, and the answer MUST be generated as if VoiceActivityDetection was set to "false". This is done on a per-codec basis (e.g., if the offerer somehow offered support for CN but set usedtx=0 for Opus, setting VoiceActivityDetection to "true" would result in an answer with CN codecs and usedtx=0). The impact of this rule is that an answerer won't try to use silence suppression with any endpoint that doesn't offer it, which makes silence suppression bilateral, even with non-JSEP endpoints 🤝
5.4. Modifying an Offer or Answer 🧚
The SDP returned from createOffer or createAnswer MUST NOT be changed before it gets passed to setLocalDescription. If you need precise control over the SDP, the aforementioned createOffer/createAnswer options or RtpTransceiver APIs MUST be used 🎛️
After calling setLocalDescription with an offer or answer, the application MAY modify the SDP to reduce its capabilities before sending it to the far side, as long as it follows the rules above that define a valid JSEP offer or answer. Likewise, an application that has received an offer or answer from a peer MAY modify the received SDP, subject to the same constraints, before calling setRemoteDescription 🌸
As always, the application is solely responsible for what it sends to the other party, and all incoming SDP will be processed by the JSEP implementation to the extent of its capabilities. It's an error to assume that all SDP is well formed; however, one should be able to assume that any implementation of this specification will be able to process, as a remote offer or answer, unmodified SDP coming from any other implementation of this specification 😭
5.5. Processing a Local Description 💖
When a SessionDescription is supplied to setLocalDescription, the following steps MUST be performed:
- If the description is of type "rollback", follow the processing defined in Section 5.7 and skip the processing described in the rest of this section 🔙
- Otherwise, the type of the SessionDescription is checked against the current state of the PeerConnection:
- If the type is "offer", the PeerConnection state MUST be either "stable" or "have-local-offer".
- If the type is "pranswer" or "answer", the PeerConnection state MUST be either "have-remote-offer" or "have-local-pranswer".
- If the type is not correct for the current state, processing MUST stop and an error MUST be returned. Big no, bestie 🚫
- The SessionDescription is then checked to make sure its contents are identical to those generated in the last call to createOffer/createAnswer, so they haven't been altered, as discussed in Section 5.4; otherwise, processing MUST stop and an error MUST be returned 🔍
- Next, the SessionDescription is parsed into a data structure, as described in Section 5.8 below 🧩
- Finally, the parsed SessionDescription is applied as described in Section 5.9 below ✨
5.6. Processing a Remote Description 📨✨
okay so bestie, when a SessionDescription gets handed to setRemoteDescription, there are steps you MUST do, no skipping, no cap 💅
- If the description is of type "rollback", you follow the processing in Section 5.7 and skip everything else in this section. Rollback is its own whole thing 🔄
- Otherwise, the type gets checked against where the PeerConnection currently is:
- If the type is "offer", the PeerConnection state MUST be either "stable" or "have-remote-offer". 🎀
- If the type is "pranswer" or "answer", the PeerConnection state MUST be either "have-local-offer" or "have-remote-pranswer". 🦋
- If the type doesn't match the current state, processing MUST stop and an error MUST be returned. Not me getting a mismatch and just vibing 💀
- Next, the SessionDescription gets parsed into a data structure, as described in Section 5.8 below. If parsing fails for any reason, processing MUST stop and an error MUST be returned. 🙈
- Finally, the parsed SessionDescription gets applied as described in Section 5.10 below. ✨
5.7. Processing a Rollback 🔙🌸
okay so a rollback can happen if the PeerConnection is in literally any state except "stable". That means both offers and provisional answers can get rolled back 🫶
Rollback can only be used to cancel proposed changes. There's no support for rolling back from a "stable" state to some previous "stable" state, so bestie, if a rollback is attempted in "stable", processing MUST stop and an error MUST be returned. 😭 This also means once the answerer has done setLocalDescription with its answer, that answer can't be rolled back. It's locked in, periodt 👑
The effect of rollback MUST be the same whether setLocalDescription or setRemoteDescription is called. Consistency is everything 💖
To actually process a rollback, a JSEP implementation abandons the current offer/answer transaction, sets the signaling state to "stable", and sets the pending local and/or remote description (see Sections 4.1.14 and 4.1.16) to "null". 🧹 Any resources or candidates that were allocated by the abandoned local description get discarded. Any media that's received gets processed according to the previous local and remote descriptions. 🌈
A rollback also disassociates any RtpTransceivers that were associated with "m=" sections by applying the rolled-back session description (see Sections 5.10 and 5.9). So some RtpTransceivers that used to be associated won't be associated with any "m=" section anymore. For those, the value of the RtpTransceiver's mid property MUST be set to "null", and the mapping between the transceiver and its "m=" section index MUST be discarded. 🦋
RtpTransceivers that were created by applying a remote offer that later got rolled back MUST be stopped and removed from the PeerConnection. 💀 However, an RtpTransceiver MUST NOT be removed if a track was attached to it via the addTrack method. This is so an app can call addTrack, then setRemoteDescription with an offer, then roll back that offer, then call createOffer and have an "m=" section for the added track show up in the generated offer. Main character energy for that track 🎬✨
5.8. Parsing a Session Description 🧩
okay so the SDP inside the session description object is a sequence of text lines, each one a key-value expression, as described in [RFC4566], Section 5. The SDP gets read line by line and turned into a data structure holding the deserialized info. SDP allows tons of line types, though, and not all of them matter for JSEP apps. For each line, the implementation first makes sure it's syntactically correct according to its defining ABNF, checks that it follows the semantics used in [RFC4566] and [RFC3264], and then either parses and stores or discards the value, as described below. 🤓
If any line isn't well formed or can't be parsed as described, the parser MUST stop with an error and reject the session description, even if the value was going to be discarded. This keeps implementations from accidentally misreading ambiguous SDP. No ambiguity allowed, bestie 🙅♀️💜
5.8.1. Session-Level Parsing 🌷
First up, the session-level lines get checked and parsed. These lines MUST occur in a specific order and with a specific syntax, as defined in [RFC4566], Section 5. Also note that while the specific line types (e.g., "v=", "c=") MUST occur in the defined order, lines of the same type (typically "a=") can show up in any order. 🎀
The following non-attribute lines aren't meaningful in the JSEP context and MAY be discarded once they've been checked. 🫧
- The "c=" line MUST be checked for syntax, but its value is only used for ICE mismatch detection, as defined in [RFC8445], Section 5.4. Note that JSEP implementations should never encounter this condition because ICE is required for WebRTC. 🧊
- The "i=", "u=", "e=", "p=", "t=", "r=", "z=", and "k=" lines MUST be checked for syntax, but their values aren't otherwise used. 🍓
The remaining non-attribute lines get processed like this:
- The "v=" line MUST have a version of 0, as specified in [RFC4566], Section 5.1. 🔢
- The "o=" line MUST be parsed as specified in [RFC4566], Section 5.2. 🧾
- The "b=" line, if present, MUST be parsed as specified in [RFC4566], Section 5.8, and the bwtype and bandwidth values stored. 📊
Finally, the attribute lines get processed. Specific processing MUST be applied for the following session-level attribute ("a=") lines: 🧚
- Any "a=group" lines are parsed as specified in [RFC5888], Section 5, and the group's semantics and mids are stored.
- If present, a single "a=ice-lite" line is parsed as specified in [RFC8839], Section 5.3, and a value indicating the presence of ice-lite is stored.
- If present, a single "a=ice-ufrag" line is parsed as specified in [RFC8839], Section 5.4, and the ufrag value is stored.
- If present, a single "a=ice-pwd" line is parsed as specified in [RFC8839], Section 5.4, and the password value is stored.
- If present, a single "a=ice-options" line is parsed as specified in [RFC8839], Section 5.6, and the set of specified options is stored.
- Any "a=fingerprint" lines are parsed as specified in [RFC8122], Section 5, and the set of fingerprint and algorithm values is stored.
- If present, a single "a=setup" line is parsed as specified in [RFC4145], Section 4, and the setup value is stored.
- If present, a single "a=tls-id" line is parsed as specified in [RFC8842], Section 5, and the attribute value is stored.
- Any "a=identity" lines are parsed and the identity values stored for subsequent verification, as specified in [RFC8827], Section 5.
- Any "a=extmap" lines are parsed as specified in [RFC5285], Section 5, and their values are stored.
Other attributes that aren't relevant to JSEP may also be present, and implementations SHOULD process any that they recognize. 🔍 As required by [RFC4566], Section 5.13, unknown attribute lines MUST be ignored. 🙈
Once all the session-level lines have been parsed, processing continues with the lines in "m=" sections. Onward we go 🚀💖
5.8.2. Media Section Parsing 🎀
okay bestie, just like the session-level lines, the media section lines MUST occur in the specific order and with the specific syntax defined in [RFC4566], Section 5. no vibes-based ordering here 😭
The m= line itself MUST be parsed as described in [RFC4566], Section 5.14, and the <media>, <port>, <proto>, and <fmt> values stored. 💖
Following the m= line, specific processing MUST be applied for the following non-attribute lines:
- As with the
c=line at the session level, thec=line MUST be parsed according to [RFC4566], Section 5.7, but its value is not used. 🦋 - The
b=line, if present, MUST be parsed as specified in [RFC4566], Section 5.8, and the bwtype and bandwidth values stored.
Specific processing MUST also be applied for the following attribute lines:
- If present, a single
a=ice-ufragline is parsed as specified in [RFC8839], Section 5.4, and the ufrag value is stored. - If present, a single
a=ice-pwdline is parsed as specified in [RFC8839], Section 5.4, and the password value is stored. - If present, a single
a=ice-optionsline is parsed as specified in [RFC8839], Section 5.6, and the set of specified options is stored. - Any
a=candidateattributes MUST be parsed as specified in [RFC8839], Section 5.1, and their values stored. ✨ - Any
a=remote-candidatesattributes MUST be parsed as specified in [RFC8839], Section 5.2, but their values are ignored. - If present, a single
a=end-of-candidatesattribute MUST be parsed as specified in [RFC8840], Section 8.1, and its presence or absence flagged and stored. - Any
a=fingerprintlines are parsed as specified in [RFC8122], Section 5, and the set of fingerprint and algorithm values is stored. 🔥
If the m= <proto> value indicates use of RTP, as described in Section 5.1.2 above, the following attribute lines MUST be processed: 🌈
- The
m=<fmt>value MUST be parsed as specified in [RFC4566], Section 5.14, and the individual values stored. - Any
a=rtpmapora=fmtplines MUST be parsed as specified in [RFC4566], Section 6, and their values stored. - If present, a single
a=ptimeline MUST be parsed as described in [RFC4566], Section 6, and its value stored. - If present, a single
a=maxptimeline MUST be parsed as described in [RFC4566], Section 6, and its value stored. - If present, a single direction attribute line (e.g.,
a=sendrecv) MUST be parsed as described in [RFC4566], Section 6, and its value stored. - Any
a=ssrcattributes MUST be parsed as specified in [RFC5576], Section 4.1, and their values stored. - Any
a=extmapattributes MUST be parsed as specified in [RFC5285], Section 5, and their values stored. - Any
a=rtcp-fbattributes MUST be parsed as specified in [RFC4585], Section 4.2, and their values stored. - If present, a single
a=rtcp-muxattribute MUST be parsed as specified in [RFC5761], Section 5.1.3, and its presence or absence flagged and stored. - If present, a single
a=rtcp-mux-onlyattribute MUST be parsed as specified in [RFC8858], Section 3, and its presence or absence flagged and stored. - If present, a single
a=rtcp-rsizeattribute MUST be parsed as specified in [RFC5506], Section 5, and its presence or absence flagged and stored. - If present, a single
a=rtcpattribute MUST be parsed as specified in [RFC3605], Section 2.1, but its value is ignored, as this information is superfluous when using ICE. - If present,
a=msidattributes MUST be parsed as specified in [RFC8830], Section 3.2, and their values stored, ignoring anyappdatafield. If noa=msidattributes are present, a random msid-id value is generated for a "default" MediaStream for the session, if not already present, and this value is stored. 🧚 - Any
a=imageattrattributes MUST be parsed as specified in [RFC6236], Section 3, and their values stored. - Any
a=ridlines MUST be parsed as specified in [RFC8851], Section 10, and their values stored. - If present, a single
a=simulcastline MUST be parsed as specified in [RFC8853], and its values stored.
Otherwise, if the m= <proto> value indicates use of SCTP, the following attribute lines MUST be processed: 🌸
- The
m=<fmt>value MUST be parsed as specified in [RFC8841], Section 4.3, and the application protocol value stored. - An
a=sctp-portattribute MUST be present, and it MUST be parsed as specified in [RFC8841], Section 5.2, and the value stored. - If present, a single
a=max-message-sizeattribute MUST be parsed as specified in [RFC8841], Section 6, and the value stored. Otherwise, use the specified default.
Other attributes that are not relevant to JSEP may also be present, and implementations SHOULD process any that they recognize. As required by [RFC4566], Section 5.13, unknown attribute lines MUST be ignored. 🙈
5.8.3. Semantics Verification 💅
Assuming that parsing completes successfully, the parsed description is then evaluated to ensure internal consistency as well as proper support for mandatory features. Specifically, the following checks are performed: 👑
- For each
m=section, valid values for each of the mandatory-to-use features enumerated in Section 5.1.1 MUST be present. These values MAY be either present at the media level or inherited from the session level.- ICE ufrag and password values, which MUST comply with the size limits specified in [RFC8839], Section 5.4.
- A tls-id value, which MUST be set according to [RFC8842], Section 5. If this is a re-offer or a response to a re-offer and the tls-id value is different from that presently in use, the DTLS connection is not being continued and the remote description MUST be part of an ICE restart, together with new ufrag and password values.
- A DTLS setup value, which MUST be set according to the rules specified in [RFC5763], Section 5 and MUST be consistent with the selected role of the current DTLS connection, if one exists and is being continued.
- DTLS fingerprint values, where at least one fingerprint MUST be present.
- All rid-ids referenced in an
a=simulcastline MUST exist asa=ridlines. 🍓 - Each
m=section is also checked to ensure that prohibited features are not used. - If the RTP/RTCP multiplexing policy is "require", each
m=section MUST contain ana=rtcp-muxattribute. If anm=section contains ana=rtcp-mux-onlyattribute, that section MUST also contain ana=rtcp-muxattribute. - If an
m=section was present in the previous answer, the state of RTP/RTCP multiplexing MUST match what was previously negotiated. 🫶
If this session description is of type "pranswer" or "answer", the following additional checks are applied:
- The session description MUST follow the rules defined in [RFC3264], Section 6, including the requirement that the number of
m=sections MUST exactly match the number ofm=sections in the associated offer. - For each
m=section, the media type and protocol values MUST exactly match the media type and protocol values in the correspondingm=section in the associated offer.
If any of the preceding checks failed, processing MUST stop and an error MUST be returned. ⭐
5.9. Applying a Local Description 🦋
The following steps are performed at the media engine level to apply a local description. If an error is returned, the session MUST be restored to the state it was in before performing these steps. 💜
First, m= sections are processed. For each m= section, the following steps MUST be performed; if any parameters are out of bounds or cannot be applied, processing MUST stop and an error MUST be returned. ✨
- If this
m=section is new, begin gathering candidates for it, as defined in [RFC8445], Section 5.1.1, unless it is definitively being bundled (either (1) this is an offer and them=section is marked bundle-only or (2) it is an answer and them=section is bundled into anotherm=section). - Or, if the ICE ufrag and password values have changed, trigger the ICE agent to start an ICE restart as described in [RFC8445], Section 9, and begin gathering new candidates for the
m=section. If this description is an answer, also start checks on that media section. 🔥 - If the
m=section<proto>value indicates use of RTP:- If there is no RtpTransceiver associated with this
m=section, find one and associate it with thism=section according to the following steps. Note that this situation will only occur when applying an offer.- Find the RtpTransceiver that corresponds to this
m=section, using the mapping between transceivers andm=section indices established when creating the offer. - Set the value of this RtpTransceiver's mid property to the MID of the
m=section.
- Find the RtpTransceiver that corresponds to this
- If RTCP mux is indicated, prepare to demux RTP and RTCP from the RTP ICE component, as specified in [RFC5761], Section 5.1.3.
- For each specified RTP header extension, establish a mapping between the extension ID and URI, as described in [RFC5285], Section 6.
- If the MID header extension is supported, prepare to demux RTP streams intended for this
m=section based on the MID header extension, as described in [RFC8843], Section 15. - For each specified media format, establish a mapping between the payload type and the actual media format, as described in [RFC3264], Section 6.1. In addition, prepare to demux RTP streams intended for this
m=section based on the media formats supported by thism=section, as described in [RFC8843], Section 9.2. 🌈 - For each specified "rtx" media format, establish a mapping between the RTX payload type and its associated primary payload type, as described in Sections 8.6 and 8.7 of [RFC4588].
- If the direction attribute is of type "sendrecv" or "recvonly", enable receipt and decoding of media. 🎀
- If there is no RtpTransceiver associated with this
Finally, if this description is of type "pranswer" or "answer", follow the processing defined in Section 5.11 below. 💖
5.10. Applying a Remote Description 🎀
okay bestie, this is where we take the remote description and actually DO something with it 💅 the steps are performed in order, and if an error gets returned, the session MUST be restored to the state it was in before performing these steps. no cap, that's the safety net 🫶
If the answer contains any "a=ice-options" attributes where "trickle" is listed as an attribute, update the PeerConnection canTrickleIceCandidates property to be "true". Otherwise, set this property to "false". ✨
The following steps MUST be performed for attributes at the session level; if any parameters are out of bounds or cannot be applied, processing MUST stop and an error MUST be returned. 🌸
- For any specified
"CT"bandwidth value, set this value as the limit for the maximum total bitrate for all"m="sections, as specified in [RFC4566], Section 5.8. Within this overall limit, the implementation can dynamically decide how to best allocate the available bandwidth between"m="sections, respecting any specific limits that have been specified for individual"m="sections. 🍓 - For any specified
"RR"or"RS"bandwidth values, handle as specified in [RFC3556], Section 2. 🦋 - Any
"AS"bandwidth value ([RFC4566], Section 5.8) MUST be ignored, as the meaning of this construct at the session level is not well defined. 🙈
For each "m=" section, the following steps MUST be performed; if any parameters are out of bounds or cannot be applied, processing MUST stop and an error MUST be returned. 💜
- If the ICE ufrag or password changed from the previous remote description:
- If the description is of type
"offer", the implementation MUST note that an ICE restart is needed, as described in [RFC8839], Section 4.4.1.1.1. 🔁 - If the description is of type
"answer"or"pranswer", then check to see if the current local description is an ICE restart, and if not, generate an error. If the PeerConnection state is"have-remote-pranswer"and the ICE ufrag or password changed from the previous provisional answer, then signal the ICE agent to discard any previous ICE checklist state for the"m="section. Finally, signal the ICE agent to begin checks. 🔥
- If the description is of type
- If the current local description indicates an ICE restart but neither the ICE ufrag nor the password has changed from the previous remote description (as prescribed by [RFC8445], Section 9), generate an error. 🙅♀️
- Configure the ICE components associated with this media section to use the supplied ICE remote ufrag and password for their connectivity checks. 🧚
- Pair any supplied ICE candidates with any gathered local candidates, as described in [RFC8445], Section 6.1.2, and start connectivity checks with the appropriate credentials. 👑
- If an
"a=end-of-candidates"attribute is present, process the end-of-candidates indication as described in [RFC8838], Section 14. 🌈 - If the
"m="section<proto>value indicates use of RTP:- If the
"m="section is being recycled (see Section 5.2.2), disassociate the currently associated RtpTransceiver by setting its mid property to"null", and discard the mapping between the transceiver and its"m="section index. ♻️ - If the
"m="section is not associated with any RtpTransceiver (possibly because it was disassociated in the previous step), either find an RtpTransceiver or create one according to the following steps:- If the
"m="section is sendrecv or recvonly, and there are RtpTransceivers of the same type that were added to the PeerConnection by addTrack and are not associated with any"m="section and are not stopped, find the first (according to the canonical order described in Section 5.2.1) such RtpTransceiver. 🔍 - If no RtpTransceiver was found in the previous step, create one with a recvonly direction. 🪄
- Associate the found or created RtpTransceiver with the
"m="section by setting the value of the RtpTransceiver's mid property to the MID of the"m="section, and establish a mapping between the transceiver and the index of the"m="section. If the"m="section does not include a MID (i.e., the remote endpoint does not support the MID extension), generate a value for the RtpTransceiver mid property, following the guidance for"a=mid"mentioned in Section 5.2.1. 🫶
- If the
- For each specified media format that is also supported by the local implementation, establish a mapping between the specified payload type and the media format, as described in [RFC3264], Section 6.1. Specifically, this means that the implementation records the payload type to be used in outgoing RTP packets when sending each specified media format, as well as the relative preference for each format that is indicated in their ordering. If any indicated media format is not supported by the local implementation, it MUST be ignored. 🎶
- For each specified
"rtx"media format, establish a mapping between the RTX payload type and its associated primary payload type, as described in [RFC4588], Section 4. If any referenced primary payload types are not present, this MUST result in an error. Note that RTX payload types may refer to primary payload types that are not supported by the local media implementation, in which case the RTX payload type MUST also be ignored. 🔁 - For each specified fmtp parameter that is supported by the local implementation, enable them on the associated media formats. ⭐
- For each specified Synchronization Source (SSRC) that is signaled in the
"m="section, prepare to demux RTP streams intended for this"m="section using that SSRC, as described in [RFC8843], Section 9.2. 🎧 - For each specified RTP header extension that is also supported by the local implementation, establish a mapping between the extension ID and URI, as described in [RFC5285], Section 5. Specifically, this means that the implementation records the extension ID to be used in outgoing RTP packets when sending each specified header extension. If any indicated RTP header extension is not supported by the local implementation, it MUST be ignored. 🧩
- For each specified RTCP feedback mechanism that is supported by the local implementation, enable them on the associated media formats. 📡
- For any specified
"TIAS"("Transport Independent Application Specific Maximum") bandwidth value, set this value as a constraint on the maximum RTP bitrate to be used when sending media, as specified in [RFC3890]. If a"TIAS"value is not present but an"AS"value is specified, generate a"TIAS"value using this formula:
- If the
TIAS = AS * 1000 * 0.95 - (50 * 40 * 8)The 1000 changes the unit from kbps to bps (as required by TIAS), and the 0.95 is to allocate 5% to RTCP. An estimate of header overhead is then subtracted out, in which the 50 is based on 50 packets per second, the 40 is based on typical header size (in bytes), and the 8 converts bytes to bits. Note that "TIAS" is preferred over "AS" because it provides more accurate control of bandwidth. 📊
- For any
"RR"or"RS"bandwidth values, handle as specified in [RFC3556], Section 2. 🦋 - Any specified
"CT"bandwidth value MUST be ignored, as the meaning of this construct at the media level is not well defined. 🙈 - If the
"m="section is of type"audio":- For each specified
"CN"media format, configure silence suppression for all supported media formats with the same clock rate, as described in [RFC3389], Section 5, except for formats that have their own internal silence suppression mechanisms. Silence suppression for such formats (e.g., Opus) is controlled via fmtp parameters, as discussed in Section 5.2.3.2. 🤫 - For each specified
"telephone-event"media format, enable dual-tone multifrequency (DTMF) transmission for all supported media formats with the same clock rate, as described in [RFC4733], Section 2.5.1.2. If there are any supported media formats that do not have a corresponding telephone-event format, disable DTMF transmission for those formats. ☎️ - For any specified
"ptime"value, configure the available media formats to use the specified packet size when sending. If the specified size is not supported for a media format, use the next closest value instead. ⏱️
- For each specified
Finally, if this description is of type "pranswer" or "answer", follow the processing defined in Section 5.11 below. 💖
5.11. Applying an Answer ✨
Okay bestie, so on top of the steps we already did for processing a local or remote description 💅, there's extra stuff to do whenever the description is pranswer or answer. 🦋
For each m= section, the following steps MUST be performed:
- If the
m=section has been rejected (i.e., the<port>value is set to zero in the answer), stop any reception or transmission of media for this section, and, unless a non-rejectedm=section is bundled with thism=section, discard any associated ICE components, as described in [RFC8839], Section 4.4.3.1. 🙅♀️ - If the remote DTLS fingerprint has changed or the value of the
a=tls-idattribute has changed, tear down the DTLS connection. 💔 This includes the case when the PeerConnection state ishave-remote-pranswer. If a DTLS connection needs to be torn down but the answer does not indicate an ICE restart or, in the case ofhave-remote-pranswer, new ICE credentials, an error MUST be generated. 🚨 If an ICE restart happens without a change in the tls-id value or fingerprint, then the same DTLS connection is continued over the new ICE channel. ✨ Note that although JSEP requires that answerers change the tls-id value if and only if the offerer does, non-JSEP answerers are permitted to change the tls-id value as long as the offer contained an ICE restart. So JSEP implementations that process DTLS data before receiving an answer MUST be prepared to receive either a ClientHello or data from the previous DTLS connection. 🫶 - If no valid DTLS connection exists, prepare to start a DTLS connection, using the specified roles and fingerprints, on any underlying ICE components, once they are active. 🌸
- If the
m=section<proto>value indicates use of RTP:- If the
m=section references RTCP feedback mechanisms that were not present in the correspondingm=section in the offer, this indicates a negotiation problem and MUST result in an error. 😭 However, new media formats and new RTP header extension values are permitted in the answer, as described in [RFC3264], Section 7 and [RFC5285], Section 6. 💜 - If the
m=section has RTCP mux enabled, discard the RTCP ICE component, if one exists, and begin or continue muxing RTCP over the RTP ICE component, as specified in [RFC5761], Section 5.1.3. Otherwise, prepare to transmit RTCP over the RTCP ICE component. If no RTCP ICE component exists because RTCP mux was previously enabled, this MUST result in an error. 💀 - If the
m=section has Reduced-Size RTCP enabled, configure the RTCP transmission for thism=section to use Reduced-Size RTCP, as specified in [RFC5506]. 🍓 - If the direction attribute in the answer indicates that the JSEP implementation should be sending media (
sendonlyfor local answers,recvonlyfor remote answers, orsendrecvfor either type of answer), choose the media format to send as the most preferred media format from the remote description that is also locally supported, as discussed in Sections 6.1 and 7 of [RFC3264]. Then start transmitting RTP media using that format once the underlying transport layers have been established. 🚀 If an SSRC has not already been chosen for this outgoing RTP stream, choose a unique random one. 🎲 If media is already being transmitted, the same SSRC SHOULD be used unless the clock rate of the new codec is different, in which case a new SSRC MUST be chosen, as specified in [RFC7160], Section 4.1. 🔄 - The payload type mapping from the remote description is used to determine payload types for the outgoing RTP streams, including the payload type for the send media format chosen above. Any RTP header extensions that were negotiated should be included in the outgoing RTP streams, using the extension mapping from the remote description. If the MID header extension has been negotiated, include it in the outgoing RTP streams, as indicated in [RFC8843], Section 15. If the RtpStreamId or RepairedRtpStreamId header extensions have been negotiated and rid-ids have been established, include these header extensions in the outgoing RTP streams, as indicated in [RFC8851], Section 4. 🧚
- If the
m=section is of typeaudio, and silence suppression was (1) configured for the send media format as a result of processing the remote description and (2) also enabled for that format in the local description, use silence suppression for outgoing media, in accordance with the guidance in Section 5.2.3.2. If these conditions are not met, silence suppression MUST NOT be used for outgoing media. 🤫 - If simulcast has been negotiated, send the appropriate number of Source RTP Streams as specified in [RFC8853], Section 5.3.3. 📺
- If the send media format chosen above has a corresponding
rtxmedia format or a FEC mechanism has been negotiated, establish a redundancy RTP stream with a unique random SSRC for each Source RTP Stream, and start or continue transmitting RTX/FEC packets as needed. 🛟 - If the send media format chosen above has a corresponding
redmedia format of the same clock rate, allow redundant encoding using the specified format for resiliency purposes, as discussed in [RFC8854], Section 3.2. Note that unlike RTX or FEC media formats, theredformat is transmitted on the Source RTP Stream, not the redundancy RTP stream. 👑 - Enable the RTCP feedback mechanisms referenced in the media section for all Source RTP Streams using the specified media formats. Specifically, begin or continue sending the requested feedback types and reacting to received feedback, as specified in [RFC4585], Section 4.2. When sending RTCP feedback, follow the rules and recommendations from [RFC8108], Section 5.4.1 to select which SSRC to use. 📨
- If the direction attribute in the answer indicates that the JSEP implementation should not be sending media (
recvonlyfor local answers,sendonlyfor remote answers, orinactivefor either type of answer), stop transmitting all RTP media, but continue sending RTCP, as described in [RFC3264], Section 5.1. 🤍
- If the
- If the
m=section<proto>value indicates use of SCTP:- If an SCTP association exists and the remote SCTP port has changed, discard the existing SCTP association. This includes the case when the PeerConnection state is
have-remote-pranswer. 🔁 - If no valid SCTP association exists, prepare to initiate an SCTP association over the associated ICE component and DTLS connection, using the local SCTP port value from the local description and the remote SCTP port value from the remote description, as described in [RFC8841], Section 10.2. 🔌
- If an SCTP association exists and the remote SCTP port has changed, discard the existing SCTP association. This includes the case when the PeerConnection state is
If the answer contains valid bundle groups, discard any ICE components for the m= sections that will be bundled onto the primary ICE components in each bundle, and begin muxing these m= sections accordingly, as described in [RFC8843], Section 7.4. 🧺
If the description is of type answer and there are still remaining candidates in the ICE candidate pool, discard them. 🗑️
6. Processing RTP/RTCP 🎀
Okay so when bundling is happening, figuring out which m= section an incoming RTP/RTCP packet belongs to is defined in [RFC8843], Section 9.2. 🧠 When you're NOT bundling, it's super easy, because the proper m= section is clear from the ICE component the RTP/RTCP came in on. 🦋
Once the proper m= section (or sections) is known, the RTP/RTCP gets delivered to the RtpTransceiver(s) tied to that m= section, and the rest of the processing happens at the RtpTransceiver level. 💅 This includes using the RID mechanism [RFC8851] and its RtpStreamId and RepairedRtpStreamId identifiers to tell apart multiple encoded streams and figure out which Source RTP stream a given redundancy RTP stream should repair. ✨
7. Examples 🧚
Note that this example section shows several SDP fragments. 📜 To fit RFC line-length restrictions, some of the SDP lines have been split into multiple lines, where leading whitespace means that a line is a continuation of the previous line. 🌈 Also, some blank lines have been added to make things easier to read, but they are not valid in SDP. 🙈
More examples of SDP for WebRTC call flows, including examples with IPv6 addresses, can be found in [SDP4WebRTC]. 💖
7.1. Simple Example ✨
okay so this section is a super simple example of a minimal audio/video call between two JSEP endpoints 📞 and it does NOT use Trickle ICE 🙅. The next section gives you the more detailed version of what can happen in a JSEP session 💖
The flow below shows Alice's endpoint starting the session with Bob's endpoint. The messages from the JavaScript app in Alice's browser to the JavaScript in Bob's browser (abbreviated "AliceJS" and "BobJS") are assumed to go over some signaling protocol through a web server 🌸 Both sides' JavaScript waits for ALL the candidates before sending the offer or answer, so the offers and answers are complete. Trickle ICE is not used here 🫶 The user agents (JSEP implementations) in Alice's and Bob's browsers, abbreviated "AliceUA" and "BobUA", are both on the default bundle policy of "balanced" and the default RTCP mux policy of "require" 🎀
// set up local media state
AliceJS->AliceUA: create new PeerConnection
AliceJS->AliceUA: addTrack with two tracks: audio and video
AliceJS->AliceUA: createOffer to get offer
AliceJS->AliceUA: setLocalDescription with offer
AliceUA->AliceJS: multiple onicecandidate events with candidates
// wait for ICE gathering to complete
AliceUA->AliceJS: onicecandidate event with null candidate
AliceJS->AliceUA: get |offer-A1| from pendingLocalDescription
// |offer-A1| is sent over signaling protocol to Bob
AliceJS->WebServer: signaling with |offer-A1|
WebServer->BobJS: signaling with |offer-A1|
// |offer-A1| arrives at Bob
BobJS->BobUA: create a PeerConnection
BobJS->BobUA: setRemoteDescription with |offer-A1|
BobUA->BobJS: ontrack events for audio and video tracks
// Bob accepts call
BobJS->BobUA: addTrack with local tracks
BobJS->BobUA: createAnswer
BobJS->BobUA: setLocalDescription with answer
BobUA->BobJS: multiple onicecandidate events with candidates
// wait for ICE gathering to complete
BobUA->BobJS: onicecandidate event with null candidate
BobJS->BobUA: get |answer-A1| from currentLocalDescription
// |answer-A1| is sent over signaling protocol
// to Alice
BobJS->WebServer: signaling with |answer-A1|
WebServer->AliceJS: signaling with |answer-A1|
// |answer-A1| arrives at Alice
AliceJS->AliceUA: setRemoteDescription with |answer-A1|
AliceUA->AliceJS: ontrack events for audio and video tracks
// media flows
BobUA->AliceUA: media sent from Bob to Alice
AliceUA->BobUA: media sent from Alice to BobHere's the SDP for |offer-A1| 🦋
v=0
o=- 4962303333179871722 1 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle ice2
a=group:BUNDLE a1 v1
a=group:LS a1 v1
m=audio 10100 UDP/TLS/RTP/SAVPF 96 0 8 97 98
c=IN IP4 203.0.113.100
a=mid:a1
a=sendrecv
a=rtpmap:96 opus/48000/2
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 telephone-event/8000
a=rtpmap:98 telephone-event/48000
a=fmtp:97 0-15
a=fmtp:98 0-15
a=maxptime:120
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=msid:47017fee-b6c1-4162-929c-a25110252400
a=ice-ufrag:ETEn
a=ice-pwd:OtSK0WpNtpUjkY4+86js7ZQl
a=fingerprint:sha-256
19:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04:
BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2
a=setup:actpass
a=tls-id:91bbf309c0990a6bec11e38ba2933cee
a=rtcp:10101 IN IP4 203.0.113.100
a=rtcp-mux
a=rtcp-rsize
a=candidate:1 1 udp 2113929471 203.0.113.100 10100 typ host
a=candidate:1 2 udp 2113929470 203.0.113.100 10101 typ host
a=end-of-candidates
m=video 10102 UDP/TLS/RTP/SAVPF 100 101 102 103
c=IN IP4 203.0.113.100
a=mid:v1
a=sendrecv
a=rtpmap:100 VP8/90000
a=rtpmap:101 H264/90000
a=fmtp:101 packetization-mode=1;profile-level-id=42e01f
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=100
a=rtpmap:103 rtx/90000
a=fmtp:103 apt=101
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=msid:47017fee-b6c1-4162-929c-a25110252400
a=ice-ufrag:BGKk
a=ice-pwd:mqyWsAjvtKwTGnvhPztQ9mIf
a=fingerprint:sha-256
19:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04:
BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2
a=setup:actpass
a=tls-id:91bbf309c0990a6bec11e38ba2933cee
a=rtcp:10103 IN IP4 203.0.113.100
a=rtcp-mux
a=rtcp-rsize
a=candidate:1 1 udp 2113929471 203.0.113.100 10102 typ host
a=candidate:1 2 udp 2113929470 203.0.113.100 10103 typ host
a=end-of-candidatesAnd here's the SDP for |answer-A1| 💅
v=0
o=- 6729291447651054566 1 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle ice2
a=group:BUNDLE a1 v1
a=group:LS a1 v1
m=audio 10200 UDP/TLS/RTP/SAVPF 96 0 8 97 98
c=IN IP4 203.0.113.200
a=mid:a1
a=sendrecv
a=rtpmap:96 opus/48000/2
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 telephone-event/8000
a=rtpmap:98 telephone-event/48000
a=fmtp:97 0-15
a=fmtp:98 0-15
a=maxptime:120
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=msid:61317484-2ed4-49d7-9eb7-1414322a7aae
a=ice-ufrag:6sFv
a=ice-pwd:cOTZKZNVlO9RSGsEGM63JXT2
a=fingerprint:sha-256
6B:8B:F0:65:5F:78:E2:51:3B:AC:6F:F3:3F:46:1B:35:
DC:B8:5F:64:1A:24:C2:43:F0:A1:58:D0:A1:2C:19:08
a=setup:active
a=tls-id:eec3392ab83e11ceb6a0990c903fbb19
a=rtcp-mux
a=rtcp-rsize
a=candidate:1 1 udp 2113929471 203.0.113.200 10200 typ host
a=end-of-candidates
m=video 10200 UDP/TLS/RTP/SAVPF 100 101 102 103
c=IN IP4 203.0.113.200
a=mid:v1
a=sendrecv
a=rtpmap:100 VP8/90000
a=rtpmap:101 H264/90000
a=fmtp:101 packetization-mode=1;profile-level-id=42e01f
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=100
a=rtpmap:103 rtx/90000
a=fmtp:103 apt=101
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=msid:61317484-2ed4-49d7-9eb7-1414322a7aae7.2. Detailed Example 🌈
this one is the bigger, more involved session between two JSEP endpoints 🧚 Trickle ICE is used in full trickle mode, with a bundle policy of "max-bundle", an RTCP mux policy of "require", and a single TURN server. At first, Alice and Bob both set up an audio channel and a data channel. Later on, Bob adds two video flows, one for his video feed and one for screen sharing, and BOTH of them support FEC 🔥 The video feed is configured for simulcast. Alice accepts these video flows but doesn't add any video flows of her own, so they're handled as recvonly. Alice also specifies a maximum video decoder resolution 👑
// set up local media state
AliceJS->AliceUA: create new PeerConnection
AliceJS->AliceUA: addTrack with an audio track
AliceJS->AliceUA: createDataChannel to get data channel
AliceJS->AliceUA: createOffer to get |offer-B1|
AliceJS->AliceUA: setLocalDescription with |offer-B1|
// |offer-B1| is sent over signaling protocol to Bob
AliceJS->WebServer: signaling with |offer-B1|
WebServer->BobJS: signaling with |offer-B1|
// |offer-B1| arrives at Bob
BobJS->BobUA: create a PeerConnection
BobJS->BobUA: setRemoteDescription with |offer-B1|
BobUA->BobJS: ontrack event with audio track from Alice
// candidates are sent to Bob
AliceUA->AliceJS: onicecandidate (host) |offer-B1-candidate-1|
AliceJS->WebServer: signaling with |offer-B1-candidate-1|
AliceUA->AliceJS: onicecandidate (srflx) |offer-B1-candidate-2|
AliceJS->WebServer: signaling with |offer-B1-candidate-2|
AliceUA->AliceJS: onicecandidate (relay) |offer-B1-candidate-3|
AliceJS->WebServer: signaling with |offer-B1-candidate-3|
WebServer->BobJS: signaling with |offer-B1-candidate-1|
BobJS->BobUA: addIceCandidate with |offer-B1-candidate-1|
WebServer->BobJS: signaling with |offer-B1-candidate-2|
BobJS->BobUA: addIceCandidate with |offer-B1-candidate-2|
WebServer->BobJS: signaling with |offer-B1-candidate-3|
BobJS->BobUA: addIceCandidate with |offer-B1-candidate-3|
// Bob accepts call
BobJS->BobUA: addTrack with local audio
BobJS->BobUA: createDataChannel to get data channel
BobJS->BobUA: createAnswer to get |answer-B1|
BobJS->BobUA: setLocalDescription with |answer-B1|
// |answer-B1| is sent to Alice
BobJS->WebServer: signaling with |answer-B1|
WebServer->AliceJS: signaling with |answer-B1|
AliceJS->AliceUA: setRemoteDescription with |answer-B1|
AliceUA->AliceJS: ontrack event with audio track from Bob
// candidates are sent to Alice
BobUA->BobJS: onicecandidate (host) |answer-B1-candidate-1|
BobJS->WebServer: signaling with |answer-B1-candidate-1|
BobUA->BobJS: onicecandidate (srflx) |answer-B1-candidate-2|
BobJS->WebServer: signaling with |answer-B1-candidate-2|
BobUA->BobJS: onicecandidate (relay) |answer-B1-candidate-3|
BobJS->WebServer: signaling with |answer-B1-candidate-3|
WebServer->AliceJS: signaling with |answer-B1-candidate-1|
AliceJS->AliceUA: addIceCandidate with |answer-B1-candidate-1|
WebServer->AliceJS: signaling with |answer-B1-candidate-2|
AliceJS->AliceUA: addIceCandidate with |answer-B1-candidate-2|
WebServer->AliceJS: signaling with |answer-B1-candidate-3|
AliceJS->AliceUA: addIceCandidate with |answer-B1-candidate-3|
// data channel opens
BobUA->BobJS: ondatachannel event
AliceUA->AliceJS: ondatachannel event
BobUA->BobJS: onopen
AliceUA->AliceJS: onopen
// media is flowing between endpoints
BobUA->AliceUA: audio+data sent from Bob to Alice
AliceUA->BobUA: audio+data sent from Alice to Bob
// some time later, Bob adds two video streams
// note: no candidates exchanged, because of bundle
BobJS->BobUA: addTrack with first video stream
BobJS->BobUA: addTrack with second video stream
BobJS->BobUA: createOffer to get |offer-B2|
BobJS->BobUA: setLocalDescription with |offer-B2|
// |offer-B2| is sent to Alice
BobJS->WebServer: signaling with |offer-B2|
WebServer->AliceJS: signaling with |offer-B2|
AliceJS->AliceUA: setRemoteDescription with |offer-B2|
AliceUA->AliceJS: ontrack event with first video track
AliceUA->AliceJS: ontrack event with second video track
AliceJS->AliceUA: createAnswer to get |answer-B2|
AliceJS->AliceUA: setLocalDescription with |answer-B2|
// |answer-B2| is sent over signaling protocol
// to Bob
AliceJS->WebServer: signaling with |answer-B2|
WebServer->BobJS: signaling with |answer-B2|
BobJS->BobUA: setRemoteDescription with |answer-B2|
// media is flowing between endpoints
BobUA->AliceUA: audio+video+data sent from Bob to Alice
AliceUA->BobUA: audio+video+data sent from Alice to BobOkay, the SDP for |offer-B1| looks like this 🎀
Okay bestie, picking up right where the last part left off 💖 Here's how the B1 candidates look 🦋
|offer-B1-candidate-1| looks like:
ufrag ATEn
index 0
mid a1
attr candidate:1 1 udp 2113929471 203.0.113.100 10100 typ host|offer-B1-candidate-2| looks like:
ufrag ATEn
index 0
mid a1
attr candidate:1 1 udp 1845494015 198.51.100.100 11100 typ srflx
raddr 203.0.113.100 rport 10100|offer-B1-candidate-3| looks like:
ufrag ATEn
index 0
mid a1
attr candidate:1 1 udp 255 192.0.2.100 12100 typ relay
raddr 198.51.100.100 rport 11100Now the SDP for |answer-B1| 🎀✨ Notice how it's basically a mirror of the offer, but with its own ICE credentials and fingerprint and a different DTLS setup role 🙈
v=0
o=- 7729291447651054566 1 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle ice2
a=group:BUNDLE a1 d1
m=audio 9 UDP/TLS/RTP/SAVPF 96 0 8 97 98
c=IN IP4 0.0.0.0
a=mid:a1
a=sendrecv
a=rtpmap:96 opus/48000/2
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 telephone-event/8000
a=rtpmap:98 telephone-event/48000
a=fmtp:97 0-15
a=fmtp:98 0-15
a=maxptime:120
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=msid:71317484-2ed4-49d7-9eb7-1414322a7aae
a=ice-ufrag:7sFv
a=ice-pwd:dOTZKZNVlO9RSGsEGM63JXT2
a=fingerprint:sha-256
7B:8B:F0:65:5F:78:E2:51:3B:AC:6F:F3:3F:46:1B:35:
DC:B8:5F:64:1A:24:C2:43:F0:A1:58:D0:A1:2C:19:08
a=setup:active
a=tls-id:7a25ab85b195acaf3121f5a8ab4f0f71
a=rtcp-mux
a=rtcp-mux-only
a=rtcp-rsize
m=application 9 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 0.0.0.0
a=mid:d1
a=sctp-port:5000
a=max-message-size:65536And here are the candidates for the answer side 💅 Same shape as before, just with the answerer's addresses 🌸
|answer-B1-candidate-1| looks like:
ufrag 7sFv
index 0
mid a1
attr candidate:1 1 udp 2113929471 203.0.113.200 10200 typ host|answer-B1-candidate-2| looks like:
ufrag 7sFv
index 0
mid a1
attr candidate:1 1 udp 1845494015 198.51.100.200 11200 typ srflx
raddr 203.0.113.200 rport 10200|answer-B1-candidate-3| looks like:
ufrag 7sFv
index 0
mid a1
attr candidate:1 1 udp 255 192.0.2.200 12200 typ relay
raddr 198.51.100.200 rport 11200Next up is |offer-B2| 🧚💜 This one is the big glow-up. On top of the audio and data sections, it adds two brand new video m= sections, and BOTH of them are offering FEC (that's the flexfec payload 104 you'll spot). One of them is also offering simulcast 🔥 Two more things to clock: the version number in the o= line got bumped from 1 to 2, and the c= line now points at the local candidate that was actually selected. The gathered candidates also show up as a=candidate lines 🫶
v=0
o=- 7729291447651054566 2 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle ice2
a=group:BUNDLE a1 d1 v1 v2
a=group:LS a1 v1
m=audio 12200 UDP/TLS/RTP/SAVPF 96 0 8 97 98
c=IN IP4 192.0.2.200
a=mid:a1
a=sendrecv
a=rtpmap:96 opus/48000/2
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 telephone-event/8000
a=rtpmap:98 telephone-event/48000
a=fmtp:97 0-15
a=fmtp:98 0-15
a=maxptime:120
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=msid:71317484-2ed4-49d7-9eb7-1414322a7aae
a=ice-ufrag:7sFv
a=ice-pwd:dOTZKZNVlO9RSGsEGM63JXT2
a=fingerprint:sha-256
7B:8B:F0:65:5F:78:E2:51:3B:AC:6F:F3:3F:46:1B:35:
DC:B8:5F:64:1A:24:C2:43:F0:A1:58:D0:A1:2C:19:08
a=setup:actpass
a=tls-id:7a25ab85b195acaf3121f5a8ab4f0f71
a=rtcp-mux
a=rtcp-mux-only
a=rtcp-rsize
a=candidate:1 1 udp 2113929471 203.0.113.200 10200 typ host
a=candidate:1 1 udp 1845494015 198.51.100.200 11200 typ srflx
raddr 203.0.113.200 rport 10200
a=candidate:1 1 udp 255 192.0.2.200 12200 typ relay
raddr 198.51.100.200 rport 11200
a=end-of-candidates
m=application 12200 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 192.0.2.200
a=mid:d1
a=sctp-port:5000
a=max-message-size:65536
m=video 12200 UDP/TLS/RTP/SAVPF 100 101 102 103 104
c=IN IP4 192.0.2.200
a=mid:v1
a=sendrecv
a=rtpmap:100 VP8/90000
a=rtpmap:101 H264/90000
a=fmtp:101 packetization-mode=1;profile-level-id=42e01f
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=100
a=rtpmap:103 rtx/90000
a=fmtp:103 apt=101
a=rtpmap:104 flexfec/90000
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=msid:71317484-2ed4-49d7-9eb7-1414322a7aae
a=rid:1 send
a=rid:2 send
a=rid:3 send
a=simulcast:send 1;2;3
m=video 12200 UDP/TLS/RTP/SAVPF 100 101 102 103 104
c=IN IP4 192.0.2.200
a=mid:v2
a=sendrecv
a=rtpmap:100 VP8/90000
a=rtpmap:101 H264/90000
a=fmtp:101 packetization-mode=1;profile-level-id=42e01f
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=100
a=rtpmap:103 rtx/90000
a=fmtp:103 apt=101
a=rtpmap:104 flexfec/90000
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=msid:81317484-2ed4-49d7-9eb7-1414322a7aaeLast but not least, the SDP for |answer-B2| 👑 The answerer says yes to both video sections, but it's only receiving video, so it marks them a=recvonly 📹 It also uses a=imageattr to cap the received resolution, and it keeps setup:passive so the existing DTLS roles stay exactly where they were 🌈
v=0
o=- 4962303333179871723 2 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle ice2
a=group:BUNDLE a1 d1 v1 v2
a=group:LS a1 v1
m=audio 12100 UDP/TLS/RTP/SAVPF 96 0 8 97 98
c=IN IP4 192.0.2.100
a=mid:a1
a=sendrecv
a=rtpmap:96 opus/48000/2
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 telephone-event/8000
a=rtpmap:98 telephone-event/48000
a=fmtp:97 0-15
a=fmtp:98 0-15
a=maxptime:120
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=msid:57017fee-b6c1-4162-929c-a25110252400
a=ice-ufrag:ATEn
a=ice-pwd:AtSK0WpNtpUjkY4+86js7ZQl
a=fingerprint:sha-256
29:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04:
BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2
a=setup:passive
a=tls-id:17f0f4ba8a5f1213faca591b58ba52a7
a=rtcp-mux
a=rtcp-mux-only
a=rtcp-rsize
a=candidate:1 1 udp 2113929471 203.0.113.100 10100 typ host
a=candidate:1 1 udp 1845494015 198.51.100.100 11100 typ srflx
raddr 203.0.113.100 rport 10100
a=candidate:1 1 udp 255 192.0.2.100 12100 typ relay
raddr 198.51.100.100 rport 11100
a=end-of-candidates
m=application 12100 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 192.0.2.100
a=mid:d1
a=sctp-port:5000
a=max-message-size:65536
m=video 12100 UDP/TLS/RTP/SAVPF 100 101 102 103
c=IN IP4 192.0.2.100
a=mid:v1
a=recvonly
a=rtpmap:100 VP8/90000
a=rtpmap:101 H264/90000
a=fmtp:101 packetization-mode=1;profile-level-id=42e01f
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=100
a=rtpmap:103 rtx/90000
a=fmtp:103 apt=101
a=imageattr:100 recv [x=[48:1920],y=[48:1080],q=1.0]
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
m=video 12100 UDP/TLS/RTP/SAVPF 100 101 102 103
c=IN IP4 192.0.2.100
a=mid:v2
a=recvonly
a=rtpmap:100 VP8/90000
a=rtpmap:101 H264/90000
a=fmtp:101 packetization-mode=1;profile-level-id=42e01f
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=100
a=rtpmap:103 rtx/90000
a=fmtp:103 apt=101
a=imageattr:100 recv [x=[48:1920],y=[48:1080],q=1.0]
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli7.3. Early Transport Warmup Example
okay so this example is the early-warmup technique from Section 4.1.10.1 in action 🫶 Alice's endpoint sends an offer to Bob's endpoint to start an audio/video call. Bob immediately answers, accepting the audio and video "m=" sections but marking them as sendonly (from his side), which means Alice won't send media yet 🌸 This lets the JSEP implementation start negotiating ICE and DTLS right away ✨ Then Bob's endpoint prompts him to pick up the call, and when he does, his endpoint sends a second offer that enables the audio and video "m=" sections, and that's when media goes both ways 💅
The vibe here is that as soon as the first answer lands, the implementation can get going on ICE and DTLS and set up the session transport 🦋 If transport setup finishes before the second offer goes out, the callee can send media the moment they answer, which keeps the perceived post-dial delay super low 😭 The second offer/answer exchange can also change the preferred codecs or other session parameters, so it's giving flexibility 🎀
This example also uses the "relay" ICE candidate policy from Section 3.5.3 to cut down on the ICE gathering and checking that's needed 🧚
// set up local media state AliceJS->AliceUA: create new PeerConnection with "relay" ICE policy AliceJS->AliceUA: addTrack with two tracks: audio and video AliceJS->AliceUA: createOffer to get |offer-C1| AliceJS->AliceUA: setLocalDescription with |offer-C1|
// |offer-C1| is sent over signaling protocol to Bob AliceJS->WebServer: signaling with |offer-C1| WebServer->BobJS: signaling with |offer-C1|
// |offer-C1| arrives at Bob BobJS->BobUA: create new PeerConnection with "relay" ICE policy BobJS->BobUA: setRemoteDescription with |offer-C1| BobUA->BobJS: ontrack events for audio and video
// a relay candidate is sent to Bob AliceUA->AliceJS: onicecandidate (relay) |offer-C1-candidate-1| AliceJS->WebServer: signaling with |offer-C1-candidate-1|
WebServer->BobJS: signaling with |offer-C1-candidate-1| BobJS->BobUA: addIceCandidate with |offer-C1-candidate-1|
// Bob prepares an early answer to warm up the // transport BobJS->BobUA: addTransceiver with null audio and video tracks BobJS->BobUA: transceiver.setDirection(sendonly) for both BobJS->BobUA: createAnswer BobJS->BobUA: setLocalDescription with answer
// |answer-C1| is sent over signaling protocol // to Alice BobJS->WebServer: signaling with |answer-C1| WebServer->AliceJS: signaling with |answer-C1|
// |answer-C1| (sendonly) arrives at Alice AliceJS->AliceUA: setRemoteDescription with |answer-C1| AliceUA->AliceJS: ontrack events for audio and video
// a relay candidate is sent to Alice BobUA->BobJS: onicecandidate (relay) |answer-B1-candidate-1| BobJS->WebServer: signaling with |answer-B1-candidate-1|
WebServer->AliceJS: signaling with |answer-B1-candidate-1| AliceJS->AliceUA: addIceCandidate with |answer-B1-candidate-1|
// ICE and DTLS establish while call is ringing
// Bob accepts call, starts media, and sends // new offer BobJS->BobUA: transceiver.setTrack with audio and video tracks BobUA->AliceUA: media sent from Bob to Alice BobJS->BobUA: transceiver.setDirection(sendrecv) for both transceivers BobJS->BobUA: createOffer BobJS->BobUA: setLocalDescription with offer
// |offer-C2| is sent over signaling protocol // to Alice BobJS->WebServer: signaling with |offer-C2| WebServer->AliceJS: signaling with |offer-C2|
// |offer-C2| (sendrecv) arrives at Alice AliceJS->AliceUA: setRemoteDescription with |offer-C2| AliceJS->AliceUA: createAnswer AliceJS->AliceUA: setLocalDescription with |answer-C2| AliceUA->BobUA: media sent from Alice to Bob
// |answer-C2| is sent over signaling protocol // to Bob AliceJS->WebServer: signaling with |answer-C2| WebServer->BobJS: signaling with |answer-C2| BobJS->BobUA: setRemoteDescription with |answer-C2|
The SDP for |offer-C1| looks like: 💜
v=0 o=- 1070771854436052752 1 IN IP4 0.0.0.0 s=- t=0 0 a=ice-options:trickle ice2 a=group:BUNDLE a1 v1 a=group:LS a1 v1
m=audio 9 UDP/TLS/RTP/SAVPF 96 0 8 97 98 c=IN IP4 0.0.0.0 a=mid:a1 a=sendrecv a=rtpmap:96 opus/48000/2 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:97 telephone-event/8000 a=rtpmap:98 telephone-event/48000 a=fmtp:97 0-15 a=fmtp:98 0-15 a=maxptime:120 a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level a=msid:bbce3ba6-abfc-ac63-d00a-e15b286f8fce a=ice-ufrag:4ZcD a=ice-pwd:ZaaG6OG7tCn4J/lehAGz+HHD a=fingerprint:sha-256 C4:68:F8:77:6A:44:F1:98:6D:7C:9F:47:EB:E3:34:A4: 0A:AA:2D:49:08:28:70:2E:1F:AE:18:7D:4E:3E:66:BF a=setup:actpass a=tls-id:9e5b948ade9c3d41de6617b68f769e55 a=rtcp-mux a=rtcp-mux-only a=rtcp-rsize
m=video 0 UDP/TLS/RTP/SAVPF 100 101 102 103 c=IN IP4 0.0.0.0 a=mid:v1 a=sendrecv a=rtpmap:100 VP8/90000 a=rtpmap:101 H264/90000 a=fmtp:101 packetization-mode=1;profile-level-id=42e01f a=rtpmap:102 rtx/90000 a=fmtp:102 apt=100 a=rtpmap:103 rtx/90000 a=fmtp:103 apt=101 a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id a=rtcp-fb:100 ccm fir a=rtcp-fb:100 nack a=rtcp-fb:100 nack pli a=msid:bbce3ba6-abfc-ac63-d00a-e15b286f8fce a=bundle-only
| offer-C1-candidate-1 | looks like: 🦋 |
ufrag 4ZcD index 0 mid a1 attr candidate:1 1 udp 255 192.0.2.100 12100 typ relay raddr 0.0.0.0 rport 0
The SDP for |answer-C1| looks like: 🌈
v=0 o=- 6386516489780559513 1 IN IP4 0.0.0.0 s=- t=0 0 a=ice-options:trickle ice2 a=group:BUNDLE a1 v1 a=group:LS a1 v1
m=audio 9 UDP/TLS/RTP/SAVPF 96 0 8 97 98 c=IN IP4 0.0.0.0 a=mid:a1 a=sendonly a=rtpmap:96 opus/48000/2 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:97 telephone-event/8000 a=rtpmap:98 telephone-event/48000 a=fmtp:97 0-15 a=fmtp:98 0-15 a=maxptime:120 a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level a=msid:751f239e-4ae0-c549-aa3d-890de772998b a=ice-ufrag:TpaA a=ice-pwd:t2Ouhc67y8JcCaYZxUUTgKw/ a=fingerprint:sha-256 A2:F3:A5:6D:4C:8C:1E:B2:62:10:4A:F6:70:61:C4:FC: 3C:E0:01:D6:F3:24:80:74:DA:7C:3E:50:18:7B:CE:4D a=setup:active a=tls-id:55e967f86b7166ed14d3c9eda849b5e9 a=rtcp-mux a=rtcp-mux-only a=rtcp-rsize
m=video 9 UDP/TLS/RTP/SAVPF 100 101 102 103 c=IN IP4 0.0.0.0 a=mid:v1 a=sendonly a=rtpmap:100 VP8/90000 a=rtpmap:101 H264/90000 a=fmtp:101 packetization-mode=1;profile-level-id=42e01f a=rtpmap:102 rtx/90000 a=fmtp:102 apt=100 a=rtpmap:103 rtx/90000 a=fmtp:103 apt=101 a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id a=rtcp-fb:100 ccm fir a=rtcp-fb:100 nack a=rtcp-fb:100 nack pli a=msid:751f239e-4ae0-c549-aa3d-890de772998b
| answer-C1-candidate-1 | looks like: 🧚 |
ufrag TpaA index 0 mid a1 attr candidate:1 1 udp 255 192.0.2.200 12200 typ relay raddr 0.0.0.0 rport 0
The SDP for |offer-C2| looks like: 🎀
v=0 o=- 6386516489780559513 2 IN IP4 0.0.0.0 s=- t=0 0 a=ice-options:trickle ice2 a=group:BUNDLE a1 v1 a=group:LS a1 v1
m=audio 12200 UDP/TLS/RTP/SAVPF 96 0 8 97 98 c=IN IP4 192.0.2.200 a=mid:a1 a=sendrecv a=rtpmap:96 opus/48000/2 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:97 telephone-event/8000 a=rtpmap:98 telephone-event/48000 a=fmtp:97 0-15 a=fmtp:98 0-15 a=maxptime:120 a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level a=msid:751f239e-4ae0-c549-aa3d-890de772998b a=ice-ufrag:TpaA a=ice-pwd:t2Ouhc67y8JcCaYZxUUTgKw/ a=fingerprint:sha-256 A2:F3:A5:6D:4C:8C:1E:B2:62:10:4A:F6:70:61:C4:FC: 3C:E0:01:D6:F3:24:80:74:DA:7C:3E:50:18:7B:CE:4D a=setup:actpass a=tls-id:55e967f86b7166ed14d3c9eda849b5e9 a=rtcp-mux a=rtcp-mux-only a=rtcp-rsize a=candidate:1 1 udp 255 192.0.2.200 12200 typ relay raddr 0.0.0.0 rport 0 a=end-of-candidates
m=video 12200 UDP/TLS/RTP/SAVPF 100 101 102 103 c=IN IP4 192.0.2.200 a=mid:v1 a=sendrecv a=rtpmap:100 VP8/90000 a=rtpmap:101 H264/90000 a=fmtp:101 packetization-mode=1;profile-level-id=42e01f a=rtpmap:102 rtx/90000 a=fmtp:102 apt=100 a=rtpmap:103 rtx/90000 a=fmtp:103 apt=101 a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id a=rtcp-fb:100 ccm fir a=rtcp-fb:100 nack a=rtcp-fb:100 nack pli a=msid:751f239e-4ae0-c549-aa3d-890de772998b
The SDP for |answer-C2| looks like: 👑
v=0 o=- 1070771854436052752 2 IN IP4 0.0.0.0 s=- t=0 0 a=ice-options:trickle ice2 a=group:BUNDLE a1 v1 a=group:LS a1 v1
m=audio 12100 UDP/TLS/RTP/SAVPF 96 0 8 97 98 c=IN IP4 192.0.2.100 a=mid:a1 a=sendrecv a=rtpmap:96 opus/48000/2 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:97 telephone-event/8000 a=rtpmap:98 telephone-event/48000 a=fmtp:97 0-15 a=fmtp:98 0-15 a=maxptime:120 a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level a=msid:bbce3ba6-abfc-ac63-d00a-e15b286f8fce a=ice-ufrag:4ZcD a=ice-pwd:ZaaG6OG7tCn4J/lehAGz+HHD a=fingerprint:sha-256 C4:68:F8:77:6A:44:F1:98:6D:7C:9F:47:EB:E3:34:A4: 0A:AA:2D:49:08:28:70:2E:1F:AE:18:7D:4E:3E:66:BF a=setup:passive a=tls-id:9e5b948ade9c3d41de6617b68f769e55 a=rtcp-mux a=rtcp-mux-only a=rtcp-rsize a=candidate:1 1 udp 255 192.0.2.100 12100 typ relay raddr 0.0.0.0 rport 0 a=end-of-candidates
m=video 12100 UDP/TLS/RTP/SAVPF 100 101 102 103 c=IN IP4 192.0.2.100 a=mid:v1 a=sendrecv a=rtpmap:100 VP8/90000 a=rtpmap:101 H264/90000 a=fmtp:101 packetization-mode=1;profile-level-id=42e01f a=rtpmap:102 rtx/90000 a=fmtp:102 apt=100 a=rtpmap:103 rtx/90000 a=fmtp:103 apt=101 a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id a=rtcp-fb:100 ccm fir a=rtcp-fb:100 nack a=rtcp-fb:100 nack pli a=msid:bbce3ba6-abfc-ac63-d00a-e15b286f8fce
8. Security Considerations 🔐✨
okay so the IETF already dropped the big security architecture docs for WebRTC as a whole, [RFC8827] [RFC8826] 📚 and this section is just the stuff that's specific to this document 💅
formally JSEP is an API, but bestie, it's way more iconic to think of it as an Internet protocol 🌐 where the application JavaScript is basically untrustworthy from the JSEP implementation's point of view 👀 so the threat model of [RFC3552] applies 🛡️ JavaScript can call the API in any order with any inputs, even malicious ones 💀 this is extra relevant for the SDP that gets handed to setLocalDescription. correct usage means the SDP was derived from createOffer or createAnswer, but there's zero guarantee apps actually do that 🙈 so the JSEP implementation MUST be prepared for the JavaScript to pass in bogus data instead 🚨
and the other way around, the app programmer needs to know that the JavaScript does NOT have complete control of endpoint behavior 🫣 one case that really deserves a spotlight: editing ICE candidates out of the SDP, or suppressing trickled candidates, does not do what you'd expect 😭 implementations will still run checks from those candidates even if they never get sent to the other side. so it's not possible to stop the remote peer from learning your public IP address just by removing server-reflexive candidates 🙅♀️ if you want to hide your public IP, you MUST instead configure the ICE agent to use only relay candidates 🧚🔒
9. IANA Considerations 🌸
this document has no IANA actions, so nothing to register here, periodt 💖
10. References 📖✨
10.1. Normative References 📚✨
Okay bestie, these are the must-reads 🫶 The spec leans on every one of them, so they're basically the homework list 💅
- [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/info/rfc2119>.
- [RFC3261] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, "SIP: Session Initiation Protocol", RFC 3261, DOI 10.17487/RFC3261, June 2002, <https://www.rfc-editor.org/info/rfc3261>. 📞
- [RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with Session Description Protocol (SDP)", RFC 3264, DOI 10.17487/RFC3264, June 2002, <https://www.rfc-editor.org/info/rfc3264>. 🤝
- [RFC3552] Rescorla, E. and B. Korver, "Guidelines for Writing RFC Text on Security Considerations", BCP 72, RFC 3552, DOI 10.17487/RFC3552, July 2003, <https://www.rfc-editor.org/info/rfc3552>. 🔐
- [RFC3605] Huitema, C., "Real Time Control Protocol (RTCP) attribute in Session Description Protocol (SDP)", RFC 3605, DOI 10.17487/RFC3605, October 2003, <https://www.rfc-editor.org/info/rfc3605>.
- [RFC3711] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. Norrman, "The Secure Real-time Transport Protocol (SRTP)", RFC 3711, DOI 10.17487/RFC3711, March 2004, <https://www.rfc-editor.org/info/rfc3711>. 🛡️
- [RFC3890] Westerlund, M., "A Transport Independent Bandwidth Modifier for the Session Description Protocol (SDP)", RFC 3890, DOI 10.17487/RFC3890, September 2004, <https://www.rfc-editor.org/info/rfc3890>.
- [RFC4145] Yon, D. and G. Camarillo, "TCP-Based Media Transport in the Session Description Protocol (SDP)", RFC 4145, DOI 10.17487/RFC4145, September 2005, <https://www.rfc-editor.org/info/rfc4145>.
- [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session Description Protocol", RFC 4566, DOI 10.17487/RFC4566, July 2006, <https://www.rfc-editor.org/info/rfc4566>. 📝
- [RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, "Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, DOI 10.17487/RFC4585, July 2006, <https://www.rfc-editor.org/info/rfc4585>.
- [RFC5124] Ott, J. and E. Carrara, "Extended Secure RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/SAVPF)", RFC 5124, DOI 10.17487/RFC5124, February 2008, <https://www.rfc-editor.org/info/rfc5124>.
- [RFC5285] Singer, D. and H. Desineni, "A General Mechanism for RTP Header Extensions", RFC 5285, DOI 10.17487/RFC5285, July 2008, <https://www.rfc-editor.org/info/rfc5285>.
- [RFC5761] Perkins, C. and M. Westerlund, "Multiplexing RTP Data and Control Packets on a Single Port", RFC 5761, DOI 10.17487/RFC5761, April 2010, <https://www.rfc-editor.org/info/rfc5761>.
- [RFC5888] Camarillo, G. and H. Schulzrinne, "The Session Description Protocol (SDP) Grouping Framework", RFC 5888, DOI 10.17487/RFC5888, June 2010, <https://www.rfc-editor.org/info/rfc5888>. 🧩
- [RFC6236] Johansson, I. and K. Jung, "Negotiation of Generic Image Attributes in the Session Description Protocol (SDP)", RFC 6236, DOI 10.17487/RFC6236, May 2011, <https://www.rfc-editor.org/info/rfc6236>.
- [RFC6347] Rescorla, E. and N. Modadugu, "Datagram Transport Layer Security Version 1.2", RFC 6347, DOI 10.17487/RFC6347, January 2012, <https://www.rfc-editor.org/info/rfc6347>. 🔒
- [RFC6716] Valin, JM., Vos, K., and T. Terriberry, "Definition of the Opus Audio Codec", RFC 6716, DOI 10.17487/RFC6716, September 2012, <https://www.rfc-editor.org/info/rfc6716>. 🎶
- [RFC6904] Lennox, J., "Encryption of Header Extensions in the Secure Real-time Transport Protocol (SRTP)", RFC 6904, DOI 10.17487/RFC6904, April 2013, <https://www.rfc-editor.org/info/rfc6904>.
- [RFC7160] Petit-Huguenin, M. and G. Zorn, Ed., "Support for Multiple Clock Rates in an RTP Session", RFC 7160, DOI 10.17487/RFC7160, April 2014, <https://www.rfc-editor.org/info/rfc7160>. ⏱️
- [RFC7587] Spittka, J., Vos, K., and JM. Valin, "RTP Payload Format for the Opus Speech and Audio Codec", RFC 7587, DOI 10.17487/RFC7587, June 2015, <https://www.rfc-editor.org/info/rfc7587>.
- [RFC7742] Roach, A.B., "WebRTC Video Processing and Codec Requirements", RFC 7742, DOI 10.17487/RFC7742, March 2016, <https://www.rfc-editor.org/info/rfc7742>. 🎥
- [RFC7850] Nandakumar, S., "Registering Values of the SDP 'proto' Field for Transporting RTP Media over TCP under Various RTP Profiles", RFC 7850, DOI 10.17487/RFC7850, April 2016, <https://www.rfc-editor.org/info/rfc7850>.
- [RFC7874] Valin, JM. and C. Bran, "WebRTC Audio Codec and Processing Requirements", RFC 7874, DOI 10.17487/RFC7874, May 2016, <https://www.rfc-editor.org/info/rfc7874>. 🎙️
- [RFC8108] Lennox, J., Westerlund, M., Wu, Q., and C. Perkins, "Sending Multiple RTP Streams in a Single RTP Session", RFC 8108, DOI 10.17487/RFC8108, March 2017, <https://www.rfc-editor.org/info/rfc8108>.
- [RFC8122] Lennox, J. and C. Holmberg, "Connection-Oriented Media Transport over the Transport Layer Security (TLS) Protocol in the Session Description Protocol (SDP)", RFC 8122, DOI 10.17487/RFC8122, March 2017, <https://www.rfc-editor.org/info/rfc8122>.
- [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <https://www.rfc-editor.org/info/rfc8174>. 🔤
- [RFC8445] Keranen, A., Holmberg, C., and J. Rosenberg, "Interactive Connectivity Establishment (ICE): A Protocol for Network Address Translator (NAT) Traversal", RFC 8445, DOI 10.17487/RFC8445, July 2018, <https://www.rfc-editor.org/info/rfc8445>. 🧭
- [RFC8826] Rescorla, E., "Security Considerations for WebRTC", RFC 8826, DOI 10.17487/RFC8826, January 2021, <https://www.rfc-editor.org/info/rfc8826>. 🛡️
- [RFC8827] Rescorla, E., "WebRTC Security Architecture", RFC 8827, DOI 10.17487/RFC8827, January 2021, <https://www.rfc-editor.org/info/rfc8827>. 🏰
- [RFC8830] Alvestrand, H., "WebRTC MediaStream Identification in the Session Description Protocol", RFC 8830, DOI 10.17487/RFC8830, January 2021, <https://www.rfc-editor.org/info/rfc8830>.
- [RFC8834] Perkins, C., Westerlund, M., and J. Ott, "Media Transport and Use of RTP in WebRTC", RFC 8834, DOI 10.17487/RFC8834, January 2021, <https://www.rfc-editor.org/info/rfc8834>. 🚚
- [RFC8838] Ivov, E., Uberti, J., and P. Saint-Andre, "Trickle ICE: Incremental Provisioning of Candidates for the Interactive Connectivity Establishment (ICE) Protocol", RFC 8838, DOI 10.17487/RFC8838, January 2021, <https://www.rfc-editor.org/info/rfc8838>. 💧
- [RFC8839] Petit-Huguenin, M., Nandakumar, S., Holmberg, C., Keränen, A., and R. Shpount, "Session Description Protocol (SDP) Offer/Answer Procedures for Interactive Connectivity Establishment (ICE)", RFC 8839, DOI 10.17487/RFC8839, January 2021, <https://www.rfc-editor.org/info/rfc8839>.
- [RFC8840] Ivov, E., Stach, T., Marocco, E., and C. Holmberg, "A Session Initiation Protocol (SIP) Usage for Incremental Provisioning of Candidates for the Interactive Connectivity Establishment (Trickle ICE)", RFC 8840, DOI 10.17487/RFC8840, January 2021, <https://www.rfc-editor.org/info/rfc8840>.
- [RFC8841] Holmberg, C., Shpount, R., Loreto, S., and G. Camarillo, "Session Description Protocol (SDP) Offer/Answer Procedures for Stream Control Transmission Protocol (SCTP) over Datagram Transport Layer Security (DTLS) Transport", RFC 8841, DOI 10.17487/RFC8841, January 2021, <https://www.rfc-editor.org/info/rfc8841>. 🦋
- [RFC8842] Holmberg, C. and R. Shpount, "Session Description Protocol (SDP) Offer/Answer Considerations for Datagram Transport Layer Security (DTLS) and Transport Layer Security (TLS)", RFC 8842, DOI 10.17487/RFC8842, January 2021, <https://www.rfc-editor.org/info/rfc8842>.
- [RFC8843] Holmberg, C., Alvestrand, H., and C. Jennings, "Negotiating Media Multiplexing Using the Session Description Protocol (SDP)", RFC 8843, DOI 10.17487/RFC8843, January 2021, <https://www.rfc-editor.org/info/rfc8843>. 🌈
- [RFC8851] Roach, A.B., Ed., "RTP Payload Format Restrictions", RFC 8851, DOI 10.17487/RFC8851, January 2021, <https://www.rfc-editor.org/info/rfc8851>.
- [RFC8852] Roach, A.B., Nandakumar, S., and P. Thatcher, "RTP Stream Identifier Source Description (SDES)", RFC 8852, DOI 10.17487/RFC8852, January 2021, <https://www.rfc-editor.org/info/rfc8852>. 🏷️
- [RFC8853] Burman, B., Westerlund, M., Nandakumar, S., and M. Zanaty, "Using Simulcast in Session Description Protocol (SDP) and RTP Sessions", RFC 8853, DOI 10.17487/RFC8853, January 2021, <https://www.rfc-editor.org/info/rfc8853>. 📺
- [RFC8854] Uberti, J., "WebRTC Forward Error Correction Requirements", RFC 8854, DOI 10.17487/RFC8854, January 2021, <https://www.rfc-editor.org/info/rfc8854>.
- [RFC8858] Holmberg, C., "Indicating Exclusive Support of RTP and RTP Control Protocol (RTCP) Multiplexing Using the Session Description Protocol (SDP)", RFC 8858, DOI 10.17487/RFC8858, January 2021, <https://www.rfc-editor.org/info/rfc8858>.
- [RFC8859] Nandakumar, S., "A Framework for Session Description Protocol (SDP) Attributes When Multiplexing", RFC 8859, DOI 10.17487/RFC8859, January 2021, <https://www.rfc-editor.org/info/rfc8859>. 🎀
10.2. Informative References ✨
Okay bestie, now the informative ones. These are the sources the doc leans on for background, so they're not strictly required reading, but they're super useful if you want the full picture 🧚
- [RFC3389] Zopf, R., "Real-time Transport Protocol (RTP) Payload for Comfort Noise (CN)", RFC 3389, DOI 10.17487/RFC3389, September 2002,
https://www.rfc-editor.org/info/rfc3389. 🎀 - [RFC3556] Casner, S., "Session Description Protocol (SDP) Bandwidth Modifiers for RTP Control Protocol (RTCP) Bandwidth", RFC 3556, DOI 10.17487/RFC3556, July 2003,
https://www.rfc-editor.org/info/rfc3556. 🦋 - [RFC3960] Camarillo, G. and H. Schulzrinne, "Early Media and Ringing Tone Generation in the Session Initiation Protocol (SIP)", RFC 3960, DOI 10.17487/RFC3960, December 2004,
https://www.rfc-editor.org/info/rfc3960. 🔔 - [RFC4568] Andreasen, F., Baugher, M., and D. Wing, "Session Description Protocol (SDP) Security Descriptions for Media Streams", RFC 4568, DOI 10.17487/RFC4568, July 2006,
https://www.rfc-editor.org/info/rfc4568. 🔐 - [RFC4588] Rey, J., Leon, D., Miyazaki, A., Varsa, V., and R. Hakenberg, "RTP Retransmission Payload Format", RFC 4588, DOI 10.17487/RFC4588, July 2006,
https://www.rfc-editor.org/info/rfc4588. 🔁 - [RFC4733] Schulzrinne, H. and T. Taylor, "RTP Payload for DTMF Digits, Telephony Tones, and Telephony Signals", RFC 4733, DOI 10.17487/RFC4733, December 2006,
https://www.rfc-editor.org/info/rfc4733. ☎️ - [RFC5245] Rosenberg, J., "Interactive Connectivity Establishment (ICE): A Protocol for Network Address Translator (NAT) Traversal for Offer/Answer Protocols", RFC 5245, DOI 10.17487/RFC5245, April 2010,
https://www.rfc-editor.org/info/rfc5245. 🧭 - [RFC5506] Johansson, I. and M. Westerlund, "Support for Reduced-Size Real-Time Transport Control Protocol (RTCP): Opportunities and Consequences", RFC 5506, DOI 10.17487/RFC5506, April 2009,
https://www.rfc-editor.org/info/rfc5506. 📉 - [RFC5576] Lennox, J., Ott, J., and T. Schierl, "Source-Specific Media Attributes in the Session Description Protocol (SDP)", RFC 5576, DOI 10.17487/RFC5576, June 2009,
https://www.rfc-editor.org/info/rfc5576. 🎬 - [RFC5763] Fischl, J., Tschofenig, H., and E. Rescorla, "Framework for Establishing a Secure Real-time Transport Protocol (SRTP) Security Context Using Datagram Transport Layer Security (DTLS)", RFC 5763, DOI 10.17487/RFC5763, May 2010,
https://www.rfc-editor.org/info/rfc5763. 🛡️ - [RFC5764] McGrew, D. and E. Rescorla, "Datagram Transport Layer Security (DTLS) Extension to Establish Keys for the Secure Real-time Transport Protocol (SRTP)", RFC 5764, DOI 10.17487/RFC5764, May 2010,
https://www.rfc-editor.org/info/rfc5764. 🗝️ - [RFC6120] Saint-Andre, P., "Extensible Messaging and Presence Protocol (XMPP): Core", RFC 6120, DOI 10.17487/RFC6120, March 2011,
https://www.rfc-editor.org/info/rfc6120. 💬 - [RFC6464] Lennox, J., Ed., Ivov, E., and E. Marocco, "A Real-time Transport Protocol (RTP) Header Extension for Client-to-Mixer Audio Level Indication", RFC 6464, DOI 10.17487/RFC6464, December 2011,
https://www.rfc-editor.org/info/rfc6464. 🎤 - [RFC8828] Uberti, J. and G. Shieh, "WebRTC IP Address Handling Requirements", RFC 8828, DOI 10.17487/RFC8828, January 2021,
https://www.rfc-editor.org/info/rfc8828. 🌐 - [SDP4WebRTC] Nandakumar, S. and C. Jennings, "Annotated Example SDP for WebRTC", Work in Progress, Internet-Draft, draft-ietf-rtcweb-sdp-14, 17 December 2020,
https://tools.ietf.org/html/draft-ietf-rtcweb-sdp-14. 📝 - [TS26.114] 3GPP, "3rd Generation Partnership Project; Technical Specification Group Services and System Aspects; IP Multimedia Subsystem (IMS); Multimedia Telephony; Media handling and interaction (Release 16)", 3GPP TS 26.114 V16.3.0, September 2019,
https://www.3gpp.org/DynaReport/26114.htm. 📞 - [W3C.webrtc] Jennings, C., Ed., Boström, H., Ed., and J. Bruaroey, Ed., "WebRTC 1.0: Real-time Communication Between Browsers", World Wide Web Consortium PR PR-webrtc-20201215, December 2020,
https://www.w3.org/TR/2020/PR-webrtc-20201215/. 🌸
Appendix A. SDP ABNF Syntax 🧩
Here's the appendix for the syntax validation we talked about in Section 5.8. It's a big list of ABNF definitions, and this table tells you which spec each attribute comes from 💅
+=========================+==========================+
| Attribute | Reference |
+=========================+==========================+
| ptime | Section 6 of [RFC4566] |
+-------------------------+--------------------------+
| maxptime | Section 6 of [RFC4566] |
+-------------------------+--------------------------+
| rtpmap | Section 6 of [RFC4566] |
+-------------------------+--------------------------+
| recvonly | Section 9 of [RFC4566] |
+-------------------------+--------------------------+
| sendrecv | Section 9 of [RFC4566] |
+-------------------------+--------------------------+
| sendonly | Section 9 of [RFC4566] |
+-------------------------+--------------------------+
| inactive | Section 9 of [RFC4566] |
+-------------------------+--------------------------+
| fmtp | Section 9 of [RFC4566] |
+-------------------------+--------------------------+
| rtcp | Section 2.1 of [RFC3605] |
+-------------------------+--------------------------+
| setup | Section 4 of [RFC4145] |
+-------------------------+--------------------------+
| fingerprint | Section 5 of [RFC8122] |
+-------------------------+--------------------------+
| rtcp-fb | Section 4.2 of [RFC4585] |
+-------------------------+--------------------------+
| extmap | Section 7 of [RFC5285] |
+-------------------------+--------------------------+
| mid | Section 4 of [RFC5888] |
+-------------------------+--------------------------+
| group | Section 5 of [RFC5888] |
+-------------------------+--------------------------+
| imageattr | Section 3.1 of [RFC6236] |
+-------------------------+--------------------------+
| extmap (encrypt option) | Section 4 of [RFC6904] |
+-------------------------+--------------------------+
| candidate | Section 5.1 of [RFC8839] |
+-------------------------+--------------------------+
| remote-candidates | Section 5.2 of [RFC8839] |
+-------------------------+--------------------------+
| ice-lite | Section 5.3 of [RFC8839] |
+-------------------------+--------------------------+
| ice-ufrag | Section 5.4 of [RFC8839] |
+-------------------------+--------------------------+
| ice-pwd | Section 5.4 of [RFC8839] |
+-------------------------+--------------------------+
| ice-options | Section 5.6 of [RFC8839] |
+-------------------------+--------------------------+
| msid | Section 3 of [RFC8830] |
+-------------------------+--------------------------+
| rid | Section 10 of [RFC8851] |
+-------------------------+--------------------------+
| simulcast | Section 5.1 of [RFC8853] |
+-------------------------+--------------------------+
| tls-id | Section 4 of [RFC8842] |
+-------------------------+--------------------------+
Table 1: SDP ABNF ReferencesAcknowledgements 🫶
Huge shoutout to Harald Alvestrand, Taylor Brandstetter, Suhas Nandakumar, and Peter Thatcher, who wrote a ton of the actual text for this doc. 👑
Bernard Aboba, Adam Bergkvist, Jan-Ivar Bruaroey, Dan Burnett, Ben Campbell, Alissa Cooper, Richard Ejzak, Stefan Håkansson, Ted Hardie, Christer Holmberg, Andrew Hutton, Randell Jesup, Matthew Kaufman, Anant Narayanan, Adam Roach, Robert Sparks, Neil Stratford, Martin Thomson, Sean Turner, and Magnus Westerlund all gave really valuable feedback too. The whole group is literally iconic 🌈
Authors' Addresses 💌
Justin Uberti Google 747 6th Street South Kirkland, WA 98033 United States of America
Email: justin@uberti.name
Cullen Jennings Cisco 400 3rd Avenue SW Calgary AB T2P 4H2 Canada
Email: fluffy@iii.ca
Eric Rescorla (editor) Mozilla
Email: ekr@rtfm.com
📦 where this page comes from
- ietf: rfc8829 2021-01
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms