RFC 9429
JavaScript Session Establishment Protocol (JSEP)
RFC 9429: JavaScript Session Establishment Protocol (JSEP) 💖
okay bestie, this is RFC 9429 🎀 It's a Standards Track document from the IETF (ISSN 2070-1721), published in April 2024. J. Uberti wrote it, C. Jennings (from Cisco) co-wrote it, and E. Rescorla, Ed. (from Windy Hill Systems, LLC) edited it 👑 It obsoletes RFC 8829. The title block doesn't list any RFC it updates.
Abstract ✨
This doc explains how a JavaScript app can take control of the signaling side of a multimedia session 🦋 It does that through the RTCPeerConnection interface from the W3C spec, and it talks about how all of this fits with the signaling protocols that already exist.
This spec obsoletes RFC 8829, so it replaces that one 💅
Status of This Memo 🌸
This is an Internet Standards Track document.
It's an IETF product, which means it has the consensus of the IETF community. It went through public review and got approved for publication by the Internet Engineering Steering Group (IESG). If you want more info on Internet Standards, check out Section 2 of RFC 7841 ✨
For the current status, any errata, and where to send feedback, go to https://www.rfc-editor.org/info/rfc9429 🧚
Copyright Notice 💜
Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved 🔒
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they stand on the publication date. Please actually read these, because they lay out your rights and restrictions for this document 🙈 Any Code Components pulled out of this document have to include the Revised BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as the Revised BSD License describes.
Table of Contents 📑
1. Introduction
1.1. General Design of JSEP
1.2. Other Approaches Considered
1.3. Changes from RFC 8829
2. Terminology
3. Semantics and Syntax
3.1. Signaling Model
3.2. Session Descriptions and State Machine
3.3. Session Description Format
3.4. Session Description Control
3.4.1. RtpTransceivers
3.4.2. RtpSenders
3.4.3. RtpReceivers
3.5. ICE
3.5.1. ICE Gathering Overview
3.5.2. ICE Candidate Trickling
3.5.2.1. ICE Candidate Format
3.5.3. ICE Candidate Policy
3.5.4. ICE Candidate Pool
3.5.5. ICE Versions
3.6. Video Size Negotiation
3.6.1. Creating an imageattr Attribute
3.6.2. Interpreting imageattr Attributes
3.7. Simulcast
3.8. Interactions with Forking
3.8.1. Sequential Forking
3.8.2. Parallel Forking
4. Interface
4.1. PeerConnection
4.1.1. Constructor
4.1.2. addTrack
4.1.3. removeTrack
4.1.4. addTransceiver
4.1.5. onaddtrack Event
4.1.6. createDataChannel
4.1.7. ondatachannel Event
4.1.8. createOffer
4.1.9. createAnswer
4.1.10. SessionDescriptionType
4.1.10.1. Use of Provisional Answers
4.1.10.2. Rollback
4.1.11. setLocalDescription
4.1.12. setRemoteDescription
4.1.13. currentLocalDescription
4.1.14. pendingLocalDescription
4.1.15. currentRemoteDescription
4.1.16. pendingRemoteDescription
4.1.17. canTrickleIceCandidates
4.1.18. setConfiguration
4.1.19. addIceCandidate
4.1.20. onicecandidate Event
4.2. RtpTransceiver
4.2.1. stop
4.2.2. stopped
4.2.3. setDirection
4.2.4. direction
4.2.5. currentDirection
4.2.6. setCodecPreferences
5. SDP Interaction Procedures
5.1. Requirements Overview
5.1.1. Usage Requirements
5.1.2. Profile Names and Interoperability
5.2. Constructing an Offer
5.2.1. Initial Offers
5.2.2. Subsequent Offers
5.2.3. Options Handling
5.2.3.1. IceRestart
5.2.3.2. VoiceActivityDetection
5.3. Generating an Answer
5.3.1. Initial Answers
5.3.2. Subsequent Answers
5.3.3. Options Handling
5.3.3.1. VoiceActivityDetection
5.4. Modifying an Offer or Answer
5.5. Processing a Local Description
5.6. Processing a Remote Description
5.7. Processing a Rollback
5.8. Parsing a Session Description
5.8.1. Session-Level Parsing
5.8.2. Media Section Parsing
5.8.3. Semantics Verification
5.9. Applying a Local Description
5.10. Applying a Remote Description
5.11. Applying an Answer
6. Processing RTP/RTCP
7. Examples
7.1. Simple Example
7.2. Detailed Example
7.3. Early Transport Warmup Example
8. Security Considerations
9. IANA Considerations
10. References
10.1. Normative References
10.2. Informative References
Appendix A. SDP ABNF Syntax
Acknowledgements
Authors' Addresses1. Introduction 🎀
This doc describes how the W3C Web Real-Time Communication (WebRTC) RTCPeerConnection interface [W3C.webrtc] gets used to set up, manage, and tear down a multimedia session ✨
1.1. General Design of JSEP 🦋
WebRTC call setup was built to focus on the media plane and leave signaling behavior up to the app as much as possible 💅 The reason is that different apps might want different protocols, like the existing SIP call signaling protocol, or something custom made for one app, maybe for a brand-new use case 🌈 In this approach, the key info that has to be swapped is the multimedia session description. It describes the transport and media configuration needed to set up the media plane.
So with all that in mind, this doc describes the JavaScript Session Establishment Protocol (JSEP), which lets JavaScript fully control the signaling state machine 🧚 JSEP assumes a JavaScript app running inside a runtime that has WebRTC APIs (the "JSEP implementation") 🍓 The JSEP implementation is almost totally separate from the core signaling flow. Instead, the JavaScript handles that flow using two interfaces: (1) passing in local and remote session descriptions and (2) working with the Interactive Connectivity Establishment (ICE) state machine [RFC8445]. The JSEP implementation and the JavaScript app together are called a "JSEP endpoint" throughout this doc 💖
This doc talks about JSEP as if it always happens between two JSEP endpoints. But real life is often a JSEP endpoint talking to some kind of server, like a gateway or a Multipoint Control Unit (MCU) 👑 The JSEP endpoint doesn't see that difference at all. It just follows whatever the API tells it to do 🫶
JSEP's session description handling is simple and straightforward ✨ Whenever an offer/answer exchange is needed, the side starting things calls the createOffer API to make an offer. The app then uses that offer to set up its local config through the setLocalDescription API. Then the offer gets sent to the other side over whatever signaling mechanism the app likes (for example, WebSockets). When the remote party gets that offer, it installs it with the setRemoteDescription API 🔥
To finish the offer/answer exchange, the remote party uses the createAnswer API to generate a proper answer, applies it with the setLocalDescription API, and sends the answer back to the initiator over the signaling channel. When the initiator gets that answer, it installs it with the setRemoteDescription API, and the initial setup is done 🎉 This whole process can repeat for more offer/answer exchanges 🔁
About ICE [RFC8445]: JSEP decouples the ICE state machine from the overall signaling state machine, and that's a big deal 😭 The ICE state machine has to stay inside the JSEP implementation because only the implementation knows about candidates and other transport info. Keeping them separate gives extra flexibility for protocols that split session descriptions from transport. For example, in traditional SIP, each offer or answer is self-contained and includes both the session descriptions and the transport info. But [RFC8840] lets SIP work with Trickle ICE [RFC8838], where the session description can be sent right away and the transport info can be sent once it's available. Sending transport info separately can make ICE and DTLS startup faster, since ICE checks can begin as soon as any transport info shows up instead of waiting for all of it 🚀 JSEP's separation of the ICE and signaling state machines means it can handle either model, which is so iconic 👑
Even though JSEP abstracts away signaling, the app still has to know about the signaling process 🙈 The app doesn't need to understand what's inside the session descriptions to set up a call. But it must call the right APIs at the right times, convert the session descriptions and ICE info into the defined messages of its chosen signaling protocol, and do the reverse conversion on the messages it gets from the other side 💀
One way to make life easier for the app is a JavaScript library that hides all this complexity from the developer 🧁 That library would implement a given signaling protocol along with its state machine and serialization code, and give the app developer a higher-level, call-oriented interface. For example, libraries exist that implement the SIP [RFC3261] and Extensible Messaging and Presence Protocol (XMPP) [RFC6120] signaling protocols on top of the JSEP API. So JSEP gives experienced developers more control without forcing extra complexity onto novice developers 🌸
Part 1 of 23 is done, bestie 🧚 more parts are coming, so stay tuned 💜
1.2. Other Approaches Considered
okay so before JSEP won, the team looked at a bunch of other options 🧐✨ bestie, you need to hear these
Option one: a lightweight signaling protocol 📮
Instead of giving session descriptions to the API, the API would produce and consume messages from this protocol. It's more high-level, which sounds iconic, BUT it puts way more control of signaling inside the JSEP implementation. That means the implementation would have to understand and handle stuff like signaling glare (see [RFC3264], Section 4). Too much drama 💀
Option two: splitting up the media control objects 🎛️
This one was about decoupling the management of the media control objects from session descriptions, so the APIs would control each component directly. It was turned down because making the app programmer deal with that level of complexity wouldn't be worth it. First, even a simple example would need a huge pile of code just to orchestrate all the interactions 😩. Second, you'd end up with a giant API surface that everyone would need to agree on and document. Plus, these API points could be called in any order, which makes for a messier set of interactions with the media subsystem than the JSEP approach, where session descriptions get evaluated and applied in a set way. Not it. 🙅♀️
Option three: keep session descriptions, move the offer/answer generation out 🧠
Here the basic session-description-oriented API stays, but the mechanism for generating offers and answers moves out of the JSEP implementation. Instead of createOffer/createAnswer methods, you'd get a getCapabilities API that gives the application the info it needs to generate its own session descriptions. The catch? The app has to do way more work. It needs to know how to build session descriptions from capabilities, and especially how to generate the correct answer from an arbitrary offer and the supported capabilities. Sure, a library like the one mentioned above could cover that, but then basically everyone is forced to use that library, even for a simple example. Providing createOffer/createAnswer avoids this problem, and honestly that's the vibe JSEP went with 💖
1.3. Changes from RFC 8829
okay this part is kinda a glow-up story 🦋 so when [RFC8829] came out, people noticed inconsistencies around BUNDLE [RFC8843] operation. This showed up in both the specification text and how implementations behaved. The spec side got fixed with an update to BUNDLE (see [RFC9143]) 🎀
The implementation side is the tea ☕. Some implementations handled the "max-bundle" bundle policy from [RFC8829] by assuming bundling had already been negotiated, instead of marking m= sections as bundle-only like the BUNDLE spec says. So to avoid surprising apps that were already relying on that pre-standard behavior, they decided to deprecate "max-bundle" and introduce an identically defined "must-bundle" policy. When you select it, you get the behavior that [RFC8829] originally specified. Bestie, that's a whole plot twist 😭🌈
2. Terminology ✨
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. 📖💜
3. Semantics and Syntax 🧩
3.1. Signaling Model
okay this is the core of it 🫶 JSEP does not specify a particular signaling model or state machine. The only thing it really needs is for both sides to exchange session descriptions the way [RFC3264] describes (offer/answer), so both sides of the session know how to conduct the session. JSEP gives you mechanisms to create offers and answers, and to apply them to a session. BUT the JSEP implementation is totally decoupled from how those offers and answers actually get communicated to the remote side. That includes addressing, retransmission, forking, and glare handling. All of that is left entirely up to the application, which has complete control over which offers and answers get handed to the implementation, and when. Main character energy for the app 👑
+-----------+ +-----------+
| Web App |<--- App-Specific Signaling -->| Web App |
+-----------+ +-----------+
^ ^
| SDP | SDP
V V
+-----------+ +-----------+
| JSEP |<----------- Media ------------>| JSEP |
| Impl. | | Impl. |
+-----------+ +-----------+
Figure 1: JSEP Signaling Model3.2. Session Descriptions and State Machine
okay so the JSEP implementation needs specific parameters 🎀 to know what to send to the other side AND how to handle the media coming in. those parameters come from swapping session descriptions in offers and answers, and there are some details in that process the JSEP APIs have to handle. it's giving detail-oriented 💅
whether a session description is about the local side or the remote side totally changes what it means ✨ like, the list of codecs you send to the remote party says what you're willing to receive. when that gets intersected with the codecs the remote side supports, it decides what the remote side should send 🦋 but ngl not every parameter plays by that rule. some are declarative, so the remote side has to either accept them or reject them altogether 💀 the TLS fingerprints [RFC8122] used with DTLS [RFC6347] [RFC9147] are a great example. they're calculated from the local certificate(s) offered, so they're not up for negotiation at all 🔒
also, different RFCs put different conditions on offers vs answers 🌸 for example, an offer can propose an arbitrary number of "m=" sections (media descriptions, see [RFC4566], Section 5.14), but an answer has to contain the exact same number as the offer. no wiggle room there, bestie 😭
and here's the lowkey tricky part 👀 the exact media parameters aren't known until an offer and an answer have been exchanged. but the offerer might get ICE checks, and maybe even media (like in a re-offer after a connection is already up), before it gets an answer. so to process that incoming media properly, the offerer's media handler needs to know the details of the offer before the answer shows up 🧚
so, to handle session descriptions the right way, the JSEP implementation needs:
- To know if a session description pertains to the local or remote side.
- To know if a session description is an offer or an answer.
- To allow the offer to be specified independently of the answer.
JSEP handles all of this by adding both setLocalDescription and setRemoteDescription methods, and having session description objects contain a type field that says what kind of session description is being supplied 💖 this covers all three needs for both players. the offerer first calls setLocalDescription(sdp [offer]) and later calls setRemoteDescription(sdp [answer]). the answerer first calls setRemoteDescription(sdp [offer]) and later calls setLocalDescription(sdp [answer]). iconic symmetry 🫶
during the offer/answer exchange, the outstanding offer is considered "pending" at both the offerer and the answerer, since it might get accepted or rejected 🍓 if this is a re-offer, each side also has "current" local and remote descriptions, which reflect the result of the last offer/answer exchange. Sections 4.1.14, 4.1.16, 4.1.13, and 4.1.15 give way more detail on pending and current descriptions, so go check those if you're curious 🌈
JSEP also lets the application treat an answer as provisional ✨ provisional answers are how an answerer sends initial session parameters back to the offerer so the session can start, while a final answer can still be specified later. the concept of a final answer matters a lot to the offer/answer model. when one arrives, any extra resources the caller allocated can be released, because now the exact session configuration is known 👑 those "resources" can include things like extra ICE components, Traversal Using Relays around NAT (TURN) candidates, or video decoders. provisional answers, on the other hand, don't deallocate anything. so multiple dissimilar provisional answers, with their own codec choices, transport parameters, etc., can be received and applied during call setup 🎀 and the final answer itself might be different from any provisional answers received. plot twist, periodt 💅
at the signaling level, [RFC3264] says only one offer can be outstanding for a given session. but at the JSEP level, a new offer can be generated at any point 🦋 for example, with SIP for signaling, if one offer is sent and then canceled with a SIP CANCEL, another offer can be generated even though no answer came back for the first one 💀 to support this, the JSEP media layer can provide an offer via the createOffer method whenever the JavaScript application needs one for signaling. the answerer can send back zero or more provisional answers, and then finally end the offer/answer exchange by sending a final answer 🌸 here's the state machine for all of this:
setRemote(OFFER) setLocal(PRANSWER)
/-----\ /-----\
| | | |
v | v |
+---------------+ | +---------------+ |
| |----/ | |----/
| have- | setLocal(PRANSWER) | have- |
| remote-offer |------------------- >| local-pranswer|
| | | |
| | | |
+---------------+ +---------------+
^ | |
| | setLocal(ANSWER) |
setRemote(OFFER) | |
| V setLocal(ANSWER) |
+---------------+ |
| | |
| |<---------------------------+
| stable |
| |<---------------------------+
| | |
+---------------+ setRemote(ANSWER) |
^ | |
| | setLocal(OFFER) |
setRemote(ANSWER) | |
| V |
+---------------+ +---------------+
| | | |
| have- | setRemote(PRANSWER) |have- |
| local-offer |------------------- >|remote-pranswer|
| | | |
| |----\ | |----\
+---------------+ | +---------------+ |
^ | ^ |
| | | |
\-----/ \-----/
setLocal(OFFER) setRemote(PRANSWER)
Figure 2: JSEP State Machineapart from these state transitions, there's no other difference in how provisional ("pranswer") and final ("answer") answers are handled 🧚
3.3. Session Description Format
JSEP's session descriptions use Session Description Protocol (SDP) syntax for their internal representation 🌷 this format isn't the most fun to manipulate from JavaScript, but it's widely accepted and gets new features all the time. any alternate encoding would have to keep up with SDP's changes, at least until it eclipsed SDP in popularity. that's a lot of pressure, honestly 😭
still, for future flexibility, the SDP syntax is wrapped in a SessionDescription object. it can be built from SDP and serialized back out to SDP 💜 if future specs agree on a JSON format for session descriptions, this object could be extended to generate and consume that JSON. future-proof era 🔮
as detailed below, most applications should be able to treat the SessionDescriptions produced and consumed by these API calls as opaque blobs ✨ meaning the application won't need to parse or understand them. no thank you to parsing SDP by hand, we love a clean abstraction 💅
3.4. Session Description Control
so the application can control common session parameters, JSEP gives it control surfaces that tell the JSEP implementation how to generate session descriptions 🎀 in most cases, this means applications don't need to modify session descriptions after they're created. chef's kiss 👩🍳
changes to these objects change the session descriptions generated by later createOffer/createAnswer calls 🦋
3.4.1. RtpTransceivers
RtpTransceivers let the application control the RTP media associated with one "m=" section 🌈 each RtpTransceiver has an RtpSender and an RtpReceiver, which the application can use to control sending and receiving RTP media. the application may also modify the RtpTransceiver directly, for instance by stopping it 🛑
RtpTransceivers generally have a 1:1 mapping with "m=" sections. but there can be more RtpTransceivers than "m=" sections when they've been created but not yet associated with an "m=" section, or when they've been stopped and disassociated from "m=" sections 🙈 an RtpTransceiver counts as associated with an "m=" section if its mid property is non-null, meaning it's set to a valid Media Identification (MID) value. otherwise, it's disassociated. the associated "m=" section is determined using a mapping between transceivers and "m=" section indices, which gets formed when creating an offer or applying a remote offer 🧩
an RtpTransceiver is never associated with more than one "m=" section, and once a session description is applied, an "m=" section is always associated with exactly one RtpTransceiver 💖 the exception: in certain cases where an "m=" section has been rejected (more on that in Section 5.2.2 below), that "m=" section gets "recycled" and associated with a new RtpTransceiver with a new MID value ♻️
RtpTransceivers can be created explicitly by the application, or implicitly by calling setRemoteDescription with an offer that adds new "m=" sections 🌸
3.4.2. RtpSenders
RtpSenders let the application control how RTP media is sent 📤 an RtpSender is conceptually responsible for the outgoing RTP stream(s) described by an "m=" section. that includes encoding the attached MediaStreamTrack, sending RTP media packets, and generating/processing the RTP Control Protocol (RTCP) for the outgoing RTP streams(s) 🎬
3.4.3. RtpReceivers
RtpReceivers let the application inspect how RTP media is received 📥 an RtpReceiver is conceptually responsible for the incoming RTP stream(s) described by an "m=" section. that includes processing received RTP media packets, decoding the incoming stream(s) to produce a remote MediaStreamTrack, and generating/processing RTCP for the incoming RTP stream(s) 🍿
3.5. ICE
3.5.1. ICE Gathering Overview
okay so JSEP only gathers ICE candidates when the app actually needs them 💅 That collecting process is called a gathering phase ✨, and it kicks off when either a brand new or recycled m= section gets added to the local session description, or when the description has new ICE credentials in it (that's an ICE restart 🔄). The app can ask for new credentials on purpose, or the JSEP implementation can add them on its own when the ICE config changes.
When the ICE config changes in a way that needs a fresh gathering phase, a needs-ice-restart bit gets flipped on 🚩. While that bit is set, calls to the createOffer API make brand new ICE credentials. The bit gets cleared when setLocalDescription is called with new ICE credentials from either an offer or an answer, so it doesn't matter whether the restart was started locally or by the other side 🦋
When a new gathering phase starts, the ICE agent lets the app know that gathering is happening through a state change event 📣 Then, every time a new candidate shows up, the agent hands it over via an onicecandidate event, and those candidates also get slotted into the current and/or pending local session description automatically 🎀 Once everything has been gathered, one last onicecandidate event gets dispatched to say "we're done" 🌸
Heads up though: gathering phases only collect candidates for m= sections that are new, recycled, or restarting. Everything else keeps using the candidates it already has 🧚 Also, if an m= section is bundled (either through a successful bundle negotiation or by being marked as bundle-only), candidates only get gathered and exchanged for that section if and only if its MID item is a BUNDLE-tag, as described in [RFC9143] 🫶
3.5.2. ICE Candidate Trickling
candidate trickling is when a caller sends candidates to the callee bit by bit, after the initial offer has already gone out 🦋 The semantics of "Trickle ICE" are defined in [RFC8838], so go check that if you want the full tea ☕ The best part is that the callee can start doing things with the call and setting up the ICE (and maybe DTLS) connections right away, without waiting for the caller to gather every possible candidate first. That means faster media setup in cases where gathering didn't happen before the call started 🚀
JSEP supports optional candidate trickling through the APIs we already covered, which give you control and feedback over the ICE candidate gathering process 💖 Apps that support trickling can send the initial offer immediately and then send each candidate as they get notified about it ✨ Apps that don't support this can just wait for the signal that gathering is complete, and then create and send their offer with all the candidates in it at that point 🌈
When trickled candidates arrive, the receiving app passes them along to its ICE agent. That's the trigger for the agent to start using those new remote candidates for connectivity checks 💅
3.5.2.1. ICE Candidate Format
In JSEP, ICE candidates get wrapped up in an IceCandidate object, and just like session descriptions, the internal representation uses SDP syntax 📝
The candidate details go in an IceCandidate field, using the same SDP syntax as the "candidate-attribute" field defined in [RFC8839], Section 5.1. Quick note: this field doesn't include the a= prefix, as you can see in this example 👀
candidate:1 1 UDP 1694498815 192.0.2.33 10000 typ hostThe IceCandidate object also has a field for which ICE username fragment (ufrag) it belongs to, as defined in [RFC8839], Section 5.4. This is how the object figures out which session description (and therefore which gathering phase) it goes with, which clears up any confusion during ICE restarts 🫧 If this field is missing from a received IceCandidate (maybe because you're talking to a non-JSEP endpoint), the most recently received session description is assumed 🙈
There are also fields that say which m= section the candidate is for, and you can identify that section in one of two ways: by an m= section index or by a MID 🔍 The m= section index is zero-based, so index N points to the N+1th m= section in the session description this IceCandidate refers to. The MID is a "media stream identification" value, as defined in [RFC5888], Section 4. It's the more robust option, since it uses the MID of the associated RtpTransceiver object to find the m= section in the session description. (That MID might have been generated locally by the answerer when it's talking to a non-JSEP endpoint that doesn't support the MID attribute, as discussed in Section 5.10 below 🧩) If the MID field is present in a received IceCandidate, it MUST be used for identification. Otherwise, the m= section index gets used instead 💜
Apart from the m= section index, every IceCandidate field is optional, and implementations MUST NOT reject a candidate just because an optional field is missing 🎀
3.5.3. ICE Candidate Policy
Normally, when the JSEP implementation gathers ICE candidates, it grabs every form of initial candidate: host, server-reflexive, and relay 🌍 But sometimes apps want more precise control over gathering because of privacy or other concerns 🙈 For example, someone might only want relay candidates so they leak as little location info as possible (just keep in mind that this choice comes with operational costs 💸). So JSEP lets the app restrict which ICE candidates get used in a session 🔒 This filtering sits on top of any restrictions the implementation enforces about which IP addresses the app is allowed to use, as discussed in [RFC8828] 🛡️
There's also the case where the app wants to switch up which candidate types are used while the session is active 🔄 The classic example: a callee might start out with only relay candidates so they don't leak their location to some random caller, and then switch to all candidates (for lower operational cost) once the user says they want to take the call 📞 For this, the JSEP implementation MUST allow the candidate policy to be changed mid-session, subject to the interactions with local policy we just talked about 👑
To run the ICE candidate policy, the JSEP implementation checks the current setting at the start of each gathering phase 🧐 During that gathering phase, the implementation MUST NOT expose candidates that the current policy disallows to the application, use them as the source of connectivity checks, or leak them indirectly through other fields, such as the raddr/rport attributes on other ICE candidates. If the application wants a different policy later, it can apply it by starting a new gathering phase through an ICE restart 💅
3.5.4. ICE Candidate Pool
JSEP apps usually get ICE gathering started by handing info to setLocalDescription, since the local description says how many ICE components will be needed and which ones need candidates ✨ But if the app already knows how many ICE components it'll use ahead of time, it can ask the implementation to pre-gather a pool of potential ICE candidates to help make media setup super speedy 🚀
When setLocalDescription is eventually called and the JSEP implementation gets ready to gather the needed ICE candidates, it SHOULD start by checking whether any candidates are already sitting in the pool 👀 If there are candidates in the pool, they SHOULD be handed to the application right away through the ICE candidate event 🎀 If the pool runs dry (maybe because more ICE components got used than expected, or because the pool didn't have enough time to gather candidates), the remaining candidates get gathered as usual 🫧 This only happens for the first offer/answer exchange. After that, the candidate pool is emptied and never used again 🧹
One place this is super useful is an app that expects an incoming call at some point and wants to cut down the time it takes to set up connectivity, so initial media doesn't get clipped ✂️ By pre-gathering candidates into the pool, the app can exchange them and start sending connectivity checks from those candidates almost the moment a call comes in 📞 Just know that holding on to pre-gathered candidates means they stay alive for as long as they might be needed, and that means the app is using up resources on the STUN/TURN servers it relies on 🥺 ("STUN" stands for "Session Traversal Utilities for NAT".)
3.5.5. ICE Versions
Okay, so this spec formally relies on [RFC8445], but when it was published, most WebRTC implementations were still using the version of ICE described in [RFC5245] 🕰️ The ice2 attribute defined in [RFC8445] can be used to detect which ICE version a remote endpoint is running, and it helps provide a smooth transition from the older spec to the newer one 🌉 Implementations MUST be able to accept remote descriptions that don't have the ice2 attribute 💖
3.6. Video Size Negotiation
Video size negotiation is how a receiver uses the a=imageattr SDP attribute [RFC6236] to say which video frame sizes it can receive 📐 A receiver might have hard limits on what its video decoder can handle, or it might have a max set by policy 🎬 By putting these limits in an a=imageattr attribute, JSEP endpoints can try to make sure the remote sender transmits video at a resolution that works for them 🌟 But when you're talking to a non-JSEP endpoint that doesn't understand this attribute, the signaled limits might get exceeded, and the JSEP implementation MUST handle that gracefully, e.g., by discarding the video 😭
Some codecs support samples with aspect ratios other than 1.0 (aka non-square pixels) 🧩 JSEP implementations won't transmit non-square pixels, but they SHOULD receive and render that video with the correct aspect ratio 🦋 Sample aspect ratio has zero impact on the size negotiation I just described, though. All dimensions get measured in pixels, whether they're square or not 📏
3.6.1. Creating an imageattr Attribute ✨
okay so first, the receiver figures out its own limits 🧚. It combines any local limits it knows about (like what its hardware decoder can actually handle, or local policy) to work out the absolute smallest and biggest sizes it can receive. If there are no known local limits at all, the a=imageattr attribute SHOULD be omitted.
If those local limits rule out receiving any video at all (the degenerate case where there are no permitted resolutions), then the a=imageattr attribute MUST be omitted, and the "m=" section MUST be marked as "sendonly"/"inactive", as appropriate 🙈 yikes, but that's the situation.
Otherwise, an a=imageattr attribute gets created with a recv direction. The resolution space from the intersection above gives it its minimum and maximum x= and y= values 💖
The rules here express a single set of preferences, so the a=imageattr q= value isn't really important. It SHOULD be set to 1.0 💅
The a=imageattr field is payload type specific. When all the video codecs supported have the same capabilities, using a single attribute with the wildcard payload type (*) is RECOMMENDED, and honestly it's the cleanest vibe 🌸 However, when the supported video codecs have different limitations, specific a=imageattr attributes MUST be inserted for each payload type.
As an example, imagine a system with a multiformat video decoder that can decode any resolution from 48x48 up to 720p. In this case, the implementation would generate this attribute:
a=imageattr:* recv [x=[48:1280],y=[48:720],q=1.0]This declaration says the receiver can decode any image resolution from 48x48 up to 1280x720 pixels. Iconic range, no notes 👑
3.6.2. Interpreting imageattr Attributes
[RFC6236] defines a=imageattr as an advisory field. Translation: it does not absolutely constrain the video formats the sender can use, it just gives an indication of the preferred values 🦋
This specification is way more specific about it, though 😤 When a MediaStreamTrack producing video at a certain resolution (the "track resolution") is attached to an RtpSender that is encoding the track video at the same or lower resolution(s) (the "encoder resolutions"), and a remote description is applied that references the sender and contains valid a=imageattr recv attributes, it MUST follow the rules below so the sender never transmits a resolution that would exceed the size criteria in the attributes. These rules MUST be followed for as long as the attributes stay present in the remote description. That includes cases where the track changes its resolution or gets replaced with a different track, so no escape hatch there 🔒
Depending on how the RtpSender is configured, it might be producing a single encoding at one resolution or, if simulcast (Section 3.7) has been negotiated, multiple encodings, each at their own specific resolution. On top of that, each encoding may have the flexibility to reduce its resolution when needed, or it may be locked to a specific output resolution 🎞️
For each encoding being produced by the RtpSender, the set of a=imageattr recv attributes in the corresponding "m=" section of the remote description gets processed to decide what should be transmitted. Only attributes that reference the media format selected for the encoding are considered, and each one is evaluated individually, starting with the attribute with the highest q= value. If multiple attributes share the same q= value, they're evaluated in the order they appear in their containing "m=" section. Also, while JSEP endpoints will include at most one a=imageattr recv attribute per media format, JSEP endpoints may receive session descriptions from non-JSEP endpoints whose "m=" sections contain multiple such attributes, so be ready for that chaos 🌈
For each a=imageattr recv attribute, the following rules apply. If processing succeeds, the encoding is transmitted accordingly, and no further attributes get considered for that encoding. Otherwise, the next attribute is evaluated, in the aforementioned order. If none of the supplied attributes can be processed successfully, the encoding MUST NOT be transmitted, and an error SHOULD be raised to the application 🚨
- The limits from the attribute are compared to the encoder resolution. Only the specific limits mentioned below are considered; any other values, such as picture aspect ratio, MUST be ignored. When considering a MediaStreamTrack producing rotated video, the unrotated resolution MUST be used for the checks. This is required regardless of whether the receiver supports performing receive-side rotation (for example through Coordination of Video Orientation (CVO) [TS26.114]), because it significantly simplifies the matching logic 🧠
- If the attribute includes a
sar=(sample aspect ratio) value set to something other than1.0, which means the receiver wants to receive non-square pixels, that cannot be satisfied and the attribute MUST NOT be used 🙅♀️ - If the encoder resolution exceeds the maximum size permitted by the attribute and the encoder is allowed to adjust its resolution, the encoder SHOULD apply downscaling to satisfy the limits. Downscaling MUST NOT change the picture aspect ratio of the encoding, ignoring any trivial differences due to rounding. For example, if the encoder resolution is 1280x720 and the attribute specifies a maximum of 640x480, the expected output resolution would be 640x360. If downscaling cannot be applied, the attribute MUST NOT be used 📐
- If the encoder resolution is less than the minimum size permitted by the attribute, the attribute MUST NOT be used, and the encoder MUST NOT apply upscaling. JSEP implementations SHOULD avoid this situation by allowing receipt of arbitrarily small resolutions, perhaps via fallback to a software decoder 🛠️
- If the encoder resolution is within the maximum and minimum sizes, no action is needed. Easy win 🥰
3.7. Simulcast 🎀
JSEP supports simulcast transmission of a MediaStreamTrack, where multiple encodings of the source media can be sent within the context of a single "m=" section. The current JSEP API is designed so applications can send simulcasted media but only receive a single encoding. This is giving multi-user scenarios, where each sending client sends multiple encodings to a server, and the server then picks the right encoding for each receiving client to forward 🧚♀️
Applications request simulcast support by configuring multiple encodings on an RtpSender. When an offer or answer is generated, these encodings get indicated through SDP markings on the corresponding "m=" section, as described below. Receivers that understand simulcast and are willing to receive it also include SDP markings to show their support, and JSEP endpoints use those markings to decide whether simulcast is permitted for a given RtpSender. If simulcast support isn't negotiated, the RtpSender will only use the first configured encoding 💅
Note that the exact simulcast parameters are up to the sending application. The SDP markings exist so the remote side can receive and demux multiple simulcast encodings, but the specific resolutions and bitrates for each encoding are purely a send-side decision in JSEP 💖
JSEP doesn't currently provide a mechanism to configure receipt of simulcast. So if the remote endpoint offers simulcast, the answer generated by a JSEP endpoint won't indicate support for receiving it, and the remote endpoint will only send a single encoding per "m=" section 😭
Also, JSEP doesn't provide a way to handle an incoming offer that requests simulcast from the JSEP endpoint. That means setting up simulcast when the JSEP endpoint receives the initial offer requires out-of-band signaling or SDP inspection. However, if the JSEP endpoint sets up simulcast in its initial offer, any established simulcast streams keep working when an incoming re-offer arrives. Future versions of this specification may add APIs for the incoming initial offer scenario 👀
When using JSEP to transmit multiple encodings from an RtpSender, the techniques from [RFC8853] and [RFC8851] are used. Specifically, when multiple encodings have been configured for an RtpSender, the "m=" section for that RtpSender includes an a=simulcast attribute, as defined in [RFC8853], Section 5.1, with a send simulcast stream description that lists each desired encoding, and no recv simulcast stream description. The "m=" section also includes an a=rid attribute for each encoding, as specified in [RFC8851], Section 4. Restriction Identifiers (RIDs, also called rid-ids or RtpStreamIds) are what let the individual encodings be told apart even though they're all part of the same "m=" section, and that's the whole trick 🦋
3.8. Interactions with Forking 🍓
Some call signaling systems allow forking, where an SDP offer may be provided to more than one device. For example, SIP [RFC3261] defines both a "parallel search" and a "sequential search". These are mostly signaling-level issues that are outside the scope of JSEP, but they do have some impact on the configuration of the media plane that's relevant 📞 When forking happens at the signaling layer, the JavaScript application running the signaling has to decide what media should be sent or received at any point in time, as well as which remote endpoint it should talk to. JSEP is what makes sure the media engine can make RTP and media perform as the application requires 🫶 The basic operations the application can tell the media engine to do are:
- Start exchanging media with a given remote peer, but keep all the resources reserved in the offer.
- Start exchanging media with a given remote peer, and free any resources in the offer that aren't being used.
3.8.1. Sequential Forking 🧚
Sequential forking means a call gets dispatched to multiple remote callees, where each callee can accept the call, but only one active session ever exists at a time. No mixing of received media happens, so no supergroup remixes 🙅♀️
JSEP handles sequential forking really well, letting the application easily control the policy for picking the desired remote endpoint. When an answer arrives from one of the callees, the application can choose to apply it as either (1) a provisional answer, which leaves the possibility of using a different answer in the future, or (2) a final answer, which ends the setup flow 💜
In a "first-one-wins" situation, the first answer gets applied as a final answer, and the application rejects any answers after that. In SIP parlance, this would be ACK + BYE 👑
In a "last-one-wins" situation, every answer gets applied as a provisional answer, and any previous call leg gets terminated. At some point the application ends the setup process, maybe with a timer. At that point, the application could reapply the pending remote description as a final answer ⭐
3.8.2. Parallel Forking ✨
okay so parallel forking is when one call gets sent out to multiple remote callees at the same time 📞📞📞, and each callee can pick up, which means you can end up with several active signaling sessions at once 😭🫶. If more than one callee starts sending media at the same time, the ways to handle that are described in [RFC3960], Section 3.1. Like, most SIP devices today only swap media with ONE device at a time and don't try to mix multiple early media audio sources together 🙅♀️, because that gets messy fast. Imagine blending a European ringback tone with a North American ringback tone 💀. The result wouldn't sound like either one, and the user would be so confused. If the signaling app only wants to exchange media with one remote endpoint at a time, then from the media engine's point of view, it's literally the exact same as the sequential forking case 🦋.
Now the parallel forking case where the JavaScript app wants to exchange media with several peers at once is a little more involved 🧚, but the app can follow the strategy that [RFC3960] describes, using UPDATE 💅. The UPDATE approach lets the signaling set up a separate media flow for every peer the app wants to talk to. In JSEP, the offer used in the UPDATE is made by creating a brand new PeerConnection (see Section 4.1) and making sure the same local media streams have been added to that new PeerConnection 🎀. Then the new PeerConnection object produces an SDP offer, and the signaling can use that offer to perform the UPDATE strategy discussed in [RFC3960] 🌸.
Because the media streams get shared, the app ends up with N parallel PeerConnection sessions, and each one has its own local and remote description plus its own local and remote addresses 💜. You can manage the media flowing from these sessions with setDirection (see Section 4.2.3), or the app can choose to play out the media from all sessions mixed together 🌈. And if the app only wants to keep one session, it can simply terminate the sessions it no longer needs 👑✨.
4. Interface 🎀
This section details the basic operations that must be present to implement JSEP functionality. The actual API exposed in the W3C API may have somewhat different syntax, but it should map easily to these concepts 🦋.
4.1. PeerConnection 💖
4.1.1. Constructor 🌸
The PeerConnection constructor lets the app specify global parameters for the media session, like the STUN/TURN servers and credentials to use when gathering candidates 🧚. It also sets the initial ICE candidate policy and pool size, plus the bundle policy to use ✨.
If an ICE candidate policy is specified, it works as described in Section 3.5.3, so the JSEP implementation only surfaces the permitted candidates (including any implementation-internal filtering) to the app, and only uses those candidates for connectivity checks 🔥. The available policies are:
all: All candidates permitted by implementation policy will be gathered and used.
relay: All candidates except relay candidates will be filtered out. This obfuscates the location information that might be ascertained by the remote peer from the received candidates. Depending on how the application deploys and chooses relay servers, this could obfuscate location to a metro or possibly even global level. 🙈
The default ICE candidate policy MUST be set to "all", because that's generally the desired policy and it also typically cuts down on application TURN server resource usage significantly 💅.
If a size is specified for the ICE candidate pool, that number is how many ICE components to pre-gather candidates for 🍓. Pre-gathering eats up STUN/TURN server resources for potentially long periods of time, so this MUST only happen upon application request 😭. That's why the default candidate pool size MUST be zero 🫶.
The app can specify its preferred policy for BUNDLE, the multiplexing mechanism defined in [RFC9143] 🦋. No matter the policy, the app will always try to negotiate bundle onto a single transport, and it will offer a single bundle group across all "m=" sections. Using that single transport depends on the answerer accepting bundle 💜. But by picking a policy from the list below, the app controls exactly how aggressively it bundles media streams together, which affects how it interoperates with a non-bundle-aware endpoint 👑. When negotiating with a non-bundle-aware endpoint, only the streams not marked as bundle-only will be established 🌈.
The available policies are:
balanced: The first "m=" section of each type (audio, video, or application) will contain transport parameters, which will allow an answerer to unbundle that section. The second and any subsequent "m=" sections of each type will be marked as bundle-only. The result is that if there are N distinct media types, then candidates will be gathered for N media streams. This policy balances the desire to multiplex with the need to ensure that basic audio and video can still be negotiated in legacy cases. When acting as answerer, if there is no bundle group in the offer, the implementation will reject all but the first "m=" section of each type. ⚖️
max-compat: All "m=" sections will contain transport parameters; none will be marked as bundle-only. This policy makes no assumptions about the remote endpoint and as such will allow all streams to be received by non-bundle-aware endpoints, but as a result requires separate candidates to be gathered for each media stream. 🧸
must-bundle: Only the first "m=" section will contain transport parameters; all streams other than the first will be marked as bundle-only. This policy presumes that the remote endpoint supports multiplexing and accordingly aims to minimize candidate gathering, at the cost of less compatibility with legacy endpoints. When acting as answerer, the implementation will reject any "m=" sections other than the first "m=" section, unless they are in the same bundle group as that "m=" section. 🚀
As it gives the best trade-off between performance and compatibility with legacy endpoints, the default bundle policy MUST be set to "balanced" 💖.
[RFC8829] defined a policy known as "max-bundle". It was defined identically to the "must-bundle" policy above, but some implementations followed an earlier, pre-standard definition instead (in which, for example, no "m=" sections were marked as bundle-only) 🙃. So "max-bundle" is considered deprecated, and implementations compliant with this specification SHOULD ignore attempts by the app to select this bundle policy (although some phase-out period may be necessary to avoid app breakage) 🥲.
The app can specify its preferred policy for RTP/RTCP multiplexing [RFC5761] using one of the following policies:
negotiate: The JSEP implementation will gather both RTP and RTCP candidates, but it also will offer "a=rtcp-mux", so it's compatible with either multiplexing or non-multiplexing endpoints 🦋.
require: The JSEP implementation will only gather RTP candidates and will insert an "a=rtcp-mux-only" indication into any new "m=" sections in offers it generates. This halves the number of candidates that the offerer needs to gather. Applying a description with an "m=" section that does not contain an "a=rtcp-mux" attribute will cause an error to be returned 🙅♀️.
The default multiplexing policy MUST be set to "require" ✨. Implementations MAY choose to reject attempts by the app to set the multiplexing policy to "negotiate" 💅.
4.1.2. addTrack 🎀
The addTrack method adds a MediaStreamTrack to the PeerConnection, using the MediaStream argument to associate the track with other tracks in the same MediaStream 🧚. That way they can be added to the same "LS" (Lip Synchronization) group when creating an offer or answer 🫶. Tracks in the same "LS" group are supposed to play back synchronized for proper lip sync, as described in [RFC5888], Section 7 👄. addTrack tries to minimize the number of transceivers. If the PeerConnection is in the "have-remote-offer" state, the track gets attached to the first compatible transceiver that was created by the most recent call to setRemoteDescription and doesn't have a local track yet 🌸. Otherwise, a new transceiver gets created, as described in Section 4.1.4 💜.
4.1.3. removeTrack 🌈
The removeTrack method removes a MediaStreamTrack from the PeerConnection, using the RtpSender argument to indicate which sender should have its track removed 👋. The sender's track is cleared, and the sender stops sending. Future calls to createOffer will mark the "m=" section associated with the sender as "recvonly" (if transceiver.direction is "sendrecv") or as "inactive" (if transceiver.direction is "sendonly") 🍓.
4.1.4. addTransceiver 🦋
The addTransceiver method adds a new RtpTransceiver to the PeerConnection 🧚. If a MediaStreamTrack argument is provided, the transceiver gets configured with that media type and the track gets attached to it. Otherwise, the app MUST explicitly specify the type 🙌. This mode is useful for creating "recvonly" transceivers, and also for creating transceivers that a track can be attached to at some later point ✨.
At the time of creation, the app can also specify a transceiver direction attribute, a set of MediaStreams that the transceiver is associated with (allowing "LS" group assignments), and a set of encodings for the media (used for simulcast as described in Section 3.7) 💖.
4.1.5. onaddtrack Event 💜
The onaddtrack event is dispatched to the app when a new remote track has been signaled as a result of a setRemoteDescription call 👀. The new track is supplied as a MediaStreamTrack object in the event, along with the MediaStream(s) the track is part of 🎁.
4.1.6. createDataChannel 🍓
The createDataChannel method creates a new data channel and attaches it to the PeerConnection 🫧. If no data channel currently exists for this PeerConnection, then a new offer/answer exchange is required 🔄. All data channels on a given PeerConnection share the same SCTP/DTLS association ("SCTP" stands for "Stream Control Transmission Protocol") and therefore the same "m=" section, so creating subsequent data channels doesn't affect the JSEP state at all 🌊.
The createDataChannel method also takes a number of arguments that the PeerConnection uses (e.g., maxPacketLifetime), but those aren't reflected in the SDP and don't affect the JSEP state 🧸.
4.1.7. ondatachannel Event 🌟
The ondatachannel event is dispatched to the app when a new data channel has been negotiated by the remote side, which can happen at any time after the underlying SCTP/DTLS association has been established 💌. The new data channel object is supplied in the event 🦋.
4.1.8. createOffer 🎀
Okay so createOffer is the method that whips up a blob of SDP containing an offer per [RFC3264] 💅 It gives you the full menu: the session configuration the app supports, descriptions of the media added to this PeerConnection, the codec, RTP, and RTCP options this implementation supports, and any candidates the ICE agent has already gathered ✨ You can also pass an options parameter for extra control over the offer, and that options parameter lets the app trigger an ICE restart to reestablish connectivity 🔥
In the initial offer, the SDP contains all the desired functionality for the session. Functionality that's supported but not desired by default may be omitted 🌸 For each SDP line, the generation follows the process defined for generating an initial offer in the specification that defines that line. The exact handling of initial offer generation is detailed in Section 5.2.1 below 🧚
In the event createOffer is called after the session is established, it generates an offer that modifies the current session based on whatever has changed, like adding or stopping RtpTransceivers, or requesting an ICE restart 🦋 For each existing stream, generating each SDP line MUST follow the process defined for generating an updated offer in the specification that defines that line. For each new stream, the generation of the SDP MUST follow the process of generating an initial offer, as mentioned above. The exact handling of subsequent offer generation is detailed in Section 5.2.2 below 💜
Session descriptions generated by createOffer MUST be immediately usable by setLocalDescription. If a system has limited resources (for example, a finite number of decoders), createOffer SHOULD return an offer that reflects the current state of the system, so that setLocalDescription will succeed when it tries to acquire those resources 🫶
Calling this method may do things such as generating new ICE credentials, but it does not change the PeerConnection state, trigger candidate gathering, or cause media to start or stop flowing. The offer is not applied, and does not become the pending local description, until setLocalDescription is called. So no sneaky changes, bestie 🙈
4.1.9. createAnswer 🌈
The createAnswer method generates a blob of SDP that contains an SDP answer per [RFC3264], with the supported configuration for the session that's compatible with the parameters supplied in the most recent call to setRemoteDescription. setRemoteDescription MUST have been called prior to calling createAnswer, so don't skip that step, periodt 😭 Like createOffer, the returned blob contains descriptions of the media added to this PeerConnection, the codec, RTP, and RTCP options supported by the application, and any candidates the ICE agent has gathered. An options parameter may be supplied to provide additional control over the generated answer ✨
As an answer, the generated SDP contains a specific configuration that says how the media plane should be established. For each SDP line, the generation of the SDP MUST follow the process defined for generating an answer in the specification that defines that line. The exact handling of answer generation is detailed in Section 5.3 below 🎀
Session descriptions generated by createAnswer MUST be immediately usable by setLocalDescription. Like createOffer, the returned description SHOULD reflect the current state of the system 🍓
Calling this method may do things such as generating new ICE credentials, but it does not change the PeerConnection state, trigger candidate gathering, or cause a media state change. The answer is not applied, and does not become the current local description, until setLocalDescription is called. Main character energy, but make it patient 👑
4.1.10. SessionDescriptionType 💖
Session description objects (RTCSessionDescription) may be of type "offer", "pranswer", "answer", or "rollback". These types tell you how the description parameter should be parsed and how the media state should change 🧚
"offer" means a description MUST be parsed as an offer, and it may include many possible media configurations. A description used as an "offer" may be applied any time the PeerConnection is in a "stable" state, or applied as an update to a previously supplied but unanswered "offer" 🦋
"pranswer" means a description MUST be parsed as an answer, but not a final answer. So it MUST NOT result in the freeing of allocated resources. It may result in the start of media transmission, if the answer does not specify an "inactive" media direction. A description used as a "pranswer" may be applied as a response to an "offer" or as an update to a previously sent "pranswer" 🌸
"answer" means a description MUST be parsed as an answer, the offer/answer exchange MUST be considered complete, and any resources (decoders, candidates) that are no longer needed SHOULD be released 💅 A description used as an "answer" may be applied as a response to an "offer" or as an update to a previously sent "pranswer" ✨
The only difference between a provisional and a final answer is that the final answer results in freeing any unused resources that were allocated because of the offer. That means the application can use some discretion about whether an answer should be applied as provisional or final, and can change the session description type as needed. For example, in a serial forking scenario, an application may receive multiple "final" answers, one from each remote endpoint. The application could choose to accept the initial answers as provisional answers, and only apply an answer as final when it receives one that meets its criteria (for example, a live user instead of voicemail) 🙈
"rollback" is a special session description type indicating that the state machine MUST be rolled back to the previous "stable" state, as described in Section 4.1.10.2. The contents MUST be empty 🔥
4.1.10.1. Use of Provisional Answers 🌈
Most applications won't need to create answers using the "pranswer" type. Sending an immediate response to an offer is good practice, since it warms up the session transport and prevents media clipping. But the preferred handling for a JSEP application is to create and send a "sendonly" final answer with a null MediaStreamTrack immediately after receiving the offer. That prevents media from being sent by the caller, and lets media be sent immediately upon answer by the callee 🫶 Later, when the callee actually accepts the call, the application can plug in the real MediaStreamTrack and create a new "sendrecv" offer to update the previous offer/answer pair and start bidirectional media flow. This could also be done with a "sendonly" "pranswer" followed by a "sendrecv" answer, but the initial pranswer leaves the offer/answer exchange open, which means the caller can't send an updated offer during that time 💀
As an example, picture a typical JSEP application that wants to set up audio and video as fast as possible. When the callee receives an offer with audio and video MediaStreamTracks, it sends an immediate answer accepting these tracks as "sendonly". That means the caller won't send the callee any media yet, and because the callee hasn't added its own MediaStreamTracks, the callee won't send any media either. Then it asks the user to accept the call and acquires the needed local tracks 🦋 Once the user accepts, the application plugs in the tracks it acquired. Since ICE handshaking and DTLS handshaking have likely completed by this point, they can start transmitting immediately. The application also sends a new offer to the remote side, indicating call acceptance and moving the audio and video to two-way media. A detailed example flow along these lines is shown in Section 7.3 🎀
Of course, some applications may not be able to do this double offer/answer exchange, particularly ones trying to gateway to legacy signaling protocols. In those cases, pranswer can still give the application a mechanism to warm up the transport 🌸
4.1.10.2. Rollback 🍓
Sometimes you want to "undo" a change made to setLocalDescription or setRemoteDescription. Picture this: a call is ongoing and one side wants to change some session parameters, so that side generates an updated offer and then calls setLocalDescription. But the remote side, either before or after setRemoteDescription, decides it doesn't want the new parameters and sends a reject message back to the offerer. Now the offerer, and possibly the answerer too, needs to return to a "stable" state and the previous local/remote description 😭 To support this, this specification introduces the concept of "rollback", which discards any proposed changes to the session and returns the state machine to the "stable" state. A rollback is performed by supplying a session description of type "rollback" with empty contents to either setLocalDescription or setRemoteDescription 💜
4.1.11. setLocalDescription 👑
The setLocalDescription method tells the PeerConnection to apply the supplied session description as its local configuration. The type field indicates whether the description should be processed as an offer, provisional answer, final answer, or rollback. Offers and answers are checked differently, using the various rules that exist for each SDP line ✨
This API changes the local media state. Among other things, it sets up local resources for receiving and decoding media. To successfully handle scenarios where the application wants to offer a change from one media format to a different, incompatible format, the PeerConnection MUST be able to simultaneously support use of both the current and pending local descriptions (for example, support the codecs that exist in either description). This dual processing begins when the PeerConnection enters the "have-local-offer" state, and it continues until setRemoteDescription is called with either (1) a final answer, at which point the PeerConnection can fully adopt the pending local description, or (2) a rollback, which results in a revert to the current local description 🌈
This API indirectly controls the candidate gathering process. When a local description is supplied and the number of transports currently in use doesn't match the number of transports needed by the local description, the PeerConnection creates transports as needed and begins gathering candidates for each transport, using ones from the candidate pool if available 🧚
If (1) setRemoteDescription was previously called with an offer, (2) setLocalDescription is called with an answer (provisional or final), (3) the media directions are compatible, and (4) media is available to send, this will result in the starting of media transmission 🦋
4.1.12. setRemoteDescription 📩
okay so setRemoteDescription tells the PeerConnection to apply the session description you hand it as the desired remote configuration 💅. Just like setLocalDescription, the type field on the description tells it how to process the thing 🎀.
this one changes your local media state. among other things, it sets up local resources for sending and encoding media 🎬.
if (1) setLocalDescription was called earlier with an offer, (2) setRemoteDescription gets an answer (provisional or final), (3) the media directions are compatible, and (4) there's media ready to send, then media transmission kicks off ✨. main character moment, fr 🔥.
4.1.13. currentLocalDescription 🧾
currentLocalDescription gives you back the current negotiated local description (aka the local description from the last successful offer/answer exchange) plus any local candidates the ICE agent has generated since the local description was set 🫶.
if an offer/answer exchange hasn't been completed yet, you get a null object back 🙈.
4.1.14. pendingLocalDescription ⏳
pendingLocalDescription returns a copy of the local description that's still in negotiation, so a local offer that was set without a matching remote answer yet 💜. it also includes any local candidates ICE has generated since that local description was set.
if the state of the PeerConnection is "stable" or "have-remote-offer", you get a null object back 🙃.
4.1.15. currentRemoteDescription 🌸
currentRemoteDescription returns a copy of the current negotiated remote description (the remote description from the last successful offer/answer exchange) 💖. on top of that, you also get any remote candidates supplied through processIceMessage since the remote description was set ✨.
same deal as before: if an offer/answer exchange hasn't been completed yet, it's a null object 🙈.
4.1.16. pendingRemoteDescription 🦋
pendingRemoteDescription gives you a copy of the remote description that's currently in negotiation, aka a remote offer set without a corresponding local answer 🧚. it also brings along any remote candidates supplied through processIceMessage since that remote description was set.
if the PeerConnection is in "stable" or "have-local-offer", it's a null object 💀.
4.1.17. canTrickleIceCandidates 🌈
the canTrickleIceCandidates property tells you whether the remote side can receive trickled candidates 🍓. it has three possible values:
null: No SDP has been received from the other side, so it is not known if it can handle trickle. This is the initial value beforesetRemoteDescriptionis called.true: SDP has been received from the other side indicating that it can support trickle.false: SDP has been received from the other side indicating that it cannot support trickle.
as described in Section 3.5.2, JSEP implementations always hand candidates to the application one at a time, which is what Trickle ICE needs 🫶. but applications can check canTrickleIceCandidates to see if their peer can actually do Trickle ICE, meaning it's safe to send an initial offer or answer and then send candidates later as they get gathered 👑. only true definitively confirms remote Trickle ICE support, so an app that compares against true will by default try half trickle on initial offers and full trickle on later interactions with a Trickle ICE-compatible agent 😭.
4.1.18. setConfiguration ⚙️
setConfiguration lets you change the PeerConnection's global configuration (the stuff originally set through the constructor parameters) during the session 🔧. when you call it matters, and the effects differ depending on which specific parameters you change:
- Changes to the STUN/TURN servers to use affect the next gathering phase. If an ICE gathering phase has already started or completed, the
'needs-ice-restart'bit mentioned in Section 3.5.1 will be set. This will cause the next call tocreateOfferto generate new ICE credentials, to force an ICE restart and kick off a new gathering phase, in which the new servers will be used. If the ICE candidate pool has a nonzero size and a local description has not yet been applied, any existing candidates will be discarded, and new candidates will be gathered from the new servers. - Any change to the ICE candidate policy affects the next gathering phase. If an ICE gathering phase has already started or completed, the
'needs-ice-restart'bit will be set. Either way, changes to the policy have no effect on the candidate pool, because pooled candidates are not made available to the application until a gathering phase occurs, so any necessary filtering can still be done on any pooled candidates. - The ICE candidate pool size MUST NOT be changed after applying a local description. If a local description has not yet been applied, any changes to the ICE candidate pool size take effect immediately. If it's increased, additional candidates are pre-gathered; if it's decreased, the now-superfluous candidates are discarded.
- The bundle and RTCP-multiplexing policies MUST NOT be changed after the construction of the PeerConnection.
calling this method may change the state of the ICE agent 🧊.
4.1.19. addIceCandidate 🧊
addIceCandidate sends an update to the ICE agent through an IceCandidate object (Section 3.5.2.1) 💌. if the IceCandidate's candidate field is non-null, it's treated as a new remote ICE candidate, which gets added to the current and/or pending remote description following the Trickle ICE rules. otherwise, the IceCandidate is treated as an end-of-candidates indication, as defined in [RFC8838], Section 14 🏁.
either way, the "m=" section index, MID, and ufrag fields from the IceCandidate decide which "m=" section and ICE candidate generation it belongs to, as described in Section 3.5.2.1 above 🎯. for an end-of-candidates indication, null values for the "m=" section index and MID fields mean the indication applies to all "m=" sections in the specified ICE candidate generation. but if both fields are null for a new remote candidate, this MUST be treated as an invalid condition, as specified below 🙅.
if any IceCandidate fields hold invalid values or an error pops up while processing the IceCandidate object, the supplied IceCandidate MUST be ignored and an error MUST be returned ❌.
otherwise, the new remote candidate or end-of-candidates indication gets handed to the ICE agent. for a new remote candidate, connectivity checks are sent to it, assuming setLocalDescription has already been called to start the ICE gathering process 🚀.
4.1.20. onicecandidate Event 🎉
the onicecandidate event gets dispatched to the application in two situations: (1) when the ICE agent discovers a new allowed local ICE candidate during ICE gathering (see Section 3.5.1, plus the restrictions in Section 3.5.3), or (2) when an ICE gathering phase completes 🎁. the event carries a single IceCandidate object, as defined in Section 3.5.2.1.
in the first case, the newly discovered candidate shows up in the IceCandidate object, and all of its fields MUST be non-null 💎. that candidate also gets added to the current and/or pending local description, following the Trickle ICE rules.
in the second case, the event's IceCandidate object MUST have its candidate field set to null to signal that the current gathering phase is done, meaning no more onicecandidate events are coming in this phase 🛑. the IceCandidate's ufrag field MUST be specified so you know which ICE candidate generation is ending 🔚. the IceCandidate's "m=" section index and MID fields MAY be specified to say the event applies to one specific "m=" section, or be set to null to say it applies to all "m=" sections in the current ICE candidate generation 🌟. this event can be used by the application to generate an end-of-candidates indication, as defined in [RFC8838], Section 13 💜.
4.2. RtpTransceiver 🎀
4.2.1. stop 🚫
the stop method stops an RtpTransceiver 🛑. after that, future calls to createOffer generate a zero port for the associated "m=" section. more details are below 👀.
4.2.2. stopped 🙅♀️
the stopped property tells you whether the transceiver has been stopped, either by calling stop or by applying an answer that rejects the associated "m=" section 🚷. in either case it's "true", and otherwise it's "false".
a stopped RtpTransceiver doesn't send any outgoing RTP or RTCP, and it doesn't process any incoming RTP or RTCP 😶. and it can't be restarted, so no comebacks 💀.
4.2.3. setDirection 🧭
setDirection sets the direction of a transceiver, which affects the direction property of the associated "m=" section on future calls to createOffer and createAnswer 💅. the permitted values are recvonly, sendrecv, sendonly, and inactive, mirroring the identically named direction attributes defined in [RFC4566], Section 6 📜.
when you create offers, the transceiver direction is directly reflected in the output, even for re-offers 📤. when you create answers, the transceiver direction gets intersected with the offered direction, as explained in Section 5.3 below 🤝.
heads up though: even though setDirection sets the direction property (Section 4.2.4) right away, that doesn't immediately change whether the transceiver's RtpSender sends or its RtpReceiver receives 👀. the direction actually in effect shows up in the currentDirection property, which only updates when an answer is applied ✨.
4.2.4. direction 🧁
the direction property shows the last value passed into setDirection 💌. if setDirection has never been called, it's set to whatever direction the transceiver was initialized with.
4.2.5. currentDirection 🔮
the currentDirection property shows the last negotiated direction for the transceiver's associated "m=" section 🤓. more specifically, it's the direction attribute [RFC3264] of the associated "m=" section in the last applied answer (provisional answers count too), with the direction reversed if it was a remote answer. for example, if the direction attribute for the associated "m=" section in a remote answer is "recvonly", then currentDirection is set to "sendonly" 🔄.
if an answer that references this transceiver hasn't been applied yet, or if the transceiver is stopped, currentDirection is set to null 🙈.
4.2.6. setCodecPreferences 🎀
okay so setCodecPreferences lets you set the codec preferences on a transceiver 💖. Those preferences decide which codecs show up in the associated m= section, and in what order, the next time you call createOffer or createAnswer 🦋. bestie, the big twist is that it does NOT pick what codec actually gets sent 🙈. It only changes which codecs the implementation says it prefers to receive, through the offer or answer. And even if setCodecPreferences excludes a codec, it can STILL be used to send until the next offer/answer exchange throws it out 💀. no cap, that's a plot twist.
The codec preferences of an RtpTransceiver can cause codecs to be excluded from later createOffer and createAnswer calls. When that happens, the matching media formats get dropped from the associated m= section 🥀. The preferences can't add media formats that wouldn't otherwise be there, so they only ever subtract ✂️.
The codec preferences can also set the order of codecs in later createOffer and createAnswer calls. In that case, the media formats in the associated m= section follow whatever order you specified 🌈.
5. SDP Interaction Procedures 👑
This section walks through the exact steps for creating and parsing SDP objects 📝.
5.1. Requirements Overview 🌸
okay this part is the checklist 📋. JSEP implementations MUST comply with the specifications listed below, since they're the ones that govern how offers and answers get created and processed.
5.1.1. Usage Requirements 💅
All session descriptions that JSEP implementations handle, both local and remote, MUST indicate support for the specifications below. If any of them are missing, that omission MUST be treated as an error 🚫.
- ICE, as specified in [RFC8445], MUST be used. Note that the remote endpoint may use a lite implementation, so implementations MUST properly handle remote endpoints that use ICE-lite. The remote endpoint may also use an older version of ICE, so implementations MUST properly handle remote endpoints that use ICE as specified in [RFC5245].
- DTLS [RFC6347] [RFC9147] or DTLS-SRTP [RFC5763] MUST be used, as appropriate for the media type, as specified in [RFC8827]. Note: RFC 8827 requires implementations to support DTLS 1.2 [RFC6347] and permits the use of DTLS 1.3 [RFC9147].
The SDP security descriptions mechanism for Secure Real-time Transport Protocol (SRTP) keying [RFC4568] MUST NOT be used, as discussed in [RFC8827] 🙅♀️.
5.1.2. Profile Names and Interoperability 🦄
For media m= sections, JSEP implementations MUST support the UDP/TLS/RTP/SAVPF profile specified in [RFC5764], as well as the TCP/DTLS/RTP/SAVPF profile specified in [RFC7850]. They MUST indicate one of these profiles for each media m= line they produce in an offer. For data m= sections, implementations MUST support the UDP/DTLS/SCTP profile as well as the TCP/DTLS/SCTP profile, and MUST indicate one of these profiles for each data m= line they produce in an offer. The exact profile is determined by the protocol associated with the current default or selected ICE candidate, as described in [RFC8839], Section 4.2.1.2 ✨.
Unfortunately, some endpoints try to be compatible by generating other profile strings even when they actually mean to support one of these profiles 😭. For example, an endpoint might generate RTP/AVP but supply a=fingerprint and a=rtcp-fb attributes, which signals that it's willing to support UDP/TLS/RTP/SAVPF or TCP/DTLS/RTP/SAVPF. To make things smoother with those endpoints, JSEP implementations MUST follow these rules when processing the media m= sections in a received offer 🧚:
- Any profile in the offer matching one of the following MUST be accepted:
RTP/AVP(defined in [RFC4566], Section 8.2.2)RTP/AVPF(defined in [RFC4585], Section 9)RTP/SAVP(defined in [RFC3711], Section 12)RTP/SAVPF(defined in [RFC5124], Section 6)TCP/DTLS/RTP/SAVP(defined in [RFC7850], Section 3.4)TCP/DTLS/RTP/SAVPF(defined in [RFC7850], Section 3.5)UDP/TLS/RTP/SAVP(defined in [RFC5764], Section 9)UDP/TLS/RTP/SAVPF(defined in [RFC5764], Section 9)
- The profile in any
m=line in any generated answer MUST exactly match the profile provided in the offer. - Because DTLS-SRTP is REQUIRED, the choice of SAVP or AVP has no effect. Support for DTLS-SRTP is determined by the presence of one or more
a=fingerprintattributes. Note that lack of ana=fingerprintattribute will lead to negotiation failure 💔. - The use of AVPF or AVP simply controls the timing rules used for RTCP feedback. If AVPF is provided or an
a=rtcp-fbattribute is present, assume AVPF timing, i.e., a default value oftrr-int=0. Otherwise, assume that AVPF is being used in an AVP-compatible mode and use a value oftrr-int=4000. - For data
m=sections, implementations MUST support receiving theUDP/DTLS/SCTP,TCP/DTLS/SCTP, orDTLS/SCTP(for backwards compatibility) profiles.
Note that re-offers by JSEP implementations MUST use the correct profile strings, even if the initial offer/answer exchange used an (incorrect) older profile string 🔁. This keeps JSEP behavior simple, with minimal downside, because any remote endpoint that fails to handle such a re-offer will also fail to handle a JSEP endpoint's initial offer 🙃.
5.2. Constructing an Offer 🎀
When createOffer is called, a new SDP description MUST be created that includes the functionality specified in [RFC8834]. The exact details of how that happens are explained below 🦋.
5.2.1. Initial Offers 💖
When createOffer is called for the first time, the result is known as the initial offer 🌷.
The first step in generating an initial offer is to generate the session-level attributes, as specified in [RFC4566], Section 5. Specifically 📝:
- The first SDP line MUST be
v=0as defined in [RFC4566], Section 5.1. - The second SDP line MUST be an
o=line as defined in [RFC4566], Section 5.2. The value of the<username>field SHOULD be-. The<sess-id>MUST be representable by a 64-bit signed integer, and the value MUST be less than (2^63)-1. This makes sure the<sess-id>value, when written as a string, is always a non-negative integer, since some SDP parsers may fail to parse a negative<sess-id>🙈. It is RECOMMENDED that the<sess-id>be built by generating a 64-bit quantity with the highest bit set to zero and the remaining 63 bits being cryptographically random 🔐. The value of the<nettype><addrtype><unicast-address>tuple SHOULD be set to a non-meaningful address, such asIN IP4 0.0.0.0, so a local IP address doesn't leak in this field. This problem is discussed in [RFC8828]. As mentioned in [RFC4566], the entireo=line needs to be unique, but picking a random number for<sess-id>is enough to make that happen ✨. - The third SDP line MUST be a
s=line as defined in [RFC4566], Section 5.3. To match theo=line, a single dash SHOULD be used as the session name, e.g.,s=-. This is different from the advice in [RFC4566], which proposes a single space. But since botho=ands=are meaningless in JSEP, using the same meaningless value just seems clearer 🤷♀️. - Session Information (
i=), URI (u=), Email Address (e=), Phone Number (p=), Repeat Times (r=), and Time Zones (z=) lines are not useful in this context and SHOULD NOT be included 🙅♀️. - Encryption Keys (
k=) lines do not provide sufficient security and MUST NOT be included 🔓. - A
t=line MUST be added, as specified in [RFC4566], Section 5.9. Both<start-time>and<stop-time>SHOULD be set to zero, e.g.,t=0 0⏰. - An
a=ice-optionsline with the "trickle" and "ice2" options MUST be added, as specified in [RFC8840], Section 4.1.1 and [RFC8445], Section 10 🧊. - If WebRTC identity is being used, an
a=identityline MUST be added, as described in [RFC8827], Section 5 🪪.
The next step is to generate m= sections, as specified in [RFC4566], Section 5.14. An "m=" section is generated for each RtpTransceiver that has been added to the PeerConnection, excluding any stopped RtpTransceivers. This happens in the order the RtpTransceivers were added to the PeerConnection 🎶. If there are no such RtpTransceivers, no "m=" sections are generated, but more can be added later, as discussed in [RFC3264], Section 5.
For each "m=" section generated for an RtpTransceiver, establish a mapping between the transceiver and the index of the generated "m=" section 🗺️.
Each "m=" section, as long as it is not marked as bundle-only, MUST contain a unique set of ICE credentials and a unique set of ICE candidates. Bundle-only "m=" sections MUST NOT contain any ICE credentials and MUST NOT gather any candidates 🚫.
For DTLS, all "m=" sections MUST use any and all certificates that have been specified for the PeerConnection. As a result, they MUST all have the same fingerprint value or values [RFC8122], or these values MUST be session-level attributes 🔏.
Each "m=" section MUST be generated as specified in [RFC4566], Section 5.14. For the m= line itself, the following rules MUST be followed 🎀:
- If the "m=" section is marked as bundle-only, then the
<port>value MUST be set to zero. Otherwise, the<port>value is set to the port of the default ICE candidate for this "m=" section. But since no candidates are available yet, the default<port>value of 9 (Discard) MUST be used, as indicated in [RFC8840], Section 4.1.1. - To properly indicate use of DTLS, the
<proto>field MUST be set toUDP/TLS/RTP/SAVPF, as specified in [RFC5764], Section 8. - If codec preferences have been set for the associated transceiver, media formats MUST be generated in the corresponding order and MUST exclude any codecs not present in the codec preferences 🎯.
- Unless excluded by the above restrictions, the media formats MUST include the mandatory audio/video codecs as specified in [RFC7874], Section 3 and [RFC7742], Section 5.
The m= line MUST be followed immediately by a c= line, as specified in [RFC4566], Section 5.7. Again, since no candidates are available yet, the c= line MUST contain the default value IN IP4 0.0.0.0, as defined in [RFC8840], Section 4.1.1 🌬️.
[RFC8859] groups SDP attributes into different categories. To avoid unnecessary duplication when bundling, attributes of category IDENTICAL or TRANSPORT MUST NOT be repeated in bundled "m=" sections. This repeats the guidance from [RFC9143], Section 7.1.3. It includes "m=" sections for which bundling has been negotiated and is still desired, as well as "m=" sections marked as bundle-only 🧩.
The following attributes, which are of a category other than IDENTICAL or TRANSPORT, MUST be included in each "m=" section 💜:
- An
a=midline, as specified in [RFC5888], Section 4. All MID values MUST be generated in a fashion that does not leak user information, e.g., randomly or using a per-PeerConnection counter. They SHOULD be 3 bytes or less, so they can fit efficiently into the RTP header extension defined in [RFC9143], Section 15.2 🪄. Note that this does not set the RtpTransceiver mid property, since that only happens when the description is applied. The generated MID value can be considered a "proposed" MID at this point 🌸.
A direction attribute 🧭 that matches the one on the transceiver it belongs to.
- For each media format on the
m=line, thea=rtpmapanda=fmtplines, as specified in [RFC4566], Section 6 and [RFC3264], Section 5.1. 📻 - For each primary codec where RTP retransmission should be used, a matching
a=rtpmapline that saysrtxwith the clock rate of the primary codec, plus ana=fmtpline that points back at the primary codec's payload type, as specified in [RFC4588], Section 8.1. 🔁 - For each Forward Error Correction (FEC) mechanism the app supports,
a=rtpmapanda=fmtplines, as specified in [RFC4566], Section 6. The FEC mechanisms that MUST be supported are listed in [RFC8854], Section 7, and the specific usage for each media type is laid out in Sections 4 and 5 of [RFC8854]. 🛡️ - If this
m=section is for media with configurable durations per packet (audio is the classic example), ana=maxptimeline. It gives the maximum amount of media, in milliseconds, that can be packed into each packet, as specified in [RFC4566], Section 6. The value is set to the smallest of the maximum duration values across all the codecs in them=section. ⏱️ - If this
m=section is for video and there are known limits on the size of images that can be decoded, ana=imageattrline, as specified in Section 3.6. 🖼️ - For each RTP header extension the app supports, an
a=extmapline, as specified in [RFC5285], Section 5. The list of header extensions that SHOULD/MUST be supported is specified in [RFC8834], Section 5.2. Any header extensions that require encryption MUST be specified as indicated in [RFC6904], Section 4. 🎁 - For each RTCP feedback mechanism the app supports, an
a=rtcp-fbline, as specified in [RFC4585], Section 4.2. The list of RTCP feedback mechanisms that SHOULD/MUST be supported is specified in [RFC8834], Section 5.1. 💬 - If the RtpTransceiver has a
sendrecvorsendonlydirection:- For each MediaStream that was associated with the transceiver when it was created via addTrack or addTransceiver, an
a=msidline, as specified in [RFC8830], Section 2, but leaving out theappdatafield. 🎬
- For each MediaStream that was associated with the transceiver when it was created via addTrack or addTransceiver, an
- If the RtpTransceiver has a
sendrecvorsendonlydirection, and the app has specified a rid-id for an encoding, or has specified more than one encoding in the RtpSenders's parameters, ana=ridline for each encoding specified. Thea=ridline is specified in [RFC8851], and its direction MUST besend. If the app has chosen a rid-id, it MUST be used. Otherwise, a rid-id MUST be generated by the implementation. rid-ids MUST be generated in a way that does not leak user information, for example randomly or with a per-PeerConnection counter (see the guidance at the end of [RFC8852], Section 3.3). They SHOULD be 3 bytes or less, so they fit neatly into the RTP header extensions defined in [RFC8852], Section 3.3. If no encodings have been specified, or only one encoding is specified without a rid-id, then noa=ridlines are generated. 🪪 - If the RtpTransceiver has a
sendrecvorsendonlydirection and more than onea=ridline has been generated, ana=simulcastline with directionsend, as defined in [RFC8853], Section 5.1. The associated set of rid-ids MUST include all of the rid-ids used in thea=ridlines for thism=section. 📡 - If (1) the bundle policy for this PeerConnection is set to
must-bundleand this is not the firstm=section, or (2) the bundle policy is set tobalancedand this is not the firstm=section for this media type, ana=bundle-onlyline. 🧺
The following attributes, which are of category IDENTICAL or TRANSPORT, MUST appear only in m= sections that either have a unique address or are associated with the BUNDLE-tag. (In initial offers, this means those m= sections that do not contain an a=bundle-only attribute.) 🔒
a=ice-ufraganda=ice-pwdlines, as specified in [RFC8839], Section 5.4. 🧊- For each desired digest algorithm, one or more
a=fingerprintlines for each of the endpoint's certificates, as specified in [RFC8122], Section 5. - An
a=setupline, as specified in [RFC4145], Section 4 and clarified for use in DTLS-SRTP scenarios in [RFC5763], Section 5. The role value in the offer MUST beactpass. 🤝 - An
a=tls-idline, as specified in [RFC8842], Section 5.2. 🪪 - An
a=rtcpline, as specified in [RFC3605], Section 2.1, containing the default value9 IN IP4 0.0.0.0, because no candidates have been gathered yet. 🫥 - An
a=rtcp-muxline, as specified in [RFC5761], Section 5.1.3. 🧵 - If the RTP/RTCP multiplexing policy is
require, ana=rtcp-mux-onlyline, as specified in [RFC8858], Section 4. 🔐 - An
a=rtcp-rsizeline, as specified in [RFC5506], Section 5. 📏
Lastly, if a data channel has been created, an m= section MUST be generated for data. The <media> field MUST be set to application, and the <proto> field MUST be set to UDP/DTLS/SCTP [RFC8841]. The <fmt> value MUST be set to webrtc-datachannel, as specified in [RFC8841], Section 4.4.2. 💬
Within the data m= section, an a=mid line MUST be generated and included as described above, along with an a=sctp-port line referencing the SCTP port number, as defined in [RFC8841], Section 5.1. If appropriate, an a=max-message-size line is also included, as defined in [RFC8841], Section 6.1. 🌷
As discussed above, the following attributes of category IDENTICAL or TRANSPORT are included only if the data m= section either has a unique address or is associated with the BUNDLE-tag (for example, if it is the only m= section):
a=ice-ufraga=ice-pwda=fingerprinta=setupa=tls-id
Once all m= sections have been generated, a session-level a=group attribute MUST be added, as specified in [RFC5888]. This attribute MUST have semantics BUNDLE and MUST include the MID identifiers of each m= section. The effect is that the JSEP implementation offers all m= sections as one bundle group. Whether the m= sections are bundle-only or not depends on the bundle policy. 🎀
The next step is to generate session-level lip sync groups, as defined in [RFC5888], Section 7. For each MediaStream referenced by more than one RtpTransceiver (by passing those MediaStreams as arguments to the addTrack and addTransceiver methods), a group of type LS MUST be added that contains the MID values for each RtpTransceiver. 👄
Attributes that SDP permits at either the session level or the media level SHOULD generally be at the media level, even if they are identical. This helps development and debugging by making individual media sections easier to understand, especially if one attribute from a set of initially identical ones is changed later. However, implementations MAY choose to aggregate attributes at the session level, and JSEP implementations MUST be prepared to receive attributes in either location. 🧠
Attributes other than the ones specified above MAY be included, except for the following attributes, which are specifically incompatible with the requirements of [RFC8834] and MUST NOT be included:
a=cryptoa=key-mgmta=ice-lite
Note that when bundle is used, any additional attributes that are added MUST follow the advice in [RFC8859] on how those attributes interact with bundle. 🫧
Note that these requirements are in some cases stricter than those of SDP. Implementations MUST be prepared to accept compliant SDP even if it would not conform to the requirements for generating SDP in this specification. 🌈
5.2.2. Subsequent Offers 💖
okay so, this part is about what happens when createOffer gets called a second (or later) time, or after a local description is already installed. it's giving "redo, but make sure you don't break what's already working" 🎀
If the previous offer was not applied using setLocalDescription, meaning the PeerConnection is still in the "stable" state, the steps for generating an initial offer MUST be followed, subject to the following restriction:
* The fields of the "o=" line MUST stay the same except for the <session-version> field, which MUST increment by one on each call to createOffer if the offer might differ from the output of the previous call to createOffer; implementations MAY opt to increment <session-version> on every call. The value of the generated <session-version> is independent of the <session-version> of the current local description; in particular, in the case where the current version is N, an offer is created and applied with version N+1, and then that offer is rolled back so that the current version is again N, the next generated offer will still have version N+2.
Note that if the application creates an offer by reading currentLocalDescription instead of calling createOffer, the returned SDP may be different than when setLocalDescription was originally called, due to the addition of gathered ICE candidates, but the <session-version> will not have changed. There are no known scenarios in which this causes problems, but if this is a concern, the solution is simply to use createOffer to ensure a unique <session-version>. 🧚
If the previous offer was applied using setLocalDescription, but a corresponding answer from the remote side has not yet been applied, meaning the PeerConnection is still in the "have-local-offer" state, an offer is generated by following the steps in the "stable" state above, along with these exceptions:
* The "s=" and "t=" lines MUST stay the same.
* If any RtpTransceiver has been added and there exists an "m=" section with a zero port in the current local description or the current remote description, that "m=" section MUST be recycled by generating an "m=" section for the added RtpTransceiver as if the "m=" section were being added to the session description (including a new MID value) and placing it at the same index as the "m=" section with a zero port.
* If an RtpTransceiver is stopped and is not associated with an "m=" section, an "m=" section MUST NOT be generated for it. This prevents adding back RtpTransceivers whose "m=" sections were recycled and used for a new RtpTransceiver in a previous offer/answer exchange, as described above.
* If an RtpTransceiver has been stopped and is associated with an "m=" section, and the "m=" section is not being recycled as described above, an "m=" section MUST be generated for it with the port set to zero and all "a=msid" lines removed.
* For RtpTransceivers that are not stopped, the "a=msid" line or lines MUST stay the same if they are present in the current description, regardless of changes to the transceiver's direction or track. If no "a=msid" line is present in the current description, "a=msid" line(s) MUST be generated according to the same rules as for an initial offer.
* Each "m=" and "c=" line MUST be filled in with the port, relevant RTP profile, and address of the default candidate for the "m=" section, as described in [RFC8839], Section 4.2.1.2 and clarified in Section 5.1.2. If no RTP candidates have yet been gathered, default values MUST still be used, as described above.
* Each "a=mid" line MUST stay the same.
* Each "a=ice-ufrag" and "a=ice-pwd" line MUST stay the same, unless the ICE configuration has changed (e.g., changes to either the supported STUN/TURN servers or the ICE candidate policy) or the IceRestart option (Section 5.2.3.1) was specified. If the "m=" section is bundled into another "m=" section, it still MUST NOT contain any ICE credentials.
* If the "m=" section is not bundled into another "m=" section, its "a=rtcp" attribute line MUST be filled in with the port and address of the default RTCP candidate, as indicated in [RFC5761], Section 5.1.3. If no RTCP candidates have yet been gathered, default values MUST be used, as described in Section 5.2.1 above.
* If the "m=" section is not bundled into another "m=" section, for each candidate that has been gathered during the most recent gathering phase (see Section 3.5.1), an "a=candidate" line MUST be added, as defined in [RFC8839], Section 5.1. If candidate gathering for the section has completed, an "a=end-of-candidates" attribute MUST be added, as described in [RFC8840], Section 8.2. If the "m=" section is bundled into another "m=" section, both "a=candidate" and "a=end-of-candidates" MUST be omitted.
* For RtpTransceivers that are still present, the "a=rid" lines MUST stay the same.
* For RtpTransceivers that are still present, any "a=simulcast" line MUST stay the same.
If the previous offer was applied using setLocalDescription, and a corresponding answer from the remote side has been applied using setRemoteDescription, meaning the PeerConnection is in the "have-remote-pranswer" state or the "stable" state, an offer is generated based on the negotiated session descriptions by following the steps mentioned for the "have-local-offer" state above.
In addition, for each existing, non-recycled, non-rejected "m=" section in the new offer, the following adjustments are made based on the contents of the corresponding "m=" section in the current local or remote description, as appropriate:
* The "m=" line and corresponding "a=rtpmap" and "a=fmtp" lines MUST only include media formats that have not been excluded by the codec preferences of the associated transceiver and also MUST include all currently available formats. Media formats that were previously offered but are no longer available (e.g., a shared hardware codec) MAY be excluded.
* Unless codec preferences have been set for the associated transceiver, the media formats on the "m=" line MUST be generated in the same order as in the most recent answer. Any media formats that were not present in the most recent answer MUST be added after all existing formats.
* The RTP header extensions MUST only include those that are supported by the application on the associated transceiver.
* The RTCP feedback mechanisms MUST only include those that are supported by the application on the associated transceiver.
* The "a=rtcp" line MUST NOT be added if the most recent answer included an "a=rtcp-mux" line.
* The "a=rtcp-mux" line MUST be the same as that in the most recent answer.
* The "a=rtcp-mux-only" line MUST NOT be added.
* The "a=rtcp-rsize" line MUST NOT be added unless present in the most recent answer.
* An "a=bundle-only" line, as defined in [RFC9143], Section 6, MUST NOT be added. Instead, JSEP implementations MUST simply omit parameters in the IDENTICAL and TRANSPORT categories for bundled "m=" sections, as described in [RFC9143], Section 7.1.3.
* Note that if media "m=" sections are bundled into a data "m=" section, then certain TRANSPORT and IDENTICAL attributes may appear in the data "m=" section even if they would otherwise only be appropriate for a media "m=" section (e.g., "a=rtcp-mux"). This cannot happen in initial offers because in the initial offer JSEP implementations always list media "m=" sections (if any) before the data "m=" section (if any), and at least one of those media "m=" sections will not have the "a=bundle-only" attribute. Therefore, in initial offers, any "a=bundle-only" "m=" sections will be bundled into a preceding non-bundle-only media "m=" section.
The "a=group:BUNDLE" attribute MUST include the MID identifiers specified in the bundle group in the most recent answer, minus any "m=" sections that have been marked as rejected, plus any newly added or re-enabled "m=" sections. In other words, the bundle attribute MUST contain all "m=" sections that were previously bundled, as long as they are still alive, as well as any new "m=" sections.
Note that if bundling has been negotiated, unbundling is no longer possible, and media sections will not be marked as bundle-only. Although this is by design, it could cause issues in the rare case of sending a subsequent offer as an initial offer to a non-bundle-aware endpoint via Third Party Call Control (3PCC), as discussed in [RFC9143], Section 7.6.
"a=group:LS" attributes are generated in the same way as for initial offers, with the additional stipulation that any lip sync groups that were present in the most recent answer MUST continue to exist and MUST contain any previously existing MID identifiers, as long as the identified "m=" sections still exist and are not rejected, and the group still contains at least two MID identifiers. This ensures that any synchronized "recvonly" "m=" sections continue to be synchronized in the new offer. 🫶
5.2.3. Options Handling 🎀
The createOffer method takes as a parameter an RTCOfferOptions object. Special processing is performed when generating an SDP description if the following options are present.
5.2.3.1. IceRestart 🔥
If the IceRestart option is specified, with a value of "true", the offer MUST indicate an ICE restart by generating new ICE ufrag and pwd attributes, as specified in [RFC8839], Section 4.4.1.1.1. If this option is specified on an initial offer, it has no effect (since a new ICE ufrag and pwd are already generated). Similarly, if the ICE configuration has changed, this option has no effect, since new ufrag and pwd attributes will be generated automatically. This option is primarily useful for reestablishing connectivity in cases where failures are detected by the application. 💅
5.2.3.2. VoiceActivityDetection 🎀
okay so silence suppression, also called DTX ("discontinuous transmission") 🤫, is lowkey genius. When nobody's talking, you switch to a special encoding that uses less bandwidth for audio. The catch is that you lose a little fidelity, so it's a trade-off, bestie 💅
If the VoiceActivityDetection option is specified with a value of "true", the offer MUST indicate support for silence suppression in the audio it receives. It does this by including comfort noise ("CN") codecs for each offered audio codec, as specified in [RFC3389], Section 5.1, except for codecs that have their own internal silence suppression support. ✨ For codecs that have their own internal silence suppression support, the appropriate fmtp parameters for each such codec MUST be specified to indicate that silence suppression for received audio is desired. For example, when using the Opus codec [RFC6716], the "usedtx=1" parameter, specified in [RFC7587], would be used in the offer. 🦋
If the VoiceActivityDetection option is specified with a value of "false", the JSEP implementation MUST NOT emit "CN" codecs. 🙈 For codecs that have their own internal silence suppression support, the appropriate fmtp parameters for each such codec MUST be specified to indicate that silence suppression for received audio is not desired. For example, when using the Opus codec, the "usedtx=0" parameter would be specified in the offer. On top of that, the implementation MUST NOT use silence suppression for media it generates, regardless of whether the "CN" codecs or related fmtp parameters appear in the peer's description. The result is that silence suppression in JSEP depends on mutual agreement of both sides, which keeps handling consistent no matter which codec is used. 🫶
The VoiceActivityDetection option does not have any impact on the setting of the "vad" value in the signaling of the client-to-mixer audio level header extension described in [RFC6464], Section 4. 💜
5.3. Generating an Answer 🦋
when createAnswer is called, a brand new SDP description MUST be created. It has to be compatible with the supplied remote description and also meet the requirements in [RFC8834]. The exact details are explained below, so let's go 🌈
5.3.1. Initial Answers 🧚
when createAnswer is called for the first time after a remote description has been provided, the result is known as the initial answer. If no remote description has been installed, an answer cannot be generated, and an error MUST be returned. 💀
Note that the remote description SDP may not have been created by a JSEP endpoint, so it may not conform to all the requirements listed in Section 5.2. For lots of cases that's fine. ✨ But if any mandatory SDP attributes are missing, or functionality listed as mandatory-to-use above is not present, this MUST be treated as an error. It MUST also cause the affected "m=" sections to be marked as rejected. 😭
The first step in generating an initial answer is to generate session-level attributes. This works the same way as in Section 5.2.1 above, with one difference: the "a=ice-options" line, with the "trickle" option as specified in [RFC8840], Section 4.1.3 and the "ice2" option as specified in [RFC8445], Section 10, is only included if such an option was present in the offer. 🎀
The next step is to generate session-level lip sync groups, as defined in [RFC5888], Section 7. For each group of type "LS" present in the offer, select the local RtpTransceivers that are referenced by the MID values in the specified group. Then figure out which of them either reference a common local MediaStream (specified in the calls to addTrack/addTransceiver used to create them) or have no MediaStream to reference because they were not created by addTrack/addTransceiver. If at least two such RtpTransceivers exist, a group of type "LS" with the MID values of these RtpTransceivers MUST be added. Otherwise, the offered "LS" group MUST be ignored, and no corresponding group is generated in the answer. 👑
As a simple example, consider the following offer of a single audio and single video track contained in the same MediaStream. SDP lines not relevant to this example have been removed for clarity. As explained in Section 5.2, a group of type "LS" has been added that references each track's RtpTransceiver. 🍓
a=group:LS a1 v1
m=audio 10000 UDP/TLS/RTP/SAVPF 0
a=mid:a1
a=msid:ms1
m=video 10001 UDP/TLS/RTP/SAVPF 96
a=mid:v1
a=msid:ms1If the answerer uses a single MediaStream when it adds its tracks, both of its transceivers will reference this stream. So the subsequent answer will contain a "LS" group identical to the one in the offer, as shown below: 🌸
a=group:LS a1 v1
m=audio 20000 UDP/TLS/RTP/SAVPF 0
a=mid:a1
a=msid:ms2
m=video 20001 UDP/TLS/RTP/SAVPF 96
a=mid:v1
a=msid:ms2However, if the answerer groups its tracks into separate MediaStreams, its transceivers will reference different streams. That means the subsequent answer will not contain a "LS" group. 😳
m=audio 20000 UDP/TLS/RTP/SAVPF 0
a=mid:a1
a=msid:ms2a
m=video 20001 UDP/TLS/RTP/SAVPF 96
a=mid:v1
a=msid:ms2bFinally, if the answerer does not add any tracks, its transceivers will not reference any MediaStreams. This causes the preferences of the offerer to be maintained, so the subsequent answer will contain an identical "LS" group. 🫧
a=group:LS a1 v1
m=audio 20000 UDP/TLS/RTP/SAVPF 0
a=mid:a1
a=recvonly
m=video 20001 UDP/TLS/RTP/SAVPF 96
a=mid:v1
a=recvonlyThe example in Section 7.2 shows a more involved case of "LS" group generation. 🔥
The next step is to generate an "m=" section for each "m=" section that is present in the remote offer, as specified in [RFC3264], Section 6. For the purposes of this discussion, any session-level attributes in the offer that are also valid as media-level attributes are considered to be present in each "m=" section. Each offered "m=" section will have an associated RtpTransceiver, as described in Section 5.10. If there are more RtpTransceivers than there are "m=" sections, the unmatched RtpTransceivers will need to be associated in a subsequent offer. 💅
For each offered "m=" section, if any of the following conditions are true, the corresponding "m=" section in the answer MUST be marked as rejected. You do this by setting the <port> in the "m=" line to zero, as indicated in [RFC3264], Section 6, and then further processing for this "m=" section can be skipped. ❌
- The associated RtpTransceiver has been stopped.
- There is no offered media format that is both supported and, if applicable, allowed by codec preferences.
- The bundle policy is "must-bundle", and this is not the first "m=" section or in the same bundle group as the first "m=" section.
- The bundle policy is "balanced", and this is not the first "m=" section for this media type or in the same bundle group as the first "m=" section for this media type.
- This "m=" section is in a bundle group, and the group's offerer tagged "m=" section is being rejected due to one of the above reasons. This requires all "m=" sections in the bundle group to be rejected, as specified in [RFC9143], Section 7.3.3.
Otherwise, each "m=" section in the answer MUST then be generated as specified in [RFC3264], Section 6.1. For the "m=" line itself, the following rules MUST be followed. 🧾
- The <port> value would normally be set to the port of the default ICE candidate for this "m=" section. But given that no candidates are available yet, the default <port> value of 9 (Discard) MUST be used, as indicated in [RFC8840], Section 4.1.1. 🙃
- The <proto> field MUST be set to exactly match the <proto> field for the corresponding "m=" line in the offer. 🎯
- If codec preferences have been set for the associated transceiver, media formats MUST be generated in the corresponding order, regardless of what was offered. They MUST also exclude any codecs not present in the codec preferences. 🌟
- Otherwise, the media formats on the "m=" line MUST be generated in the same order as those offered in the current remote description, excluding any currently unsupported formats. Any currently available media formats that are not present in the current remote description MUST be added after all existing formats. 🍰
- In either case, the media formats in the answer MUST include at least one format that is present in the offer. They MAY include formats that are locally supported but not present in the offer, as mentioned in [RFC3264], Section 6.1. If no common format exists, the "m=" section is rejected as described above. 🥺
The "m=" line MUST be followed immediately by a "c=" line, as specified in [RFC4566], Section 5.7. Again, as no candidates are available yet, the "c=" line MUST contain the default value "IN IP4 0.0.0.0", as defined in [RFC8840], Section 4.1.3. 🌬️
If the offer supports bundle, all "m=" sections to be bundled MUST use the same ICE credentials and candidates. All "m=" sections not being bundled MUST use unique ICE credentials and candidates. Each "m=" section MUST contain the following attributes, which are of attribute types other than IDENTICAL or TRANSPORT: 🔐
- If and only if present in the offer, an "a=mid" line, as specified in [RFC5888], Section 9.1. The MID value MUST match the one specified in the offer. 🏷️
- A direction attribute. You figure this out by applying the rules regarding the offered direction specified in [RFC3264], Section 6.1, and then intersecting with the direction of the associated RtpTransceiver. For example, if an "m=" section is offered as "sendonly" and the local transceiver is set to "sendrecv", the result in the answer is a "recvonly" direction. 🧭
- For each media format on the "m=" line, "a=rtpmap" and "a=fmtp" lines, as specified in [RFC4566], Section 6 and [RFC3264], Section 6.1. 📝
- If "rtx" is present in the offer, then for each primary codec where RTP retransmission should be used, add a corresponding "a=rtpmap" line indicating "rtx" with the clock rate of the primary codec. Also add an "a=fmtp" line that references the payload type of the primary codec, as specified in [RFC4588], Section 8.1. 🔁
- For each FEC mechanism supported by the application, "a=rtpmap" and "a=fmtp" lines, as specified in [RFC4566], Section 6. The FEC mechanisms that MUST be supported are specified in [RFC8854], Section 7, and the specific usage for each media type is outlined in Sections 4 and 5 of [RFC8854]. 🛡️
- If this "m=" section is for media with configurable durations of media per packet, e.g., audio, an "a=maxptime" line, as described in Section 5.2. ⏱️
- If this "m=" section is for video media and there are known limitations on the size of images that can be decoded, an "a=imageattr" line, as specified in Section 3.6. 🖼️
- For each RTP header extension supported by the application and present in the offer, an "a=extmap" line, as specified in [RFC5285], Section 5. The list of header extensions that SHOULD/ MUST be supported is specified in [RFC8834], Section 5.2. Any header extensions that require encryption MUST be specified as indicated in [RFC6904], Section 4. 🌈
- For each RTCP feedback mechanism your app supports that also shows up in the offer, you add an
a=rtcp-fbline, as specified in [RFC4585], Section 4.2. 💅 The list of RTCP feedback mechanisms that SHOULD/MUST be supported lives in [RFC8834], Section 5.1 🧚 - If the RtpTransceiver has a
sendrecvorsendonlydirection: 🎀- For each MediaStream that was associated with the transceiver when it was created via addTrack or addTransceiver, you add an
a=msidline, as specified in [RFC8830], Section 2, but you leave out theappdatafield 🦋
- For each MediaStream that was associated with the transceiver when it was created via addTrack or addTransceiver, you add an
Every m= section that is NOT bundled into another m= section MUST have these attributes (they're category IDENTICAL or TRANSPORT, so they're the "same for everyone" kind of vibe) ✨
a=ice-ufraganda=ice-pwdlines, as specified in [RFC8839], Section 5.4. 🔑- For each desired digest algorithm, one or more
a=fingerprintlines for each of the endpoint's certificates, as specified in [RFC8122], Section 5. 🔐 - An
a=setupline, as specified in [RFC4145], Section 4 and clarified for use in DTLS-SRTP scenarios in [RFC5763], Section 5. The role value in the answer MUST beactiveorpassive. When the offer contains theactpassvalue, which is always the case with JSEP endpoints, the answerer SHOULD use theactiverole. Offers from non-JSEP endpoints MAY send other values fora=setup, and in that case the answer MUST use a value consistent with the value in the offer. 🫶 - An
a=tls-idline, as specified in [RFC8842], Section 5.3. 🌸 - If it's in the offer, an
a=rtcp-muxline, as specified in [RFC5761], Section 5.1.3. Otherwise, you add ana=rtcpline, as specified in [RFC3605], Section 2.1, containing the default value9 IN IP4 0.0.0.0(because no candidates have been gathered yet, so it's giving placeholder energy) 🌈 - If it's in the offer, an
a=rtcp-rsizeline, as specified in [RFC5506], Section 5. 🍓
If a data channel m= section has been offered, you MUST also generate an m= section for data. The <media> field MUST be set to application, and the <proto> and <fmt> fields MUST be set to exactly match the fields in the offer. Lowkey the most important part of this whole bit 😭
Inside the data m= section, you MUST generate an a=mid line and include it as described above, along with an a=sctp-port line that references the SCTP port number, as defined in [RFC8841], Section 5.1. And, if it's appropriate, you add an a=max-message-size line, as defined in [RFC8841], Section 6.1. 💜
As we talked about above, the following attributes of category IDENTICAL or TRANSPORT are only included if the data m= section is NOT bundled into another m= section:
a=ice-ufraga=ice-pwda=fingerprinta=setupa=tls-id
Okay so here's the plot twist 🙈 If media m= sections are bundled into a data m= section, then some TRANSPORT and IDENTICAL attributes can show up in the data m= section too, even if they'd normally only be right for a media m= section (e.g., a=rtcp-mux). Bundling is a whole mood 🎭
If a=group attributes with semantics BUNDLE are offered, the matching session-level a=group attributes MUST be added, as specified in [RFC5888]. These attributes MUST have semantics BUNDLE and MUST include all MID identifiers from the offered bundle groups that have not been rejected. Also, no matter whether a=bundle-only was in the offer, all m= sections in the answer MUST NOT have an a=bundle-only line. Not me doing a whole rule just to say "don't" 💀
Attributes that are common between all m= sections MAY be moved up to the session level, but only if they're explicitly defined to be valid at the session level. ✨
The attributes that are prohibited when creating offers are ALSO prohibited when creating answers. No loopholes, bestie 🚫
5.3.2. Subsequent Answers
When createAnswer gets called a second (or later) time, or it gets called after a local description has already been installed, the processing is kinda different from an initial answer. 🌀
If the previous answer was NOT applied using setLocalDescription, meaning the PeerConnection is still in the have-remote-offer state, you MUST follow the steps for generating an initial answer, subject to this restriction:
- The fields of the
o=line MUST stay the same, EXCEPT for the<session-version>field, which MUST increment if the session description changes in any way from the previously generated answer. 🔢
If any session description was previously supplied to setLocalDescription, you generate an answer by following the steps in the have-remote-offer state above, along with these exceptions:
- The
s=andt=lines MUST stay the same. 🎀 - Each
m=andc=line MUST be filled in with the port and address of the default candidate for them=section, as described in [RFC8839], Section 4.2.1.2. Note that in certain cases them=line protocol may not match that of the default candidate, because them=line protocol value MUST match what was supplied in the offer, as described above. Plot twist, but it's still the rule 🧚 - Each
a=ice-ufraganda=ice-pwdline MUST stay the same, UNLESS them=section is restarting. In that case, new ICE credentials MUST be created as specified in [RFC8839], Section 4.4.1.1.1. If them=section is bundled into anotherm=section, it STILL MUST NOT contain any ICE credentials. 🔑 - Each
a=tls-idline MUST stay the same, UNLESS the offerer'sa=tls-idline changed. In that case, a new tls-id value MUST be created, as described in [RFC8842], Section 5.2. 🌸 - Each
a=setupline MUST use anactiveorpassiverole value that's consistent with the existing DTLS association, IF the association is being continued by the offerer. 🫶 - RTCP multiplexing MUST be used, and an
a=rtcp-muxline MUST be inserted if and only if them=section previously used RTCP multiplexing. 🌈 - If the
m=section is NOT bundled into anotherm=section AND RTCP multiplexing is not active, ana=rtcpattribute line MUST be filled in with the port and address of the default RTCP candidate. If no RTCP candidates have been gathered yet, default values MUST be used, as described in Section 5.3.1 above. 🍓 - If the
m=section is NOT bundled into anotherm=section, then for each candidate that was gathered during the most recent gathering phase (see Section 3.5.1), ana=candidateline MUST be added, as defined in [RFC8839], Section 5.1. If candidate gathering for the section has completed, ana=end-of-candidatesattribute MUST be added, as described in [RFC8840], Section 8.2. If them=section IS bundled into anotherm=section, botha=candidateanda=end-of-candidatesMUST be omitted. 🦋 - For RtpTransceivers that are not stopped, the
a=msidline(s) MUST stay the same, regardless of changes to the transceiver's direction or track. If noa=msidline is present in the current description,a=msidline(s) MUST be generated according to the same rules as for an initial answer. 💖
5.3.3. Options Handling
The createAnswer method takes an RTCAnswerOptions object as a parameter. The set of parameters for RTCAnswerOptions is different from the ones supported in RTCOfferOptions. The IceRestart option is unnecessary, because ICE credentials will automatically be changed for all m= sections where the offerer chose to do an ICE restart. Big brain energy, no notes 🧠
The following option is supported in RTCAnswerOptions. ✨
5.3.3.1. VoiceActivityDetection
Silence suppression in the answer is handled as described in Section 5.2.3.2, with one exception: if support for silence suppression was NOT indicated in the offer, the VoiceActivityDetection parameter has no effect, and the answer MUST be generated as if VoiceActivityDetection was set to false. This is done on a per-codec basis. For example, if the offerer somehow offered support for CN but set usedtx=0 for Opus, setting VoiceActivityDetection to true would result in an answer with CN codecs and usedtx=0. The impact of this rule is that an answerer will not try to use silence suppression with any endpoint that doesn't offer it, which makes silence suppression support bilateral, even with non-JSEP endpoints. Fair is fair 🤝💅
5.4. Modifying an Offer or Answer
The SDP returned from createOffer or createAnswer MUST NOT be changed before you pass it to setLocalDescription. If you need precise control over the SDP, the createOffer/createAnswer options or the RtpTransceiver APIs MUST be used instead. Don't touch the SDP by hand, bestie 🙅♀️
After calling setLocalDescription with an offer or answer, the application MAY modify the SDP to reduce its capabilities before sending it to the far side, as long as it follows the rules above that define a valid JSEP offer or answer. Likewise, an application that has received an offer or answer from a peer MAY modify the received SDP, subject to the same constraints, before calling setRemoteDescription. 🦋
As always, the application is solely responsible for what it sends to the other party, and all incoming SDP will be processed by the JSEP implementation to the extent of its capabilities. It is an error to assume that all SDP is well formed. However, you should be able to assume that any implementation of this specification will be able to process, as a remote offer or answer, unmodified SDP coming from any other implementation of this specification. Trust but verify, periodt 🫶✨
5.5. Processing a Local Description
When a SessionDescription is supplied to setLocalDescription, the following steps MUST be performed: 📋
- If the description is of type
rollback, follow the processing defined in Section 5.7 and skip the processing described in the rest of this section. 🌀 - Otherwise, the type of the SessionDescription is checked against the current state of the PeerConnection:
- If the type is
offer, the PeerConnection state MUST be eitherstableorhave-local-offer. - If the type is
pransweroranswer, the PeerConnection state MUST be eitherhave-remote-offerorhave-local-pranswer.
- If the type is
If the type is not correct for the current state, processing MUST stop and an error MUST be returned. 🚨
The SessionDescription is then checked to make sure its contents are identical to those generated in the last call to createOffer/createAnswer, and thus have not been altered, as discussed in Section 5.4. Otherwise, processing MUST stop and an error MUST be returned. No sneaky edits allowed 🙈
Next, the SessionDescription is parsed into a data structure, as described in Section 5.8 below. 🧩
Finally, the parsed SessionDescription is applied as described in Section 5.9 below. 👑
5.6. Processing a Remote Description ✨
okay so when a SessionDescription gets handed to setRemoteDescription 💅, the following steps MUST be performed:
- If the description is of type
rollback, follow the processing defined in Section 5.7 and skip the processing described in the rest of this section. - Otherwise, the type of the SessionDescription gets checked against the current state of the PeerConnection:
- If the type is
offer, the PeerConnection state MUST be eitherstableorhave-remote-offer. - If the type is
pransweroranswer, the PeerConnection state MUST be eitherhave-local-offerorhave-remote-pranswer.
- If the type is
- If the type is not correct for the current state, processing MUST stop and an error MUST be returned. 🙅♀️
- Next, the SessionDescription gets parsed into a data structure, as described in Section 5.8 below. If parsing fails for any reason, processing MUST stop and an error MUST be returned.
- Finally, the parsed SessionDescription gets applied as described in Section 5.10 below. 🎀
5.7. Processing a Rollback 🔄
a rollback can happen if the PeerConnection is in any state except for stable. so both offers and provisional answers can be rolled back, lowkey iconic. rollback is only for canceling proposed changes. there is no support for rolling back from a stable state to a previous stable state, so that's not a thing. if a rollback is attempted in the stable state, processing MUST stop and an error MUST be returned. and note that once the answerer has performed setLocalDescription with its answer, that can't be rolled back. no takebacks 😭
The effect of rollback MUST be the same regardless of whether setLocalDescription or setRemoteDescription is called. consistency is everything 🫶
To process rollback, a JSEP implementation abandons the current offer/answer transaction, sets the signaling state to stable, and sets the pending local and/or remote description (see Sections 4.1.14 and 4.1.16) to null. any resources or candidates that were allocated by the abandoned local description get discarded. any media that gets received is processed according to the previous local and remote descriptions. 🦋
A rollback disassociates any RtpTransceivers that were associated with m= sections by the application of the rolled-back session description (see Sections 5.10 and 5.9). this means some RtpTransceivers that were previously associated will no longer be associated with any m= section. in those cases, the value of the RtpTransceiver's mid property MUST be set to null, and the mapping between the transceiver and its m= section index MUST be discarded. RtpTransceivers that were created by applying a remote offer that was later rolled back MUST be stopped and removed from the PeerConnection. however, an RtpTransceiver MUST NOT be removed if a track was attached to it via the addTrack method. this is so an app can call addTrack, then call setRemoteDescription with an offer, then roll back that offer, then call createOffer and have an m= section for the added track show up in the generated offer. the main character energy of addTrack 🧚
5.8. Parsing a Session Description 📜
The SDP in the session description object is a sequence of text lines, each one a key-value expression, as described in [RFC4566], Section 5. the SDP gets read line by line and turned into a data structure holding the deserialized info. SDP allows lots of line types, though, and not all of them matter for JSEP apps. for each line, the implementation first makes sure it's syntactically correct according to its defining ABNF, checks that it follows the semantics used in [RFC4566] and [RFC3264], and then either parses and stores or discards the value, as described below. 🤓
If any line is not well formed or can't be parsed as described, the parser MUST stop with an error and reject the session description, even if the value was going to be discarded. this keeps implementations from accidentally misreading ambiguous SDP. no vibes-based parsing allowed 🙅♀️
5.8.1. Session-Level Parsing 🌸
First up, the session-level lines get checked and parsed. these lines MUST occur in a specific order, and with a specific syntax, as defined in [RFC4566], Section 5. note that while the specific line types (e.g., v=, c=) MUST occur in the defined order, lines of the same type (typically a=) can show up in any order. 💜
The following non-attribute lines aren't meaningful in the JSEP context and MAY be discarded once they've been checked. 🗑️
- The
c=line MUST be checked for syntax, but its value is only used for ICE mismatch detection, as defined in [RFC8445], Section 5.4. note that JSEP implementations should never encounter this condition because ICE is required for WebRTC. 🧊 - The
i=,u=,e=,p=,t=,r=,z=, andk=lines MUST be checked for syntax, but their values aren't otherwise used.
The remaining non-attribute lines get processed like this:
- The
v=line MUST have a version of 0, as specified in [RFC4566], Section 5.1. - The
o=line MUST be parsed as specified in [RFC4566], Section 5.2. - The
b=line, if present, MUST be parsed as specified in [RFC4566], Section 5.8, and the bwtype and bandwidth values stored.
Finally, the attribute lines get processed. Specific processing MUST be applied for the following session-level attribute (a=) lines: ✨
- Any
a=grouplines are parsed as specified in [RFC5888], Section 5, and the group's semantics and MID values are stored. - If present, a single
a=ice-liteline is parsed as specified in [RFC8839], Section 5.3, and a value indicating the presence of ana=ice-liteline is stored. - If present, a single
a=ice-ufragline is parsed as specified in [RFC8839], Section 5.4, and the ufrag value is stored. - If present, a single
a=ice-pwdline is parsed as specified in [RFC8839], Section 5.4, and the password value is stored. - If present, a single
a=ice-optionsline is parsed as specified in [RFC8839], Section 5.6, and the set of specified options is stored. - Any
a=fingerprintlines are parsed as specified in [RFC8122], Section 5, and the set of fingerprint and algorithm values is stored. - If present, a single
a=setupline is parsed as specified in [RFC4145], Section 4, and the setup value is stored. - If present, a single
a=tls-idline is parsed as specified in [RFC8842], Section 5, and the attribute value is stored. - Any
a=identitylines are parsed and the identity values stored for subsequent verification, as specified in [RFC8827], Section 5. - Any
a=extmaplines are parsed as specified in [RFC5285], Section 5, and their values are stored.
Other attributes that aren't relevant to JSEP may also be present, and implementations SHOULD process any that they recognize. As required by [RFC4566], Section 5.13, unknown attribute lines MUST be ignored. 🙈
Once all the session-level lines have been parsed, processing continues with the lines in m= sections. 🦋
5.8.2. Media Section Parsing 🎀
Okay so, just like the session-level lines, the media section lines MUST show up in the specific order and use the specific syntax defined in [RFC4566], Section 5. Order matters here, bestie 💅
The m= line itself MUST be parsed as described in [RFC4566], Section 5.14, and the <media>, <port>, <proto>, and <fmt> values stored. 🌸
After the m= line, specific processing MUST be applied to these non-attribute lines:
- Same energy as the
c=line at the session level: thec=line MUST be parsed according to [RFC4566], Section 5.7, but its value is not used. 🙈 - The
b=line, if present, MUST be parsed as specified in [RFC4566], Section 5.8, and the bwtype and bandwidth values stored. 💜
Specific processing MUST also be applied to these attribute lines:
- If present, a single
a=ice-ufragline is parsed as specified in [RFC8839], Section 5.4, and the ufrag value is stored. 🦋 - If present, a single
a=ice-pwdline is parsed as specified in [RFC8839], Section 5.4, and the password value is stored. 🔐 - If present, a single
a=ice-optionsline is parsed as specified in [RFC8839], Section 5.6, and the set of specified options is stored. ✨ - Any
a=candidateattributes MUST be parsed as specified in [RFC8839], Section 5.1, and their values stored. 🧚 - Any
a=remote-candidatesattributes MUST be parsed as specified in [RFC8839], Section 5.2, but their values are ignored. 🫶 - If present, a single
a=end-of-candidatesattribute MUST be parsed as specified in [RFC8840], Section 8.1, and its presence or absence flagged and stored. 🏁 - Any
a=fingerprintlines are parsed as specified in [RFC8122], Section 5, and the set of fingerprint and algorithm values is stored. 🔥
If the m= <proto> value indicates use of RTP, as described in Section 5.1.2 above, the following attribute lines MUST be processed. This is the main-character section, so pay attention 👑
- The
m=<fmt>value MUST be parsed as specified in [RFC4566], Section 5.14, and the individual values stored. 🍓 - Any
a=rtpmapora=fmtplines MUST be parsed as specified in [RFC4566], Section 6, and their values stored. 🌈 - If present, a single
a=ptimeline MUST be parsed as described in [RFC4566], Section 6, and its value stored. ⏱️ - If present, a single
a=maxptimeline MUST be parsed as described in [RFC4566], Section 6, and its value stored. ⏲️ - If present, a single direction attribute line (e.g., "a=sendrecv") MUST be parsed as described in [RFC4566], Section 6, and its value stored. ↔️
- Any
a=ssrcattributes MUST be parsed as specified in [RFC5576], Section 4.1, and their values stored. 🎵 - Any
a=extmapattributes MUST be parsed as specified in [RFC5285], Section 5, and their values stored. 🗺️ - Any
a=rtcp-fbattributes MUST be parsed as specified in [RFC4585], Section 4.2, and their values stored. 💬 - If present, a single
a=rtcp-muxattribute MUST be parsed as specified in [RFC5761], Section 5.1.3, and its presence or absence flagged and stored. 🔀 - If present, a single
a=rtcp-mux-onlyattribute MUST be parsed as specified in [RFC8858], Section 3, and its presence or absence flagged and stored. 🔒 - If present, a single
a=rtcp-rsizeattribute MUST be parsed as specified in [RFC5506], Section 5, and its presence or absence flagged and stored. 📏 - If present, a single
a=rtcpattribute MUST be parsed as specified in [RFC3605], Section 2.1, but its value is ignored, as this information is superfluous when using ICE. 🙅♀️ - If present,
a=msidattributes MUST be parsed as specified in [RFC8830], Section 3.2, and their values stored, ignoring any "appdata" field. If noa=msidattributes are present, a random msid-id value is generated for a "default" MediaStream for the session, if not already present, and this value is stored. 🎀 - Any
a=imageattrattributes MUST be parsed as specified in [RFC6236], Section 3, and their values stored. 🖼️ - Any
a=ridlines MUST be parsed as specified in [RFC8851], Section 10, and their values stored. 🆔 - If present, a single
a=simulcastline MUST be parsed as specified in [RFC8853], and its values stored. 📡
Otherwise, if the m= <proto> value indicates use of SCTP, the following attribute lines MUST be processed. Different vibe, same rules 🦋
- The
m=<fmt>value MUST be parsed as specified in [RFC8841], Section 4.3, and the application protocol value stored. 🧩 - An
a=sctp-portattribute MUST be present, and it MUST be parsed as specified in [RFC8841], Section 5.2, and the value stored. 🚪 - If present, a single
a=max-message-sizeattribute MUST be parsed as specified in [RFC8841], Section 6, and the value stored. Otherwise, use the specified default. 📦
Other attributes that aren't relevant to JSEP may also show up, and implementations SHOULD process any that they recognize. As required by [RFC4566], Section 5.13, unknown attribute lines MUST be ignored. Not my problem, not my line 🙃
5.8.3. Semantics Verification 🔍
Assuming parsing finishes successfully, the parsed description then gets checked to make sure it's internally consistent and properly supports the mandatory features. The following checks are performed:
- For each "m=" section, valid values for each of the mandatory-to-use features enumerated in Section 5.1.1 MUST be present. These values MAY be either present at the media level or inherited from the session level. 💖
- ICE ufrag and password values, which MUST comply with the size limits specified in [RFC8839], Section 5.4. 📐
- A tls-id value, which MUST be set according to [RFC8842], Section 5. If this is a re-offer or a response to a re-offer and the tls-id value is different from that presently in use, the DTLS connection is not being continued and the remote description MUST be part of an ICE restart, together with new ufrag and password values. 🔄
- A DTLS setup value, which MUST be set according to the rules specified in [RFC5763], Section 5 and MUST be consistent with the selected role of the current DTLS connection, if one exists and is being continued. 🤝
- DTLS fingerprint values, where at least one fingerprint MUST be present.
- All rid-ids referenced in an
a=simulcastline MUST exist asa=ridlines. 🆗 - Each "m=" section is also checked to ensure that prohibited features are not used. 🚫
- If the RTP/RTCP multiplexing policy is "require", each "m=" section MUST contain an
a=rtcp-muxattribute. If an "m=" section contains ana=rtcp-mux-onlyattribute, that section MUST also contain ana=rtcp-muxattribute. 🧷 - If an "m=" section was present in the previous answer, the state of RTP/RTCP multiplexing MUST match what was previously negotiated. 🪞
If this session description is of type "pranswer" or "answer", the following additional checks are applied:
- The session description MUST follow the rules defined in [RFC3264], Section 6, including the requirement that the number of "m=" sections MUST exactly match the number of "m=" sections in the associated offer. 🔢
- For each "m=" section, the media type and protocol values MUST exactly match the media type and protocol values in the corresponding "m=" section in the associated offer. 🎯
If any of the preceding checks failed, processing MUST stop and an error MUST be returned. 🛑💀
5.9. Applying a Local Description 🛠️
The following steps are performed at the media engine level to apply a local description. If an error is returned, the session MUST be restored to the state it was in before performing these steps. Undo button, babe ⏪
First, "m=" sections are processed. For each "m=" section, the following steps MUST be performed; if any parameters are out of bounds or cannot be applied, processing MUST stop and an error MUST be returned. 🧐
- If this "m=" section is new, begin gathering candidates for it, as defined in [RFC8445], Section 5.1.1, unless it is definitively being bundled (either (1) this is an offer and the "m=" section is marked as bundle-only or (2) it is an answer and the "m=" section is bundled into another "m=" section). 🌱
- Or, if the ICE ufrag and password values have changed, trigger the ICE agent to start an ICE restart as described in [RFC8445], Section 9, and begin gathering new candidates for the "m=" section. If this description is an answer, also start checks on that media section. 🔁
- If the "m=" section
<proto>value indicates use of RTP: 🎬- If there is no RtpTransceiver associated with this "m=" section, find one and associate it with this "m=" section according to the following steps. Note that this situation will only occur when applying an offer. 🧭
- Find the RtpTransceiver that corresponds to this "m=" section, using the mapping between transceivers and "m=" section indices established when creating the offer. 🗂️
- Set the value of this RtpTransceiver's mid property to the MID of the "m=" section. 🏷️
- If RTCP mux is indicated, prepare to demux RTP and RTCP from the RTP ICE component, as specified in [RFC5761], Section 5.1.3. 🧃
- For each specified RTP header extension, establish a mapping between the extension ID and URI, as described in [RFC5285], Section 6. 🔗
- If the MID header extension is supported, prepare to demux RTP streams intended for this "m=" section based on the MID header extension, as described in [RFC9143], Section 15. 🎧
- For each specified media format, establish a mapping between the payload type and the actual media format, as described in [RFC3264], Section 6.1. In addition, prepare to demux RTP streams intended for this "m=" section based on the media formats supported by this "m=" section, as described in [RFC9143], Section 9.2. 🎞️
- For each specified "rtx" media format, establish a mapping between the RTX payload type and its associated primary payload type, as described in Sections 8.6 and 8.7 of [RFC4588]. 🔁🎶
- If the direction attribute is of type "sendrecv" or "recvonly", enable receipt and decoding of media. 🎧✨
- If there is no RtpTransceiver associated with this "m=" section, find one and associate it with this "m=" section according to the following steps. Note that this situation will only occur when applying an offer. 🧭
Finally, if this description is of type "pranswer" or "answer", follow the processing defined in Section 5.11 below. Almost there, we're in the home stretch 🦋
5.10. Applying a Remote Description ✨
okay bestie, time to apply a remote description 💅 these steps run in order, and if an error comes back, the session MUST be restored to the state it was in before any of this started. rollback is the vibe when things go sideways 😭
If the answer has any a=ice-options attributes that list trickle, update the PeerConnection canTrickleIceCandidates property to true. Otherwise, set this property to false. 🦋
The session-level steps MUST be performed for attributes at the session level. If any parameters are out of bounds or cannot be applied, processing MUST stop and an error MUST be returned. 🚫
- For any specified
CTbandwidth value, set this value as the limit for the maximum total bitrate for allm=sections, as specified in [RFC4566], Section 5.8. Within this overall limit, the implementation can dynamically decide how to best allocate the available bandwidth betweenm=sections, respecting any specific limits that have been specified for individualm=sections. 🍓 - For any specified
RRorRSbandwidth values, handle as specified in [RFC3556], Section 2. - Any
ASbandwidth value ([RFC4566], Section 5.8) MUST be ignored, as the meaning of this construct at the session level is not well defined. 🙈
For each m= section, the following steps MUST be performed. If any parameters are out of bounds or cannot be applied, processing MUST stop and an error MUST be returned. 🌸
- If the ICE ufrag or password changed from the previous remote description:
- If the description is of type
offer, the implementation MUST note that an ICE restart is needed, as described in [RFC8839], Section 4.4.1.1.1. 🔥 - If the description is of type
answerorpranswer, then check to see if the current local description is an ICE restart, and if not, generate an error. If the PeerConnection state ishave-remote-pranswerand the ICE ufrag or password changed from the previous provisional answer, then signal the ICE agent to discard any previous ICE checklist state for them=section. Finally, signal the ICE agent to begin checks. ✨
- If the description is of type
- If the current local description indicates an ICE restart but neither the ICE ufrag nor the password has changed from the previous remote description (as prescribed by [RFC8445], Section 9), generate an error.
- Configure the ICE components associated with this media section to use the supplied ICE remote ufrag and password for their connectivity checks. 🔑
- Pair any supplied ICE candidates with any gathered local candidates, as described in [RFC8445], Section 6.1.2, and start connectivity checks with the appropriate credentials. 🧚
- If an
a=end-of-candidatesattribute is present, process the end-of-candidates indication as described in [RFC8838], Section 14. - If the
m=section<proto>value indicates use of RTP:- If the
m=section is being recycled (see Section 5.2.2), disassociate the currently associatedRtpTransceiverby setting itsmidproperty tonull, and discard the mapping between the transceiver and itsm=section index. 🔄 - If the
m=section is not associated with anyRtpTransceiver(possibly because it was disassociated in the previous step), either find anRtpTransceiveror create one according to the following steps:- If the
m=section issendrecvorrecvonly, and there areRtpTransceivers of the same type that were added to thePeerConnectionbyaddTrackand are not associated with anym=section and are not stopped, find the first (according to the canonical order described in Section 5.2.1) suchRtpTransceiver. 👑 - If no
RtpTransceiverwas found in the previous step, create one with arecvonlydirection. - Associate the found or created
RtpTransceiverwith them=section by setting the value of theRtpTransceiver'smidproperty to the MID of them=section, and establish a mapping between the transceiver and the index of them=section. If them=section does not include a MID (i.e., the remote endpoint does not support the MID extension), generate a value for theRtpTransceivermidproperty, following the guidance fora=midmentioned in Section 5.2.1. 🦋
- If the
- For each specified media format that is also supported by the local application, establish a mapping between the specified payload type and the media format, as described in [RFC3264], Section 6.1. Specifically, this means that the implementation records the payload type to be used in outgoing RTP packets when sending each specified media format, as well as the relative preference for each format that is indicated in their ordering. If any indicated media format is not supported by the local application, it MUST be ignored. 🎀
- For each specified
rtxmedia format, establish a mapping between the RTX payload type and its associated primary payload type, as described in [RFC4588], Section 4. If any referenced primary payload types are not present, this MUST result in an error. Note that RTX payload types may refer to primary payload types that are not supported by the local media implementation, in which case the RTX payload type MUST also be ignored. 😭 - For each specified fmtp parameter that is supported by the local application, enable them on the associated media formats. 💖
- For each specified Synchronization Source (SSRC) that is signaled in the
m=section, prepare to demux RTP streams intended for thism=section using that SSRC, as described in [RFC9143], Section 9.2. 🌈 - For each specified RTP header extension that is also supported by the local application, establish a mapping between the extension ID and URI, as described in [RFC5285], Section 5. Specifically, this means that the implementation records the extension ID to be used in outgoing RTP packets when sending each specified header extension. If any indicated RTP header extension is not supported by the local application, it MUST be ignored. 💜
- For each specified RTCP feedback mechanism that is also supported by the local application, enable them on the associated media formats. 🫶
- For any specified
TIAS("Transport Independent Application Specific (maximum)") bandwidth value, set this value as a constraint on the maximum RTP bitrate to be used when sending media, as specified in [RFC3890]. If aTIASvalue is not present but anASvalue is specified, generate aTIASvalue using this formula:
- If the
TIAS = AS * 1000 * 0.95 - (50 * 40 * 8)The 1000 changes the unit from kbps to bps (as required by TIAS), and the 0.95 is to allocate 5% to RTCP. An estimate of header overhead is then subtracted out, in which the 50 is based on 50 packets per second, the 40 is based on typical header size (in bytes), and the 8 converts bytes to bits. Note that TIAS is preferred over AS because it provides more accurate control of bandwidth. 🧮
- For any
RRorRSbandwidth values, handle as specified in [RFC3556], Section 2. - Any specified
CTbandwidth value MUST be ignored, as the meaning of this construct at the media level is not well defined. 🙈 - If the
m=section is of typeaudio:- For each specified
CNmedia format, configure silence suppression for all supported media formats with the same clock rate, as described in [RFC3389], Section 5, except for formats that have their own internal silence suppression mechanisms. Silence suppression for such formats (e.g., Opus) is controlled via fmtp parameters, as discussed in Section 5.2.3.2. 🤫 - For each specified
telephone-eventmedia format, enable dual-tone multifrequency (DTMF) transmission for all supported media formats with the same clock rate, as described in [RFC4733], Section 2.5.1.2. If there are any supported media formats that do not have a correspondingtelephone-eventformat, disable DTMF transmission for those formats. 📞 - For any specified
ptimevalue, configure the available media formats to use the specified packet size when sending. If the specified size is not supported for a media format, use the next closest value instead. ⏱️
- For each specified
Finally, if this description is of type pranswer or answer, follow the processing defined in Section 5.11 below. ✨🎀
5.11. Applying an Answer 💅✨
okay bestie so on top of all the steps we already covered for processing a local or remote description 🧚, these extra steps kick in when we're processing a description of type "pranswer" or "answer" 💖
For each "m=" section, the following steps MUST be performed:
- If the
"m="section has been rejected (i.e., the<port>value is set to zero in the answer), stop any reception or transmission of media for this section, and, unless a non-rejected"m="section is bundled with this"m="section, discard any associated ICE components, as described in the second bullet item in [RFC8839], Section 4.4.3.1. 🙅♀️ - If the remote DTLS fingerprint has been changed or the value of
a=tls-idhas changed, tear down the DTLS connection. This includes the case when the PeerConnection state is"have-remote-pranswer". If a DTLS connection needs to be torn down but the answer does not indicate an ICE restart or, in the case of"have-remote-pranswer", new ICE credentials, an error MUST be generated. If an ICE restart is performed without a change in the tls-id value or fingerprint, then the same DTLS connection is continued over the new ICE channel. Note that although JSEP requires that answerers change the tls-id value if and only if the offerer does, non-JSEP answerers are permitted to change the tls-id value as long as the offer contained an ICE restart. Thus, JSEP implementations that process DTLS data prior to receiving an answer MUST be prepared to receive either a ClientHello or data from the previous DTLS connection. 🔐🎀 - If no valid DTLS connection exists, prepare to start a DTLS connection, using the specified roles and fingerprints, on any underlying ICE components, once they are active. 🌸
- If the
"m="section<proto>value indicates use of RTP:- If the
"m="section references RTCP feedback mechanisms that were not present in the corresponding"m="section in the offer, this indicates a negotiation problem and MUST result in an error. However, new media formats and new RTP header extension values are permitted in the answer, as described in [RFC3264], Section 7 and [RFC5285], Section 6. 🚫 - If the
"m="section has RTCP mux enabled, discard the RTCP ICE component, if one exists, and begin or continue muxing RTCP over the RTP ICE component, as specified in [RFC5761], Section 5.1.3. Otherwise, prepare to transmit RTCP over the RTCP ICE component; if no RTCP ICE component exists because RTCP mux was previously enabled, this MUST result in an error. 🔁 - If the
"m="section has Reduced-Size RTCP enabled, configure the RTCP transmission for this"m="section to use Reduced-Size RTCP, as specified in [RFC5506]. 🍓 - If the direction attribute in the answer indicates that the JSEP implementation should be sending media (
"sendonly"for local answers,"recvonly"for remote answers, or"sendrecv"for either type of answer), choose the media format to send as the most preferred media format from the remote description that is also locally supported, as discussed in Sections 6.1 and 7 of [RFC3264], and start transmitting RTP media using that format once the underlying transport layers have been established. If an SSRC has not already been chosen for this outgoing RTP stream, choose a unique random one. If media is already being transmitted, the same SSRC SHOULD be used unless the clock rate of the new codec is different, in which case a new SSRC MUST be chosen, as specified in [RFC7160], Section 4.1. 🎙️ - The payload type mapping from the remote description is used to determine payload types for the outgoing RTP streams, including the payload type for the send media format chosen above. Any RTP header extensions that were negotiated should be included in the outgoing RTP streams, using the extension mapping from the remote description. If the MID header extension has been negotiated, include it in the outgoing RTP streams, as indicated in [RFC9143], Section 15. If the RtpStreamId or RepairedRtpStreamId header extensions have been negotiated and rid-ids have been established, include these header extensions in the outgoing RTP streams, as indicated in [RFC8851], Section 4. 🧩
- If the
"m="section is of type"audio", and silence suppression was (1) configured for the send media format as a result of processing the remote description and (2) also enabled for that format in the local description, use silence suppression for outgoing media, in accordance with the guidance in Section 5.2.3.2. If these conditions are not met, silence suppression MUST NOT be used for outgoing media. 🤫 - If simulcast has been negotiated, send the appropriate number of Source RTP Streams as specified in [RFC8853], Section 5.3.3. 📺
- If the send media format chosen above has a corresponding
"rtx"media format or a FEC mechanism has been negotiated, establish a redundancy RTP stream with a unique random SSRC for each Source RTP Stream, and start or continue transmitting RTX/FEC packets as needed. 🛟 - If the send media format chosen above has a corresponding
"red"media format of the same clock rate, allow redundant encoding using the specified format for resiliency purposes, as discussed in [RFC8854], Section 3.2. Note that unlike RTX or FEC media formats, the"red"format is transmitted on the Source RTP Stream, not the redundancy RTP stream. 🍒 - Enable the RTCP feedback mechanisms referenced in the media section for all Source RTP Streams using the specified media formats. Specifically, begin or continue sending the requested feedback types and reacting to received feedback, as specified in [RFC4585], Section 4.2. When sending RTCP feedback, follow the rules and recommendations from [RFC8108], Section 5.4.1 to select which SSRC to use. 💬
- If the direction attribute in the answer indicates that the JSEP implementation should not be sending media (
"recvonly"for local answers,"sendonly"for remote answers, or"inactive"for either type of answer), stop transmitting all RTP media, but continue sending RTCP, as described in [RFC3264], Section 5.1. 🛑
- If the
- If the
"m="section<proto>value indicates use of SCTP:- If an SCTP association exists and the remote SCTP port has changed, discard the existing SCTP association. This includes the case when the PeerConnection state is
"have-remote-pranswer". 🔄 - If no valid SCTP association exists, prepare to initiate an SCTP association over the associated ICE component and DTLS connection, using the local SCTP port value from the local description and the remote SCTP port value from the remote description, as described in [RFC8841], Section 10.2. 🦋
- If an SCTP association exists and the remote SCTP port has changed, discard the existing SCTP association. This includes the case when the PeerConnection state is
If the answer contains valid bundle groups, discard any ICE components for the "m=" sections that will be bundled onto the primary ICE components in each bundle, and begin muxing these "m=" sections accordingly, as described in [RFC9143], Section 7.4. 📦
If the description is of type "answer" and there are still remaining candidates in the ICE candidate pool, discard them. 🧹
6. Processing RTP/RTCP 🦋
When bundling, associating incoming RTP/RTCP with the proper "m=" section is defined in [RFC9143], Section 9.2. When not bundling, the proper "m=" section is clear from the ICE component over which the RTP/RTCP is received. 📡
Once the proper "m=" section or sections are known, RTP/RTCP is delivered to the RtpTransceiver(s) associated with the "m=" section(s) and further processing of the RTP/RTCP is done at the RtpTransceiver level. This includes using the RID mechanism [RFC8851] and its associated RtpStreamId and RepairedRtpStreamId identifiers to distinguish between multiple encoded streams and determine which Source RTP Stream should be repaired by a given redundancy RTP stream. 🌈
7. Examples 🧚♀️
Note that this example section shows several SDP fragments. To accommodate RFC line-length restrictions, some of the SDP lines have been split into multiple lines, where leading whitespace indicates that a line is a continuation of the previous line. In addition, some blank lines have been added to improve readability but are not valid in SDP. 📝
More examples of SDP for WebRTC call flows, including examples with IPv6 addresses, can be found in [SDP4WebRTC]. 💜
7.1. Simple Example ✨
okay bestie, this section is the tiny starter call 💖 it shows a super minimal audio/video call between two JSEP endpoints, and it does NOT use Trickle ICE 🙅♀️ the next section is the longer, more detailed version, so stay tuned 🦋
the flow below is Alice's endpoint starting the session with Bob's endpoint 📞 the messages between the JavaScript in Alice's browser and the JavaScript in Bob's browser (abbreviated "AliceJS" and "BobJS") are assumed to zip over some signaling protocol through a web server 🌸 both JS sides wait for all the candidates before sending the offer or answer, so the offers and answers are complete and Trickle ICE is not used. no trickle, just vibes 🫶 the user agents (the JSEP implementations in the browsers, "AliceUA" and "BobUA") are both on the default bundle policy of "balanced" and the default RTCP mux policy of "require" 💅
// set up local media state
AliceJS->AliceUA: create new PeerConnection
AliceJS->AliceUA: addTrack with two tracks: audio and video
AliceJS->AliceUA: createOffer to get offer
AliceJS->AliceUA: setLocalDescription with offer
AliceUA->AliceJS: multiple onicecandidate events with candidates
// wait for ICE gathering to complete
AliceUA->AliceJS: onicecandidate event with null candidate
AliceJS->AliceUA: get |offer-A1| from pendingLocalDescription
// |offer-A1| is sent over signaling protocol to Bob
AliceJS->WebServer: signaling with |offer-A1|
WebServer->BobJS: signaling with |offer-A1|
// |offer-A1| arrives at Bob
BobJS->BobUA: create a PeerConnection
BobJS->BobUA: setRemoteDescription with |offer-A1|
BobUA->BobJS: ontrack events for audio and video tracks
// Bob accepts call
BobJS->BobUA: addTrack with local tracks
BobJS->BobUA: createAnswer
BobJS->BobUA: setLocalDescription with answer
BobUA->BobJS: multiple onicecandidate events with candidates
// wait for ICE gathering to complete
BobUA->BobJS: onicecandidate event with null candidate
BobJS->BobUA: get |answer-A1| from currentLocalDescription
// |answer-A1| is sent over signaling protocol
// to Alice
BobJS->WebServer: signaling with |answer-A1|
WebServer->AliceJS: signaling with |answer-A1|
// |answer-A1| arrives at Alice
AliceJS->AliceUA: setRemoteDescription with |answer-A1|
AliceUA->AliceJS: ontrack events for audio and video tracks
// media flows
BobUA->AliceUA: media sent from Bob to Alice
AliceUA->BobUA: media sent from Alice to Bobthe SDP for |offer-A1| looks like this 🎀
v=0
o=- 4962303333179871722 1 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle ice2
a=group:BUNDLE a1 v1
a=group:LS a1 v1
m=audio 10100 UDP/TLS/RTP/SAVPF 96 0 8 97 98
c=IN IP4 203.0.113.100
a=mid:a1
a=sendrecv
a=rtpmap:96 opus/48000/2
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 telephone-event/8000
a=rtpmap:98 telephone-event/48000
a=fmtp:97 0-15
a=fmtp:98 0-15
a=maxptime:120
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=msid:47017fee-b6c1-4162-929c-a25110252400
a=ice-ufrag:ETEn
a=ice-pwd:OtSK0WpNtpUjkY4+86js7ZQl
a=fingerprint:sha-256
19:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04:
BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2
a=setup:actpass
a=tls-id:91bbf309c0990a6bec11e38ba2933cee
a=rtcp:10101 IN IP4 203.0.113.100
a=rtcp-mux
a=rtcp-rsize
a=candidate:1 1 udp 2113929471 203.0.113.100 10100 typ host
a=candidate:1 2 udp 2113929470 203.0.113.100 10101 typ host
a=end-of-candidates
m=video 10102 UDP/TLS/RTP/SAVPF 100 101 102 103
c=IN IP4 203.0.113.100
a=mid:v1
a=sendrecv
a=rtpmap:100 VP8/90000
a=rtpmap:101 H264/90000
a=fmtp:101 packetization-mode=1;profile-level-id=42e01f
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=100
a=rtpmap:103 rtx/90000
a=fmtp:103 apt=101
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=msid:47017fee-b6c1-4162-929c-a25110252400
a=ice-ufrag:BGKk
a=ice-pwd:mqyWsAjvtKwTGnvhPztQ9mIf
a=fingerprint:sha-256
19:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04:
BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2
a=setup:actpass
a=tls-id:91bbf309c0990a6bec11e38ba2933cee
a=rtcp:10103 IN IP4 203.0.113.100
a=rtcp-mux
a=rtcp-rsize
a=candidate:1 1 udp 2113929471 203.0.113.100 10102 typ host
a=candidate:1 2 udp 2113929470 203.0.113.100 10103 typ host
a=end-of-candidatesthe SDP for |answer-A1| looks like this 🌈
v=0
o=- 6729291447651054566 1 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle ice2
a=group:BUNDLE a1 v1
a=group:LS a1 v1
m=audio 10200 UDP/TLS/RTP/SAVPF 96 0 8 97 98
c=IN IP4 203.0.113.200
a=mid:a1
a=sendrecv
a=rtpmap:96 opus/48000/2
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 telephone-event/8000
a=rtpmap:98 telephone-event/48000
a=fmtp:97 0-15
a=fmtp:98 0-15
a=maxptime:120
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=msid:61317484-2ed4-49d7-9eb7-1414322a7aae
a=ice-ufrag:6sFv
a=ice-pwd:cOTZKZNVlO9RSGsEGM63JXT2
a=fingerprint:sha-256
6B:8B:F0:65:5F:78:E2:51:3B:AC:6F:F3:3F:46:1B:35:
DC:B8:5F:64:1A:24:C2:43:F0:A1:58:D0:A1:2C:19:08
a=setup:active
a=tls-id:eec3392ab83e11ceb6a0990c903fbb19
a=rtcp-mux
a=rtcp-rsize
a=candidate:1 1 udp 2113929471 203.0.113.200 10200 typ host
a=end-of-candidates
m=video 10200 UDP/TLS/RTP/SAVPF 100 101 102 103
c=IN IP4 203.0.113.200
a=mid:v1
a=sendrecv
a=rtpmap:100 VP8/90000
a=rtpmap:101 H264/90000
a=fmtp:101 packetization-mode=1;profile-level-id=42e01f
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=100
a=rtpmap:103 rtx/90000
a=fmtp:103 apt=101
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=msid:61317484-2ed4-49d7-9eb7-1414322a7aae7.2. Detailed Example 🎀
this section is a more involved session between two JSEP endpoints 🧠 trickle ICE is used in full trickle mode, with a bundle policy of "must-bundle", an RTCP mux policy of "require", and a single TURN server 🔥 at first, Alice and Bob both set up an audio channel and a data channel 🎧 later, Bob adds two video flows, one for his video feed and one for screen sharing, both supporting FEC 📺 and his video feed is configured for simulcast 👑 Alice accepts these video flows but doesn't add any video of her own, so they get handled as "recvonly" 👀 Alice also specifies a maximum video decoder resolution 🌈
// set up local media state
AliceJS->AliceUA: create new PeerConnection
AliceJS->AliceUA: addTrack with an audio track
AliceJS->AliceUA: createDataChannel to get data channel
AliceJS->AliceUA: createOffer to get |offer-B1|
AliceJS->AliceUA: setLocalDescription with |offer-B1|
// |offer-B1| is sent over signaling protocol to Bob
AliceJS->WebServer: signaling with |offer-B1|
WebServer->BobJS: signaling with |offer-B1|
// |offer-B1| arrives at Bob
BobJS->BobUA: create a PeerConnection
BobJS->BobUA: setRemoteDescription with |offer-B1|
BobUA->BobJS: ontrack event with audio track from Alice
// candidates are sent to Bob
AliceUA->AliceJS: onicecandidate (host) |offer-B1-candidate-1|
AliceJS->WebServer: signaling with |offer-B1-candidate-1|
AliceUA->AliceJS: onicecandidate (srflx) |offer-B1-candidate-2|
AliceJS->WebServer: signaling with |offer-B1-candidate-2|
AliceUA->AliceJS: onicecandidate (relay) |offer-B1-candidate-3|
AliceJS->WebServer: signaling with |offer-B1-candidate-3|
WebServer->BobJS: signaling with |offer-B1-candidate-1|
BobJS->BobUA: addIceCandidate with |offer-B1-candidate-1|
WebServer->BobJS: signaling with |offer-B1-candidate-2|
BobJS->BobUA: addIceCandidate with |offer-B1-candidate-2|
WebServer->BobJS: signaling with |offer-B1-candidate-3|
BobJS->BobUA: addIceCandidate with |offer-B1-candidate-3|
// Bob accepts call
BobJS->BobUA: addTrack with local audio
BobJS->BobUA: createDataChannel to get data channel
BobJS->BobUA: createAnswer to get |answer-B1|
BobJS->BobUA: setLocalDescription with |answer-B1|
// |answer-B1| is sent to Alice
BobJS->WebServer: signaling with |answer-B1|
WebServer->AliceJS: signaling with |answer-B1|
AliceJS->AliceUA: setRemoteDescription with |answer-B1|
AliceUA->AliceJS: ontrack event with audio track from Bob
// candidates are sent to Alice
BobUA->BobJS: onicecandidate (host) |answer-B1-candidate-1|
BobJS->WebServer: signaling with |answer-B1-candidate-1|
BobUA->BobJS: onicecandidate (srflx) |answer-B1-candidate-2|
BobJS->WebServer: signaling with |answer-B1-candidate-2|
BobUA->BobJS: onicecandidate (relay) |answer-B1-candidate-3|
BobJS->WebServer: signaling with |answer-B1-candidate-3|
WebServer->AliceJS: signaling with |answer-B1-candidate-1|
AliceJS->AliceUA: addIceCandidate with |answer-B1-candidate-1|
WebServer->AliceJS: signaling with |answer-B1-candidate-2|
AliceJS->AliceUA: addIceCandidate with |answer-B1-candidate-2|
WebServer->AliceJS: signaling with |answer-B1-candidate-3|
AliceJS->AliceUA: addIceCandidate with |answer-B1-candidate-3|
// data channel opens
BobUA->BobJS: ondatachannel event
AliceUA->AliceJS: ondatachannel event
BobUA->BobJS: onopen
AliceUA->AliceJS: onopen
// media is flowing between endpoints
BobUA->AliceUA: audio+data sent from Bob to Alice
AliceUA->BobUA: audio+data sent from Alice to Bob
// some time later, Bob adds two video streams
// note: no candidates exchanged, because of bundle
BobJS->BobUA: addTrack with first video stream
BobJS->BobUA: addTrack with second video stream
BobJS->BobUA: createOffer to get |offer-B2|
BobJS->BobUA: setLocalDescription with |offer-B2|
// |offer-B2| is sent to Alice
BobJS->WebServer: signaling with |offer-B2|
WebServer->AliceJS: signaling with |offer-B2|
AliceJS->AliceUA: setRemoteDescription with |offer-B2|
AliceUA->AliceJS: ontrack event with first video track
AliceUA->AliceJS: ontrack event with second video track
AliceJS->AliceUA: createAnswer to get |answer-B2|
AliceJS->AliceUA: setLocalDescription with |answer-B2|
// |answer-B2| is sent over signaling protocol
// to Bob
AliceJS->WebServer: signaling with |answer-B2|
WebServer->BobJS: signaling with |answer-B2|
BobJS->BobUA: setRemoteDescription with |answer-B2|
// media is flowing between endpoints
BobUA->AliceUA: audio+video+data sent from Bob to Alice
AliceUA->BobUA: audio+video+data sent from Alice to Bobthe SDP for |offer-B1| looks like this 💖
v=0
o=- 4962303333179871723 1 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle ice2
a=group:BUNDLE a1 d1
m=audio 9 UDP/TLS/RTP/SAVPF 96 0 8 97 98
c=IN IP4 0.0.0.0
a=mid:a1
a=sendrecv
a=rtpmap:96 opus/48000/2
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 telephone-event/8000
a=rtpmap:98 telephone-event/48000
a=fmtp:97 0-15
a=fmtp:98 0-15
a=maxptime:120
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=msid:57017fee-b6c1-4162-929c-a25110252400
a=ice-ufrag:ATEn
a=ice-pwd:AtSK0WpNtpUjkY4+86js7ZQl
a=fingerprint:sha-256
29:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04:
BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2
a=setup:actpass
a=tls-id:17f0f4ba8a5f1213faca591b58ba52a7
a=rtcp-mux
a=rtcp-mux-only
a=rtcp-rsize
m=application 0 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 0.0.0.0
a=mid:d1
a=sctp-port:5000
a=max-message-size:65536
a=bundle-onlyNext up, the three candidates that offer B1 gathers, and they're all sitting on the same ufrag and mid, so they're a whole crew 🌸. |offer-B1-candidate-1| looks like this:
ufrag ATEn
index 0
mid a1
attr candidate:1 1 udp 2113929471 203.0.113.100 10100 typ host|offer-B1-candidate-2| looks like this:
ufrag ATEn
index 0
mid a1
attr candidate:1 1 udp 1845494015 198.51.100.100 11100 typ srflx
raddr 203.0.113.100 rport 10100And |offer-B1-candidate-3| looks like this:
ufrag ATEn
index 0
mid a1
attr candidate:1 1 udp 255 192.0.2.100 12100 typ relay
raddr 198.51.100.100 rport 11100Now the answer SDP for |answer-B1| 💅✨:
v=0
o=- 7729291447651054566 1 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle ice2
a=group:BUNDLE a1 d1
m=audio 9 UDP/TLS/RTP/SAVPF 96 0 8 97 98
c=IN IP4 0.0.0.0
a=mid:a1
a=sendrecv
a=rtpmap:96 opus/48000/2
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 telephone-event/8000
a=rtpmap:98 telephone-event/48000
a=fmtp:97 0-15
a=fmtp:98 0-15
a=maxptime:120
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=msid:71317484-2ed4-49d7-9eb7-1414322a7aae
a=ice-ufrag:7sFv
a=ice-pwd:dOTZKZNVlO9RSGsEGM63JXT2
a=fingerprint:sha-256
7B:8B:F0:65:5F:78:E2:51:3B:AC:6F:F3:3F:46:1B:35:
DC:B8:5F:64:1A:24:C2:43:F0:A1:58:D0:A1:2C:19:08
a=setup:active
a=tls-id:7a25ab85b195acaf3121f5a8ab4f0f71
a=rtcp-mux
a=rtcp-mux-only
a=rtcp-rsize
m=application 9 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 0.0.0.0
a=mid:d1
a=sctp-port:5000
a=max-message-size:65536The answerer has its own three candidates too, so here they are 🦋. |answer-B1-candidate-1| looks like this:
ufrag 7sFv
index 0
mid a1
attr candidate:1 1 udp 2113929471 203.0.113.200 10200 typ host|answer-B1-candidate-2| looks like this:
ufrag 7sFv
index 0
mid a1
attr candidate:1 1 udp 1845494015 198.51.100.200 11200 typ srflx
raddr 203.0.113.200 rport 10200And |answer-B1-candidate-3| looks like this:
ufrag 7sFv
index 0
mid a1
attr candidate:1 1 udp 255 192.0.2.200 12200 typ relay
raddr 198.51.100.200 rport 11200Okay, the SDP for |offer-B2| is next and it's giving main character energy 👑🎀. Besides the brand new video m= sections (both offering FEC, and one of them offering simulcast too 🔥), look at what else changed. The version number in the o= line got bumped, the c= line now shows which local candidate got picked ✨, and the gathered candidates show up as a=candidate lines 🧚.
v=0
o=- 7729291447651054566 2 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle ice2
a=group:BUNDLE a1 d1 v1 v2
a=group:LS a1 v1
m=audio 12200 UDP/TLS/RTP/SAVPF 96 0 8 97 98
c=IN IP4 192.0.2.200
a=mid:a1
a=sendrecv
a=rtpmap:96 opus/48000/2
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 telephone-event/8000
a=rtpmap:98 telephone-event/48000
a=fmtp:97 0-15
a=fmtp:98 0-15
a=maxptime:120
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=msid:71317484-2ed4-49d7-9eb7-1414322a7aae
a=ice-ufrag:7sFv
a=ice-pwd:dOTZKZNVlO9RSGsEGM63JXT2
a=fingerprint:sha-256
7B:8B:F0:65:5F:78:E2:51:3B:AC:6F:F3:3F:46:1B:35:
DC:B8:5F:64:1A:24:C2:43:F0:A1:58:D0:A1:2C:19:08
a=setup:actpass
a=tls-id:7a25ab85b195acaf3121f5a8ab4f0f71
a=rtcp-mux
a=rtcp-mux-only
a=rtcp-rsize
a=candidate:1 1 udp 2113929471 203.0.113.200 10200 typ host
a=candidate:1 1 udp 1845494015 198.51.100.200 11200 typ srflx
raddr 203.0.113.200 rport 10200
a=candidate:1 1 udp 255 192.0.2.200 12200 typ relay
raddr 198.51.100.200 rport 11200
a=end-of-candidates
m=application 12200 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 192.0.2.200
a=mid:d1
a=sctp-port:5000
a=max-message-size:65536
m=video 12200 UDP/TLS/RTP/SAVPF 100 101 102 103 104
c=IN IP4 192.0.2.200
a=mid:v1
a=sendrecv
a=rtpmap:100 VP8/90000
a=rtpmap:101 H264/90000
a=fmtp:101 packetization-mode=1;profile-level-id=42e01f
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=100
a=rtpmap:103 rtx/90000
a=fmtp:103 apt=101
a=rtpmap:104 flexfec/90000
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=msid:71317484-2ed4-49d7-9eb7-1414322a7aae
a=rid:1 send
a=rid:2 send
a=rid:3 send
a=simulcast:send 1;2;3
m=video 12200 UDP/TLS/RTP/SAVPF 100 101 102 103 104
c=IN IP4 192.0.2.200
a=mid:v2
a=sendrecv
a=rtpmap:100 VP8/90000
a=rtpmap:101 H264/90000
a=fmtp:101 packetization-mode=1;profile-level-id=42e01f
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=100
a=rtpmap:103 rtx/90000
a=fmtp:103 apt=101
a=rtpmap:104 flexfec/90000
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=msid:81317484-2ed4-49d7-9eb7-1414322a7aaeAnd last but not least, the SDP for |answer-B2| 🌈💜. The answerer accepts both video m= sections, uses a=recvonly to say "one-way video only, bestie" 🙈, and uses a=imageattr to cap the received resolution. Also peep the a=setup:passive line, which keeps the existing DTLS roles exactly where they were 🫶.
v=0
o=- 4962303333179871723 2 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle ice2
a=group:BUNDLE a1 d1 v1 v2
a=group:LS a1 v1
m=audio 12100 UDP/TLS/RTP/SAVPF 96 0 8 97 98
c=IN IP4 192.0.2.100
a=mid:a1
a=sendrecv
a=rtpmap:96 opus/48000/2
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 telephone-event/8000
a=rtpmap:98 telephone-event/48000
a=fmtp:97 0-15
a=fmtp:98 0-15
a=maxptime:120
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=msid:57017fee-b6c1-4162-929c-a25110252400
a=ice-ufrag:ATEn
a=ice-pwd:AtSK0WpNtpUjkY4+86js7ZQl
a=fingerprint:sha-256
29:E2:1C:3B:4B:9F:81:E6:B8:5C:F4:A5:A8:D8:73:04:
BB:05:2F:70:9F:04:A9:0E:05:E9:26:33:E8:70:88:A2
a=setup:passive
a=tls-id:17f0f4ba8a5f1213faca591b58ba52a7
a=rtcp-mux
a=rtcp-mux-only
a=rtcp-rsize
a=candidate:1 1 udp 2113929471 203.0.113.100 10100 typ host
a=candidate:1 1 udp 1845494015 198.51.100.100 11100 typ srflx
raddr 203.0.113.100 rport 10100
a=candidate:1 1 udp 255 192.0.2.100 12100 typ relay
raddr 198.51.100.100 rport 11100
a=end-of-candidates
m=application 12100 UDP/DTLS/SCTP webrtc-datachannel
c=IN IP4 192.0.2.100
a=mid:d1
a=sctp-port:5000
a=max-message-size:65536
m=video 12100 UDP/TLS/RTP/SAVPF 100 101 102 103
c=IN IP4 192.0.2.100
a=mid:v1
a=recvonly
a=rtpmap:100 VP8/90000
a=rtpmap:101 H264/90000
a=fmtp:101 packetization-mode=1;profile-level-id=42e01f
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=100
a=rtpmap:103 rtx/90000
a=fmtp:103 apt=101
a=imageattr:100 recv [x=[48:1920],y=[48:1080],q=1.0]
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
m=video 12100 UDP/TLS/RTP/SAVPF 100 101 102 103
c=IN IP4 192.0.2.100
a=mid:v2
a=recvonly
a=rtpmap:100 VP8/90000
a=rtpmap:101 H264/90000
a=fmtp:101 packetization-mode=1;profile-level-id=42e01f
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=100
a=rtpmap:103 rtx/90000
a=fmtp:103 apt=101
a=imageattr:100 recv [x=[48:1920],y=[48:1080],q=1.0]
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli7.3. Early Transport Warmup Example ✨
Okay so this one shows off the early-warmup trick from Section 4.1.10.1 and it's giving main character energy 💖. Alice's endpoint sends an offer to Bob's endpoint to kick off an audio/video call 📞. Bob instantly replies with an answer that accepts the audio/video "m=" sections but marks them as "sendonly" (from his side), which basically means Alice won't send media yet 🙈. This lets the JSEP implementation start negotiating ICE and DTLS right away ✨. Then Bob's endpoint asks him to answer the call, and when he does, his endpoint sends a second offer that enables the audio and video "m=" sections, so media goes both ways 🦋. The big win is that once that first answer lands, the implementation can already do ICE and DTLS negotiation and set up the session transport. If transport setup finishes before the second offer goes out, the callee can transmit media the moment they pick up, which keeps the perceived post-dial delay super low 💅. The second offer/answer exchange can also swap the preferred codecs or other session parameters 🎀.
This example also uses the "relay" ICE candidate policy from Section 3.5.3 to cut down on the ICE gathering and checking needed 🧚.
Here's the whole call flow, bestie 🍓:
// set up local media state
AliceJS->AliceUA: create new PeerConnection with "relay" ICE policy
AliceJS->AliceUA: addTrack with two tracks: audio and video
AliceJS->AliceUA: createOffer to get |offer-C1|
AliceJS->AliceUA: setLocalDescription with |offer-C1|
// |offer-C1| is sent over signaling protocol to Bob
AliceJS->WebServer: signaling with |offer-C1|
WebServer->BobJS: signaling with |offer-C1|
// |offer-C1| arrives at Bob
BobJS->BobUA: create new PeerConnection with "relay" ICE policy
BobJS->BobUA: setRemoteDescription with |offer-C1|
BobUA->BobJS: ontrack events for audio and video
// a relay candidate is sent to Bob
AliceUA->AliceJS: onicecandidate (relay) |offer-C1-candidate-1|
AliceJS->WebServer: signaling with |offer-C1-candidate-1|
WebServer->BobJS: signaling with |offer-C1-candidate-1|
BobJS->BobUA: addIceCandidate with |offer-C1-candidate-1|
// Bob prepares an early answer to warm up the
// transport
BobJS->BobUA: addTransceiver with null audio and video tracks
BobJS->BobUA: transceiver.setDirection(sendonly) for both
BobJS->BobUA: createAnswer
BobJS->BobUA: setLocalDescription with answer
// |answer-C1| is sent over signaling protocol
// to Alice
BobJS->WebServer: signaling with |answer-C1|
WebServer->AliceJS: signaling with |answer-C1|
// |answer-C1| (sendonly) arrives at Alice
AliceJS->AliceUA: setRemoteDescription with |answer-C1|
AliceUA->AliceJS: ontrack events for audio and video
// a relay candidate is sent to Alice
BobUA->BobJS: onicecandidate (relay) |answer-B1-candidate-1|
BobJS->WebServer: signaling with |answer-B1-candidate-1|
WebServer->AliceJS: signaling with |answer-B1-candidate-1|
AliceJS->AliceUA: addIceCandidate with |answer-B1-candidate-1|
// ICE and DTLS establish while call is ringing
// Bob accepts call, starts media, and sends
// new offer
BobJS->BobUA: transceiver.setTrack with audio and video tracks
BobUA->AliceUA: media sent from Bob to Alice
BobJS->BobUA: transceiver.setDirection(sendrecv) for both
transceivers
BobJS->BobUA: createOffer
BobJS->BobUA: setLocalDescription with offer
// |offer-C2| is sent over signaling protocol
// to Alice
BobJS->WebServer: signaling with |offer-C2|
WebServer->AliceJS: signaling with |offer-C2|
// |offer-C2| (sendrecv) arrives at Alice
AliceJS->AliceUA: setRemoteDescription with |offer-C2|
AliceJS->AliceUA: createAnswer
AliceJS->AliceUA: setLocalDescription with |answer-C2|
AliceUA->BobUA: media sent from Alice to Bob
// |answer-C2| is sent over signaling protocol
// to Bob
AliceJS->WebServer: signaling with |answer-C2|
WebServer->BobJS: signaling with |answer-C2|
BobJS->BobUA: setRemoteDescription with |answer-C2|The SDP for |offer-C1| looks like:
v=0
o=- 1070771854436052752 1 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle ice2
a=group:BUNDLE a1 v1
a=group:LS a1 v1
m=audio 9 UDP/TLS/RTP/SAVPF 96 0 8 97 98
c=IN IP4 0.0.0.0
a=mid:a1
a=sendrecv
a=rtpmap:96 opus/48000/2
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 telephone-event/8000
a=rtpmap:98 telephone-event/48000
a=fmtp:97 0-15
a=fmtp:98 0-15
a=maxptime:120
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=msid:bbce3ba6-abfc-ac63-d00a-e15b286f8fce
a=ice-ufrag:4ZcD
a=ice-pwd:ZaaG6OG7tCn4J/lehAGz+HHD
a=fingerprint:sha-256
C4:68:F8:77:6A:44:F1:98:6D:7C:9F:47:EB:E3:34:A4:
0A:AA:2D:49:08:28:70:2E:1F:AE:18:7D:4E:3E:66:BF
a=setup:actpass
a=tls-id:9e5b948ade9c3d41de6617b68f769e55
a=rtcp-mux
a=rtcp-mux-only
a=rtcp-rsize
m=video 0 UDP/TLS/RTP/SAVPF 100 101 102 103
c=IN IP4 0.0.0.0
a=mid:v1
a=sendrecv
a=rtpmap:100 VP8/90000
a=rtpmap:101 H264/90000
a=fmtp:101 packetization-mode=1;profile-level-id=42e01f
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=100
a=rtpmap:103 rtx/90000
a=fmtp:103 apt=101
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=msid:bbce3ba6-abfc-ac63-d00a-e15b286f8fce
a=bundle-only| offer-C1-candidate-1 | looks like: |
ufrag 4ZcD
index 0
mid a1
attr candidate:1 1 udp 255 192.0.2.100 12100 typ relay
raddr 0.0.0.0 rport 0The SDP for |answer-C1| looks like:
v=0
o=- 6386516489780559513 1 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle ice2
a=group:BUNDLE a1 v1
a=group:LS a1 v1
m=audio 9 UDP/TLS/RTP/SAVPF 96 0 8 97 98
c=IN IP4 0.0.0.0
a=mid:a1
a=sendonly
a=rtpmap:96 opus/48000/2
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 telephone-event/8000
a=rtpmap:98 telephone-event/48000
a=fmtp:97 0-15
a=fmtp:98 0-15
a=maxptime:120
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=msid:751f239e-4ae0-c549-aa3d-890de772998b
a=ice-ufrag:TpaA
a=ice-pwd:t2Ouhc67y8JcCaYZxUUTgKw/
a=fingerprint:sha-256
A2:F3:A5:6D:4C:8C:1E:B2:62:10:4A:F6:70:61:C4:FC:
3C:E0:01:D6:F3:24:80:74:DA:7C:3E:50:18:7B:CE:4D
a=setup:active
a=tls-id:55e967f86b7166ed14d3c9eda849b5e9
a=rtcp-mux
a=rtcp-mux-only
a=rtcp-rsize
m=video 9 UDP/TLS/RTP/SAVPF 100 101 102 103
c=IN IP4 0.0.0.0
a=mid:v1
a=sendonly
a=rtpmap:100 VP8/90000
a=rtpmap:101 H264/90000
a=fmtp:101 packetization-mode=1;profile-level-id=42e01f
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=100
a=rtpmap:103 rtx/90000
a=fmtp:103 apt=101
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=msid:751f239e-4ae0-c549-aa3d-890de772998b| answer-C1-candidate-1 | looks like: |
ufrag TpaA
index 0
mid a1
attr candidate:1 1 udp 255 192.0.2.200 12200 typ relay
raddr 0.0.0.0 rport 0The SDP for |offer-C2| looks like:
v=0
o=- 6386516489780559513 2 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle ice2
a=group:BUNDLE a1 v1
a=group:LS a1 v1
m=audio 12200 UDP/TLS/RTP/SAVPF 96 0 8 97 98
c=IN IP4 192.0.2.200
a=mid:a1
a=sendrecv
a=rtpmap:96 opus/48000/2
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 telephone-event/8000
a=rtpmap:98 telephone-event/48000
a=fmtp:97 0-15
a=fmtp:98 0-15
a=maxptime:120
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=msid:751f239e-4ae0-c549-aa3d-890de772998b
a=ice-ufrag:TpaA
a=ice-pwd:t2Ouhc67y8JcCaYZxUUTgKw/
a=fingerprint:sha-256
A2:F3:A5:6D:4C:8C:1E:B2:62:10:4A:F6:70:61:C4:FC:
3C:E0:01:D6:F3:24:80:74:DA:7C:3E:50:18:7B:CE:4D
a=setup:actpass
a=tls-id:55e967f86b7166ed14d3c9eda849b5e9
a=rtcp-mux
a=rtcp-mux-only
a=rtcp-rsize
a=candidate:1 1 udp 255 192.0.2.200 12200 typ relay
raddr 0.0.0.0 rport 0
a=end-of-candidates
m=video 12200 UDP/TLS/RTP/SAVPF 100 101 102 103
c=IN IP4 192.0.2.200
a=mid:v1
a=sendrecv
a=rtpmap:100 VP8/90000
a=rtpmap:101 H264/90000
a=fmtp:101 packetization-mode=1;profile-level-id=42e01f
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=100
a=rtpmap:103 rtx/90000
a=fmtp:103 apt=101
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=msid:751f239e-4ae0-c549-aa3d-890de772998bThe SDP for |answer-C2| looks like:
v=0
o=- 1070771854436052752 2 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle ice2
a=group:BUNDLE a1 v1
a=group:LS a1 v1
m=audio 12100 UDP/TLS/RTP/SAVPF 96 0 8 97 98
c=IN IP4 192.0.2.100
a=mid:a1
a=sendrecv
a=rtpmap:96 opus/48000/2
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:97 telephone-event/8000
a=rtpmap:98 telephone-event/48000
a=fmtp:97 0-15
a=fmtp:98 0-15
a=maxptime:120
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:2 urn:ietf:params:rtp-hdrext:ssrc-audio-level
a=msid:bbce3ba6-abfc-ac63-d00a-e15b286f8fce
a=ice-ufrag:4ZcD
a=ice-pwd:ZaaG6OG7tCn4J/lehAGz+HHD
a=fingerprint:sha-256
C4:68:F8:77:6A:44:F1:98:6D:7C:9F:47:EB:E3:34:A4:
0A:AA:2D:49:08:28:70:2E:1F:AE:18:7D:4E:3E:66:BF
a=setup:passive
a=tls-id:9e5b948ade9c3d41de6617b68f769e55
a=rtcp-mux
a=rtcp-mux-only
a=rtcp-rsize
a=candidate:1 1 udp 255 192.0.2.100 12100 typ relay
raddr 0.0.0.0 rport 0
a=end-of-candidates
m=video 12100 UDP/TLS/RTP/SAVPF 100 101 102 103
c=IN IP4 192.0.2.100
a=mid:v1
a=sendrecv
a=rtpmap:100 VP8/90000
a=rtpmap:101 H264/90000
a=fmtp:101 packetization-mode=1;profile-level-id=42e01f
a=rtpmap:102 rtx/90000
a=fmtp:102 apt=100
a=rtpmap:103 rtx/90000
a=fmtp:103 apt=101
a=extmap:1 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:3 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=msid:bbce3ba6-abfc-ac63-d00a-e15b286f8fce8. Security Considerations 🔐
okay bestie, the IETF already wrote separate docs about the security architecture for WebRTC as a whole: [RFC8827] [RFC8826]. this section is just about the stuff specific to this document 💅
so, JSEP is formally an API, but honestly? think of it as an Internet protocol, where the application JavaScript is untrustworthy from the JSEP implementation's point of view 👀 the threat model of [RFC3552] applies here. like, JavaScript can call the API in any order and with any inputs, including malicious ones 💀 this is extra relevant for the SDP you pass to setLocalDescription. correct API usage means passing in SDP that was derived from createOffer or createAnswer, but there is zero guarantee apps actually do that 🙈 The JSEP implementation MUST be prepared for the JavaScript to pass in bogus data instead.
now the other direction, which is just as important 🫶 app devs need to know the JavaScript does NOT have complete control of endpoint behavior. one case that really bears mentioning: editing ICE candidates out of the SDP or suppressing trickled candidates does not do what you'd expect 😭 implementations will still perform checks from those candidates even if they are not sent to the other side. so, for instance, you can't stop the remote peer from learning an endpoint's public IP address by removing server-reflexive candidates.
Endpoints that wish to conceal their public IP address MUST instead configure the ICE agent to use only relay candidates. 🚨
9. IANA Considerations 🎀
good news: this document has no IANA actions, so there's nothing to register here 🍓
10. References 📚
10.1. Normative References 📚✨
Okay bestie, these are the normative refs, the ones you actually need to read to get the rest of this spec 🫶
- [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>. - [RFC3261] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, "SIP: Session Initiation Protocol", RFC 3261, DOI 10.17487/RFC3261, June 2002,
<https://www.rfc-editor.org/info/rfc3261>. - [RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with Session Description Protocol (SDP)", RFC 3264, DOI 10.17487/RFC3264, June 2002,
<https://www.rfc-editor.org/info/rfc3264>. - [RFC3552] Rescorla, E. and B. Korver, "Guidelines for Writing RFC Text on Security Considerations", BCP 72, RFC 3552, DOI 10.17487/RFC3552, July 2003,
<https://www.rfc-editor.org/info/rfc3552>. - [RFC3605] Huitema, C., "Real Time Control Protocol (RTCP) attribute in Session Description Protocol (SDP)", RFC 3605, DOI 10.17487/RFC3605, October 2003,
<https://www.rfc-editor.org/info/rfc3605>. - [RFC3711] Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. Norrman, "The Secure Real-time Transport Protocol (SRTP)", RFC 3711, DOI 10.17487/RFC3711, March 2004,
<https://www.rfc-editor.org/info/rfc3711>. - [RFC3890] Westerlund, M., "A Transport Independent Bandwidth Modifier for the Session Description Protocol (SDP)", RFC 3890, DOI 10.17487/RFC3890, September 2004,
<https://www.rfc-editor.org/info/rfc3890>. - [RFC4145] Yon, D. and G. Camarillo, "TCP-Based Media Transport in the Session Description Protocol (SDP)", RFC 4145, DOI 10.17487/RFC4145, September 2005,
<https://www.rfc-editor.org/info/rfc4145>. - [RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session Description Protocol", RFC 4566, DOI 10.17487/RFC4566, July 2006,
<https://www.rfc-editor.org/info/rfc4566>. - [RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, "Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, DOI 10.17487/RFC4585, July 2006,
<https://www.rfc-editor.org/info/rfc4585>. - [RFC5124] Ott, J. and E. Carrara, "Extended Secure RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/SAVPF)", RFC 5124, DOI 10.17487/RFC5124, February 2008,
<https://www.rfc-editor.org/info/rfc5124>. - [RFC5285] Singer, D. and H. Desineni, "A General Mechanism for RTP Header Extensions", RFC 5285, DOI 10.17487/RFC5285, July 2008,
<https://www.rfc-editor.org/info/rfc5285>. - [RFC5761] Perkins, C. and M. Westerlund, "Multiplexing RTP Data and Control Packets on a Single Port", RFC 5761, DOI 10.17487/RFC5761, April 2010,
<https://www.rfc-editor.org/info/rfc5761>. - [RFC5888] Camarillo, G. and H. Schulzrinne, "The Session Description Protocol (SDP) Grouping Framework", RFC 5888, DOI 10.17487/RFC5888, June 2010,
<https://www.rfc-editor.org/info/rfc5888>. - [RFC6236] Johansson, I. and K. Jung, "Negotiation of Generic Image Attributes in the Session Description Protocol (SDP)", RFC 6236, DOI 10.17487/RFC6236, May 2011,
<https://www.rfc-editor.org/info/rfc6236>. - [RFC6347] Rescorla, E. and N. Modadugu, "Datagram Transport Layer Security Version 1.2", RFC 6347, DOI 10.17487/RFC6347, January 2012,
<https://www.rfc-editor.org/info/rfc6347>. - [RFC6716] Valin, JM., Vos, K., and T. Terriberry, "Definition of the Opus Audio Codec", RFC 6716, DOI 10.17487/RFC6716, September 2012,
<https://www.rfc-editor.org/info/rfc6716>. - [RFC6904] Lennox, J., "Encryption of Header Extensions in the Secure Real-time Transport Protocol (SRTP)", RFC 6904, DOI 10.17487/RFC6904, April 2013,
<https://www.rfc-editor.org/info/rfc6904>. - [RFC7160] Petit-Huguenin, M. and G. Zorn, Ed., "Support for Multiple Clock Rates in an RTP Session", RFC 7160, DOI 10.17487/RFC7160, April 2014,
<https://www.rfc-editor.org/info/rfc7160>. - [RFC7587] Spittka, J., Vos, K., and JM. Valin, "RTP Payload Format for the Opus Speech and Audio Codec", RFC 7587, DOI 10.17487/RFC7587, June 2015,
<https://www.rfc-editor.org/info/rfc7587>. - [RFC7742] Roach, A.B., "WebRTC Video Processing and Codec Requirements", RFC 7742, DOI 10.17487/RFC7742, March 2016,
<https://www.rfc-editor.org/info/rfc7742>. - [RFC7850] Nandakumar, S., "Registering Values of the SDP 'proto' Field for Transporting RTP Media over TCP under Various RTP Profiles", RFC 7850, DOI 10.17487/RFC7850, April 2016,
<https://www.rfc-editor.org/info/rfc7850>. - [RFC7874] Valin, JM. and C. Bran, "WebRTC Audio Codec and Processing Requirements", RFC 7874, DOI 10.17487/RFC7874, May 2016,
<https://www.rfc-editor.org/info/rfc7874>. - [RFC8108] Lennox, J., Westerlund, M., Wu, Q., and C. Perkins, "Sending Multiple RTP Streams in a Single RTP Session", RFC 8108, DOI 10.17487/RFC8108, March 2017,
<https://www.rfc-editor.org/info/rfc8108>. - [RFC8122] Lennox, J. and C. Holmberg, "Connection-Oriented Media Transport over the Transport Layer Security (TLS) Protocol in the Session Description Protocol (SDP)", RFC 8122, DOI 10.17487/RFC8122, March 2017,
<https://www.rfc-editor.org/info/rfc8122>. - [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017,
<https://www.rfc-editor.org/info/rfc8174>. - [RFC8445] Keranen, A., Holmberg, C., and J. Rosenberg, "Interactive Connectivity Establishment (ICE): A Protocol for Network Address Translator (NAT) Traversal", RFC 8445, DOI 10.17487/RFC8445, July 2018,
<https://www.rfc-editor.org/info/rfc8445>. - [RFC8826] Rescorla, E., "Security Considerations for WebRTC", RFC 8826, DOI 10.17487/RFC8826, January 2021,
<https://www.rfc-editor.org/info/rfc8826>. - [RFC8827] Rescorla, E., "WebRTC Security Architecture", RFC 8827, DOI 10.17487/RFC8827, January 2021,
<https://www.rfc-editor.org/info/rfc8827>. - [RFC8829] Uberti, J., Jennings, C., and E. Rescorla, Ed., "JavaScript Session Establishment Protocol (JSEP)", RFC 8829, DOI 10.17487/RFC8829, January 2021,
<https://www.rfc-editor.org/info/rfc8829>. - [RFC8830] Alvestrand, H., "WebRTC MediaStream Identification in the Session Description Protocol", RFC 8830, DOI 10.17487/RFC8830, January 2021,
<https://www.rfc-editor.org/info/rfc8830>. - [RFC8834] Perkins, C., Westerlund, M., and J. Ott, "Media Transport and Use of RTP in WebRTC", RFC 8834, DOI 10.17487/RFC8834, January 2021,
<https://www.rfc-editor.org/info/rfc8834>. - [RFC8838] Ivov, E., Uberti, J., and P. Saint-Andre, "Trickle ICE: Incremental Provisioning of Candidates for the Interactive Connectivity Establishment (ICE) Protocol", RFC 8838, DOI 10.17487/RFC8838, January 2021,
<https://www.rfc-editor.org/info/rfc8838>. - [RFC8839] Petit-Huguenin, M., Nandakumar, S., Holmberg, C., Keränen, A., and R. Shpount, "Session Description Protocol (SDP) Offer/Answer Procedures for Interactive Connectivity Establishment (ICE)", RFC 8839, DOI 10.17487/RFC8839, January 2021,
<https://www.rfc-editor.org/info/rfc8839>. - [RFC8840] Ivov, E., Stach, T., Marocco, E., and C. Holmberg, "A Session Initiation Protocol (SIP) Usage for Incremental Provisioning of Candidates for the Interactive Connectivity Establishment (Trickle ICE)", RFC 8840, DOI 10.17487/RFC8840, January 2021,
<https://www.rfc-editor.org/info/rfc8840>. - [RFC8841] Holmberg, C., Shpount, R., Loreto, S., and G. Camarillo, "Session Description Protocol (SDP) Offer/Answer Procedures for Stream Control Transmission Protocol (SCTP) over Datagram Transport Layer Security (DTLS) Transport", RFC 8841, DOI 10.17487/RFC8841, January 2021,
<https://www.rfc-editor.org/info/rfc8841>. - [RFC8842] Holmberg, C. and R. Shpount, "Session Description Protocol (SDP) Offer/Answer Considerations for Datagram Transport Layer Security (DTLS) and Transport Layer Security (TLS)", RFC 8842, DOI 10.17487/RFC8842, January 2021,
<https://www.rfc-editor.org/info/rfc8842>. - [RFC8851] Roach, A.B., Ed., "RTP Payload Format Restrictions", RFC 8851, DOI 10.17487/RFC8851, January 2021,
<https://www.rfc-editor.org/info/rfc8851>. - [RFC8852] Roach, A.B., Nandakumar, S., and P. Thatcher, "RTP Stream Identifier Source Description (SDES)", RFC 8852, DOI 10.17487/RFC8852, January 2021,
<https://www.rfc-editor.org/info/rfc8852>. - [RFC8853] Burman, B., Westerlund, M., Nandakumar, S., and M. Zanaty, "Using Simulcast in Session Description Protocol (SDP) and RTP Sessions", RFC 8853, DOI 10.17487/RFC8853, January 2021,
<https://www.rfc-editor.org/info/rfc8853>. - [RFC8854] Uberti, J., "WebRTC Forward Error Correction Requirements", RFC 8854, DOI 10.17487/RFC8854, January 2021,
<https://www.rfc-editor.org/info/rfc8854>. - [RFC8858] Holmberg, C., "Indicating Exclusive Support of RTP and RTP Control Protocol (RTCP) Multiplexing Using the Session Description Protocol (SDP)", RFC 8858, DOI 10.17487/RFC8858, January 2021,
<https://www.rfc-editor.org/info/rfc8858>. - [RFC8859] Nandakumar, S., "A Framework for Session Description Protocol (SDP) Attributes When Multiplexing", RFC 8859, DOI 10.17487/RFC8859, January 2021,
<https://www.rfc-editor.org/info/rfc8859>. - [RFC9143] Holmberg, C., Alvestrand, H., and C. Jennings, "Negotiating Media Multiplexing Using the Session Description Protocol (SDP)", RFC 9143, DOI 10.17487/RFC9143, February 2022,
<https://www.rfc-editor.org/info/rfc9143>. - [RFC9147] Rescorla, E., Tschofenig, H., and N. Modadugu, "The Datagram Transport Layer Security (DTLS) Protocol Version 1.3", RFC 9147, DOI 10.17487/RFC9147, April 2022,
<https://www.rfc-editor.org/info/rfc9147>.
10.2. Informative References 📚🌸
okay these are the informative refs, so they're good background reading and not required to get the spec working 🫶 still worth a peek if you're curious
[RFC3389] Zopf, R., "Real-time Transport Protocol (RTP) Payload for
Comfort Noise (CN)", RFC 3389, DOI 10.17487/RFC3389,
September 2002, <https://www.rfc-editor.org/info/rfc3389>.
[RFC3556] Casner, S., "Session Description Protocol (SDP) Bandwidth
Modifiers for RTP Control Protocol (RTCP) Bandwidth",
RFC 3556, DOI 10.17487/RFC3556, July 2003,
<https://www.rfc-editor.org/info/rfc3556>.
[RFC3960] Camarillo, G. and H. Schulzrinne, "Early Media and Ringing
Tone Generation in the Session Initiation Protocol (SIP)",
RFC 3960, DOI 10.17487/RFC3960, December 2004,
<https://www.rfc-editor.org/info/rfc3960>.
[RFC4568] Andreasen, F., Baugher, M., and D. Wing, "Session
Description Protocol (SDP) Security Descriptions for Media
Streams", RFC 4568, DOI 10.17487/RFC4568, July 2006,
<https://www.rfc-editor.org/info/rfc4568>.
[RFC4588] Rey, J., Leon, D., Miyazaki, A., Varsa, V., and R.
Hakenberg, "RTP Retransmission Payload Format", RFC 4588,
DOI 10.17487/RFC4588, July 2006,
<https://www.rfc-editor.org/info/rfc4588>.
[RFC4733] Schulzrinne, H. and T. Taylor, "RTP Payload for DTMF
Digits, Telephony Tones, and Telephony Signals", RFC 4733,
DOI 10.17487/RFC4733, December 2006,
<https://www.rfc-editor.org/info/rfc4733>.
[RFC5245] Rosenberg, J., "Interactive Connectivity Establishment
(ICE): A Protocol for Network Address Translator (NAT)
Traversal for Offer/Answer Protocols", RFC 5245,
DOI 10.17487/RFC5245, April 2010,
<https://www.rfc-editor.org/info/rfc5245>.
[RFC5506] Johansson, I. and M. Westerlund, "Support for Reduced-Size
Real-Time Transport Control Protocol (RTCP): Opportunities
and Consequences", RFC 5506, DOI 10.17487/RFC5506, April
2009, <https://www.rfc-editor.org/info/rfc5506>.
[RFC5576] Lennox, J., Ott, J., and T. Schierl, "Source-Specific
Media Attributes in the Session Description Protocol
(SDP)", RFC 5576, DOI 10.17487/RFC5576, June 2009,
<https://www.rfc-editor.org/info/rfc5576>.
[RFC5763] Fischl, J., Tschofenig, H., and E. Rescorla, "Framework
for Establishing a Secure Real-time Transport Protocol
(SRTP) Security Context Using Datagram Transport Layer
Security (DTLS)", RFC 5763, DOI 10.17487/RFC5763, May
2010, <https://www.rfc-editor.org/info/rfc5763>.
[RFC5764] McGrew, D. and E. Rescorla, "Datagram Transport Layer
Security (DTLS) Extension to Establish Keys for the Secure
Real-time Transport Protocol (SRTP)", RFC 5764,
DOI 10.17487/RFC5764, May 2010,
<https://www.rfc-editor.org/info/rfc5764>.
[RFC6120] Saint-Andre, P., "Extensible Messaging and Presence
Protocol (XMPP): Core", RFC 6120, DOI 10.17487/RFC6120,
March 2011, <https://www.rfc-editor.org/info/rfc6120>.
[RFC6464] Lennox, J., Ed., Ivov, E., and E. Marocco, "A Real-time
Transport Protocol (RTP) Header Extension for Client-to-
Mixer Audio Level Indication", RFC 6464,
DOI 10.17487/RFC6464, December 2011,
<https://www.rfc-editor.org/info/rfc6464>.
[RFC8828] Uberti, J. and G. Shieh, "WebRTC IP Address Handling
Requirements", RFC 8828, DOI 10.17487/RFC8828, January
2021, <https://www.rfc-editor.org/info/rfc8828>.
[RFC8843] Holmberg, C., Alvestrand, H., and C. Jennings,
"Negotiating Media Multiplexing Using the Session
Description Protocol (SDP)", RFC 8843,
DOI 10.17487/RFC8843, January 2021,
<https://www.rfc-editor.org/info/rfc8843>.
[SDP4WebRTC]
Nandakumar, S. and C. F. Jennings, "Annotated Example SDP
for WebRTC", Work in Progress, Internet-Draft, draft-ietf-
rtcweb-sdp-14, 17 December 2020,
<https://datatracker.ietf.org/doc/html/draft-ietf-rtcweb-
sdp-14>.
[TS26.114] 3GPP, "3rd Generation Partnership Project; Technical
Specification Group Services and System Aspects; IP
Multimedia Subsystem (IMS); Multimedia Telephony; Media
handling and interaction (Release 18)", 3GPP TS 26.114
V18.6.0, March 2024,
<https://www.3gpp.org/DynaReport/26114.htm>.
[W3C.webrtc]
Jennings, C., Ed., Castelli, F., Ed., Boström, H., Ed.,
and J-I. Bruaroey, Ed., "WebRTC: Real-time Communication
Between Browsers", W3C Recommendation, March 2023,
<https://www.w3.org/TR/2023/REC-webrtc-20230306/>.Appendix A. SDP ABNF Syntax 🎀
okay so for the syntax validation we do in Section 5.8, this is the list of ABNF definitions we lean on, and it's giving organised 💅 each attribute gets pointed right at the spot where it's defined
+=========================+===============================+
| Attribute | Reference |
+=========================+===============================+
| ptime | Section 6 of [RFC4566] |
+-------------------------+-------------------------------+
| maxptime | Section 6 of [RFC4566] |
+-------------------------+-------------------------------+
| rtpmap | Section 6 of [RFC4566] |
+-------------------------+-------------------------------+
| recvonly | Sections 6 and 9 of [RFC4566] |
+-------------------------+-------------------------------+
| sendrecv | Sections 6 and 9 of [RFC4566] |
+-------------------------+-------------------------------+
| sendonly | Sections 6 and 9 of [RFC4566] |
+-------------------------+-------------------------------+
| inactive | Sections 6 and 9 of [RFC4566] |
+-------------------------+-------------------------------+
| fmtp | Sections 6 and 9 of [RFC4566] |
+-------------------------+-------------------------------+
| rtcp | Section 2.1 of [RFC3605] |
+-------------------------+-------------------------------+
| setup | Section 4 of [RFC4145] |
+-------------------------+-------------------------------+
| fingerprint | Section 5 of [RFC8122] |
+-------------------------+-------------------------------+
| rtcp-fb | Section 4.2 of [RFC4585] |
+-------------------------+-------------------------------+
| extmap | Section 7 of [RFC5285] |
+-------------------------+-------------------------------+
| mid | Section 4 of [RFC5888] |
+-------------------------+-------------------------------+
| group | Section 5 of [RFC5888] |
+-------------------------+-------------------------------+
| imageattr | Section 3.1 of [RFC6236] |
+-------------------------+-------------------------------+
| extmap (encrypt option) | Section 4 of [RFC6904] |
+-------------------------+-------------------------------+
| candidate | Section 5.1 of [RFC8839] |
+-------------------------+-------------------------------+
| remote-candidates | Section 5.2 of [RFC8839] |
+-------------------------+-------------------------------+
| ice-lite | Section 5.3 of [RFC8839] |
+-------------------------+-------------------------------+
| ice-ufrag | Section 5.4 of [RFC8839] |
+-------------------------+-------------------------------+
| ice-pwd | Section 5.4 of [RFC8839] |
+-------------------------+-------------------------------+
| ice-options | Section 5.6 of [RFC8839] |
+-------------------------+-------------------------------+
| msid | Section 3 of [RFC8830] |
+-------------------------+-------------------------------+
| rid | Section 10 of [RFC8851] |
+-------------------------+-------------------------------+
| simulcast | Section 5.1 of [RFC8853] |
+-------------------------+-------------------------------+
| tls-id | Section 4 of [RFC8842] |
+-------------------------+-------------------------------+
Table 1: SDP ABNF ReferencesAcknowledgements 🫶
okay the big thanks 🥹 Harald Alvestrand, Taylor Brandstetter, Suhas Nandakumar, and Peter Thatcher wrote a ton of the text for RFC 8829 (and therefore this document), so they're literally the reason this spec exists 👑
and a huge shoutout to Bernard Aboba, Adam Bergkvist, Jan-Ivar Bruaroey, Dan Burnett, Ben Campbell, Alissa Cooper, Richard Ejzak, Stefan Håkansson, Ted Hardie, Christer Holmberg, Andrew Hutton, Randell Jesup, Matthew Kaufman, Anant Narayanan, Adam Roach, Robert Sparks, Neil Stratford, Martin Thomson, Sean Turner, and Magnus Westerlund 🌈 they all gave really valuable feedback on RFC 8829 (and therefore this document), which is so iconic of them 💖
Authors' Addresses 💌
here's how to reach the authors, bestie 🦋
Justin Uberti
Email: justin@uberti.name
Cullen Jennings
Cisco
400 3rd Avenue SW
Calgary AB T2P 4H2
Canada
Email: fluffy@iii.ca
Eric Rescorla (editor)
Windy Hill Systems, LLC
Email: ekr@rtfm.com📦 where this page comes from
- ietf: rfc9429 2024-04
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms