RFC 5567
An Architectural Framework for Media Server Control
RFC 5567: An Architectural Framework for Media Server Control 🎀
- Series: Network Working Group
- Number: RFC 5567
- Editor: T. Melanchuk, Ed. (Rain Willow Communications)
- Category: Informational
- Date: June 2009
- Obsoletes/Updates: This document doesn't list any RFCs it obsoletes or updates. 🫶
Status of This Memo 📋
okay so this memo just gives info to the Internet community. It is not a standard of any kind 🙅♀️. Anyone can share it freely, no cap.
Copyright Notice ©️
Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved. 💅
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents in effect on the date of publication of this document (http://trustee.ietf.org/license-info). Please review these documents carefully, as they describe your rights and restrictions with respect to this document.
This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The person(s) controlling the copyright in some of this material may not have granted the IETF Trust the right to allow modifications of such material outside the IETF Standards Process. Without obtaining an adequate license from the person(s) controlling the copyright in such materials, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process, except to format it for publication as an RFC or to translate it into languages other than English. 🙈
Abstract 🦋
this doc lays out an architectural framework for Media Server control 🧚. The main goal is to define the logical entities that exist in the Media Server control world, and to define the naming conventions and the interactions between them.
Table of Contents 🍓
1. Introduction ....................................................2
2. Terminology .....................................................3
3. Architecture Overview ...........................................4
4. SIP Usage .......................................................7
5. Media Control for IVR Services .................................10
5.1. Basic IVR Services ........................................11
5.2. IVR Services with Mid-Call Controls .......................11
5.3. Advanced IVR Services .....................................11
6. Media Control for Conferencing Services ........................12
6.1. Creating a New Conference .................................14
6.2. Adding a Participant to a Conference ......................14
6.3. Media Controls ............................................15
6.4. Floor Control .............................................16
7. Security Considerations ........................................21
8. Acknowledgments ................................................22
9. Contributors ...................................................22
10. Informative References ........................................231. Introduction ✨
okay so Application Servers host one or more instances of a communications application 📱. Media Servers handle the real-time media processing functions 🎶. This doc is the core architectural framework that lets Application Servers control Media Servers. You'll get an overview of the core logical entities and how they interact in Section 3. The requirements for Media Server control are defined in [RFC5167] 📝.
The Session Initiation Protocol (SIP) [RFC3261] is the session establishment protocol used in this architecture 🔥. Application Servers use it for two things: terminating media streams on Media Servers, and creating and managing control channels between themselves and Media Servers 💜. The detailed model for Media Server control, along with a description of SIP usage, is in Section 4. It's giving thorough 😭.
Several services are described using the framework in this document 🌈. Use cases for Interactive Voice Response (IVR) services are in Section 5, and conferencing use cases are in Section 6 📞.
2. Terminology 📖
bestie, these are the terms this doc defines for Media Server control. Pay attention, the definitions matter 🤓:
- Application Server (AS): A functional entity that hosts one or more instances of a communication application. The application server may include the conference policy server, the focus, and the conference notification server, as defined in [RFC4353]. It may also include communication applications that use IVR or announcement services 🎀.
- Media Functions: Functions available on a Media Server that are used to supply media services to the AS. Some examples are Dual-Tone Multi-Frequency (DTMF) detection, mixing, transcoding, playing announcement, recording, etc. 🎹
- Media Resource Broker (MRB): A logical entity that is responsible for both the collection of appropriate published Media Server (MS) information and supplying of appropriate MS information to consuming entities. The MRB is an optional entity and will be discussed in a separate document. Lowkey a background character for now 🫣.
- Media Server (MS): The media server includes the mixer as defined in [RFC4353]. The media server plays announcements, and it processes media streams for functions like DTMF detection and transcoding. It may also record media streams for supporting IVR functions like announcing conference participants. In the architecture for the 3GPP IP Multimedia Subsystem (IMS), a Media Server is referred to as a Media Resource Function (MRF) 🌸.
- Media Services: Application service requiring media functions such as Interactive Voice Response (IVR) or media conferencing 💅.
- Media Session: From the Session Description Protocol (SDP) specification [RFC4566]: "A multimedia session is a set of multimedia senders and receivers and the data streams flowing from senders to receivers. A multimedia conference is an example of a multimedia session." 🦋
- MS Control Channel: A reliable transport connection between the AS and MS used to exchange MS Control PDUs. Implementations must support the Transport Control Protocol (TCP) [RFC0793] and may support the Stream Control Transmission Protocol (SCTP) [RFC4960]. Implementations must support TLS [RFC5246] as a transport-level security mechanism, although its use in deployments is optional. Security first, periodt 🔐.
- MS Control Dialog: A SIP dialog that is used for establishing a control channel between the user agent (UA) and the MS 🤝.
- MS Control Protocol: The protocol used for by an AS to control an MS. The MS Control Protocol assumes a reliable underlying transport protocol for the MS Control Channel 🧠.
- MS Media Dialog: A SIP dialog between the AS and MS that is used for establishing media sessions between a user device such as a SIP phone and the MS 📲.
The definitions for AS, MS, and MRB above are taken from [RFC5167] ✨.
3. Architecture Overview ✨
okay so bestie, a Media Server (MS) is basically a network device that processes media streams 🎀 and some of the media processing it can do looks like this:
- Control of the Real-Time Protocol (RTP) [RFC3550] streams using the Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF) [RFC4585].
- Mixing of incoming media streams.
- Media stream source (for multimedia announcements).
- Media stream processing (e.g., transcoding, DTMF detection).
- Media stream sink (for multimedia recordings).
An MS supplies one or more media processing functionalities to an Application Server (AS) 💅, and those can include things that aren't on that list above. An AS can send a particular call to a suitable MS, either by discovering what capabilities a specific MS offers or by going through a Media Resource Broker 🦋.
The type of processing an MS does on media streams is specified and controlled by an AS. Application Servers are logical entities that can run one or more instances of a communications application, so they're basically the brains of the operation 🧠✨. Examples of Application Servers that can work with a Media Server are an AS acting as a Conference 'Focus' as defined in [RFC4353], or an IVR application that uses an MS to play announcements and detect DTMF key presses 📞.
Application Servers use SIP to set up control channels between themselves and MSs. An MS Control Channel implements a reliable transport protocol that carries the MS Control Protocol, and the SIP dialog used to set up a control channel is called an MS Control Dialog 🌸.
Application Servers terminate SIP [RFC3261] signaling from SIP User Agents, and they may terminate other signaling that's outside the scope of this document. They use SIP Third Party Call Control [RFC3725] (3PCC) to establish, maintain, and tear down media streams from those SIP UAs to a Media Server. A SIP dialog that an AS uses to set up a media session on an MS is called an MS Media Dialog 🫶.
Media streams go directly between SIP User Agents and Media Servers 🌈. Media Servers support multiple types of media. Common RTP media types include audio and video, but other things like text and the Binary Floor Control Protocol (BFCP) [RFC4583] are possible too. This basic architecture, showing session establishment signaling between a single AS and MS, is shown in Figure 1 below 🧚.
+-------------+ +--------------+
| | SIP (MS Control Dialog) | |
| Application |<----------------------->| Media |
| Server | | Server |
| |<----------------------->| |
+-------------+ SIP (MS Media Dialog) +--------------+
^ ^
\ | RTP/SRTP
\ | audio/
\ | video/etc)
\ |
\ v
\ +--------------+
\ SIP | |
+-------------->| SIP |
| User Agent |
| |
+--------------+Figure 1: Basic Signaling Architecture
The architecture must support a many-to-many relationship between Application Servers and Media Servers 🔥. In real-world deployments, an Application Server may interact with multiple Media Servers, and/or a Media Server may be controlled by more than one Application Server.
Application Servers can use the SIP URI as described in [RFC4240] to request basic functions from Media Servers 💖. Basic functions are the ones that need no mid-call interactions between the AS and MS. Examples are simple announcement-playing or basic conference-mixing where the AS doesn't need to explicitly control the mixing.
Most services, however, have interactions between the AS and MS during a call or conference 👑. The types of interactions can be generalized as follows:
- commands from an AS to an MS to request the application or configuration of a function. The request may apply to a single media stream, multiple media streams associated with multiple SIP dialogs, or to properties of a conference mix.
- responses from an MS to an AS reporting on the status of particular commands.
- notifications from an MS to an AS that report results from commands or notify changes to subscribed status.
Commands, responses, and notifications are carried over one or more dedicated control channels between the Application Server and the Media Server 🍓. Dedicated control channels provide reliable, sequenced, peer-to-peer transport for Media Server control interactions. Implementations must support the Transport Control Protocol (TCP) [RFC0793] and may support the Stream Control Transmission Protocol (SCTP) [RFC4960]. Because MS control needs sequenced reliable delivery of messages, unreliable protocols such as the User Datagram Protocol (UDP) are not suitable 🙈. Implementations must support TLS [RFC5246] as a transport-level security mechanism, although its use in deployments is optional 🔐. A dedicated control channel is shown in Figure 2 below.
+-------------+ +--------------+
| | | |
| Application | MS ctrl channel | Media |
| Server |<------------------->| Server |
| | | |
+-------------+ +--------------+
^ ^ ^
RTP/SRTP | | |
(audio/ | | |
video/etc) | | |
| | v
+---|-v-------+
+-|---v-------+ |
+-|-----------+ | |
| | | |
| SIP | | |
| User Agent | |-+
| |-+
+-------------+Figure 2: Media Server Control Architecture
Both Application Servers and Media Servers may interact with other servers for specific purposes beyond the scope of this document 🌷. For example, Application Servers will often talk to other infrastructure components, which are usually based on deployment requirements, with links to back-office data stores and applications. Media Servers will often retrieve announcements from external file servers. Also, many Media Servers support IVR dialog services using VoiceXML [W3C.REC-voicexml20-20040316]. In this case, the MS interacts with other servers using HTTP during standard VoiceXML processing. VoiceXML Media Servers may also interact with speech engines (for example, using the Media Resource Control Protocol version 2 (MRCPv2)) for speech recognition and generation purposes 🎙️.
Some specific types of interactions between Application and Media servers are also out of scope for this document. MS resource reservation is one such interaction 🙅. Also, any interactions between Application Servers, or between Media Servers, are out of scope too.
Okay bestie, let's get into the SIP stuff 💅✨ This part is where the Media Server and Application Server start actually talking to each other, so buckle up 🎀
4. SIP Usage ✨
okay so the Session Initiation Protocol [RFC3261] was built by the IETF to start, manage, and end multimedia sessions 🦋 and honestly? it's been absolutely iconic. it's blown up like crazy since it first showed up and it's now the main VoIP protocol out there 💖 it got picked as the foundation for stuff like the IP Multimedia Subsystem (IMS) in 3GPP and it showed up in tons of the early live VoIP setups 🌸 media servers aren't a new idea in IP telephony either, and people have proposed loads of signaling protocols and techniques to control them 🧚 the most popular approaches so far mix SIP with different markup languages to send media service requests and responses back and forth 💜
so, as we talked about in Section 3 and as Figure 1 shows, this document's logical architecture is all about the Application Server (AS) and the Media Server (MS) working together 🫶 SIP interactions can be split up into "MS media dialogs," which an AS and an MS use to set up media sessions between an endpoint and a Media Server, and "MS control dialogs," which set up and keep up the MS control channels 🔥
SIP is the main signaling protocol for session signaling, and it's used for every media session pointed at a Media Server, like this document describes 👑 Media Servers may support other signaling protocols, but we're not looking at that kind of interaction here 🙈 Application Servers may terminate non-SIP signaling protocols, but when they talk to a Media Server they MUST gateway those requests over to SIP 💀
SIP is also used to create, manage, and tear down the dedicated MS control channel(s) 🎀 control channels give you reliable, sequenced delivery of MS Control Protocol messages 💅 the Application and Media Servers use the SDP attributes defined in [RFC4145] so SIP can negotiate the control channel 🦋 a control channel gets closed when SIP ends the matching MS control dialog ✨ more details and example flows live in the SIP Control Framework [SIP-CTRL-FW] 😭 that framework also has basic control message semantics for the kinds of interactions we covered in Section 3 🌈 it uses "packages" so domain-specific protocols can be defined in the Extensible Markup Language (XML) [W3C.REC-xml-20060816] format 🧚 the MS Control Protocol is made up of one or more packages for the SIP Control Framework 💖
using SIP for both media and control dialogs gives you a bunch of inherent perks compared to other options 🍓 here's the list:
- SIP location and rendezvous capabilities, as defined in [RFC3263] 🗺️ these give you the core mechanisms for routing a SIP request based on stuff like DNS SRV and NAPTR records, and the SIP infrastructure leans on those techniques hard 💅
- SIP's security and identity properties, like using TLS to connect to another SIP-based entity reliably and securely 🔐 SIP has a few identity mechanisms you can use: [RFC3261] gives you an intra-domain digest-based mechanism, and [RFC4474] defines a certificate-based inter-domain identity mechanism 🫶 SIP with S/MIME lets you secure payloads with encrypted and signed certificate techniques 👑
- SIP has seriously powerful and dynamic media-negotiation properties, as defined in [RFC3261] and [RFC3264] 🔥
- You can pick an appropriate SIP entity based on capability sets, as discussed in [RFC3840] 🦋 this is a really powerful function because Media Servers can convey a specific capability set, and then the AS is free to pick the MS that fits what it needs 💖
- using SIP also keeps things consistent with IETF protocols and usages 🌸 SIP was meant for creating and managing media sessions, so this is a correct use of the protocol ✨
as I mentioned earlier in this section, media services using SIP are pretty well understood 🧚 some older proposals suggested using the SIP INFO [RFC2976] method as the transport between the AS and MS 😬 but using SIP INFO like that is not advised, for a few reasons 🙈
- INFO is an opaque request with no specific semantics. a SIP endpoint that gets an INFO request has no idea what to do with it based on SIP signaling 💀
- SIP INFO was never built to carry generic session control information along the signaling path. it should really only be used for optional application information, e.g., carrying mid-call Public Switched Telephone Network (PSTN) signaling messages between PSTN gateways 🍓
- SIP INFO travels along the signaling path, which is an inefficient way to send control messages that could be routed directly between the AS and MS 😩
- [RFC3261] has rules for using an unreliable protocol like UDP. when a packet gets close to the Maximum Transmission Unit (MTU), the protocol should be switched to TCP. that's not ideal when you're constantly dealing with big payloads, like XML-formatted MS control messages 🥺
5. Media Control for IVR Services 🎀
okay so one of the main jobs of a Media Server is to help an Application Server that's running IVR services by doing media processing on media streams 📞 "IVR" is kind of a generic term, but the range of media functions an MS offers covers what user interaction dialogs need 💜 those functions include media transcoding, basic announcements, user input detection (via DTMF or speech), and media recording 🎙️
a particular IVR or user dialog app usually needs several of those media functions at once 🧩 IVR dialogs can range a lot in complexity, from a simple single announcement playback all the way up to complex voice mail apps 📼
like we already covered, an AS uses SIP [RFC3261] and SDP [RFC4566] to set up and configure media sessions to a Media Server 🦋 the AS uses the MS control channel, which was set up with SIP, to invoke IVR requests and to receive responses and notifications 🔔 this topology is shown in Figure 3 below 👇
+-------------+ SIP +-------------+
| Application |<---------------------------->| Media |
| Server | (media & MS Control dialogs) | Server |
| | | |
| | MS Control Protocol (IVR) | |
| |<---------------------------->| (IVR media |
| (App logic) | (CtrlChannel) | functions) |
+-------------+ +-------------+
^ ^^
\ || R
\ || T
\ || P
\ || /
\ || S
\ || R
\ || T
\ || P
\ vv
\ call signaling +-----------+
---------------------------->| User |
(e.g., SIP) | Equipment |
+-----------+
Figure 3: IVR Topologythe variety of Application Server IVR services means the Media Server has to support different levels of media functions, as described in the following subsections ✨
5.1. Basic IVR Services 📣
for simple basic announcement requests, the MS control channel (the one shown in Figure 3 above) is not required 🙌 simple announcement requests can be invoked on the Media Server using the SIP URI mechanism defined in [RFC4240] 🦋 this interface doesn't allow any digit detection, user input collection, or mid-call dialog control 🙈 but lots of apps only need basic media services, so the processing burden on the Media Server for more complex AS interactions wouldn't be needed in that case 💅
5.2. IVR Services with Mid-Call Controls 🎛️
for more complex IVR dialogs that need mid-call interaction and control between the Application Server and the Media Server, the MS control channel (shown in Figure 3 above) is used to invoke specific media functions on the Media Server 🔥 these functions include, but aren't limited to, complex announcements with barge-in facility, user-input detection and reporting (e.g., DTMF) to an Application Server, DTMF and voice-activity controlled recordings, and so on 🎀 composite services, like play-collect and play-record, are also covered by this model 🫶
mid-call control also lets Application Servers subscribe to IVR-related events, and lets the Media Server notify the AS when those events happen 🔔 examples are announcement completion events, record completion events, and reporting of collected DTMF digits 💖
5.3. Advanced IVR Services 👑
okay so IVR services with mid-call control already give you a full set of media functions that a Media Server is expected to have 💅 but the advanced IVR services model goes a level higher, letting application logic described at a higher level of abstraction, like VoiceXML provides, run on the Media Server 🧚 invoking VoiceXML IVR dialogs can be done with the "Prompt and Collect" mechanism of [RFC4240] 🦋 the IVR control protocol can also be extended so VoiceXML requests can be invoked over the MS control channel too ✨ VoiceXML IVR services running on the Media Server might need an HTTP interface (not shown in Figure 3) between the Media Server and one or more back-end servers that host or generate VoiceXML documents 🌐 those back-end servers may or may not be physically separate from the Application Server 💜
6. Media Control for Conferencing Services ✨
okay so [RFC4353] is the one that lays out the overall architecture and protocol pieces you need for multipoint conferencing with SIP 🎀 then the framework for centralized conferencing [RFC5239] builds on that and adds a protocol between the user and the conferencing server 💅 [RFC4353] describes how the conferencing server gets broken down, but it leaves the specifics open 🫶
this section breaks that decomposition down and talks about what each of the decomposed functional units actually does 🦋 the conferencing factory and the conference focus both live inside the Application Server described in this document 💖
so an Application Server uses SIP Third Party Call Control [RFC3725] to set up media sessions from SIP user agents to a Media Server 🧚 the AS uses that same mechanism, as described in this section, to add and remove participants from a conference, and also to handle the media streams set up for each user 🌈 the XCON framework was designed to be protocol-agnostic about the Call Signaling Protocol users use to join a conference, which means an XCON-compliant Application Server has to handle gatewaying the non-SIP signaling negotiations ✨ the goal is to get valid SIP media sessions running between itself and the Media Server, while keeping the non-SIP interaction with the user totally transparent 🙈
+------------+ +------------+
| | SIP (2m+1c) | |
| Application|-------------| Media |
| Server | | Server |
| (Focus) |-------------| (Mixer) |
| | CtrlChannel | |
+------------+ +------------+
| \ .. .
| \\ RTP... .
| \\ .. .
| H.323 \\ ... .
SIP | \\ ... .RTP
| ..\ .
| ... \\ .
| ... \\ .
| .. \\ .
| ... \\ .
| .. \ .
+-----------+ +-----------+
|Participant| |Participant|
+-----------+ +-----------+
Figure 4: Conference Topologyto top it off, the Application Server uses a dedicated Media Server control channel to set up and manage media conferences on the Media Server 🫧 that complements what 3PCC and the XCON control protocol already do 💜 Figure 4 shows the signaling and media paths for a two-participant conference 🍓 the three SIP dialogs between the AS and MS set up one control session (1c) and two media sessions (2m), coming from the participants (one originally signaled using H.323 and then gatewayed into SIP, and one signaled directly in SIP) 👑
as the conference focus, the Application Server is in charge of setting up and managing a media conference on the Media Servers 🌸 the goal is making sure every media stream in a conference is available to its participants 🔥 it does this by using one or more mixer entities (as described in RFC 4353), whose role as part of the Media Server is covered in this section ✨ the Application Server's services include, but are not limited to, setting up, handling, and destroying a new media conference, adding and removing participants, managing media streams in a conference, controlling the layout and mixing configuration for each media type, allowing per-user custom media profiles, and so on 💅
as a mixer entity in a multimedia conferencing scenario, the Media Server receives a set of media streams of the same type (after transcoding if needed) 🦋 it then combines the received media in a type-specific way and redistributes the result to each authorized participant 🎀 the Application Server handles how each media stream gets combined, plus all the media-related policies, through a dedicated MS control channel 🫶
to summarize: the AS needs to be able to manage Media Servers at both the conference level and the participant level 👑
6.1. Creating a New Conference
when a new conference gets created, either because of a previous conference scheduling or because the first participant dialed into a specified URI, the Application Server must take care of creating a media conference on the Media Server 💖 it does that by sending an explicit request to the Media Server 🧚 this can be done with an MS control channel message 🌈 the request may contain detailed info about the desired settings and policies for the conference (for example, the media to involve, the mixing configuration for them, the relevant identifiers, and so on) ✨ the Media Server validates the request and takes care of allocating the resources needed to set up the media conference 🙈
Application Servers may use mechanisms other than sending requests over the control channel to establish conferences on a Media Server 🍓 they can then use the control channel to control the conference afterwards 💅 examples of other ways to create a conference include using the Request-URI mechanism of [RFC4240] or the procedures defined in [RFC4579] 🦋
once that's done, the MS tells the Application Server what the result of the request was 🫧 each conference is referred to by a specific identifier, and both the Application Server and the Media Server include it in later transactions about the same conference (for example, to modify the settings of an extant conference) 💜
6.2. Adding a Participant to a Conference
as mentioned before, an Application Server uses SIP 3PCC to set up media sessions from SIP user agents to a Media Server 👑 the URI the AS uses in the INVITE to the MS may be one associated with the conference on the MS 🌸 but more likely, the media sessions are first set up to the Media Server using a URI for the Media Server, and then joined to the conference later using the MS Control Protocol 🔥 that way, IVR dialogs can happen before someone joins the conference ✨
the AS, acting as a 3PCC, correlates the media session negotiation between the UA and the MS so it can set up all the media streams the conference policies need 💅
6.3. Media Controls
the XCON Common Data Model [XCON-DM] currently defines some basic media-related controls 🦋 conference-aware participants can use them in several ways, for example through an XCON conference control protocol or through IVR dialogs 🎀 these controls let participants change the volume of their own audio in the conference, configure the layout they want for incoming video streams, mute and unmute themselves, and pause and unpause their own video stream 🫶 conference-aware participants use these controls by sending dedicated conference control protocol requests to the Application Server 💖 the AS validates those requests, translates them into the Media Server Control Protocol, and then forwards them over the MS Control Channel to the MS 🧚 following the AS's directives, the Media Server manipulates the involved media streams accordingly 🌈
+------------+ +------------+
| | 'Include audio | |
| Application| sent by user X | Media |
| Server | in conf Y mix' | Server |
| (Focus) |----------------->| (Mixer) |
| | (MS CtrlChn) | |
+------^-----+ +------------+
| ..
| ...
| 'Unmute me' ... RTP
| (XCON) ...
| ...
| ...
+-----------+ ...
|Participant|...
+-----------+
Figure 5: Conferencing Example: Unmuting A Participantthe Media Server may need to tell the AS about events like in-band DTMF tones during the conference 🙈
6.4. Floor Control
the XCON framework adds "floor control" as an enhancement on top of [RFC4575] 👑 floor control is a way to manage joint or exclusive access to shared resources in a (multiparty) conferencing environment 🌸 it is NOT a mandatory mechanism for a conferencing system implementation, but it gives advanced media input control features to conference-aware participants 🫧 it allows coordinated and moderated access to any set of resources the conferencing system provides ✨ to make that work, a so-called floor gets associated with a set of resources, which represents the right for participants to access and manipulate those resources 💅 to use floor control, you need a specific protocol, the Binary Floor Control Protocol, which is specified in [RFC4582] 🦋 [RFC4583] gives SIP UAs a way to set up a BFCP connection to the Floor Control Server, using a Connection-Oriented Media (COMEDIA) [RFC4145] negotiation to exploit floor control 🎀
in the context of AS-MS interaction, floor control is another way to control participants' media streams 💖 a typical example is a floor tied to the right to access the shared audio channel in a conference 🧚 a participant who gets that floor is granted the right to talk by the conferencing system, which means the MS includes their audio frames in the overall audio conference mix 🌈 when the floor is revoked, the participant is muted and their audio is left out of the final mix 🫶
the BFCP defines a Floor Control Server (FCS) and the floor chair 🍓 it's clear that the floor chair, who decides on floor requests, is part of the application logic 👑 which means that when the floor chair role is automated in a conference, it will normally be part of the AS 💜
the example makes it clear that the Floor Control Server and the Media Server can interact directly or indirectly, so each floor can be correctly bound to its related set of media resources 🔥 there's also a similar interaction between the Floor Control Server and the Application Server, since the AS has to know every association between floors and resources 🌸 that way it can orchestrate the bindings with whichever element is responsible for those resources (for example, the Media Server for audio and/or video streams) and the operations on them (for example, muting or unmuting a participant in a conference) ✨ for this reason, the Floor Control Server can be co-located with either the Media Server or the Application Server, as long as both elements are allowed to interact with the Floor Control Server through some kind of protocol 💅
okay bestie, so there are TWO ways the floor control stuff can work 🦋 and we're gonna look at both so you can see how all the pieces talk to each other 💅 In both topologies, the parts involved are the same, they just sit in different places, and that changes the whole vibe ✨
When the AS and the FCS are co-located, it's honestly so simple 🫶 It's basically a twist on the case in Figure 5. The only big difference is that the AS is acting because of a change in the floor control status, not because a participant asked for some specific control themselves 💜 The sequence diagram in Figure 6 shows how everyone talks in a normal situation. It assumes the BFCP connection between the UA and the FCS (the FCS is co-located with the AS, remember) is already negotiated and set up, and that the UA already knows all the important identifiers and the floors-resources-associations (for example, through [RFC4583]) 🌸 It also assumes the AS already set up the media mixing on the MS using the MS control channel. Right now, every frame the UA tries to send on its media stream is getting dropped by the MS, because the UA isn't allowed to use the resource yet 🙈 For a SIP UA, this can come from a 'sendonly' field on the media stream in a re-INVITE that the MS sent 💀 And it's worth pointing out that the AS has to make sure no user media control mechanisms (like the ones in the previous sub-section) can override the floor control 🔥
UA AS MS
(Floor Participant) (FCS)
| | |
|<===================== One-way RTP stream ======================|
| | |
| FloorRequest(BFCP) | |
|------------------------------------>| |
| | |
| FloorRequestStatus[PENDING](BFCP) | |
|<------------------------------------| |
| |--+ apply |
| | | policies |
| |<-+ to request |
| | |
| FloorRequestStatus[ACCEPTED](BFCP) | |
|<------------------------------------| |
| | |
. . .
. . .
| | |
| FloorRequestStatus[GRANTED](BFCP) | |
|<------------------------------------| |
| | 'Unmute UA' (CtrlChn) |
| |------------------------->|
| | |
|<==================== Bidirectional RTP stream ================>|
| | |
. . .
. . .
Figure 6: Conferencing Example: Floor Control Call FlowSo here's the flow, bestie 🎀 The UA is also a floor participant, and it sends a "FloorRequest" to the floor control server (the FCS, which is co-located with the AS), basically saying "hey, I want the floor for the audio stream in the conference" 🍓 The AS replies with a "FloorRequestStatus" that says PENDING, which means no decision has been made yet ⏳ Then the AS, following the BFCP policies for this conference, decides on the request and accepts it ✨ Quick note: this decision might get passed to another participant instead, if they were previously assigned as the chair of the floor 👑 Assuming it's accepted, the AS tells the UA with a new "FloorRequestStatus" that now says ACCEPTED 💖 And here's the tea: ACCEPTED only means the request was accepted, NOT that the floor is granted yet 😭 Once the queue management in the FCS says, according to the specified scheduling algorithms, that the earlier floor request from the UA can be granted, the AS sends a new "FloorRequestStatus" with GRANTED, and it also unmutes the participant in the conference by sending a directive to the MS through the control channel 🔥 Once the UA gets the notice that its request was granted, it can start sending its media, knowing its stream won't be dropped by the MS anymore 🦋 If the session was previously updated with a 'sendonly' on the media stream, the MS has to originate another re-INVITE saying the media stream flow is now bidirectional ('sendrecv') 💅
As I said before, this scenario has an automated floor chair, where the AS makes the floor request decisions based on some policies 🤖 The case with a real person as the chair is basically the same, except the incoming request isn't handled directly by the AS according to its policies. Instead, it gets forwarded to the floor control participant that the chair UA is using 💜 Then the chair UA tells the AS-FCS what it decided using a 'ChairAction' message 🫶
The rest of this section explores the other scenario, where the AS and FCS talk through the MS control channel instead 🌈 This one follows the H.248.19 document related to conferencing in 3GPP, so it's giving standards-approved vibes ✨ The next sequence diagram shows the same use case as the last topology, so it makes exactly the same assumptions as before 💖 That means the BFCP connection between the UA and the FCS is already negotiated and set up, and the UA already knows all the relevant identifiers and floors-resources-associations. The AS also already set up the media mixing on the MS using the MS control channel. The difference this time is that the AS has to do extra setup, like identifying the BFCP-moderated resources, establishing basic policies and instructions about the chair identifiers for each resource, and subscribing to events of interest, because the FCS isn't co-located with the AS anymore 🧚 On top of that, a BFCP session has been established between the AS (acting as a floor chair in this scenario) and the FCS (MS) 👑 Again, every frame the UA tries to send on the media stream is getting dropped by the MS, since the UA still isn't authorized to use the resource. For a SIP UA, this can come from a 'sendonly' field on the media stream in a re-INVITE from the MS 🙈 And once more, the AS has to make sure that no user media control mechanisms (like the ones mentioned in the previous sub-section) can override the floor control 🔥
UA AS MS
(Floor Participant) (Floor Chair) (FCS)
| | |
|<===================== One-way RTP stream ======================|
| | |
| FloorRequest(BFCP) | |
|--------------------------------------------------------------->|
| | |
| | FloorRequestStatus[PENDING](BFCP) |
|<---------------------------------------------------------------|
| | FloorRequestStatus[PENDING](BFCP) |
| |<-----------------------------------|
| | |
| | ChairAction[ACCEPTED] (BFCP) |
| |----------------------------------->|
| | ChairActionAck (BFCP) |
| |<-----------------------------------|
| | |
| | FloorRequestStatus[ACCEPTED](BFCP) |
|<---------------------------------------------------------------|
| | |
. . .
. . .
| | |
| | FloorRequestStatus[GRANTED](BFCP) |
|<---------------------------------------------------------------|
| | 'Floor has been granted' (CtrlChn) |
| |<-----------------------------------|
| | |
|<==================== Bidirectional RTP stream ================>|
| | |
. . .
. . .
Figure 7: Conferencing Example: Floor Control Call FlowOkay so bestie, the UA is a floor participant too 🎀 It sends a FloorRequest to the floor control server (FCS), which is co-located with the MS, saying it wants the floor for the audio stream in the conference 🎤✨. The MS replies with a FloorRequestStatus set to PENDING, which just means "nobody's decided yet" ⏳. Then it passes the request along to the AS, which is playing the floor chair in this example 👑.
The AS checks the BFCP policies for this conference and decides to accept the request ✅. It tells the MS through a BFCP ChairAction message. The MS acknowledges the ChairAction, then tells the UA with a fresh FloorRequestStatus, this time ACCEPTED 💅. But ACCEPTED doesn't mean the floor is hers yet, bestie. It only means the request got a yes 🙈.
Once the MS's queue management says, based on its scheduling algorithms, that the floor can be granted, the MS sends the UA a FloorRequestStatus with GRANTED 🎉 and unmutes the participant in the conference 🔊. When the UA gets that GRANTED notification, it can start sending its media, and it knows the MS won't drop the stream now 🦋.
Now here's a detail you can't miss 👀. If the session was earlier updated with a 'sendonly' on the media stream, the MS must send out a new re-INVITE saying the stream is now 'sendrecv', which means two-way 🔁💜.
This scenario has an automated floor chair, where the AS uses its own policies to make the floor decisions 🤖. A real person in the chair role works the same way, except the incoming request goes to the floor control participant that the chair UA is using instead of to the AS 🧑💻. The decision still comes back in a 'ChairAction' message, same as before 💖.
Another common setup is a BFCP-moderated conference with no chair at all 🌸. Here the MS has to handle incoming requests using predefined policies, like always accepting new ones ⭐. Nobody outside the MS has to make any decisions, because the policies in the MS decide everything instantly ⚡.
As we said before, the case where the FCS is co-located with the AS is way simpler to understand and use 🧚. When the AS fully controls the FCS, queue management included, the AS just tells the MS about floor status changes directly 👑. For example, it can use the control channel to tell the MS to unmute a participant who got the floor for the audio media stream 🎙️.
7. Security Considerations 💀🔐
Okay, security is serious, bestie 😭. This document only describes the architecture for controlling a Media Server 🏗️. It's mostly about how Application Servers and Media Servers talk to each other 🫶. User agents talk to Application Servers using signaling protocols like SIP, and those conversations are out of scope here 📞. Application Servers are responsible for using their signaling protocol's security mechanisms, plus their own application policies, so that only authorized users get service 🛡️.
Media interactions between user agents and Media Servers are also out of scope 🎧. Those are controlled by the Application Servers, which must make sure proper security mechanisms are used 🔒. For example, if the MS is acting as the FCS, the AS sets up the BFCP connection between the user agent and the MS using SIP and the SDP mechanisms in [RFC4583] ☎️. And BFCP [RFC4582] strongly requires TLS for BFCP 🙌.
Media Servers are valuable network resources, so they need protection from unauthorized access 💎. Application Servers use SIP and related standards to set up control channels to Media Servers and to set up media sessions, including BFCP sessions, between an MS and end users 🦋. Media Servers use SIP's security mechanisms to authenticate requests from Application Servers and to make sure those requests haven't been tampered with 🕵️. Using SIP's security mechanisms means only authorized Application Servers can set up sessions with an MS and use its resources through those sessions 👑.
Control channels between an AS and an MS carry the MS control protocol 📡. That protocol affects both what end users see and the resources used on a Media Server 😳. TLS [RFC5246] must be implemented as the transport-level security mechanism for control channels, so the integrity of MS control interactions is protected 🔐.
The resources of an MS can be shared by more than one AS 🤝. Media Servers must stop one AS from accessing or changing resources assigned to another AS 🚫. One way to do this is for the MS to tie ownership of a resource to the AS that first allocated it, and then make sure future requests about that resource match up with the AS that owns it and is responsible for it 💅.
8. Acknowledgments 🌈
The authors want to thank Spencer Dawkins for his detailed reviews and comments, Gary Munson for suggestions, and Xiao Wang for review and feedback 🫶💜.
9. Contributors ✨
This document came out of the Media Control Architecture Design Team 🧩. Besides the editor, these people were on the design team and made substantial text contributions to this document 🎀:
Chris Boulton: cboulton@ubiquity.net
Martin Dolly: mdolly@att.com
Roni Even: roni.even@polycom.co.il
Lorenzo Miniero: lorenzo.miniero@unina.it
Adnan Saleem: Adnan.Saleem@radisys.com10. Informative References 📚
Here's the full reading list, bestie, copied exactly 🤓🧚:
[RFC0793] Postel, J., "Transmission Control Protocol", STD 7,
RFC 793, September 1981.
[RFC2976] Donovan, S., "The SIP INFO Method", RFC 2976,
October 2000.
[RFC3261] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston,
A., Peterson, J., Sparks, R., Handley, M., and E.
Schooler, "SIP: Session Initiation Protocol", RFC 3261,
June 2002.
[RFC3263] Rosenberg, J. and H. Schulzrinne, "Session Initiation
Protocol (SIP): Locating SIP Servers", RFC 3263,
June 2002.
[RFC3264] Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model
with Session Description Protocol (SDP)", RFC 3264,
June 2002.
[RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V.
Jacobson, "RTP: A Transport Protocol for Real-Time
Applications", STD 64, RFC 3550, July 2003.
[RFC3725] Rosenberg, J., Peterson, J., Schulzrinne, H., and G.
Camarillo, "Best Current Practices for Third Party Call
Control (3pcc) in the Session Initiation Protocol (SIP)",
BCP 85, RFC 3725, April 2004.
[RFC3840] Rosenberg, J., Schulzrinne, H., and P. Kyzivat,
"Indicating User Agent Capabilities in the Session
Initiation Protocol (SIP)", RFC 3840, August 2004.
[RFC4145] Yon, D. and G. Camarillo, "TCP-Based Media Transport in
the Session Description Protocol (SDP)", RFC 4145,
September 2005.
[RFC4240] Burger, E., Van Dyke, J., and A. Spitzer, "Basic Network
Media Services with SIP", RFC 4240, December 2005.
[RFC4353] Rosenberg, J., "A Framework for Conferencing with the
Session Initiation Protocol (SIP)", RFC 4353,
February 2006.
[RFC4474] Peterson, J. and C. Jennings, "Enhancements for
Authenticated Identity Management in the Session
Initiation Protocol (SIP)", RFC 4474, August 2006.
[RFC4566] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session
Description Protocol", RFC 4566, July 2006.
[RFC4575] Rosenberg, J., Schulzrinne, H., and O. Levin, "A Session
Initiation Protocol (SIP) Event Package for Conference
State", RFC 4575, August 2006.
[RFC4579] Johnston, A. and O. Levin, "Session Initiation Protocol
(SIP) Call Control - Conferencing for User Agents",
BCP 119, RFC 4579, August 2006.
[RFC4582] Camarillo, G., Ott, J., and K. Drage, "The Binary Floor
Control Protocol (BFCP)", RFC 4582, November 2006.
[RFC4583] Camarillo, G., "Session Description Protocol (SDP) Format
for Binary Floor Control Protocol (BFCP) Streams",
RFC 4583, November 2006.
[RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey,
"Extended RTP Profile for Real-time Transport Control
Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585,
July 2006.
[RFC4960] Stewart, R., "Stream Control Transmission Protocol",
RFC 4960, September 2007.
[RFC5167] Dolly, M. and R. Even, "Media Server Control Protocol
Requirements", RFC 5167, March 2008.
[RFC5239] Barnes, M., Boulton, C., and O. Levin, "A Framework for
Centralized Conferencing", RFC 5239, June 2008.
[RFC5246] Dierks, T. and E. Rescorla, "The Transport Layer Security
(TLS) Protocol Version 1.2", RFC 5246, August 2008.
[SIP-CTRL-FW]
Boulton, C., Melanchuk, T., and S. McGlashan, "Media
Control Channel Framework", Work in Progress,
February 2009.
[W3C.REC-voicexml20-20040316]
Carter, J., Tryphonas, S., Danielsen, P., Burnett, D.,
Rehor, K., McGlashan, S., Ferrans, J., Porter, B., Lucas,
B., and A. Hunt, "Voice Extensible Markup Language
(VoiceXML) Version 2.0", World Wide Web Consortium
Recommendation REC-voicexml20-20040316, March 2004,
<http://www.w3.org/TR/2004/REC-voicexml20-20040316>.
[W3C.REC-xml-20060816]
Sperberg-McQueen, C., Paoli, J., Bray, T., Maler, E., and
F. Yergeau, "Extensible Markup Language (XML) 1.0 (Fourth
Edition)", World Wide Web Consortium Recommendation REC-
xml-20060816, August 2006,
<http://www.w3.org/TR/2006/REC-xml-20060816>.
[XCON-DM] Novo, O., Camarillo, G., Morgan, D., and J. Urpalainen,
"Conference Information Data Model for Centralized
Conferencing (XCON)", Work in Progress, April 2009.Author's Address 🦋
Tim Melanchuk (editor)
Rain Willow Communications
EMail: tim.melanchuk@gmail.comThat's the whole thing, bestie! 🌈💖 Protocols are so iconic when you understand them 💅✨
📦 where this page comes from
- ietf: rfc5567 2009-06
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms