RFC 5707
Media Server Markup Language (MSML)
RFC 5707: Media Server Markup Language (MSML) 🎀
Okay bestie, here's the quick glow-up of the title block 💅 This is an Independent Submission, so it's not coming through the IETF stream. The category is Informational ✨ (not a Standards Track thing), and the ISSN is 2070-1721. It was published in February 2010 🌸
The authors are A. Saleem and Y. Xin from RadiSys, plus G. Sharratt, who was a Consultant at the time 🫶 The header doesn't list anything this RFC obsoletes or updates, so there's no drama there 🙈
Abstract ✨
MSML (Media Server Markup Language) is a language for controlling and invoking lots of different kinds of services on IP media servers 🎧 RadiSys started driving the MSML control interface, and then Intel, Dialogic, and other folks in the industry gave it a bunch of big contributions 🙌
With MSML, clients can decide how multimedia sessions interact on a media server, and they can apply services to one person or to a whole group 👑 It's super handy for conference stuff like video layout and audio mixing, and it can also set up sidebar conferences, personal mixes, and the properties of media streams 🎀 Clients can also define media processing dialogs, which can be part of how apps interact with users or conferences 💖 Transforming media streams to and from users or conferences is one of those interactions, and so are interactive voice response (IVR) dialogs. MSML is what specifies all of this 🦋 On top of that, MSML clients can invoke dialogs with individual users or with groups of conference participants using VoiceXML 🧚
Status of This Memo 📜
This document is not an Internet Standards Track specification. It's published just for informational purposes 🌈
This is a contribution to the RFC Series, and it's independent of any other RFC stream 💜 The RFC Editor picked to publish it at their own discretion, and they make no statement about how valuable it is for implementation or deployment 🍓 Documents the RFC Editor approves for publication are not a candidate for any level of Internet Standard. See Section 2 of RFC 5741 for that ✨
You can find info about the current status of this document, any errata, and how to give feedback at http://www.rfc-editor.org/info/rfc5707 🔥
IESG Note ⚠️
This RFC is not a candidate for any level of Internet Standard. The IETF disclaims any knowledge of how fit this RFC is for any purpose. In particular, the decision to publish is not based on IETF review for things like security, congestion control, or inappropriate interaction with deployed protocols. The RFC Editor chose to publish this at their discretion, so readers should be careful when evaluating its value for implementation and deployment. See RFC 3932 for more information 💀
Copyright Notice 🎀
Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved 💖
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info), as they're in effect on the date this document was published. Please read these documents carefully, because they describe your rights and restrictions with respect to this document 📜
Table of Contents 🗺️
1. Introduction ....................................................4
2. Glossary ........................................................5
3. MSML SIP Usage ..................................................6
3.1. SIP INFO ...................................................7
3.2. SIP Control Framework ......................................8
4. Language Structure .............................................15
4.1. Package Scheme ............................................15
4.2. Profile Scheme ............................................18
5. Execution Flow .................................................19
6. Media Server Object Model ......................................21
6.1. Objects ...................................................21
6.2. Identifiers ...............................................23
7. MSML Core Package ..............................................26
7.1. <msml> ....................................................26
7.2. <send> ....................................................26
7.3. <result> ..................................................27
7.4. <event> ...................................................27
8. MSML Conference Core Package ...................................28
8.1. Conferences ...............................................28
8.2. Media Streams .............................................29
8.3. <createconference> ........................................31
8.4. <modifyconference> ........................................33
8.5. <destroyconference> .......................................34
8.6. <audiomix> ................................................35
8.7. <videolayout> .............................................36
8.8. <join> ....................................................43
8.9. <modifystream> ............................................45
8.10. <unjoin> .................................................46
8.11. <monitor> ................................................47
8.12. <stream> .................................................47
9. MSML Dialog Packages ...........................................51
9.1. Overview ..................................................51
9.2. Primitives ................................................53
9.3. Events ....................................................55
9.4. MSML Dialog Usage with SIP ................................56
9.5. MSML Dialog Structure and Modularity ......................57
9.6. MSML Dialog Core Package ..................................58
9.7. MSML Dialog Base Package ..................................63
9.8. MSML Dialog Group Package .................................81
9.9. MSML Dialog Transform Package .............................85
9.10. MSML Dialog Speech Package ...............................88
9.11. MSML Dialog Fax Detection Package ........................92
9.12. MSML Dialog Fax Send/Receive Package .....................93
10. MSML Audit Package ...........................................100
10.1. MSML Audit Core Package .................................100
10.2. MSML Audit Conference Package ...........................102
10.3. MSML Audit Connection Package ...........................106
10.4. MSML Audit Dialog Package ...............................108
10.5. MSML Audit Stream Package ...............................110
11. Response Codes ...............................................111
12. MSML Conference Examples .....................................113
12.1. Establishing a Dial-In Conference .......................113
12.2. Example of a Sidebar Audio Conference ...................117
12.3. Example of Removing a Conference ........................118
12.4. Example of Modifying Video Layout .......................118
13. MSML Dialog Examples .........................................120
13.1. Announcement ............................................120
13.2. Voice Mail Retrieval ....................................120
13.3. Play and Record .........................................122
13.4. Speech Recognition ......................................125
13.5. Play and Collect ........................................125
13.6. User Controlled Gain ....................................128
14. MSML Audit Examples ..........................................128
14.1. Audit All Conferences ...................................128
14.2. Audit Conference Dialogs ................................129
14.3. Audit Conference Streams ................................130
14.4. Audit All Connections ...................................131
14.5. Audit Connection Dialogs ................................131
14.6. Audit Connection Streams ................................132
14.7. Audit Connection with Selective States ..................133
15. Future Work ..................................................134
16. XML Schema ...................................................134
16.1. MSML Core ...............................................136
16.2. MSML Conference Core Package ............................140
16.3. MSML Dialog Packages ....................................148
16.4. MSML Audit Packages .....................................170
17. Security Considerations ......................................176
18. IANA Considerations ..........................................176
18.1. IANA Registrations for 'application' MIME Media Type ....176
18.2. IANA Registrations for 'text' MIME Media Type ...........178
18.3. URN Sub-Namespace Registration ..........................179
18.4. XML Schema Registration .................................180
19. References ...................................................181
19.1. Normative References ....................................181
19.2. Informative References ..................................182
Acknowledgments ..................................................1831. Introduction ✨
Okay so media servers have these dynamic pools of media resources 🎀. Control agents and other users of media servers (called media server clients) can define and create tons of different services based on how they configure and use those resources 💅. And honestly? That configuration, plus how the resources interact, often gets changed dynamically during a call to match how an app is interacting with a user 🦋.
For example, a call can start with an initial IVR dialog before it gets placed into a conference 🎤. Calls can hop from a main conference to a sidebar conference and then back again 💖. Individual calls can also be directly bridged to make small n-way calls or simple sidebars 🌸. None of these change the SIP [n1] dialog or RTP [i3] session 🙈. Yet they all DO affect the media flow and processing inside the media server, so it's a whole vibe 😭.
The Media Server Markup Language (MSML) is an XML [n2] language used to control the flow of media streams, plus the services applied to those streams inside a media server 👑. It's used to invoke lots of different types of services on individual sessions, groups of sessions, and conferences 🧚. MSML lets you create conferences, bridge different sessions together, and bridge sessions into conferences 🌈.
MSML can also create user interaction dialogs and allows media transforms to be applied to media streams ✨. Media interaction dialogs made with MSML let you build IVR dialog sessions for individual users, as well as for groups of users in a conference 🫶. Dialogs can also be specified using other languages, like VoiceXML [n5], which supports complete single-party application logic executed on the media server 💜.
MSML is a transport independent language, so it doesn't rely on underlying transport mechanisms, and its language semantics are independent of transport 🦋. That said, SIP is a typical, commonly used transport mechanism for MSML, invoked using the SIP URI scheme 💅. This specification defines using MSML dialogs with SIP as the transport mechanism 🎀.
A network connection can be established with the media server using SIP 📞. Media received and transmitted on that connection flows through different media resources on the media server, depending on the requested service 🍓. Basic Network Media Services with SIP [n7] defines conventions for associating a basic service with a SIP Request-URI 🌸. MSML lets a control agent dynamically apply and change services during the lifetime of the SIP dialog 🔥.
MSML was designed to handle control and manipulation of media processing operations (for example, announcement, IVR, play and record, automatic speech recognition (ASR), text to speech (TTS), fax, and video) 🎙️. It also handles control and relationships of media streams (for example, simple and advanced conferencing) 👯. It provides a general-purpose media server control architecture 👑. MSML can additionally be used to invoke other, more complex IVR languages such as VoiceXML 💖.
The MSML control interface has been widely deployed across the industry, with numerous client-side and server-side implementations, since 2003 🥹. The in-service commercial deployments cover a wide variety of applications, including (but not limited to) IP multimedia conferencing, network voice services, IVR, IVVR (interactive voice and video response), and voice/video mail 🌈.
2. Glossary 📖
- Media Server: a general-purpose platform for executing real-time media processing tasks 🧚. This is a logical function that maps either to a single physical device or to a portion of a physical device 💜.
- Media Server Client: an application that originates MSML requests to a media server, also referred to as a control agent in this specification 🫶.
- Network Connection: a participant that represents the termination on a media server of one or more RTP [i3] sessions (for example, audio and video) associated with a call 📞. Network connections are established and removed using a session establishment protocol such as SIP 🔥. An instance of a network connection is independent of MSML processing instructions applied to it ✨.
- Dialog: an automated IVR participant 🎙️. Examples of dialogs may be announcement players, IVR interfaces, or voice recorders 🎀. Dialogs may be defined in MSML or using VoiceXML [n5] 💅.
- Conference: an intermediary function that provides multimedia mixing and other advanced conferencing services 👯. This specification currently considers conferences with audio and/or video media types, but is extensible to other media types 🌸.
- Identifier: a name that is used to refer to a specific instance of an object on the media server, such as a conference or a dialog 🍓. Identifiers are composed of one or more terms, where each term identifies an object class and instance 🌈.
- Object: the generic term for a media server entity that terminates, originates, or processes media 🦋. This specification defines four classes of objects and specifies mechanisms to create them, join them together, and destroy them 👑.
- Participant Object: an object in a media server that sources original media in a call and/or receives and terminates media in a call 💖.
- Intermediary Object: an object in a media server that acts on media within a call for the benefit of the participants 🥹.
- Independent Object: an object that can exist on a media server independent of other objects 🧚.
- Operator: an intermediary transformer that modifies or transforms a media stream 🔥. Examples of operators may be audio gain controls, video scaling, or voice masking 🎙️. MSML defines operators as media transform objects, which transform media using operations such as gain control, when applied to media streams 💅.
- Media Stream: a single media flow between two objects 🫶. A media stream has a media type and may be unidirectional or bidirectional ✨.
3. MSML SIP Usage 📞
SIP is used to create and modify media sessions with a media server according to the procedures defined in RFC 3261 [n1] 💅. Often, SIP third party call control [i4] will be used to create sessions to a media server on behalf of end users 🎀. MSML is used to define and change the service that a user connected to a media server receives 🦋. MSML clients are application servers, soft-switches, or other forms of control agents, and they SHOULD have an authorized security relationship with the media server 🔐. MSML itself does not define authorization mechanisms 🙅♀️.
MSML transactions are originated based on events that occur in the application domain 🌸. These events may be independent from any media or user interaction 🌈. For example, an application may want to play an announcement to a conference warning that its scheduled completion time is approaching 📢. Applications themselves are structured in lots of different ways 🧩. Their structure and requirements contribute to their selection of protocols and languages 💖. To accommodate differing application needs, MSML is designed to be neutral to other languages and independent of the transport used to carry it 👑.
MSML is purposely designed to be transport independent 🦋. In this release of the specification, SIP INFO [i5] and SIP Control Framework [i11] have been chosen as transport mechanisms for MSML, as described in the following sections 🎀.
3.1. SIP INFO 🎀
SIP INVITE and INFO [i5] requests and responses MAY be used to carry MSML 💅. INFO requests allow asynchronous mid-call messages within SIP with few additional semantics 🌸. Plus, there are existing widely deployed implementations of that method, it aids in initial developments that are closely coupled with SIP session establishment, and it allows MSML to be directly associated with user dialogs when third party call control is used 🙌.
Although INFO is sometimes considered not to be a suitable general-purpose transport mechanism for messages within SIP, there have been proposals to make it more acceptable 🤔. MSML may evolve to include other SIP usage and/or to work with other protocols or as a stand-alone protocol established through SIP, in future releases of this document 🔮.
MSML supports several models for client interaction 🧚. When clients use 3PCC to establish media sessions on behalf of end users, clients will have a SIP dialog for each media session 📞. MSML MAY be sent on these dialogs 🦋. However, the targets of MSML actions are not inferred from the session associated with the SIP dialog 🚫. The targets of MSML actions are always explicitly specified using identifiers, as previously defined 🎯.
An application, after interacting with a user, may want to affect multiple objects within a media server 👯. For example, tones or messages are often played to a conference when connections are added or removed 🔔. A separate message may also be played to a participant as they join, or to moderators 💌. Explicit identifiers (that is, not inferred from a transport mechanism) allow these multiple actions to be easily grouped into a single transaction sent on any SIP dialog 🍓.
MSML also supports a model of dedicated control associations 👑. This supports decoupled application architectures, where a client can control media server services without also establishing all of the media sessions itself 🥰. Control associations are created using SIP, but they do not have any associated media session 🙈. Although INFO messages will initially be sent on this SIP dialog, just as with dialogs associated with media sessions, it is possible that in the future, the SIP dialog will be used to establish a separate control session (defined in SDP [n9]) that does not use SIP as the transport for MSML messages 🔮.
A media server using MSML also sends asynchronous events to a client using MSML scripts in SIP INFO 💫. Events are sent based on previous MSML requests, and they are sent within the SIP dialog on which the MSML request that caused the event to be generated was received 📬. If this dialog no longer exists when the event is generated, the event is discarded 🗑️.
Events may be generated during the execution of a dialog created by a <dialogstart> element 🎙️. For example, dialogs can send events based on user input 👂. VoiceXML dialogs, on the other hand, generally interact with other servers outside of MSML using HTTP 🌐.
An event is also generated when the execution of a dialog terminates, because of either completion or failure ✅. The exact information returned depends on the dialog language, the capabilities of the dialog execution environment, and what was requested by the dialog 🧾. Both MSML and VoiceXML [n5] allow information to be returned when they exit 📤. These events may be sent in a SIP INFO or a SIP BYE 📨. SIP BYE is used when the dialog itself specifies that the connection should be disconnected, for example, through the use of the <disconnect> element 👋.
Conferences may also generate events based upon their configuration 🎪. An example of this is the notification of the set of active speakers 🗣️.
3.2. SIP Control Framework ✨
okay so the SIP Control Framework [i11] MAY be used as a transport mechanism for MSML 🚚💨 it's basically a delivery truck for MSML commands
the Control Framework gives you a generic way to set up and report on remotely started commands 📦 it leans on a bunch of stuff from the Session Initiation Protocol (SIP) [n1] to find its partner and set up a reliable channel for back-and-forth control chats 🫶 and honestly? compared to SIP INFO, this one is the more general-purpose option 💅 it isn't stuck with the limits that SIP INFO has, so it's giving freedom 🦋
the framework also brings in the idea of a Control Package 🎀 that's basically a specific, explicit way of using the Control Framework for one particular set of interactions. this spec already defines a bunch of packages so MSML can control the media server in lots of ways 🌈 we're talking basic dialog, advanced conferencing, advanced dialog, and audit service 👑 every one of these packages gets its own unique Control Package name so MSML can be used with the Control Framework 🧚
this section is where we cover the mandatory info that MUST be specified when you define a Control Framework Package, as laid out in SIP Control Framework [i11] 📝 no skipping this one, bestie 😭
3.2.1. Control Framework Package Names
the Control Framework [i11] says every Control Package definition has to specify and register a unique name 🔥
the MSML spec uses a hierarchical naming scheme so you can see which packages inherit from which 🌸 for example, package "msml-x" is derived from package "msml", and package "msml-x-y" is derived from package "msml-x" 🧠 it's like a family tree, iykyk 🌳
here's the list of Control Package names that the MSML spec reserves 💖
"msml": this Control Package supports MSML Core Package as specified
in section 7.
"msml-conf": this Control Package supports MSML Conference Core
Package as specified in section 8.
"msml-dialog": this Control Package supports MSML Dialog Core Package
as specified in section 9.6.
"msml-dialog-base": this Control Package supports MSML Dialog Base
Package as specified in section 9.7.
"msml-dialog-group": this Control Package supports MSML Dialog Group
Package as specified in section 9.8.
"msml-dialog-transform": this Control Package supports MSML Dialog
Transform Package as specified in section 9.9.
"msml-dialog-speech": this Control Package supports MSML Dialog
Speech Package as specified in section 9.10.
"msml-dialog-fax-detect": this Control Package supports MSML Dialog
Fax Detection Package as specified in section 9.11.
"msml-dialog-fax-sendrecv": this Control Package supports MSML Dialog
Fax Send/Receive Package as specified in section 9.12.
"msml-audit": this Control Package supports MSML Audit Core Package
as specified in section 10.1.
"msml-audit-conf": this Control Package supports MSML Audit
Conference Package as specified in section 10.2.
"msml-audit-conn": this Control Package supports MSML Audit
Connection Package as specified in section 10.3.
"msml-audit-dialog": this Control Package supports MSML Audit Dialog
Package as specified in section 10.4.
"msml-audit-stream": this Control Package supports MSML Audit Stream
Package as specified in section 10.5.so an application server using the Control Framework as its transport for MSML MUST use one or more package names, depending on what it needs from the media server 🎯 no vibes-based guessing here 😤 the package name(s) show up in the "Control-Packages" SIP header that's in the SIP INVITE dialog request which creates the control channel, as specified in [i11] 💌
and last but not least, the "Control-Packages" value MAY be re-negotiated using the SIP re-INVITE mechanism 🔄 flexible energy, periodt 💅✨
3.2.2. Control Framework Messages
okay so the way CONTROL, response, and REPORT messages from [i11] get used is different for every single Control Package in MSML 💅 so each one gets its own little section below ✨
MSML Core Package "msml" 🎀
The application server may send a CONTROL message with a body of MSML request using the following elements to the MS:
<msml>: the root element that may contain a list of child elements that request a specific operation. The child elements are defined in extended packages (e.g.,"msml-conf"and"msml-dialog"). This element is also the root element that contains an MSML result and event.<send>: sends an event to the specified recipient within the media server. Specific event types are defined within the extended packages.
The media server replies with a response message containing a MSML result using the following elements:
<result>: reports the results of an MSML transaction.
The media server MAY send the MSML event to the application server, in a REPORT or CONTROL message, using the element <event>. The actual content of the <event> and which Control Framework message to use are defined within the extended packages. 🦋
MSML Conference Core Package "msml-conf" 🏠
This package extends the MSML Core Package to define a framework for creation, manipulation, and deletion of a conference. 🎪
The AS can send a CONTROL message with a body of the MSML request that contains one or multiple conference-related commands to the MS. The MS then replies with a response message with a body of the MSML result to indicate whether or not the request has been fulfilled. 💌
During the lifetime of a conference, whenever an event occurs, the media server MAY send CONTROL messages containing MSML events to notify the application server. The application server SHOULD reply with a response message with no MSML body to acknowledge the event has been received. 🔔
This package does NOT use the REPORT message. 🙅♀️
Dialog Core Package "msml-dialog" 💬
This package extends the MSML Core Package to define the structural framework and abstractions for MSML dialogs. 🧱
The application server MAY send CONTROL messages containing a MSML request using the following elements:
<dialogstart>: instantiate an MSML media dialog on a connection or a conference.<dialogend>: terminates an MSML dialog.<send>: sends an event and an optional namelist to the dialog, dialog group, or dialog primitive.<exit>: used by the dialog description language to cause the execution of the MSML dialog to terminate.
For the <dialogstart> command, the response message MUST contain an MSML result that indicates that the dialog has been started successfully. The MSML result MAY contain <dialogid> to return the dialog identifier, if the identifier was assigned by the media server. Subsequently, zero or more MSML events MAY be initiated by the media server in (update) REPORT messages to report information gathered during the dialog. Finally, an MSML event "msml.dialog.exit" SHOULD be generated in a (terminate) REPORT message when the dialog terminates (e.g., MSML execution of <exit>). ⏯️
For the <dialogend> and <send> commands, the response message contains the final MSML result that indicates that the request has either been fulfilled or rejected. ✅
Dialog Base Package "msml-dialog-base" 🧁
This package extends the MSML Dialog Core Package to define a set of base functionality for MSML dialogs. The extension defines individual media primitives, including <play>, <dtmfgen>, <tonegen>, <record>, <dtmf> and <collect>, to be used as child element of <dialogstart>. This package does not change the framework message usage as defined by the MSML Dialog Core Package. 🎵
Dialog Transform Package "msml-dialog-transform" 🌀
This package extends the MSML Dialog Core Package to define a set of transform primitives that works as filter on half-duplex media streams. The extension defines transform primitives, including <vad>, <gain>, <agc>, <gate>, <clamp> and <relay>, that MAY be used as child elements of <dialogstart>. This package does not change the framework message usage as defined by the MSML Dialog Core Package. 🎚️
Dialog Group Package "msml-dialog-group" 🫂
This package extends the MSML Dialog Core, Base, and Transform Packages to define a single control flow construct that specifies concurrent execution of multiple media primitives. The extension defines the <group> element that MAY be used as a child element of <dialogstart> to enclose multiple media primitives, such that they can be executed concurrently. This package does not change the framework message usage as defined by the MSML Dialog Core Package. 👯♀️
Dialog Speech Package "msml-dialog-speech" 🗣️
This package extends the MSML Dialog Core and MSML Base Package to define functionality that MAY be used for automatic speech recognition and text to speech. The extension extends the <dialogstart> and the <play> elements.
For <dialogstart>, it defines a new child element <speech> to activate grammars or user input rules associated with speech recognition. For <play>, it defines a new child element <tts> to initiate the text-to-speech service.
This package does not change the framework message usage as defined by the MSML Dialog Core Package. 🤖
Dialog Fax Detection Package "msml-dialog-fax-detect" 📠
This package extends the MSML Dialog Core Package to define primitives provide fax detection service. The extension defines a primitive <faxdetect> to be used as a child element of <dialogstart>. This package does not change the framework message usage as defined by the MSML Dialog Core Package. 🔍
Dialog Fax Send/Receive Package "msml-dialog-fax-sendrecv" 📨
This package extends the MSML Dialog Core Package to define primitives that allow a media server to provide fax send or receive service. The extension defines new primitives <faxsend> and <faxrcv>, to be used as a child element of <dialogstart>. This package does not change the framework message usage as defined by the MSML Dialog Core Package. 📬
Dialog Audit Core Package "msml-audit" 🕵️♀️
This package extends the MSML Core Package to define a framework for auditing media resource(s) allocated on the media server. 🧾
This package follows a simple request/response transaction, allowing the application server to send CONTROL messages containing MSML <audit> requests. The media server MUST reply with a response message containing the result. The result is contained within the <auditresult> element, returning the queried state information. 📋
This package does NOT use the REPORT message. 🙅♀️
Dialog Audit Conference Package "msml-audit-conf" 🎤
This package extends the MSML Audit Core Package to define conference specific states that MAY be queried via the <audit> command and the corresponding response MUST be returned by the <auditresult> element. This package does not change the framework message usage as defined by the MSML Audit Core Package. 📊
Dialog Audit Connection Package "msml-audit-conn" 🔌
This package extends the MSML Audit Core Package to define connection specific states that MAY be queried via the <audit> command and the corresponding response MUST be returned by the <auditresult> element. This package does not change the framework message usage as defined by the MSML Audit Core Package. 📡
Dialog Audit Dialog Package "msml-audit-dialog" 💭
This package extends the MSML Audit Core Package to define dialog specific states that MAY be queried via the <audit> command and the corresponding response MUST be returned by the <auditresult> element. This package does not change the framework message usage as defined by the MSML Audit Core Package. 🗂️
Dialog Audit Stream Package "msml-audit-stream" 🌊
This package extends the MSML Audit Core Package to define stream specific states that MAY be queried via the <audit> command and the corresponding response MUST returned by the <auditresult> element. This package does not change the framework message usage as defined by the MSML Audit Core Package. 💧
3.2.3. Common XML Support
The XML schema described in [i11] MUST be supported by all Control Packages defined by MSML. However, the "connection-id" value MUST be constructed as defined by MSML (i.e., the identifier MUST contain a local dialog tag only, while the SIP Control Framework [i11] requires that the "connection-id" contain both local and remote dialog tags). 🧩
3.2.4. Control Message Body
A valid CONTROL body message MUST conform to the MSML schema, as included in this specification, for the MSML package(s) used. 📦
3.2.5. REPORT Message Body
A valid REPORT body message MUST conform to the MSML schema, as included in this specification, for the MSML package(s) used. 📝
4. Language Structure ✨
4.1. Package Scheme ✨
okay so the big way MSML gets extended is the "package" 🎀 A package is a bundle of one or more XML schemas that add new features by using new or extended elements and attributes. Every package except the ones in this document lives in its own standards doc, like an Internet Draft or an RFC. Any package that extends base MSML MUST include references to the MSML base set of schemas from the Internet Drafts. A schema in a package MUST only extend MSML, meaning it must not change the existing spec. no touching the original 🙅♀️
A given MSML script will reference all the schemas for the packages whose stuff it actually uses. A script MUST reference the MSML base and optionally any extension package(s). see the IANA Considerations section 📋
Each package MUST define its own namespace so that elements or attributes with the same name in different packages don't clash 💅 And if a script uses an element or attribute that lives in a package (instead of the base), it MUST prefix that name with the namespace name. it's giving organized 🗂️
MSML is built from a core package that gives you the structure without any specific features. Extra packages sit on top of the core and add the functional stuff. You can mix any combination of extra packages with the core. The diagram below shows the MSML packages defined in this document 🦋
+--------------------------------------------------------+
| MSML Core |
+--------------------------------------------------------+
/ \ \
+--------+ +--------+ +-------+
| Dialog | | Conf | | Audit |
| Core | | Core | | Core |
+--------+ +--------+ +-------+
________ \_______________________________________ |
------------------------------------------------ |
/ \ \ \ \ \ |
+------+ +---------+ +------+ +------+ +------+ +-------+ |
|Dialog| |Dialog | |Dialog| |Dialog| |Dialog| |Dialog | |
|Base | |Transform| |Group | |Speech| |Fax | |Fax | |
+------+ +---------+ +------+ +------+ |Detect| |Send/ | |
+------+ |Receive| |
+-------+ |
________________________|
-------------------------
/ \ \ \
+-----+ +-----+ +------+ +------+
|Audit| |Audit| |Audit | |Audit |
|Conf | |Conn | |Dialog| |Stream|
+-----+ +-----+ +------+ +------+- MSML Core Package (Mandatory) 💜
This describes the minimum base framework that MUST be implemented to support additional core packages. non-negotiable bestie 👑
- MSML Conference Core Package (Conditionally Mandatory, for Conferencing) 🎤
This describes the audio and multimedia basic and advanced conferencing package. It MAY be implemented, so it's a vibe choice 🌸
- MSML Dialog Core Package (Conditionally Mandatory, for Dialogs) 💬
This describes the dialog core package, which MUST be implemented for any dialog services. However, systems that only do conferencing MAY omit support for MSML dialogs. The MSML Dialog Core Package is the framework that the other dialog packages plug into. The MSML Dialog Base Package MUST be supported, while all the other dialog packages MAY be supported.
- MSML Dialog Base Package (Conditionally Mandatory, for Dialogs)
- MSML Dialog Group Package (Optional)
- MSML Dialog Transform Package (Optional)
- MSML Dialog Fax Detection Package (Optional)
- MSML Dialog Fax Send/Receive Package (Optional)
- MSML Dialog Speech Package (Optional)
- MSML Audit Core Package (Conditionally Mandatory, for Auditing) 🔍
This describes the audit core package, which MUST be implemented to support auditing services. It's the framework the additional audit packages are supported within.
- MSML Audit Conference Package (Conditionally Mandatory, for Auditing Conference, Conference Dialog, and Conference Stream)
- MSML Audit Connection Package (Conditionally Mandatory, for Auditing Connection, Connection Dialog, and Connection Stream)
- MSML Audit Dialog Package (Conditionally Mandatory, for Auditing Dialog, and MUST be used with either MSML Audit Conference Package or MSML Audit Connection Package)
- MSML Audit Stream Package (Conditionally Mandatory, for Auditing Stream, and MUST be used with either MSML Audit Conference Package or MSML Audit Connection Package)
The formal way to extend MSML dialogs is to define a new package 🧁 The new package MUST provide a text description of what extensions are included and how they work. It MUST also define an XML schema file (if applicable) that defines the new package, which can be done through extension, restriction of an existing package, or a specific profile of an existing package. Dependencies on other packages MUST be stated. For example, a package that extends or restricts another one depends on the original package's spec. Finally, the new package MUST be assigned a unique name and version. receipts please 📝
Here are the types of things you can define in new packages ✨
- new primitives
- extensions to existing primitives (events, shadow variables, attributes, content)
- new recognition grammars for existing primitives
- new markup languages for speech generation
- languages for specifying a topology schema
- new predefined topology schemas
- new variables / segment types (sets & languages)
- new control flow elements
MSML packages get assembled into a specific MSML profile that different implementations can share 🤝 The base MSML dialog profiles defined in this document are made up of the MSML Core Package, MSML Dialog Core Package, MSML Dialog Base Package, MSML Dialog Group Package, MSML Transform Package, MSML Fax Packages, and the MSML Speech Package.
MSML extension packages that define primitives MUST define the following for each primitive within the package:
- the function that the primitive performs
- the attributes that may be used to tailor its behavior
- the events that it is capable of understanding
- the shadow variables that provide access to information determined as a result of the primitive's operation
The mechanism for making sure a media server and its client share a compatible set of packages is not defined here 🙈 For now, the expectation is that provisioning will be used, possibly alongside a future auditing capability. Also, when used in SIP networks, packages could be defined using feature tags, and the procedures for Indicating User Agent Capabilities in SIP [i1] could be used to let a media server describe its capabilities to other user agents. it's a whole vibe 💫
4.2. Profile Scheme 🦋
Not every device or app using MSML needs the entire MSML schema 💅 For example, a media processing device might only support audio announcements, only audio simple conferencing, or only multimedia IVR. It's highly desirable to have a system that describes exactly what portion of MSML a particular media processing device or control agent supports.
The package scheme we just talked about lets MSML functionality be grouped into packages that rely on the MSML core package. It lets a portion of the full MSML spec be implemented on a per-package basis, and it also sets up a framework for future extension packages. But inside a given package, sometimes only a subset of the package's functionality is needed. To support subsets of packages at a finer granularity than the package level, a profile scheme is required 🎀
MSML package profiles identify a subset of a given MSML package with specific definitions of elements and attributes. Each MSML package profile MUST be accompanied by one or more corresponding schemas. Using the examples above, there could be an audio announcements profile of the MSML Dialog Base Package, an audio simple conferencing profile of the MSML Conference Core Package, and a multimedia IVR profile of the MSML Dialog Base Package. so many possibilities 🌈
MSML package profiles MUST be published separately from the MSML specification, in one or more standards documents (e.g., Internet Drafts or RFCs) dedicated to MSML package profiles. Profiles would not be registered with IANA, and any organization is additionally free to create its own profile(s) if it needs to. no gatekeeping here, we love a creator 🫶
5. Execution Flow 🔥
okay so MSML runs on a model where there's one single control context inside a media server 🧚. That context can have one or many SIP [n1] dialogs attached to it 💜. It's assumed that any SIP dialogs tied to the MSML control context were already authorized, as appropriate, by mechanisms outside the scope of MSML 🫶. Not MSML's job, bestie 💅.
The control context keeps track of the state of all the media objects and media streams inside the media server ✨. It receives and processes every MSML request from authorized SIP dialogs, and it grabs every event that media objects generate internally and sends it out on the right SIP dialog 📮. An MSML request can create brand new media objects and streams, and it can modify or destroy any existing ones 🦋.
A single MSML request can be just one action for the media server to handle, in which case it's giving very simple command request energy 🎀. But honestly, it's often more natural for a client to ask for several actions at once, or to want a few actions tightly coordinated by the media server 👑. Multiple MSML elements received in a single request MUST be processed sequentially in document order 📋.
An example of the first situation is creating a conference and joining it with an initial participant 🌸. An example of the second is unjoining one or more participants from a main conference and joining them to a sidebar conference 🍓. In the first case, network latency may not be a big deal, but it's just simpler for the client to combine the requests 💖. In the second case, the extra network latency between separate requests could mean the participant hears audio loss, and that is NOT it 😭.
Each MSML request is processed as a single transaction 🎀. A media server MUST ensure that it has the necessary resources available to carry out the complete transaction before executing any elements of the request 🔍. If it doesn't have enough resources, it MUST return a 520 response and MUST NOT execute the transaction 🚫.
The MSML request MUST be checked for well-formedness and validated against the schema before any elements get executed 📝. This lets XML [n2] errors get reported right away, and it cuts down on failures in the middle of a transaction, which would otherwise leave only part of it executed 🙈.
Each element is expected to execute immediately ⚡. Elements such as <dialogstart>, which take an unpredictable amount of time, are "forked" and executed in a separate thread (see MSML Dialog Packages) 🧵. Once they're successfully forked, execution just carries on with the element after the </dialogstart> ✨. So MSML doesn't give you mechanisms to sequence or coordinate other operations with dialog elements 🙃.
Processing within a transaction MUST stop if any errors occur 🛑. Elements that were already executed before the error are NOT rolled back, so they stay done 😬. It's the client's responsibility to decide what to do next based on the results in the response 🧠. Most elements MAY contain an optional "mark" attribute 🔖. The value of that attribute from the last successfully executed element MUST be returned in an error response 📍. Also, errors that happen during the execution of a dialog occur outside the context of an MSML transaction, so those get reported in an asynchronous event 📣.
Transaction results come back as part of the SIP request response 💌. They show whether the transaction succeeded or failed 💯. The result MUST also include identifiers for any objects the media server created for which the client didn't provide an instance name 🆔. And if the transaction fails, the reason for the failure MUST be returned, and an indication of how much of the transaction was executed before the failure occurred SHOULD be returned too 📊. Iconic behavior, honestly 👑.
6. Media Server Object Model 🦋
Media servers are general-purpose platforms for running real-time media processing tasks 🌈. Those tasks range from simple stuff, like playing announcements, to seriously complex stuff, like speech interfaces, centralized multimedia conferencing, and sophisticated gaming applications 🎮.
Calls get set up on a media server using SIP 📞. Clients will often use SIP third party call control (3PCC) [i4] to set up calls to a media server on behalf of end users 🫶. MSML doesn't require 3PCC, though. It only needs the client and the media server to share a common identifier for the call and its associated RTP [i3] sessions 🔗.
Objects are the entities that source, sink, or modify media streams 🎀. A media stream is a bidirectional or unidirectional media flow between objects on a media server 🌊. The subsections below define the classes of objects that exist on a media server and how they're identified in MSML 🧚.
6.1. Objects 💖
A media object is an endpoint of one or more media streams 🎯. It can be a connection that terminates RTP sessions coming in from the network, or a resource that transforms or manipulates media 🛠️. MSML defines four classes of media objects, and each class defines the basic properties of how object instances are used within a media server 🍭. Most classes, though, need the client to define what specific instances actually do, using MSML or other languages such as VoiceXML 📜.
These are the classes of media processing objects that are defined. The class names are in parentheses 💅:
- network connection (conn)
- conference (conf)
- dialog (dialog)
A network connection is an abstraction for the media processing resources involved in terminating the RTP session(s) of a call 📡. For audio services, a connection instance presents a full-duplex audio stream interface within a media server 🎧. Multimedia connections have multiple media streams of different media types, and each one corresponds to an RTP session 🎬. Network connections get instantiated through SIP [n1] 💫.
A conference represents the media resources and state information needed for a single logical mix of each media type in the conference (e.g., audio and video) 🎤. MSML models multiple mixes or views of the same media type as separate conferences 🪞. Each conference has multiple inputs, and those inputs can be divided into classes so an application can request different media treatment for different participants 💜. For example, the video streams for some participants might be assigned to fixed regions of the screen, while the ones for other participants only show up when they're speaking 🎥.
A conference has a single logical output per media type 📺. For each participant, that output is the audio conference mix, minus any audio that participant contributed, plus the video mix shared by all conference participants 🌷. Video conferences using voice activated switching have an optional ability to show the previous speaker to the current speaker 🙌.
Conferences are instantiated using the <createconference> element 🏗️. The content of the <createconference> element specifies the parameters of the audio and/or video mixes 🧾.
Dialogs are a class of objects that represent automated participants 🤖. They're similar to network connections from a media flow perspective, and they may have one or more media streams as the abstraction for their interface within a media server 🌀. The difference is that dialogs are created and destroyed through MSML, and the media server itself implements the dialog participant 🪄. Dialogs are instantiated through the <dialogstart> element 🚀. The contents of the <dialogstart> element define the desired or expected dialog behavior 📋. Dialogs can also be invoked by referencing VoiceXML as the dialog description language 🎙️.
Operators are functions used to filter or transform a media stream 🎚️. The function an operator instance fulfills is defined as a property of the media stream 🧩. Operators may be unidirectional or bidirectional, and they have a media type 🎨. Unidirectional operators reflect simple atomic functions, like automatic gain control, filtering tones out of conferences, or applying specific gain values to a stream 🔊. They have a single media input, connected to the media stream from one object, and a single media output, connected to the media stream of a different object 🔀.
Bidirectional operators have two media inputs and two media outputs 👯. One input and output is associated with the stream to one object, and the other input and output is associated with a stream to a different object 💞. Bidirectional objects may treat the media differently in each direction 🙃. For example, an operator could change the media sent to a connection based on speech it recognized, or on dual-tone multi-frequency (DTMF) it received from that connection 📞. Operators are implicitly instantiated when streams are created or modified using the elements <join> and <modifystream>, respectively 🧠.
The relationships between the different object classes (conf, conn, and dialog) are shown in the figure below 🖼️.
+--------------------------------------+
| Media Server |
| |
|------+ ,---. |
| | +------+ / \ |
<== RTP ==>| conn |<---->| oper |<---->( conf ) |
| | +------+ \ / |
|------+ `---' |
| ^ ^ |
| | | |
| | +------+ +------+ | |
| | | | | | | |
| +-->|dialog| |dialog|<---+ |
| | | | | |
| +------+ +------+ |
+--------------------------------------+A single, full-duplex instance of each object class is shown, along with the common relationships between them 🧩. An operator (such as gain) sits between a connection and a conference, and dialogs are shown participating both with an individual connection and with a conference 🎀. The figure isn't meant to imply only one-to-one relationships 🙅. Conferences will often have hundreds of participants, and either connections or conferences may be interacting with more than one dialog 🥳. For example, one dialog might be recording a conference while other dialogs announce participants joining or leaving it 🎙️✨.
6.2. Identifiers 🦋
okay bestie, so objects get referenced with identifiers made of one or more terms 💅 Each term says what kind of object it is (the object class) and which one specific object it means (the instance). The two get split by a colon : inside the term.
Identifiers get handed out the moment an object is first created ✨ The MSML client or the media server can both pick the instance name. If the client doesn't pick one, the media server picks it for them 🎀 Whatever name the server picks gets sent back to the client as a complete object identifier in the response to the request that made the object.
Some objects are totally fine existing on their own 🌸 Connections can be created through SIP at any time, and then MSML can link their media to other objects to build services. Conferences can be created with specific resources reserved, just waiting for participants to connect 🧚
So connections and conferences are independent objects because they can exist on a standalone basis. Their identifiers are just a single term, like conf:abc or conn:1234. If a client picks its own instance names for independent objects, it must use globally unique instance names 🌈 One way to get that is to put the client's domain name into the name.
Dialogs exist to provide a service to independent objects 💖 They can act as a participant in a conference, or interact with a connection kinda like a two-participant call. Dialogs depend on independent objects existing, and that shows up in how their identifiers are built. Operators change the media flow between other objects, like adding gain between a connection and a conference. Since operators are just media transform primitives defined as properties of the media stream, they don't get identifiers and are created implicitly ✨
Dialog identifiers are a list of terms separated by slashes (/). The left-most term has to specify a conference or a connection, and that becomes the root. Here's an example of a dialog acting as a conference participant:
conf:abc/dialog:recorderAll objects except connections are created using MSML 💅 Connections are created when media sessions get established through SIP. There are a few options that clients and media servers can use to set up a shared instance name for a connection and its media streams.
When media servers support multiple media types, the instance name SHOULD be a call identifier that can be used to identify the collection of RTP sessions associated with a call 🦋 When MSML is used together with SIP and third party call control, the call identifier MUST be the same as the local tag the media server assigned to identify the SIP dialog. That's the tag the media server adds to the "To" header in its response to an initial INVITE transaction. RFC 3261 requires those tag values to be globally unique 😭
Here's an example of a connection identifier: conn:74jgd63956ts.
With third party call control, the MSML client acts as a back-to-back user agent (B2BUA) to set up the media sessions 🫶 SIP dialogs get established between the client and the media server, which lets the media server's local tag be used as a connection identifier. If third party call control isn't used, a SIP event package MAY be used so a media server can tell a subscribed client about new sessions 💜
Identifiers let every object on a media server be uniquely addressed, but they can also point at multiple objects at once ✨ Right now there are two ways to do that:
- wildcards
- common instance names
An identifier can reference multiple objects when a wildcard is used as an instance name. MSML reserves the instance name made of a single asterisk (*) to mean all objects that share the same identifier root and class 👑 Instance names containing an asterisk cannot be created. Wildcards MUST only be used as the right-most term of an identifier, and they MUST NOT be used as part of the root for dialog identifiers 🙈 Wildcards are only allowed where explicitly indicated below.
These are valid wildcards:
conf:abc/dialog:*
conn:*And this is illegal wildcard usage:
conf:*/dialog:73849Even though identifiers share a common syntax, MSML elements restrict which classes of objects are valid in a given context 💀 For example, it's valid to join two connections together, but it's not valid to join two IVR dialogs. Not me trying to join two IVR dialogs 😭
7. MSML Core Package ✨
This section describes the core MSML package, which MUST be supported in order to use any other MSML packages 💖 The core package is basically a framework with no explicit functionality of its own. Functional packages get used on top of it 🎀
7.1. <msml> 🌸
<msml> is the root element. When a media server receives it, it defines the set of operations that make up a single MSML request 🧚 Operations are requested by what's inside the element. Each operation MAY appear zero or more times as a child of <msml>. Specific operations are defined in the conference package and in the set of dialog packages 🦋
The results of a request, or the contents of events a media server sends, are also wrapped in the <msml> element ✨ The results of the transaction go in as a body in the response to the SIP request that carried the transaction. That response contains any identifiers the media server assigned to newly created objects. Every message a media server generates is correlated to an object identifier. Objects and identifiers are covered in section 6 (Media Server Object Model) 💅
Attributes:
version: "1.1" Mandatory
7.2. <send> 💅
Events are used to affect how different objects behave inside a media server 🔥 The <send> element sends an event to a specified recipient within the media server.
Attributes:
event: the name of an event. Mandatory.target: an object identifier. When the identifier is for a dialog, it may optionally be appended with a slash "/" followed by the target to be included in an MSML dialog<send>. Mandatory.valuelist: a list of zero or more parameters that are included with the event.mark: a token that can be used to identify execution progress in the case of errors. The value of the mark attribute from the last successfully executed MSML element is returned in an error response. Therefore, the value of all mark attributes within an MSML document should be unique 🍓
7.3. <result> 🎀
The <result> element reports the results of an MSML transaction 🫶 It's included as a body in the final response to the SIP request that started the transaction. An optional child element <description> can include text that explains what error responses mean 😭 Response codes are defined in section 11 (Response Codes).
Attributes:
response: a numeric code showing the overall success or failure of the transaction, and for a failure, the reason why. Mandatory.mark: in the case of an error, the value of the mark attribute from the last successfully executed element that included the mark attribute.
If something fails, a description of the reason SHOULD be given using the child element <description> ✨
Three other child elements let the response include identifiers for objects that the request created but that didn't have instance names specified by the client 🌈 Those elements are <confid> and <dialogid>, for objects created through a <createconference> and <dialogstart> respectively.
7.4. <event> 🧚
The <event> element is how a media server notifies an event to a media server client 🔥 The MSML Core Package defines three types of events: "msml.dialog.exit", "msml.conf.nomedia", and "msml.conf.asn". These correspond to a dialog terminating, a conference being automatically deleted when the last participant leaves, and the notification of the current set of active speakers for a conference, in that order 💜 Events can also be generated by an executing dialog. In that case, the event type is specified by the dialog (see MSML Dialog Core Package <send>).
Attributes:
name: the type of event. If the event is generated because of the execution of an MSML dialog<send>, the value MUST be the value of the "event" attribute from the<send>element within the MSML Dialog Core Package. If the event is generated because of the execution of an<exit>, the value MUST be "moml.exit". If the event is generated because of the execution of a<disconnect>, the value MUST be "moml.disconnect". If the event is generated because of an error, the value must be "moml.error". Mandatory.id: the identifier of the conference or dialog that generated the event or caused the event to be generated. Mandatory.
<event> has two children, <name> and <value>, which hold the name and value respectively of each namelist item associated with the event 🎀
8. MSML Conference Core Package 👑
8.1. Conferences 🌸
A conference has a mixer for each type of media it supports 💖 Each mix has a matching description that defines how the media from participants contributes to that mix. A mixer has multiple inputs that get combined in a media-specific way to create a single logical output 🦋
The elements that describe the mix for each media type are called mixer description elements. They are:
<audiomix> defines the parameters for mixing audio media.
<videolayout> defines the composition of a video window.
Those elements, defined in sections 8.6 (Audio Mix) and 8.7 (Video Layout) respectively, get used as content of the <createconference> element to set up a conference's initial properties ✨ They get used inside the <modifyconference> element to change a conference's properties once it's been created, or inside the <destroyconference> element to remove individual mixes from the conference 💅
Conferences can be terminated by an MSML client, either by using the <destroyconference> element to remove the entire conference or by removing the last mixer(s) associated with it 🙈 A media server can also terminate conferences automatically, based on criteria specified when the conference was created. When the conference gets deleted, any remaining participants will have their associated SIP dialogs either left unchanged or deleted, depending on the value of the "term" attribute specified when the conference was created 🫶
8.2. Media Streams ✨
okay bestie, so every object has at least one media input and one media output for each type of media it supports 🎀 each object class is the one that decides how many input and output objects of that class it supports 💅 media streams get created when objects are joined, either explicitly with <join> or implicitly when dialogs are created with <dialogstart> 🦋 dialog creation happens in two stages: first the resources for the dialog instance get allocated and configured, and then those resources get implicitly joined to the dialog target while the dialog is running 🧚 for the full scoop, check out the MSML Dialog Base Package 📚
a join operation creates a bidirectional audio stream between two objects by default 🎧 video and unidirectional streams can also be created 🌈 a media stream is made by connecting the output of one object to the input of another object, and then the other way around too (assuming it's a bidirectional or full-duplex join) 💖
many objects can only take a single input for each type of media 🫶 in this specification, only the conference object class can take an arbitrary number of inputs 👑 so if a stream gets requested to an object that already has a stream of the same type plugged into its single input, the result depends on what kind of media stream it is 🤔
audio mixing works by summing audio signals 🎶 and automatically mixing audio streams is super useful! like, you can bridge two streams to make a simple three-way call, or bridge a private announcement with a (whispered) conference mix for one participant 🤫 for general conferences though, an MSML client SHOULD create an audio conference and then join participants to it 🌸 and conference mixers SHOULD subtract each participant's audio from the mix so they don't hear themselves 🙈
when a media server gets a request to join an audio stream to the single audio input of an object that already has an audio stream connected, it SHOULD automatically bridge the new stream with the existing one, making a mix of the two audio streams ✨ how many streams can be bridged this way is implementation specific 🍓 it is RECOMMENDED that a media server support bridging at least two streams 💜 and a media server that cannot bridge a new stream with any existing streams MUST fail the operation requesting the join 💀
video is a whole different vibe 🎬 there are lots of ways two video streams can be combined and shown, like side by side in separate panes, picture in picture, or one pane showing only one stream at a time based on something like active speaker 📺 each option makes a very different presentation and needs really different media resources, so automatic bridging of video is not supported 🙅♀️ a media server MUST fail requests to join a new video stream to an object that only supports a single video input and already has a video stream connected to that input 😭 if you want an object to have multiple video streams joined to it, the object itself has to be capable of supporting multiple video streams 🎀 conference objects can support multiple video streams and give you a way to specify the mixing presentation for them 🦋
a media server MUST NOT establish any streams unless it can create all the streams requested by an operation 🚫 streams can only be created if both objects support a media type and at least one of these conditions is true:
- Each object that is to receive media is not already receiving a stream of that type.
- Any object that is to receive media and is already receiving a stream of that type supports receiving an additional stream of that type. The only class of objects defined in this specification that directly support receiving multiple streams of the same type are conferences.
- The media server is able to automatically bridge media streams for an object that is to receive media and that is already receiving a stream of the requested type. The only type of media defined in this specification that MAY be automatically bridged is audio.
the directionality of media streams on a connection is modeled separately from what SDP [n9] allows for the matching RTP [i3] sessions 🎀 media servers MUST respect the SDP in what they actually transmit, but they MUST NOT let the SDP change the directionality when joining streams inside the media server 💅
8.3. <createconference> 🎪
<createconference> is used to allocate and configure the media mixing resources for conferences 🎪 the properties for each type of media mix the conference needs get described inside the content of the <createconference> element 🧁 mixer descriptions are covered in the Audio Mix and Video Layout sections 🎵 when no mixer descriptions are specified, the default behavior MUST be equivalent to including a single <audiomix> 🫶
clients can ask a media server to automatically delete a conference when a specified condition happens, using the "deletewhen" attribute 🗑️ a value of "nomedia" means the conference MUST be deleted when no participants are left, and when that happens, an "msml.conf.nomedia" event MUST be notified to the MSML client 👀 a value of "nocontrol" means the conference MUST be deleted when the SIP [n1] dialog carrying the <createconference> element gets terminated, and when that happens, a media server MUST terminate all participant dialogs by sending a BYE for their associated SIP dialog 💀 and a value of "never" MUST leave the ability to delete a conference under the control of the MSML client 🦋
Attributes:
- name: the instance name of the conference. If the attribute is not present, the media server MUST assign a globally unique name for the conference. If the attribute is present but the name is already in use, an error (432) will result and MSML document execution MUST stop. Events that the conference generates use this name as the value of their "id" attribute (see section 7.4 (
<event>)). 🏷️ - deletewhen: defines whether a media server should automatically delete the conference. Possible values are "nomedia", "nocontrol", and "never". Default is "nomedia". 🗑️
- term: when true, the media server MUST send a BYE request on all SIP dialogs still associated with the conference when the conference is deleted. Setting term equal to false lets clients start dialogs on connections once the conference has completed. Default is "true". 🎬
- mark: a token that MAY be used to identify execution progress in the case of errors. The value of the mark attribute from the last successfully executed MSML element is returned in an error response. Therefore, the value of all mark attributes within an MSML document should be unique. 🔖
here's an example of creating an audio conference 🎤 this conference lets at most two participants contend to be heard, and it reports the set of active speakers no more frequently than every 10 seconds ⏰
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<createconference name="example">
<audiomix>
<n-loudest n="3"/>
<asn ri="10s"/>
</audiomix>
</createconference>
</msml>8.3.1. <reserve> 🛎️
conference resources can be reserved by including the <reserve> element as a child of <createconference> 💼 <reserve> lets you specify a set of resources that a media server will reserve for the conference ✨ any requests for resources beyond those that have been reserved should be honored on a best-effort basis by a media server 🌟
Attributes:
- required: boolean that specifies whether
<createconference>should fail if the requested resources are not available. When set to false, the conference will be created, with no reserved resources, if the complete reservation cannot be honored. Default is "true". 🥺
8.3.1.1. <resource> 🧩
the resources to be reserved are defined using <resource> 📦 the contents of these elements describe a resource to be reserved 🎀 descriptions are implementation dependent 🤷♀️ media servers that support MSML dialogs may use the elements from that package as the basis for resource descriptions 💅 each resource element may use the attribute "n" to define the quantity of the resource to reserve 🔢
Attributes:
- n: number of resources to be reserved. Default is 1. 🔢
- type: specifies whether the resource is to be reserved by each individual participant or reserved as a shared conference resource. Valid values for this attribute are "individual" or "shared". Default is "individual". 🙋♀️
for example, the following creates a conference and reserves two types of resources 🫧 one resource element can represent resources shared by all participants of the conference, while the other can represent resources reserved for each of the expected participants 💖
<createconference>
<reserve>
<resource n="20">
<!--description of resources used by each participant-->
</resource>
<resource n="2" type="shared">
<!--description of the shared conference resources-->
</resource>
</reserve>
</createconference>8.4. <modifyconference> 🎀
okay bestie, so ✨ every property of an audio mix or the presentation of a video mix can be changed while the conference is alive, using <modifyconference> 💖. audio mix changes go in an <audiomix> child of <modifyconference>, and that same move can add an audio mixer if there wasn't one yet 🔥. video presentation changes go in a <videolayout> child, and it works the same way for adding a video mixer if none was allocated before 🦋
mixers get removed by putting a mixer description element inside <destroyconference/> 💅
features and presentation aspects get enabled, added, or modified by including the element(s) that define them inside a mixer description 🫶. you have to include the complete specification of the element, just like you would when the conference gets created. the new definition completely replaces any previous definition, so no leftovers 💀. only the stuff defined by elements in the mixer descriptions gets affected. any existing configuration aspects of a conference that you don't specify in <modifyconference/> MUST maintain their current state in the media server 👑
for example, if an MSML client wanted to change the minimum reporting interval for active speaker notification from the one shown in the Conference Examples section (<createconference>), it would send this to the media server:
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<modifyconference id="conf:example">
<audiomix>
<asn ri="4"/>
</audiomix>
</modifyconference>
</msml>this would also turn on active speaker notification if it wasn't already enabled 🎀. the N-loudest mixing is unaffected 💜
multiple elements MAY be included in the mixer descriptions, similar to when conferences are created 🧚. like, in a video conference, the video mix description (<videolayout>) could say the layout should change so the regions currently showing participants get smaller, and new region(s) get created to support more participants. it's giving main character energy 😭. a media server MUST make all of the requested changes or none of the requested changes 🔥
more examples of modifying conferences are in the Conference Examples section 🌈
Attributes:
- id: the identifier for a conference. Wildcards MUST NOT be used. Mandatory.
- mark: a token that can be used to identify execution progress in the case of errors. The value of the mark attribute from the last successfully executed MSML element is returned in an error response. Therefore, the value of all "mark" attributes within an MSML document SHOULD be unique. 🍓
8.5. <destroyconference> 💀
<destroyconference> is used to delete mixers, or to delete the entire conference and all of its state and shared resources 🫧. when a mixer is removed, all of the streams joined to that mixer get unjoined. when a conference is destroyed, SIP dialogs for any remaining participants MUST be maintained or removed based on the value of the "term" attribute when the conference was created 👑
when there's no element content, <destroyconference/> deletes the entire conference 💅. individual mixers get removed by including a mixer description element that identifies the mix (or mixes) to be removed as content of <destroyconference/>. <audiomix/> is used to remove audio mixers, and <videolayout/> is used to remove video mixers 🦋. when one or more mixer descriptions are specified, the media server MUST only delete the specified mixer and MUST NOT affect any other existing mixers 🔥. when <audiomix/> or <videolayout/> is identified for individual removal, other feature aspects of the mix MUST NOT be included. if they are specified anyway, the media server MUST ignore those elements 🙈. when the last mixer is removed from a conference, a media server MUST remove all conference state, leaving or removing any remaining SIP dialogs as described above 🌸
Attributes:
- id: the identifier for a conference. Mandatory.
- mark: a token that can be used to identify execution progress in the case of errors. The value of the mark attribute from the last successfully executed MSML element is returned in an error response. Therefore, the value of all "mark" attributes within an MSML document SHOULD be unique. 🍓
8.6. <audiomix> 🎤
the properties of the overall audio mix are specified using the <audiomix> element ✨
Attributes:
- id: an optional identifier for the audio mix.
- samplerate: an integer value that specifies the sample rate (in Hz) for the audio mixer. Optional, default value of 8000. 💜
an example of the description for an audio mix looks like this:
<audiomix id="mix1">
<asn ri="10s"/>
<n-loudest n="3"/>
</audiomix>8.6.1. <n-loudest> 🔊
the <n-loudest> element says that participants contend to be included in the conference mix based on their audio energy 🏆. when the element is not present, all participants are mixed 🫶
Attributes:
- n: the number of participants that will be included in the audio mix based on having the greatest audio energy. Mandatory.
8.6.2. <asn> 👀
the <asn> element enables notification of active speakers 🗣️. active speakers MUST be notified using the <event> element with an event name of msml.conf.asn. the namelist of the event consists of the set of active speakers, and the name of each item is the string speaker, with a value of the connection identifier for that connection 💖
Attributes:
- ri: the minimum reporting interval defines the minimum duration of time that must pass before changes to active speakers will be reported. A value of zero disables active speaker notification. 🙅♀️
- asth: specifies the active speaker threshold (in units of dBm0). Valid value range is 0 to -96. Optional, default is -96. 🎚️
an example of an active speaker notification looks like this:
<event name="msml.conf.asn" id="conf:example">
<name>speaker</name>
<value>conn:hd93tg5hdf</value>
<name>speaker</name>
<value>conn:w8cn59vei7</value>
<name>speaker</name>
<value>conn:p78fnh6sek47fg</value> </event>8.7. <videolayout> 📺
a video layout is specified using the <videolayout> element 🎬. it's used as a container to hold the elements that describe all of the properties of a video mix. the parameters of the window that displays the video mix are defined by the <root> element 🪟. when the video mix is composed of multiple panes, the location and characteristics of those panes are defined by one or more <region> elements. a <region> element is not required when only a single video stream is displayed at one time and none of the visual attributes of regions are required 🧚
some regions may display a video stream based on selection criteria, instead of continuously showing one participant's stream 🎓. one example is a distance learning lecture, where the instructor sees each of the students periodically displayed in a region. when a region is used to display one of a number of streams, it is placed as a child of a <selector> element 💅
Attributes:
- type: specifies the language used to define the layout. Layouts defined using MSML MUST use the value
text/msml-basic-layout. This is the same convention as defined for the layout package from the W3C SMIL 2.0 specification [i6]. The default when omitted istext/msml-basic-layout. 📜 - id: an optional identifier for the video layout. 🆔
8.7.1. <root> 🪟
the <root> element describes the root window, or virtual screen, that the conference video mix will be displayed in 🌈. simple conferences can display participant video directly within the root window, but more complex conferences will use regions for this purpose. areas of the window that aren't used to display video will show the root window background 🖼️
all video presentations require a root window. it MUST be present when a video mix is created, and it cannot be deleted; however, its attributes MAY be changed using the <modifyconference> element 🔒✨
Attributes:
- size: the size of the root window, specified as one of the five standard common intermediate formats (e.g., CIF, QCIF). 📐
- backgroundcolor: the color for the root window background, defined using the values for the "background-color" property of the CSS2 specification [n10]. 🎨
- backgroundimage: the URI for an image to be displayed as the root window background. Transparent portions of the image allow the background color to show through. 🌅
8.7.2. <region> 🎀
Okay bestie, <region> elements are the video panes that show participant video streams 📺✨. They always get rendered on top of the root window 🪟.
The size of a region is set relative to the root window with the relativesize attribute 📐. Those sizes are fractions, like 1/4 or 1/3, which keep the aspect ratio of the original video and make scaling way more efficient 💅.
Regions get placed on the root window with the position attributes top and left 📍. They mark the top left corner of the region as an offset from the top left corner of the root window. You can give them as a number of pixels, or as a percent of the vertical or horizontal dimension of the root window. Percent values end with a percent (%) character. 33% and 67% should be read as 1/3 and 2/3, which makes it super easy to line regions up with sizes that are relative to the root window 🧩.
Here's an example of a video layout with six regions 🌈:
+-------+---+
| | 2 |
| 1 +---+
| | 3 |
+---+---+---+
| 6 | 5 | 4 |
+---+---+---+ <videolayout type="text/msml-basic-layout">
<root size="CIF"/>
<region id="1" left="0" top="0" relativesize="2/3"/>
<region id="2" left="67%" top="0" relativesize="1/3"/>
<region id="3" left="67%" top="33%" relativesize="1/3">
<region id="4" left="67%" top="67%" relativesize="1/3"/>
<region id="5" left="33%" top="67%" relativesize="1/3"/>
<region id="6" left="0" top="67%" relativesize="1/3"/>
</videolayout>Which part of the root window a region covers depends on its position and its size 🗺️. When regions overlap, they stack up by their priority attribute 🥞. The region with the highest priority value sits below all the others and gets hidden by anything overlapping it 🫥. The region with the lowest non-zero priority value sits on top of everything and never gets hidden by overlaps 👑. priority takes values between 0 and 1. A value of zero disables the region, frees up any resources tied to it, and unjoins any video stream that was showing in it 🚫.
Regions that don't set a priority get one from the media server when the conference is created 🤖. The first region inside the <videolayout> element without a priority gets a priority of one, the second gets two, and so on. So every region that doesn't set a priority ends up underneath every region that does 🛏️. Among the no-priority regions, they stack from top to bottom in the order they show up inside <videolayout> 📚.
For example, say a layout looks like this:
<videolayout>
<root size="CIF"/>
<region id="a" ... priority=".3" .../>
<region id="b" ... />
<region id="c" ... priority=".2" ...>
<region id="d" ... />
</videolayout>Then the layering goes, from top to bottom: c, a, b, d 🥰.
Anything of a region that hangs off the edge of the root window gets cropped ✂️. For example, this layout:
<videolayout>
<root size="CIF"/>
<region id="foo" left="50%" top="50%" relativesize="2/3"/>
</videolayout>would look roughly like this:
+-----------+
| root |
|background |
| +-----+--
| | |//
| | foo |//
+-----+-----+//
|////////Visual attributes are how you style the look of individual regions 🎨. You can add a border, plus a title and/or a logo. Text and logos show up as images on top of the region's video, and they sit below every region with a lower priority 💖. The visual attributes are title, titletextcolor, titlebackgroundcolor, bordercolor, borderwidth, and logo 🦋.
Visual attributes can also be set on individual streams (Video Stream Properties) 🎞️. When both a region and a stream set visual attributes, the stream ones MUST take precedence 🔥. That way, streams that get picked for display automatically (Stream Selection) still get their proper text and logos 🫶. The region's visual attributes only appear when no stream is linked to that region 🌸.
Two more attributes on a region, blank and freeze, control the state of the video shown in it 🧊. When blank or freeze is set to "true", the media server MUST display the region either as a blank region, or with the video image frozen on the last frame it received 🥶.
These two attributes live on regions and aren't allowed on streams, because that seems to be the common use case 💅. If you applied them to streams, only that one stream would be affected inside a selector while the others kept displaying normally 🙃. Outside of personal mixing scenarios, you can get the same result by having the participant mute their own transmission to the media server 🤫.
Here are the attributes for each region 📝:
id: a name that can be used to refer to the region 🏷️.left: the position of the region from the left side of the root window ⬅️.top: the position of the region from the top of the root window ⬆️.relativesize: the size of the region, expressed as a fraction of the root window size 📏.priority: a number between 0 and 1 that sets the precedence when rendering overlapping regions. A value of zero disables the region 🥇.title: text to be displayed as the title for the region 📰.titletextcolor: the color of the text 🌷.titlebackgroundcolor: the color of the text background 🎨.bordercolor: the color of the region border 🖍️.borderwidth: the width of the region border 📏.logo: the URI of an image file to be displayed 🖼️.freeze: a boolean value, with a default of"false", that defines whether the video image should be frozen at the currently displayed frame 🧊.blank: a boolean value, with a default of"false", that defines whether the region should display black instead of the associated video stream 🖤.
8.7.3. <selector> 💅
It's really common to want one of several video streams picked automatically for display 🎯. The <selector> element defines the selection criteria and its parameters 📋. The selection algorithm is set by the method attribute 🧠. The selection methods defined right now let you do voice activated switching, or step through the associated video streams one at a time 🔁.
The regions that will display the selected video stream go inside the <selector> element as child elements 🪆. Putting regions inside a <selector> doesn't change how they're laid out next to regions that aren't part of the selection 🧭. For simple video conferences that show video right in the root window, you can nest the <root> element as a child of <selector> 🌟. Region elements MUST NOT be used in that case 🙅♀️.
For example, here's a super common layout where the active speaker's video shows in the big region ("1") at the top left, while five other participants are in regions around the edge 🎤💜:
+-------+---+
| | 2 |
| 1 +---+
| | 3 |
+---+---+---+
| 6 | 5 | 4 |
+---+---+---+ <videolayout type="text/msml-basic-layout">
<root size="CIF"/>
<selector id="switch" method="vas">
<region id="1" left="0" top="0" relativesize="2/3"/>
</selector>
<region id="2" left="67%" top="0" relativesize="1/3"/>
<region id="3" left="67%" top="33%" relativesize="1/3">
<region id="4" left="67%" top="67%" relativesize="1/3"/>
<region id="5" left="33%" top="67%" relativesize="1/3"/>
<region id="6" left="0" top="67%" relativesize="1/3"/>
</videolayout>Every selector method has to be defined so that it still works when there's only one region as its child 🫧. Selector methods that handle more than one child region MUST specify how they work across those multiple regions 🧩. Media server implementations MAY support only a single region for methods that are defined to allow multiple regions 🤏.
A participant's video gets its selector or region through the display attribute of <stream>, used during a join operation 🚪. Picking a selector means the stream gets shown according to that selector's method 🎬. Picking a region supports continuous presence display of participants 👯. Some streams can be joined with both a selector and a region 💫. In that case, the <blankothers> attribute decides whether the streams tied to a continuous presence region should get blanked when that stream is selected for display in one of the selector regions 🙈.
These attributes are common to all selector methods 📝:
id: a name that can be used to refer to the selector 🏷️.method: the name of the method used to select the video stream. A value of"vas"(see the following section, Voice Activated Switching) MAY be specified 🎙️.status: specifies whether the selector is"active"or"disabled"🟢.blankothers: when"true", video streams that are also displayed in continuous presence regions will have those continuous presence regions blanked when the stream is displayed in a selection region 🫣.
8.7.3.1. Voice Activated Switching ("vas") 🦋
Voice activated switching (VAS) shows the video stream that matches the participant who's speaking right now 🗣️✨. You specify it with a selector method value of "vas" 💜.
If the active speaker's video isn't currently showing in a selection region, it replaces the video in whichever region is showing the speaker who was least recently active 🔄. If the active speaker's video is already showing in a selection region, nothing changes at all 🙌. When VAS is applied to a single region, the current speaker just ends up displayed in that region 👑.
Attributes:
si: switching interval, which is the minimum period of time that must elapse before the video is allowed to switch to the active speaker ⏳.speakersees: defines whether the active speaker sees the "current" speaker (themselves) or the "previous" speaker 👀.
8.8. <join> 🔗
okay so <join> is the element that creates one or more streams between two independent objects 💅 streams can be audio or video, and they can be bidirectional or unidirectional. a bidirectional stream is basically two unidirectional streams glued together, and you can tweak each one on its own 🫶 the streams you want get specified by <stream> elements (section <stream>) as the content of <join> ✨
no content at all? then <join> defaults to a bidirectional audio stream 🎧 so if you've only got one type of stream between two objects, or you've only got a unidirectional stream, you can use <join> to add another media type or make the stream bidirectional by including the right <stream> elements. to turn a bidirectional stream into a unidirectional one, use <unjoin> (section <unjoin>) to remove the direction you don't need anymore 🦋
<stream> elements also set the properties of a stream, like gain, voice masking, or tone clamping for audio, or labels and other visual stuff for video 🌸 properties are often set asymmetrically, meaning one direction only. if one direction should behave differently from the other, you need two <stream> elements inside the <join>, one for each direction, when you're creating a bidirectional stream 💜
If a media server can provide services using both compressed or uncompressed media, the MSML client may need to distinguish within requests which format is to be used. When compressed streams are created, both objects must use the same media format or an error response (450) is generated.
Attributes:
- id1: an identifier of either a connection or conference. Wildcards MUST NOT be used. Mandatory. Any other object class results in a 440 error. 🙅♀️
- id2: an identifier of either a connection or conference. Wildcards MUST NOT be used. Mandatory. Any other object class results in a 440 error.
- mark: a token that can be used to identify execution progress in the case of errors. The value of the mark attribute from the last successfully executed MSML element is returned in an error response. So the value of all mark attributes within an MSML document SHOULD be unique. 🏷️
okay now the fun part, a call center coaching scenario 🎀 a supervisor can listen to the convo between an agent and a customer and drop hints to the agent, and the customer can't hear those hints. one join sets up a stream between the agent and the customer, another sets up a stream between the agent and the supervisor, and a third join makes a half-duplex stream from the customer to the supervisor. the media server automatically bridges the media streams from the customer and the supervisor for the agent, and from the customer and the agent for the supervisor. iconic teamwork 👑
assuming these connections, each with a single audio stream:
conn:supervisor
conn:agent
conn:customerthis is how you create the media flows we just talked about:
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<join id1="conn:supervisor" id2="conn:agent"/>
<join id1="conn:agent" id2="conn:customer"/>
<join id1="conn:supervisor" id2="conn:customer">
<stream media="audio" dir="to-id1"/>
</join>
</msml>here's joining a participant to a multimedia conference 📺 it assumes the conference has a video presentation region named "topright". the "display" attribute is explained in the section Video Stream Properties.
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<join id1="conn:hd83t5hf7g3" id2="conf:example">
<stream media="audio"/>
<stream media="video" dir="from-id1" display="topright"/>
<stream media="video" dir="to-id1"/>
</join>
</msml>8.9. <modifystream> 🪄
media streams can have different properties, like the gain on an audio stream or a visual label on a video stream 🎬 these properties get specified as the content of <stream> elements (section <stream>). <modifystream> is how you change the properties of a stream: you include one or more <stream> elements that should have their properties changed.
okay this part is important 🚨 stream properties MUST be set as specified by the <stream> element used as a child of the <modifystream> element. any properties you leave out of the <stream> element when modifying a stream MUST remain unchanged. setting a property for only one direction of a bidirectional stream MUST NOT affect the other direction. if you want to change the directionality of streams, issue an <unjoin> followed by a <join>. any streams between the two objects that aren't included within <modifystream> MUST NOT be affected 🙌
Attributes:
- id1: an identifier of either a conference or a connection. The instance name MUST NOT contain a wildcard if "id2" contains a wildcard. Mandatory.
- id2: an identifier of either a conference or a connection. The instance name MUST NOT contain a wildcard if "id1" contains a wildcard. Mandatory.
- mark: a token that can be used to identify execution progress in the case of errors. The value of the mark attribute from the last successfully executed MSML element is returned in an error response. So the value of all mark attributes within an MSML document is RECOMMENDED to be unique. 🏷️
8.10. <unjoin> 🧹
<unjoin> removes one or more media streams between two objects. if there's no content in the <stream> element, then all media streams between the objects MUST be removed 💀 you can remove individual streams by specifying them with <stream> elements, and the streams you didn't specify MUST NOT be removed. to change a bidirectional stream into a unidirectional one, unjoin the direction you no longer need using the <unjoin> element. operator elements MUST NOT be specified within <stream> elements when you're unjoining streams with the <unjoin> element. any stream operators that do get specified MUST be ignored 🫠
<unjoin> and <join> can be used together to move a media stream, like from a main conference to a sidebar conference 🧚
Attributes:
- id1: an identifier of either a conference or a connection. The instance name MUST NOT contain a wildcard if "id2" contains a wildcard. Mandatory.
- id2: an identifier of either a conference or a connection. The instance name MUST NOT contain a wildcard if "id1" contains a wildcard. Mandatory.
- mark: a token that can be used to identify execution progress in the case of errors. The value of the mark attribute from the last successfully executed MSML element is returned in an error response. So the value of all mark attributes within an MSML document SHOULD be unique. 🏷️
this example removes a participant from a conference and plays a leave tone for everyone still in it 🔔
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<unjoin id1="conn:jd73ht89sf489f" id2="conf:1"/>
<dialogstart target="conf:1" type="application/moml+xml">
<play>
<audio uri="file://leave_tone.wav"/>
</play>
</dialogstart>
</msml>8.11. <monitor> 👀
<monitor> is a specialized unidirectional join that copies the media destined for a connection object. one use for <monitor> is quality monitoring within a conference 📊 the media stream can be removed using the <unjoin> element (see the section <unjoin>).
Attributes:
- id1: an identifier of the connection to be monitored. Mandatory. Any other object class results in a 440 error. Wildcards MUST NOT be used. 🙅♀️
- id2: an identifier of the object that is to receive the copy of the media destined to id1. id2 may be a connection or a conference. Mandatory. Any other object class results in a 440 error. Wildcards MUST NOT be used.
- compressed: "true" or "false". Specifies whether the join should occur before or after compression. When "true", id2 must be a connection using the same media format as id1 or an error response (450) is generated. Default is "false".
- mark: a token that can be used to identify execution progress in the case of errors. The value of the mark attribute from the last successfully executed MSML element is returned in an error response. So the value of all mark attributes within an MSML document SHOULD be unique. 🏷️
8.12. <stream> 🎶
individual streams get specified using the <stream> element 🎵 it MAY be included as a child element in any of the stream manipulation elements <join>, <modifystream>, or <unjoin>.
the type of stream is set with a "media" attribute, using values that match the top-level MIME media types defined in RFC 2046 [i7]. this specification only covers audio and video media. other specifications may define procedures for additional types 🎞️
a stream is bidirectional when there's no direction attribute "dir" present. a stream is unidirectional when a direction attribute is present. the "dir" attribute MUST have a value of "from-id1" or "to-id1", depending on the direction you need. these values are relative to the identifier attributes of the parent element 🧭
the compressed attribute tells you whether the stream is compressed, when that matters 🗜️ what happens when the attribute isn't present is implementation specific. joining compressed streams acts much like an RTP [i3] relay.
the properties of media streams get specified as the content of <stream> elements when the element is a child of <join> or <modifystream>. <stream> elements MUST NOT have any content when they're used as a child of <unjoin> to identify specific streams to remove 🚫
some properties are defined in MSML as extra attributes or child elements of <stream> that are specific to the media type. the ones for audio streams and video streams are defined in the following two sub-sections. operators, which you can think of as properties of the media stream, MAY be specified as child elements of the <stream> element ✨
Attributes:
- media: "audio" or video". Mandatory
- dir: "from-id1" or "to-id1".
- compressed: "true" or "false". Specifies whether the stream uses compressed media. Default is implementation specific.
8.12.1. Audio Stream Properties ✨
Okay so audio mixes can be set up to only mix the N-loudest people 🎤, but lowkey there are some "preferred" participants who should always get to talk. 💖 When audio streams join a conference that uses N-loudest audio mixing, you have to identify the preferred streams. 🫶
A preferred audio stream is identified with the "preferred" attribute. The "preferred" attribute MAY be used for an audio stream that is input to a conference and MUST NOT be used for other streams. 💅
Additional attributes of the <stream> element for audio streams are:
Attributes:
preferred: a boolean value that defines whether the stream does not contend for N-loudest mixing. A value of"true"means that the stream MUST always be mixed while a value of"false"means that the stream MAY contend for mixing into a conference when N-loudest mixing is enabled. Default is"false". 🍓
There are two elements that can be used to change the characteristics of an audio stream, as defined below. 🎀
8.12.1.1. <gain>
The <gain> element can be used to adjust the volume of an audio media stream. 🔊 You can set it to a specific gain amount, have it automatically adjust the gain to a target level, or mute the stream. 🤫
Attributes:
id: an optional identifier that may be referenced elsewhere for sending events to the gain primitive. 🧚amt: a specific gain to apply, specified in dB, or the string"mute", which indicates that the stream should be muted. This attribute MUST NOT be used if"agc"is present. 🙈agc: a boolean indicating whether automatic gain control is to be used. This attribute MUST NOT be used if"amt"is present. 💜tgtlvl: the desired target level for AGC, specified in dBm0. This attribute MUST be specified if"agc"is set to"true". This attribute MUST NOT be specified if"agc"is not present. 🌈maxgain: the maximum gain that AGC may apply. Maxgain is specified in dB. This attribute MUST be used if"agc"is present and MUST NOT be used when"agc"is not present. 👑
8.12.1.2. <clamp>
The <clamp> element is used to filter tones and/or audio-band DTMF from a media stream. 🧹
Attributes:
dtmf: a boolean indicating whether DTMF tones should be removed. 📵tone: a boolean indicating whether other tones should be removed. 🔇
8.12.2. Video Stream Properties 🎬
Video mixes define a presentation that may have multiple regions, such as a quad-split. 🪟 Each region displays the video from one or more participants. When video streams are joined to such a conference, the region that will display the video needs to be specified as part of the join operation. 🦋
The region that will display the video is specified using the "display" attribute. The "display" attribute MUST be used for a video stream that is input to a conference and MUST NOT be used for other streams. The value of the attribute MUST identify a <region> (see the section <region>) or a <selector> (see the section <selector>) that is defined for the conference. A stream MUST NOT be directly joined to a region that is defined within a selector. 🚫 Changing the value of the "display" attribute can be used to change where in a video presentation layout a video stream is displayed. 🌸
Additional attributes of the <stream> element for video streams are:
Attributes:
display: the identifier of a video layout region or selector that is to be used to display the video stream. 🎞️override: specifies whether or not the given video stream is the override source in the region defined by the"display"attribute. Valid values are"true"or"false". Optional, default value is"false". Only a video stream that is input to a conference can be the override source. A particular region can have at most one override source at a time. The most recently joined video stream with this attribute set to"true"becomes the override source. When there's an override source in place, its video is always displayed in the region, regardless of what video selection algorithm (either a selector or continuous presence mode) is configured for that region. Once the override source is cleared, the conference MUST revert back to the original video selection algorithm. 👑🔥
8.12.2.1. <visual>
Some regions of video conferences may display different streams automatically, such as when voice activated switching is used. 🗣️ Connections MAY also be joined directly without the use of video mixing. In these cases, the <visual> element may be used to define visual display properties for a stream. ✨
The <visual> element MAY use any of the visual attributes defined for regions (see the section <region>). This allows the visual aspects of regions within a <selector> to be tailored to the selected video stream, or for streams that are directly joined to display a name or logo. 🏷️💅
9. MSML Dialog Packages 🧚
9.1. Overview
MSML Dialog Packages define an XML [n2] language for composing complex media objects from a vocabulary of simple media resource objects called primitives. 🧩 It is primarily a descriptive or declarative language for describing media processing objects. MSML dialogs operate on a single or multiple streams that are identified by the MSML document outside the scope of the MSML Dialog Package. 📄
MSML dialogs are intended to be used in different environments. 🌍 As such, the language itself does not define how an MSML dialog is used. Each environment in which an MSML dialog is used must define how it is used, the set of services provided, and the mechanism for passing information between the environment and the MSML dialog. The specific mechanisms used to realize the interface between an MSML dialog and its environment are platform specific. 🛠️
MSML Dialog Packages provide two models for access to media resources and service creation building blocks. 💜 Both models MAY be used in conjunction with each other in a complementary manner. The first model (referred to as "Media Primitives and Composites", part of the mandatory MSML Dialog Base Package) contains media primitives (such as digit collection and announcements) and composite functions (such as play and collect combined as a single operation). The second model (referred to as "Media Groups", part of the optional MSML Dialog Group Package) allows the ability to define complex customized interactions, via event passing mechanisms, between media primitives, if required. 🎀
MSML Dialog Core Package
Defines core framework over which all MSML Dialog Packages
operate.
MSML Dialog Base Package
Media Primitives
<dtmf> or <collect>
DTMF digit collection
<play>
Playing of Announcements
<dtmfgen>
Generation of DTMF digits
<tonegen>
Tone genration
<record>
Media recording
Media Composites
<collect>
Supports play and collect operation.
Composite function with inclusion of play.
<record>
Supports play and record operation.
Composite function with inclusion of play.
MSML Dialog Group Package
<group>
Allows grouping of media primitives for parallel
execution, with an event exchange mechanism
between the media primitives to achieve
customized media operations. All the above media
primitive elements are accepted within the
group.The following operations MUST be supported using the elements described above, using either the MSML Dialog Base Package or the MSML Dialog Group Package. 🍬
Announcement only
<play>
Collection only
<dtmf> or <collect>
Recording only
<record>
Play and Collect
<collect>
<play/>
</collect>
Play and Record
<record>
<play/>
</record>Additional MSML Dialog Packages are:
- MSML Dialog Transform Package 🔄
- MSML Dialog Speech Package 🗨️
- MSML Fax Detection Package 📠
- MSML Fax Send/Receive Package 📨
MSML dialogs MAY be used to simply expose primitive media resource objects, but they will be used more often to describe dialog operations and media transformation objects that can be controlled via user interaction. 🫶
MSML dialogs do not contain any computation or flow control constructs. 🙅♀️ There are no results automatically generated when media operations complete. Results MUST be explicitly requested using a <send> or <exit> element within the definition of the MSML dialog. 💀
9.2. Primitives 🎀
okay bestie, primitives are the little workers that each do ONE job on a media stream (or on a bunch of streams) ✨ like generating audio or video, recognizing speech or DTMF, or adjusting the gain 🔊. You can compose them so they run concurrently, which is giving multitasking energy 💅. Primitives that are not composed for concurrent execution MUST simply execute sequentially, in the order they show up in an MSML document. Anything running concurrently inside the same MSML object (the one defined in a single MSML document) MAY chat with each other through events (see MSML Dialog Group Package) 💬.
Primitives get sorted into these descriptive categories 🗂️:
- Recognizers have a media input but no output. They let you recognize or detect different things inside a media stream and generate events based on what they receive 👀.
- Transformers have one media input and one output, and they may send and receive events 🔁.
- Sources and sinks generate or consume media. They have either a media input or a media output but not both. They may receive and generate events too 🎤.
- Composites combine underlying primitives to give higher-level user interaction, without needing specific event-based exchanges between those primitives. They're the simpler option for more common services, like play and collect or play and record 🎁.
Primitives may define different media processing behavior (states) based on the events they receive 🎭. Primitives that support different processing states must define their default starting state and should support the "initial" attribute so that state can be specified when the primitive is instantiated 🌸. All primitives must support the "terminate" event class 🛑.
The following types of primitives are defined in this specification 📋:
Recognizers Transformers Source/Sink Composites
------------------------------------------------------
dtmf/collect agc play dtmf/collect
faxdetect clamp record record
speech gain dtmfgen
vad gate tonegen
relay faxsend
faxrcvPrimitives have shadow variables, similar to the ones in VoiceXML [n5], and they get values assigned automatically whenever a primitive is used ✨. When an MSML dialog context starts up, every shadow variable has the string value "undefined". Each primitive has its own set of shadow variables, and those are global in scope across the whole MSML dialog context 🌍.
Names SHOULD be assigned to individual primitives when more than one primitive of the same type is used within one MSML document 🏷️. If a primitive hasn't been named and gets instantiated a second time, its shadow variables get overwritten 😬.
Shadow variables cannot be modified under user control 🙅♀️. They may be returned from the MSML dialog context using the <send> element 📤.
9.3. Events 💌
Events are how primitives talk to each other, and how an MSML context talks to its outside world 🗣️. That outside world is defined by the way the MSML context was invoked. This will often be through MSML, but other languages and protocols such as SIP may also be used 📞.
Every primitive and group conceptually implements its own event queue. Events sent to them get placed into their queue and are removed and processed in order 📥. Primitives within a group conceptually have their own thread of execution 🧵. Since servicing events from multiple queues is asynchronous, you can't assume that several events sent in sequence to different queues will be processed in the order they were sent 🙃. Say recognition led to events going to both a <play> and a <record>, in that order. The <record> could totally process its event before the <play> 😳.
Each primitive defines the set of events it supports and how it behaves when handling each one 🎛️. That means tons of behaviors can be defined. For example, VCR-style controls can be built by making primitives that support an event for each control, and media recognition or detection can be what triggers those events 📼.
Events can also come from somewhere else, like a control agent, and just get received by the primitive that implements the control 🕹️. Examples include adjusting volume (gain) and pausing and resuming both announcement playout and record creation ⏯️.
Primitives act on events based on the longest match of an event name 🔍. Event names are a period '.' delimited sequence of tokens. The first token, the root of the name, can be thought of as an event class 🏛️. Matching lets a standard meaning be defined and then extended based on what triggers an event's generation. For example, a record primitive behaves differently depending on whether it completed because a user stopped speaking or because it was cancelled. The recording is kept in the first case but not the second 🎙️.
Longest match also means new recognizers can be created and used without changing how existing primitives are defined 🧠. For example, a face recognition capability could generate a terminate.frowning event when a user looks puzzled 🤨. No primitive directly defines that event, but it will still trigger a generic terminate action. Primitives that need specialized behavior for frowning can be extended to support it. The event can also be exported from the MSML context without requiring the primitives that receive it to understand facial expressions 🫶.
9.4. MSML Dialog Usage with SIP 📞
MSML dialogs MAY be used directly with SIP for dialog interactions (e.g., IVR or fax) 🎧. They can be invoked initially as part of the "Prompt and Collect" service described in "Basic Network Media Services with SIP" [n7]. That defines service indicators for a small number of well-defined services, using the user part of the SIP Request-URI (R-URI) 🔑.
The prompt and collect service uses "dialog" as its service indicator, and URI parameters further refine the specific IVR request 🎀. This document defines an additional parameter "msml-param" for the dialog service indicator, like this:
dialog-parameters = ";" ( dialog-param [ vxml-parameters ] )
| moml-param
dialog-param = "voicexml=" dialog-url
moml-param = "moml=" moml-urlThere are no additional URI parameters when MSML is used as the dialog language 🙌.
MSML dialogs define discrete IVR dialog commands 📦. These commands MAY be included directly in the body of the INVITE to the "dialog" service indicator by using the "cid" [n8] URL scheme. That scheme identifies a message body part, which in this case would contain the MSML dialog request 💌. Note that a multipart message body containing a single part MUST be present, even if the INVITE does not contain an SDP offer ⚠️. Subsequent MSML dialog requests are sent in the body of SIP INFO messages, same as all messages from a media server 📨.
An example of the SIP URI described above is:
sip:dialog@mediaserver.example.net;\
moml=cid:14864099865376@appserver.example.netThe body part that contained the MSML dialog referenced by the URL would have a Content-Id header of:
Content-Id: <14864099865376@appserver.example.net>The results of executing an <exit> or <disconnect>, or of executing a <send> whose "target" attribute value equals "source", are notified in SIP INFO messages using the <event> element from the MSML Core package 🔔. No messages are sent if execution completes normally without executing one of these elements 🤫.
If there's an error during validation or execution, a media server MUST notify the error as described above, and it must include the namelist items moml.error.status and moml.error.description 🚨. The values for these items are defined in section 11 📚.
A restricted subset of MSML dialogs can also be used with the "Announcement" service defined in [n7] 📢. That service uses "annc" as its service indicator and defines parameters that describe an announcement. The "play=" parameter identifies the URL of a prompt or a provisioned announcement sequence. Its value can refer to an MSML dialog body part using a "cid" URL as described above. That body part must only contain the <play> primitive 🎵.
Using MSML dialogs enhances the announcement service 💖. The client can specify a sequence of audio segments instead of needing each sequence to be provisioned, and video is supported too 🎬. Plus, MSML dialogs define a standard set of variables, whereas [n7] defines a parameterization mechanism but does not formally specify any semantics 🤓.
If a media server does not understand the "cid" scheme, or does not understand MSML dialogs, it must respond with the SIP response code 488 - not acceptable here 🙅♀️. If the MSML dialog body contains elements other than the <play> primitive, or there are errors during validation, a media server must respond with 400 - bad request 💀. Finally, if there's a discrepancy between parameters specified in the Request-URI and the corresponding attributes defined in the MSML dialog body, the Request-URI parameters must be silently ignored 🫥.
MSML dialogs MUST NOT change the operation of the announcement service from what's defined in [n7] 🙅♀️. When the announcement completes, a media server issues a SIP BYE request 👋. The INFO method MUST NOT used with the announcement service 🚫.
9.5. MSML Dialog Structure and Modularity 🧩
MSML is structured as a set of packages, and only the core and base packages are required 📦. The Dialog Core Package defines the framework for MSML requests to a media server, without any specific functionality 🦴. It consists of the "primitive" abstraction, an abstract element for control flow, the sequential execution model, and the <send> element 🔧. Basically, the MSML Dialog Core Package lets you execute a sequence of one or more media processing primitives, with the ability to notify events to the invocation environment 🔔.
Primitives are contained within the MSML Dialog Base Package, which defines the basic <play>, <record>, <dtmf>, <dtmfgen>, <tonegen>, and <collect> elements 🎁. Another package, the MSML Dialog Transform Package, defines the simple half-duplex filters 🎚️. More advanced primitives are defined in the speech and fax packages 🗣️. The MSML speech package depends on the MSML Dialog Base Package, since it extends <play> by adding synthesized speech 🤖. Finally, the group execution model, which is currently the only element that changes the flow of control, is defined in a separate MSML Dialog Group Package 🌈. All of these packages are optional, with one exception: the MSML Dialog Core and MSML Dialog Base Packages MUST be implemented to provide the minimal functionality 💜.
9.6. MSML Dialog Core Package ✨
okay so the MSML Dialog Core Package is giving structure 🎀 It sets up the framework and abstractions for MSML dialogs (through its schema), and it also defines the basic elements that aren't part of the core primitive or control abstractions. This package depends on the MSML Core Package, so it's not doing this alone 🫶 Events from dialogs, like prompt completion, digits collected, or dialog termination, get passed up by the media server through the MSML Core Package (see MSML Core Package <event>).
Dialogs run totally independently from the MSML core context 💅 When a dialog starts, MSML allocates the dialog control resources and, if that works, gets them running. The core keeps going without waiting for the dialog to finish. That fork from the core context happens through the <dialogstart> element. Media streams get created between the dialog target and other internal media server resources while the dialog runs. Stream creation has to follow the requirements in the MSML Core Package, plus the media stream rules in the MSML Conference Core Package. no cap, the rules are important 🔥
9.6.1. <dialogstart> 🧚
The <dialogstart> element starts an MSML media dialog on connections or conferences 🦋 The dialog can be inline or pulled in by a URI [n6]. Inline dialogs MUST be composed of any of the MSML Dialog Packages. MSML dialogs MAY be defined externally as VoiceXML [n5]. And if there's a src attribute holding a URI, the MSML dialog description MUST NOT be inline. that's a hard no 🙅♀️
When the dialog finishes, the originator gets notified with a "msml.dialog.exit" event. Any results the dialog returns on exit get sent along as a namelist in that event 💜
The"msml.dialog.exit"event also fires when a dialog fails. That can happen if there's an error fetching an external document, or an error inside the dialog's execution thread. In that case the event carries a namelist with the items"dialog.exit.status"and"dialog.exit.description", so the client knows what went wrong and why. The values for those items are defined in this package and in the MSML Core Package. Any info from the failed dialog can come back as extra namelist items too.
Attributes:
target: an identifier of a connection or a conference that will interact with the dialog. The identifier must not contain wildcards. Mandatory. 🎯src: the URL of the dialog description. MUST NOT be used if the MSML dialog description is inline. Otherwise, an error (422) will result and MSML document execution will stop. 🚫type: a MIME type that identifies the type of language used to describe the dialog.application/moml+xmlandapplication/vxml+xmlare used to identify MSML dialogs and VoiceXML [n5] respectively. Mandatory. 🏷️name: an instance name for the dialog. If the attribute isn't there, the media server assigns an identifier to the dialog. If the attribute is present but the name is already linked to the target, an error (431) will result and MSML document execution will stop. Any results the dialog generates get correlated to its identifier. 🫧mark: a token that helps identify execution progress when errors happen. Themarkvalue from the last successfully executed MSML element comes back in an error response. So allmarkattributes in an MSML document should be unique. 🔖
Here are some examples, because you know I love a good example 🌸 First, starting an external MSML dialog on a connection:
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<dialogstart target="conn:abcd1234"
type="application/moml+xml"
name="sample"
src="http://server.example.com/scripts/foo.moml"/>
</msml>Next, an inline embedded MSML dialog on a connection:
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<dialogstart target="conn:abcd1234" name="sample">
<play>
<audio uri="file://clip1.wav"/>
<audio uri="http://host1/clip2.wav"/>
<tts uri="http://host2/text.ssml"/>
<var type="date" subtype="mdy" value="20030601"/>
</play>
<send target="source"
event="done"
namelist="play.amt play.end"/>
</dialogstart>
</msml>And last, a VoiceXML dialog started on a connection:
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<dialogstart target="conn:abcd1234"
type="application/vxml+xml"
name="sample"
src="http://server.example.com/scripts/foo.vxml"/>
</msml>If that dialog fails after its execution thread has started (say the VoiceXML document fetch fails), the event coming back looks like this 😭
<?xml version="1.0" encoding="UTF-8"?>
<event name="msml.dialog.exit"
id="conn:abcd1234/dialog:sample">
<name>dialog.exit.status</name>
<value>423</value>
<name>dialog.exit.description</name>
<value>External document fetch error</value>
</event>9.6.2. <dialogend> 💀
<dialogend> is how you terminate an MSML dialog that was created with <dialogstart> before it finishes on its own 🙈 How it works depends on the dialog language the executing context is using. For VoiceXML, a "connection.disconnected" event gets thrown to the VoiceXML application. For MSML dialog, a "terminate" event gets sent to the MSML core context.
<dialogend> gives the running dialog a chance to wrap up gracefully before it generates a "msml.dialog.exit" event. Dialog results can still come back, packed in a namelist on that event ✨
Attributes:
id: the identifier of a dialog. Mandatory. 🪪mark: a token that helps identify execution progress when errors happen. Themarkvalue from the last successfully executed MSML dialog element comes back in an error response. So allmarkattributes in an MSML document should be unique. 🔖
For example, if the dialog from the previous example were still running, this would end it and generate an "msml.dialog.exit" event:
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<dialogend id="conn:abcd1234/dialog:sample"/>
</msml>9.6.3. <send> 📨
The <send> element sends an event, plus an optional namelist, to the recipient named in the target attribute. Event names are defined by the recipient. When the recipient is an MSML dialog group or primitive, the events are defined in this document. Other recipients MAY use names that suit their environment 🌈
The target attribute says who gets the event. Recipients MAY be other MSML dialog primitives or groups running inside the object, the object itself, or the environment that invoked the MSML dialog. Sending events to media primitives or groups is supported by the MSML Dialog Group Package. Any target that's unknown inside the object is assumed to belong to the external environment. By convention, the string "source" SHOULD used to address that environment, but any target name that's different from the MSML dialog namespace MAY be used 🦋
Attributes:
event: the name of an event. Mandatory. 📣target: the recipient of the event. The recipient MUST be an MSML dialog primitive, the currently executing group, or the MSML dialog environment. A primitive is specified by a primitive type, optionally followed by a period '.' and the identifier of a primitive. Identifiers are only needed when more than one primitive of the same type exists in the object. The executing group is specified with the token "group". The environment is specified with the token "source", optionally followed by a period '.' and any environment-specific target. Mandatory. 🎯namelist: a list of zero or more shadow variables that are included with the event. 🧾
9.6.4. <exit> 👋
The <exit> element makes the MSML dialog stop executing 🥀
Attributes:
namelist: a list of one or more shadow variables that MAY optionally be sent to the context that invoked the MSML Dialog object. 🎁
9.6.5. <disconnect> 🔌
The <disconnect> element is like <exit>, but it has an extra meaning: it tells the context that invoked the MSML dialog to disconnect from a media server, along with the media stream tied to the object. How the disconnect happens depends on how the media stream was first set up. If SIP was used, a <disconnect> makes a media server issue a BYE request. That request gets sent for the SIP dialog attached to the media session the MSML dialog was running on 📞
Attributes:
namelist: a list of one or more shadow variables that MAY optionally be sent to the context that invoked the MSML dialog object. 🎀
9.7. MSML Dialog Base Package 🌷
The MSML Dialog Base Package sets up a required set of base functionality for the media server 💖 It covers individual media primitives, like playing an announcement or collecting digits, plus composite operations like play and collect. When this package is used together with the MSML Dialog Group Package, the event-based mechanism is what controls the primitives. This package can also be combined with the MSML Speech Package to level up prompts with TTS and level up input collection with ASR 🤖✨
In the sections that follow, the subsections of a primitive define child elements of that primitive. They aren't primitives themselves 🧚 They don't receive events or populate shadow variables, so don't go looking for them there 🙈
9.7.1. <play> 🎀
okay so <play> is the element that makes the audio or video stream, bestie 🎶 It MUST play in sequence the media made by its child elements <audio>, <video>, <media>, <tts>, and <var>. When it stops, either because a terminate event shows up or because all the media generation is done, the <playexit> element runs (if you've got one). At least one media generation element must be present 💅
<play> has two states: generate and suspend. Media gets generated in generate and paused in suspend. Once it's in suspend, generation picks back up when the generate event arrives. The default starting state is generate ✨
Audio MAY be generated in different languages by setting xml:lang on <play> and/or on its child elements. The children inherit the language, but each child MAY set its own. Except for physical audio clips, it's an error if a language is asked for and the media server can't render the audio in that language 🙈
Attributes:
id: an optional identifier that can be referenced elsewhere for sending events to the play primitive 🆔interval: the delay between stopping one iteration and starting the next. It does nothing unlessiterateis also specified. Default is no interval.iterate: how many times the media from the child media elements gets played. Each iteration is a full play of every child media element, in document order. Defaults to once'1'.initial: the initial state for the play element. Default is"generate".maxtime: the maximum allowed time for the <play> to complete ⏰barge: whether audio announcements can be interrupted by DTMF detected during play-out. The DTMF digit that barges the announcement is stored in the digit buffer. Valid values are"true"or"false", and it's mandatory. When barge is applied to a conference target, DTMF digits detected from any conference participant MUST terminate the announcement 📞cleardb: whether the digit buffer gets cleared before the announcement starts. Valid values are"true"or"false", and it's mandatory 🧹offset: an offset, measured in units of time, where the <play> starts generating media. Offset is only valid when all child media elements are <audio>.skip: an amount of time used to skip through the media when "forward" and "backward" events arrive. Default is 3 s (three seconds) ⏩xml:lang: the language to use for content that can be rendered in different languages 🌍
Events:
The following describes input events to the media primitive object. The MSML Dialog Group Package allows an event exchange mechanism between primitives 🫶
pause: makes the play enter the suspend state ⏸️resume: makes the play enter the generate state ▶️forward: skips forward through the media. It only has an effect when all child media elements are <audio>.backward: skips backward through the media. It only has an effect when all child media elements are <audio>.restart: skips to the beginning of the media. It only has an effect when all child media elements are <audio>.toggle-state: flips between the suspend and generate states 🔁terminate: ends the play and assigns values to the shadow variables 🛑
Shadow Variables:
play.amt: the length of time the media was generated before the play was stopped. This doesn't include any time that passed while the play was in the suspend state.play.end: the event that caused the play to stop. When the play stops because all media generation has completed, end gets assigned the value"play.complete"✅
Note: thebargeandcleardbattributes are a simplified way to control play operations with implicit DTMF, no <group> or event exchange needed 🧚 When you use <play> inside the group framework andbargeis specified, detecting a barge condition generates an implicit terminate event to the play primitive.
Here's what the child elements of <play> look like 👑
9.7.1.1. <audio> 🎧
The <audio> element identifies prerecorded audio to play. Local URI references may resolve to a single physical audio clip, a logical clip, or a provisioned sequence of clips (physical or logical). A logical clip is one that can be rendered differently based on the language attribute. Logical clips are provisioned for each of the languages a media server supports. Remote URI references get resolved according to the capabilities of the remote server 🌐
Attributes:
uri: the location of the audio to be played. The file and http schemes are supported. Mandatory.format: the encoding and file type of the audio resource. The format attribute is defined as a string of the form"audio/<filetype>;codecs=<codec>". The keyword'audio'identifies audio content. The codecs field identifies the codec used to decode the audio file's content. If the format attribute isn't specified, the filetype MUST be determined from the URI, and the codec information MUST be determined from the media resource 🔍audiosamplerate: the audio sample rate in kHz. If it's not specified, the sample rate SHOULD be determined from the media resource.audiosamplesize: the audio sample size in bits. If it's not specified, the sample size SHOULD be determined from the media resource.iterate: how many times the audio is to be played. Defaults to once'1'.xml:lang: the language to use when the URI identifies a logical clip, either directly or as part of a sequence 🗣️
9.7.1.2. <video> 🎬
The <video> element identifies prerecorded multimedia to play. Contents identified by the URI attribute may contain audio only, video only, or both audio and video. The media server SHOULD attempt to play both audio and video from the identified URI, if both are available in the content 🍿
Attributes:
uri: the location of the video or multimedia to be played. The file and http schemes are supported. Mandatory.format: the encoding and file type of the video or multimedia resource. The format attribute is defined as a string of the form"video/<filetype>;codecs=<codecx>,<codecy>". The keyword'video'identifies video-only media or media containing both audio and video. Thecodecsfield identifies the audio and/or video codecs used to decode the file content, and the order of the codec values doesn't matter. When you've got audio and video content and use the'video'keyword, thecodecs=<codecx>,<codecy>field MAY be used to identify the audio codec and the video codec. If it's not specified, the codec information SHOULD be determined from the media file 🎞️audiosamplerate: the audio sample rate in kHz. If it's not specified, the sample rate SHOULD be determined from the media file.audiosamplesize: the audio sample size in bits. If it's not specified, the sample size SHOULD be determined from the media file.codecconfig: an optional special instruction string for codec configuration. The default is to send no special configuration string to the codec.profile: a video profile name specific to the codec. If it's not specified, the default video profile of the codec SHOULD be selected.level: a video profile level for the codec. The default is to send no profile information to the codec and let the codec pick an internal default 📊imagewidth: the width of the video image in pixels. The default is to use the image width information from the media file.imageheight: the height of the video image in pixels. The default is to use the image height information from the media file.maxbitrate: the bitrate of the video signal in kbps. The default is to use the maximum bitrate information from the media file.framerate: the video frame rate in frames per second. The default is to use the frame rate information from the media file.iterate: how many times the media content is to be played. Defaults to once'1'.
9.7.1.3. <media> 🎥
The <media> element identifies multimedia content for play. All content of the <media> element MUST start to play concurrently. This element may be used to generate a multimedia stream from two independent media resources, one identifying audio and the other identifying video 🫧
The <media> element MUST contain at least one child element. The valid child elements of <media> are <audio> and <video>, as described earlier. The <media> element MUST contain at most one <audio> element or at most one <video> element 🙅♀️
9.7.1.4. <var> 🧠
The <var> element specifies the generation of audio from a variable, using prerecorded audio segments. A variable represents a semantic concept (like a date or a number) and dynamically produces the right speech for it 🗓️
Prerecorded audio lets an application vendor or service provider pick the exact voice for their audio, so they completely control the "sound and feel" of the service their end users get. It sounds super high quality and lets the variables blend seamlessly into the surrounding audio segments 💖
Text to speech (TTS) using Speech Synthesis Markup Language (SSML) [n11] can also be used to render variables, but it may not sound as good, or give as complete control over the "sound and feel" or user experience. TTS is normally used for reading text like emails, and for very large vocabularies such as stock names. TTS makes a very clear difference between the variables and the surrounding audio segments. (See MSML Dialog Speech Package.) 🤖
Attributes:
type: the type of variable. Mandatory. The variable type must be one of"date","digits","duration","month","money","number","silence","time", or"weekday".subtype: an optional clarification of the type. The specific values depend on the type 🔎value: the text that should be rendered in a way that fits thetypeandsubtypeattributes. Mandatory.xml:lang: the language to use when rendering the variable 🌏
9.7.1.5. <playexit> 🚪
The <playexit> element MUST be invoked when generation of all content of the <play> has come to completion. The contents of this element MAY be used to send events 📨
Attributes:
- none 🫥
9.7.2. <dtmfgen> 🎀
okay so the DTMF generator is the one that originates one or more DTMF digits in sequence 📞✨ it's giving phone keypad, bestie 🫶
Attributes:
id: an optional identifier that may be referenced elsewhere for sending events to the dtmfgen primitive 💖digits: a string of characters from the alphabet0-9a-d#*that correspond to a sequence of DTMF tones. Mandatory. 🔥level: used to define the power level for which the tones will be generated. Expressed in dBm0 in a range of 0 to -96 dBm0. Larger negative values express lower power levels. Note that values lower than -55 dBm0 will be rejected by most receivers (TR-TSY-000181, ITU-T Q.24A). Default is -6 dBm0. 🌸dur: the duration in milliseconds for which each tone should be generated. Implementations may round the value if they only support discrete durations. Default is 100 ms. ⭐interval: the duration in milliseconds of a silence interval following each generated tone. Implementations may round the value if they only support discrete durations. Default is 100 ms. 🦋
Events:
terminate: terminates DTMF generation and assigns values to the shadow variables. 💅
Shadow Variables:
dtmfgen.end: contains the event that caused DTMF generation to stop. 👑
The following sections describe the child elements of <dtmfgen>. 🧚
9.7.2.1. <dtmfgenexit> 🌈
ngl this one is so important 😭 the <dtmfgenexit> element MUST be invoked when the DTMF generation operation completes or is terminated as a result of receiving the terminate event. it MAY be used to send events when the DTMF generation has completed. no cap, that's the whole vibe 💜
Attributes:
- none
9.7.3. <tonegen> 🍓
the tone generator is where it gets customized, periodt ✨ you can compose a sequence of varying tones with optional silence intervals using the <tonegen> element. its child elements, namely <tone> and <silence>, specify a single tone or a sequence of tones 🎶
Attributes:
id: an optional identifier that may be referenced elsewhere for sending events to the tonegen primitive 🦋iterate: a numeric value specifying the total number of iterations. A value of'forever'represents infinite repetitions. Optional. Default is 1. 💖
Events:
terminate: terminates tone generation and assigns values to the shadow variables. 🔥
Shadow Variables:
tonegen.end: contains the event that caused tone generation to stop. 🌸
The following sections describe the child elements of <tonegen>. 🫶
9.7.3.1. <tone> 💅
the <tone> element specifies a single tone with an optional silence interval, and honestly the main character energy is real 👑 the tone specification consists of two tone frequencies, their attenuation values, a duration of the tone, and the number of times to repeat the tone.
Attributes:
duration: time duration or length of the individual tone, specified in"ms"or"s"in increments of 10 ms. A value of 0 represents an infinite duration. Mandatory. ⭐iterate: specifies the number of times to execute the contents of<tone>element. A value of'forever'represents infinite repetitions. Optional. Default is 1. 🌈
Events:
- none
Child Elements:
The child elements of <tone> element specify a single tone and an optional silence interval to be inserted at the end of tone generation. A tone is defined by <tone1> and <tone2> elements. Each <tone> element MUST contain at least one of <tone1> or <tone2>, or MAY contain <tone1> and <tone2> exactly once. 🧚
<tone1>
Attributes:
freq: specifies the frequency of the first tone in"Hz", ranging from 0 to 3999 Hz. Mandatory. 🎀atten: specifies the attenuation level expressed in dBm0, ranging from 0 to -96 dBm0. Mandatory. 💜
<tone2>
Attributes:
freq: specifies the frequency of the second tone in"Hz", ranging from 0 to 3999 Hz. Mandatory. 🍓atten: specifies the attenuation level expressed in dBm0, ranging from 0 to -96 dBm0. Mandatory. ✨
<silence>- Refer to the silence element definition below. 🦋
9.7.3.2. <silence> 🙈
the <silence> element inserts a silence interval as optional content of <tonegen> or <tone> elements. it's the pause that makes the whole thing slay 🤫
Attributes:
duration: specifies the amount of silence interval in"ms"or"s", in increments of 10ms. Mandatory. 🌸
Events:
- none
9.7.3.3. <tonegenexit> 🌈
same energy as the dtmfgen exit, bestie 💖 the <tonegenexit> element MUST be invoked when the tone generation operation completes or is terminated as a result of receiving the terminate event. the <tonegenexit> element MAY be used to send events when the tone generation has completed. iykyk 🫶
Attributes:
- none
9.7.4. <record> 🎀
okay bestie, <record> is the element that literally makes recordings 🎙️ 💖 Like <play>, it has two states, create and suspend ✨ In create, received media becomes part of the recording. In suspend, that media gets discarded 🫶
Recording MUST be terminated when a terminate event is received, or when a nospeech event is received and no audio has been recorded yet 💀 <record> also tells the different terminate events apart 👑
An optional <play> element MAY be specified as a child element of <record> 🧚 This makes a full play-record combo, where the prompts inside <play> get played before the recording starts 🎀
Note: prespeech, postspeech, and termkey give you a simpler way to control record operations using implicit DTMF and VAD, no <group> or event exchange mechanism needed 🙈
Attributes:
id: an optional identifier that can be referenced elsewhere for sending events to the record primitive 🦋append: a boolean that decides whether the recording is allowed to be appended to an existing file if dest already exists. Default is"false". This attribute is ignored if the scheme is http 🌸dest: the destination for the recording. It will contain audio only, video only, or both audio and video, depending on the stream(s) being recorded. Recording MAY be either local or external based on the attribute value. File and http schemes are supported 🍓audiodest: the destination for the audio-only recording. Recording MAY be either local or external based on the attribute value. All combinations of dest, audiodest, and videodest are valid. File and http schemes are supported 🎧videodest: the destination for the video-only recording. Recording MAY be either local or external based on the attribute value. All combinations of dest, audiodest, and videodest are valid. File and http schemes are supported 📹format: defines the encoding and file type of the recording. It's a string shaped like"audio|video/filetype;codecs=x,y". The keywordaudiomeans an audio-only recording, whilevideomeans a video-only recording or an audio plus video recording. The codecs field names the audio and/or video codecs to use, and the order of the codec values doesn't matter. When you record audio and video using thevideokeyword, the codecs=x,y field MAY be used to identify the audio codec and the video codec. Mandatory 💅codecconfig: an optional special instruction string for codec configuration. Default is to send no special configuration string to the codec 🧩audiosamplerate: the audio sample rate in kHz. If it's not specified, the sample rate SHOULD be determined from the media source 🔊audiosamplesize: the audio sample size in bits. If it's not specified, the sample size SHOULD be determined from the media source 🎚️profile: a video profile name specific to the codec. If it's not specified, the default video profile of the codec SHOULD be selected for the recording 🎬level: a video profile level passed to the codec. Default is to send no profile information to the codec and let the codec pick an internal default 🌈imagewidth: the width of the video image in pixels. Default is to use the image width information from the media source 📐imageheight: the height of the video image in pixels. Default is to use the image height information from the media source 📏maxbitrate: the bitrate of the video signal in kbps. Default is to use the maximum bitrate information from the media source ⭐framerate: the video frame rate in frames per second. Default is to use the frame rate information from the media source 🎞️initial: defines the initial state for the record element. Default is"create", which starts the recording as soon as the<record>element is executed. Theinitialattribute only applies when<record>is used within the<group>structure 🚀maxtime: defines the maximum length of the recording in units of time. Mandatory ⏳prespeech: a timer value, in seconds, for detecting absence of audio energy at the start of the record operation. If no audio energy is detected for the time set by prespeech, the recording gets terminated. Default is 0 s, which doesn't activate the prespeech timer 🤫postspeech: a timer value, in seconds, for detecting absence of audio energy while the recording is in progress. If absence of audio energy is detected for the amount set by the postspeech timer, the recording gets terminated. Default is 0 s, which disables terminating a recording due to postspeech silence 😶termkey: a single DTMF key that, when detected, terminates the recording. Without this attribute, the recording can't be terminated by DTMF digits. When termkey is specified, the detected DTMF digit terminates the recording and is not entered in the digit buffer 🔑
Events:
The following describes input events to the media primitive object. The MSML Dialog Group Package allows an event exchange mechanism between primitives 💌
pause: makes the record enter the suspend state. Received media is discarded ⏸️resume: makes the record resume if it was suspended. Otherwise it does nothing ▶️toggle-state: toggles the suspend / create state 🔁terminate: terminates the recording and assigns values to the shadow variables 🛑terminate.cancelled: terminates the recording and assigns values to the shadow variables. If the dest attribute used the file scheme, the local recording is deleted. Applications are responsible for removing external files created using the http scheme 🗑️terminate.finalsilence: terminates the recording and assigns values to the shadow variables. If the dest attribute used the file scheme, the final silence is removed from the recording ✂️nospeech: terminates the recording and assigns values to the shadow variables if it's received and no recording has been created yet. Thenospeechevent is ignored if audio has already been recorded 🙉
Shadow Variables:
record.len: the actual length of the recording, measured in units of time. This doesn't include time that may have passed while the record was in the suspend state ⌛record.end: contains the event that caused the record to terminate. When the record terminates because maxtime is exceeded, end is assigned the value"record.complete.maxlength"🏁record.recordid: contains the value of thedestattribute, if supplied. Otherwise it contains a media server assigned record identifier 🪪
Record termination due to prespeech silence gets the value "record.failed.prespeech". Record termination due to postspeech silence gets the value "record.complete.postspeech". Record termination due to DTMF detection gets the value "record.complete.termkey" 💅
The following sections describe the child elements of <record> 🧚
9.7.4.1. <play>
The optional <play> element, as a child element of <record>, lets a prompt play before recording starts. The record operation starts at the end of the play sequence, or if the play is barged by DTMF, assuming barge=true is specified for <play> 🎵 For the full scoop, check out the <play> element 💖
9.7.4.2. <tonegen>
The optional <tonegen> element, as a child element of <record>, lets a tone or sequence of tones play before recording starts. The record operation starts at the end of the tone generation 🔔 For the complete description, refer to the <tonegen> element ✨
9.7.4.3. <recordexit>
The <recordexit> element MUST be invoked when the record operation completes, or when the recording is terminated because it received the terminate event 🦋 The <recordexit> element MAY be used to send events when the recording has completed 📨
Attributes:
- none 🫶
9.7.5. <dtmf> or <collect> ✨
Okay bestie, DTMF input has a few jobs in MSML dialogs 🎀 First, it can trigger events that change how other primitives process media 🦋 Second, it collects DTMF digits from a media stream and reports them back to the user of the MSML dialog 📲 And ngl, a lot of the time DTMF detection does both at once 💅 Barge is the classic example: a prompt gets stopped because of DTMF input, but there might still be more digits to collect 🙈
DTMF detection can handle multiple recognition patterns at the same time 🌈 Each pattern can send a different event, which is how you build DTMF controls 🎮 Or, one pattern can represent a whole collection while another pattern, a substring of the first, acts as the barge indication 💜
An optional <play> element MAY be specified as a child element of <dtmf> or <collect> 🎶 That gives you a complete play-collect operation, where the prompts inside <play> get played before DTMF digit collection 🧚
Heads up 🚨: all patterns share the same digit collection buffer, inter-digit timing, a single <nomatch> element, and a single <noinput> element. So multiple patterns might not be the move for simultaneous collections that serve different purposes 🙅 When that's what you need, separate <dtmf> elements should be used instead 🫶
<dtmf> terminates if any of the <pattern>, <noinput>, or <nomatch> elements are matched the maximum number of times they're allowed 💀 The number of times they may match can be set as an attribute of <dtmf> or of the individual child elements 🔥
Element identifier <dtmf> is equivalent to <collect> ✨ However, <collect> is the preferred name 💅 MSML clients SHOULD use <collect>, while MSML servers SHOULD support both 🫶
Attributes:
id: an optional identifier that may be referenced elsewhere for sending events to this primitive 🎀cleardb: a boolean that says whether the buffer for digit collection gets cleared of any collected digits when the element is instantiated. If it's set to false, any digits currently in the buffer MUST be immediately compared against the pattern elements 🧹fdt: defines the first-digit timer value. The first-digit timer starts when DTMF detection is first invoked. If no DTMF digits are detected during this initial interval, the<noinput>element MUST be invoked. Optional, default is 0 s (wait forever for the first digit) ⏳idt: defines the inter-digit timer used when digits are being collected. When specified, the timer starts when the first digit is detected and restarts on each subsequent digit. Timer expiration applies to all patterns. After that, if any patterns remain active and a nomatch element is specified, the nomatch is executed and DTMF input MUST terminate. The idt attribute should only be used when digit collection is being performed. Optional, default is 4 s 🕰️edt: defines the extra-digit timer value. It specifies how long the media server MUST wait after a match to detect a termination key, if one is specified by the<pattern>element. Optional, default is 4 s 🔑starttimer: a boolean that defines whether the first digit timer (fdt) starts initially. When set to false, the starttimer event must be received for it to start. Default is"false"🚦iterate: specifies the number of times the<pattern>,<noinput>, and<nomatch>elements may be executed, unless those elements specify differently. The value"forever"MAY be used to indicate that these may be executed any number of times. Default is once1🔁ldd: defines the minimum duration a digit must be held for it to be detected as a long DTMF digit. A long DTMF digit event MUST be treated as a single DTMF event, and MUST contain an extra characterLat the end to be told apart from the other regular digit events. For example,#Land#are different DTMF events 🧐 Optional, default of 0 s. A value of 0 s disables long DTMF digit detection and reporting. The attribute value is an integer with a valid range from 100 ms to 100 s (units MUST be supplied) 📏
Events:
The following describes input events to the media primitive object. The MSML Dialog Group Package allows an event exchange mechanism between primitives 🤝
starttimer: starts the first digit timer (fdt) if it hasn't already been started. Has no effect otherwise 🏁terminate: terminates the DTMF input and assigns values to the shadow variables 🛑
Shadow Variables:
dtmf.digits: the string of DTMF digits that have been received (the contents of the digit buffer) 📝dtmf.len: the number of digits in the digit buffer 🔢dtmf.last: the last digit in the digit buffer ☝️dtmf.end: contains the event that caused the<dtmf>to terminate, or is assigned one of"dtmf.match","dtmf.noinput", or"dtmf.nomatch", depending on which of the corresponding elements reached its maximum 🏆
The following sections describe the child elements of <dtmf> or <collect> 🧸
9.7.5.1. <play>
The optional <play> element, as a child element of <dtmf> or <collect>, lets a prompt play before DTMF digit collection 🎵 DTMF digit collection starts at the end of the play sequence, or if the play gets barged by DTMF, assuming barge=true is specified for <play>. For the full scoop, check out the <play> element 📖
9.7.5.2. <pattern>
The <pattern> element describes one or more DTMF digits that are to be recognized 🎯 When the pattern is matched, the child elements MUST be executed 🙌
Attributes:
digits: the digit pattern that should be matched. Mandatory 🔢format: an enumerated value that defines the format used to express the digit pattern. The format may be"mgcp"or"megaco"for patterns expressed as a digit map from those specifications, or as one of the simple built-in formats defined within this specification. Currently, a single built-in format,"moml+digits", is defined. It allows a match based on either one or more specific digits, or on a specific length specification with an optional return key."moml+digits"is the default 🧩iterate: specifies the number of times the<pattern>may be matched. The value"forever"may be used to indicate that<pattern>may be matched any number of times. This value overrides any specified in<dtmf>. Default is once1🔁
9.7.5.3. <detect>
The contents of the <detect> element MUST be executed whenever any DTMF is first detected 👀 It MUST be matched at most once 🎈
Attributes:
- none 🫧
9.7.5.4. <noinput>
The <noinput> element is used when DTMF is being collected 🎤 Children of the <noinput> element MUST be executed when DTMF has not been detected and the first digit timeout occurs ⏰
Attributes:
iterate: specifies the number of times the<noinput>may be triggered. The value"forever"may be used to indicate that<noinput>may be triggered any number of times. This value overrides any specified in<dtmf>. Default is once1🔁
9.7.5.5. <nomatch>
The <nomatch> element is used when DTMF is being collected 🎤 Children of the <nomatch> element MUST be executed when it is determined that none of the individual patterns can be matched 🚫
Attributes:
iterate: specifies the number of times the<nomatch>may be triggered. The value"forever"may be used to indicate that<nomatch>may be triggered any number of times. This value overrides any specified in<dtmf>. Default is once1🔁
9.7.5.6. <dtmfexit>
The <dtmfexit> element MUST be invoked when the dtmf input completes because one of <pattern>, <noinput>, or <nomatch> has occurred its maximum number of times 🚪
Attributes:
- None 🫧
9.7.6. <moml>
The root element <moml> MUST be used when the document is a stand-alone MSML dialog, where the invoking application media type indicates 'application/moml+xml' 📄 Additionally, for backwards compatibility, the <moml> element MUST be used within <dialogstart>, which contains an inline embedded MSML dialog 🧁
Valid contents of <moml> are all elements described within this MSML Dialog Base Package 🧺
Attributes:
version:"1.0"Mandatory 🏷️id: an identifier unique to this object. Events returned from the MSML dialog (the"target"attribute of a<send>is equal to"source") will be correlated with this identifier. Mandatory 🪪
Events:
terminate: terminates the MOML context. A terminate event gets sent to the currently executing<group>or primitive 💥
9.8. MSML Dialog Group Package ✨
okay so the group package is the thing that lets you run a bunch of primitives at the same time 🎀 You wrap them in a <group> element and suddenly they're all doing their thing concurrently. Groups define how media flows between those primitives, and each group has one or more inputs and one or more outputs. A <group> is basically a declaration of a fancy media processing operation 💅 The event interaction between primitives (see the following subsection) lives inside one or more groups. BUT groups don't scope events. They just say "these are running at the same time," and a primitive has to be executing in order to receive an event 🦋
Using groups is totally optional. It's just what you reach for when you want complex services built out of events flowing between primitives. For simpler stuff like play/collect or play/record, you don't need the group mechanism at all 🌸 The MSML Dialog Group Package depends on the MSML Dialog Base Package, so that's a must-have bestie 💜
Groups can also describe media objects that transform a stream while letting the application or the user control the transformation. Like, a gain control that reacts to user speech or DTMF input? Iconic 👑 In that case, a recognition primitive sends events to a gain control primitive.
Groups have one attribute that defines the media flow inside them. They also have a dimension, which is how many media inputs and outputs they have. Right now, dimensions of 1 and 2 are supported based on the group topology. Dimension 1 is a group with one input and one output, and dimension 2 is a group with two inputs and two outputs 🍓
Media flow to and from the primitives in a group is based on the topology attribute of the <group> element. That attribute names a topology schema, and the schema implies the group's dimension ✨
There are a few common ways primitives get wired together, and a schema is basically a convenient template you can slap onto multiple primitives without spelling out every single media relationship yourself 🧚 These two schemas are initially defined for one-dimensional groups:
- parallel: media sent to the group goes to every primitive that has an input. The group bridges the output from every primitive that has an output into a single common group output.
- serial: the first primitive listed in the group receives the media sent to the group. Its output gets connected to the input of the next primitive defined within the group, and so on, until the last primitive in the group becomes the group output.
Here are the two topologies in diagrams. The one on the left is parallel and the one on the right is serial 🌈
/-> P1 --\
/ \
G(in) +---> P2 ----> G(out) G(in) --> P1 --> P2 --> P3 --> G(out)
\ /
\-> P3 --/More complex media flows MAY be created by nesting serial and parallel groups inside each other. For example, the diagram below shows a serial group nested inside a star topology. Bestie, this is where it gets spicy 🔥
/-----> P1 ------------------------\
/ \
Gs(in) +-> Gp(in) --> P2 --> P3 --> Gp(out) -+> Gs(out)Why would you do this? Picture a record operation where DTMF gets clamped out of the recording itself, but a DTMF key press still stops the recording. In that setup, P1 is a DTMF recognizer, P2 is a clamp primitive, and P3 is a recorder. The example below shows it. It leaves out child elements and attributes that aren't related to the core concept. The next section covers sending events, and the details of each primitive are in section 4 📼
<group topology="parallel">
<dtmf/>
<group topology="serial">
<clamp/>
<record/>
</group>
</group>There's also one schema, "fullduplex", for a two-dimensional group. A full-duplex two-dimensional group has exactly two immediate children. Those children can be primitives or other one-dimensional groups. A "fullduplex" group must only be used as the top-most group and must not be nested. Each primitive (P1) and group (G2) becomes one half of the full-duplex group, as shown in the diagram below 💖
G-A(in1) +-> G2 --> G-B(out1)
G-A(out2) <-- P1 <-+ G-B(in2)Full-duplex groups are symmetrical when both halves are the same, and asymmetrical when they differ. Asymmetric groups need a name for each side. The left side is the input of the first child of the full-duplex group combined with the output of the second child. The right side is the reverse. In the diagram above, these sides are labeled A and B respectively 🦋
The user-operated gain control from earlier in this subsection is a perfect full-duplex example. The gain should operate on the audio the user hears, but the gain is controlled by recognizing things like DTMF or spoken commands in the media the user originates. The XML below does exactly that and matches the media flow in the diagram above. If the user's audio isn't needed for anything except controlling the gain, then the <relay> isn't required and the internal group could be left out. A complete XML description is in the examples section 🎀
<group topology="fullduplex">
<group topology="parallel">
<dtmf/>
<relay/>
</group>
<gain/>
</group>Primitives within a group MUST begin concurrently but MAY finish asynchronously, based on the events they receive or when their task completes 💅 A group MUST terminate when all of the primitives within it have completed. If the group contains a <groupexit> element, then the contents of that element MUST be executed as part of group termination 🫶
A group itself MAY receive a terminate event requesting termination. A terminate event sent to the group causes a terminate event to be sent to each of its currently active primitives. The <groupexit> element is NOT executed until every primitive has processed its terminate event 😭
9.8.1. <group>
The <group> element lets the contained primitives execute concurrently 🌸
Attributes:
topology: specifies a schema that defines the flow of media within the group. Three schemas are initially defined."fullduplex"is for use with two-dimensional groups."parallel"and"serial"are for use with one-dimensional groups. The definitions of these topologies are in section 9.8. Mandatory.id: identifies the name of the group. Mandatory when groups are nested.
Events:
terminate: causes a terminate event to be sent to each element contained within the group.
9.8.2. <groupexit>
The <groupexit> element allows events to be sent when group processing completes. Group processing completes when all contained primitives terminate 🧚
Attributes:
- none
Events:
- none
9.9. MSML Dialog Transform Package 🦋
The MSML Dialog Transform Package gathers together the simple primitives that work as filters on half-duplex media streams 💜
9.9.1. <vad>
Voice activity detection (VAD) is used to detect voice and silence when speech recognition isn't required. Like speech and DTMF, a VAD has different media conditions it can match. Those conditions can be qualified by a minimum length of time they must last before they count as recognized ✨
Attributes:
id: an optional identifier that may be referenced elsewhere for sending events to the vad primitive.starttimer: a boolean value that defines whether the timer is started to allow recognition of the initial condition (voice, silence). When set tofalse, the starttimer event must be received in order for the initial condition to be recognized. The timer does not affect recognition of the transition conditions. Default is"false".
Events:
starttimer: starts the timer to allow recognition of the initial condition if it has not already been started. Has no effect otherwise.terminate: terminates voice activity detection.
Shadow Variables:
- none
The following sections describe the child elements of <vad> 🌈
9.9.1.1. <voice>, <silence>, <tvoice>, <tsilence>
Each child element corresponds to a condition that a VAD can detect. The first two detect when voice or silence has been initially present for a minimum length of time since the VAD was started. The last two require that a transition to the voice or silence condition occurs first 🍓
Attributes:
len: the length of time the condition must persist in order to be recognized. Mandatory. For<tvoice>and<tsilence>, the length of time applies only to the final recognized condition.sen: the maximum length of time the condition not being detected may occur without causing the detector to begin measuring that condition.
9.9.2. <gain>
Gain MAY be used to adjust the gain of a media stream by a specific amount 🔥 Applying <gain> removes any previous connection AGC setting used by the <agc> element.
Attributes:
id: an optional identifier that may be referenced elsewhere for sending events to the gain primitive.incr: an increment, expressed in dB, used to adjust the gain when "louder" and "softer" events are received. Default is 3 dB.amt: a specific gain to apply, specified in dB. Mandatory.
Events:
mute: self-explanatory 🙈unmute: self-explanatory 🙈reset: sets the gain to zero dB.louder: makes the audio on a stream louder.softer: makes the audio on a stream quieter.amt: sets the gain to the specified value between -96 dB and 96 dB.
9.9.3. <agc>
Automatic gain control MAY be used to have a media server automatically adjust the gain of a media stream 🤖 Applying <agc> removes any previous connection gain setting used by the <gain> element.
Attributes:
id: an optional identifier that may be referenced elsewhere for sending events to the gain primitive.tgtlvl: the desired target level for AGC, specified in dBm0 with a valid range of -40 to 0. Mandatory.maxgain: an optional attribute used to specify the maximum gain that AGC will apply, specified in dBm0 with a valid range of 0 to 40, with a default of 10.
Events:
mute: self-explanatory 🙈unmute: self-explanatory 🙈
9.9.4. <gate>
The <gate> element is a simple filter that passes or halts media, regardless of the media stream's format, based on the events it receives. <gate> shares the same mute and unmute events as the gain primitives <gain> and <agc>, for compatibility 💅
Attributes:
id: an optional identifier that may be referenced elsewhere for sending events to the gate primitive.initial: the values"pass"and"halt"define whether media is initially allowed to pass. Default is to pass.
Events:
mute: halts media flow through the primitive.unmute: allows media to pass through the primitive.
9.9.5. <clamp> 🫧
okay so this one is giving DTMF filter energy 🎀. This element MAY be used to filter DTMF tones out of a media stream. Anything that isn't a DTMF tone just passes straight through unchanged ✨.
id: an optional identifier that can be referenced elsewhere for sending events to the clamp primitive.
Events: none.
9.9.6. <relay> 🦋
this one is so simple it's iconic 💅. The <relay> is a basic primitive that copies whatever comes in to the output. no drama, no notes 🫶.
id: an optional identifier that can be referenced elsewhere for sending events to the relay primitive.
Events: none.
9.10. MSML Dialog Speech Package 🎙️
the MSML speech package is where the voice stuff lives 🗣️✨. It defines functionality that MAY be used for automatic speech recognition, <speech>, and it extends the <play> primitive from the MSML Dialog Base Package so it can do speech synthesis too 🌈. Because of that, this package depends on the MSML Dialog Base Package, so bestie you need that one first 💜.
9.10.1. <speech> 👂
the <speech> element switches on grammars or user input rules tied to speech recognition 🔥. If you specify more than one grammar, they all get switched on. All the active grammars share the same timers, recognition attributes, and <noinput> and <nomatch> elements, which is so iconic and efficient 🙌. Each grammar can have its own <match> element too.
<speech> ends if any of the <grammar>, <noinput>, or <nomatch> elements hit the max number of matches they're allowed. The number of allowed matches can be set as an attribute on <speech> itself or on the individual child elements 🎀.
Attributes:
id: an optional identifier that can be referenced elsewhere for sending events to the speech primitive.noint: sets a time window in which speech input has to start. If it doesn't, the associated<noinput>element gets invoked 😭.norect: sets a maximum time window in which speech has to begin to be matched. If that doesn't happen, the associated<nomatch>element gets invoked 💀.spcmplt: sets how much silence is needed after speech before a result gets finalized, in the case where there's a complete match of an active grammar 🤏. After that silence, the right<match>element gets triggered if the result is above the confidence level. If not, a<nomatch>element gets triggered.spincmplt: sets how much silence is needed after speech before a result gets finalized, in the case where there's an incomplete match of all the active grammars 🥲. After that silence, the<nomatch>element gets triggered.confidence: the minimum confidence level the recognizer has to have before it counts a recognition result as matching a grammar. It's an integer from 1-100 💯.sens: sets how sensitive the recognizer is when it's deciding whether speech is present 👀. Lower sensitivity may be needed so the recognizer still works well with lots of background noise or line echo 🔊.starttimer: a boolean that decides whether the no input (noint) and no recognition (norect) timers start right away. When it's set tofalse, thestarttimerevent has to be received before they start. The default is"false"🕰️.iterate: sets how many times the<grammar>,<noinput>, and<nomatch>elements can run, unless those elements say something different. The value"forever"can be used to mean they can run any number of times ♾️. The default is once'1'🧚.
Events:
sens: sets the sensitivity of the recognizer as described above 🎚️.starttimer: starts the no input (noint) and no recognition (norect) timers if they haven't been started already. It does nothing otherwise 🙃.terminate: ends the speech input and assigns values to the shadow variables 🛑.
Shadow Variables:
speech.end: holds the event that caused the<speech>to end, or it gets set to one of"speech.match","speech.noinput", or"speech.nomatch", depending on which of the matching elements hit its max 🏁.speech.results: holds the results of a matched grammar. The results are formatted with the Natural Language Semantics Markup Language (NLSML) [n4] 🧾. When this variable gets referenced to return results, the results come back as a separate MIME entity 📦.
The following sections describe the child elements of <speech> 👇.
9.10.1.1. <grammar> 📖
the <grammar> element specifies and activates a speech grammar using Speech Recognition Grammar Specification (SRGS) [n3] XML notation 🧠. Grammars can be referenced by a URI or defined inline. Child elements of <match> MUST be executed when the specified speech grammar is matched 🙌.
Attributes:
uri: specifies where the SRGS grammar lives when it isn't defined inline 📍.iterate: specifies how many times the<grammar>can be matched. The value"forever"MAY be used to mean<grammar>can be matched any number of times ♾️. This value overrides any value set in<speech>. The default is once'1'✨.
9.10.1.2. <match> 🎯
<match> is a child of <grammar>, and it specifies the actions to take when the matching grammar gets matched 💅.
9.10.1.3. <noinput> 🤫
the <noinput> element gets used when speech is being recognized 🎙️. Children of the <noinput> element MUST be executed when speech hasn't been detected and the no input timeout (noint) happens 😬.
Attributes:
iterate: specifies how many times the<noinput>can be triggered. The value"forever"may be used to mean<noinput>can be triggered any number of times ♾️. This value overrides any value set in<speech>. The default is once'1'🧚.
9.10.1.4. <nomatch> 🙅♀️
the <nomatch> element also gets used when speech is being recognized 🎧. Children of the <nomatch> element MUST be executed when it's determined that none of the active grammars will match 🚫.
Attributes:
iterate: specifies the maximum number of times the<nomatch>can be triggered. The value"forever"MAY be used to mean<nomatch>can be triggered any number of times ♾️. This value overrides any value set in<speech>. The default is once'1'✨.
9.10.1.5. <speechexit> 🚪
the <speechexit> element MUST be invoked when the speech input completes because one of <grammar>, <noinput>, or <nomatch> has happened its maximum number of times 🏃♀️💨.
Attributes: none
9.10.2. <play> 🔊
the <play> element, as defined in the MSML Dialog Base Package, gets a new child element that synthesizes speech 🤖🎶. From an XML perspective, <tts> is a member of a media substitution group 🧩. See the schema at the end of this document for details 📄.
The following sections describe the child elements of <play> 👇.
9.10.2.1. <tts> 🗣️
contents of the <tts> element get rendered with text-to-speech services, and they must be compliant with the SSML specification [n11] 💖. Element content MAY be plain text, contain the SSML <speak> element, or the uri attribute should identify where the text to be rendered lives 📍.
Attributes:
uri: identifies the location of the text to be rendered. Thefileandhttpschemes are supported 🌸.iterate: specifies how many times the text-to-speech block gets rendered. It defaults to once'1'🧚.xml:lang: specifies the language to use when it isn't explicitly set as an attribute on<speak>🌍.
9.11. MSML Dialog Fax Detection Package 📠
the Fax Detection Package defines primitives that let a media server provide facsimile detection services 🦖✨.
9.11.1. <faxdetect> 🔍
fax tone detection is used to spot the T.30 Calling Tone (CNG) or the Called Station Identification (CED) tone in a media stream 👂💜. Child elements of <faxdetectexit> MUST be executed when a CNG tone is detected 🙌.
id: an optional identifier that can be referenced elsewhere for sending events to the faxdetect primitive.
Events:
terminate: ends fax tone detection and assigns values to the associated shadow variables 🛑.
Shadow Variables:
faxdetect.tone: a string that specifies the fax tone type the media server detected. Values supported SHOULD include"CED","CNG", or an empty string 🎀. The empty string MUST be used if fax tone detection ended before a fax tone was detected, which results in the<faxdetectexit>element getting executed 🫠.faxdetect.end: a string value that specifies why<faxdetect>ended 🏁. Values supported SHOULD include"faxdetect.complete"(because a CED or CNG tone was detected),"faxdetect.failed.noresource"(failed because the media server ran out of resources),"faxdetect.failed"(failed for any other reason),"faxdetect.terminated"(terminated by<dialogend>), or undefined 🌈.
9.11.2. <faxdetectexit> 🚪
the <faxdetectexit> element MUST be invoked when fax detection, which was started via <faxdetect>, ends 🏃♀️. Its child elements, <send> and <exit>, let the media server report events 📣.
Attributes: none
9.12. MSML Dialog Fax Send/Receive Package 📨
9.12.1. <faxsend> 📠
okay so <faxsend> is the calling fax terminal energy 💅 it's the one that sends a set of pages, main character behavior fr. It can also ask the called terminal to send pages back instead of, or on top of, receiving them. The actual fax images to send get defined by the <sendobj> elements, which we'll get to in a sec.
Asking the other side for pages happens when you include <rxpoll> inside <faxsend>. That can go along with <sendobj> or replace it. One <sendobj> (at a minimum) or <rxpoll> element has to be there, no cap. If both show up, the media server sends its pages first and then polls the other terminal for pages 🫶
Since fax is its own media type, <faxsend> isn't really expected to mix with other primitives 🦋 Instead it talks fax protocols with a remote fax terminal (or gateway) and sends requested status events back to whatever invoked it. While the fax is happening, shadow variables keep track of the progress and parameters of each stage. 📊
Status events get requested by including one or more status request elements. Each one matches a stage or event in fax operation and makes a predefined event fire when it happens. The only thing getting these events is expected to be a fax control agent, so the authors made it simpler: each event comes with a predefined namelist of shadow variables, and you don't have to pick them yourself 🧚 They also say this might get revisited later to allow tailored namelists, once there's more implementation experience. Status requests apply to both sending and polling operation 💜
Attributes:
lclid: the identifier that a media server uses to identify itself.minspeed: the minimum acceptable speed to negotiate for the operation.maxspeed: the maximum speed to negotiate for the operation. This attribute is primarily for testing purposes.ecm: specifies whether Error Correction Mode (ECM) is allowed to be used if supported by the remote terminal. Defaults to"true".
Events:
terminate: terminates the fax send operation. 🛑
Shadow Variables:
fax.rmtid: the identifier of the remote fax terminal. 📞fax.rate: the negotiated speed for the operation.fax.resolution: identifies the resolution of the image. Both metric- and inch-based resolutions are defined. Metric-based resolutions are75x75,150x150,204x98,204x196,204x391, and408x391. Inch-based resolutions are200x200,300x300,400x400, and600x600. 🖼️fax.pagesize: identifies the negotiated page size. Metric sizes are"A3","A4","A5","A6", and"B4". Inch-based page sizes are"Letter"and"Legal". 📄fax.encoding: identifies the image encoding utilized. Valid values are"MH","R","MMR", and"JPEG".fax.ecm: identifies whether ECM operation was used.fax.pagebadlines: the number of bad lines in a page.fax.objbadlines: the number of bad lines in an object.fax.opbadlines: the number of bad lines in an operation.fax.objuri: the objuri of the current object.fax.resendcount: the number of pages resent due to errors. 😬fax.totalpages: the number of pages processed or stored.fax.totalobjects: the count of the objects used in the operation.fax.duration: the duration of the operation expressed as a duration in seconds and milliseconds (e.g.,"23s250ms"). ⏱️fax.result: contains the reason that caused the fax operation to complete. When the operation completes successfully, the value will be assigned"fax.success". Other values include"fax.partial","fax.nofax","fax.remotedisconnect","fax.uri.access.error", and"fax.invalid.startpage". the full drama of how a fax ends 🎬
The following sections describe the child elements of <faxsend>. 🎀
9.12.1.1. <sendobj>
<sendobj> is used to define a fax transmission. There MAY be multiple instances of the element, which will be transmitted in order. 📑
Attributes:
objuri: a URI that points to the fax image that will be transmitted. Mandatory.startpage: the first page of a multi-page objuri to send.pagecount: page count.
9.12.1.2. <hdrfooter>
<hdrfooter> describes the header/footer that a media server MAY put on pages. The header or footer may be defined as the content of the <format> child element. The <format> element is only allowed if the type attribute has a value of "header" or "footer". 🦄
Attributes:
type: specifies whether a header or a footer should be put on pages and identifies the source of the header or footer. The following enumerated values may be used:"header"indicates that the media server should put a header on pages using the contents of the<format>element."nohdr"indicates that there should be no header or footer."footer"indicates that the media server should put a footer on pages using the contents of the<format>element.
style: defines the style of insertion onto a fax page that a media server should use for the header or footer. Valid styles are"append","overlay", or"replace". 🎨
<format> is a child of the <hdrfooter> element that defines the style format to be used for the header or footer. It uses a "C" language style format statement (as shown below) to define the contents and layout of the header or footer. It's giving printf vibes ✨
code length name format
%a 3 day of week 3-character abbreviation
%d 2 date 01-31
%m 2 month 01-12
%y 2 year 00-99
%Y 4 year 0000-9999
%I 2 12 hour 01-12
%H 2 24 hour 00-23
%M 2 minute 00-59
%S 2 seconds 00-59
%p 2 AM/PM AM or PM
%P 2 page number 01-99
%T 2 total pages 01-99
%l 20 local ID (sender) 0-9, + or spaces
%r 20 remote ID (rcvr) 0-9, + or spaces
%% 1 percent display % in header/ftr9.12.1.3. <rxpoll>
<rxpoll> provides the information necessary for a receive polling operation to occur. The object(s) to be received are defined by one or more <rcvobj> elements. The <rcvobj> is defined further under the child elements of <faxrcv>. The <rxpoll> element MAY also include a description of the header/footer that a media server SHOULD put on received pages. The <hdrfooter> element and its usage is described above. 🔁
Attributes:
rmtid: specifies the identifier of the remote fax terminal that is to be associated with a polling operation. A media server MUST NOT execute a polling operation unless the value ofrmtidmatches that of the connected remote machine. Mandatory. 🔒
9.12.1.4. <faxstart>
The <faxstart> element requests that an event be sent when fax operation has begun. When triggered, the following will be executed: 🚀
<send target="source" event="fax.start"/>9.12.1.5. <faxnegotiate>
The <faxnegotiate> element requests that an event be sent when a negotiation has been completed. Multiple events MAY be sent each time a Digital Command Signal (DCS) frame is sent or received. When triggered, the following will be executed: 🤝
<send target="source" event="fax.negotiate"
namelist="fax.rmtid
fax.rate
fax.resolution
fax.pagesize
fax.encoding
fax.ecm"/>9.12.1.6. <faxpagedone>
The <faxpagedone> element requests that an event be sent when a page has been sent or received. When triggered, the following will be executed: 📃✅
<send target="source" event="fax.pagedone"
namelist="fax.resolution
fax.pagesize
fax.encoding
fax.pagebadlines
fax.resendcount"/>9.12.1.7. <faxobjectdone>
The <faxobjectdone> element requests that an event be sent when an objuri has been completed. When triggered, the following will be executed: 🎉
<send target="source" event="fax.objectdone"
namelist="fax.objuri
fax.objbadlines
fax.resendcount
fax.totalpages
fax.result"/>9.12.1.8. <faxopcomplete>
The <faxopcomplete> element requests that an event be sent when an operation has been completed. When triggered, the following will be executed: 👑
<send target="source" event="fax.opcomplete"
namelist="fax.totalpages
fax.opbadlines
fax.resendcount
fax.totalobjects
fax.duration
fax.result"/>9.12.1.9. <faxpollstarted>
The <faxpollstarted> element requests that an event be sent when a polling operation has started. When triggered, the following will be executed: 🌸
<send target="source" event="fax.opcomplete"
namelist="fax.rmtid
fax.rate
fax.resolution
fax.pagesize
fax.encoding
fax.ecm"/>9.12.2. <faxrcv> ✨
okay so the <faxrcv> primitive is the called fax terminal's whole personality 📠 Usually it's all about receiving pages, but it can also send pages instead of receiving them, or on top of receiving them 💅 The fax objects it's going to receive get defined by the <rcvobj> elements, which we'll get to in a sec 🫶
A media server SHOULD send pages as a polled terminal when the <txpoll> element is included as part of <faxrcv>. That element can show up alongside <rcvobj> or instead of it. One <rcvobj> or <txpoll> element must be present, bestie 🔥 When both are there, a media server SHOULD first receive pages, and then it'll let the other terminal poll it to request pages 🌈
Fax is its own distinct media type, so <faxrcv> isn't really expected to interact with other primitives 🧚 Instead, it talks to a remote fax terminal over fax protocols and sends the requested status events back to whatever invoked it 💜 While fax is happening, shadow variables record the progress and parameters of each stage 📊
Status events get requested by including one or more status request elements ✨ These map to different stages or events in fax operation, and when those happen, the predefined events get sent to the invoking environment 🔔 The only recipient of these events is expected to be a fax control agent, so requests are simplified by tying a predefined namelist of shadow variables to each event 🎀 This decision may be revisited to allowed tailored namelists based on further implementation experience 🙈 Status requests apply to both receiving and polling operation, so they're iconic like that 👑
Attributes:
id: an optional identifier that may be referenced elsewhere for sending events to the faxrecv primitive.lclid: the identifier that a media server uses to identify itself.ecm: specifies whether ECM mode is allowed to be used if supported by the remote terminal. Defaults to"true".
Events:
terminate: terminates the fax reception operation.
Shadow Variables:
<faxrcv> supports the same set of shadow variables as <faxsend> 🦋
The following sections describe the child elements of <faxrcv>. 🌸
In addition to the elements defined below, <faxrcv> MAY also have the following child elements, which were defined under <faxsend>:
<hdrfooter><faxstart><faxnegotiate><faxpagedone><faxobjectdone><faxopcomplete><faxpollstarted>
Their meaning and usage are the same as previously defined. 💖
9.12.2.1. <rcvobj>
<rcvobj> is used to define the fax objects a media server will receive 📥 You can have multiple instances of it, and they get used in order, so no skipping ahead 🍓
Attributes:
objuri: a URI that points to the location where a received image is to be stored. Mandatory.maxpages: the maximum number of pages that will be stored inobjuri.
9.12.2.2. <txpoll>
<txpoll> provides the info for a polling operation to happen as part of a fax receive operation 🔁 An object, or multiple objects, to be sent can be supplied by one or more <sendobj> elements 📤 If there are multiple occurrences, a media server MUST select the <sendobj> element whose rmtid attribute matches that of the remote terminal. Main character energy, the right one gets picked every time 👑
The <sendobj> element was already defined earlier as a child element of <faxsend> 💅 The <txpoll> element gets extended with an rmtid attribute that specifies the identifier of the remote fax terminal, and it's used to select the specific <sendobj> to send 🎀
A media server SHOULD put a header/footer on transmitted pages based on any <hdrfooter> element included as part of <txpoll> 🫶
Attributes:
rmtid: specifies the identifier of the remote fax terminal that is to be associated with a polling operation. A media server MUST NOT execute a polling operation unless the value ofrmtidmatches that of the connected remote machine. Mandatory.
10. MSML Audit Package 🦋
10.1. MSML Audit Core Package
This section describes the MSML Audit Core Package that MAY be implemented to support auditing services 🔍
Audit requests and results can vary based on the info being audited 📋 The MSML Audit Core Package sets up the framework for sending audit requests, defines a state list, and builds audit results. The additional audit packages define package-specific state lists and the audit result content that goes with them. The additional audit packages MUST be defined within the framework specified by the Audit Core Package. It's giving structure, honestly 💜
10.1.1. <audit>
The <audit> element is an optional child element of <msml>, which MAY be used by MSML clients to perform state auditing of current media resources that are allocated and in use by the media server 🌈 The requested state information comes back in an MSML response ✨
Attributes:
queryid: the identifier of the MSML object being queried by the MSML client. Mandatory. Supported object types: conference or connection. Wildcards are allowed.statelist: a list of one or more state parameters that are being queried. Optional. If not present, the media server SHOULD return the id of the audited object only. Each object type may contain a set of states. If the"statelist"contains any state that does not match the audited object type, the request MUST be rejected 🙅♀️mark: in the case of an error, the value of themarkattribute from the last successfully executed element that included themarkattribute.
State Parameters:
The state parameter MUST be named using a dot-notation format "audit.X.a.b.c...", where X is the mandatory field that indicates the class name of the object (e.g., "conf" or "conn"), and "a.b.c..." is the optional field used to describe the actual name of the state parameter in a hierarchical manner 🧩 The wildcard "*" MAY be used as part of a state name. However, it MUST only be used in the last field of the dot-notation (e.g., "audit.conf.*" is valid, but "audit.conf.*.a" is invalid). When a wildcard is used, it's equivalent to querying all the states below the specified level 🌸 Each field (e.g., within "a.b.c...") results in individual element names <a>, <b>, and <c> in the audit result, which contain the corresponding state value. The parent/child relationship between these elements follows the hierarchy of the state name (i.e., <c> is a child element of <b>, and <b> is a child element of <a>) 👑
10.1.2. <auditresult>
The <auditresult> element is an optional child element of <result>, and it MUST be used by the media server to return the audit result 📦 A specific instance of the <auditresult> element contains the state information of a single active object. So if multiple objects are within the scope of the audit request, one <auditresult> element per object MUST be present. A zero occurrence of <auditresult> means there are no active resources within the scope of the audit request 🫠
Attributes:
targetid: the identifier of a conference or connection. Mandatory. Wildcard is not allowed.
The <auditresult> may contain child elements that return additional state information, corresponding to the "statelist" attribute in the <audit> request. The child element names correspond to the fields of the state parameter name (e.g., "a.b.c..."), following the same hierarchical structure 🧚
10.2. MSML Audit Conference Package
This section describes the MSML Audit Conference Package that MUST be implemented to support auditing of conference services 🎪 The MSML Audit Conference Package follows the framework specified by the MSML Audit Core Package. This package defines the state parameter list and audit result for conference auditing 💖
10.2.1. State Parameters
All conference state parameter names MUST be prefixed by "audit.conf" 🏷️
confconfig: query the conference's general configuration.confconfig.audiomix: query the audio mixer's general configuration in the conference.confconfig.audiomix.asn: query the current ASN setting in the audio mixer.confconfig.audiomix.n-loudest: query the current n-loudest setting in the audio mixer.confconfig.videolayout: query the video layout's general configuration in the conference.confconfig.videolayout.root: query the root window setting of the video layout.confconfig.videolayout.selector: query the video stream selector setting of the video layout.confconfig.controller: query who the conference controller is.dialog: query the active dialog information on the conference. See MSML Audit Dialog Package for details.stream: query the active stream information on the conference. See MSML Audit Stream Package for details.
10.2.2. <auditresult>
The <auditresult> attribute of "targetid" is required to indicate results for auditing a conference 🎯
The <auditresult> element may optionally contain the following child elements, which return additional conference state information if the corresponding states are queried and available ✨
10.2.2.1. confconfig
Hmm, I need to keep heading levels consistent with the depth rule I've been using, so I'll correct that below.
The <confconfig> element is used to return the general configuration state(s) of a conference, using the following attributes 🌈
Attributes:
deletewhen: as defined by<createconference>element in MSML Conference Core Package.term: as defined by<createconference>element in MSML Conference Core Package.
10.2.2.2. confconfig.audiomix
The <audiomix> element contains the general audio mixer configuration using the following attributes 🎚️
Attributes:
id: as defined by<audiomix>element in MSML Conference Core Package.samplerate: as defined by<audiomix>element in MSML Conference Core Package.
10.2.2.3. confconfig.audiomix.asn
The <asn> element contains the current ASN setting of an audio mixer, if ASN is enabled. The state values are included in the following attributes 🔊
Attributes:
ri: as defined by<asn>element in MSML Conference Core Package.asth: as defined by<asn>element in MSML Conference Core Package.
10.2.2.4. confconfig.audiomix.n-loudest
The <n-loudest> element contains the current n-loudest setting of the audio mixer. The state values are included in the following attributes 🔝
Attributes:
n: as defined by<n-loudest>element in MSML Conference Core Package.
10.2.2.5. confconfig.videolayout
The <videolayout> element contains the general video layout configuration using the following attributes 🎬
Attributes:
id: as defined by<videolayout>in MSML Conference Core Package.type: as defined by<videolayout>in MSML Conference Core Package.
10.2.2.6. confconfig.videolayout.root
The <root> element is used to contain root window settings 🪟
Attributes:
size: as defined by<root>element in MSML Conference Core Package.backgroundcolor: as defined by<root>element in MSML Conference Core Package.Backgroundimage: as defined by<root>element in MSML Conference Core Package.
10.2.2.7. confconfig.videolayout.selector
Okay so the <selector> element is where all the selector settings live 🎬✨
Attributes:
id: as defined by<selector>element in MSML Conference Core Package.method: as defined by<selector>element in MSML Conference Core Package.status: as defined by<selector>element in MSML Conference Core Package.blankothers: as defined by<selector>element in MSML Conference Core Package.si: as defined by<selector>element in MSML Conference Core Package when selector method is"vas".speakersees: as defined by<selector>element in MSML Conference Core Package when selector method is"vas".
10.2.2.8. confconfig.controller
The <controller> element is used to return the conference controller id in its content 💅 The conference controller is the SIP dialog that carries the <createconference> request. The return value is the MSML connection id.
10.2.2.9. dialog
If conference dialog state is queried, the audit result comes back using the <dialog> element, as specified in the MSML Audit Dialog Package 📞
10.2.2.10. stream
If conference stream state is queried, the audit result comes back using the <stream> element, as specified in the MSML Audit Stream Package 🌊
10.3. MSML Audit Connection Package
This section describes the MSML Audit Connection Package, which MAY be implemented to support auditing connection services 🦋 It follows the framework specified by the MSML Audit Core Package, and it defines the state parameter list and audit result for connection auditing.
10.3.1. State Parameters
Connection state parameter names are prefixed by "audit.conn" 🎀
sipdialog: queries the identifier of the SIP dialog with which the connection is associated.sipdialog.localseq: queries one of the SIP dialog states, the local sequence number.sipdialog.remoteseq: queries one of the SIP dialog states, the remote sequence number.sipdialog.localURI: queries one of the SIP dialog states, the local URI.sipdialog.remoteURI: queries one of the SIP dialog states, the remote URI.sipdialog.remotetarget: queries one of the SIP dialog states, the remote target.sipdialog.routeset: queries one of the SIP dialog states, the route set.localsdp: queries the local SDP body of the connection.remotesdp: queries the remote SDP body of the connection.dialog: queries the active dialog information on the connection. See MSML Audit Dialog Package for details.stream: queries the active stream information on the connection. See MSML Audit Stream Package for details.
10.3.2. <auditresult>
The <auditresult> attribute targetid MUST specify a connection identifier for a connection result 🔥
The <auditresult> element MAY contain the following child elements optionally, to return extra connection state information if the matching states are queried and available 💖
10.3.2.1. sipdialog
The <sipdialog> element contains the associated SIP dialog information. The SIP dialog ID information is returned using the following attributes ✨
Attributes:
callid: call-ID value as defined in [n1]. Mandatory.localtag: local-tag value as defined in [n1]. Mandatory.remotetag: remote-tag value as defined in [n1]. Mandatory.
This element can contain the following child elements optionally, to return additional SIP dialog state information to the client if the matching states are queried and available 🫶
10.3.2.2. sipdialog.localseq
The <localseq> element contains the local sequence number. The local sequence number is one of the SIP dialog states as defined in [n1] 🌸
10.3.2.3. sipdialog.remoteseq
The <remoteseq> element contains the remote sequence number. The remote sequence number is one of the SIP dialog states as defined in [n1] 🌸
10.3.2.4. sipdialog.localuri
The <localuri> element contains the local URI value. The local URI is one of the SIP dialog states as defined in [n1] 🧚
10.3.2.5. sipdialog.remoteuri
The <remoteuri> element contains the remote URI value. The remote URI is one of the SIP dialog states as defined in [n1] 🧚
10.3.2.6. sipdialog.remotetarget
The <remotetarget> element contains the remote target value. The remote target is one of the SIP dialog states as defined in [n1] 🎯
10.3.2.7. sipdialog.routeset
The <routeset> element contains the route-set value, which is an ordered list of URIs separated by comma. The route set is one of the SIP dialog states as defined in [n1] 🛣️
10.3.2.8. localsdp
The <localsdp> element contains the local SDP body 🎶
10.3.2.9. remotesdp
The <remotesdp> element contains the remote SDP body 🎶
10.3.2.10. dialog
If the connection dialog state is queried, the audit result returns the queried information using the <dialog> element, as specified in the MSML Audit Dialog Package 📞
10.3.2.11. stream
If the connection stream state is queried, the audit result returns the queried information using the <stream> element, as specified in the MSML Audit Stream Package 🌊
10.4. MSML Audit Dialog Package
This section describes the MSML Audit Dialog Package, which MAY be implemented to support auditing dialogs 💜 It follows the framework specified by the MSML Audit Core Package.
The MSML Audit Dialog Package must be used together with either the MSML Audit Conference Package or MSML Audit Connection Package, since dialogs apply to conferences or connections. Bestie, that's the whole point 🙈
10.4.1. State Parameters
Dialog state parameter names are prefixed by dialog. Since this package must be used together with the MSML Audit Conference Package or MSML Audit Connection Package, the complete dialog state name must be prefixed by "audit.conf.dialog" or "audit.conn.dialog", depending on the context in which the dialog state is queried 🧩
dialog: queries the number of active dialog(s) running on the target (a conference or connection). Basic dialog information will be returned.dialog.duration: queries the amount of time a dialog has been running.dialog.primitive: queries the media primitive currently being executed by the dialog.dialog.controller: queries the dialog controller.
10.4.2. <dialog>
The <dialog> element is a child element of <auditresult>, and it contains the active dialog information on the target identified by the attribute targetid of the <audioresult> element 🎀
Basic dialog information is returned using the following attributes.
Attributes:
src: as defined by the<dialogstart>element in the MSML Dialog Core Package.type: as defined by the<dialogstart>element in the MSML Dialog Core Package. Mandatory.name: as defined by the<dialogstart>element in the MSML Dialog Core Package. Mandatory.
This element may contain the following child elements optionally, to return additional dialog information if the matching state parameter has been queried and the state value is available 💅
10.4.2.1. <duration>
Hold up, the <duration> element returns how long a dialog has been running on the specified target ⏱️ The duration value goes in the element content, and it is a positive integer value in units of seconds.
10.4.2.2. <primitive>
The <primitive> element returns the currently active media primitive in its content. The active media primitive is the primitive that is being executed right now 🎬 Possible return values are play, dtmf, collect, dtmfgen, tonegen, record, or none.
10.4.2.3. <controller>
The <controller> element returns the dialog controller id in its content. The dialog controller is the SIP dialog that carries the <dialogstart> request. The returned value is the MSML connection id 👑
10.5. MSML Audit Stream Package
This section describes the MSML Audit Stream Package, which MAY be implemented to support auditing streams 🌈 It follows the framework specified by the MSML Audit Core Package.
The MSML Audit Stream Package MUST be used together with either the MSML Audit Conference Package or the MSML Audit Connection Package, since streams apply between conferences, between connections, or between conferences and connections. No cap, it's a must 💀
10.5.1. State Parameters
Stream state parameter names are prefixed by stream. Since this package must be used together with the MSML Audit Conference Package or MSML Audit Connection Package, the complete stream state name must be prefixed by "audit.conf.stream" or "audit.conn.stream", depending on the context in which the stream state is queried 🦋
stream: queries the number of active streams created on the audited object. Basic stream information will be returned.stream.clamp: queries the clamping status.stream.gain: queries the gain control information.stream.visual: queries the visual setting.
10.5.2. <stream>
The <stream> element is a child element of <auditresult>, and it contains the active stream information on the target identified by the attribute targetid of the <audioresult> element 🌊
Basic stream information is returned using the following attributes.
Attributes:
joinwith: an identifier of either a connection or a conference with which the audited object is joined. Mandatory. Wildcard is not allowed.media: as defined by the<stream>element in the MSML Conference Core Package. Mandatory.dir: direction of stream, from the audited target's perspective,"from"or"to". Mandatory.compressed: as defined by the<stream>element in the MSML Conference Core Package.display: as defined by the<stream>element in the MSML Conference Core Package.override: as defined by the<stream>element in the MSML Conference Core Package.preferred: as defined by the<stream>element in the MSML Conference Core Package.
This element MAY contain the following child elements that optionally return additional stream information, if the corresponding state parameter is queried and the state value is available ✨
10.5.2.1. <clamp>
The <clamp> element is included if stream clamping is active. The currently active clamping state values are returned using the attributes as defined by the <clamp> element in the MSML Conference Core Package 🧊
10.5.2.2. <gain>
The <gain> element is included if stream gain is active. The current gain control state values are returned using the attributes as defined by the <gain> element in the MSML Conference Core Package 🔊
10.5.2.3. <visual>
The <visual> element is included if stream visual display is active. The current visual display settings are returned using the attributes as defined by the <visual> element in the MSML Conference Core Package 👀
11. Response Codes ✨
okay so response codes are how the server tells you what happened, whether it went great or totally flopped 💀. when something fails, the right code and description must get passed back to whatever invoked it, no exceptions bestie 🫶
these codes show up as the value of the response attribute on the <result> element. some of them can also ride along inside a namelist on an "msml.dialog.exit" event when an MSML dialog fails 🦋
Here's the full lineup:
Informational (1xx)
Reserved for future use
Success (200)
200 OK
Request Error (4xx)
400 Bad Request
401 Unknown Element
402 Unsupported Element
403 Missing mandatory element content
404 Forbidden element content
405 Invalid element content
406 Unknown attribute
407 Attribute not supported
408 Missing mandatory attribute
409 Forbidden attribute is present
410 Invalid attribute value
420 Unsupported media description language
421 Unknown media description language
422 Ambiguous request (both URI and inline description)
423 External document fetch error
424 Syntax error in foreign language
425 Semantic error in foreign language
426 Unknown error executing foreign language
430 Object does not exist
431 Object instance name already used
432 Conference name already in use
433 reserved
434 External document fetch error
440 Cannot join objects of the specified class
441 Objects have incompatible media types
442 reserved
443 reserved
444 Number of media inputs exceeded
450 Objects have incompatible media formats
451 Incompatible media stream format
Server Error (5xx)
500 Internal media server error
503 Service Unavailable
510 Not in service
511 Service Unavailable
520 No resource to fulfill request
521 Internal limit exceeded12. MSML Conference Examples 🎀
these examples zoom in on the MSML Conference Core Package, which is what a control agent (CA) uses to steer services on a media server (MS) 🧚 they show how the SIP signaling that sets up the media sessions lines up with the MSML service control commands. to keep things short, only the MSML message content is shown. the examples assume the CA and MS use the IPv4 address and UDP port number of the audio stream (on the MS) to identify the MSML connection 💖
12.1. Establishing a Dial-In Conference 📞
UA Control Agent Media Server
| | |
| | INVITE F1 |
| |-------------------------->|
| | 200 F2 |
| |<--------------------------|
| | ACK F3 |
| |-------------------------->|
| | |
| | createconference> F4 |
| |-------------------------->|
| | 200 F5 |
| |<--------------------------|
| INVITE (SDP UA) F6 | |
|------------------------>| |
| | INVITE (SDP UA) F7 |
| |-------------------------->|
| | 200 (SDP MS) F8 |
| |<--------------------------|
| | ACK F9 |
| |-------------------------->|
| 200 (SDP MS) F10 | |
|<------------------------| |
| ACK F11 | |
|------------------------>| |
| | <dialogstart> F12 |
| |-------------------------->|
| | 200 F13 |
| |<--------------------------|
| | HTTP interactions F14 |
| |<------------------------->|
| | <event>(dialog.exit) F15 |
| |<--------------------------|
| | <join> F16 |
| |-------------------------->|
| | 200 F17 |
| |<--------------------------|
| ... | ... |
| | |
| | <dialogstart> F18 |
| |-------------------------->|
| | 200 F19 |
| |-------------------------->|
| | HTTP interactions F20 |
| |<--------------------------|
| | <event>(dialog.exit) F21 |
| |-------------------------->|
| ... | ... |
| | |Steps 1-3 set up an MSML control channel for the conference 🔐. you could also already have a control channel running that handles every CA/MS interaction, since one channel per conference is just one possible setup. right now MSML rides on SIP INFO requests and responses over that SIP dialog. there's a proposal to use that same message exchange to set up a TCP channel for MSML, kind of like how the Media Resource Control Protocol v2 (MRCPv2) does it. that approach would need a request identifier added to the <msml> element so requests and responses can be matched up. today it leans on the SIP INFO request and response for that job. the MSML messages below don't specify a transport, but they assume the request/response correlation comes from the transport messages ✨
Step 4: create a conference that mixes the loudest two speakers and reports those speakers back to the control agent every 10 seconds 🎤. the media server will automatically end any remaining media sessions, delete the conference and its associated resources, and do the same when the control channel goes down 💀
<msml version="1.1">
<createconference name="exampleConf" deletewhen="nocontrol">
<audiomix>
<n-loudest n="3"/>
<asn ri="10s"/>
</audiomix>
</createconference>
</msml>Step 5: the conference got created successfully 🎉
<msml version="1.1">
<result response="200"/>
</msml>Steps 6-11: standard 3PCC setup for a user-initiated media session to a media server 📱. this is basically the equivalent of a dial-in conference participant. the "To:" header the MS sent back in the 200 response to Step F8 looked like this:
To: <sip:msml@ms.example.com>;tag=jd87dfg4hStep 12: start an initial dialog with the participant to ask for their name, which conference they want, and so on 🙋♀️. the dialog wraps up by telling the participant they're joining the conference. if this isn't the first participant, the dialog can also announce who else is already there 👯
<msml version="1.1">
<dialogstart target="conn:jd87dfg4h" name="12345"
type="application/vxml+xml"
src="http://server.example.com/scripts/initial.vxml"/>
</msml>Step 13: the dialog started successfully, and the dialog identifier comes back in the response 🦋
<msml version="1.1">
<result response="200"/>
<dialogid>conn:jd87dfg4h/dialog:12345</dialogid>
</msml>Step 14: a sequence of HTTP VoiceXML dialog interactions happens here 🗣️
Step 15: the VoiceXML browser exits, but the participant doesn't get disconnected 👀. if a namelist had been set inside the VoiceXML <exit> element, it would be included in the <event> sent to the CA 📨
<msml version="1.1">
<event name="msml.dialog.exit"
id="conn:jd87dfg4h/dialog:12345"/>
</msml>Step 16: join the participant to the conference, and automatically adjust the volume of their outgoing audio to a target level of -20 dBm0 🔊✨
<msml version="1.1">
<join id1="conn:jd87dfg4h" id2="conf:exampleConf">
<stream media="audio" dir="from-id1">
<gain agc="true" tgtlvl="-20"/>
</stream>
<stream media="audio" dir="to-id1"/>
</msml>Step 17: they're successfully joined to the conference 🥳
<msml version="1.1">
<result response="200"/>
</msml>Steps 6 through 17 get repeated for the second participant 👭
Step 18: play a join tone or message to announce the new participant to the conference 🔔
<msml version="1.1">
<dialogstart target="conf:exampleConf"
type="application/vxml+xml"
src="http://server.example.com/scripts/joinmsg.vxml"/>
</msml>Step 19: the dialog started successfully and the dialog identifier comes back. since no name attribute was given in <dialogstart>, the media server assigned its own unique identifier 🪄
<msml version="1.1">
<result response="200"/>
<dialogid>conf:ExampleConf/dialog:j6fs8745</dialogid>
</msml>Step 20: HTTP VoiceXML dialog interaction(s) go down here 💬
Step 21: the VoiceXML browser exits 👋
<msml version="1.1">
<event name="msml.dialog.exit"
id="conf:ExampleConf/dialog:j6fs8745"/>
</msml>Steps 6 through 21 get repeated for the third and every later participant 🌈💖
12.2. Example of a Sidebar Audio Conference 🎧
okay so this example picks up right where the last one left off, with the conference already set up 💅 Now we make a sidebar conference that hears the main conference as a whisper 🤫 Three participants get moved into the sidebar. After a bit, they get sent back to the main conference and the sidebar is deleted. bye sidebar 👋
Step 1 is where the sidebar gets created 🎀 It's joined half-duplex to the main conference, and a manual gain object goes into the media stream. Then the three participants get moved from the main conference over to the sidebar. Not shown here, but a CA could put the "mark" attribute in each element so it can recover if something goes wrong halfway through a transaction 🧚
<msml version="1.1">
<createconference name="sidebarConf"
deletewhen="nomedia">
<audiomix/>
</createconference>
<join id1="conf:sidebarConf" id2="conf:exampleConf">
<stream media="audio" dir="to-id1">
<gain amt="-20"/>
</stream>
</join>
<unjoin id1="conn:gs5s4-1" id2="conf:exampleConf"/>
<join id1="conn:gs5s4-1" id2="conf:sidebarConf"/>
<unjoin id1="conn:hd764gr9-2" id2="conf:exampleConf"/>
<join id1="conn:hd764gr9-2" id2="conf:sidebarConf"/>
<unjoin id1="conn:h37frdvgs65-3" id2="conf:exampleConf"/>
<join id1="conn:h37frdvgs65-3" id2="conf:sidebarConf"/>
</msml>Step 2 is the MS saying "sidebar created and everyone's in!" ✨ 200 means slay 💖
<msml version="1.1">
<result response="200"/>
</msml>Step 3 is when the sidebar chat is done and everyone gets rejoined to the main conference 🌸 The MS destroys the sidebar on its own once the last media stream is removed, which is what was set up when the sidebar got created. so tidy 🧹
<msml version="1.1">
<unjoin id1="conn:gs5s4-1" id2="conf:sidebarConf"/>
<join id1="conn:gs5s4-1" id2="conf:exampleConf"/>
<unjoin id1="conn:hd764gr9-2" id2="conf:sidebarConf"/>
<join id1="conn:hd764gr9-2" id2="conf:exampleConf"/>
<unjoin id1="conn:h37frdvgs65-3" id2="conf:sidebarConf"/>
<join id1="conn:h37frdvgs65-3" id2="conf:exampleConf"/>
</msml>Step 4 is the final confirmation: everyone's back in the main conference and the sidebar is gone 🦋
<msml version="1.1">
<result response="200"/>
</msml>12.3. Example of Removing a Conference 🫶
this one assumes a conference set up like the first example, with an MSML control channel that belongs to that conference, and the conference is set to get deleted when that channel goes away (this is done using SIP) 📞
Steps 1-2: the CA sends a SIP BYE for the dialog that set up the conference control channel 👋
Steps 3-6: the MS starts ending the media sessions for everyone still hanging out in the conference 🏃♀️
The MS deletes the conference and frees up all its resources once the last participant is gone 🧹💜
12.4. Example of Modifying Video Layout 🎬
so say a conference named "example" gets created with these mixer descriptions. picture a 2x2 grid of boxes, numbered 1 through 4 🍓
+---+---+
| 1 | 2 |
+---+---+
| 3 | 4 |
+---+---+<createconference name="quad-split">
<audiomix>
<n-loudest n="3"/>
<asn ri="10s"/>
</audiomix>
<videolayout>
<root size="CIF" background="white" />
<selector id="default" method="vas" si="500ms">
<region id="1" left="0" top="0" relativesize="1/4"/>
</selector>
<region id="2" left="50%" top="0" relativesize="1/4"/>
<region id="3" left="0%" top="50%" relativesize="1/4">
<region id="4" left="50%" top="50%" relativesize="1/4"/>
</videolayout>
</createconference>next, this one changes the video window to QCIF and sets the background back to the default "black" 🖤
<modifyconference id="conf:example">
<videolayout>
<root size="4CIF"/>
</videolayout>
</modifyconference>the relative location of the regions doesn't change 💅 but the sizes of the regions do, since they're relative to the size of the root window. so the layout looks exactly the same, just half the size ✨
and this one freezes the video in region "2" without touching any of its other attributes 🧊
<modifyconference id="conf:example">
<videolayout>
<region id="2" left="50%" top="0" relativesize="1/4"
freeze="true"/>
</videolayout>
</modifyconference>13. MSML Dialog Examples 🌈
these examples focus on the MSML Dialog Base Package and the MSML Dialog Group Package 🧚
13.1. Announcement 📢
this is a simple announcement scenario 🎀 two recorded audio files get played in order, then some generated speech, then a variable. the results get reported once the media generation is done ✅
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<dialogstart target="conn:12345" name="12345">
<play>
<audio uri="file://clip1.wav"/>
<audio uri="http://host1/clip2.wav"/>
<tts uri="http://host2/text.ssml"/>
<var type="date" subtype="mdy" value="20030601"/>
</play>
<send target="source" event="done" namelist="play.amt
play.end"/>
</dialogstart>
</msml>13.2. Voice Mail Retrieval 📼
here's a voice mail retrieval example 💖 it plays a message, and the user can pause and resume it by pressing '5' to toggle. the whole thing ends when the play finishes or the user presses '#' 🙈
while the message is playing, the user can skip forward and backward through it, and rewind back to the beginning ⏪
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<dialogstart target="conn:12345" name="12345">
<group topology="parallel">
<play>
<audio uri="file://message.wav"/>
<playexit>
<send target="group" event="terminate"/>
</playexit>
</play>
<dtmf iterate="forever">
<pattern digits="5">
<send target="play" event="toggle-state"/>
</pattern>
<pattern digits="6">
<send target="play" event="forward"/>
</pattern>
<pattern digits="7">
<send target="play" event="backward"/>
</pattern>
<pattern digits="8">
<send target="play" event="restart"/>
</pattern>
<pattern digits="#">
<send target="play" event="terminate"/>
</pattern>
</dtmf>
</group>
</dialogstart>
</msml>13.3. Play and Record 🎙️
this one is the big boss of the examples 😤 it plays a prompt and records, and it uses voice activity and DTMF detection to decide what happens next. any DTMF input or voice activity barges the play and starts the record 🔥 but if the prompt got barged with the '#' digit, the record terminates without starting at all 🚫
when the play ends, it sends a starttimer event to the VAD so it can watch for an initial silence 👂 if the VAD detects 3 seconds of initial silence, the recording terminates without starting ⏹️
once the recording has resumed (based on voice detection), it can end in a few ways. it ends after 5 seconds of silence, or after 60 seconds have passed, or if a '#' key gets recognized 🔚 every bit of this behaviour can be changed by tweaking what gets recognized and which events get sent 🛠️ the following example uses the MSML Dialog Group Package 💅
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<dialogstart target="conn:12345" name="12345">
<group topology="parallel">
<play>
<audio uri="file://prompt.wav"/>
<playexit>
<send target="vad" event="starttimer"/>
</playexit>
</play>
<dtmf>
<pattern digits="#">
<send target="record" event="terminate.termkey"/>
</pattern>
<detect>
<send target="play" event="terminate"/>
</detect>
</dtmf>
<vad>
<voice len="10ms">
<send target="play" event="terminate"/>
<send target="record" event="resume"/>
</voice>
<silence len="3s">
<send target="record" event="nospeech"/>
</silence>
<tsilence len="5s">
<send target="record" event="terminate.finalsilence"/>
</tsilence>
</vad>
<record initial="suspend" maxtime="60s"
dest="file://record.wav" format="g729">
<recordexit>
<send target="group" event="terminate"/>
</recordexit>
</record>
<groupexit>
<send target="source" event="done"
namelist="record.len record.end"/>
</groupexit>
</group>
</dialogstart>
</msml>the next one does the exact same thing, as described above, but with the MSML Dialog Base Package instead 🧁 it uses the <record> composite mechanism for the play and record operation, so it's way more compact 💜
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<dialogstart target="conn:12345" name="12345">
<record prespeech="3s" postspeech="5s" maxtime="60s" termkey="#"
dest="file://record.wav" format="g729">
<play barge="true">
<audio uri="file://prompt.wav"/>
</play>
<recordexit>
<send target="source" event="done"
namelist="record.len record.end"/>
</recordexit>
</record>
</dialogstart>
</msml>13.4. Speech Recognition 🗣️
this simple example asks the user to say the name of a city, then sends back the result 🌍
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<dialogstart target="conn:12345" name="12345">
<group topology="parallel">
<play>
<audio uri="file://prompt.wav"/>
</play>
<speech>
<grammar version="1.0">
<rule id="city" scope="public">
<item>
<one-of>
<item>vancouver</item>
<item>new york</item>
<item>london</item>
</one-of>
</item>
</rule>
<match>
<send target="group" event="terminate"/>
</match>
</grammar>
<noinput>
<send target="group" event="terminate"/>
</noinput>
<nomatch>
<send target="group" event="terminate"/>
</nomatch>
</speech>
<groupexit>
<send target="source" event="done"
namelist="speech.end speech.results"/>
</groupexit>
</group>
</dialogstart>
</msml>13.5. Play and Collect 🎀
okay so this one prompts a user to enter 4 DTMF digits, ended by the '#' key (that's the xxxx# you'll see below) 💅 the prompt can be barged, and the user gets 10 seconds to start entering input or no input gets indicated 🫶
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<dialogstart target="conn:12345" name="12345">
<group topology="parallel">
<play>
<audio uri="file://prompt.wav"/>
<playexit>
<send target="dtmf" event="starttimer"/>
</playexit>
</play>
<dtmf fdt="10s" idt="16s">
<pattern digits="xxxx#">
<send target="group" event="terminate"/>
</pattern>
<detect>
<send target="play" event="terminate"/>
</detect>
<noinput>
<send target="group" event="terminate"/>
</noinput>
<nomatch>
<send target="group" event="terminate"/>
</nomatch>
</dtmf>
<groupexit>
<send target="source" event="done"
namelist="dtmf.digits dtmf.end"/>
</groupexit>
</group>
</dialogstart>
</msml>The following does the exact same thing as above, but using the MSML Dialog Base Package and the <collect> composite mechanism for the play and collect operation ✨
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<dialogstart target="conn:12345" name="12345">
<collect fdt="10s" idt="16s">
<play barge="true">
<audio uri="file://prompt.wav"/>
</play>
<pattern digits="xxxx#">
<send target="source" event="done"
namelist="dtmf.digits dtmf.end"/>
</pattern>
<noinput>
<send target="source" event="done"
namelist="dtmf.end"/>
</noinput>
<nomatch>
<send target="source" event="done"
namelist="dtmf.end"/>
</nomatch>
</collect>
</dialogstart>
</msml>13.6. User Controlled Gain 🌈
this one is so cool 😭 it shows how nesting groups can make an arbitrary full-duplex media control. DTMF gets detected on media flowing in one direction and is used to adjust the gain on media flowing the opposite way. On top of that, the stream used for DTMF detection has DTMF removed and its gain auto-adjusted before it leaves the group. This widget could sit between a conference participant and a conference mixer 🦋
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.0">
<dialogstart target="conn:12345" name="12345">
<group topology="fullduplex">
<group topology="parallel">
<dtmf>
<pattern digits="1" iterate="forever">
<send target="gain" event="louder"/>
</pattern>
<pattern digits="2" iterate="forever">
<send target="gain" event="softer"/>
</pattern>
</dtmf>
<group topology="serial">
<clamp/>
<agc tgtlvl="0"/>
</group>
</group>
<gain amt="0" incr="5"/>
</group>
</dialogstart>
</msml>14. MSML Audit Examples ✨
these examples cover the MSML Audit Conference Package and the MSML Audit Connection Package, and how they work together with the MSML Audit Dialog Package or/and the MSML Audit Stream Package 🔥
14.1. Audit All Conferences 👑
this one audits every active conference on the media server and asks for the conference configurations 💜
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<audit queryid="conf:*" statelist="audit.conf.confconfig.*"/>
</msml>the result below assumes two conferences are allocated by the media server. Conference "conf:1" has both an audio mixer (with ASN enabled) and a video layout (vas) created, while conference "conf:2" only has an audio mixer created, with ASN disabled 🌸
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<result response="200">
<auditresult targetid="conf:1">
<confconfig deletewhen="nocontrol" term="true">
<audiomix id="audiomix1">
<asn ri="5s"/>
<n-loudest n="16"/>
</audiomix>
<videolayout id="videolayout1"
type="text/msml-basic-layout">
<selector id="selector1" method="vas" si="5s"
speakersees="current">
<root size="CIF"/>
</selector>
</videolayout>
<controller>conn:1234</controller>
</confconfig>
</auditresult>
<auditresult targetid="conf:2">
<confconfig deletewhen="nomedia" term="true">
<audiomix id="audiomix2">
<n-loudest n="1"/>
</audiomix>
<controller>conn:1234</controller>
</confconfig>
</auditresult>
</result>
</msml>14.2. Audit Conference Dialogs 🧚
this example audits the active dialogs on one specific conference. The request asks for every dialog state that's available 🍓
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<audit queryid="conf:1" statelist="audit.conf.dialog.*"/>
</msml>the example result assumes one dialog is running on conference "conf:1". It's been going for 60 seconds, and right now it's doing a record operation 🎙️
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<result response="200">
<auditresult targetid="conf:1">
<dialog name="sample">
<duration>60</duration>
<primitive>record</primitive>
<controller>conn:1234</controller>
</dialog>
</auditresult>
</result>
</msml>14.3. Audit Conference Streams 🦋
this request audits the active streams on one specific conference and asks for every stream state that's available 🫶
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<audit queryid="conf:1" statelist="audit.conf.stream.*"/>
</msml>the example result assumes three audio participants in the conference. Connection "conn:1234" is a talk-listen participant with both clamp and gain control enabled. Connection "conn:1235" is talk-only. Connection "conn:1236" is listen-only with automatic gain control enabled 🎧
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<result response="200">
<auditresult targetid="conf:1">
<stream joinwith="conn:1234" media="audio" dir="to">
<clamp dtmf="true" tone="false"/>
<gain amt="-10"/>
</stream>
<stream joinwith="conn:1234" media="audio" dir="from">
<gain amt="10"/>
</stream>
<stream joinwith="conn:1235" media="audio" dir="to">
</stream>
<stream joinwith="conn:1236" media="audio" dir="from">
<gain agc="true" tgtlvl="0" maxgain="10"/>
</stream>
</auditresult>
</result>
</msml>14.4. Audit All Connections 💅
this request audits every active connection on the media server. No extra state gets queried 🌈
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<audit queryid="conn:*"/>
</msml>the example result assumes five connections are currently allocated by the media server 💖
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<result response="200">
<auditresult targetid="conn:1230"/>
<auditresult targetid="conn:1231"/>
<auditresult targetid="conn:1232"/>
<auditresult targetid="conn:1233"/>
<auditresult targetid="conn:1234"/>
</result>
</msml>14.5. Audit Connection Dialogs 🌸
this request audits the active dialogs on one specific connection. No extra dialog state is queried ⭐
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<audit queryid="conn:1234" statelist="audit.conn.dialog"/>
</msml>the example result assumes three dialogs running on that connection 🧚
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<result response="200">
<auditresult targetid="conn:1234">
<dialog name="sample1"/>
<dialog name="sample2"/>
<dialog name="sample3"/>
</auditresult>
</result>
</msml>14.6. Audit Connection Streams 🫶
this request audits the active streams on one specific connection. No extra stream state is queried 💜
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<audit queryid="conn:1234" statelist="audit.conn.stream"/>
</msml>the example result assumes three audio streams between the target connection and other MSML objects. One is a bidirectional stream between the target connection and a conference, and the other two are unidirectional streams between two other connections 🦋
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<result response="200">
<auditresult targetid="conn:1234">
<stream joinwith="conf:1" media="audio" dir="to"/>
<stream joinwith="conf:1" media="audio" dir="from"/>
<stream joinwith="conn:1235" media="audio" dir="to"/>
<stream joinwith="conn:1236" media="audio" dir="from"/>
</auditresult>
</result>
</msml>14.7. Audit Connection with Selective States ✨
okay so this one is an audit of a single connection 🔍 bestie wants to see its SIP dialog ID and the SDP info, and you ask for just those pieces. super efficient, no extra baggage 💅
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<audit queryid="conn:1234" statelist="audit.conn.sipdialog
audit.conn.localsdp audit.conn.remotesdp"/>
</msml>
<?xml version="1.0" encoding="UTF-8"?>
<msml version="1.1">
<result response="200">
<auditresult targetid="conn:1234">
<sipdialog callid="ABCD@10.0.0.10:5060"
localtag="sdfjsiodf"
remotetag="zvnmviuhd8"/>
<localsdp>
v=0
o=- 31691 31691 IN IP4 ms5mpc11.lab.radisys.com
s=media server session
t=0 0
m=audio 33794 RTP/AVP 0
c=IN IP4 10.3.5.111
a=rtpmap:0 PCMU/8000
a=sendrecv
m=video 32770 RTP/AVP 34
c=IN IP4 10.3.5.11
b=AS:48
a=rtpmap:34 H263/90000
a=fmtp:34 CIF=1
a=sendrecv
</localsdp>
<remotesdp>
v=0
o=- 12345 12345 IN IP4 10.0.0.88
s=RadiSys SIP Media Server session
t=0 0
c=IN IP4 10.0.0.126
b=AS:128
m=audio 10000 RTP/AVP 0
a=rtpmap:0 PCMU/8000
a=ptime:20
a=sendrecv
m=video 10002 RTP/AVP 34
a=rtpmap:34 H263/90000
a=fmtp:34 CIF=1
a=sendrecv
</remotesdp>
</auditresult>
</result>
</msml>15. Future Work 🔮
okay so these are the things that might show up in future versions of this document, no promises yet 🧚 the big ones:
- Ability for MSML clients to audit or query the media server for the supported set of MSML packages and profiles 👀
- Ability to version MSML packages and profiles, plus a naming scheme for MSML extension packages 💜
16. XML Schema 🎀
the MSML spec is a whole set of XML schemas 🫶 you can use all of them together, or just a subset for each MSML package. the sections below define a complete set covering every MSML package, so it's giving full coverage 💖
each package has a single schema file, <package-name>-datatypes.xsd, which other extended packages can include. every package can also optionally have another schema file, <package_name>.xsd, that you can use directly to build or validate MSML scripts for that package ✨
the complete MSML schema, msml.xsd, pulls in all the individual MSML packages 🌈
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-core-datatypes.xsd"/>
<xs:include schemaLocation="msml-conf-core-datatypes.xsd"/>
<xs:include schemaLocation="msml-dialog-core-datatypes.xsd"/>
<xs:include schemaLocation="msml-dialog-base-datatypes.xsd"/>
<xs:include schemaLocation="msml-dialog-transform-datatypes.xsd"/>
<xs:include schemaLocation="msml-dialog-group-datatypes.xsd"/>
<xs:include schemaLocation="msml-dialog-speech-datatypes.xsd"/>
<xs:include schemaLocation="msml-dialog-fax-detect-datatypes.xsd"/>
<xs:include schemaLocation="msml-dialog-fax-sendrecv-
datatypes.xsd"/>
<xs:include schemaLocation="msml-audit-core-datatypes.xsd"/>
<xs:include schemaLocation="msml-audit-conf-datatypes.xsd"/>
<xs:include schemaLocation="msml-audit-conn-datatypes.xsd"/>
<xs:include schemaLocation="msml-audit-dialog-datatypes.xsd"/>
<xs:include schemaLocation="msml-audit-stream-datatypes.xsd"/>
<xs:element name="msml">
<xs:complexType>
<xs:choice>
<xs:group ref="msmlRequestType" maxOccurs="unbounded"/>
<xs:element name="event">
<xs:complexType>
<xs:choice maxOccurs="unbounded">
<xs:sequence>
<xs:element name="name" type="msmlEventNameValue.datatype"/>
<xs:element name="value">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:pattern value="[a-zA-Z0-9.]+"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
</xs:sequence>
</xs:choice>
<xs:attribute name="name" type="msmlEventName.datatype"
use="required"/>
<xs:attribute name="id" type="msmlEventSource.datatype"
use="required"/>
</xs:complexType>
</xs:element>
<xs:element name="result">
<xs:complexType>
<xs:choice>
<xs:element ref="description" minOccurs="0"/>
<xs:sequence>
<xs:element ref="msmlResultSimple" minOccurs="0"
maxOccurs="unbounded"/>
<xs:element ref="msmlResultComplex" minOccurs="0"
maxOccurs="unbounded"/>
</xs:sequence>
</xs:choice>
<xs:attribute name="response">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:pattern value="\d{3}"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="mark" type="mark.datatype"/>
</xs:complexType>
</xs:element>
</xs:choice>
<xs:attribute name="version" type="xs:string" use="required"
fixed="1.1"/>
</xs:complexType>
</xs:element>
</xs:schema>16.1. MSML Core 🧩
16.1.1. msml-core.xsd 💅
ok this one is the core package's own schema, and it's giving a lot of déjà vu because it's basically the same as the full one above, just only pulling in msml-core-datatypes.xsd 🫶
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified" attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-core-datatypes.xsd"/>
<xs:element name="msml">
<xs:complexType>
<xs:choice>
<xs:group ref="msmlRequestType" maxOccurs="unbounded"/>
<xs:element name="event">
<xs:complexType>
<xs:choice maxOccurs="unbounded">
<xs:sequence>
<xs:element name="name" type="msmlEventNameValue.datatype"/>
<xs:element name="value">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:pattern value="[a-zA-Z0-9.]+"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
</xs:sequence>
</xs:choice>
<xs:attribute name="name" type="msmlEventName.datatype"
use="required"/>
<xs:attribute name="id" type="msmlEventSource.datatype"
use="required"/>
</xs:complexType>
</xs:element>
<xs:element name="result">
<xs:complexType>
<xs:choice>
<xs:element ref="description" minOccurs="0"/>
<xs:sequence>
<xs:element ref="msmlResultSimple" minOccurs="0"
maxOccurs="unbounded"/>
<xs:element ref="msmlResultComplex" minOccurs="0"
maxOccurs="unbounded"/>
</xs:sequence>
</xs:choice>
<xs:attribute name="response">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:pattern value="\d{3}"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="mark" type="mark.datatype"/>
</xs:complexType>
</xs:element>
</xs:choice>
<xs:attribute name="version" type="xs:string" use="required"
fixed="1.1"/>
</xs:complexType>
</xs:element>
</xs:schema>16.1.2. msml-core-datatypes.xsd ✨
Okay bestie, this is the datatypes file for the core schema 💅 Every value here is copied exactly from the original.
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<xs:group name="msmlRequestType">
<xs:choice>
<xs:element ref="msmlRequest"/>
<xs:element name="send">
<xs:complexType>
<xs:complexContent>
<xs:extension base="msmlRequestType">
<xs:attribute name="event" type="msmlEvent.datatype"
use="required"/>
<xs:attribute name="target" type="msmlTarget.datatype"
use="required"/>
<xs:attribute name="valuelist" type="xs:string"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
</xs:choice>
</xs:group>
<xs:element name="msmlRequest" type="msmlRequestType"
abstract="true"/>
<xs:complexType name="msmlRequestType">
<xs:attribute ref="mark"/>
</xs:complexType>
<xs:element name="msmlResultSimple" type="msmlResultSimpleType"
abstract="true"/>
<xs:element name="msmlResultComplex" type="msmlResultComplexType"
abstract="true"/>
<xs:simpleType name="msmlResultSimpleType">
<xs:restriction base="xs:string"/>
</xs:simpleType>
<xs:complexType name="msmlResultComplexType"/>
<xs:element name="description" type="xs:string"/>
<xs:attribute name="mark" type="mark.datatype"/>
<xs:simpleType name="msmlInstanceID.datatype">
<xs:restriction base="xs:string">
<xs:pattern value="[a-zA-Z0-9.:\-_]+"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="connID.datatype">
<xs:restriction base="xs:string">
<xs:pattern value="conn:[a-zA-Z0-9.:\-_]+"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="confID.datatype">
<xs:restriction base="xs:string">
<xs:pattern value="conf:[a-zA-Z0-9.:\-_]+"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="dialogID.datatype">
<xs:restriction base="xs:string">
<xs:pattern value="conf:[a-zA-Z0-9.:\-_]+/dialog:[a-zA-Z0-9.:\-_]+"/>
<xs:pattern value="conn:[a-zA-Z0-9.:\-_]+/dialog:[a-zA-Z0-9.:\-_]+"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="independentID.datatype">
<xs:restriction base="xs:string">
<xs:pattern value="conf:[a-zA-Z0-9.:\-_]+"/>
<xs:pattern value="conn:[a-zA-Z0-9.:\-_]+"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="dialogLanguage.datatype">
<xs:restriction base="xs:string">
<xs:enumeration value="application/moml+xml"/>
<xs:enumeration value="application/voicexml+xml"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="msmlEvent.datatype">
<xs:restriction base="xs:string"/>
</xs:simpleType>
<xs:simpleType name="msmlSend.datatype">
<xs:restriction base="xs:string"/>
</xs:simpleType>
<xs:simpleType name="msmlEventName.datatype">
<xs:restriction base="xs:string">
<xs:pattern value="msml.dialog.exit"/>
<xs:pattern value="msml.conf.asn"/>
<xs:pattern value="msml.conf.nomedia"/>
<xs:pattern value="msml.dialog.exit"/>
<xs:pattern value="[a-zA-Z0-9.:_\-]+"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="msmlTarget.datatype">
<xs:restriction base="xs:string">
<xs:pattern
value="conf:[a-zA-Z0-9.:_\-]+(/oper:[a-zA-Z0-9.:_\-]+|\*)*"/>
<xs:pattern
value="conn:[a-zA-Z0-9.:_\-]+(/oper:[a-zA-Z0-9.:_\-]+|\*)+"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="msmlEventSource.datatype">
<xs:restriction base="xs:string">
<xs:pattern value="conf:[a-zA-Z0-9.:_\-]+"/>
<xs:pattern value="(conf:[a-zA-Z0-9.:_\-]+|conn:[a-zA-Z0-9.:_\-
]+)/dialog:[a-zA-Z0-9.:_\-]+"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="msmlEventNameValue.datatype">
<xs:restriction base="xs:string"/>
</xs:simpleType>
<xs:simpleType name="mark.datatype">
<xs:restriction base="xs:string">
<xs:pattern value="[a-zA-Z0-9.:\-_]+"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="boolean.datatype">
<xs:restriction base="xs:string">
<xs:enumeration value="true"/>
<xs:enumeration value="false"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="posDuration.datatype">
<xs:restriction base="xs:string">
<xs:pattern value="(\+)?([0-9]*\.)?[0-9]+(ms|s)"/>
</xs:restriction>
</xs:simpleType>
</xs:schema>16.2. MSML Conference Core Package 🎀
16.2.1. msml-conf-core.xsd 💖
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-core.xsd"/>
<xs:include schemaLocation="msml-conf-core-datatypes.xsd"/>
</xs:schema>16.2.2. msml-conf-core-datatypes.xsd 🦋
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-core-datatypes.xsd"/>
<xs:element name="createconference" substitutionGroup="msmlRequest">
<xs:complexType>
<xs:complexContent>
<xs:extension base="msmlRequestType">
<xs:all>
<xs:element name="audiomix" type="audioMixType" minOccurs="0"/>
<xs:element name="videolayout" type="videoLayoutType"
minOccurs="0"/>
<xs:element name="reserve" minOccurs="0">
<xs:complexType>
<xs:sequence>
<xs:element name="resource" maxOccurs="unbounded">
<xs:complexType>
<xs:sequence>
<xs:any namespace="##other" processContents="lax"
minOccurs="0" maxOccurs="unbounded"/>
</xs:sequence>
<xs:attribute name="n" type="xs:positiveInteger"
default="1"/>
<xs:anyAttribute namespace="##any"/>
</xs:complexType>
</xs:element>
</xs:sequence>
<xs:attribute name="required" type="boolean.datatype"
default="true"/>
</xs:complexType>
</xs:element>
</xs:all>
<xs:attribute name="name" type="msmlInstanceID.datatype"/>
<xs:attribute name="deletewhen" default="never">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="nomedia"/>
<xs:enumeration value="nocontrol"/>
<xs:enumeration value="never"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="term" type="boolean.datatype" default="true"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="modifyconference" substitutionGroup="msmlRequest">
<xs:complexType>
<xs:complexContent>
<xs:extension base="msmlRequestType">
<xs:all>
<xs:element name="audiomix" type="audioMixType" minOccurs="0"/>
<xs:element name="videolayout" type="videoLayoutType"
minOccurs="0"/>
</xs:all>
<xs:attribute name="id" type="confID.datatype" use="required"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="destroyconference" substitutionGroup="msmlRequest">
<xs:complexType>
<xs:complexContent>
<xs:extension base="msmlRequestType">
<xs:all>
<xs:element name="audiomix" type="basicAudioMixType"
minOccurs="0"/>
<xs:element name="videolayout" type="basicVideoLayoutType"
minOccurs="0"/>
</xs:all>
<xs:attribute name="id" type="confID.datatype" use="required"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="join" substitutionGroup="msmlRequest">
<xs:complexType>
<xs:complexContent>
<xs:extension base="msmlRequestType">
<xs:sequence>
<xs:element name="stream" type="streamType" minOccurs="0"
maxOccurs="4"/>
</xs:sequence>
<xs:attribute name="id1" type="independentID.datatype"
use="required"/>
<xs:attribute name="id2" type="independentID.datatype"
use="required"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="modifystream" substitutionGroup="msmlRequest">
<xs:complexType>
<xs:complexContent>
<xs:extension base="msmlRequestType">
<xs:sequence>
<xs:element name="stream" type="streamType" maxOccurs="4"/>
</xs:sequence>
<xs:attribute name="id1" type="independentID.datatype"
use="required"/>
<xs:attribute name="id2" type="independentID.datatype"
use="required"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="unjoin" substitutionGroup="msmlRequest">
<xs:complexType>
<xs:complexContent>
<xs:extension base="msmlRequestType">
<xs:sequence>
<xs:element name="stream" type="basicStreamType" minOccurs="0"
maxOccurs="4"/>
</xs:sequence>
<xs:attribute name="id1" type="independentID.datatype"
use="required"/>
<xs:attribute name="id2" type="independentID.datatype"
use="required"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="monitor" substitutionGroup="msmlRequest">
<xs:complexType>
<xs:complexContent>
<xs:extension base="msmlRequestType">
<xs:attribute name="id1" type="connID.datatype" use="required"/>
<xs:attribute name="id2" type="independentID.datatype"
use="required"/>
<xs:attribute name="compressed" type="boolean.datatype"
default="false"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="confid" type="msmlResultSimpleType"
substitutionGroup="msmlResultSimple"/>
<xs:complexType name="basicStreamType">
<xs:attribute name="dir">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="to-id1"/>
<xs:enumeration value="from-id1"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="media">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="audio"/>
<xs:enumeration value="video"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="compressed" type="boolean.datatype"/>
</xs:complexType>
<xs:complexType name="streamType">
<xs:complexContent>
<xs:extension base="basicStreamType">
<xs:choice minOccurs="0" maxOccurs="unbounded">
<xs:element name="gain">
<xs:complexType>
<xs:attribute name="amt" use="optional">
<xs:simpleType>
<xs:restriction base="xs:integer">
<xs:minInclusive value="-96"/>
<xs:maxInclusive value="96"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="agc" type="boolean.datatype"/>
<xs:attribute name="tgtlvl" use="optional">
<xs:simpleType>
<xs:restriction base="xs:nonPositiveInteger">
<xs:minInclusive value="-40"/>
<xs:maxInclusive value="0"/> </xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="maxgain" default="10">
<xs:simpleType>
<xs:restriction base="xs:nonNegativeInteger">
<xs:minInclusive value="0"/>
<xs:maxInclusive value="40"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
</xs:complexType>
</xs:element>
<xs:element name="clamp">
<xs:complexType>
<xs:attribute name="dtmf" type="boolean.datatype"/>
<xs:attribute name="tones" type="boolean.datatype"/>
</xs:complexType>
</xs:element>
<xs:element name="visual"/>
</xs:choice>
<xs:attribute name="preferred" type="boolean.datatype"
default="false"/>
<xs:attribute name="display" type="xs:string"/>
<xs:attribute name="override" type="boolean.datatype"
default="false"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
<xs:complexType name="basicAudioMixType">
<xs:attribute name="id" type="xs:string" use="optional"/>
<xs:attribute name="samplerate" type="xs:positiveInteger"
use="optional" default="8000"/>
</xs:complexType>
<xs:complexType name="audioMixType">
<xs:complexContent>
<xs:extension base="basicAudioMixType">
<xs:all>
<xs:element name="asn" minOccurs="0">
<xs:complexType>
<xs:attribute name="ri" type="posDuration.datatype"/>
<xs:attribute name="asth" default="-96">
<xs:simpleType>
<xs:restriction base="xs:nonPositiveInteger">
<xs:minInclusive value="-96"/>
<xs:maxInclusive value="0"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
</xs:complexType>
</xs:element>
<xs:element name="n-loudest" minOccurs="0">
<xs:complexType>
<xs:attribute name="n" type="xs:positiveInteger" use="required"/>
</xs:complexType>
</xs:element>
</xs:all>
</xs:extension>
</xs:complexContent>
</xs:complexType>
<xs:complexType name="basicVideoLayoutType">
<xs:attribute name="id" type="xs:string" use="required"/>
<xs:attribute name="type" type="xs:string" use="required"
fixed="text/msml-basic-layout"/>
</xs:complexType>
<xs:complexType name="videoLayoutType">
<xs:complexContent>
<xs:extension base="basicVideoLayoutType">
<xs:choice>
<xs:element name="selector">
<xs:complexType>
<xs:complexContent>
<xs:extension base="selectorType">
<xs:choice>
<xs:element name="root" type="rootType" minOccurs="0"/>
<xs:element name="region" minOccurs="0">
<xs:complexType>
<xs:attribute name="id" type="xs:string" use="required"/>
<xs:attribute name="left" type="xs:positiveInteger"/>
<xs:attribute name="top" type="xs:positiveInteger"/>
<xs:attribute name="relativeSize">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="1/4"/>
<xs:enumeration value="1/3"/>
<xs:enumeration value="2/3"/>
<xs:enumeration value="3/4"/>
<xs:enumeration value="1"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="priority">
<xs:simpleType>
<xs:restriction base="xs:float">
<xs:minInclusive value="0"/>
<xs:maxExclusive value="1"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="title" type="xs:string"/>
<xs:attribute name="titleTextColor" type="xs:string"/>
<xs:attribute name="titleBackgroundColor" type="xs:string"/>
<xs:attribute name="borderColor" type="xs:string"/>
<xs:attribute name="borderWidth" type="xs:positiveInteger"/>
<xs:attribute name="logo" type="xs:anyURI"/>
</xs:complexType>
</xs:element>
</xs:choice>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="root" type="rootType"/>
<xs:element name="region" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xs:complexContent>
<xs:extension base="regionType"/>
</xs:complexContent>
</xs:complexType>
</xs:element>
</xs:choice>
</xs:extension>
</xs:complexContent>
</xs:complexType>
<xs:complexType name="regionType">
<xs:attribute name="id" type="xs:string" use="required"/>
<xs:attribute name="left" type="xs:positiveInteger"/>
<xs:attribute name="top" type="xs:positiveInteger"/>
<xs:attribute name="relativeSize">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="1/4"/>
<xs:enumeration value="1/3"/>
<xs:enumeration value="2/3"/>
<xs:enumeration value="3/4"/>
<xs:enumeration value="1"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="priority">
<xs:simpleType>
<xs:restriction base="xs:float">
<xs:minInclusive value="0"/>
<xs:maxExclusive value="1"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="title" type="xs:string"/>
<xs:attribute name="titleTextColor" type="xs:string"/>
<xs:attribute name="titleBackgroundColor" type="xs:string"/>
<xs:attribute name="borderColor" type="xs:string"/>
<xs:attribute name="borderWidth" type="xs:positiveInteger"/>
<xs:attribute name="logo" type="xs:anyURI"/>
</xs:complexType>
<xs:complexType name="selectorType">
<xs:attribute name="id" type="xs:string" use="required"/>
<xs:attribute name="method" use="required">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="vas"/>
<xs:enumeration value="sequence"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="status" default="active">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="active"/>
<xs:enumeration value="disabled"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="si" type="posDuration.datatype" default="1s"/>
<xs:attribute name="blankothers" type="xs:boolean" default="false"/>
<xs:attribute name="speakersees" default="current">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="current"/>
<xs:enumeration value="previous"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
</xs:complexType>
<xs:complexType name="rootType">
<xs:attribute name="size" default="CIF">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="16CIF"/>
<xs:enumeration value="4CIF"/>
<xs:enumeration value="CIF"/>
<xs:enumeration value="QCIF"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="backgroundcolor" type="xs:string"
default="black"/>
<xs:attribute name="backgroundimage" type="xs:anyURI"/>
</xs:complexType>
<xs:simpleType name="confclass.datatype">
<xs:restriction base="xs:string">
<xs:enumeration value="standard"/>
<xs:enumeration value="preferred"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="conferenceType.datatype">
<xs:restriction base="xs:string">
<xs:enumeration value="audio.basic"/>
<xs:enumeration value="audio.advanced"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="duplex.datatype">
<xs:restriction base="xs:string">
<xs:enumeration value="half"/>
<xs:enumeration value="full"/>
</xs:restriction>
</xs:simpleType>
</xs:schema>16.3. MSML Dialog Packages ✨
okay the schema is giving 💅 and now we get the dialog packages, which are basically their own little family of schemas 🎀 they pull in the core stuff with xs:include, so the core bits just come along for the ride 🦋
16.3.1. msml-dialog-core.xsd 💖
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-core.xsd"/>
<xs:include schemaLocation="msml-dialog-core-datatypes.xsd"/>
</xs:schema>16.3.2. msml-dialog-core-datatypes.xsd 🎀
<?xml version="1.0" encoding="UTF-8"?>bestie, that's where this part stops 🫶
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-core-datatypes.xsd"/>
<xs:group name="momlRequest">
<xs:choice>
<xs:group ref="executeType"/>
<xs:group ref="sendType"/>
</xs:choice>
</xs:group>
<xs:element name="dialogstart" substitutionGroup="msmlRequest">
<xs:complexType>
<xs:complexContent>
<xs:extension base="msmlRequestType">
<xs:choice>
<xs:group ref="momlRequest" minOccurs="0"/>
</xs:choice>
<xs:attribute name="target" type="independentID.datatype"
use="required"/>
<xs:attribute name="type" type="dialogLanguage.datatype"
use="required"/>
<xs:attribute name="name" type="msmlInstanceID.datatype"/>
<xs:attribute name="src" type="xs:anyURI" use="optional"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="dialogend" substitutionGroup="msmlRequest">
<xs:complexType>
<xs:complexContent>
<xs:extension base="msmlRequestType">
<xs:attribute name="id" type="dialogID.datatype" use="required"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="dialogid" type="msmlResultSimpleType"
substitutionGroup="msmlResultSimple"/>
<xs:group name="executeType">
<xs:choice>
<xs:element ref="primitive" maxOccurs="unbounded"/>
<xs:element ref="control" maxOccurs="unbounded"/>
</xs:choice>
</xs:group>
<xs:element name="primitive" type="primitiveType" abstract="true"/>
<xs:complexType name="primitiveType">
<xs:attribute name="id" type="momlID.datatype"/>
</xs:complexType>
<xs:element name="control" abstract="true"/>
<xs:group name="sendType">
<xs:choice>
<xs:choice>
<xs:element name="exit" type="exitType"/>
<xs:element name="disconnect" type="exitType"/>
</xs:choice>
<xs:sequence>
<xs:element ref="send" maxOccurs="unbounded"/>
<xs:choice minOccurs="0">
<xs:element name="exit" type="exitType"/>
<xs:element name="disconnect" type="exitType"/>
</xs:choice>
</xs:sequence>
</xs:choice>
</xs:group>
<xs:element name="send">
<xs:complexType>
<xs:attribute name="event" type="momlEvent.datatype" use="required"/>
<xs:attribute name="target" type="momlTarget.datatype"
use="required"/>
<xs:attribute name="namelist" type="momlNamelist.datatype"/>
</xs:complexType>
</xs:element>
<xs:complexType name="exitType">
<xs:attribute name="namelist" type="momlNamelist.datatype"/>
</xs:complexType>
<xs:simpleType name="momlID.datatype">
<xs:restriction base="xs:string">
<xs:pattern value="[a-zA-Z0-9][a-zA-Z0-9._\-]*"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="momlEvent.datatype">
<xs:restriction base="xs:string">
<xs:pattern value="[a-zA-Z0-9][a-zA-Z0-9._\-]*"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="momlNamelist.datatype">
<xs:restriction base="xs:string"/>
</xs:simpleType>
<xs:simpleType name="dtmfDigits.datatype">
<xs:restriction base="xs:string">
<xs:pattern value="[0-9#*]+"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="iterate.datatype">
<xs:union memberTypes="xs:positiveInteger">
<xs:simpleType>
<xs:restriction base="xs:negativeInteger">
<xs:minInclusive value="-1"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="forever"/>
</xs:restriction>
</xs:simpleType>
</xs:union>
</xs:simpleType>
<xs:simpleType name="momlTarget.datatype">
<xs:restriction base="xs:string">
<xs:pattern value="[a-zA-Z0-9][a-zA-Z0-9._\-]*"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="duration.datatype">
<xs:restriction base="xs:string">
<xs:pattern value="(\+|\-)?([0-9]*\.)?[0-9]+(ms|s)"/>
</xs:restriction>
</xs:simpleType>
</xs:schema>16.3.3. msml-dialog-base.xsd ✨
Okay, next up is the base schema, and it's basically a little bundle of includes, bestie 🧚 Here it is:
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="unqualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-core.xsd"/>
<xs:include schemaLocation="msml-dialog-core-datatypes.xsd"/>
<xs:include schemaLocation="msml-dialog-base-datatypes.xsd"/>
</xs:schema>16.3.4. msml-dialog-base-datatypes.xsd 💜
Now the base datatypes, and this one is giving main character energy 💅 It has the play and record elements, so let's look:
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="unqualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-dialog-core-datatypes.xsd"/>
<xs:import namespace="http://www.w3.org/XML/1998/namespace"
schemaLocation="http://www.w3.org/2001/xml.xsd"/>
<xs:element name="play" substitutionGroup="primitive">
<xs:complexType>
<xs:complexContent>
<xs:extension base="primitiveType">
<xs:sequence>
<xs:choice maxOccurs="unbounded">
<xs:element name="audio" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xs:attribute name="uri" type="xs:anyURI" use="required"/>
<xs:attribute name="iterate" type="iterate.datatype"
default="1"/>
<xs:attribute name="format" type="xs:string" use="optional"/>
<xs:attribute name="audiosamplerate" type="xs:positiveInteger"
use="optional"/>
<xs:attribute name="audiosamplesize" type="xs:positiveInteger"
use="optional"/>
<xs:attribute ref="xml:lang"/>
</xs:complexType>
</xs:element>
<xs:element name="video" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xs:attribute name="uri" type="xs:anyURI" use="required"/>
<xs:attribute name="iterate" type="iterate.datatype"
use="optional" default="1"/>
<xs:attribute name="format" type="xs:string" use="optional"/>
<xs:attribute name="audiosamplerate" type="xs:positiveInteger"
use="optional"/>
<xs:attribute name="audiosamplesize" type="xs:positiveInteger"
use="optional"/>
<xs:attribute name="codecconfig" type="xs:string"
use="optional"/>
<xs:attribute name="profile" type="xs:string" use="optional"/>
<xs:attribute name="level" type="xs:string" use="optional"/>
<xs:attribute name="imagewidth" type="xs:positiveInteger"
use="optional"/>
<xs:attribute name="imageheight" type="xs:positiveInteger"
use="optional"/>
<xs:attribute name="maxbitrate" type="xs:positiveInteger"
use="optional"/>
<xs:attribute name="framerate" type="xs:positiveInteger"
use="optional"/>
</xs:complexType>
</xs:element>
<xs:element name="media" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xs:choice minOccurs="0" maxOccurs="unbounded">
<xs:element name="audio" minOccurs="0">
<xs:complexType>
<xs:attribute name="uri" type="xs:anyURI" use="required"/>
<xs:attribute name="iterate" type="iterate.datatype"
default="1"/>
<xs:attribute name="format" type="xs:string"
use="optional"/>
<xs:attribute name="audiosamplerate"
type="xs:positiveInteger" use="optional"/>
<xs:attribute name="audiosamplesize"
type="xs:positiveInteger" use="optional"/>
<xs:attribute ref="xml:lang"/>
</xs:complexType>
</xs:element>
<xs:element name="video" minOccurs="0">
<xs:complexType>
<xs:attribute name="uri" type="xs:anyURI" use="required"/>
<xs:attribute name="iterate" type="iterate.datatype"
use="optional" default="1"/>
<xs:attribute name="format" type="xs:string"
use="optional"/>
<xs:attribute name="audiosamplerate"
type="xs:positiveInteger" use="optional"/>
<xs:attribute name="audiosamplesize"
type="xs:positiveInteger" use="optional"/>
<xs:attribute name="codecconfig" type="xs:string"
use="optional"/>
<xs:attribute name="profile" type="xs:string"
use="optional"/>
<xs:attribute name="level" type="xs:string" use="optional"/>
<xs:attribute name="imagewidth" type="xs:positiveInteger"
use="optional"/>
<xs:attribute name="imageheight" type="xs:positiveInteger"
use="optional"/>
<xs:attribute name="maxbitrate" type="xs:positiveInteger"
use="optional"/>
<xs:attribute name="framerate" type="xs:positiveInteger"
use="optional"/>
</xs:complexType>
</xs:element>
</xs:choice>
</xs:complexType>
</xs:element>
<xs:element ref="smedia" minOccurs="0" maxOccurs="unbounded"/>
</xs:choice>
<xs:choice minOccurs="0">
<xs:element name="playexit">
<xs:complexType>
<xs:group ref="sendType"/>
</xs:complexType>
</xs:element>
</xs:choice>
</xs:sequence>
<xs:attribute name="interval" type="posDuration.datatype"
use="optional"/>
<xs:attribute name="iterate" type="iterate.datatype" use="optional"
default="1"/>
<xs:attribute name="offset" type="duration.datatype"
use="optional"/>
<xs:attribute name="initial" use="optional" default="generate">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="generate"/>
<xs:enumeration value="suspend"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="maxtime" type="posDuration.datatype"
use="optional"/>
<xs:attribute name="skip" type="duration.datatype" use="optional"
default="3s"/>
<xs:attribute name="barge" type="boolean.datatype" use="optional"
default="false"/>
<xs:attribute name="cleardb" type="boolean.datatype" use="optional"
default="false"/>
<xs:attribute ref="xml:lang"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="record" substitutionGroup="primitive">
<xs:complexType>
<xs:complexContent>
<xs:extension base="primitiveType">
<xs:choice minOccurs="0">
<xs:element ref="play" minOccurs="0" maxOccurs="unbounded"/>
<xs:element ref="tonegen" minOccurs="0" maxOccurs="unbounded"/>
<xs:element name="recordexit">
<xs:complexType>
<xs:group ref="sendType"/>
</xs:complexType>
</xs:element>
</xs:choice>
<xs:attribute name="append" type="boolean.datatype" use="optional"
default="false"/>
<xs:attribute name="dest" type="xs:anyURI" use="optional"/>
<xs:attribute name="audiodest" type="xs:anyURI" use="optional"/>
<xs:attribute name="videodest" type="xs:anyURI" use="optional"/>
<xs:attribute name="format" use="required">
<xs:simpleType>
<xs:restriction base="xs:string"/>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="codecconfig" use="optional">
<xs:simpleType>
<xs:restriction base="xs:string"/>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="audiosamplerate" type="xs:positiveInteger"
use="optional"/> <xs:attribute name="audiosamplesize" type="xs:positiveInteger"
use="optional"/>
<xs:attribute name="profile" use="optional">
<xs:simpleType>
<xs:restriction base="xs:string"/>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="level" use="optional">
<xs:simpleType>
<xs:restriction base="xs:string"/>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="imagewidth" type="xs:positiveInteger"
use="optional"/>
<xs:attribute name="imageheight" type="xs:positiveInteger"
use="optional"/>
<xs:attribute name="maxbitrate" type="xs:positiveInteger"
use="optional"/>
<xs:attribute name="framerate" type="xs:positiveInteger"
use="optional"/>
<xs:attribute name="maxtime" type="posDuration.datatype"
use="required"/>
<xs:attribute name="initial" use="optional" default="create">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="create"/>
<xs:enumeration value="suspend"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="prespeech" type="posDuration.datatype"
use="optional" default="0s"/>
<xs:attribute name="postspeech" type="posDuration.datatype"
use="optional" default="0s"/>
<xs:attribute name="termkey" use="optional">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:pattern value="[0-9#*ABCD]"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="dtmf" substitutionGroup="primitive">
<xs:complexType>
<xs:complexContent>
<xs:extension base="primitiveType">
<xs:sequence>
<xs:element name="pattern" maxOccurs="unbounded">
<xs:complexType>
<xs:group ref="sendType"/>
<xs:attribute name="digits" type="xs:string" use="required"/>
<xs:attribute name="format">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="mgcp"/>
<xs:enumeration value="megaco"/>
<xs:enumeration value="moml+digits"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="iterate" type="iterate.datatype"
default="1"/>
</xs:complexType>
</xs:element>
<xs:element name="detect" minOccurs="0">
<xs:complexType>
<xs:group ref="sendType"/>
</xs:complexType>
</xs:element>
<xs:element name="noinput" type="iterateSendType" minOccurs="0"/>
<xs:element name="nomatch" type="iterateSendType" minOccurs="0"/>
<xs:element name="dtmfexit" minOccurs="0">
<xs:complexType>
<xs:group ref="sendType"/>
</xs:complexType>
</xs:element>
<xs:element ref="play" minOccurs="0"/>
</xs:sequence>
<xs:attribute name="cleardb" type="boolean.datatype"
default="true"/>
<xs:attribute name="fdt" type="posDuration.datatype" default="0s"/>
<xs:attribute name="idt" type="posDuration.datatype" default="4s"/>
<xs:attribute name="edt" type="posDuration.datatype" default="4s"/>
<xs:attribute name="starttimer" type="boolean.datatype"
default="false"/>
<xs:attribute name="iterate" type="iterate.datatype" default="1"/>
<xs:attribute name="ldd" type="posDuration.datatype" default="0s"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="collect" substitutionGroup="primitive">
<xs:complexType>
<xs:complexContent>
<xs:extension base="primitiveType">
<xs:sequence>
<xs:element name="pattern" maxOccurs="unbounded">
<xs:complexType>
<xs:group ref="sendType"/>
<xs:attribute name="digits" type="xs:string" use="required"/>
<xs:attribute name="format">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="mgcp"/>
<xs:enumeration value="megaco"/>
<xs:enumeration value="moml+digits"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="iterate" type="iterate.datatype"
default="1"/>
</xs:complexType>
</xs:element>
<xs:element name="detect" minOccurs="0">
<xs:complexType>
<xs:group ref="sendType"/>
</xs:complexType>
</xs:element>
<xs:element name="noinput" type="iterateSendType" minOccurs="0"/>
<xs:element name="nomatch" type="iterateSendType" minOccurs="0"/>
<xs:element name="dtmfexit" minOccurs="0">
<xs:complexType>
<xs:group ref="sendType"/>
</xs:complexType>
</xs:element>
<xs:element ref="play" minOccurs="0"/>
</xs:sequence>
<xs:attribute name="cleardb" type="boolean.datatype"
default="true"/>
<xs:attribute name="fdt" type="posDuration.datatype" default="0s"/>
<xs:attribute name="idt" type="posDuration.datatype" default="4s"/>
<xs:attribute name="edt" type="posDuration.datatype" default="4s"/>
<xs:attribute name="starttimer" type="boolean.datatype"
default="false"/>
<xs:attribute name="iterate" type="iterate.datatype" default="1"/>
<xs:attribute name="ldd" type="posDuration.datatype"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="dtmfgen" substitutionGroup="primitive">
<xs:complexType>
<xs:complexContent>
<xs:extension base="primitiveType">
<xs:choice minOccurs="0">
<xs:element name="dtmfgenexit">
<xs:complexType>
<xs:group ref="sendType"/>
</xs:complexType>
</xs:element>
</xs:choice>
<xs:attribute name="level" use="optional" default="-6">
<xs:simpleType>
<xs:restriction base="xs:nonPositiveInteger">
<xs:maxInclusive value="0"/>
<xs:minInclusive value="-96"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="digits" type="dtmfDigits.datatype"
use="required"/>
<xs:attribute name="dur" type="posDuration.datatype" use="optional"
default="100ms"/>
<xs:attribute name="interval" type="posDuration.datatype"
use="optional" default="100ms"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="tonegen" substitutionGroup="primitive">
<xs:complexType>
<xs:complexContent>
<xs:extension base="primitiveType">
<xs:choice minOccurs="0">
<xs:element name="tonegenexit" minOccurs="0">
<xs:complexType>
<xs:group ref="sendType"/>
</xs:complexType>
</xs:element>
<xs:element name="tone" maxOccurs="unbounded">
<xs:complexType>
<xs:sequence>
<xs:element name="tone1">
<xs:complexType>
<xs:attribute name="freq" use="required">
<xs:simpleType>
<xs:restriction base="xs:unsignedInt">
<xs:minInclusive value="0"/>
<xs:maxInclusive value="3999"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="atten" use="required">
<xs:simpleType>
<xs:restriction base="xs:nonPositiveInteger">
<xs:minInclusive value="-96"/>
<xs:maxInclusive value="0"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
</xs:complexType>
</xs:element>
<xs:element name="tone2">
<xs:complexType>
<xs:attribute name="freq" use="required">
<xs:simpleType>
<xs:restriction base="xs:unsignedInt">
<xs:minInclusive value="0"/>
<xs:maxInclusive value="3999"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="atten" use="required">
<xs:simpleType>
<xs:restriction base="xs:nonPositiveInteger">
<xs:minInclusive value="-96"/>
<xs:maxInclusive value="0"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
</xs:complexType>
</xs:element>
<xs:element name="silence" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xs:attribute name="duration" type="duration.datatype"
use="required"/>
</xs:complexType>
</xs:element>
</xs:sequence>
<xs:attribute name="duration" use="required">
<xs:simpleType>
<xs:restriction base="duration.datatype"/>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="iterate" type="iterate.datatype"
use="optional" default="1"/>
</xs:complexType>
</xs:element>
<xs:element name="silence" minOccurs="0" maxOccurs="unbounded">
<xs:complexType>
<xs:attribute name="duration" type="duration.datatype"
use="required"/>
</xs:complexType>
</xs:element>
</xs:choice>
<xs:attribute name="iterate" type="iterate.datatype" use="optional"
default="1"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:complexType name="iterateSendType">
<xs:group ref="sendType"/>
<xs:attribute name="iterate" type="iterate.datatype" default="1"/>
</xs:complexType>
<xs:element name="smedia" type="smediaType" abstract="true"/>
<xs:complexType name="smediaType">
<xs:attribute ref="xml:lang"/>
<xs:attribute name="iterate" type="iterate.datatype"/>
</xs:complexType>
<xs:element name="var" substitutionGroup="smedia">
<xs:complexType>
<xs:complexContent>
<xs:extension base="smediaType">
<xs:attribute name="type" use="required">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="date"/>
<xs:enumeration value="digits"/>
<xs:enumeration value="duration"/>
<xs:enumeration value="month"/>
<xs:enumeration value="money"/>
<xs:enumeration value="number"/>
<xs:enumeration value="silence"/>
<xs:enumeration value="time"/>
<xs:enumeration value="weekday"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="subtype" type="xs:string" use="optional"/>
<xs:attribute name="value" type="xs:string" use="required"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
</xs:schema>16.3.5. msml-dialog-transform.xsd 🎀
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="unqualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-core.xsd"/>
<xs:include schemaLocation="msml-dialog-core-datatypes.xsd"/>
<xs:include schemaLocation="msml-dialog-transform-datatypes.xsd"/>
</xs:schema>16.3.6. msml-dialog-transform-datatypes.xsd 🧚
<?xml version="1.0" encoding="UTF-8"?><xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="unqualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-dialog-core-datatypes.xsd"/>
<xs:import namespace="http://www.w3.org/XML/1998/namespace"
schemaLocation="http://www.w3.org/2001/xml.xsd"/>
<xs:element name="vad" substitutionGroup="primitive">
<xs:complexType>
<xs:complexContent>
<xs:extension base="primitiveType">
<xs:all>
<xs:element name="voice" type="vadPatternType" minOccurs="0"/>
<xs:element name="silence" type="vadPatternType" minOccurs="0"/>
<xs:element name="tvoice" type="vadPatternType" minOccurs="0"/>
<xs:element name="tsilence" type="vadPatternType" minOccurs="0"/>
</xs:all>
<xs:attribute name="starttimer" type="boolean.datatype"
default="false"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="gain" substitutionGroup="primitive">
<xs:complexType>
<xs:complexContent>
<xs:extension base="primitiveType">
<xs:attribute name="incr" default="3">
<xs:simpleType>
<xs:restriction base="xs:positiveInteger">
<xs:maxInclusive value="96"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="amt" use="required">
<xs:simpleType>
<xs:restriction base="xs:integer">
<xs:minInclusive value="-96"/>
<xs:maxInclusive value="96"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="agc" substitutionGroup="primitive">
<xs:complexType>
<xs:complexContent>
<xs:extension base="primitiveType">
<xs:attribute name="tgtlvl" use="required">
<xs:simpleType>
<xs:restriction base="xs:nonPositiveInteger">
<xs:minInclusive value="-40"/>
<xs:maxInclusive value="0"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="maxgain" default="10">
<xs:simpleType>
<xs:restriction base="xs:nonNegativeInteger">
<xs:minInclusive value="0"/>
<xs:maxInclusive value="40"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="gate" substitutionGroup="primitive">
<xs:complexType>
<xs:complexContent>
<xs:extension base="primitiveType">
<xs:attribute name="initial" default="pass">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="pass"/>
<xs:enumeration value="halt"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="clamp" substitutionGroup="primitive">
<xs:complexType>
<xs:complexContent>
<xs:extension base="primitiveType"/>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="relay" substitutionGroup="primitive">
<xs:complexType>
<xs:complexContent>
<xs:extension base="primitiveType"/>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:complexType name="vadPatternType">
<xs:group ref="sendType"/>
<xs:attribute name="iterate" type="iterate.datatype" default="1"/>
<xs:attribute name="len" type="posDuration.datatype" use="required"/>
<xs:attribute name="sen" type="posDuration.datatype" use="optional"/>
</xs:complexType>
</xs:schema>16.3.7. msml-dialog-group.xsd ✨
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="unqualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-core.xsd"/>
<xs:include schemaLocation="msml-dialog-core-datatypes.xsd"/>
<xs:include schemaLocation="msml-dialog-base-datatypes.xsd"/>
<xs:include schemaLocation="msml-dialog-group-datatypes.xsd"/>
</xs:schema>16.3.8. msml-dialog-group-datatypes.xsd 💅
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="unqualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-core-datatypes.xsd"/>
<xs:include schemaLocation="msml-dialog-core-datatypes.xsd"/>
<xs:include schemaLocation="msml-dialog-base-datatypes.xsd"/>
<xs:include schemaLocation="msml-dialog-transform-datatypes.xsd"/>
<xs:element name="group" substitutionGroup="control">
<xs:complexType>
<xs:sequence>
<xs:group ref="executeType"/>
<xs:element name="groupexit" minOccurs="0">
<xs:complexType>
<xs:group ref="sendType"/>
</xs:complexType>
</xs:element>
</xs:sequence>
<xs:attribute name="id" type="momlID.datatype"/>
<xs:attribute name="topology" use="required">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="serial"/>
<xs:enumeration value="parallel"/>
<xs:enumeration value="fullduplex"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
</xs:complexType>
</xs:element>
</xs:schema>16.3.9. msml-dialog-speech.xsd 🎀
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-core.xsd"/>
<xs:include schemaLocation="msml-dialog-core-datatypes.xsd"/>
<xs:include schemaLocation="msml-dialog-base-datatypes.xsd"/>
<xs:include schemaLocation="msml-dialog-speech-datatypes.xsd"/>
</xs:schema>16.3.10. msml-dialog-speech-datatypes.xsd 🦋
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-dialog-core-datatypes.xsd"/>
<xs:include schemaLocation="msml-dialog-base-datatypes.xsd"/>
<xs:include schemaLocation="http://www.w3.org/TR/2002/WD-speech-
synthesis-20020405/synthesis-core.xsd"/>
<xs:include schemaLocation="http://www.w3.org/TR/speech-
grammar/grammar-core.xsd"/>
<xs:element name="speech" substitutionGroup="primitive">
<xs:complexType>
<xs:complexContent>
<xs:extension base="primitiveType">
<xs:sequence>
<xs:element name="grammar" maxOccurs="unbounded">
<xs:complexType>
<xs:complexContent>
<xs:extension base="grammar">
<xs:choice>
<xs:element name="match" type="iterateSendType"
minOccurs="0"/>
</xs:choice>
<xs:attribute name="uri" type="xs:anyURI"/>
<xs:attribute name="iterate" type="iterate.datatype"
default="1"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="noinput" type="iterateSendType" minOccurs="0"/>
<xs:element name="nomatch" type="iterateSendType" minOccurs="0"/>
<xs:element name="speechexit" minOccurs="0">
<xs:complexType>
<xs:group ref="sendType"/>
</xs:complexType>
</xs:element>
</xs:sequence>
<xs:attribute name="noint" type="posDuration.datatype"/>
<xs:attribute name="norect" type="posDuration.datatype"/>
<xs:attribute name="spcmplt" type="posDuration.datatype"/>
<xs:attribute name="confidence">
<xs:simpleType>
<xs:restriction base="xs:positiveInteger">
<xs:maxInclusive value="100"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="sens" type="xs:positiveInteger"/>
<xs:attribute name="starttimer" type="boolean.datatype"
default="false"/>
<xs:attribute name="iterate" type="iterate.datatype" default="1"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="tts" type="smediaType" substitutionGroup="smedia"/>
</xs:schema>16.3.11. msml-dialog-fax-detect.xsd 🧚
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-core.xsd"/>
<xs:include schemaLocation="msml-dialog-core-datatypes.xsd"/>
<xs:include schemaLocation="msml-dialog-fax-detect-datatypes.xsd"/>
</xs:schema>16.3.12. msml-dialog-fax-detect-datatypes.xsd 🌸
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-dialog-core-datatypes.xsd"/>
<xs:element name="faxdetect" substitutionGroup="primitive">
<xs:complexType>
<xs:complexContent>
<xs:extension base="primitiveType">
<xs:choice minOccurs="0">
<xs:element name="faxdetectexit">
<xs:complexType>
<xs:group ref="sendType"/>
</xs:complexType>
</xs:element>
</xs:choice>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
</xs:schema>16.3.13. msml-dialog-fax-sendrecv.xsd 🍓
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-core.xsd"/>
<xs:include schemaLocation="msml-dialog-core-datatypes.xsd"/>
<xs:include schemaLocation="msml-dialog-fax-sendrecv-datatypes.xsd"/>
</xs:schema>16.3.14. msml-dialog-fax-sendrecv-datatypes.xsd 🫶
<?xml version="1.0" encoding="UTF-8"?> <xs:include schemaLocation="msml-dialog-core-datatypes.xsd"/>
<xs:element name="faxsend" substitutionGroup="primitive">
<xs:complexType>
<xs:complexContent>
<xs:extension base="primitiveType">
<xs:sequence>
<xs:element name="sendobj" type="sendobjType" minOccurs="0"
maxOccurs="unbounded"/>
<xs:element name="hdrfooter" type="hdrfooterType" minOccurs="0"/>
<xs:element name="rxpoll" minOccurs="0">
<xs:complexType>
<xs:sequence>
<xs:element name="rcvobj" type="rcvobjType"
maxOccurs="unbounded"/>
<xs:element name="hdrfooter" type="hdrfooterType"
minOccurs="0"/>
</xs:sequence>
<xs:attribute name="rmtid" type="faxid.datatype"
use="required"/>
</xs:complexType>
</xs:element>
<xs:group ref="faxstatusrequest"/>
</xs:sequence>
<xs:attribute name="lclid" type="faxid.datatype" use="optional"/>
<xs:attribute name="minspeed" type="faxspeed.datatype"
use="optional"/>
<xs:attribute name="maxspeed" type="faxspeed.datatype"
use="optional"/>
<xs:attribute name="ecm" type="boolean.datatype" use="optional"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="faxrecv" substitutionGroup="primitive">
<xs:complexType>
<xs:complexContent>
<xs:extension base="primitiveType">
<xs:sequence>
<xs:element name="rcvobj" type="rcvobjType" minOccurs="0"
maxOccurs="unbounded"/>
<xs:element name="hdrfooter" type="hdrfooterType" minOccurs="0"/>
<xs:element name="txpoll" minOccurs="0">
<xs:complexType>
<xs:sequence>
<xs:element name="sendobj" type="sendobjType"
maxOccurs="unbounded"/>
<xs:element name="hdrfooter" type="hdrfooterType"
minOccurs="0"/>
</xs:sequence>
<xs:attribute name="rmtid" type="faxid.datatype"/>
</xs:complexType>
</xs:element>
<xs:group ref="faxstatusrequest"/>
</xs:sequence>
<xs:attribute name="lclid" type="faxid.datatype" use="optional"/>
<xs:attribute name="ecm" type="boolean.datatype" default="true"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:group name="faxstatusrequest">
<xs:sequence>
<xs:element name="faxstart" minOccurs="0"/>
<xs:element name="faxnegotiate" minOccurs="0"/>
<xs:element name="faxpagedone" minOccurs="0"/>
<xs:element name="faxobjectdone" minOccurs="0"/>
<xs:element name="faxopcomplete" minOccurs="0"/>
<xs:element name="faxpollstart" minOccurs="0"/>
</xs:sequence>
</xs:group>
<xs:complexType name="hdrfooterType">
<xs:choice>
<xs:element name="format" type="xs:string" minOccurs="0"
maxOccurs="unbounded"/>
</xs:choice>
<xs:attribute name="type" type="hdrfooter.datatype"/>
<xs:attribute name="style" type="hdrfooterstyle.datatype"/>
</xs:complexType>
<xs:complexType name="formatType">
<xs:simpleContent>
<xs:extension base="xs:string">
<xs:attribute name="style">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="append"/>
<xs:enumeration value="overlay"/>
<xs:enumeration value="replace"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
</xs:extension>
</xs:simpleContent>
</xs:complexType>
<xs:complexType name="rcvobjType">
<xs:attribute name="objuri" type="xs:anyURI" use="required"/>
<xs:attribute name="maxpages" type="xs:positiveInteger"/>
</xs:complexType>
<xs:complexType name="sendobjType">
<xs:attribute name="objuri" type="xs:anyURI" use="required"/>
<xs:attribute name="startpage" type="xs:positiveInteger"/>
<xs:attribute name="pagecount" type="xs:positiveInteger"/>
</xs:complexType>
<xs:simpleType name="faxid.datatype">
<xs:restriction base="xs:string">
<xs:pattern value="[0-9+*- ]{20}"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="faxspeed.datatype">
<xs:restriction base="xs:string">
<xs:enumeration value="2400"/>
<xs:enumeration value="4800"/>
<xs:enumeration value="7200"/>
<xs:enumeration value="9600"/>
<xs:enumeration value="12000"/>
<xs:enumeration value="14400"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="hdrfooter.datatype">
<xs:restriction base="xs:string">
<xs:enumeration value="header"/>
<xs:enumeration value="footer"/>
<xs:enumeration value="autohdr"/>
<xs:enumeration value="nohdr"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="hdrfooterstyle.datatype">
<xs:restriction base="xs:string">
<xs:enumeration value="append"/>
<xs:enumeration value="overlay"/>
<xs:enumeration value="replace"/>
</xs:restriction>
</xs:simpleType>
</xs:schema>16.4. MSML Audit Packages ✨
okay, the audit packages are up next and they're giving main character energy 💅🦋
16.4.1. msml-audit-core.xsd
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-core.xsd"/>
<xs:include schemaLocation="msml-core-datatypes.xsd"/>
<xs:include schemaLocation="msml-audit-core-datatypes.xsd"/>
</xs:schema>16.4.2. msml-audit-core-datatypes.xsd
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-core-datatypes.xsd"/>
<xs:element name="audit" substitutionGroup="msmlRequest">
<xs:complexType>
<xs:complexContent>
<xs:extension base="msmlRequestType">
<xs:attribute name="queryid" type="auditQueryId.datatype"
use="required"/>
<xs:attribute name="statelist" type="auditStateList.datatype"
use="optional"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="auditresult" substitutionGroup="msmlResultComplex">
<xs:complexType>
<xs:complexContent>
<xs:extension base="msmlResultComplexType">
<xs:choice maxOccurs="unbounded">
<xs:element ref="stateParameter"/>
<xs:element ref="stateParameterSimple"/>
</xs:choice>
<xs:attribute name="targetid" type="independentID.datatype"
use="required"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="stateParameter" type="stateParameterType"
abstract="true"/>
<xs:element name="stateParameterSimple" type="stateParameterSimpleType"
abstract="true"/>
<xs:complexType name="stateParameterType"/>
<xs:simpleType name="stateParameterSimpleType">
<xs:restriction base="xs:string"/>
</xs:simpleType>
<xs:simpleType name="auditQueryId.datatype">
<xs:restriction base="xs:string">
<xs:pattern value="conf:[a-zA-Z0-9.:\-_]+"/>
<xs:pattern value="conn:[a-zA-Z0-9.:\-_]+"/>
<xs:pattern value="conf:\*"/>
<xs:pattern value="conn:\*"/>
</xs:restriction>
</xs:simpleType>
<xs:simpleType name="auditStateList.datatype">
<xs:restriction base="xs:string"/>
</xs:simpleType>
</xs:schema>16.4.3. msml-audit-conf.xsd
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-core.xsd"/>
<xs:include schemaLocation="msml-core-datatypes.xsd"/>
<xs:include schemaLocation="msml-audit-core-datatypes.xsd"/>
<xs:include schemaLocation="msml-audit-dialog-datatypes.xsd"/>
<xs:include schemaLocation="msml-audit-stream-datatypes.xsd"/>
<xs:include schemaLocation="msml-audit-conf-datatypes.xsd"/>
</xs:schema>16.4.4. msml-audit-conf-datatypes.xsd
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-conf-core-datatypes.xsd"/>
<xs:include schemaLocation="msml-audit-core-datatypes.xsd"/>
<xs:element name="confconfig" substitutionGroup="stateParameter">
<xs:complexType>
<xs:complexContent>
<xs:extension base="stateParameterType">
<xs:sequence>
<xs:element name="audiomix" type="audioMixType" minOccurs="0"
maxOccurs="unbounded"/>
<xs:element name="videolayout" type="videoLayoutType"
minOccurs="0" maxOccurs="unbounded"/>
<xs:element name="controller" type="connID.datatype"
minOccurs="0"/>
</xs:sequence>
<xs:attribute name="deletewhen" use="optional" default="never">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="nomedia"/>
<xs:enumeration value="nocontrol"/>
<xs:enumeration value="never"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="term" type="boolean.datatype" use="optional"
default="true"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
</xs:schema>16.4.5. msml-audit-conn.xsd
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-core.xsd"/>
<xs:include schemaLocation="msml-audit-core-datatypes.xsd"/>
<xs:include schemaLocation="msml-audit-dialog-datatypes.xsd"/>
<xs:include schemaLocation="msml-audit-stream-datatypes.xsd"/>
<xs:include schemaLocation="msml-audit-conn-datatypes.xsd"/>
</xs:schema>16.4.6. msml-audit-conn-datatypes.xsd
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-audit-core-datatypes.xsd"/>
<xs:element name="sipdialog" substitutionGroup="stateParameter">
<xs:complexType>
<xs:complexContent>
<xs:extension base="stateParameterType">
<xs:sequence>
<xs:element name="localseq" type="xs:integer" minOccurs="0"/>
<xs:element name="remoteseq" type="xs:int" minOccurs="0"/>
<xs:element name="localuri" type="xs:string" minOccurs="0"/>
<xs:element name="remoteuri" type="xs:string" minOccurs="0"/>
<xs:element name="remotetarget" type="xs:string" minOccurs="0"/>
<xs:element name="routeset" type="xs:string" minOccurs="0"/>
</xs:sequence>
<xs:attribute name="callid" type="xs:string" use="required"/>
<xs:attribute name="localtag" type="xs:string" use="required"/>
<xs:attribute name="remotetag" type="xs:string" use="required"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
<xs:element name="localsdp" type="stateParameterSimpleType"
substitutionGroup="stateParameterSimple"/>
<xs:element name="remotesdp" type="stateParameterSimpleType"
substitutionGroup="stateParameterSimple"/>
</xs:schema>16.4.7. msml-audit-dialog-datatypes.xsd ✨
okay so Audit Dialog stuff needs either the Audit Conf Package or the Audit Conn Package, bestie 💖
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-audit-core-datatypes.xsd"/>
<xs:element name="dialog" substitutionGroup="stateParameter">
<xs:complexType>
<xs:complexContent>
<xs:extension base="stateParameterType">
<xs:sequence>
<xs:element name="duration" type="xs:positiveInteger"
minOccurs="0"/>
<xs:element name="primitive" minOccurs="0">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:pattern value="play"/>
<xs:pattern value="dtmf"/>
<xs:pattern value="collect"/>
<xs:pattern value="dtmfgen"/>
<xs:pattern value="tonegen"/>
<xs:pattern value="record"/>
<xs:pattern value="none"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
<xs:element name="controller" type="connID.datatype"
minOccurs="0"/>
</xs:sequence>
<xs:attribute name="name" type="msmlInstanceID.datatype"
use="required"/>
<xs:attribute name="src" type="xs:anyURI" use="optional"/>
<xs:attribute name="type" type="dialogLanguage.datatype"
use="required"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
</xs:schema>16.4.8. msml-audit-stream-datatypes.xsd 🎀
Audit Stream functionality needs either the Audit Conf Package or the Audit Conn Package too, so same energy as the dialog schema 🫶
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
elementFormDefault="qualified"
attributeFormDefault="unqualified">
<xs:include schemaLocation="msml-audit-core-datatypes.xsd"/>
<xs:element name="stream" substitutionGroup="stateParameter">
<xs:complexType>
<xs:complexContent>
<xs:extension base="stateParameterType">
<xs:all>
<xs:element name="clamp" minOccurs="0">
<xs:complexType>
<xs:attribute name="dtmf" type="boolean.datatype"/>
<xs:attribute name="tones" type="boolean.datatype"/>
</xs:complexType>
</xs:element>
<xs:element name="gain" minOccurs="0">
<xs:complexType>
<xs:attribute name="amt" use="optional">
<xs:simpleType>
<xs:restriction base="xs:integer">
<xs:minInclusive value="-96"/>
<xs:maxInclusive value="96"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="agc" type="boolean.datatype"/>
<xs:attribute name="tgtlvl" use="optional">
<xs:simpleType>
<xs:restriction base="xs:nonPositiveInteger">
<xs:minInclusive value="-40"/>
<xs:maxInclusive value="0"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="maxgain" default="10">
<xs:simpleType>
<xs:restriction base="xs:nonNegativeInteger">
<xs:minInclusive value="0"/>
<xs:maxInclusive value="40"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
</xs:complexType>
</xs:element>
<xs:element name="visual" minOccurs="0"/>
</xs:all>
<xs:attribute name="joinwith" type="independentID.datatype"
use="required"/>
<xs:attribute name="media" use="required">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:pattern value="audio"/>
<xs:pattern value="video"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="dir" use="required">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:pattern value="from"/>
<xs:pattern value="to"/>
</xs:restriction>
</xs:simpleType>
</xs:attribute>
<xs:attribute name="compressed" type="boolean.datatype"/>
<xs:attribute name="preferred" type="boolean.datatype"
default="false"/>
<xs:attribute name="display" type="xs:string"/>
<xs:attribute name="override" type="boolean.datatype"
default="false"/>
</xs:extension>
</xs:complexContent>
</xs:complexType>
</xs:element>
</xs:schema>17. Security Considerations 🔐
bestie, this part is serious, so listen up 👑 MSML is an XML-based language, which means the security considerations from RFC 3023 [i2] apply to it. 😭
Media server interfaces that MSML drives are under the explicit control of a SIP application server. SIP call legs are what carry the XML-based MSML transactions over to the media server. For the security and integrity of MSML transactions, whenever they are needed, you SHOULD use sips: and TLS to encrypt and authenticate the SIP control channel that carries the MSML payloads. More info on the security, privacy, and integrity of the MSML media types lives in the IANA Considerations section 🦋
Media streams, like audio/video, MAY optionally be protected, encrypted/decrypted, and authenticated with Secure Real Time Protocol (SRTP), wherever media stream security is needed. Media negotiation sets the required level of security and is started by the clients. That part is outside the scope of the control interface MSML specifies, so it's not MSML's job 💅
18. IANA Considerations 🌈
18.1. IANA Registrations for 'application' MIME Media Type 💜
The following registrations have been made:
Type Name: "application"
Subtype names:
'application/vnd.radisys.msml+xml',
'application/vnd.radisys.moml+xml',
'application/vnd.radisys.msml-conf+xml',
'application/vnd.radisys.msml-dialog+xml',
'application/vnd.radisys.msml-dialog-base+xml',
'application/vnd.radisys.msml-dialog-group+xml',
'application/vnd.radisys.msml-dialog-speech+xml',
'application/vnd.radisys.msml-dialog-transform+xml',
'application/vnd.radisys.msml-dialog-fax-detect+xml',
'application/vnd.radisys.msml-dialog-fax-sendrecv+xml',
'application/vnd.radisys.msml-audit+xml',
'application/vnd.radisys.msml-audit-conf+xml',
'application/vnd.radisys.msml-audit-conn+xml',
'application/vnd.radisys.msml-audit-dialog+xml',
'application/vnd.radisys.msml-audit-stream+xml'Required parameters: none
Optional parameters: charset
charset semantics as specified in RFC 3023 [i2] for "application/xml" media type.
Encoding considerations:
As specified in RFC 3023 [i2].
Security Considerations:
Media types included in this section are XML based, so security considerations as defined by RFC 3023 [i10] apply to them. 🙈
These media types don't have any active or executable content, because the content itself just controls the underlying media streams 💖
Secure exchange of content tied to these media types, for authentication and privacy whenever that applies, shall require setting up a secure control channel with sips: and TLS.
Privacy and integrity of media content tied to these media types need thinking about when apps using them swap personal info, like personal identification codes or conference access codes. Whenever that content needs secure transport and authentication, a secure channel using sips: and TLS MUST be used, because these media types don't provide any built-in security mechanisms of their own. This part is non-negotiable, bestie 🚨
Interoperability considerations:
As specified in RFC 3023 [i2] and as specified within this document.
Published specification: RFC 5707
Intended applications for these media types:
Multimedia Conferencing, Interactive Voice Response systems
Additional information:
Magic number(s): None
File extension(s): None
Macintosh file type code(s): None
Person & email address to contact for further information:
Adnan Saleem <adnan.saleem@radisys.com>
Intended usage: COMMON
18.2. IANA Registrations for 'text' MIME Media Type 🍓
The following registrations are planned:
text/vnd.radisys.msml-basic-layout
Required parameters: none
Optional parameters: charset
charset semantics as specified in RFC 3023 [i2] for "text/xml" media type.
Encoding considerations: As specified in RFC 3023 [i2].
Security Considerations:
Media types included in this section are XML based, so security considerations as defined by RFC 3023 [i10] apply to them. 🙈
The media type defined in this section doesn't have any active or executable content. It only defines a visual layout scheme for a video conference. Setting up the active connections tied to the video conference is outside the scope of this media type 🎀
Since this media type only defines a visual layout scheme, with no reference or information about client connections or participants within the conference, privacy and integrity concerns don't apply to it ✨
Interoperability considerations:
As specified in RFC 3023 [i2] and as specified within this document.
Published specification: RFC 5707
Intended applications for these media types:
Multimedia Conferencing, Interactive Voice Response systems
Additional information:
Magic number(s): None
File extension(s): None
Macintosh file type code(s): None
Person & email address to contact for further information:
Adnan Saleem <adnan.saleem@radisys.com>
Intended usage: COMMON
18.3. URN Sub-Namespace Registration 🧚
The namespace URI for elements defined in this specification is a URN [i8]. It uses the namespace identifier 'ietf' defined by [i9] and extended by RFC 3688 [i10].
The following registrations of URN Sub-Namespaces are planned:
XML namespace: urn:ietf:params:xml:ns:msml
XML:
BEGIN
<?xml version="1.0"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML Basic 1.0//EN"
"http://www.w3.org/TR/xhtml-basic/xhtml-basic10.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="content-type"
content="text/html;charset=iso-8859-1"/>
<title>Media Server Markup Language Namespace</title>
</head>
<body>
<h1>Namespace for Media Server Markup Language</h1>
<h2>urn:ietf:params:xml:ns:msml</h2>
<p>See MSML <a
href="http://www.rfc-editor.org/rfc/rfc5707.txt">RFC 5707</a></p>
</body>
</html>
END18.4. XML Schema Registration 💖
This section registers an XML schema per the procedures in [i10].
URI: urn:ietf:params:xml:schema:msml
Registrant Contact:
Adnan Saleem (adnan.saleem@radisys.com) and authors listed within this document.
The XML for this schema can be found as the sole content of Section 16. 🌸
19. References 📚
19.1. Normative References ✨
okay bestie, these are the must-reads 📚 Everything here is required reading to understand the spec, no skipping!
[n1] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, "SIP: Session Initiation Protocol", RFC 3261, June 2002. 📞
[n2] Bray, T., Paoli, J., Sperberg-McQueen, C., and E. Maler, "Extensible Markup Language (XML) 1.0 (Second Edition)," W3C First Edition REC-xml-20001006, October 2000. 🧾
[n3] World Wide Web Consortium, "Speech Recognition Grammar Specification Version 1.0" (SRGS), W3C Candidate Recommendation, March 16, 2004 🎙️
[n4] World Wide Web Consortium, "Natural Language Semantics Markup Language (NLSML) for the Speech Interface Framework", W3C Working Draft 20, November 2000. 💬
[n5] World Wide Web Consortium, "Voice Extensible Markup Language (VoiceXML) Version 2.0, W3C Candidate Recommendation, March 16, 2004. 🗣️
[n6] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform Resource Identifier (URI): Generic Syntax", STD 66, RFC 3986, January 2005. 🔗
[n7] Burger, E., Ed., Van Dyke, J., and A. Spitzer, "Basic Network Media Services with SIP", RFC 4240, December 2005. 🎬
[n8] Levinson, E., "Content-ID and Message-ID Uniform Resource Locators", RFC 2392, August 1998. 🏷️
[n9] Handley, M., Jacobson, V., and C. Perkins, "SDP: Session Description Protocol", RFC 4566, July 2006. 🧩
[n10] Bos, B., Lie, H., Tantek, C., and Hickson, I., "Cascading Style Sheets, level 2 (CSS2) Specification," W3C REC CR-CSS21-, July 2007. 💅
[n11] Burnett, D., Walker, M., and Hunt, A., "Speech Synthesis Markup Language (SSML) Version 1.0", W3C Recommendation, 7 September 2004. 🔊
19.2. Informative References 💖
these ones are the extra-credit reading, totally worth a look if you wanna go deeper 🤓
[i1] Rosenberg, J., Schulzrinne, H., and P. Kyzivat, "Indicating User Agent Capabilities in the Session Initiation Protocol (SIP)", RFC 3840, August 2004. 🪪
[i2] Murata, M., St. Laurent, S., and D. Kohn, "XML Media Types", RFC 3023, January 2001. 🏷️
[i3] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, July 2003. 🎥
[i4] Rosenberg, J., Peterson, J., Schulzrinne, H., and G. Camarillo, "Best Current Practices for Third Party Call Control (3pcc) in the Session Initiation Protocol (SIP)", BCP 85, RFC 3725, April 2004. 🤝
[i5] Donovan, S., "The SIP INFO Method", RFC 2976, October 2000. ℹ️
[i6] Ossenbruggen, J., Rutledge, L., Saccocio, B., Schmitz, P., Kate, W., Ayars, J., Bulterman, D., Cohen, A., Day, K., Hodge, E., Hoschka, P., Hyche, E., Jourdan, M., Kubota, K., Lanphier, R., Laya'da, N., Michel, T., and D. Newman, "Synchronized Multimedia Integration Language (SMIL 2.0) Specification," W3C REC REC-smil2-20050107, January 2005. 🎞️
[i7] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types", RFC 2046, November 1996. 📎
[i8] Moats, R., "URN Syntax", RFC 2141, May 1997. 🪄
[i9] Moats, R., "A URN Namespace for IETF Documents", RFC 2648, August 1999. 🗂️
[i10] Mealling, M., "The IETF XML Registry", BCP 81, RFC 3688, January 2004. 📋
[i11] Boulton, C., Melanchuk, T., McGlashan, S., and A. Shiratzky, "A Control Framework for the Session Initiation Protocol (SIP)", Work in Progress, February 2007. 🚧
Acknowledgments 🫶
the squad that made this whole thing possible, literally iconic of them 👑
Sergiu Stambolian of RadiSys provided key insights, both theoretic and through development experience, on several versions of the document. 🧠
Stephen Buko and George Raskulinec of Intel made numerous valuable contributions towards enhancements of multimedia playback and record operations. 🎶
Gene Shtirmer of Intel provided review feedback on several revisions and feature enhancement suggestions. 📝
David Asher of NMS Communications provided valuable insights towards creation of standard profiles and a modularization scheme based on packages for better interoperability. 🧩
Gilles Compienne of Ubiquity Software has provided feedback on several earlier versions of this document. 💜
Chris Boulton and Ben Smith, both of Ubiquity, and Michael Rice of VocalData helped clarify several issues, while Bruce Walsh and Kevin Fitzgerald, both of Spectel/Avaya, provided important feedback. ✨
Cliff Schornak of Commetrex significantly contributed to the facsimile work. 📠
Peter Danielsen of Lucent has contributed thoughtful and detailed reviews for several earlier versions of the document. 🌸
Authors' Addresses 🎀
these are the brains behind it, so you know where to find them 🦋
Adnan Saleem RadiSys 4190 Still Creek Drive, Suite 300 Burnaby, BC, V5C 6C6 Canada
Phone: +1 604 918 6376 EMail: adnan.saleem@radisys.com
Yong Xin RadiSys 4190 Still Creek Drive, Suite 300 Burnaby, BC, V5C 6C6 Canada
Phone: +1 604 918 6383 EMail: yong.xin@radiSys.com
Garland Sharratt Consultant Vancouver, BC Canada
EMail: garland.sharratt@gmail.com
and that's a wrap, bestie! 🌈💅✨
📦 where this page comes from
- ietf: rfc5707 2010-02
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms