RFC 4463
A Media Resource Control Protocol (MRCP) Developed by Cisco, Nuance, and Speechworks
RFC 4463: A Media Resource Control Protocol (MRCP) Developed by Cisco, Nuance, and Speechworks 🎀
Okay bestie, here's the title block tea ☕. This is RFC 4463, published in April 2006 under the Network Working Group. It's Informational, which means it's sharing info for the Internet community and is NOT a standard. ✨ The authors are S. Shanmugham (Cisco Systems, Inc.), P. Monaco (Nuance Communications), and B. Eberman (Speechworks Inc.). The header doesn't list anything it obsoletes or updates. 💅
Status of This Memo
Okay so this memo just gives info to the Internet community. It does not specify an Internet standard of any kind 🙈. Distribution of this memo is unlimited, so share away 🦋.
Copyright Notice
Copyright (C) The Internet Society (2006) 📜.
IESG Note
This one's a big disclaimer, so read carefully 👀. This RFC is not a candidate for any level of Internet Standard. The IETF doesn't know whether this RFC is fit for any purpose, and the decision to publish was NOT based on IETF review for stuff like security, congestion control, or bad interaction with protocols already out there ⚠️. The RFC Editor picked to publish it at their own discretion. Readers should be cautious about how much they trust it for implementing and deploying 🙃. See RFC 3932 for more info.
Also, this document uses a MIME type application/mrcp, which has NOT been registered with the IANA, so it's not recognized as a standard IETF MIME type 🥲. Still, it's an ancestor to ongoing standardization in this space, so publishing it is meaningful 🌈.
Abstract
This doc describes a Media Resource Control Protocol (MRCP) that was developed together by Cisco Systems, Inc., Nuance Communications, and Speechworks, Inc. 🤝 It's published as an RFC as input for further IETF development in this area 🧚.
MRCP controls media service resources like speech synthesizers, recognizers, signal generators, signal detectors, fax servers, etc., over a network 📞. It's designed to work with streaming protocols like RTSP (Real Time Streaming Protocol) or SIP (Session Initiation Protocol), which help set up control connections to external media streaming devices. It also works with media delivery mechanisms like RTP (Real Time Protocol) 🎶.
Table of Contents
1. Introduction ....................................................3
2. Architecture ....................................................4
2.1. Resources and Services .....................................4
2.2. Server and Resource Addressing .............................5
3. MRCP Protocol Basics ............................................5
3.1. Establishing Control Session and Media Streams .............5
3.2. MRCP over RTSP .............................................6
3.3. Media Streams and RTP Ports ................................8
4. Notational Conventions ..........................................8
5. MRCP Specification ..............................................9
5.1. Request ...................................................10
5.2. Response ..................................................10
5.3. Event .....................................................12
5.4. Message Headers ...........................................12
6. Media Server ...................................................19
6.1. Media Server Session ......................................19
7. Speech Synthesizer Resource ....................................21
7.1. Synthesizer State Machine .................................22
7.2. Synthesizer Methods .......................................22
7.3. Synthesizer Events ........................................23
7.4. Synthesizer Header Fields .................................23
7.5. Synthesizer Message Body ..................................29
7.6. SET-PARAMS ................................................32
7.7. GET-PARAMS ................................................32
7.8. SPEAK .....................................................33
7.9. STOP ......................................................34
7.10. BARGE-IN-OCCURRED ........................................35
7.11. PAUSE ....................................................37
7.12. RESUME ...................................................37
7.13. CONTROL ..................................................38
7.14. SPEAK-COMPLETE ...........................................40
7.15. SPEECH-MARKER ............................................41
8. Speech Recognizer Resource .....................................42
8.1. Recognizer State Machine ..................................42
8.2. Recognizer Methods ........................................42
8.3. Recognizer Events .........................................43
8.4. Recognizer Header Fields ..................................43
8.5. Recognizer Message Body ...................................51
8.6. SET-PARAMS ................................................56
8.7. GET-PARAMS ................................................56
8.8. DEFINE-GRAMMAR ............................................57
8.9. RECOGNIZE .................................................60
8.10. STOP .....................................................63
8.11. GET-RESULT ...............................................64
8.12. START-OF-SPEECH ..........................................64
8.13. RECOGNITION-START-TIMERS .................................65
8.14. RECOGNITON-COMPLETE ......................................65
8.15. DTMF Detection ...........................................67
9. Future Study ...................................................67
10. Security Considerations .......................................67
11. RTSP-Based Examples ...........................................67
12. Informative References ........................................74
Appendix A. ABNF Message Definitions ..............................76
Appendix B. Acknowledgements ......................................841. Introduction ✨
Okay so MRCP is made to give a client device that needs audio/video stream processing a way to control processing resources on the network 🔥. These media processing resources can be speech recognizers (a.k.a. Automatic-Speech-Recognition (ASR) engines), speech synthesizers (a.k.a. Text-To-Speech (TTS) engines), fax, signal detectors, and more 🎙️. MRCP lets you build distributed Interactive Voice Response platforms, like VoiceXML [6] interpreters 📱. The protocol defines the requests, responses, and events needed to control those media processing resources 💜. It also defines the state machine for each resource and the required state transitions for each request and server-generated event 🧠.
MRCP doesn't handle how the control session gets set up with the server. Instead it relies on the Real Time Streaming Protocol (RTSP) [2] to set up and maintain the session 🙌. The session control protocol is also in charge of setting up the media connection from the client to the network server. MRCP and its messaging are designed to be carried over RTSP or another protocol as a MIME-type, similar to the Session Description Protocol (SDP) [5] 🌸.
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [8] 🫶.
2. Architecture 🏗️
So the system has two main players 👯. There's a client that needs media streams generated or processed, and a server that has the resources or devices to do that processing or generating. The client sets up a control session with the server using a protocol like RTSP 🤳. That also sets up and establishes the RTP stream between the client and the server, or another RTP endpoint 🎧. Each resource needed to process or generate the stream gets addressed by a URL. Then the client can use MRCP messages to control the media resources and change how they process or generate the media stream 🦋.
|--------------------|
||------------------|| |----------------------|
|| Application Layer|| ||--------------------||
||------------------|| || TTS | ASR | Fax ||
|| ASR/TTS API || ||Plugin|Plugin|Plugin||
||------------------|| || on | on | on ||
|| MRCP Core || || MRCP | MRCP | MRCP ||
|| Protocol Stack || ||--------------------||
||------------------|| || RTSP Stack ||
|| RTSP Stack || || ||
||------------------|| ||--------------------||
|| TCP/IP Stack ||========IP=========|| TCP/IP Stack ||
||------------------|| ||--------------------||
|--------------------| |----------------------|
MRCP client Real-time Streaming MRCP
media server2.1. Resources and Services 🍓
The server is set up to offer a certain set of resources and services to the client. These resources come in 3 types 🧁.
Transmission Resources
These are resources that can generate real-time streams, like signal generators that make tones and sounds at certain frequencies and patterns, and speech synthesizers that generate spoken audio streams, etc. 🔊
Reception Resources
These are resources that receive and process streaming data, like signal detectors and speech recognizers 👂.
Dual Mode Resources
These are resources that both send and receive data, like a fax resource that can send or receive fax through a two-way RTP stream 📠.
2.2. Server and Resource Addressing 🗺️
The server as a whole is addressed using a container URL. The individual resources the server offers are reached by individual resource URLs inside that container URL 💌.
RTSP Example:
A media server or container URL looks like this 👇
rtsp://mediaserver.com/media/It may contain one or more resource URLs of the form 💕
rtsp://mediaserver.com/media/speechrecognizer/
rtsp://mediaserver.com/media/speechsynthesizer/
rtsp://mediaserver.com/media/fax/3. MRCP Protocol Basics 💖
The MRCP message format is text based, with ways to carry embedded binary data 📝. That's how stuff like recognition grammars, recognition results, and synthesizer speech markup can travel in the MRCP message between the client and the server resource 🧩. The protocol does NOT handle session control management, media management, reliable sequencing and delivery, or server or resource addressing. Those get left to a protocol like SIP or RTSP 🙃. MRCP focuses on controlling and communicating with the resource processing the stream, and defines the requests, responses, and events needed to do that periodt 💅.
3.1. Establishing Control Session and Media Streams
okay so the control session between client and server gets set up with something like RTSP 💖 That same handshake also sets up the RTP streams between server and client, allocating ports and sorting out transport parameters as needed ✨ Each control session gets its own unique session-id, and how that session-id is formatted, used, and lives its life is all handled by RTSP 🎀 Resources inside the session are addressed by their individual resource URLs 🦋
MRCP is giving tunneling queen 👑 It's built to ride through another protocol like RTSP and level up what that protocol can do. MRCP leans on RTSP headers for sequencing, reliability, and addressing, so messages arrive reliably, in the right order, and at the right resource 💅 The MRCP messages ride along in the RTSP message body. The media server works out where each message goes by checking the session-level headers and the URL info 🧚 And yes, another protocol, like SIP [4], could also be used to tunnel MRCP messages 🌸
3.2. MRCP over RTSP
RTSP supports both TCP and UDP for the client to talk to the server, and the RTSP URL is what tells them apart 🦋 Every MRCP based media server MUST support TCP for transport and MAY support UDP 💜
Over in RTSP land, the ANNOUNCE method/response MUST be used to carry MRCP request/responses between client and server 🎀 MRCP messages MUST NOT be communicated in the RTSP SETUP or TEARDOWN messages 🙈
Here's the tea 🍓 Right now every RTSP message is a request/response, and RTSP doesn't do asynchronous events at all. That's because RTSP was designed to run over TCP or UDP, so it couldn't assume the underlying protocol would be reliable 😭 So when you're doing MRCP over RTSP, an asynchronous event from the MRCP server gets wrapped up in a server-initiated ANNOUNCE method/response instead ✨ A future RTSP extension that lets the server send asynchronous events to the client would give us a different vehicle for carrying those MRCP events 🌈
An RTSP session gets created when the client sends an RTSP SETUP message to a server, addressed either to a server URL or to any one of its resource URLs, without a session-id in it 💖 The server sets up a session context and replies to the client with a session-id 🫶 That same sequence also sets up the RTP transport parameters between client and server, and then the server is ready to receive or send media streams 🔥 If the client wants to attach an additional resource to a session that already exists, the client should send that session's ID in the subsequent SETUP message 🧚
Now the rule for media servers: when a media server doing MRCP over RTSP gets a PLAY, RECORD, or PAUSE RTSP method aimed at an MRCP resource URL, it should respond with an RTSP 405 "Method not Allowed" response 🙅♀️ For these resources, the only RTSP methods allowed are SETUP, TEARDOWN, DESCRIBE, and ANNOUNCE 💅
Example 1 is the vibe check, bestie. Read it all the way through so you can see the whole call flow 👀
Example 1:
C->S: ANNOUNCE rtsp://media.server.com/media/synthesizer RTSP/1.0
CSeq:4
Session:12345678
Content-Type:application/mrcp
Content-Length:223
SPEAK 543257 MRCP/1.0
Voice-gender:neutral
Voice-category:teenager
Prosody-volume:medium
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<sentence>The subject is <prosody
rate="-20%">ski trip</prosody></sentence>
</paragraph>
</speak>
S->C: RTSP/1.0 200 OK
CSeq: 4
Session:12345678
RTP-Info:url=rtsp://media.server.com/media/synthesizer;
seq=9810092;rtptime=3450012
Content-Type:application/mrcp
Content-Length:52
MRCP/1.0 543257 200 IN-PROGRESS
S->C: ANNOUNCE rtsp://media.server.com/media/synthesizer RTSP/1.0
CSeq:6
Session:12345678
Content-Type:application/mrcp
Content-Length:123
SPEAK-COMPLETE 543257 COMPLETE MRCP/1.0
C->S: RTSP/1.0 200 OK
CSeq:6To keep things brief 🙈, most examples from here on only show the MRCP messages and skip the RTSP messages and headers they're tunneled in. RTSP messages that aren't carrying an MRCP message, like responses, are left out too 💜
3.3. Media Streams and RTP Ports
A single set of RTP/RTCP ports is negotiated and shared between the MRCP client and server whenever multiple media processing resources are used in one session 🎀 Think automatic speech recognition (ASR) engines and text to speech (TTS) engines hanging out together 🌈 The individual resource instances allocated on the server under a common session identifier all feed from or to that one RTP stream ✨
The client can send multiple media streams toward the server, and they're told apart by using different synchronized source (SSRC) identifier values 🦋 The server can do the same, using multiple Synchronized Source (SSRC) identifier values to tell apart media streams coming from the individual transmission resource URLs, if there's more than one 🫶 The resources might also team up and send just one stream to the client instead. That's totally up to how the media server is implemented 💅
4. Notational Conventions ✨
Lots of the definitions and syntax here are identical to HTTP/1.1, so this spec just points to the section where they live instead of copying them over 🧚 For brevity, [HX.Y] refers to Section X.Y of the current HTTP/1.1 specification (RFC 2616 [1]) 💜
Every mechanism in this document is described in both prose and an augmented Backus-Naur form (ABNF) similar to the one used in [H2.1] 🎀 The full details are in RFC 4234 [3] 🌸
The ABNF that comes with the descriptive text is informative, which means it might not be complete 🙈 The complete message format in ABNF is in Appendix A, and that's the normative format definition, so that's the one to trust 👑
5. MRCP Specification 🔥
The MRCP PDU is textual, using an ISO 10646 character set in the UTF-8 encoding (RFC 3629 [12]) so lots of different languages can be represented 🌈 For compact representations, MRCP also lets other character sets like ISO 8859-1 be used when that's wanted 🍓 The MRCP protocol headers and field names only use the US-ASCII subset of UTF-8 💅 Internationalization applies to certain fields like grammar, results, speech markup, and so on, not to MRCP as a whole ✨
Lines are terminated by CRLF, but receivers SHOULD be prepared to also interpret CR and LF by themselves as line terminators 💖 Also, some parameters in the PDU may hold binary data or a record that spans multiple lines 🦋 Those fields come with a length value attached to the parameter, which tells you how many octets immediately follow it 🫶
The whole MRCP PDU rides in the body of the session level message as a MIME entity of type application/mrcp 🎀 The individual MRCP messages don't have addressing info saying which resource the request/response is to or from 🙈 Instead, the MRCP message relies on the header of the session level message carrying it to deliver the request to the right resource, or to figure out who a response or event came from 🧚
The MRCP message set has three flavors: requests from the client to the server, responses from the server to the client, and asynchronous events from the server to the client 🌸 Every one of them is made of a start-line, one or more header fields (also known as "headers"), an empty line (a line with nothing before the CRLF) that marks the end of the header fields, and an optional message body 💜
generic-message = start-line
message-header
CRLF
[ message-body ]
message-body = *OCTET
start-line = request-line / status-line / event-lineThe message-body holds resource-specific and message-specific data that needs to travel between client and server as a MIME entity 🌈 What the info is and which MIME types carry it get spelled out later, when each specific message is covered ✨
5.1. Request
An MRCP request is a Request line, followed by zero or more parameters as part of the message headers, plus an optional message body with data specific to that request 💅
The Request message from client to server has, in its first line, the method to be applied, a method tag for that request, and the version of the protocol in use 🎀
request-line = method-name SP request-id SP
mrcp-version CRLFThe request-id field is a unique identifier that the client creates and sends to the server 🦋 The server resource should use this identifier in its response to this request 🫶 If the request doesn't complete with the response, future asynchronous events tied to this request MUST carry the request-id 🔥
request-id = 1*DIGITThe method-name field says which specific request the client is making to the server 🧚 Each resource supports a certain list of requests or methods that can be issued to it, and those get covered in later sections 🌈
method-name = synthesizer-method
/ recognizer-methodThe mrcp-version field is the MRCP protocol version the client is using 💖
mrcp-version = "MRCP" "/" 1*DIGIT "." 1*DIGIT5.2. Response
Once the server resource has received and interpreted the request message, it responds with an MRCP response message 🌸 It's made of a status line, optionally followed by a message body ✨
response-line = mrcp-version SP request-id SP status-code SP
request-state CRLFThe mrcp-version field here is similar to the one in the Request Line, and it shows which MRCP protocol version the server is running 💜
The request-id in the response MUST match the one sent in the corresponding request message 🎀
The status-code field is a 3-digit code showing the success, failure, or other status of the request 🙈
The request-state field says whether the job started by the Request is PENDING, IN-PROGRESS, or COMPLETE 👑 COMPLETE means the Request was processed all the way through, and there won't be any more events from that resource to the client with that request-id 💅 PENDING means the job has been placed on a queue and will be processed in first-in-first-out order 🧚 IN-PROGRESS means the request is being processed and isn't finished yet 🔥 While a request is PENDING or IN-PROGRESS, further Event messages will be delivered with that request-id 🦋
request-state = "COMPLETE"
/ "IN-PROGRESS"
/ "PENDING"5.2.1. Status Codes
Status codes fall into two big buckets: the Success (2XX) codes and the Failure (4XX) codes 🌈
5.2.1.1. Success 2xx ✨
200 Success
201 Success with some optional parameters ignored.5.2.1.2. Failure 4xx 💅
401 Method not allowed
402 Method not valid in this state
403 Unsupported Parameter
404 Illegal Value for Parameter
405 Not found (e.g., Resource URI not initialized
or doesn't exist)
406 Mandatory Parameter Missing
407 Method or Operation Failed (e.g., Grammar compilation
failed in the recognizer. Detailed cause codes MAY BE
available through a resource specific header field.)
408 Unrecognized or unsupported message entity
409 Unsupported Parameter Value
421-499 Resource specific Failure codes5.3. Event 🔔
okay so the server resource might need to tell the client that something changed or that a certain event happened 👀 these messages are used when a request doesn't finish right away and the response comes back with a status of PENDING or IN-PROGRESS. the in-progress results and events for the request get sent to the client through the event message from the server. events carry the request-id of the request that's in progress and generating them, plus a status value. the status value is COMPLETE if the request is done and this was the last event, otherwise it's IN-PROGRESS. bestie, that's the whole vibe 🫶
event-line = event-name SP request-id SP request-state SP
mrcp-version CRLFthe mrcp-version used here is identical to the one used in the Request/Response Line, and it shows which version of the MRCP protocol the server is running 🌸
the request-id in the event should match the one sent in the request that caused the event. no mix-ups, periodt ✨
the request-state tells you if the Request/Command causing this event is complete or still in progress, and it's the same one mentioned in Section 5.2. the final event will have a COMPLETE status to show the request is finished 🎀
the event-name identifies what kind of event the media resource generated. the valid event names depend on which resource is generating them, and we'll cover those in later sections 🦋
event-name = synthesizer-event
/ recognizer-event5.4. Message Headers 🎀
MRCP header fields (the general-header ones from Section 5.4 and the resource-specific-header ones from Sections 7.4 and 8.4) follow the same generic format as Section 2.1 of RFC 2822 [7]. each header field is a name, then a colon (":"), then the field value 💖 field names are case-insensitive. the field value MAY be preceded by any amount of linear whitespace (LWS), though a single SP is preferred. headers can also be stretched over multiple lines, as long as each extra line starts with at least one SP or HT 🌈
message-header = 1*(generic-header / resource-header)the order of header fields with different field names doesn't matter when they're received 🙌 but it's "good practice" to send the general-header fields first, then the request-header or response-header fields, and end with the entity-header fields. very organized, very iconic 💅
multiple message-header fields with the same field-name MAY be present in a message if and only if the entire field value for that header is defined as a comma-separated list (i.e., #(values)).
it MUST be possible to combine multiple header fields into one "field-name:field-value" pair without changing what the message means, by appending each later field-value to the first one, separated by a comma. so the order of header fields with the same field-name does matter for how the combined value gets read, which is why a proxy MUST NOT change the order of these field values when it forwards a message. not me being dramatic, but this is a real rule 😭
Generic Headers 🌸
generic-header = active-request-id-list
/ proxy-sync-id
/ content-id
/ content-type
/ content-length
/ content-base
/ content-location
/ content-encoding
/ cache-control
/ logging-tagall headers in MRCP are case insensitive, just like HTTP and RTSP header definitions. consistency is giving ✨
5.4.1. Active-Request-Id-List 🧚
in a request, this field lists the request-ids it should apply to. it's super useful when you've got multiple Requests that are PENDING or IN-PROGRESS and you want this one to hit only some of them, specifically 🎯
in a response, this field returns the list of request-ids that the operation modified, or that were in progress, or that just completed. there could be one or more requests that returned a request-state of PENDING or IN-PROGRESS. when a method that affects one or more PENDING or IN-PROGRESS requests gets sent from the client to the server, the response MUST contain the list of request-ids that were affected in this header field 🫶
the active-request-id-list is only used in requests and responses, never in events 🙅♀️
for example, say a STOP request with no active-request-id-list gets sent to a synthesizer resource (a wildcard STOP) that has one or more SPEAK requests in the PENDING or IN-PROGRESS state. all SPEAK requests MUST be cancelled, including the one that's IN-PROGRESS. on top of that, the response to the STOP request would contain the request-id of every SPEAK request that got terminated, in the active-request-id-list. and in this case, no SPEAK-COMPLETE or RECOGNITION-COMPLETE events will be sent for those terminated requests. ghosted, but on purpose 💀
active-request-id-list = "Active-Request-Id-List" ":" request-id
*("," request-id) CRLF5.4.2. Proxy-Sync-Id 🔥
when any server resource generates a barge-in-able event, it creates a unique Tag and sends it as a header field in an event to the client. the client then acts as a proxy to the server resource and sends a BARGE-IN-OCCURRED method (Section 7.10) to the synthesizer server resource, using the Proxy-Sync-Id it got from the server resource. when the recognizer and synthesizer resources are part of the same session, they may choose to work together for quicker interaction and response 🏃♀️ and here's where the proxy-sync-id helps: it lets the resource receiving the event (proxied by the client) figure out whether this event was already handled through a direct interaction between the resources. main character energy for this header, honestly 👑
proxy-sync-id = "Proxy-Sync-Id" ":" 1*ALPHA CRLF5.4.3. Accept-Charset 🦋
see [H14.2]. this specifies the acceptable character set for entities returned in the response or in events associated with this request. it's useful for picking the character set to use in the Natural Language Semantics Markup Language (NLSML) results of a RECOGNITON-COMPLETE event 🌈
5.4.4. Content-Type 💜
see [H14.17]. heads up: the content types suitable for MRCP are limited to speech markup, grammar, recognition results, etc., and they're specified later in this document 📜 the multi-part content type "multi-part/mixed" is supported for communicating several of the above-mentioned contents at once. in that case, the body parts cannot contain any MRCP specific headers 🙈
5.4.5. Content-Id 🎀
this field contains an ID or name for the content, so it can be referred to later. the definition of this field conforms to RFC 2392 [14], RFC 2822 [7], and RFC 2046 [13], and it's needed in multi-part messages 🧩 in MRCP, whenever the content needs to be stored (by the client or the server), it's stored associated with this ID. then the content can be referenced during the session in URI form, using the session:URI scheme described in a later section 🔗
5.4.6. Content-Base 🌸
the content-base entity-header field may be used to specify the base URI for resolving relative URLs within the entity.
content-base = "Content-Base" ":" absoluteURI CRLFnote, though, that the base URI of the contents within the entity-body may be redefined within that entity-body. an example of this would be a multi-part MIME entity, which can have multiple entities inside it 🪆
5.4.7. Content-Encoding 🍓
the content-encoding entity-header field is used as a modifier to the media-type. when it's present, its value tells you what additional content coding has been applied to the entity-body, and so what decoding mechanisms must be applied to get the media-type referenced by the content-type header field. content-encoding is mainly used to let a document be compressed without losing the identity of its underlying media type 🗜️✨
content-encoding = "Content-Encoding" ":"
*WSP content-coding
*(*WSP "," *WSP content-coding *WSP )
CRLF
content-coding = token
token = 1*(alphanum / "-" / "." / "!" / "%" / "*"
/ "_" / "+" / "`" / "'" / "~" )content coding is defined in [H3.5]. an example of its use is:
Content-Encoding:gzip
if multiple encodings have been applied to an entity, the content codings MUST be listed in the order in which they were applied. order matters, bestie 🫶
5.4.8. Content-Location 💖
the content-location entity-header field MAY BE used to supply the resource location for the entity enclosed in the message, when that entity is accessible from a location separate from the requested resource's URI 🗺️
content-location = "Content-Location" ":" ( absoluteURI /
relativeURI ) CRLFthe content-location value is a statement of where the resource for this particular entity was located at the time of the request. the media server MAY use this header field to optimize certain operations. when you provide this header field, the entity being sent should not have been modified from what was retrieved from the content-location URI 🧐
for example, say the client provided a grammar markup inline, and it had previously retrieved it from a certain URI. that URI can be provided as part of the entity, using the content-location header field. this lets a resource like the recognizer look into its cache to see whether this grammar was already retrieved, compiled, and cached. if so, it might optimize by using the previously compiled grammar object 🧠⚡
if the content-location is a relative URI, the relative URI is interpreted relative to the content-base URI 🌈
5.4.9. Content-Length 🙈
this field contains the length of the content of the message body (i.e., everything after the double CRLF that follows the last header field). unlike HTTP, it MUST be included in all messages that carry content beyond the header portion of the message 📏 if it's missing, a default value of zero is assumed. it's interpreted according to [H14.13] ✨
5.4.10. Cache-Control 🎀
okay so if the media server is planning on doing caching, it MUST follow the cache correctness rules of HTTP 1.1 (RFC2616) whenever it accesses and caches HTTP URIs 💅 the expires and cache-control headers of the cached URI or document must be honored, and they will literally always win over the Cache-Control defaults set by this header field 👑 this header is just for setting the default caching behavior on the media server for the session or request ✨ its scope depends on the method it's sent on! if it rides along on a SET-PARAMS method, it SHOULD apply to every document request the media server might make during that session 🦋 if it's sent on any other messages, it MUST only apply to the document requests the media server needs to make for that method 🌸 and an empty cache-control header on a GET-PARAMS method is basically the media server going "ok, tell me your current cache-control settings" 🫶
cache-control = "Cache-Control" ":" *WSP cache-directive
*( *WSP "," *WSP cache-directive *WSP )
CRLF
cache-directive = "max-age" "=" delta-seconds
/ "max-stale" "=" delta-seconds
/ "min-fresh" "=" delta-seconds
delta-seconds = 1*DIGITso here, delta-seconds is a time value written as an integer number of seconds in decimal, counted from the moment the media server received the message response or data 🧚
these directives let the media server override the basic expiration mechanism 💜
max-age
this one says the client is OK with the media server using a response whose age is no greater than the specified time in seconds 🍓 unless a max-stale directive is also included, the client is NOT willing to let the media server use a stale response 🙈
Wait, correction for the record: the client is not willing to accept the media server using a stale response. That's the whole vibe ✨
min-fresh
this means the client is willing to accept the media server using a response whose freshness lifetime is no less than its current age plus the specified time in seconds 🌈 basically the client wants a response that will still be fresh for at least that many seconds 🎀
max-stale
this means the client is willing to accept the media server using a response that has gone past its expiration time 💀 if max-stale gets a value, the client will accept a response that is past its expiration time by no more than that many seconds 🦋 if no value is given to max-stale, the client will accept a stale response of any age, no questions asked 🔥
the media server cache MAY BE requested to use stale response/data without validation, but only if that doesn't conflict with any "MUST"-level requirements about cache validation in the HTTP 1.1 specification for that URI (like a "must-revalidate" cache-control directive) 🫶 this is one of those "read the fine print" moments bestie 🙈
and if both the MRCP cache-control directive and the cached entry on the media server include "max-age" directives, the lesser of the two values is used to decide how fresh the cached entry is for that request 💅
5.4.11. Logging-Tag 🧚
this header field MAY BE sent as part of a SET-PARAMS/GET-PARAMS method to set the logging tag for logs the media server generates 🌸 once it's set, the value sticks around until a new value is set or the session ends 💖 the MRCP server should give admins a way to subset its output logs, so they can look at or pull out just the part of the log file from when the logging tag was set to a certain value 🦋
MRCP clients using this feature should be careful that no two clients use the same logging tag 🙈 if two clients do specify the same logging tag, the effect on the MRCP server's output logs is undefined 💀
logging-tag = "Logging-Tag" ":" 1*ALPHA CRLF6. Media Server 🎬
the capability of media server resources can be found using the RTSP DESCRIBE mechanism 💜 when a client sends an RTSP DESCRIBE method for a media resource URI, the media server response MUST contain an SDP description in its body that describes the capabilities of the media server resource 👑 that SDP description MUST contain, at a minimum, the media header (m-line) describing the codec and other media-related features it supports 🌈 it MAY contain another SDP header as well, but support for it is optional ✨
the way SDP messages show up in the RTSP message body and get used follows the SIP RFC 2543 [4], but it's limited to media-related negotiation and description 🎀
6.1. Media Server Session 🦋
as discussed in Section 3.2, a client/server should share one RTSP session-id for the different resources it may use under the same session 💖 the client MUST allocate a set of client RTP/RTCP ports for a new session, and it MUST NOT send a Session-ID in the SETUP message for the first resource 🫶 then the server creates a Session-ID, allocates a set of server RTP/RTCP ports, and responds to the SETUP message 🌸
if the client wants to open more resources with the same server under the same session, it sends the session-id it got in the earlier SETUP response in the SETUP for the new resource 🧚 a SETUP message with an existing session-id tells the server that this new resource will feed from or into the same RTP/RTCP stream as that existing session 💅
if the client wants to open a resource from a media server that isn't where the first resource came from, it sends separate SETUP requests with no session-id header field in them 🍓 each server allocates its own session-id and returns it in the response, and each one also comes back with its own set of RTP/RTCP ports 🌈 this happens when the synthesizer engine and the recognition engine live on different servers 🔥
the RTSP SETUP method SHOULD contain an SDP description of the media stream being set up 💜 the RTSP SETUP response MUST contain an SDP description of the media stream it expects to receive and send on that session 👑
the SDP description in the SETUP method from the client SHOULD describe the required media parameters like codec, Named Signaling Event (NSE) payload types, etc. 🎀 it could have multiple media headers (i.e., m-lines) so the client can give the media server more than one option to choose from 🦋
the SDP description in the SETUP response should reflect the media parameters the media server will actually be using for the stream 🌸 it should be within the choices that were specified in the SDP of the SETUP method, if one was provided ✨
example time 💖
C->S:
SETUP rtsp://media.server.com/recognizer/ RTSP/1.0
CSeq:1
Transport:RTP/AVP;unicast;client_port=46456-46457
Content-Type:application/sdp
Content-Length:190
v=0
o=- 123 456 IN IP4 10.0.0.1
s=Media Server
p=+1-888-555-1212
c=IN IP4 0.0.0.0
t=0 0
m=audio 46456 RTP/AVP 0 96
a=rtpmap:0 pcmu/8000
a=rtpmap:96 telephone-event/8000
a=fmtp:96 0-15
S->C:
RTSP/1.0 200 OK
CSeq:1
Session:0a030258_00003815_3bc4873a_0001_0000
Transport:RTP/AVP;unicast;client_port=46456-46457;
server_port=46460-46461
Content-Length:190
Content-Type:application/sdp
v=0
o=- 3211724219 3211724219 IN IP4 10.3.2.88
s=Media Server
c=IN IP4 0.0.0.0
t=0 0
m=audio 46460 RTP/AVP 0 96
a=rtpmap:0 pcmu/8000
a=rtpmap:96 telephone-event/8000
a=fmtp:96 0-15if an SDP description wasn't provided in the RTSP SETUP method, the media server may decide on the parameters of the stream, but it MUST specify what it chooses in the SETUP response 🫶 an SDP announcement only comes back in a response to a SETUP message that doesn't specify a Session 💅 so the server won't return an SDP announcement for the synthesizer SETUP of a session that's already been established with a recognizer 🙈
C->S:
SETUP rtsp://media.server.com/recognizer/ RTSP/1.0
CSeq:1
Transport:RTP/AVP;unicast;client_port=46498
S->C:
RTSP/1.0 200 OK
CSeq:1
Session:0a030258_000039dc_3bc48a13_0001_0000
Transport:RTP/AVP;unicast; client_port=46498;
server_port=46502-46503
Content-Length:193
Content-Type:application/sdp
v=0
o=- 3211724947 3211724947 IN IP4 10.3.2.88
s=Media Server
c=IN IP4 0.0.0.0
t=0 0
m=audio 46502 RTP/AVP 0 101
a=rtpmap:0 pcmu/8000
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-157. Speech Synthesizer Resource 🗣️
this resource can turn text the client provides into a speech stream in real time ✨ depending on how it's implemented and what it can do, the client can control stuff like voice characteristics, speaker speed, and so on 🌈
the synthesizer resource gets controlled by MRCP requests from the client 💖 it can respond to those requests, or it can generate asynchronous events to the server to flag certain conditions while it's processing the stream 🦋
7.1. Synthesizer State Machine 🧠
the synthesizer keeps track of states because it needs to correlate MRCP requests from the client 💅 the state transitions below describe the states of the synthesizer and reflect the request sitting at the head of the queue 👑 a SPEAK request in the PENDING state can be deleted or stopped by a STOP request, and that doesn't change the state of the resource 🫶
Idle Speaking Paused
State State State
| | |
|----------SPEAK------->| |--------|
|<------STOP------------| CONTROL |
|<----SPEAK-COMPLETE----| |------->|
|<----BARGE-IN-OCCURRED-| |
| |--------| |
| CONTROL |-----------PAUSE--------->|
| |------->|<----------RESUME---------|
| | |----------|
| | PAUSE |
| | |--------->|
| |--------|----------| |
| BARGE-IN-OCCURRED | SPEECH-MARKER |
| |------->|<---------| |
|----------| | |------------|
| STOP | SPEAK |
| | | |----------->|
|<---------| |
|<-------------------STOP--------------------------|7.2. Synthesizer Methods 🎶
the synthesizer supports the following methods 💜
synthesizer-method = "SET-PARAMS"
/ "GET-PARAMS"
/ "SPEAK"
/ "STOP"
/ "PAUSE"
/ "RESUME"
/ "BARGE-IN-OCCURRED"
/ "CONTROL"7.3. Synthesizer Events ⭐
the synthesizer may generate the following events 🌸
synthesizer-event = "SPEECH-MARKER"
/ "SPEAK-COMPLETE"7.4. Synthesizer Header Fields ✨
okay so this is the bit where the synthesizer's header fields live 💅 A synthesizer message can carry header fields that hold request options and extra info to augment the Request, Response, or Event it's attached to 🎀
synthesizer-header = jump-target ; Section 7.4.1
/ kill-on-barge-in ; Section 7.4.2
/ speaker-profile ; Section 7.4.3
/ completion-cause ; Section 7.4.4
/ voice-parameter ; Section 7.4.5
/ prosody-parameter ; Section 7.4.6
/ vendor-specific ; Section 7.4.7
/ speech-marker ; Section 7.4.8
/ speech-language ; Section 7.4.9
/ fetch-hint ; Section 7.4.10
/ audio-fetch-hint ; Section 7.4.11
/ fetch-timeout ; Section 7.4.12
/ failed-uri ; Section 7.4.13
/ failed-uri-cause ; Section 7.4.14
/ speak-restart ; Section 7.4.15
/ speak-length ; Section 7.4.16Here's the cheat sheet of which header fields work with which methods, events, and responses 🍓
Parameter Support Methods/Events/Response
jump-target MANDATORY SPEAK, CONTROL
logging-tag MANDATORY SET-PARAMS, GET-PARAMS
kill-on-barge-in MANDATORY SPEAK
speaker-profile OPTIONAL SET-PARAMS, GET-PARAMS,
SPEAK, CONTROL
completion-cause MANDATORY SPEAK-COMPLETE
voice-parameter MANDATORY SET-PARAMS, GET-PARAMS,
SPEAK, CONTROL
prosody-parameter MANDATORY SET-PARAMS, GET-PARAMS,
SPEAK, CONTROL
vendor-specific MANDATORY SET-PARAMS, GET-PARAMS
speech-marker MANDATORY SPEECH-MARKER
speech-language MANDATORY SET-PARAMS, GET-PARAMS, SPEAK
fetch-hint MANDATORY SET-PARAMS, GET-PARAMS, SPEAK
audio-fetch-hint MANDATORY SET-PARAMS, GET-PARAMS, SPEAK
fetch-timeout MANDATORY SET-PARAMS, GET-PARAMS, SPEAK
failed-uri MANDATORY Any
failed-uri-cause MANDATORY Any
speak-restart MANDATORY CONTROL
speak-length MANDATORY SPEAK, CONTROL7.4.1. Jump-Target 🦋
this one is the skip-ahead / rewind param 🏃♀️ It MAY BE specified in a CONTROL method to jump forward or rewind backward on an active SPEAK request. A + or - means relative to whatever is currently playing. It MAY BE specified in a SPEAK request too, to set an offset into the speech markup where the SPEAK request should start talking from. Which speech length units work is up to the synthesizer implementation ngl. If it doesn't support a unit or the operation, the resource SHOULD respond with a status code of 404 "Illegal or Unsupported value for parameter" 💀
jump-target = "Jump-Size" ":" speech-length-value CRLF
speech-length-value = numeric-speech-length
/ text-speech-length
text-speech-length = 1*ALPHA SP "Tag"
numeric-speech-length= ("+" / "-") 1*DIGIT SP
numeric-speech-unit
numeric-speech-unit = "Second"
/ "Word"
/ "Sentence"
/ "Paragraph"7.4.2. Kill-On-Barge-In 🙈
this param MAY BE sent as part of the SPEAK method to turn on kill-on-barge-in support 🔥 When it's on, the SPEAK method gets interrupted by DTMF input caught by a signal detector resource, or by the start of speech sensed or recognized by the speech recognizer resource. Iconic interruption behavior, honestly 👑
kill-on-barge-in = "Kill-On-Barge-In" ":" boolean-value CRLF
boolean-value = "true" / "false"If the recognizer or signal detector is on the same server as the synthesizer, that server should be intelligent enough to spot their interactions through their shared RTSP session-id and work together to provide kill-on-barge-in support 🧠 The client needs to send a BARGE-IN-OCCURRED method to the synthesizer resource whenever it gets a barge-in-able event from the synthesizer or signal detector resource. These resources MAY BE local or distributed 🌍 If this field isn't specified, the value defaults to "true" ✨
7.4.3. Speaker Profile 🧚
this one MAY BE part of the SET-PARAMS/GET-PARAMS or SPEAK request from the client to the server 💜 It names the speaker's profile with a URI, which can point to a set of voice parameters like gender, accent, and so on.
speaker-profile = "Speaker-Profile" ":" uri CRLF7.4.4. Completion Cause 👑
this header field MUST be specified in a SPEAK-COMPLETE event coming from the synthesizer resource to the client 🎀 It tells you the reason the SPEAK request finished, and it's giving very important info vibes 😭
completion-cause = "Completion-Cause" ":" 1*DIGIT SP 1*ALPHA
CRLFHere's the full list of cause codes, bestie 📋
Cause-Code Cause-Name Description
000 normal SPEAK completed normally.
001 barge-in SPEAK request was terminated because
of barge-in.
002 parse-failure SPEAK request terminated because of a
failure to parse the speech markup text.
003 uri-failure SPEAK request terminated because, access
to one of the URIs failed.
004 error SPEAK request terminated prematurely due
to synthesizer error.
005 language-unsupported
Language not supported.7.4.5. Voice-Parameters 🎀
this set of params defines the voice of the speaker, so it's the whole aesthetic ✨
voice-parameter = "Voice-" voice-param-name ":"
voice-param-value CRLFvoice-param-name can be any one of the attribute names under the voice element in W3C's Speech Synthesis Markup Language Specification [9] 📖 voice-param-value is any one of the value choices for that voice element attribute, as listed in the section above.
These header fields MAY BE sent in a SET-PARAMS/GET-PARAMS request to define or get default values for the entire session, or they MAY BE sent in a SPEAK request to define default values for just that speak request 🌸 Plus, these attributes can also be part of the speech text marked up in Speech Synthesis Markup Language (SSML) 💅
These voice parameter header fields can also be sent in a CONTROL method to change a SPEAK request that's already in progress, so the behavior shifts on the fly 🦋 If the synthesizer resource doesn't support this operation, it should respond back to the client with a status of unsupported 🙈
7.4.6. Prosody-Parameters 🌈
this set of params defines the prosody of the speech, aka the rhythm and vibe of how it sounds 🎶
prosody-parameter = "Prosody-" prosody-param-name ":"
prosody-param-value CRLFprosody-param-name can be any one of the attribute names under the prosody element in W3C's Speech Synthesis Markup Language Specification [9] 📖 prosody-param-value is any one of the value choices for the matching prosody element attribute, as listed in the section above.
These header fields MAY BE sent in a SET-PARAMS/GET-PARAMS request to define or get default values for the entire session, or they MAY BE sent in a SPEAK request to define default values for that speak request 🍓 Again, these attributes can also be part of the speech text marked up in SSML ✨
The prosody parameter header fields in the SET-PARAMS or SPEAK request only apply if the speech data is of type text/plain and does not use a speech markup format. Read that twice, it matters 🙈
These prosody parameter header fields MAY also be sent in a CONTROL method to affect a SPEAK request in progress and change its behavior on the fly 🔥 If the synthesizer resource doesn't support this operation, it should respond back to the client with a status of unsupported 💀
7.4.7. Vendor-Specific Parameters 💜
this set of headers lets the client set vendor-specific parameters 🧚♀️
vendor-specific = "Vendor-Specific-Parameters" ":"
vendor-specific-av-pair
*[";" vendor-specific-av-pair] CRLF
vendor-specific-av-pair = vendor-av-pair-name "="
vendor-av-pair-valueThis header MAY BE sent in the SET-PARAMS/GET-PARAMS method and is used to set vendor-specific parameters on the server side 🖥️ The vendor-av-pair-name can be any vendor-specific field name and follows the XML vendor-specific attribute naming convention 📝 The vendor-av-pair-value is the value you're setting the attribute to, and it needs to be quoted 🎀
When you want the server to tell you the current value of these parameters, this header can go in the GET-PARAMS method with a list of the vendor-specific attribute names you want, separated by a semicolon 🫶
7.4.8. Speech Marker 🍓
this header field holds a marker tag that can be embedded in the speech data 🏷️ Most speech markup formats have ways to embed marker fields between speech texts. The synthesizer generates SPEECH-MARKER events when it reaches those marker fields. This field SHOULD be part of the SPEECH-MARKER event, and it contains the marker tag values 🦋
speech-marker = "Speech-Marker" ":" 1*ALPHA CRLF7.4.9. Speech Language 🌍
this header field specifies the default language of the speech data, but only if the speech data doesn't already say what language it's in 🗣️ Its values should follow RFC 3066 [16] 📚 This MAY occur in SPEAK, SET-PARAMS, or GET-PARAMS requests 💖
speech-language = "Speech-Language" ":" 1*ALPHA CRLF7.4.10. Fetch Hint ⭐
when the synthesizer needs to fetch documents or other resources like speech markup or audio files, this header field controls the URI access properties 🧲 It defines when the synthesizer should retrieve content from the server. A value of "prefetch" means a file may be downloaded as soon as the request is received, whereas "safe" means a file should only be downloaded when it's actually needed 🌸 The default value is "prefetch" ✨ This header field MAY occur in SPEAK, SET-PARAMS, or GET-PARAMS requests 🫶
fetch-hint = "Fetch-Hint" ":" 1*ALPHA CRLF7.4.11. Audio Fetch Hint 🦋
same energy as the last one, but this time it's about audio files specifically 🎧 When the synthesizer needs to fetch things like speech audio files, this header field controls the URI access properties. It defines whether the synthesizer can try to speed things up by pre-fetching audio. The value is either "safe", which means audio is only fetched when it's needed and never before; "prefetch", which permits but doesn't require the platform to pre-fetch the audio; or "stream", which lets it stream the audio fetches 🌊 The default value is "prefetch" 💅 This header field MAY occur in SPEAK, SET-PARAMS, or GET-PARAMS requests 💜
audio-fetch-hint = "Audio-Fetch-Hint" ":" 1*ALPHA CRLF7.4.12. Fetch Timeout ⏳
when the synthesizer needs to fetch documents or other resources like speech audio files, this header field controls the URI access properties again 🔁 It sets the synthesizer's timeout for resources the media server may need to fetch from the network. It's specified in milliseconds ⏱️ The default value is platform-dependent 🤷♀️ This header field MAY occur in SPEAK, SET-PARAMS, or GET-PARAMS 🎀
fetch-timeout = "Fetch-Timeout" ":" 1*DIGIT CRLF7.4.13. Failed URI 🔗
okay so when a synthesizer method needs the synthesizer to fetch or access a URI and that access fails 💀, the media server SHOULD give you the failed URI in this header field in the method response 🫶.
failed-uri = "Failed-URI" ":" Url CRLF7.4.14. Failed URI Cause 🙈
same energy here bestie: when a synthesizer method needs to fetch or access a URI and it fails, the media server SHOULD send the URI specific or protocol-specific response code through this header field in the method response. this field is alphanumeric so it can handle every protocol, because some of them send back a response string instead of a numeric code 🧚
failed-uri-cause = "Failed-URI-Cause" ":" 1*ALPHA CRLF7.4.15. Speak Restart ⏪
picture this: a CONTROL jump backward request hits a synthesizer resource that's currently speaking, and the jump goes beyond the start of the speech. the current SPEAK request restarts from the very beginning of its speech data, and the CONTROL response would include this header to say "restart happened" 🔄. this header MAY occur in the CONTROL response ✨.
speak-restart = "Speak-Restart" ":" boolean-value CRLF7.4.16. Speak Length 📏
this parameter MAY BE specified in a CONTROL method to control how much speech gets spoken, measured relative to the current speaking point in the active SPEAK request. a "-" value is illegal in this field 🚫. if a field with a Tag unit is specified, the media must speak until the tag is reached or the SPEAK request completes, whichever comes first. this also MAY BE specified in a SPEAK request to set the length to speak in the speech data, relative to where that SPEAK request starts. the speech length units supported depend on the synthesizer implementation 🎀. if it doesn't support a unit or the operation, the resource SHOULD respond with a status code of 404 "Illegal or Unsupported value for parameter" 💅.
speak-length = "Speak-Length" ":" speech-length-value
CRLF7.5. Synthesizer Message Body 💬
a synthesizer message can carry extra info in its message body, tied to the Method, Response, or Event it belongs to 🌸.
7.5.1. Synthesizer Speech Data 🗣️
the marked-up text for the synthesizer to speak is sent as a MIME entity in the message body. you can send the message inline (embedded right in the body) or by reference (giving a URI to the data) 🦋. either way, the data and the markup format used for the speech have to be supported by the media server, no exceptions 👑.
all media servers MUST support plain text speech data and W3C's Speech Synthesis Markup Language [9] at a minimum, so they MUST support the MIME types text/plain and application/synthesis+ssml at a minimum 💖.
if the speech data is specified by URI reference, the MIME type text/uri-list is used to list one or more URIs that point to what needs to be spoken 🔗. if you give a list of speech URIs, the speech data from each URI gets spoken in the order the URIs are listed ✨.
if the data is a mix of URIs and inline speech data, you use the multipart/mixed MIME type and embed MIME blocks for text/uri-list, application/synthesis+ssml or text/plain 🧩. the character set and encoding can be specified following the standard MIME type definitions. the multi-part MIME block can also hold actual audio data in .wav or Sun audio format 🎧. this is for when the client has audio clips it recorded, stored in memory or on a local device, and needs to play as part of the SPEAK request. the client can send those audio MIME parts as part of the multi-part MIME block, and the speech markup (which is another part in that block, per the multipart/mixed spec) references the audio 🫶.
Example 1:
Content-Type:text/uri-list
Content-Length:176
http://www.cisco.com/ASR-Introduction.sml
http://www.cisco.com/ASR-Document-Part1.sml
http://www.cisco.com/ASR-Document-Part2.sml
http://www.cisco.com/ASR-Conclusion.smlExample 2:
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<sentence>The subject is <prosody
rate="-20%">ski trip</prosody></sentence>
</paragraph>
</speak>Example 3:
Content-Type:multipart/mixed; boundary="--break"
--break
Content-Type:text/uri-list
Content-Length:176
http://www.cisco.com/ASR-Introduction.sml
http://www.cisco.com/ASR-Document-Part1.sml
http://www.cisco.com/ASR-Document-Part2.sml
http://www.cisco.com/ASR-Conclusion.sml
--break
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<sentence>The subject is <prosody
rate="-20%">ski trip</prosody></sentence>
</paragraph>
</speak>
--break7.6. SET-PARAMS ⚙️
the SET-PARAMS method goes from client to server and tells the synthesizer resource to set its default synthesizer context parameters, like voice characteristics and prosody 🎛️. if the server accepts and sets every parameter, it MUST return a Response-Status of 200 🏆. if it chooses to ignore some optional parameters, it MUST return 201 🙃.
if some of the parameters being set are unsupported or have illegal values, the server still accepts and sets the rest, and it MUST respond with a Response-Status of 403 or 404 🚨. it MUST also include in the response the header fields that could not be set 📝.
Example:
C->S:SET-PARAMS 543256 MRCP/1.0
Voice-gender:female
Voice-category:adult
Voice-variant:3
S->C:MRCP/1.0 543256 200 COMPLETE7.7. GET-PARAMS 🔍
the GET-PARAMS method goes from client to server and asks the synthesizer resource for its current synthesizer context parameters, like voice characteristics and prosody 👀. the client SHOULD list the parameters it wants to read by sending a set of empty parameter header fields 📋. if no specific list is given, the server SHOULD return all the settable parameters, including vendor-specific parameters and their current values 🌈. but the wildcard use can be super intensive, since the number of settable parameters can be huge depending on the vendor 😭. so it is RECOMMENDED that the client does not use the wildcard GET-PARAMS operation very often 🫠.
Example:
C->S:GET-PARAMS 543256 MRCP/1.0
Voice-gender:
Voice-category:
Voice-variant:
Vendor-Specific-Parameters:com.mycorp.param1;
com.mycorp.param2
S->C:MRCP/1.0 543256 200 COMPLETE
Voice-gender:female
Voice-category:adult
Voice-variant:3
Vendor-Specific-Parameters:com.mycorp.param1="Company Name";
com.mycorp.param2="124324234@mycorp.com"7.8. SPEAK 🎤
the SPEAK method goes from client to server and gives the synthesizer resource the speech text, kicking off speech synthesis and streaming 🚀. SPEAK can carry voice and prosody header fields that define how the voice behaves, plus the actual marked-up text to say 💅. if specific voice and prosody parameters are set inside the speech markup text, those take precedence over the values in the header fields and over anything set with an earlier SET-PARAMS request 👑.
there are 3 levels of scope when applying voice parameters 🧠. the highest precedence is whatever's specified inside the speech markup text. next are the ones in the header fields of the SPEAK request, which only apply to that SPEAK request. last are the session default values set with SET-PARAMS, which apply for the whole session going forward 🦋.
if the resource is idle and the SPEAK request is being actively processed, the resource responds with a success status code and a request-state of IN-PROGRESS ✅.
if the resource is in the speaking or paused states (so it's in the middle of processing a previous SPEAK request), the status comes back as success with a request-state of PENDING ⏳. that means this SPEAK request is queued up and gets processed after the currently active SPEAK request finishes 🎀.
for the synthesizer resource, SPEAK is the only request that can return a request-state of IN-PROGRESS or PENDING 💜. when the text is done being synthesized, the resource issues a SPEAK-COMPLETE event with the request-id of the SPEAK message and a request-state of COMPLETE 🏁.
Example:
C->S:SPEAK 543257 MRCP/1.0
Voice-gender:neutral
Voice-category:teenager
Prosody-volume:medium
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<sentence>The subject is <prosody
rate="-20%">ski trip</prosody></sentence>
</paragraph>
</speak>
S->C:MRCP/1.0 543257 200 IN-PROGRESS
S->C:SPEAK-COMPLETE 543257 COMPLETE MRCP/1.0
Completion-Cause:000 normal7.9. STOP 🛑
The STOP method goes from the client to the server, and it tells the resource to stop speaking if it's currently talking. Literally the "hush, bestie" move 🤫
The STOP request can include an active-request-id-list header field to stop zero or more specific SPEAK requests that might be sitting in the queue, and it gets back a 200(Success) response code. ✨ If there's no active-request-id-list header field in the STOP request, it terminates all outstanding SPEAK requests. Total clean sweep 🧹
If a STOP request successfully kills one or more PENDING or IN-PROGRESS SPEAK requests, the response message body has an active-request-id-list header field listing the SPEAK request-ids that got terminated. Otherwise, there's no active-request-id-list header field in the response at all. 💀 No SPEAK-COMPLETE events get sent for these terminated requests, so don't go looking for them 🙈
If a SPEAK request that was IN-PROGRESS and speaking got stopped, the next pending SPEAK request (if there is one) becomes IN-PROGRESS and moves to the speaking state. Queue glow-up 💅
If a SPEAK request that was IN-PROGRESS and in the paused state got stopped, the next pending SPEAK request (if any) becomes IN-PROGRESS and moves to the paused state. Same energy, just on pause ⏸️
Example:
C->S:SPEAK 543258 MRCP/1.0
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<sentence>The subject is <prosody
rate="-20%">ski trip</prosody></sentence>
</paragraph>
</speak>
S->C:MRCP/1.0 543258 200 IN-PROGRESS
C->S:STOP 543259 200 MRCP/1.0
S->C:MRCP/1.0 543259 200 COMPLETE
Active-Request-Id-List:5432587.10. BARGE-IN-OCCURRED 🎀
The BARGE-IN-OCCURRED method is how the client tells the speech resource about a barge-in-able event it spotted. It's the "excuse me, I'm interrupting" move 🙋♀️
This event is useful in two scenarios:
- The client has detected some events like DTMF digits or other barge-in-able events and wants to communicate that to the synthesizer.
- The recognizer resource and the synthesizer resource are in different servers. In which case the client MUST act as a Proxy and receive event from the recognition resource, and then send a BARGE-IN-OCCURRED method to the synthesizer. In such cases, the BARGE-IN-OCCURRED method would also have a
proxy-sync-idheader field received from the resource generating the original event. 🔁
If a SPEAK request is active with kill-on-barge-in enabled, and the BARGE-IN-OCCURRED event is received, the synthesizer should stop streaming out audio. ✋ It should also terminate any speech requests queued behind the current active one, irrespective of whether they have barge-in enabled or not. If a barge-in-able prompt was playing and it got terminated, the response MUST contain the request-ids of all SPEAK requests that were terminated in its active-request-id-list. There will be no SPEAK-COMPLETE events generated for these requests. Gone, gone, gone 👋
If the synthesizer and the recognizer are on the same server, they could be optimized for a quicker kill-on-barge-in response by having them interact directly based on a common RTSP session-id. ⚡ In these cases, the client MUST still proxy the recognition event through a BARGE-IN-OCCURRED method, but the synthesizer resource may have already stopped and sent a SPEAK-COMPLETE event with a barge-in completion cause code. If there were no SPEAK requests terminated as a result of the BARGE-IN-OCCURRED method, the response would still be a 200 success, but MUST not contain an active-request-id-list header field. 🙃
C->S:SPEAK 543258 MRCP/1.0
Voice-gender:neutral
Voice-category:teenager
Prosody-volume:medium
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<sentence>The subject is <prosody
rate="-20%">ski trip</prosody></sentence>
</paragraph>
</speak>
S->C:MRCP/1.0 543258 200 IN-PROGRESS
C->S:BARGE-IN-OCCURRED 543259 200 MRCP/1.0
Proxy-Sync-Id:987654321
S->C:MRCP/1.0 543259 200 COMPLETE
Active-Request-Id-List:5432587.11. PAUSE ⏸️
The PAUSE method goes from the client to the server and tells the resource to pause speech, if it's speaking something. Pause it, pause it, pause it 🫶
If a PAUSE method is issued on a session when a SPEAK is not active, the server SHOULD respond with a status of 402 or "Method not valid in this state". 🙅♀️ If a PAUSE method is issued on a session when a SPEAK is active and paused, the server SHOULD respond with a status of 200 or "Success". If a SPEAK request was active, the server MUST return an active-request-id-list header with the request-id of the SPEAK request that was paused. Receipts, always 🧾
C->S:SPEAK 543258 MRCP/1.0
Voice-gender:neutral
Voice-category:teenager
Prosody-volume:medium
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<sentence>The subject is <prosody
rate="-20%">ski trip</prosody></sentence>
</paragraph>
</speak>
S->C:MRCP/1.0 543258 200 IN-PROGRESS
C->S:PAUSE 543259 MRCP/1.0
S->C:MRCP/1.0 543259 200 COMPLETE
Active-Request-Id-List:5432587.12. RESUME ▶️
The RESUME method goes from the client to the server and tells a paused synthesizer resource to pick up where it left off and keep talking. Unpause era 🌈
If a RESUME method is issued on a session when a SPEAK is not active, the server SHOULD respond with a status of 402 or "Method not valid in this state". If a RESUME method is issued on a session when a SPEAK is active and speaking (i.e., not paused), the server SHOULD respond with a status of 200 or "Success". If a SPEAK request was active, the server MUST return an active-request-id-list header with the request-id of the SPEAK request that was resumed.
Example:
C->S:SPEAK 543258 MRCP/1.0
Voice-gender:neutral
Voice-category:teenager
Prosody-volume:medium
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<sentence>The subject is <prosody
rate="-20%">ski trip</prosody></sentence>
</paragraph>
</speak>
S->C:MRCP/1.0 543258 200 IN-PROGRESS
C->S:PAUSE 543259 MRCP/1.0
S->C:MRCP/1.0 543259 200 COMPLETE
Active-Request-Id-List:543258
C->S:RESUME 543260 MRCP/1.0
S->C:MRCP/1.0 543260 200 COMPLETE
Active-Request-Id-List:5432587.13. CONTROL 🎛️
The CONTROL method goes from the client to the server and tells a synthesizer that's already speaking to change what it's saying on the fly. It's giving remote control, and it's iconic 📺 This method is used to make the synthesizer jump forward or backward in what it's being spoken, change speaker rate and speaker parameters, etc. It affects the active or IN-PROGRESS SPEAK request. Depending on the implementation and capability of the synthesizer resource, it may allow this operation or one or more of its parameters. Not every synth is built the same, babe 💁♀️
When a CONTROL to jump forward is issued and the operation goes beyond the end of the active SPEAK method's text, the request succeeds. A SPEAK-COMPLETE event follows the response to the CONTROL method. If there are more SPEAK requests in the queue, the synthesizer resource will continue to process the next SPEAK method. ⏭️ When a CONTROL to jump backwards is issued and the operation jumps to the beginning of the speech data of the active SPEAK request, the response to the CONTROL request contains the speak-restart header. ⏮️
These two behaviors can be used to rewind or fast-forward across multiple speech requests, if the client wants to break up a speech markup text into multiple SPEAK requests. Scrubbing through your audio like a pro 🎬
If a SPEAK request was active when the CONTROL method was received, the server MUST return an active-request-id-list header with the Request-id of the SPEAK request that was active.
Example:
C->S:SPEAK 543258 MRCP/1.0
Voice-gender:neutral
Voice-category:teenager
Prosody-volume:medium
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<sentence>The subject is <prosody
rate="-20%">ski trip</prosody></sentence>
</paragraph>
</speak>
S->C:MRCP/1.0 543258 200 IN-PROGRESS
C->S:CONTROL 543259 MRCP/1.0
Prosody-rate:fast
S->C:MRCP/1.0 543259 200 COMPLETE
Active-Request-Id-List:543258
C->S:CONTROL 543260 MRCP/1.0
Jump-Size:-15 Words
S->C:MRCP/1.0 543260 200 COMPLETE
Active-Request-Id-List:5432587.14. SPEAK-COMPLETE 🏁
This is an Event message from the synthesizer resource to the client, and it says the SPEAK request is done. Main character exit moment 🎬 The request-id header field WILL match the request-id of the SPEAK request that started the speech that just finished. The request-state field should be COMPLETE, which signals that this is the last Event with that request-id, and that the request with that request-id is now complete. ✅ The completion-cause header field gives the cause code for how the request wrapped up, like whether the SPEAK finished normally, hit an error, or got cut off by kill-on-barge-in, etc. 🧐
Example:
C->S:SPEAK 543260 MRCP/1.0
Voice-gender:neutral
Voice-category:teenager
Prosody-volume:medium
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<sentence>The subject is <prosody
rate="-20%">ski trip</prosody></sentence>
</paragraph>
</speak>
S->C:MRCP/1.0 543260 200 IN-PROGRESS
S->C:SPEAK-COMPLETE 543260 COMPLETE MRCP/1.0
Completion-Cause:000 normal7.15. SPEECH-MARKER 🧚
okay so this one's an event the synthesizer sends the client when it hits a marker tag in the speech markup it's currently working through 💅 the request-id in the header matches the SPEAK request that kicked things off, and the request-state is IN-PROGRESS because the speech isn't done yet, there's still more to say 🎀 the marker it just hit gets returned in the speech-marker header field, so the client knows exactly where the synthesizer is in the markup 📍
Example:
C->S:SPEAK 543261 MRCP/1.0
Voice-gender:neutral
Voice-category:teenager
Prosody-volume:medium
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<mark name="here"/>
<sentence>The subject is
<prosody rate="-20%">ski trip</prosody>
</sentence>
<mark name="ANSWER"/>
</paragraph>
</speak>
S->C:MRCP/1.0 543261 200 IN-PROGRESS
S->C:SPEECH-MARKER 543261 IN-PROGRESS MRCP/1.0
Speech-Marker:here
S->C:SPEECH-MARKER 543261 IN-PROGRESS MRCP/1.0
Speech-Marker:ANSWER
S->C:SPEAK-COMPLETE 543261 COMPLETE MRCP/1.0
Completion-Cause:000 normal8. Speech Recognizer Resource 🎧
okay the speech recognizer is literally the queen of listening 👑 it takes in an incoming voice stream and gives the client back an interpretation of what was said, in text form 📝
8.1. Recognizer State Machine 🔄
the recognizer is driven by MRCP requests from the client 💌 but it can also answer those requests or send asynchronous events to the server to flag certain things happening while it processes the stream 👀 so the recognizer keeps track of states to match up each request from the client, and the transitions are laid out below ✨
Idle Recognizing Recognized
State State State
| | |
|---------RECOGNIZE---->|---RECOGNITION-COMPLETE-->|
|<------STOP------------|<-----RECOGNIZE-----------|
| | |
| | |-----------|
| |--------| GET-RESULT |
| START-OF-SPEECH | |---------->|
|------------| |------->| |
| | |----------| |
| DEFINE-GRAMMAR | RECOGNITION-START-TIMERS |
|<-----------| |<---------| |
| | |
| | |
|-------| | |
| STOP | |
|<------| | |
| |
|<-------------------STOP--------------------------|
|<-------------------DEFINE-GRAMMAR----------------|8.2. Recognizer Methods 🛠️
the recognizer supports these methods, and here's the full lineup 💅
recognizer-method = SET-PARAMS
/ GET-PARAMS
/ DEFINE-GRAMMAR
/ RECOGNIZE
/ GET-RESULT
/ RECOGNITION-START-TIMERS
/ STOP8.3. Recognizer Events 📣
the recognizer can generate these events, and that's the whole list 🫶
recognizer-event = START-OF-SPEECH
/ RECOGNITION-COMPLETE8.4. Recognizer Header Fields 🧾
a recognizer message can carry header fields with request options and extra info that go along with the Method, Response, or Event message it's attached to 🎀 here's the full set, with a pointer to the section for each one 🔍
recognizer-header = confidence-threshold ; Section 8.4.1
/ sensitivity-level ; Section 8.4.2
/ speed-vs-accuracy ; Section 8.4.3
/ n-best-list-length ; Section 8.4.4
/ no-input-timeout ; Section 8.4.5
/ recognition-timeout ; Section 8.4.6
/ waveform-url ; Section 8.4.7
/ completion-cause ; Section 8.4.8
/ recognizer-context-block ; Section 8.4.9
/ recognizer-start-timers ; Section 8.4.10
/ vendor-specific ; Section 8.4.11
/ speech-complete-timeout ; Section 8.4.12
/ speech-incomplete-timeout; Section 8.4.13
/ dtmf-interdigit-timeout ; Section 8.4.14
/ dtmf-term-timeout ; Section 8.4.15
/ dtmf-term-char ; Section 8.4.16
/ fetch-timeout ; Section 8.4.17
/ failed-uri ; Section 8.4.18
/ failed-uri-cause ; Section 8.4.19
/ save-waveform ; Section 8.4.20
/ new-audio-channel ; Section 8.4.21
/ speech-language ; Section 8.4.22
Parameter Support Methods/Events
confidence-threshold MANDATORY SET-PARAMS, RECOGNIZE
GET-RESULT
sensitivity-level Optional SET-PARAMS, GET-PARAMS,
RECOGNIZE
speed-vs-accuracy Optional SET-PARAMS, GET-PARAMS,
RECOGNIZE
n-best-list-length Optional SET-PARAMS, GET-PARAMS,
RECOGNIZE, GET-RESULT
no-input-timeout MANDATORY SET-PARAMS, GET-PARAMS,
RECOGNIZE
recognition-timeout MANDATORY SET-PARAMS, GET-PARAMS,
RECOGNIZE
waveform-url MANDATORY RECOGNITION-COMPLETE
completion-cause MANDATORY DEFINE-GRAMMAR, RECOGNIZE,
RECOGNITON-COMPLETE
recognizer-context-block Optional SET-PARAMS, GET-PARAMS
recognizer-start-timers MANDATORY RECOGNIZE
vendor-specific MANDATORY SET-PARAMS, GET-PARAMS
speech-complete-timeout MANDATORY SET-PARAMS, GET-PARAMS
RECOGNIZE
speech-incomplete-timeout MANDATORY SET-PARAMS, GET-PARAMS
RECOGNIZE
dtmf-interdigit-timeout MANDATORY SET-PARAMS, GET-PARAMS
RECOGNIZE
dtmf-term-timeout MANDATORY SET-PARAMS, GET-PARAMS
RECOGNIZE
dtmf-term-char MANDATORY SET-PARAMS, GET-PARAMS
RECOGNIZE
fetch-timeout MANDATORY SET-PARAMS, GET-PARAMS
RECOGNIZE, DEFINE-GRAMMAR
failed-uri MANDATORY DEFINE-GRAMMAR response,
RECOGNITION-COMPLETE
failed-uri-cause MANDATORY DEFINE-GRAMMAR response,
RECOGNITION-COMPLETE
save-waveform MANDATORY SET-PARAMS, GET-PARAMS,
RECOGNIZE
new-audio-channel MANDATORY RECOGNIZE
speech-language MANDATORY SET-PARAMS, GET-PARAMS,
RECOGNIZE, DEFINE-GRAMMAR8.4.1. Confidence Threshold 🎯
when the recognition resource matches a spoken phrase to some part of the grammar, it attaches a confidence level to that conclusion 💯 the confidence-threshold parameter tells the recognizer which confidence level counts as a successful match 🙌 it's an integer from 0-100 showing how sure the recognizer is about its recognition ✨ if the recognizer decides that its confidence in every recognition result is below the confidence threshold, then it MUST return no-match as the recognition result 🚫 this header field MAY occur in RECOGNIZE, SET-PARAMS, or GET-PARAMS 💜 the default value is platform specific 🌸
confidence-threshold = "Confidence-Threshold" ":" 1*DIGIT CRLF8.4.2. Sensitivity Level 🎙️
to filter out background noise so it doesn't get mistaken for speech 🙈 the recognizer MAY support a variable level of sound sensitivity 🔊 the sensitivity-level parameter lets the client set that value on the recognizer 💖 this header field MAY occur in RECOGNIZE, SET-PARAMS, or GET-PARAMS 💅 a higher value means higher sensitivity 📈 the default value is platform specific 🌈
sensitivity-level = "Sensitivity-Level" ":" 1*DIGIT CRLF8.4.3. Speed Vs Accuracy ⚡
depending on the implementation and capability of the recognizer resource, it may be tunable toward Performance or Accuracy 🎢 higher accuracy may mean more processing and higher CPU utilization, which means fewer calls per media server, and vice versa 🖥️ this parameter can be tuned on the resource with the speed-vs-accuracy header 🔧 this header field MAY occur in RECOGNIZE, SET-PARAMS, or GET-PARAMS 💫 a higher value means higher speed 🏎️ the default value is platform specific 🍓
speed-vs-accuracy = "Speed-Vs-Accuracy" ":" 1*DIGIT CRLF8.4.4. N Best List Length 🏆
sometimes the recognizer matches an incoming stream against the grammar and comes up with more than one possible match, because of confidence levels on certain words or conversation paths 🤔 if this header field isn't specified, the recognition resource only returns the best match above the confidence threshold by default 🥇 the client can set this parameter to ask the recognition resource to send back more than one alternative 🙋♀️ every alternative still has to be above the confidence-threshold 🎯 a value greater than one does NOT guarantee the recognizer will send the requested number of alternatives, so don't count on it 🫠 this header field MAY occur in RECOGNIZE, SET-PARAMS, or GET-PARAMS 💜 the minimum value is 1, and the default value is also 1 🧚
n-best-list-length = "N-Best-List-Length" ":" 1*DIGIT CRLF8.4.5. No Input Timeout ⏰
when recognition starts and no speech gets detected for a certain amount of time, the recognizer can send a RECOGNITION-COMPLETE event to the client and end the recognition operation 🛑 the no-input-timeout header field sets that timeout value, in milliseconds ⏱️ this header field MAY occur in RECOGNIZE, SET-PARAMS, or GET-PARAMS 🦋 the value ranges from 0 to MAXTIMEOUT, where MAXTIMEOUT is platform specific 🌟 the default value is platform specific too 🎀
no-input-timeout = "No-Input-Timeout" ":" 1*DIGIT CRLF8.4.6. Recognition Timeout ✨
okay so, when recognition starts and nothing gets matched for a while, the recognizer can send a RECOGNITION-COMPLETE event to the client and end the recognition operation 💅 the recognition-timeout parameter field sets that timeout, measured in milliseconds 🌸 the value can run from 0 up to MAXTIMEOUT, and MAXTIMEOUT depends on the platform 🦋 the default is 10 seconds. This header field MAY occur in RECOGNIZE, SET-PARAMS or GET-PARAMS 💜
recognition-timeout = "Recognition-Timeout" ":" 1*DIGIT CRLF8.4.7. Waveform URL 🎀
bestie, this one is a whole vibe 🎧 if the save-waveform header field is set to true, the recognizer MUST record the incoming audio stream of the recognition into a file and give the client a URI to get to it 🔥 this header MUST be present in the RECOGNITION-COMPLETE event whenever save-waveform was set to true. If some error stopped the server from recording, the URL value of the header MUST be NULL 💀 otherwise the generated URL SHOULD be globally unique across the server and all of its recognition sessions ✨ and the URL SHOULD BE available until the session gets torn down 🫶
waveform-url = "Waveform-URL" ":" Url CRLF8.4.8. Completion Cause 🦋
this header field MUST be part of a RECOGNITION-COMPLETE event that goes from the recognizer resource to the client 😭 it tells you why the RECOGNIZE method finished 🧚 it MUST BE sent in the DEFINE-GRAMMAR and RECOGNIZE responses too, but only when they come back with a failure status and a COMPLETE state. Here's the whole lineup:
Cause-Code Cause-Name Description
000 success RECOGNIZE completed with a match or
DEFINE-GRAMMAR succeeded in
downloading and compiling the
grammar
001 no-match RECOGNIZE completed, but no match
was found
002 no-input-timeout
RECOGNIZE completed without a match
due to a no-input-timeout
003 recognition-timeout
RECOGNIZE completed without a match
due to a recognition-timeout
004 gram-load-failure
RECOGNIZE failed due grammar load
failure.
005 gram-comp-failure
RECOGNIZE failed due to grammar
compilation failure.
006 error RECOGNIZE request terminated
prematurely due to a recognizer
error.
007 speech-too-early
RECOGNIZE request terminated because
speech was too early.
008 too-much-speech-timeout
RECOGNIZE request terminated because
speech was too long.
009 uri-failure Failure accessing a URI.
010 language-unsupported
Language not supported.8.4.9. Recognizer Context Block 🌈
this parameter MAY BE sent as part of a SET-PARAMS or GET-PARAMS request 💖 if GET-PARAMS includes this header field with no value, that's the client asking the recognizer to hand back its recognizer context block 🍓 the response to that MAY contain a recognizer context block as a message entity. If the server does return a context block, the response MUST contain this header field, and its value MUST match the content-id of that entity 🔥
If SET-PARAMS includes this header field, it MUST contain a message entity holding the recognizer context data, plus a content-id that matches this header field 💅 that content-id should match the one that came along with the context data during the GET-PARAMS operation 🫶
recognizer-context-block = "Recognizer-Context-Block" ":"
1*ALPHA CRLF8.4.10. Recognition Start Timers ⏱️
this parameter MAY BE sent as part of the RECOGNIZE request 🌸 a value of "false" tells the recognizer to start recognizing but not start the no-input timer yet 🙈 the recognizer should not start those timers until the client sends a RECOGNITION-START-TIMERS request to it 🦋 this is super useful when the recognizer and synthesizer engines aren't in the same session. Picture it: a kill-on-barge-in prompt is playing, and you need the RECOGNIZE request active at the same time so it can catch the barge-in and act on it 🎀 but you don't want the no-input timers ticking until the prompt is done 👑 the default is "true" ✨
recognizer-start-timers = "Recognizer-Start-Timers" ":"
boolean-value CRLF8.4.11. Vendor Specific Parameters 🧚
this set of headers lets the client set vendor-specific parameters 💜 it can be sent in the SET-PARAMS method to set vendor-specific parameters on the server 💅 the vendor-av-pair-name can be any vendor-specific field name, as long as it follows the XML vendor-specific attribute naming convention 🌈 the vendor-av-pair-value is what you set the attribute to, and it needs to be quoted 🫶
when you want to ask the server for the current values of these parameters, the header can go in the GET-PARAMS method with a semicolon-separated list of the vendor-specific attribute names you're after 🍓 this header field MAY occur in SET-PARAMS or GET-PARAMS ✨
8.4.12. Speech Complete Timeout 🎀
this header field sets how much silence you need after the user's speech before the recognizer finalizes a result (either accepting it or throwing a nomatch event) 🌸 the speech-complete-timeout gets used when the recognizer has a complete match of an active grammar, and it decides how long to wait for more input before declaring a match 🦋 the incomplete timeout is the other one, used when the speech is an incomplete match to an active grammar 💖 the value is in milliseconds.
speech-complete-timeout = "Speech-Complete-Timeout" ":"
1*DIGIT CRLFA long speech-complete-timeout delays the result and makes the computer's response feel slow 🐢 a short one can chop an utterance up in the wrong places 😭 reasonable complete timeout values are typically somewhere between 0.3 seconds and 1.0 seconds 💅 the value can run from 0 to MAXTIMEOUT, which is platform specific, and the default is platform specific too 🌈 this header field MAY occur in RECOGNIZE, SET-PARAMS, or GET-PARAMS 🫶
8.4.13. Speech Incomplete Timeout 🦋
this header field sets the required silence after user speech before a recognizer finalizes a result 🔥 the incomplete timeout applies when the speech before the silence is an incomplete match of all active grammars 💅 once it triggers, the partial result gets rejected (with a nomatch event) 🙈 the value is in milliseconds, it ranges from 0 to MAXTIMEOUT (platform specific), and the default is platform specific ✨
speech-incomplete-timeout = "Speech-Incomplete-Timeout" ":"
1*DIGIT CRLFThe speech-incomplete-timeout also applies when the speech before the silence is a complete match of an active grammar, but it's still possible to say more and keep matching the grammar 🧚 the complete timeout, by contrast, is for when the speech is a complete match and no further words can be spoken 👑
A long speech-incomplete-timeout also delays the result and makes the response slow 🐢 and a short one can break an utterance up inappropriately 😭 the speech-incomplete-timeout is usually longer than the speech-complete-timeout so users can pause mid-utterance (to breathe, for example) 🫶 this header field MAY occur in RECOGNIZE, SET-PARAMS, or GET-PARAMS 💜
8.4.14. DTMF Interdigit Timeout 🌸
this header field sets the inter-digit timeout to use when recognizing DTMF input 🎀 the value is in milliseconds, it ranges from 0 to MAXTIMEOUT (platform specific), and the default is 5 seconds 🦋 this header field MAY occur in RECOGNIZE, SET-PARAMS, or GET-PARAMS ✨
dtmf-interdigit-timeout = "DTMF-Interdigit-Timeout" ":"
1*DIGIT CRLF8.4.15. DTMF Term Timeout 🍓
this header field sets the terminating timeout for DTMF input recognition 💅 the value is in milliseconds, it ranges from 0 to MAXTIMEOUT (platform specific), and the default is 10 seconds 🌈 this header field MAY occur in RECOGNIZE, SET-PARAMS, or GET-PARAMS 🫶
dtmf-term-timeout = "DTMF-Term-Timeout" ":" 1*DIGIT CRLF8.4.16. DTMF-Term-Char 💜
this header field sets the terminating DTMF character for DTMF input recognition 🧚 the default is NULL, which is specified as an empty header field 🙈 this header field MAY occur in RECOGNIZE, SET-PARAMS, or GET-PARAMS 🌸
dtmf-term-char = "DTMF-Term-Char" ":" CHAR CRLF8.4.17. Fetch Timeout 🦋
when the recognizer needs to fetch grammar documents, this header field controls the URI access properties 🔥 it sets the recognizer's timeout for finishing the fetch of the resources the media server needs from the network 💅 the value is in milliseconds, it ranges from 0 to MAXTIMEOUT (platform specific), and the default is platform specific too ✨ this header field MAY occur in RECOGNIZE, SET-PARAMS, or GET-PARAMS 🫶
8.4.18. Failed URI 🎀
when a recognizer method needs the recognizer to fetch or access a URI and that access fails, the media server SHOULD provide the failed URI in this header field in the method response 😭 🌸
8.4.19. Failed URI Cause 🌈
same situation as before: a recognizer method needs to fetch or access a URI and it fails 💀 in that case the media server SHOULD provide the URI-specific or protocol-specific response code through this header field in the method response 🧚 this field is defined as alphanumeric on purpose, so it works for every protocol, including the ones that send back a response string instead of a numeric code 👑
8.4.20. Save Waveform 💖
this header field lets the client tell the recognizer that it MUST save the audio stream that was recognized 🎧 the recognizer MUST then record the recognized audio and make it available to the client as a URI, returned in the waveform-uri header field in the RECOGNITION-COMPLETE event 🔥 if there was an error recording the stream, or the audio clip is otherwise not available, the recognizer MUST return an empty waveform-uri header field 💀 the default value for this field is "false" 🫶
save-waveform = "Save-Waveform" ":" boolean-value CRLF8.4.21. New Audio Channel 🎙️✨
okay so this header field MAY BE specified in a RECOGNIZE message 💖 It lets the client tell the media server "heads up, from this point on I'm sending audio from a NEW audio source, channel, or speaker" 📞 If the recognition resource already collected any line statistics or info, it MUST discard it and start fresh for this RECOGNIZE. This is the move when the client MAY want to reuse an open recognition session with the media server for multiple telephone calls 🌸
new-audio-channel = "New-Audio-Channel" ":" boolean-value CRLF8.4.22. Speech Language 🌍💅
This header field specifies the language of the recognition grammar data within a session or request, but only if it isn't specified within the data itself 🫶 The value should follow RFC 3066 [16] for its values. This MAY occur in DEFINE-GRAMMAR, RECOGNIZE, SET-PARAMS, or GET-PARAMS requests 🧚
8.5. Recognizer Message Body 💬
A recognizer message can carry extra data tied to the method, response, or event 🍓 The client may send the grammar to be recognized in DEFINE-GRAMMAR or RECOGNIZE requests. When the grammar is sent in the DEFINE-GRAMMAR method, the server should be able to download, compile and optimize the grammar 🔥 The RECOGNIZE request MUST contain a list of grammars that need to be active during the recognition. The server resource may send the recognition results in the RECOGNITION-COMPLETE event or the GET-RESULT response. This data is carried in the message body of the corresponding MRCP message 👑
8.5.1. Recognizer Grammar Data 📚✨
Recognizer grammar data from the client to the server can be provided inline or by reference, and either way it's carried as MIME entities in the message body of the MRCP request message 🎀 The grammar, whether inline or by reference, is the one used to match during recognition. This data is written in one of the standard grammar specification formats, like W3C's XML or ABNF, or Sun's Java Speech Grammar Format, etc. All media servers MUST support W3C's XML based grammar markup format [11] (MIME-type application/grammar+xml) and SHOULD support the ABNF form (MIME-type application/grammar) 🦋
When a grammar is specified inline in the message, the client MUST provide a content-id for that grammar as part of the content headers. The server MUST store the grammar associated with that content-id for the duration of the session. A stored grammar can be overwritten by defining a new grammar with the same content-id. Grammars that have been associated with a content-id can be referenced through a special "session:" URI scheme 💜
Example:
session:help@root-level.storeIf grammar data needs to be specified by external URI reference, the MIME-type text/uri-list is used to list the one or more URIs that will specify the grammar data. All media servers MUST support the HTTP URI access mechanism 🌈
If the data to be defined is a mix of URI and inline grammar data, the multipart/mixed MIME-type is used and embedded with the MIME-blocks for text/uri-list, application/grammar or application/grammar+xml. The character set and encoding used in the grammar data may be specified according to standard MIME-type definitions 💅
When more than one grammar URI or inline grammar block is specified in a message body of the RECOGNIZE request, it's an active list of grammar alternatives to listen for. The ordering of the list implies the precedence of the grammars, with the first grammar in the list having the highest precedence 👑
Example 1:
Content-Type:application/grammar+xml
Content-Id:request1@form-level.store
Content-Length:104
<?xml version="1.0"?>
<!-- the default grammar language is US English -->
<grammar xml:lang="en-US" version="1.0">
<!-- single language attachment to tokens -->
<rule id="yes">
<one-of>
<item xml:lang="fr-CA">oui</item>
<item xml:lang="en-US">yes</item>
</one-of>
</rule>
<!-- single language attachment to a rule expansion -->
<rule id="request">
may I speak to
<one-of xml:lang="fr-CA">
<item>Michel Tremblay</item>
<item>Andre Roy</item>
</one-of>
</rule>
<!-- multiple language attachment to a token -->
<rule id="people1">
<token lexicon="en-US,fr-CA"> Robert </token>
</rule>
<!-- the equivalent single-language attachment expansion -->
<rule id="people2">
<one-of>
<item xml:lang="en-US">Robert</item>
<item xml:lang="fr-CA">Robert</item>
</one-of>
</rule>
</grammar>Example 2:
Content-Type:text/uri-list
Content-Length:176
session:help@root-level.store
http://www.cisco.com/Directory-Name-List.grxml
http://www.cisco.com/Department-List.grxml
http://www.cisco.com/TAC-Contact-List.grxml
session:menu1@menu-level.storeExample 3:
Content-Type:multipart/mixed; boundary="--break"
--break
Content-Type:text/uri-list
Content-Length:176
http://www.cisco.com/Directory-Name-List.grxml
http://www.cisco.com/Department-List.grxml
http://www.cisco.com/TAC-Contact-List.grxml
--break
Content-Type:application/grammar+xml
Content-Id:request1@form-level.store
Content-Length:104
<?xml version="1.0"?>
<!-- the default grammar language is US English -->
<grammar xml:lang="en-US" version="1.0">
<!-- single language attachment to tokens -->
<rule id="yes">
<one-of>
<item xml:lang="fr-CA">oui</item>
<item xml:lang="en-US">yes</item>
</one-of>
</rule>
<!-- single language attachment to a rule expansion -->
<rule id="request">
may I speak to
<one-of xml:lang="fr-CA">
<item>Michel Tremblay</item>
<item>Andre Roy</item>
</one-of>
</rule>
<!-- multiple language attachment to a token -->
<rule id="people1">
<token lexicon="en-US,fr-CA"> Robert </token>
</rule>
<!-- the equivalent single-language attachment expansion -->
<rule id="people2">
<one-of>
<item xml:lang="en-US">Robert</item>
<item xml:lang="fr-CA">Robert</item>
</one-of>
</rule>
</grammar>
--break8.5.2. Recognizer Result Data 🏆
Recognition result data from the server is carried in the MRCP message body of the RECOGNITION-COMPLETE event or the GET-RESULT response message, as MIME entities. All media servers MUST support W3C's Natural Language Semantics Markup Language (NLSML) [10] as the default standard for returning recognition results to the client, and hence MUST support the MIME-type application/x-nlsml ✨
Example 1:
Content-Type:application/x-nlsml
Content-Length:104
<?xml version="1.0"?>
<result grammar="http://theYesNoGrammar">
<interpretation>
<instance>
<myApp:yes_no>
<response>yes</response>
</myApp:yes_no>
</instance>
<input>ok</input>
</interpretation>
</result>8.5.3. Recognizer Context Block 🧠🌸
When the client has to change recognition servers within a call, this is a block of data that the client MAY collect from the first media server and provide to the second media server. This might happen because the client needs different language support, or because the media server issued an RTSP RE-DIRECT 🔁 The first recognizer may have collected acoustic and other data during its recognition. When we switch recognition servers, passing this data along may allow the second recognition server to provide better recognition based on the acoustic data the previous recognizer collected 🦋 This block of data is vendor-specific and MUST be carried as MIME-type application/octets in the body of the message 🫶
This block of data is communicated in the SET-PARAMS and GET-PARAMS method/response messages. In the GET-PARAMS method, if an empty recognizer-context-block header field is present, then the recognizer should return its vendor-specific context block in the message body as a MIME-entity with a specific content-id. The content-id value should also be specified in the recognizer-context-block header field in the GET-PARAMS response. The SET-PARAMS request wishing to provide this vendor-specific data should send it in the message body as a MIME-entity with the same content-id that it received from the GET-PARAMS. The content-id should also be sent in the recognizer-context-block header field of the SET-PARAMS message 💜
Each automatic speech recognition (ASR) vendor choosing to use this mechanism to hand off recognizer context data among its servers should distinguish its vendor-specific block of data from other vendors by choosing a unique content-id that they should recognize 👑
8.6. SET-PARAMS ⚙️
The SET-PARAMS method, from the client to the server, tells the recognizer resource to set and modify recognizer context parameters like recognizer characteristics, result detail level, etc. Some standard parameters are discussed in the following sections. If the server resource does not recognize an OPTIONAL parameter, it MUST ignore that field 🙈 Lots of the parameters in the SET-PARAMS method can also be used in another method, like the RECOGNIZE method. The difference is that when you set something like the sensitivity-level using SET-PARAMS, it applies to all future requests, whenever applicable. On the other hand, when you pass sensitivity-level in a RECOGNIZE request, it only applies to that one request 💅
Example:
C->S:SET-PARAMS 543256 MRCP/1.0
Sensitivity-Level:20
Recognition-Timeout:30
Confidence-Threshold:85
S->C:MRCP/1.0 543256 200 COMPLETE8.7. GET-PARAMS 🔍
The GET-PARAMS method, from the client to the server, asks the recognizer resource for its current default parameters, like sensitivity-level, n-best-list-length, etc. The client can request specific parameters from the server by sending it one or more empty parameter headers with no values. The server should then return the settings for those specific parameters only. When the client does not send a specific list of empty parameter headers, the recognizer should return the settings for all parameters. The wildcard use can be very intensive, since the number of settable parameters can be large depending on the vendor. Hence, it is RECOMMENDED that the client does not use the wildcard GET-PARAMS operation very often 🥹
Example:
C->S:GET-PARAMS 543256 MRCP/1.0
Sensitivity-Level:
Recognition-Timeout:
Confidence-threshold:
S->C:MRCP/1.0 543256 200 COMPLETE
Sensitivity-Level:20
Recognition-Timeout:30
Confidence-Threshold:858.8. DEFINE-GRAMMAR 💅
okay so DEFINE-GRAMMAR is the client handing the server a grammar and saying "bestie, define this, download it if you need to, and compile it" 🧚
If the server resource is already in the recognition state, the DEFINE-GRAMMAR request MUST respond with a failure status. No sneaking one in mid-recognition 🙅♀️
If the resource is idle and it can successfully load and compile the grammar, the status MUST return a success code and the request-state MUST be COMPLETE ✨
If the recognizer can't define the grammar (the download failed, the compile failed, or the grammar was in an unsupported form), the MRCP response for DEFINE-GRAMMAR MUST contain a failure status code of 407, plus a Completion-Cause header field that explains why 💀
Example:
C->S:DEFINE-GRAMMAR 543257 MRCP/1.0
Content-Type:application/grammar+xml
Content-Id:request1@form-level.store
Content-Length:104
<?xml version="1.0"?>
<!-- the default grammar language is US English -->
<grammar xml:lang="en-US" version="1.0">
<!-- single language attachment to tokens -->
<rule id="yes">
<one-of>
<item xml:lang="fr-CA">oui</item>
<item xml:lang="en-US">yes</item>
</one-of>
</rule>
<!-- single language attachment to a rule expansion -->
<rule id="request">
may I speak to
<one-of xml:lang="fr-CA">
<item>Michel Tremblay</item>
<item>Andre Roy</item>
</one-of>
</rule>
</grammar>
S->C:MRCP/1.0 543257 200 COMPLETE
Completion-Cause:000 success
C->S:DEFINE-GRAMMAR 543258 MRCP/1.0
Content-Type:application/grammar+xml
Content-Id:helpgrammar@root-level.store
Content-Length:104
<?xml version="1.0"?>
<!-- the default grammar language is US English -->
<grammar xml:lang="en-US" version="1.0">
<rule id="request">
I need help
</rule>
</grammar>
S->C:MRCP/1.0 543258 200 COMPLETE
Completion-Cause:000 success
C->S:DEFINE-GRAMMAR 543259 MRCP/1.0
Content-Type:application/grammar+xml
Content-Id:request2@field-level.store
Content-Length:104
<?xml version="1.0"?>
<!-- the default grammar language is US English -->
<grammar xml:lang="en-US" version="1.0">
<rule id="request">
I need help
</rule>
S->C:MRCP/1.0 543258 200 COMPLETE
Completion-Cause:000 success
C->S:DEFINE-GRAMMAR 543259 MRCP/1.0
Content-Type:application/grammar+xml
Content-Id:request2@field-level.store
Content-Length:104
<?xml version="1.0"?>
<grammar xml:lang="en">
<import uri="session:politeness@form-level.store"
name="polite"/>
<rule id="basicCmd" scope="public">
<example> please move the window </example>
<example> open a file </example>
<ruleref import="polite#startPolite"/>
<ruleref uri="#command"/>
<ruleref import="polite#endPolite"/>
</rule>
<rule id="command">
<ruleref uri="#action"/> <ruleref uri="#object"/>
</rule>
<rule id="action">
<choice>
<item weight="10" tag="OPEN"> open </item>
<item weight="2" tag="CLOSE"> close </item>
<item weight="1" tag="DELETE"> delete </item>
<item weight="1" tag="MOVE"> move </item>
</choice>
</rule>
<rule id="object">
<count number="optional">
<choice>
<item> the </item>
<item> a </item>
</choice>
</count>
<choice>
<item> window </item>
<item> file </item>
<item> menu </item>
</choice>
</rule>
</grammar>
S->C:MRCP/1.0 543259 200 COMPLETE
Completion-Cause:000 success
C->S:RECOGNIZE 543260 MRCP/1.0
N-Best-List-Length:2
Content-Type:text/uri-list
Content-Length:176
session:request1@form-level.store
session:request2@field-level.store
session:helpgramar@root-level.store
S->C:MRCP/1.0 543260 200 IN-PROGRESS
S->C:START-OF-SPEECH 543260 IN-PROGRESS MRCP/1.0
S->C:RECOGNITION-COMPLETE 543260 COMPLETE MRCP/1.0
Completion-Cause:000 success
Waveform-URL:http://web.media.com/session123/audio.wav
Content-Type:applicationt/x-nlsml
Content-Length:276
<?xml version="1.0"?>
<result x-model="http://IdentityModel"
xmlns:xf="http://www.w3.org/2000/xforms"
grammar="session:request1@form-level.store">
<interpretation>
<xf:instance name="Person">
<Person>
<Name> Andre Roy </Name>
</Person>
</xf:instance>
<input> may I speak to Andre Roy </input>
</interpretation>
</result>8.9. RECOGNIZE ✨
the RECOGNIZE method is the client going "okay recognizer, start listening, and here's the grammar to match against" 🎀 It can also carry parameters that control the sensitivity, the confidence level, and how much detail you get back in the results 🔥 Those parameters override the current defaults that were set by an earlier SET-PARAMS method.
If the resource is in the recognition state, the RECOGNIZE request MUST respond with a failure status.
If the resource is in the Idle state and it successfully starts the recognition, the server MUST return a success code and a request-state of IN-PROGRESS. That means the recognizer is active and the client should expect further events with this request-id 🦋
If the resource can't start a recognition, it MUST return a failure status code of 407 and contain a completion-cause header field describing why it failed 😭
This is the only request that can return a request-state of IN-PROGRESS on the recognizer resource, which just means recognition is happening right now. When recognition finishes (by matching one of the grammar alternatives, by timing out without a match, or for some other reason), the recognizer resource MUST send the client a RECOGNITON-COMPLETE event with the result of the recognition and a request-state of COMPLETE 🏁
For big grammars that take forever to compile, or grammars you keep reusing, the client could send a DEFINE-GRAMMAR request ahead of time. Then the client can send RECOGNIZE and point at the grammar through the "session:" special URI. This also works more generally if the client wants to restart recognition with a previous inline grammar 🔁
Okay, the race condition bit: the audio and the messages travel over separate communication paths, so there can be a race between when the audio starts flowing and when the RECOGNIZE method shows up. If the client starts sending audio at the same moment it sends RECOGNIZE, either the audio or the RECOGNIZE could reach the recognizer first. Another example is a client that keeps sending audio to the Media server and uses RECOGNIZE to tell the Media server when to recognize. There are a bunch of ways to resolve this, and the mechanism is left up to the implementers of recognizer Media servers 🛠️
Example:
C->S:RECOGNIZE 543257 MRCP/1.0
Confidence-Threshold:90
Content-Type:application/grammar+xml
Content-Id:request1@form-level.store
Content-Length:104
<?xml version="1.0"?>
<!-- the default grammar language is US English -->
<grammar xml:lang="en-US" version="1.0">
<!-- single language attachment to tokens -->
<rule id="yes">
<one-of>
<item xml:lang="fr-CA">oui</item>
<item xml:lang="en-US">yes</item>
</one-of>
</rule>
<!-- single language attachment to a rule expansion -->
<rule id="request">
may I speak to
<one-of xml:lang="fr-CA">
<item>Michel Tremblay</item>
<item>Andre Roy</item>
</one-of>
</rule>
</grammar>
S->C:MRCP/1.0 543257 200 IN-PROGRESS
S->C:START-OF-SPEECH 543257 IN-PROGRESS MRCP/1.0
S->C:RECOGNITION-COMPLETE 543257 COMPLETE MRCP/1.0
Completion-Cause:000 success
Waveform-URL:http://web.media.com/session123/audio.wav
Content-Type:application/x-nlsml
Content-Length:276
<?xml version="1.0"?>
<result x-model="http://IdentityModel"
xmlns:xf="http://www.w3.org/2000/xforms"
grammar="session:request1@form-level.store">
<interpretation>
<xf:instance name="Person">
<Person>
<Name> Andre Roy </Name>
</Person>
</xf:instance>
<input> may I speak to Andre Roy </input>
</interpretation>
</result>8.10. STOP 🛑
STOP is the client telling the resource "okay, cancel any recognition that's running" 🙅♀️ If a RECOGNIZE request is active and the STOP request successfully ends it, the response header carries an active-request-id-list header field with the request-id of the RECOGNIZE request that got terminated. In that case, no RECOGNITION-COMPLETE event will be sent for the terminated request 🫶
If there was no recognition active, the response MUST NOT contain an active-request-id-list header field. Either way, the response MUST contain a status of 200(Success) ✨
Example:
C->S:RECOGNIZE 543257 MRCP/1.0
Confidence-Threshold:90
Content-Type:application/grammar+xml
Content-Id:request1@form-level.store
Content-Length:104
<?xml version="1.0"?>
<!-- the default grammar language is US English -->
<grammar xml:lang="en-US" version="1.0">
<!-- single language attachment to tokens -->
<rule id="yes">
<one-of>
<item xml:lang="fr-CA">oui</item>
<item xml:lang="en-US">yes</item>
</one-of>
</rule>
<!-- single language attachment to a rule expansion -->
<rule id="request">
may I speak to
<one-of xml:lang="fr-CA">
<item>Michel Tremblay</item>
<item>Andre Roy</item>
</one-of>
</rule>
</grammar>
S->C:MRCP/1.0 543257 200 IN-PROGRESS
C->S:STOP 543258 200 MRCP/1.0
S->C:MRCP/1.0 543258 200 COMPLETE
Active-Request-Id-List:5432578.11. GET-RESULT ✨
okay so the client can send GET-RESULT when the recognizer is already in the recognized state 💅 this lets the client grab results for a recognition that's already finished 🎀. It's super handy if the client is like "ugh, I want more alternatives" or "I need more info" 🙈. When the media server gets this request, it should re-compute and return the results based on the recognition constraints in the GET-RESULT request 🧚.
the GET-RESULT request can ask for stuff like a different confidence-threshold or an n-best-list-length 🌈 this part is optional, and the automatic speech recognition (ASR) engine is allowed to say "nope, unsupported feature" and return a status like that 💀
Example:
C->S:GET-RESULT 543257 MRCP/1.0
Confidence-Threshold:90
S->C:MRCP/1.0 543257 200 COMPLETE
Content-Type:application/x-nlsml
Content-Length:276
<?xml version="1.0"?>
<result x-model="http://IdentityModel"
xmlns:xf="http://www.w3.org/2000/xforms"
grammar="session:request1@form-level.store">
<interpretation>
<xf:instance name="Person">
<Person>
<Name> Andre Roy </Name>
</Person>
</xf:instance>
<input> may I speak to Andre Roy </input>
</interpretation>
</result>8.12. START-OF-SPEECH 🎙️
this one is an event from the recognizer to the client saying "hey, I heard speech!" 👂 it's really useful for kill-on-barge-in situations, especially when the synthesizer and the recognizer are in different sessions, so the synthesizer has no clue there's incoming audio 😭 in that case the client has to be the middle-person 🫶 and send BARGE-IN-OCCURRED to the synthesizer resource. The recognizer also drops a unique proxy-sync-id in the header for this event, and that same ID gets sent along in the BARGE-IN-OCCURRED method to the synthesizer. so iconic, right? 🦋
This event should be generated irrespective of whether the synthesizer and recognizer are in the same media server or not.
8.13. RECOGNITION-START-TIMERS ⏰
the client sends this request to the recognition resource when it knows a kill-on-barge-in prompt is done playing 🎶 this matters when the recognition and synthesizer engines aren't in the same session 💜 here's the vibe: while a kill-on-barge-in prompt is playing, you want the RECOGNIZE request already active so it can detect and do kill-on-barge-in 👑 but you DON'T want the no-input timers starting until the prompt is finished 🙅♀️ The recognizer-start-timers header field in the RECOGNIZE request lets the client say whether the timers should start or not. The recognizer should not start the timers until the client sends a RECOGNITION-START-TIMERS method to the recognizer 🌸
8.14. RECOGNITON-COMPLETE 🏁
this is an event from the recognizer resource to the client that says the recognition is done ✅ the result goes in the MRCP body of the message 🍓 The request-state field MUST be COMPLETE, which means this is the last event for that request-id and that request is officially complete 🔥 the recognizer context keeps the results and the audio waveform input of that recognition until the next RECOGNIZE request comes in 🧠 A URL to the audio waveform MAY BE returned to the client in a waveform-url header field in the RECOGNITION-COMPLETE event 🎧 the client can use that URI to retrieve or play back the audio 🦋
Example:
C->S:RECOGNIZE 543257 MRCP/1.0
Confidence-Threshold:90
Content-Type:application/grammar+xml
Content-Id:request1@form-level.store
Content-Length:104
<?xml version="1.0"?>
<!-- the default grammar language is US English -->
<grammar xml:lang="en-US" version="1.0">
<!-- single language attachment to tokens -->
<rule id="yes">
<one-of>
<item xml:lang="fr-CA">oui</item>
<item xml:lang="en-US">yes</item>
</one-of>
</rule>
<!-- single language attachment to a rule expansion -->
<rule id="request">
may I speak to
<one-of xml:lang="fr-CA">
<item>Michel Tremblay</item>
<item>Andre Roy</item>
</one-of>
</rule>
</grammar>
S->C:MRCP/1.0 543257 200 IN-PROGRESS
S->C:START-OF-SPEECH 543257 IN-PROGRESS MRCP/1.0
S->C:RECOGNITION-COMPLETE 543257 COMPLETE MRCP/1.0
Completion-Cause:000 success
Waveform-URL:http://web.media.com/session123/audio.wav
Content-Type:application/x-nlsml
Content-Length:276
<?xml version="1.0"?>
<result x-model="http://IdentityModel"
xmlns:xf="http://www.w3.org/2000/xforms"
grammar="session:request1@form-level.store">
<interpretation>
<xf:instance name="Person">
<Person>
<Name> Andre Roy </Name>
</Person>
</xf:instance>
<input> may I speak to Andre Roy </input>
</interpretation>
</result>8.15. DTMF Detection 🔢
digits sent as DTMF tones get delivered to the automatic speech recognition (ASR) engine in the RTP stream, following RFC 2833 [15] 📞 The automatic speech recognizer (ASR) needs to support RFC 2833 [15] to recognize digits 💖 if it doesn't support RFC 2833 [15], it'll have to process the audio stream and pull the audio tones out of it on its own 🛠️
9. Future Study 🔮
okay so parts of the recognizer could eventually get pushed down into Digital Signal Processors (DSPs) on the Voice Browser/Gateway or IP Phones 📱 for example, the gateway could do voice activity detection to cut down on network bandwidth and the CPU load on the automatic speech recognition (ASR) server 🧚 these extensions are being saved for later study and aren't covered in this document 🌈
10. Security Considerations 🔒
the MRCP protocol can carry sensitive stuff like account numbers, passwords, etc. 😱 that's why it's SO important that the client has the option of secure communication with the server, for both the control messages and the media 🫣 the client isn't required to use it, though. If all MRCP communications happen in a trusted domain behind a firewall, this may not be necessary 🏰 But if the client or server is deployed on an insecure network, any communication crossing that insecure network needs protection 🚨 In those cases, the following additional security functionality MUST be supported on the MRCP server 💀 MRCP servers MUST implement Transport Layer Security (TLS) to secure the RTSP communication, i.e., the RTSP stack SHOULD support the rtsps: URI form 🔐 MRCP servers MUST support Secure Real-Time Transport Protocol (SRTP) as an option to send and receive media 🎀
11. RTSP-Based Examples ✨
okay so this section is the fun one 🎀 it walks through a full speech session between a client and a server, step by step, so you can see exactly what goes over the wire. the server and client agree on one Session ID, 12345678, plus a set of RTP/RTCP ports on both sides, and then everything happens in order 💅
Opening the synthesizer. this is the first resource for the session 🧚 the client asks for it with a SETUP, and the server answers with the ports it picked:
C->S:SETUP rtsp://media.server.com/media/synthesizer RTSP/1.0
CSeq:2
Transport:RTP/AVP;unicast;client_port=46456-46457
Content-Type:application/sdp
Content-Length:190
v=0
o=- 123 456 IN IP4 10.0.0.1
s=Media Server
p=+1-888-555-1212
c=IN IP4 0.0.0.0
t=0 0
m=audio 0 RTP/AVP 0 96
a=rtpmap:0 pcmu/8000
a=rtpmap:96 telephone-event/8000
a=fmtp:96 0-15
S->C:RTSP/1.0 200 OK
CSeq:2
Transport:RTP/AVP;unicast;client_port=46456-46457;
server_port=46460-46461
Session:12345678
Content-Length:190
Content-Type:application/sdp
v=0
o=- 3211724219 3211724219 IN IP4 10.3.2.88
s=Media Server
c=IN IP4 0.0.0.0
t=0 0
m=audio 46460 RTP/AVP 0 96
a=rtpmap:0 pcmu/8000
a=rtpmap:96 telephone-event/8000
a=fmtp:96 0-15Opening a recognizer resource. 🦋 this one reuses the existing session ID and ports, so no new session is made:
C->S:SETUP rtsp://media.server.com/media/recognizer RTSP/1.0
CSeq:3
Transport:RTP/AVP;unicast;client_port=46456-46457;
mode=record;ttl=127
Session:12345678
S->C:RTSP/1.0 200 OK
CSeq:3
Transport:RTP/AVP;unicast;client_port=46456-46457;
server_port=46460-46461;mode=record;ttl=127
Session:12345678Starting speech. an ANNOUNCE message carrying the MRCP SPEAK request is what kicks things off 🔥
C->S:ANNOUNCE rtsp://media.server.com/media/synthesizer RTSP/1.0
CSeq:4
Session:12345678
Content-Type:application/mrcp
Content-Length:456
SPEAK 543257 MRCP/1.0
Kill-On-Barge-In:false
Voice-gender:neutral
Voice-category:teenager
Prosody-volume:medium
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>You have 4 new messages.</sentence>
<sentence>The first is from <say-as
type="name">Stephanie Williams</say-as> <mark
name="Stephanie"/>
and arrived at <break/>
<say-as type="time">3:45pm</say-as>.</sentence>
<sentence>The subject is <prosody
rate="-20%">ski trip</prosody></sentence>
</paragraph>
</speak>
S->C:RTSP/1.0 200 OK
CSeq:4
Session:12345678
RTP-Info:url=rtsp://media.server.com/media/synthesizer;
seq=9810092;rtptime=3450012
Content-Type:application/mrcp
Content-Length:456
MRCP/1.0 543257 200 IN-PROGRESSThe speech marker. 💖 the synthesizer hits the special marker in the message and dutifully tells the client about it:
S->C:ANNOUNCE rtsp://media.server.com/media/synthesizer RTSP/1.0
CSeq:5
Session:12345678
Content-Type:application/mrcp
Content-Length:123
SPEECH-MARKER 543257 IN-PROGRESS MRCP/1.0
Speech-Marker:Stephanie
C->S:RTSP/1.0 200 OK
CSeq:5The SPEAK request finishes. ✨ the synthesizer is done, and it lets the client know:
S->C:ANNOUNCE rtsp://media.server.com/media/synthesizer RTSP/1.0
CSeq:6
Session:12345678
Content-Type:application/mrcp
Content-Length:123
SPEAK-COMPLETE 543257 COMPLETE MRCP/1.0
C->S:RTSP/1.0 200 OK
CSeq:6Asking the customer. 🌸 the recognizer gets a RECOGNIZE request so it can listen for the customer's choices:
C->S:ANNOUNCE rtsp://media.server.com/media/recognizer RTSP/1.0
CSeq:7
Session:12345678
RECOGNIZE 543258 MRCP/1.0
Content-Type:application/grammar+xml
Content-Length:104
<?xml version="1.0"?>
<!-- the default grammar language is US English -->
<grammar xml:lang="en-US" version="1.0">
<!-- single language attachment to a rule expansion -->
<rule id="request">
Can I speak to
<one-of xml:lang="fr-CA">
<item>Michel Tremblay</item>
<item>Andre Roy</item>
</one-of>
</rule>
</grammar>
S->C:RTSP/1.0 200 OK
CSeq:7
Content-Type:application/mrcp
Content-Length:123
MRCP/1.0 543258 200 IN-PROGRESSThe question itself. 👑 the client sends the next SPEAK inside an ANNOUNCE, asking the user the question. ngl this is the bit with the good advice: when you play a prompt with kill-on-barge-in and you're asking for input, it is generally RECOMMENDED that the client send the RECOGNIZE request ahead of the SPEAK request for optimum performance and user experience. that way the recognizer is guaranteed to be online before the prompt starts playing, so the user's speech won't get chopped off at the beginning (especially for power users 🙈)
C->S:ANNOUNCE rtsp://media.server.com/media/synthesizer RTSP/1.0
CSeq:8 Session:12345678 Content-Type:application/mrcp
Content-Length:733
SPEAK 543259 MRCP/1.0
Kill-On-Barge-In:true
Content-Type:application/synthesis+ssml
Content-Length:104
<?xml version="1.0"?>
<speak>
<paragraph>
<sentence>Welcome to ABC corporation.</sentence>
<sentence>Who would you like Talk to.</sentence>
</paragraph>
</speak>
S->C:RTSP/1.0 200 OK
CSeq:8
Content-Type:application/mrcp
Content-Length:123
MRCP/1.0 543259 200 IN-PROGRESSBarge-in! 🫶 since the last SPEAK had Kill-On-Barge-In set to "true", the synthesizer gets cut off the moment the user starts talking, and the client is notified.
Now, the recognition and synthesizer resources are in the same session, so they work together to deliver kill-on-barge-in 🧠 if they were in different sessions, it would have taken a few more messages before the client got the SPEAK-COMPLETE event from the synthesizer. but whether the synthesizer and recognizer share a session or not, the recognizer MUST generate the START-OF-SPEECH event to the client.
S->C:ANNOUNCE rtsp://media.server.com/media/recognizer RTSP/1.0
CSeq:9
Session:12345678
Content-Type:application/mrcp
Content-Length:273
START-OF-SPEECH 543258 IN-PROGRESS MRCP/1.0
C->S:RTSP/1.0 200 OK
CSeq:9The client should have then blindly turned around and issued a BARGE-IN-OCCURRED method to the synthesizer resource 🙃 the synthesizer, if kill-on-barge-in was enabled on the current SPEAK request, would then have interrupted it and sent a SPEAK-COMPLETE event to the client. but in this example, since the synthesizer and recognizer share a session, the client didn't send BARGE-IN-OCCURRED to the synthesizer. it just assumed kill-on-barge-in was implemented between the two resources in the same session and worked. iykyk 😭
The completion-cause code is what tells you whether this was a normal finish or a kill-on-barge-in interruption 🍓
S->C:ANNOUNCE rtsp://media.server.com/media/synthesizer RTSP/1.0
CSeq:10
Session:12345678
Content-Type:application/mrcp
Content-Length:273
SPEAK-COMPLETE 543259 COMPLETE MRCP/1.0
Completion-Cause:000 normal
C->S:RTSP/1.0 200 OK
CSeq:10Results! 💜 the recognition resource matched the spoken stream to a grammar and produced results, which the server sends back as part of the RECOGNITION-COMPLETE event:
S->C:ANNOUNCE rtsp://media.server.com/media/recognizer RTSP/1.0
CSeq:11
Session:12345678
Content-Type:application/mrcp
Content-Length:733
RECOGNITION-COMPLETE 543258 COMPLETE MRCP/1.0
Completion-Cause:000 success
Waveform-URL:http://web.media.com/session123/audio.wav
Content-Type:application/x-nlsml
Content-Length:104
<?xml version="1.0"?>
<result x-model="http://IdentityModel"
xmlns:xf="http://www.w3.org/2000/xforms"
grammar="session:request1@form-level.store">
<interpretation>
<xf:instance name="Person">
<Person>
<Name> Andre Roy </Name>
</Person>
</xf:instance>
<input> may I speak to Andre Roy </input>
</interpretation>
</result>
C->S:RTSP/1.0 200 OK
CSeq:11Tidying up. 🎀 the client tears down the synthesizer first:
C->S:TEARDOWN rtsp://media.server.com/media/synthesizer RTSP/1.0
CSeq:12
Session:12345678
S->C:RTSP/1.0 200 OK
CSeq:12We're done with the resources and tearing them down 🧹 when the last of the resources for this session is released, the Session-ID and the RTP/RTCP ports are released too. the recognizer goes last:
C->S:TEARDOWN rtsp://media.server.com/media/recognizer RTSP/1.0
CSeq:13
Session:12345678
S->C:RTSP/1.0 200 OK
CSeq:13and that's the whole conversation, bestie 🌈✨
12. Informative References 📚
okay bestie, here's the reading list 🧚 every doc this spec leans on, with all the deets
[1] Fielding, R., Gettys, J., Mogul, J., Frystyk. H., Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext transfer protocol -- HTTP/1.1", RFC 2616, June 1999.
[2] Schulzrinne, H., Rao, A., and R. Lanphier, "Real Time Streaming Protocol (RTSP)", RFC 2326, April 1998
[3] Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", RFC 4234, October 2005.
[4] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, "SIP: Session Initiation Protocol", RFC 3261, June 2002.
[5] Handley, M. and V. Jacobson, "SDP: Session Description Protocol", RFC 2327, April 1998.
[6] World Wide Web Consortium, "Voice Extensible Markup Language (VoiceXML) Version 2.0", W3C Candidate Recommendation, March 2004.
[7] Resnick, P., "Internet Message Format", RFC 2822, April 2001.
[8] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
[9] World Wide Web Consortium, "Speech Synthesis Markup Language (SSML) Version 1.0", W3C Candidate Recommendation, September 2004.
[10] World Wide Web Consortium, "Natural Language Semantics Markup Language (NLSML) for the Speech Interface Framework", W3C Working Draft, 30 May 2001.
[11] World Wide Web Consortium, "Speech Recognition Grammar Specification Version 1.0", W3C Candidate Recommendation, March 2004.
[12] Yergeau, F., "UTF-8, a transformation format of ISO 10646", STD 63, RFC 3629, November 2003.
[13] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types", RFC 2046, November 1996.
[14] Levinson, E., "Content-ID and Message-ID Uniform Resource Locators", RFC 2392, August 1998.
[15] Schulzrinne, H. and S. Petrack, "RTP Payload for DTMF Digits, Telephony Tones and Telephony Signals", RFC 2833, May 2000.
[16] Alvestrand, H., "Tags for the Identification of Languages", BCP 47, RFC 3066, January 2001.
Appendix A. ABNF Message Definitions 🎀
this is the grammar for every single message, copied exactly, so grab a snack and let's go 🍓
ALPHA = %x41-5A / %x61-7A ; A-Z / a-z
CHAR = %x01-7F ; any 7-bit US-ASCII character,
; excluding NUL
CR = %x0D ; carriage return
CRLF = CR LF ; Internet standard newline
DIGIT = %x30-39 ; 0-9
DQUOTE = %x22 ; " (Double Quote)
HEXDIG = DIGIT / "A" / "B" / "C" / "D" / "E" / "F"
HTAB = %x09 ; horizontal tab
LF = %x0A ; linefeed
OCTET = %x00-FF ; 8 bits of data
SP = %x20 ; space
WSP = SP / HTAB ; white space
LWS = [*WSP CRLF] 1*WSP ; linear whitespace
SWS = [LWS] ; sep whitespace
UTF8-NONASCII = %xC0-DF 1UTF8-CONT
/ %xE0-EF 2UTF8-CONT
/ %xF0-F7 3UTF8-CONT
/ %xF8-Fb 4UTF8-CONT
/ %xFC-FD 5UTF8-CONT
UTF8-CONT = %x80-BF
param = *pchar
quoted-string = SWS DQUOTE *(qdtext / quoted-pair )
DQUOTE
qdtext = LWS / %x21 / %x23-5B / %x5D-7E
/ UTF8-NONASCII
quoted-pair = "\" (%x00-09 / %x0B-0C
/ %x0E-7F)
token = 1*(alphanum / "-" / "." / "!" / "%" / "*"
/ "_" / "+" / "`" / "'" / "~" )
reserved = ";" / "/" / "?" / ":" / "@" / "&" / "="
/ "+" / "$" / ","
mark = "-" / "_" / "." / "!" / "~" / "*" / "'"
/ "(" / ")"
unreserved = alphanum / mark
char = unreserved / escaped /
":" / "@" / "&" / "=" / "+" / "$" / ","
alphanum = ALPHA / DIGIT
escaped = "%" HEXDIG HEXDIG
absoluteURI = scheme ":" ( hier-part / opaque-part )
relativeURI = ( net-path / abs-path / rel-path )
[ "?" query ]
hier-part = ( net-path / abs-path ) [ "?" query ]
net-path = "//" authority [ abs-path ]
abs-path = "/" path-segments
rel-path = rel-segment [ abs-path ]
rel-segment = 1*( unreserved / escaped / ";" / "@"
/ "&" / "=" / "+" / "$" / "," )
opaque-part = uric-no-slash *uric
uric = reserved / unreserved / escaped
uric-no-slash = unreserved / escaped / ";" / "?" / ":"
/ "@" / "&" / "=" / "+" / "$" / ","
path-segments = segment *( "/" segment )
segment = *pchar *( ";" param )
scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )
authority = srvr / reg-name
srvr = [ [ userinfo "@" ] hostport ]
reg-name = 1*( unreserved / escaped / "$" / ","
/ ";" / ":" / "@" / "&" / "=" / "+" )
query = *uric
userinfo = ( user ) [ ":" password ] "@"
user = 1*( unreserved / escaped
/ user-unreserved )
user-unreserved = "&" / "=" / "+" / "$" / "," / ";"
/ "?" / "/"
password = *( unreserved / escaped /
"&" / "=" / "+" / "$" / "," )
hostport = host [ ":" port ]
host = hostname / IPv4address / IPv6reference
hostname = *( domainlabel "." ) toplabel [ "." ]
domainlabel = alphanum
/ alphanum *( alphanum / "-" ) alphanum
toplabel = ALPHA / ALPHA *( alphanum / "-" )
alphanum
IPv4address = 1*3DIGIT "." 1*3DIGIT "." 1*3DIGIT "."
1*3DIGIT
IPv6reference = "[" IPv6address "]"
IPv6address = hexpart [ ":" IPv4address ]
hexpart = hexseq / hexseq "::" [ hexseq ] / "::"
[ hexseq ]
hexseq = hex4 *( ":" hex4)
hex4 = 1*4HEXDIG
port = 1*DIGIT
generic-message = start-line
message-header
CRLF
[ message-body ]
message-body = *OCTET
start-line = request-line / status-line / event-line
request-line = method-name SP request-id SP
mrcp-version CRLF
status-line = mrcp-version SP request-id SP
status-code SP request-state CRLF
event-line = event-name SP request-id SP
request-state SP mrcp-version CRLF
message-header = 1*(generic-header / resource-header)
generic-header = active-request-id-list
/ proxy-sync-id
/ content-id
/ content-type
/ content-length
/ content-base
/ content-location
/ content-encoding
/ cache-control
/ logging-tag
; -- content-id is as defined in RFC 2392 and RFC 2046
mrcp-version = "MRCP" "/" 1*DIGIT "." 1*DIGIT
request-id = 1*DIGIT
status-code = 1*DIGIT
active-request-id-list = "Active-Request-Id-List" ":"
request-id *("," request-id) CRLF
proxy-sync-id = "Proxy-Sync-Id" ":" 1*ALPHA CRLF
content-length = "Content-Length" ":" 1*DIGIT CRLF
content-base = "Content-Base" ":" absoluteURI CRLF
content-type = "Content-Type" ":" media-type
media-type = type "/" subtype *( ";" parameter )
type = token
subtype = token
parameter = attribute "=" value
attribute = token
value = token / quoted-string
content-encoding = "Content-Encoding" ":"
*WSP content-coding
*(*WSP "," *WSP content-coding *WSP )
CRLF
content-coding = token
content-location = "Content-Location" ":"
( absoluteURI / relativeURI ) CRLF
cache-control = "Cache-Control" ":"
*WSP cache-directive
*( *WSP "," *WSP cache-directive *WSP )
CRLF
cache-directive = "max-age" "=" delta-seconds
/ "max-stale" "=" delta-seconds
/ "min-fresh" "=" delta-seconds
logging-tag = "Logging-Tag" ":" 1*ALPHA CRLF
resource-header = recognizer-header
/ synthesizer-header
method-name = synthesizer-method
/ recognizer-method
event-name = synthesizer-event
/ recognizer-event
request-state = "COMPLETE"
/ "IN-PROGRESS"
/ "PENDING"
synthesizer-method = "SET-PARAMS"
/ "GET-PARAMS"
/ "SPEAK"
/ "STOP"
/ "PAUSE"
/ "RESUME"
/ "BARGE-IN-OCCURRED"
/ "CONTROL"
synthesizer-event = "SPEECH-MARKER"
/ "SPEAK-COMPLETE"
synthesizer-header = jump-target
/ kill-on-barge-in
/ speaker-profile
/ completion-cause
/ voice-parameter
/ prosody-parameter
/ vendor-specific
/ speech-marker
/ speech-language
/ fetch-hint
/ audio-fetch-hint
/ fetch-timeout
/ failed-uri
/ failed-uri-cause
/ speak-restart
/ speak-length
recognizer-method = "SET-PARAMS"
/ "GET-PARAMS"
/ "DEFINE-GRAMMAR"
/ "RECOGNIZE"
/ "GET-RESULT"
/ "RECOGNITION-START-TIMERS"
/ "STOP"
recognizer-event = "START-OF-SPEECH"
/ "RECOGNITION-COMPLETE"
recognizer-header = confidence-threshold
/ sensitivity-level
/ speed-vs-accuracy
/ n-best-list-length
/ no-input-timeout
/ recognition-timeout
/ waveform-url
/ completion-cause
/ recognizer-context-block
/ recognizer-start-timers
/ vendor-specific
/ speech-complete-timeout
/ speech-incomplete-timeout
/ dtmf-interdigit-timeout
/ dtmf-term-timeout
/ dtmf-term-char
/ fetch-timeout
/ failed-uri
/ failed-uri-cause
/ save-waveform
/ new-audio-channel
/ speech-language
jump-target = "Jump-Size" ":" speech-length-value CRLF
speech-length-value = numeric-speech-length
/ text-speech-length
text-speech-length = 1*ALPHA SP "Tag"
numeric-speech-length =("+" / "-") 1*DIGIT SP
numeric-speech-unit numeric-speech-unit = "Second"
/ "Word"
/ "Sentence"
/ "Paragraph"
delta-seconds = 1*DIGIT
kill-on-barge-in = "Kill-On-Barge-In" ":" boolean-value CRLF
boolean-value = "true" / "false"
speaker-profile = "Speaker-Profile" ":" absoluteURI CRLF
completion-cause = "Completion-Cause" ":" 1*DIGIT SP
1*ALPHA CRLF
voice-parameter = "Voice-" voice-param-name ":"
voice-param-value CRLF
voice-param-name = 1*ALPHA
voice-param-value = 1*alphanum
prosody-parameter = "Prosody-" prosody-param-name ":"
prosody-param-value CRLF
prosody-param-name = 1*ALPHA
prosody-param-value = 1*alphanum
vendor-specific = "Vendor-Specific-Parameters" ":"
vendor-specific-av-pair
*[";" vendor-specific-av-pair] CRLF
vendor-specific-av-pair = vendor-av-pair-name "="
vendor-av-pair-value
vendor-av-pair-name = 1*ALPHA
vendor-av-pair-value = 1*alphanum
speech-marker = "Speech-Marker" ":" 1*ALPHA CRLF
speech-language = "Speech-Language" ":" 1*ALPHA CRLF
fetch-hint = "Fetch-Hint" ":" 1*ALPHA CRLF
audio-fetch-hint = "Audio-Fetch-Hint" ":" 1*ALPHA CRLF
fetch-timeout = "Fetch-Timeout" ":" 1*DIGIT CRLF
failed-uri = "Failed-URI" ":" absoluteURI CRLF
failed-uri-cause = "Failed-URI-Cause" ":" 1*ALPHA CRLF
speak-restart = "Speak-Restart" ":" boolean-value CRLF
speak-length = "Speak-Length" ":" speech-length-value
CRLF
confidence-threshold = "Confidence-Threshold" ":"
1*DIGIT CRLF
sensitivity-level = "Sensitivity-Level" ":" 1*DIGIT CRLF
speed-vs-accuracy = "Speed-Vs-Accuracy" ":" 1*DIGIT CRLF
n-best-list-length = "N-Best-List-Length" ":" 1*DIGIT CRLF
no-input-timeout = "No-Input-Timeout" ":" 1*DIGIT CRLF
recognition-timeout = "Recognition-Timeout" ":" 1*DIGIT CRLF
waveform-url = "Waveform-URL" ":" absoluteURI CRLF
recognizer-context-block = "Recognizer-Context-Block" ":"
1*ALPHA CRLF
recognizer-start-timers = "Recognizer-Start-Timers" ":"
boolean-value CRLF
speech-complete-timeout = "Speech-Complete-Timeout" ":"
1*DIGIT CRLF
speech-incomplete-timeout = "Speech-Incomplete-Timeout" ":"
1*DIGIT CRLF
dtmf-interdigit-timeout = "DTMF-Interdigit-Timeout" ":"
1*DIGIT CRLF
dtmf-term-timeout = "DTMF-Term-Timeout" ":" 1*DIGIT CRLF
dtmf-term-char = "DTMF-Term-Char" ":" CHAR CRLF
save-waveform = "Save-Waveform" ":" boolean-value CRLF
new-audio-channel = "New-Audio-Channel" ":"
boolean-value CRLFAppendix B. Acknowledgements 🌈
okay bestie, big shoutout to everyone who helped make this happen 🫶 these are the legends who get the credit:
- Andre Gillet (Nuance Communications) 💜
- Andrew Hunt (SpeechWorks) ✨
- Aaron Kneiss (SpeechWorks) 🌸
- Kristian Finlator (SpeechWorks) 🦋
- Martin Dragomirecky (Cisco Systems, Inc.) 🎀
- Pierre Forgues (Nuance Communications) 🍓
- Suresh Kaliannan (Cisco Systems, Inc.) 🧚
- Corey Stohs (Cisco Systems, Inc.) 👑
- Dan Burnett (Nuance Communications) 💖
Authors' Addresses 💅
here's where you can find the authors, no cap 🫶
Saravanan Shanmugham
Cisco Systems, Inc.
170 W. Tasman Drive
San Jose, CA 95134
EMail: sarvi@cisco.com
Peter Monaco
Nuasis Corporation
303 Bryant St.
Mountain View, CA 94041
EMail: peter.monaco@nuasis.com
Brian Eberman
Speechworks, Inc.
695 Atlantic Avenue
Boston, MA 02111
EMail: brian.eberman@speechworks.comFull Copyright Statement ⭐
okay so the legal bit, but make it cute 🎀 The Internet Society owns the copyright here, dated 2006. This document is subject to the rights, licenses, and restrictions in BCP 78, which you can read at www.rfc-editor.org/copyright.html, and otherwise the authors keep all their rights 💜
The document and the info in it are given to you AS IS. The contributor, the organization they represent or are sponsored by (if any), the Internet Society, and the IETF all say they don't offer any warranties, express or implied. That includes no promise that using the info won't infringe anyone's rights, and no implied warranties of merchantability or fitness for a particular purpose 💀
Intellectual Property 🦋
the IETF doesn't take a side on whether any IPR claims about the tech in here are valid or how far they reach, and it doesn't say whether licenses would be available. It also hasn't checked for any rights on its own 🙈 If you want the rules on rights in RFC docs, check out BCP 78 and BCP 79.
Copies of IPR disclosures sent to the IETF Secretariat, plus any licensing assurances, or the results of trying to get a general license or permission for implementers or users, can be found in the IETF's online IPR repository at http://www.ietf.org/ipr 🌸
And if you know about any copyrights, patents, patent applications, or other proprietary rights that might cover tech needed to implement this standard, the IETF wants to hear about it! Send that info to ietf-ipr@ietf.org ✨
Acknowledgement 🧚
Funding for the RFC Editor function comes from the IETF Administrative Support Activity (IASA). Thank you, IASA, you keep the lights on 💖
📦 where this page comes from
- ietf: rfc4463 2006-04
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms