RFC 4313
Requirements for Distributed Control of Automatic Speech Recognition (ASR), Speaker Identification/Speaker Verification (SI/SV), and Text-to-Speech (TTS) Resources
Status of this Memo 🌸
Okay, so this memo is giving information, not a standard. It's for the Internet community, and it does not specify an Internet standard of any kind. Distribution is totally unlimited, so share it with everyone 💅.
Copyright Notice 🎀
Copyright (C) The Internet Society (2005).
Abstract ✨
This doc is about what's needed for a protocol that controls distributed speech processing of audio streams 🎧. When it says speech processing, it means three things: automatic speech recognition (ASR), speaker recognition (which covers both speaker identification (SI) and speaker verification (SV)), and text-to-speech (TTS). SIP and Real Time Streaming Protocol (RTSP) already handle rendezvous and control for general media streams, but speech processing brings extra needs that none of the existing IETF protocols cover 💖.
Table of Contents 📖
1. Introduction ....................................................3
1.1. Document Conventions .......................................3
2. SPEECHSC Framework ..............................................4
2.1. TTS Example ................................................5
2.2. Automatic Speech Recognition Example .......................6
2.3. Speaker Identification example .............................6
3. General Requirements ............................................7
3.1. Reuse Existing Protocols ...................................7
3.2. Maintain Existing Protocol Integrity .......................7
3.3. Avoid Duplicating Existing Protocols .......................7
3.4. Efficiency .................................................8
3.5. Invocation of Services .....................................8
3.6. Location and Load Balancing ................................8
3.7. Multiple Services ..........................................8
3.8. Multiple Media Sessions ....................................8
3.9. Users with Disabilities ....................................9
3.10. Identification of Process That Produced Media or
Control Output ............................................9
4. TTS Requirements ................................................9
4.1. Requesting Text Playback ...................................9
4.2. Text Formats ...............................................9
4.2.1. Plain Text ..........................................9
4.2.2. SSML ................................................9
4.2.3. Text in Control Channel ............................10
4.2.4. Document Type Indication ...........................10
4.3. Control Channel ...........................................10
4.4. Media Origination/Termination by Control Elements .........10
4.5. Playback Controls .........................................10
4.6. Session Parameters ........................................11
4.7. Speech Markers ............................................11
5. ASR Requirements ...............................................11
5.1. Requesting Automatic Speech Recognition ...................11
5.2. XML .......................................................11
5.3. Grammar Requirements ......................................12
5.3.1. Grammar Specification ..............................12
5.3.2. Explicit Indication of Grammar Format ..............12
5.3.3. Grammar Sharing ....................................12
5.4. Session Parameters ........................................12
5.5. Input Capture .............................................12
6. Speaker Identification and Verification Requirements ...........13
6.1. Requesting SI/SV ..........................................13
6.2. Identifiers for SI/SV .....................................13
6.3. State for Multiple Utterances .............................13
6.4. Input Capture .............................................13
6.5. SI/SV Functional Extensibility ............................13
7. Duplexing and Parallel Operation Requirements ..................13
7.1. Full Duplex Operation .....................................14
7.2. Multiple Services in Parallel .............................14
7.3. Combination of Services ...................................14
8. Additional Considerations (Non-Normative) ......................14
9. Security Considerations ........................................15
9.1. SPEECHSC Protocol Security ................................15
9.2. Client and Server Implementation and Deployment ...........16
9.3. Use of SPEECHSC for Security Functions ....................16
10. Acknowledgements ..............................................17
11. References ....................................................18
11.1. Normative References .....................................18
11.2. Informative References ...................................181. Introduction ✨
Lots of IETF protocols already exist for this stuff 💅. SIP [6] handles setting up and tearing down media sessions. Media Gateway Control Protocol (MGCP) [7] and Media Gateway Controller (MEGACO) [8] handle low-level media control. RTSP [9] handles recording and playback. This document is all about the requirements for one or more protocols that control network elements doing Automated Speech Recognition (ASR), speaker identification or verification (SI/SV), and turning text into audio, which is also called Text-to-Speech (TTS). So many multimedia apps would love having ASR and TTS as a distributed network resource 🎀. The scope here is only the distributed control of ASR, SI/SV, and TTS servers.
A huge range of systems could use one unified way to control TTS, ASR, and SI/SV 🌈. Think Voice over IP (VoIP) gateways to the Public Switched Telephone Network (PSTN), IP telephones, media servers, and wireless mobile devices that get their speech services from servers on the network.
There are already a bunch of proprietary ASR and TTS APIs, plus two IETF documents that tackle this problem [13], [14]. But those existing docs have real issues 😬. They mix up the semantics of existing protocols, and they're so close to other protocols that they're confusing for the person actually implementing them.
This document lays out requirements for protocols that support distributed speech processing of audio streams 🦋. To keep things simple and avoid confusion with existing protocol proposals, the requirements are framed as a "framework" for the distributed control of speech resources. That framework is called "SPEECHSC", short for Speech Services Control 💜.
1.1. Document Conventions 🍓
In this document, the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" are to be interpreted as described in RFC 2119 [3].
2. SPEECHSC Framework 🧚
Here's Figure 1, the SPEECHSC framework for speech processing 👑.
+-------------+
| Application |
| Server |\
+-------------+ \ SPEECHSC
SIP, VoiceXML, / \
etc. / \
+------------+ / \ +-------------+
| Media |/ SPEECHSC \---| ASR, SI/SV, |
| Processing |-------------------------| and/or TTS |
RTP | Entity | RTP | Server |
=====| |=========================| |
+------------+ +-------------+
Figure 1: SPEECHSC FrameworkSo the "Media Processing Entity" is a network element that processes media 🎶. It could be a pure media handler, or it could also have an associated SIP user agent, VoiceXML browser, or some other control entity. The "ASR, SI/SV, and/or TTS Server" is the network element that does the back-end speech processing 🔥. It can output an RTP stream made from text input (that's TTS), or send back recognition results based on an RTP stream coming in (that's ASR and SI/SV). The "Application Server" is the network element that tells the Media Processing Entity what changes to make to the media stream 💅. Those instructions can be set up through a session protocol like SIP, or passed along in a client/server exchange like VoiceXML. The framework lets either the Media Processing Entity or the Application Server control the ASR or TTS Server using SPEECHSC as a control protocol. That's why SPEECHSC shows up twice in the diagram, and honestly it's giving symmetry 🦋.
The entities can live in one physical box each, or get mixed together in different combos 🌸. For example, a VoiceXML [11] gateway can put the ASR and TTS functions on the same platform as the Media Processing Entity. Heads up, though: VoiceXML gateways themselves are outside the scope of this protocol 😭. You can also combine the Application Server and the Media Processing Entity, like you'd do in an interactive voice response (IVR) platform 📞.
You can also split the Media Processing Entity into one entity that controls the media endpoints and other entities that process media directly 💜. That's what a decomposed gateway using MGCP or MEGACO looks like. Again, though, this decomposition is orthogonal to the scope of SPEECHSC 🙈. The subsections that follow give some example use cases of SPEECHSC, one each for TTS, ASR, and SI/SV. They're only meant to be illustrative. They don't restrict the framework's scope or limit the decomposition or configuration to what's shown in the examples ✨.
2.1. TTS Example 🎀
okay so this one is a super simple SPEECHSC setup for a Text-to-Speech service 🔊 It's for playing announcements to someone on a phone with no screen, so there's no way to show them text errors 🫶 The scenario is in Figure 2 below. In this figure, the VoIP gateway is playing two roles at once: it's the Media Processing Entity and the Application Server from Figure 1 💅
+---------+
_| SIP |
_/ | Server |
+-----------+ SIP/ +---------+
| | _/
+-------+ | VoIP |_/
| POTS |___| Gateway | RTP +---------+
| Phone | | (SIP UA) |=========| |
+-------+ | |\_ | SPEECHSC|
+-----------+ \ | TTS |
\__ | Server |
SPEECHSC | |
\_| |
+---------+Figure 2: Text-to-Speech Example of SPEECHSC
so the POTS phone on the left tries to make a call 📞 The VoIP gateway (acting as a SIP UA) tries to set up a SIP session to finish the call, but it gets an error, like a SIP "486 Busy Here" response 😭 Without SPEECHSC, the gateway would most likely just play a busy signal to the POTS phone. But with SPEECHSC access to a TTS server, it can give a spoken error message 🦋 So the gateway builds a text error string using info from the SIP messages, like "Your call to 978-555-1212 did not go through because the called party was busy" ✨ Then it uses SPEECHSC to set up an association with a SPEECHSC server, opens an RTP stream between itself and the server, and sends a TTS request for that error message. The message gets played to the user on the POTS phone 💖
2.2. Automatic Speech Recognition Example
this example shows a VXML-enabled media processing entity and its application server using SPEECHSC to power an ASR-based user interface through an Interactive Voice Response (IVR) system 🎧 The VXML-client is the "media processing entity," and the IVR application server is the "application server" from Figure 1. The setup is in Figure 3 below 🌸
+------------+
| IVR |
_|Application |
VXML_/ +------------+
+-----------+ __/
| |_/ +------------+
PSTN Trunk | VoIP | SPEECHSC| |
=============| Gateway |---------| SPEECHSC |
|(VXML voice| | ASR |
| browser) |=========| Server |
+-----------+ RTP +------------+Figure 3: Automatic Speech Recognition Example
in this one, people call in to get stock quotes 📈 The VoIP gateway picks up their PSTN call. An IVR application sends VXML scripts to the gateway to run the conversation 💅 The VXML interpreter on the gateway sends the user's media stream to the SPEECHSC ASR server, and uses SPEECHSC to control that ASR server.
So, for example, the user says the name of a stock after an IVR prompt. The SPEECHSC ASR server tries to recognize the name and sends the results back to the VXML gateway 👑 The gateway then uses standard VXML mechanisms to tell the IVR Application what was recognized. The IVR Application can then go look up the info 🔍 And the answer can be sent back to the user with text-to-speech! This example doesn't show that part, but it works the same way as the scenario in section Section 2.1 🫶
2.3. Speaker Identification example
this one is about using speaker identification for voice-actuated login on an IP phone 🎤 The setup is in Figure 4 below. In this figure, the IP Phone is playing both the "Media Processing Entity" and the "Application Server" roles from Figure 1 🧚
+-----------+ +---------+
| | RTP | |
| IP |=========| SPEECHSC|
| Phone | | TTS |
| |_________| Server |
| | SPEECHSC| |
+-----------+ +---------+Figure 4: Speaker Identification Example
here, a user speaks into a SIP phone to get "logged in" to it, so they can make and receive calls using their identity and preferences 🥹 The IP phone uses the SPEECHSC framework to set up an RTP stream between itself and the SPEECHSC SI/SV server, and to request verification. The SV server checks the user's identity and sends the result back to the phone via SPEECHSC, including the necessary login credentials 🔐 The IP Phone can then use that identity directly to identify the user on outgoing calls, fetch the user's preferences from a configuration server, or request authorization from an Authentication, Authorization, and Accounting (AAA) server. Any combo of those works too 💜
Since this example uses SPEECHSC for a security-related function, be sure to check out the related material in Section 9 🚨
3. General Requirements 📋
3.1. Reuse Existing Protocols
to the extent feasible, the SPEECHSC framework SHOULD use existing protocols 🤝
3.2. Maintain Existing Protocol Integrity
while meeting the requirement in Section 3.1, the SPEECHSC framework MUST NOT redefine the semantics of an existing protocol. Said differently, we will not break existing protocols or cause backward-compatibility problems 🙅♀️
3.3. Avoid Duplicating Existing Protocols
to the extent feasible, SPEECHSC SHOULD NOT duplicate the functionality of existing protocols. For example, network announcements using SIP [12] and RTSP [9] already define how to request playback of audio 🔁 The focus of SPEECHSC is new functionality that existing protocols don't address, or extending existing protocols within the limits of the requirement in Section 3.2 ✨ If an existing protocol can be gracefully extended to support SPEECHSC requirements, those extensions are acceptable ways to meet the requirements 💖
as a corollary, the SPEECHSC should not require a separate protocol to perform functions that could be easily added into the SPEECHSC protocol (like redirecting media streams, or discovering capabilities), unless it's similarly easy to embed that protocol directly into the SPEECHSC framework 🧩
3.4. Efficiency
the SPEECHSC framework SHOULD use protocol elements known to result in efficient operation ⚡ Techniques to consider include:
- Re-use of transport connections across sessions
- Piggybacking of responses on requests in the reverse direction
- Caching of state across requests
3.5. Invocation of Services
the SPEECHSC framework MUST be compliant with the IAB Open Pluggable Edge Services (OPES) [4] framework 🌐 So the applicability of the SPEECHSC protocol will be specified as occurring between clients and servers, at least one of which is operating directly on behalf of the user who requested the service 🫶
3.6. Location and Load Balancing
to the extent feasible, the SPEECHSC framework SHOULD use existing schemes for service location and load balancing, like the Service Location Protocol [13] or DNS SRV records [14] 🗺️ If those facilities aren't deemed adequate, the SPEECHSC framework MAY define additional load balancing techniques 🌈
3.7. Multiple Services
the SPEECHSC framework MUST permit multiple services to operate on a single media stream, so that the same or different servers can be doing speech recognition, speaker identification or verification, etc., in parallel 🎶
3.8. Multiple Media Sessions
the SPEECHSC framework MUST allow a 1:N mapping between session and RTP channels 🎀 For example, a single session might include an outbound RTP channel for TTS, an inbound one for ASR, and a different inbound one for SI/SV (for instance, if different elements on the Media Resource Element process them) 🦋
Note: All of these can be described via SDP, so if SDP is used for media channel description, this requirement is met "for free" 🥳
3.9. Users with Disabilities
the SPEECHSC framework must have sufficient capabilities to address the critical needs of people with disabilities 🫶 In particular, the set of requirements in RFC 3351 [5] MUST be taken into account by the framework. It's also important that implementers of SPEECHSC clients and servers know that some interaction modalities of SPEECHSC may be inconvenient or just plain inappropriate for disabled users 💜 Hearing-impaired individuals may find TTS of limited use. Speech-impaired users may not be able to use ASR or SI/SV capabilities 😭 So systems employing SPEECHSC MUST provide alternative interaction modes or avoid using speech processing entirely 🌸
3.10. Identification of Process That Produced Media or Control Output
the client of a SPEECHSC operation SHOULD be able to figure out, via the SPEECHSC framework, which speech process produced the output 🔍 For example, an RTP stream with spoken TTS output should be identifiable as TTS output, and the recognized utterance from ASR should be identifiable as having come from ASR processing 🧠
4. TTS Requirements 🎙️
4.1. Requesting Text Playback
the SPEECHSC framework MUST allow a Media Processing Entity or Application Server, using a control protocol, to request that the TTS Server play back text as voice in an RTP stream 💅
4.2. Text Formats
4.2.1. Plain Text
the SPEECHSC framework MAY assume that all TTS servers can read plain text 📝 For reading plain text, the framework MUST allow the language and voicing to be indicated via session parameters. For finer control over those properties, see [1] 🎀
4.2.2. SSML
the SPEECHSC framework MUST support Speech Synthesis Markup Language (SSML)[1] <speak> basics, and SHOULD support other SSML tags 🫧 The framework assumes all TTS servers can read SSML formatted text. Internationalization of TTS in the SPEECHSC framework, including multi-lingual output within a single utterance, is handled with SSML xml:lang tags 🌍
4.2.3. Text in Control Channel
the SPEECHSC framework assumes all TTS servers accept text over the SPEECHSC connection, for reading over the RTP connection 💌 The framework assumes the server can accept text either "by value" (embedded in the protocol) or "by reference" (for example, by de-referencing a Uniform Resource Identifier (URI) embedded in the protocol) 🔗
4.2.4. Document Type Indication
a document type specifies the syntax the text to be read is encoded in 📄 The SPEECHSC framework MUST be capable of explicitly indicating the document type of the text to be processed, instead of forcing the server to infer the content by other means 🧚♀️
4.3. Control Channel 🎀
okay so the SPEECHSC framework MUST be able to set up the control channel between the client and server on a per-session basis 💖. a session is loosely a single "call" or "dialog" ✨. the protocol SHOULD be able to keep one long-lived control channel going for multiple sessions back to back 🔁, and it MAY also do shorter time horizons too, like as tiny as processing one single utterance 🦋.
4.4. Media Origination/Termination by Control Elements 🌸
bestie, the SPEECHSC framework MUST NOT force the controlling element (the application server or media processing entity) to accept or originate media streams 🙅♀️. the media streams MAY source & sink from the controlled element (ASR, TTS, etc.) instead, which is so clean 💅.
4.5. Playback Controls 🎬
the SPEECHSC framework MUST support "VCR controls" for controlling the playout of streaming media output from SPEECHSC processing 📼, and it MUST allow for servers with varying capabilities to accommodate those controls 🫶. the protocol SHOULD let clients say which controls they want to use, and let servers report which ones they honor 🤝. these capabilities include:
- The ability to jump in time to the location of a specific marker. 📍
- The ability to jump in time, forwards or backwards, by a specified amount of time. Valid time units MUST include seconds, words, paragraphs, sentences, and markers. ⏱️
- The ability to increase and decrease playout speed. 🏃♀️
- The ability to fast-forward and fast-rewind the audio, where snippets of audio are played as the server moves forwards or backwards in time. ⏩
- The ability to pause and resume playout. ⏸️
- The ability to increase and decrease playout volume. 🔊
these controls SHOULD be made easily available to users through the client user interface and through per-user customization capabilities of the client 💖. this is literally so important for hearing-impaired users 🥺, who will likely want settings and control regimes that are different from what works for non-impaired users. main character accessibility energy 👑.
4.6. Session Parameters 🧁
the SPEECHSC framework MUST support specifying session parameters, like language, prosody, and voicing 🗣️. slay, the basics.
4.7. Speech Markers 🏷️
the framework MUST accommodate speech markers, with capability at least as flexible as what SSML [1] provides ✨. it MUST further provide an efficient mechanism for reporting that a marker has been reached during playout 📣.
5. ASR Requirements 🎧
5.1. Requesting Automatic Speech Recognition 🎙️
the SPEECHSC framework MUST allow a Media Processing Entity or Application Server to ask the ASR Server to perform automatic speech recognition on an RTP stream, with the results coming back over SPEECHSC 💌.
5.2. XML 📄
the framework assumes that all ASR servers support the VoiceXML speech recognition grammar specification (SRGS) for speech recognition [2]. no cap, this one is the foundation 🧱.
5.3.1. Grammar Specification 📜
the framework assumes all ASR servers can accept grammar specifications either "by value" (embedded in the protocol) or "by reference" (for example, by de-referencing a URI embedded in the protocol) 🔗. the latter MUST allow indicating a grammar that is already known to, or otherwise "built in" to, the server 🧠. the framework and protocol SHOULD further exploit the ability to store and later retrieve, by reference, large grammars that were originally supplied by the client 💾.
5.3.2. Explicit Indication of Grammar Format 🏷️
the SPEECHSC framework protocol MUST be able to explicitly convey the grammar format in which the grammar is encoded 🙌, and MUST be extensible so it can convey new grammar formats as they get defined 🌈.
5.3.3. Grammar Sharing 🤝
the SPEECHSC framework SHOULD exploit sharing grammars across sessions for servers that are capable of doing so 🍓. this supports applications with large grammars that it's unrealistic to load dynamically. an example is a city-country grammar for a weather service 🌦️.
5.4. Session Parameters ⚙️
the SPEECHSC framework MUST accommodate, at a minimum, all of the protocol parameters currently defined in Media Resource Control Protocol (MRCP) [10]. in addition, there SHOULD be a capability to reset parameters within a session 🔄.
5.5. Input Capture 📼
the SPEECHSC framework MUST support a method that directs the ASR Server to capture the input media stream for later analysis and tuning of the ASR engine 🔧.
6. Speaker Identification and Verification Requirements 🕵️♀️
6.1. Requesting SI/SV 🪪
the SPEECHSC framework MUST allow a Media Processing Entity to ask the SI/SV Server to perform speaker identification or verification on an RTP stream, with results returned over SPEECHSC 💫.
6.2. Identifiers for SI/SV 🔑
the SPEECHSC framework MUST accommodate an identifier for each verification resource and permit control of that resource by ID 🆔. the reason is that voiceprint format and contents are vendor specific, so IDs keep things clean 💅.
6.3. State for Multiple Utterances 🧩
the SPEECHSC framework MUST work with SI/SV servers that maintain state to handle multi-utterance verification 🫶. it's giving continuity.
6.4. Input Capture 🎚️
the SPEECHSC framework MUST support a method for capturing the input media stream for later analysis and tuning of the SI/SV engine. the framework may assume all servers are capable of doing so. in addition, the framework assumes that the captured stream contains enough timestamp context (e.g., the NTP time range from the RTP Control Protocol (RTCP) packets, which corresponds to the RTP timestamps of the captured input) to figure out, after the fact, exactly when the verification was requested ⏰. receipts, bestie 🧾.
6.5. SI/SV Functional Extensibility 🧪
the SPEECHSC framework SHOULD be extensible to additional functions associated with SI/SV, such as prompting, utterance verification, and retraining 🌱.
7. Duplexing and Parallel Operation Requirements 🔁
okay so one super important thing for an interactive speech-driven system is that the user's perception of quality depends hard on being able to interrupt a prompt or rendered TTS with their own speech 🗯️. interrupting, or barging, the speech output takes more than just energy detection from the user's direction. lowkey a lot of advanced systems halt the media toward the user by using the ASR engine to decide whether an utterance is likely real speech, as opposed to, say, a cough 🤧.
7.1. Full Duplex Operation 🎧
to get low latency between utterance detection and halting playback, many implementations combine the speaking and ASR functions ⚡. the SPEECHSC framework MUST support such full-duplex implementations 💖.
7.2. Multiple Services in Parallel 🎯
good spoken user interfaces typically depend on how easily the user can get their task done ✨. when you're using speaker identification or verification tech, the UI improvements often come from combining the different technologies: simultaneous identity claim and verification (on the same utterance), and simultaneous knowledge and voice verification (using ASR and verification at the same time). using ASR and verification on the same utterance is in fact the only way to support rolling or dynamically-generated challenge phrases (e.g., "say 51723") 🔢. the SPEECHSC framework MUST support such parallel service implementations 🫶.
7.3. Combination of Services 🧩
it is optionally of interest that the SPEECHSC framework support more complex remote combination and control of speech engines 👀:
- Combination in series of engines that may then act on the input or output of ASR, TTS, or Speaker recognition engines. The control MAY then extend beyond such engines to include other audio input and output processing and natural language processing. 🔗
- Intermediate exchanges and coordination between engines. 🤝
- Remote specification of flows between engines. 🌊
these capabilities MAY benefit from service discovery mechanisms (e.g., engines, properties, and states discovery) 🔍.
8. Additional Considerations (Non-Normative) 💭
the framework assumes that Session Description Protocol (SDP) will be used to describe media sessions and streams. it further assumes RTP carriage of media 🎶. however, since SDP can be used to describe other media transport schemes (e.g., ATM), those could be used too, as long as they provide the necessary elements (e.g., explicit timestamps) 🧐.
the working group will not be defining distributed speech recognition (DSR) methods, as exemplified by the European Telecommunications Standards Institute (ETSI) Aurora project 🙅♀️. the working group will also not be recreating functionality that's available in other protocols, such as SIP or SDP 💅.
TTS looks very much like playing back a file 📁. extending RTSP looks promising when someone needs VCR controls or markers in the text to be spoken ⏯️. when you don't need VCR controls, SIP in a framework such as Network Announcements [12] works directly, no modifications needed 🦋.
ASR has an entirely different set of characteristics 🌪️. for barge-in support, ASR needs real-time return of intermediate results. barring the discovery of a good reuse model for an existing protocol, this will most likely become the focus of SPEECHSC 🎯.
9. Security Considerations 🔐
bestie, protocols for speech processing must take security and privacy into account 🫣. a lot of speech tech deals with sensitive info, like using Text-to-Speech to read out financial information 💸. likewise, popular uses for automatic speech recognition include executing financial transactions and shopping 🛍️. iykyk, this is serious stuff.
there are at least three aspects of speech processing security that intersect with the SPEECHSC requirements: securing the SPEECHSC protocol itself, implementing and deploying the servers that run the protocol, and making sure that using the technology to provide security functions is appropriate 🛡️. each of these aspects is discussed in the following subsections. while some of these considerations are, strictly speaking, out of scope of the protocol itself, they will be carefully considered and accommodated during protocol design, and will be called out as part of the applicability statement that comes with the protocol specification(s) 📋. privacy considerations are discussed as well 🙈.
9.1. SPEECHSC Protocol Security 🔒
the SPEECHSC protocol MUST in all cases support authentication, authorization, and integrity, and SHOULD support confidentiality 💜. for privacy-sensitive applications, the protocol MUST support confidentiality 🚨. the vision is that, rather than building protocol-specific security mechanisms into SPEECHSC itself, the resulting protocol will use the security machinery of either a containing protocol or the transport it runs on 🧚. for example, we will consider solutions such as using Transport Layer Security (TLS) for securing the control channel, and Secure Realtime Transport Protocol (SRTP) for securing the media channel 🎀. third-party dependencies that would need transitive trust will be minimized, or explicitly dealt with through the authentication and authorization aspects of the protocol design 👑.
9.2. Client and Server Implementation and Deployment ✨
okay so, the stuff SPEECHSC carries can be seriously sensitive 🥺, so clients and servers have to lock things down so the data stays confidential and intact, both when it's turned into spoken form and when it's turned back again 🔐💖. And it gets extra intense for speaker verification and identification, because those functions use voiceprints 🎙️ that need their privacy, confidentiality, and integrity protected, no exceptions 💅. Input capture is also a whole thing. It's useful for analysis and tuning 📊, but recording what users say can be a privacy risk, since those utterances could get revealed or replayed somewhere they shouldn't be 😭. Implementers have to be super careful about guarding any centralized voiceprint database, plus the recordings the voiceprints came from 🍓. These are the specific moves recommended to cut down on those threats:
- End-to-end authentication, confidentiality, and integrity protection (like TLS) for access to the database, so there's less exposure to outside attackers 🛡️.
- Database protection like read/write access control and local login authentication, so insiders have less exposure too 👀.
- Copies of the database, especially ones kept at off-site locations, need the same protection as the live operational database 📦.
Right now, leaking this data isn't an exploitable threat, but it might become one later 🔮. We'll look at the specific vulnerabilities that could become possible in the next subsection 🕵️♀️. For now it's smart to do things like encrypting the voiceprint database and only letting people in through programming interfaces that enforce proper authorization machinery 🔑✨.
9.3. Use of SPEECHSC for Security Functions 🦋
Speaker identification or verification can both be used directly as an authentication technology 🎀. Authorization decisions can be tied to speaker verification directly, through challenge-response protocols, or indirectly with speaker identification, using access control lists or other identity-based authorization mechanisms 🗝️. Once you do that, there are extra security concerns that need protocol security mechanisms for clients and servers to handle 😤. For example, if someone can mess with the media stream of a speaker verification request, they could let the wrong person in or lock the right person out, either through impersonation or by garbling things with noise injection 🙈. That's why it's so critical to properly secure both the control and data channels, as recommended above 🔒. These issues are specific to using SI/SV for authentication, so they should be carefully considered 💜:
- Stolen voiceprints, or the recorded samples used to build them, are a future threat to speaker identification and verification as a biometric authentication technology 😱. One plausible attack (not feasible today) is feeding the voiceprint information into a text-to-speech synthesis system as parametric input, so it could mimic the user's voice well enough to match the voiceprint 🤖. Recording a pretty big pile of voice samples isn't that hard to do sneakily, so being able to build voiceprints for this attack would make voice-based biometric authentication highly vulnerable, even with advanced challenge-response techniques 💀. People using speaker verification for authentication should keep a close eye on tech developments for these future vulnerabilities 👀 (kind of like how users of other authentication technologies should watch advances in factoring, which can be used to break asymmetric keying systems 🧠).
- Just like other biometric authentication technologies, speech identification has a downside: revocation isn't possible 😭. Once the biometric information is compromised, it can be used for identification and authentication in other independent systems 🚨.
- Enrollment procedures can be vulnerable to impersonation if they aren't protected by both protocol security mechanisms and some independent proof of identity 🧐. (Proof of identity may not be needed in systems that only need to verify continuity of identity since enrollment, as opposed to association with a particular individual. 🫶)
If you want more on using SI/SV as an authentication technology, plus some recommendations on its advantages and vulnerabilities, check out Chapter 5 of [15] 📚✨.
10. Acknowledgements ✨
Eric Burger wrote the original version of these requirements and has kept contributing actively the whole way through their development 🙌. He's basically a co-author in everything but formal authorship 👑. He's acknowledged here instead because it's better for working group co-chairs to have roles that don't conflict with how documents progress 💅.
11. References 📚
11.1. Normative References 🫶
- [1] Walker, M., Burnett, D., and A. Hunt, "Speech Synthesis Markup Language (SSML) Version 1.0", W3C REC REC-speech-synthesis-20040907, September 2004.
- [2] McGlashan, S. and A. Hunt, "Speech Recognition Grammar Specification Version 1.0", W3C REC REC-speech-grammar-20040316, March 2004.
- [3] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
- [4] Floyd, S. and L. Daigle, "IAB Architectural and Policy Considerations for Open Pluggable Edge Services", RFC 3238, January 2002.
- [5] Charlton, N., Gasson, M., Gybels, G., Spanner, M., and A. van Wijk, "User Requirements for the Session Initiation Protocol (SIP) in Support of Deaf, Hard of Hearing and Speech-impaired Individuals", RFC 3351, August 2002.
11.2. Informative References 🌈
- [6] Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, "SIP: Session Initiation Protocol", RFC 3261, June 2002.
- [7] Andreasen, F. and B. Foster, "Media Gateway Control Protocol (MGCP) Version 1.0", RFC 3435, January 2003.
- [8] Groves, C., Pantaleo, M., Ericsson, LM., Anderson, T., and T. Taylor, "Gateway Control Protocol Version 1", RFC 3525, June 2003.
- [9] Schulzrinne, H., Rao, A., and R. Lanphier, "Real Time Streaming Protocol (RTSP)", RFC 2326, April 1998.
- [10] Shanmugham, S., Monaco, P., and B. Eberman, "MRCP: Media Resource Control Protocol", Work in Progress.
- [11] World Wide Web Consortium, "Voice Extensible Markup Language (VoiceXML) Version 2.0", W3C Working Draft , April 2002,
<http://www.w3.org/TR/2002/WD-voicexml20-20020424/>. - [12] Burger, E., Ed., Van Dyke, J., and A. Spitzer, "Basic Network Media Services with SIP", RFC 4240, December 2005.
- [13] Guttman, E., Perkins, C., Veizades, J., and M. Day, "Service Location Protocol, Version 2", RFC 2608, June 1999.
- [14] Gulbrandsen, A., Vixie, P., and L. Esibov, "A DNS RR for specifying the location of services (DNS SRV)", RFC 2782, February 2000.
- [15] Committee on Authentication Technologies and Their Privacy Implications, National Research Council, "Who Goes There?: Authentication Through the Lens of Privacy", Computer Science and Telecommunications Board (CSTB) , 2003,
<http://www.nap.edu/catalog/10656.html/ >.
Author's Address 💌
David R. Oran
Cisco Systems, Inc.
7 Ladyslipper Lane
Acton, MA
USA
EMail: oran@cisco.comFull Copyright Statement ©
Copyright (C) The Internet Society (2005).
This document is covered by the rights, licenses and restrictions in BCP 78, and apart from what's set out there, the authors keep all their rights 💜.
This document and the info in it is provided "AS IS" 🫠. The contributor, the organization they represent or are sponsored by (if any), the Internet Society, and the Internet Engineering Task Force all disclaim every warranty, express or implied. That includes any warranty that using the information here won't infringe any rights, and any implied warranties of merchantability or fitness for a particular purpose 🙅♀️.
Intellectual Property 🔍
The IETF doesn't take a position on whether any Intellectual Property Rights or other rights claimed for the technology in this document are valid or in scope, or on whether a license under those rights might be available. It also doesn't claim to have independently tried to find any such rights 🕵️. Info about how rights work for RFC documents is in BCP 78 and BCP 79 📖.
Copies of IPR disclosures sent to the IETF Secretariat, any assurances of licenses that will be available, and the results of any attempt to get a general license or permission for implementers or users to use those proprietary rights can all be found in the IETF's online IPR repository at http://www.ietf.org/ipr 🌸.
The IETF also invites anyone to bring to its attention any copyrights, patents, patent applications, or other proprietary rights that might cover technology needed to implement this standard. Send that info to the IETF at ietf-ipr@ietf.org 💌.
Acknowledgement 🎀
Funding for the RFC Editor function is currently provided by the Internet Society 💖.
📦 where this page comes from
- ietf: rfc4313 2005-12
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms