RFC 9993

RTP Payload Format for Haptics

📅 published May 2026, from ietf

RFC 9993: RTP Payload Format for Haptics 💖

Internet Engineering Task Force (IETF) HS Yang Request for Comments: 9993 X. de Foy Updates: 9695 InterDigital Category: Standards Track May 2026 ISSN: 2070-1721

okay bestie, let's get into it 🎀 this RFC is by HS Yang and X. de Foy, both from InterDigital, and it's a Standards Track doc (so it's the real deal, not a someday-maybe ✨). It was published in May 2026</pick>, and it updates RFC 9695 🦋. That means it doesn't replace the old one, it adds stuff to it. Yes, I'm obsessed with this framing 😭

Also, it's part 1 of 5, so more is coming, babe 🌈

Abstract ✨

Okay so the big idea is an RTP payload format for MPEG-I haptic data 🫶 A haptic media stream is made of MIHS units, and each one has a MIHS unit header plus zero or more MIHS packets. The RTP payload header lets you pack a MIHS unit into an RTP packet payload, and also split a MIHS unit across several RTP packets when it's too big. Lowkey genius 💅

Here's the tea 🍵: the original haptics/hmpg subtype registration in RFC 9695 had no required or optional parameters at all. This memo fixes that by updating RFC 9695 and adding optional parameters to the haptics/hmpg registration. It also gives SDP usage info for the haptics media type 💜

Status of This Memo 🌸

This is an Internet Standards Track document, so it has the official stamp of approval 👑

It's a product of the IETF and represents the consensus of the IETF community. It went through public review and got approved for publication by the Internet Engineering Steering Group (IESG). If you want more on Internet Standards, check out Section 2 of RFC 7841 🎀

Current status, errata, and where to send feedback live at https://www.rfc-editor.org/info/rfc9993 🔥

Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication. Please read these carefully, because they're about your rights and restrictions for this doc. Code Components pulled out of this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come without warranty as described in the Revised BSD License ⚠️

Table of Contents 💜

  • 1. Introduction
  • 2. Conventions
  • 3. Definitions
  • 4. Haptic Format Description
    • 4.1. Overview of Haptic Coding
    • 4.2. MIHS Format
  • 5. Payload Format for Haptics
    • 5.1. RTP Header Usage
    • 5.2. Payload Header
    • 5.3. Payload Structures
      • 5.3.1. Single Unit Payload Structure
      • 5.3.2. Fragmented Unit Payload Structure
      • 5.3.3. Aggregation Packet Payload Structure
    • 5.4. MIHS Units Transmission and Reception Considerations
  • 6. Payload Format Parameters
    • 6.1. Optional Parameters Definition
    • 6.2. SDP Parameter Registration
  • 7. SDP Considerations
    • 7.1. SDP Offer/Answer Considerations
    • 7.2. Declarative SDP Considerations
  • 8. Congestion Control Considerations
  • 9. Security Considerations
  • 10. IANA Considerations
    • 10.1. Media Type Registration Update
    • 10.2. New SDP Parameters Media Registration
  • 11. References
    • 11.1. Normative References
    • 11.2. Informative References
  • Acknowledgments
  • Authors' Addresses

1. Introduction ✨

Haptics gives users tactile effects alongside audio and video, so they get that full sensory immersion experience 🤯 Haptic data mostly goes to devices that act as actuators, and it tells them how to operate based on the values in the haptic effects. The IETF registered haptics as a primary media type, same vibe as audio and video [RFC9695] 💅

The MPEG Haptics Coding standard [ISO.IEC.23090-31] defines the data formats, metadata, and codec architecture for encoding, decoding, synthesizing, and transmitting haptic signals 🧚 Inside that standard, a haptic media stream is made of MIHS units, each with a MIHS unit header and zero or more MIHS packets. A MIHS unit is a packetization unit that's built for streaming, and it's kind of like the Network Abstraction Layer (NAL) unit you'll find in some video specs 🎬

This document shows how haptic data (MIHS units) can travel over RTP. It follows the recommendations in [RFC8088] and [RFC2736] for RTP payload format writers 📝 It does not specify lip sync or any other synchronization between haptics and audio/video, so don't come for that here 🙈 It also specifies the SDP parameters and SDP offer/answer considerations for the haptics media type 🌸

2. Conventions 💜

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be read as described in BCP 14 [RFC2119] [RFC8174], but only when they show up in all capitals, exactly as shown here. Capitals are the magic word, babe, so lowercase doesn't count 🔥

3. Definitions 🎀

This document uses the definitions from the MPEG Haptics Coding standard [ISO.IEC.23090-31]. A few of them are repeated here so you don't have to dig 🫶

  • Actuator: A component of a device that renders haptic sensations 💥
  • Avatar: A representation of a body, or part of one 🧍‍♀️
  • Band: A component of a channel that holds effects for a specific range of frequencies 🎶
  • Channel: A component of a perception with one or more bands, rendered on a device at a specific body location 📍
  • Device: A physical system with one or more actuators set up to render a haptic sensation matching a given signal 🤖
  • Effect: A component of a band that defines a signal, made of a haptic waveform or one or more haptic keyframes 🌊
  • Experience: The top-level haptic component that holds perceptions and metadata 👑
  • Haptics: Tactile sensations 🫠
  • Keyframe: A component of an effect that maps a position in time or space to an effect parameter, like amplitude or frequency 📈
  • Metadata: Global info about an experience, perception, channel, or band 🗂️
  • MIHS unit: A packetization unit of the MPEG-I Haptic Stream format, used as the payload unit in the format this memo describes. See Section 4 for the details 🧩
  • Modality: The type of haptics, such as vibration, force, pressure, position, velocity, or temperature 🌡️
  • Perception: A haptic perception that holds channels of one specific modality 👀
  • Signal: A representation of the haptics for one modality, to be rendered on a device 🎧
  • Hmpg format: A binary compressed format for haptics data. The info is stored in binary form, and compression is applied at the band level. The haptics/hmpg media subtype is registered in [RFC9695] and updated by this memo 💾
  • Independent unit: A MIHS unit that can be decoded without any earlier units. Independent units carry timing info and are also called "sync units" in [ISO.IEC.23090-31] ⏱️
  • Dependent unit: A MIHS unit that needs earlier units to be decoded. Dependent units don't carry timing info and are also called "non-sync units" in [ISO.IEC.23090-31] 🔗
  • Time-independent effect: A haptic effect that happens regardless of time. The tactile feel of a texture is the classic example. These get encoded in spatial MIHS units, see Section 4.2 🧱
  • Time-dependent effect: A haptic effect that changes over time. Vibration and force are the big examples, and they get encoded in temporal MIHS units, see Section 4.2 ⏳

4. Haptic Format Description 🦋

4.1. Overview of Haptic Coding ✨

The MPEG Haptics Coding standard gives methods for efficient transmission and rendering of haptic signals, so experiences feel immersive 🌈 It handles several perception types, including the most common vibrotactile (the sense of touch that picks up vibrations) and kinesthetic perceptions (tactile resistance or force). It also covers rarer ones, like temperature or texture 🌡️

It supports two ways of encoding haptic signals: a "quantized" approach based on samples of measured data, and a "descriptive" approach where the signal is synthesized from a combination of functions 🧪 Both kinds of data can be encoded in a text-based exchange format based on JSON (.hjif) or in a binary packetized format for distribution and streaming (.hmpg). That binary one is called the MIHS format, and it's the foundation for the RTP payload format in this document 💜

4.2. MIHS Format 💖

MIHS is a stream format that transports haptic data. Haptic data, effects included, gets packetized according to the MIHS format and delivered to actuators, which then operate on the effects they're given 🚀 The MIHS format has two levels of packetization: MIHS units and MIHS packets.

MIHS units are made of a MIHS unit header plus zero or more MIHS packets. There are four types of MIHS units 😤 Here they are:

  • Initialization MIHS unit: Carries MIHS packets with the metadata needed to reset and initialize a haptic decoder, including a timestamp ⏰
  • Temporal MIHS unit: Holds one or more MIHS packets that define time-dependent effects, like pressure, velocity, and acceleration. Its duration is a positive number 🎢
  • Spatial MIHS unit: Holds one or more MIHS packets for time-independent effects, like vibrotactile texture, stiffness, and friction. Its duration is always zero 🧱
  • Silent MIHS unit: Says there's no effect during a time interval. Its duration is a positive number 🤫

A MIHS unit can be marked as independent or dependent 🔗 When a decoder handles an independent unit, it resets the previous effects, so the haptic experience doesn't depend on any earlier MIHS unit. A dependent unit continues previous MIHS units, so it cannot be decoded and rendered on its own without having decoded earlier unit(s) first. Initialization and spatial MIHS units are always independent. Temporal and silent MIHS units can be either one 🫶

Figure 1 shows a succession of MIHS units in a MIHS stream 👀

+--------+ +-------+ +-------------+ +-------------+ +-----------+
|Initial*| |Spatial| |Temporal Unit| |Temporal Unit| |Silent Unit|
| Unit   |-| Unit  |-|   (indep.)  |-| (dependent) |-| (indep.)  |
+--------+ +-------+ +-------------+ +-------------+ +-----------+
 *Initialization

Figure 1: Example of MIHS Stream 🎬

Part 1 of 5 is done, bestie 💅 Part 2 is next, and it's where we get into Section 5 and the payload format itself. Stay tuned 🫶✨

5.1. RTP Header Usage ✨

okay so the RTP header is defined in [RFC3550] and shown in Figure 2 💅 unless I say otherwise below, the fields in Figure 2 mean the same thing as in Section 5.1 of [RFC3550] 🎀

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |V=2|P|X|  CC   |M|     PT      |       Sequence Number         |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                        Timestamp (TS)                         |
   +---------------------------------------------------------------+
   |           Synchronization Source (SSRC) Identifier            |
   +---------------------------------------------------------------+
   |            Contributing Source (CSRC) Identifiers             |
   |                             ....                              |
   +---------------------------------------------------------------+

                      Figure 2: RTP Header for Haptics

Marker bit (M): 1 bit. 🦋 The marker bit SHOULD be set to one in the first non-silent RTP packet after a period of haptic silence 🌸 This helps the jitter buffer adapt and lets the haptics device do its washout (aka reset to a neutral position) right before the burst starts, with minimal impact on the end user's experience 🫶 The marker bit in all other packets MUST be set to zero ✨

Timestamp (TS): 32 bits. 💜 A timestamp showing the sampling time of the first sample of the MIHS unit in the RTP payload 🧚 The clock frequency MUST be set to the sample rate of the encoded haptic data, and it's conveyed out of band (e.g., as an SDP parameter) 🍓

5.2. Payload Header ✨

The RTP payload header comes right after the RTP header 💖 Figure 3 shows the RTP payload header for haptics 🎀

   +-+-+-+-+-+-+-+-+
   |0|1|2|3|4|5|6|7|
   +-+-+-+-+-+-+-+-+
   |D| UT  |   L   |
   +-+-----+-------+

                  Figure 3: RTP Payload Header for Haptics

D (Dependency): 1 bit. 🌈 This tells you whether the MIHS unit in the payload is dependent (value one) or independent (value zero) 👑

UT (Unit Type): 3 bits. 💅 This tells you the type of MIHS unit in the payload. The UT values are listed in Table 1 🦋

L (MIHS Layer): 4 bits. 🔥 This is an integer for the priority order of the MIHS unit in the payload, set by the haptic sender (e.g., the haptic codec) based on what the app needs 🫶 For example, the sender might use the MIHS layer to prioritize the perceptions with the biggest impact on the end-user experience 💖 Zero means the highest priority. The meaning of each individual MIHS layer isn't specified; the application gets to decide that 🧚 If the sender doesn't use the L field to show priority order, the L value is '0' 🌸

5.3. Payload Structures ✨

There are three types of RTP packet payload structures 🌈 A single unit packet has one MIHS unit in the payload. A fragmentation unit has a subset of a MIHS unit. An aggregation packet has multiple MIHS units in the payload 💅 The unit type (UT) field of the RTP payload header, as shown in Table 1, tells you both the payload structure and, for the single-unit structure, the type of MIHS unit in the payload 👑

       +===========+===================+==========================+
       | Unit Type | Payload Structure | Packet Type Name         |
       +===========+===================+==========================+
       |     0     | N/A               | Unassigned               |
       +-----------+-------------------+--------------------------+
       |     1     | Single            | Initialization MIHS Unit |
       +-----------+-------------------+--------------------------+
       |     2     | Single            | Temporal MIHS Unit       |
       +-----------+-------------------+--------------------------+
       |     3     | Single            | Spatial MIHS Unit        |
       +-----------+-------------------+--------------------------+
       |     4     | Single            | Silent MIHS Unit         |
       +-----------+-------------------+--------------------------+
       |     5     | Aggr              | Single-Time Aggregation  |
       |           |                   | Packet (STAP)            |
       +-----------+-------------------+--------------------------+
       |     6     | Aggr              | Multi-Time Aggregation   |
       |           |                   | Packet (MTAP)            |
       +-----------+-------------------+--------------------------+
       |     7     | Frag              | Fragmentation Unit       |
       +-----------+-------------------+--------------------------+

               Table 1: Payload Structure Type for Haptics

The payload structures are shown in Figure 4 🦋 The single unit structure is specified in Section 5.3.1 💜 The fragmented unit structure is specified in Section 5.3.2 🍓 The aggregation packet structure is specified in Section 5.3.3 🧚 The padding in the figures of these sections is the RTP padding defined in [RFC3550] 🌸

                                               +-------------------+
                                               |     RTP Header    |
                                               +-------------------+
                                               | RTP Payload Header|
                         +-------------------+ |   (UT = Aggr)     |
                         |     RTP Header    | +-------------------+
   +-------------------+ +-------------------+ |  MIHS Unit 1 Size |
   |     RTP Header    | | RTP Payload Header| +-------------------+
   +-------------------+ |   (UT = Frag)     | |    MIHS Unit 1    |
   | RTP Payload Header| +-------------------+ +-------------------+
   +-------------------+ |     FU Header     | |  MIHS Unit 2 Size |
   |    RTP Payload    | +-------------------+ +-------------------+
   | (Single MIHS unit)| |    RTP Payload    | |       ...         |
   +-------------------+ +-------------------+ +-------------------+

   (a) single unit      (b)fragmentation unit (c) aggregation packet

                      Figure 4: RTP Transmission Modes

5.3.1. Single Unit Payload Structure ✨

In the single unit structure (see Figure 5) 🎀 the RTP packet has the RTP header, then the payload header, then one single MIHS unit 💖 The payload header follows the structure in Section 5.2 🌸 The payload carries a MIHS unit as defined in [ISO.IEC.23090-31] 🫶

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          RTP Header                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |Payload Header |                                               |
   +---------------+                                               |
   |                        MIHS Unit Data                         |
   |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                               |...OPTIONAL RTP Padding        |
   +-------------------------------+-------------------------------+

                  Figure 5: Single Unit Payload Structure

5.3.2. Fragmented Unit Payload Structure ✨

In the fragmented unit structure (see Figure 6) 💜 the RTP packet has the RTP header, then the payload header, then a Fragmented Unit (FU) header, then a MIHS unit fragment 🦋 The payload header follows the structure in Section 5.2 🍓 The UT field of the payload header is 7 🧚 The FU header follows the structure in Figure 7 🌸 When fragmenting, all RTP payload header fields MUST remain unchanged across all fragments 🔥

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                          RTP Header                           |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |Payload Header | FU Header     |                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               |
   |                     MIHS Unit Fragment                        |
   |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                               |...OPTIONAL RTP Padding        |
   +-------------------------------+-------------------------------+

               Figure 6: Fragmentation Unit Payload Structure

FU headers are how you split one MIHS unit across multiple RTP packets 🦋 Fragments of the same MIHS unit MUST be sent in consecutive order with ascending RTP sequence numbers 💅 (no other RTP packets from the same RTP stream can be sent between the first and last fragment) 🫶 FUs MUST NOT be nested, i.e., an FU MUST NOT contain a subset of another FU 🚫

Figure 7 shows an FU header with these fields 💖

   +---+---+---+---+---+---+---+---+
   | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
   +---+---+---+---+---+---+---+---+
   |FUS|FUE|   RSV     |     UT    |
   +---+---+-----------+-----------+

                    Figure 7: Fragmentation Unit Header

FUS (Fragmented Unit Start): 1 bit. 🌈 This MUST be set to 1 for the first fragment and 0 for the other fragments 👑

FUE (Fragmented Unit End): 1 bit. 🔥 This MUST be set to 1 for the last fragment and 0 for the other fragments 💅

The combination FUS=1 and FUE=1 MUST NOT occur; such packets are invalid 💀

RSV (Reserved): 3 bits. 🧚 These bits MUST be set to 0 by the sender and ignored by the receiver 🌸

UT (Unit Type): 3 bits. 💜 This tells you the type of MIHS unit this fragment belongs to, using the values in Table 1 🍓

The thing about MIHS unit fragmentation in RTP is that a media receiver can get some fragments but not others 😭 The missing fragments typically won't be retransmitted by RTP, so you end up with partially received MIHS units 🦋 Depending on the implementation, those can be dropped or used by the decoding app 🫶 If consecutive fragments with FUE and FUS are lost, the receiver might be able to tell that the surrounding fragments belong to a different partially received MIHS unit (e.g., if the UT field has a different value) 🔮 It's giving tricky, but it's manageable bestie ✨

5.3.3. Aggregation Packet Payload Structure 🧚

okay so aggregation packets are giving bundle energy 💅. The RTP packet has an RTP header, then a payload header, and then, for each aggregated MIHS unit, a MIHS unit size followed by the MIHS unit itself. The payload header uses the same structure from Section 5.2, so no surprises there 🫶

        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                          RTP Header                           |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |RTP Payl. Head.|       MIHS Unit 1 Size        |               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+               |
       |                                                               |
       |                         MIHS Unit 1                           |
       :                                                               :
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |        MIHS Unit 2 Size     |                                 |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                                 |
       |                           MIHS Unit 2                         |
       |                                                               |
       |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                               |...OPTIONAL RTP Padding        |
       +-------------------------------+-------------------------------+

Figure 8: Single-Time Aggregation Packet

Figure 8 is a Single-Time Aggregation Packet (STAP), and it's all about bundling MIHS units that share the same timestamp 🎀. For example, if two frequencies are used for the same content, you can send them together in one STAP, no cap ✨. Multiple spatial units can also go in a STAP together, since this type of haptics data is time independent 🌸. The MIHS unit length field is 16 bits and holds the length of the MIHS unit after it, in bytes. And the UT field of the payload header is 5 for this one 💜.

        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                          RTP Header                           |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |RTP Payl. Head.|       MIHS Unit 1 Size        |   TS Offset   |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |    TS Offset  |                                               |
       +-+-+-+-+-+-+-+-+                                               |
       |                           MIHS Unit 1                         |
       |                                                               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |       MIHS Unit 2 Size        |          TS Offset            |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                                                               |
       |                           MIHS Unit 2                         |
       |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                               |...OPTIONAL RTP Padding        |
       +-------------------------------+-------------------------------+

Figure 9: Multiple-Time Aggregation Packet

Figure 9 is the Multi-Time Aggregation Packet (MTAP) 🦋. It carries multiple MIHS units with different timestamps inside one RTP packet. Multi-time aggregation can help cut down on the number of packets when some delay is totally okay 😌. The UT field is 6 here, and the MIHS unit length field is still 16 bits, in bytes. The timestamp offset field (TS offset) is 16 bits and only shows up in the MTAP case. It MUST be set to (time of the MIHS unit - RTP timestamp of the packet). The earliest aggregation unit's timestamp offset MUST always be zero, which means the RTP timestamp of the MTAP is identical to the earliest MIHS unit time 🔥.

5.4. MIHS Units Transmission and Reception Considerations 🌈

okay, here's the tea on streaming MIHS units over RTP 🍓. The following considerations apply.

The MIHS format lets units have variable duration. It uses initialization MIHS units to declare the duration of the non-zero duration MIHS units that follow, plus the maximum variation of that duration. A sender SHOULD set constant or low-variability durations (e.g., lower than the playout buffer) in initialization MIHS units for RTP streaming 🫶. This helps the receiver figure out early (e.g., with a timer) when a unit has been lost, and it makes the decoder more robust to RTP packet loss. But if a sender sends MIHS units with high duration variations, the receiver MAY need to wait a long time (e.g., the upper bound of the duration variation) to determine if a MIHS unit was lost in transmission 😭. Whether that's acceptable is application dependent, and the application can configure the encoder to generate MIHS units with lengths that have the right amount of variation.

The MIHS format uses silent MIHS units to signal haptic silence 🤫. A sender MAY decide not to send silent units to save network resources. From a receiver's standpoint, a missed MIHS unit could come from a silent unit that was never sent or from a lost packet, so a sender MAY send one, or a few, MIHS silent units at the beginning of a haptic silence. If a media receiver gets a MIHS silent unit, the receiver SHOULD assume that silence is intended until a non-silent MIHS unit is received. This can reduce the number of false detections of lost RTP packets by the decoder 💀.

In some multimedia conference scenarios using an RTP video mixer (e.g., when adding or selecting a new source), it is recommended to use Full Intra Request (FIR) feedback messages [RFC5104] with haptics 🎧. The purpose of the FIR message is to make an encoder send a decoder refresh point at the earliest opportunity. For haptics, an appropriate decoder refresh point is an initialization MIHS unit. That initialization MIHS unit point lets a decoder reset to a known state and decode all MIHS units that follow it 💖.

6. Payload Format Parameters 🎀

This section describes payload format parameters 💅. Section 6.1 specifies new optional parameters, and Section 6.2 further registers a new token in the media subregistry of the "Session Description Protocol (SDP) Parameters" registry group ✨.

6.1. Optional Parameters Definition 🦋

It is optional to include the SDP parameters in this section 🌸. Some parameters have a default value that MUST be inferred if the parameter is not present in the SDP, unless an out-of-band agreement indicates a different value, as described in Section 7.1. The values of the SDP parameters indicated in this section are based on the current version of the MPEG Haptics Coding standard (ISO/IEC 23090-31:2025) and may be different in future versions of [ISO.IEC.23090-31].

  • ver: This parameter provides the year of the edition and amendment of ISO/IEC 23090-31 that this file conforms to, as defined in [ISO.IEC.23090-31]: MPEG_haptics object.version is a string that may contain values such as XXXX or XXXX-Y, where XXXX is the year of publication and Y is the amendment number, if any. For the initial (and current) version of the MPEG Haptics Coding standard (ISO/IEC 23090-31:2025), the value is "2025". When ver is not present, a default value of "2025" SHOULD be inferred 🌈.
  • profile: This parameter indicates the profile used to generate the encoded stream, as defined in [ISO.IEC.23090-31]: MPEG_haptics object.profile is a string that may contain the value "simple-parametric" or "main". When profile is not present, the default value "main" SHOULD be inferred 💜.
  • lvl: This parameter indicates the level used to generate the encoded stream, as defined in [ISO.IEC.23090-31]: MPEG_haptics object.level is an integer that may contain the value 1 or 2. When lvl is not present, the default value 2 SHOULD be inferred ⭐.
  • maxlod: This parameter indicates the maximum level of details (LOD) to use for the avatar(s). The avatar LOD is defined in [ISO.IEC.23090-31]: MPEG_haptics.avatar object.lod is an integer that may contain the value 0 or a positive integer 🧚.
  • avtypes: This parameter indicates, using a comma-separated list, the types of haptic perception represented by the avatar(s). The avatar type is defined in [ISO.IEC.23090-31]: MPEG_haptics.avatar object.type is a string that may contain the value "Vibration", "Pressure", "Temperature", or "Custom" 🫶.
  • modalities: This parameter indicates, using a comma-separated list, haptic perception modalities (e.g., pressure, acceleration, velocity, position, temperature, etc.). The perception modality is defined in [ISO.IEC.23090-31]: MPEG_haptics.perception object.perception_modality is a string that may contain the value "Pressure", "Acceleration", "Velocity", "Position", "Temperature", "Vibrotactile", "Water", "Wind", "Force", "Electrotactile", "Vibrotactile Texture", "Stiffness", "Friction", "Humidity", "User-defined Temporal", "User-defined Spatial", or "Other" 🔥.
  • bodypartmask: This parameter is an integer that indicates, using a bitmask, the location of the devices or actuators on the body. The body part mask is defined in [ISO.IEC.23090-31]: MPEG_haptics.reference_device object.body_part_mask is a 32-bit integer that may hold a bit mask using bit positions defined in Table 7 of [ISO.IEC.23090-31] 🍓.
  • maxfreq: This parameter is an integer that indicates the maximum frequency of haptic data for vibrotactile perceptions (Hz). Maximum frequency is defined in [ISO.IEC.23090-31]: MPEG_haptics.reference_device object.maximum_frequency 🌸.
  • minfreq: This parameter is an integer that indicates the minimum frequency of haptic data for vibrotactile perceptions (Hz). Minimum frequency is defined in [ISO.IEC.23090-31]: MPEG_haptics.reference_device object.minimum_frequency 🦋.
  • dvctypes: This parameter indicates, using a comma-separated list, the types of actuators. The device type is defined in [ISO.IEC.23090-31]: MPEG_haptics.reference_device object.type is a string that may contain the value "LRA", "VCA", "ERM", "Piezo", or "Unknown" 💜.
  • silencesupp: This parameter is an integer that indicates whether silence suppression should be used (value 1) or not (value 0). When silencesupp is not present, the default value 0 SHOULD be inferred 🤫.

6.2. SDP Parameter Registration ✨

okay so this memo registers a haptics token in the media subregistry of the "Session Description Protocol (SDP) Parameters" registry group 🎀 it has all the required info from the SDP registration procedure in Section 8.2 of [RFC8866], and here's the whole form:

Contact name:  Hyunsik Yang

Contact email address:  hyunsik.yang@interdigital.com

Name being defined (as it will appear in SDP):  haptics

Type of name:  media

Description:  The 'haptics' media type for the Session Description
   Protocol is used to describe a media stream whose content can be
   rendered as touch-related sensations.  The media subtype further
   describes the specific format of the haptics stream.  The
   'haptics' media type for SDP is used to establish haptics media
   streams.

Reference:  RFC 9993

7. SDP Considerations 🦋

bestie, mapping our payload format media type onto the SDP fields is done according to [RFC8866], so that's the source of truth 💅

The media name in the m= line of SDP MUST be haptics. 🌸

The encoding name in the a=rtpmap line of SDP MUST be hmpg. ✨

The clock rate in the a=rtpmap line may be any sampling rate, typically 8000. 🧚

The optional parameters (defined in Section 6.1), when present, MUST be included in the a=fmtp line of SDP. It's written as a media type string, a semicolon-separated list of parameter=value pairs. Parameter values, including string values, MUST be written without quotation marks ("") in SDP. Parameter values that are strings are not case sensitive and SHOULD be written in lowercase 🫶

here's an example of the hmpg RTP payload in SDP, periodt:

m=haptics 43291 UDP/TLS/RTP/SAVPF 115
a=rtpmap:115 hmpg/8000
a=fmtp:115 profile=main;lvl=1;ver=2025

7.1. SDP Offer/Answer Considerations 🎀

okay this part is about negotiating haptic using the offer/answer procedure from [RFC3264], and these considerations apply 👑

When used for a unidirectional stream, the SDP parameters represent the properties of the sender (on the sending side) and of the receiver (on the receiving side). When used for a sendrecv stream, the SDP parameters represent the properties of the receiver.

The receiver properties expressed using the SDP parameters ver, profile, and lvl correspond to implementation capabilities. The ver, profile, and lvl parameters MUST be used symmetrically in SDP offer and answer. That means their values in the answer MUST match those in the offer, either explicitly signaled or implicitly inferred. In the same session, ver, profile, and lvl MUST NOT be changed in subsequent offers or answers 🔥

the properties expressed using SDP parameters other than ver, profile, and lvl are recommendations for efficient data transmission, and they're not binding. A sender is encouraged but not required to conform to the parameters the receiver specifies 🌈 these properties MAY be set to different values in offers and answers, and they MAY be updated in subsequent offers or answers.

any receiver compliant with [ISO.IEC.23090-31] MUST be capable of decoding any stream with a compatible version, profile, and level. A receiver supporting a more general profile will accept a stream for the same or a less general profile (e.g., "main" is more general than "simple-parametric"). A receiver supporting a given level will accept a stream for the same or a lower level. A receiver supporting a given version will accept a stream for the same version and MAY accept other versions. A receiver MAY ignore any part of a received stream, e.g., a part it doesn't have support for rendering 💀

the haptic signal can be sampled at different rates. The MPEG Haptics Coding standard doesn't mandate a specific frequency, and a typical sample rate is 8000 Hz ✨

the parameter ver indicates the version of the haptic standard specification. If it isn't specified, the value "2025" indicating the MPEG Haptics Coding standard ISO/IEC 23090-31:2025 [ISO.IEC.23090-31] SHOULD be inferred, although the sender and receiver MAY use a specific value based on an out-of-band agreement. The parameter profile restricts the number of tools used (e.g., the simple-parametric profile enables simpler implementations than the main profile). If it isn't specified, the most general profile "main" SHOULD be inferred, although the sender and receiver MAY use a specific value based on an out-of-band agreement. The parameter lvl further characterizes implementations within a given profile, e.g., according to the maximum supported number of channels, bands, and perceptions. If it isn't specified, the most general level "2" SHOULD be inferred, although the sender and receiver MAY use a specific version based on an out-of-band agreement 🦋

other parameters can indicate bitstream properties as well as receiver capabilities. The parameters maxlod, avtypes, bodypartmask, maxfreq, minfreq, dvctypes, and modalities can be sent by a sender to reflect the characteristics of bitstreams, and they can be set by a receiver to reflect the nature and capabilities of local actuator devices or a preferred set of bitstream properties. For example, different receivers MAY have different sets of local actuators, in which case these parameters can select a stream adapted to the receiver. In some other cases, some receivers MAY indicate a preference for a set of bitstream properties such as perceptions, min/max frequency, or body part mask, which contribute the most to the user experience for a given application. In that case, these parameters can select a stream that includes and possibly prioritizes those properties. For example, if the haptic stream server provides more information than the body mask the receiver specified, the extra information can either be integrated into a single effect or ignored by the receiver 🍓

the parameter silencesupp can indicate sender and receiver capabilities or preferences. It says whether silence suppression should be used, as described in Section 5.4. If it isn't specified, the value "0", meaning no silence suppression, SHOULD be inferred, although the sender and receiver MAY use silence suppression based on an out-of-band agreement 🙈

7.2. Declarative SDP Considerations 🌸

when haptic content over RTP is offered with SDP in a declarative style, the parameters that can indicate both bitstream properties and receiver capabilities are used to indicate only bitstream properties. For example, here the parameters maxlod, bodypartmask, maxfreq, minfreq, dvctypes, and modalities declare the values the bitstream uses, not the capabilities for receiving bitstreams. A receiver of the SDP is required to support all parameters and values of the parameters provided. Otherwise, the receiver MUST reject or not participate in the session. It falls on the creator of the session to use values that the receiving application is expected to support 😭

8. Congestion Control Considerations 💜

the general congestion control considerations for transporting RTP data apply to MPEG-I haptic data over RTP as well [RFC3550] 🫶

it's possible to adapt network bandwidth usage by adjusting either the encoder bit rate or the stream content (e.g., the LOD, body parts, actuator frequency range, target device types, and modalities). The considerations in this section apply to best-effort networks and controlled environments ✨

in case of congestion, a receiver or intermediate node MAY prioritize independent packets over dependent ones, since the non-reception of an independent MIHS unit can prevent the decoding of multiple subsequent dependent MIHS units. In case of congestion, a receiver or intermediate node MAY prioritize initialization MIHS units over other units, since these contain metadata used to reinitialize the decoder. Additionally, a receiver or intermediate node MAY drop silent units before other types, as a receiver MAY interpret a missing unit as silence. It's also possible, using the layer field of the RTP payload header, to allocate MIHS units to different layers based on their content, so the haptic data that contributes the most to the user experience gets prioritized 👑 In case of congestion, intermediate nodes and receivers SHOULD use the MIHS layer value to determine the relative importance of haptic RTP packets 🔥

receivers should monitor timestamps and treat gaps as loss of the corresponding MIHS units. MIHS units, as defined in [ISO.IEC.23090-31], should be checked for structural integrity according to their type. When CRC16 or CRC32 information is present in MIHS units, receivers must validate data integrity, and units failing Cyclic Redundancy Checks (CRCs) should be treated as lost. Receivers should further monitor indicators of service degradation such as unexpected silent gaps, repeated decoder reinitializations, or decoding failures. Receivers should report packet loss to the sender using RTCP Receiver Reports [RFC3550] and, when available, may report detailed loss and jitter metrics using mechanisms described in [RFC4585] 🥹💅

9. Security Considerations 🔐✨

okay bestie, security is serious, but it's also my favorite part to talk about 💅 The RTP payload format has the usual RTP security threats, plus some extra drama because haptic devices touch the real physical world 🫶 Common RTP threats are covered in [RFC3550], and in RTP profiles like RTP/AVP [RFC3551], RTP/AVPF [RFC4585], RTP/SAVP [RFC3711], and RTP/SAVPF [RFC5124].

Haptic sensors and actuators live in the physical environment, so it's giving high stakes 😭 Sensors can leak information, and actuators can get damaged if someone tampers with the data. Misusing actuator features (force, position, temperature, vibration, electrotactile, etc.) can actually hurt the user 💀 For example, someone could set keyframe parameters (like amplitude, position, and frequency) or channel gain to a value outside the permissible range. Individual devices can add their own protections to cut down on these risks, but sometimes harm can happen even with values that are fine for that one device 🙈 Causing contact with the physical environment or triggering unexpected force feedback are both examples. So each haptic system should set up its own system-dependent security measures, and those are more error prone. To make attackers' lives harder, haptic transmission should be protected against malicious traffic injection and tampering 🛡️

Now, "Securing the RTP Framework: Why RTP Does Not Mandate a Single Media Security Solution" [RFC7202] explains that a payload format isn't supposed to decide which solutions meet the basic security goals (confidentiality, integrity, and source authenticity) for RTP in general 🌸 That job belongs to the application developer. You can find guidance on security options and important things to consider in "Options for Securing RTP Sessions" [RFC7201]. Heads up though: [RFC7201] is now considered dated, and several of the mechanisms it describes have since evolved 📅

Applications SHOULD use appropriate and current strong security mechanisms 🔥 For modern best practices, applications can look at these options:

  • (D)TLS-based protection: For guidance on using TLS 1.3 and DTLS, applications should check [BCP195], which has up-to-date recommendations.
  • IPsec-based protection: The current, relevant protocol specs are [RFC4303] ("IP Encapsulating Security Payload (ESP)") and [RFC7296] ("Internet Key Exchange Protocol Version 2 (IKEv2)") 🦋

This document doesn't pick one specific security mechanism. Instead, applications are responsible for choosing mechanisms that follow current best practices for confidentiality, integrity, and source authentication. They should also keep up with the changing security landscape beyond what [RFC7201] covers 🧚

The haptic codec used with this payload format uses a compression algorithm (see Sections 8.2.8.5 and 8.3.3.2 in [ISO.IEC.23090-31]) 🗜️ An attacker could inject pathological datagrams into the stream that are complicated to decode and overload the receiver, similar to what's described in [RFC3551]. Yikes, but noted 😤

End-to-end security (authentication, integrity, or confidentiality protection) will stop a Media-Aware Network Element (MANE) from doing media-aware operations, other than discarding complete packets 🚧 With confidentiality protection, it can't even discard packets in a media-aware way. So if a MANE is supposed to do those things, it's required to be a trusted entity that's included in the security context establishment 👑

10. IANA Considerations 📋

10.1. Media Type Registration Update

Okay, so this memo updates the 'hmpg' haptic subtype defined in [RFC9695] 🎀 It's for the MPEG-I haptics streamable binary coding format described in ISO/IEC 23090-31: Haptics coding [ISO.IEC.23090-31]. The memo adds optional parameters for this type, and you can find them in Section 6.1. Originally, the 'haptics/hmpg' registration from [RFC9695] didn't have any required or optional parameters. This document adds optional ones to unlock extra features while staying backward compatible ✨

There's also a mapping of the parameters into SDP [RFC8866] for apps that use SDP 💖 Other control protocols that don't use SDP could get equivalent parameters defined somewhere else. The receiver MUST ignore any parameter that isn't specified in this memo. No cap, that's a must-do 🙅‍♀️

IANA updated the registration for 'haptics', which is described in Section 6.2. It lives in the "haptics" registry inside the "Media Types" registry group, and this document got added as an additional reference 🌈

Here are the entries that identify the updates to the 'haptics/hmpg' registration:

Type name:  haptics

Subtype name:  hmpg

The following entries are replaced by this memo:

Optional parameters:  See Section 6.1 of RFC 9993

Person & email address to contact for further information:  Yeshwant
   Muthusamy (yeshwant@yeshvik.com) and Hyunsik Yang
   (hyunsik.yang@interdigital.com)

10.2. New SDP Parameters Media Registration

IANA registered the following in the "media" registry, which is part of the "Session Description Protocol (SDP) Parameters" registration group 🦋

                 +=======+==========+===========+
                 |  Type | SDP Name | Reference |
                 +=======+==========+===========+
                 | media | haptics  | RFC 9993  |
                 +-------+----------+-----------+

                             Table 2

11. References 📚

11.1. Normative References

11.2. Informative References

Acknowledgments 🫶

Thanks to Philippe Guillotel, Quentin Galvane, Jonathan Lennox, Marius Kleidl, and Stephan Wenger for the comments and discussions about this document 💜

Authors' Addresses 💌

Hyunsik Yang InterDigital United States of America Email: hyunsik.yang@interdigital.com

Xavier de Foy InterDigital Canada Email: xavier.defoy@interdigital.com

📦 where this page comes from