RFC 9695

The 'haptics' Top-Level Media Type

📅 published March 2025, from ietf

Internet Engineering Task Force (IETF) ✨ RFC 9695: The 'haptics' Top-Level Media Type 🦋

Okay bestie, so this RFC is by Y. K. Muthusamy and C. Ullrich, published March 2025 🌸. It's on the Standards Track 💅, which means it's an actual Internet Standard, not just a vibe. It doesn't obsolete or update any other RFCs, so it's a fresh start 🧚. Its ISSN is 2070-1721 📚.

Main quest: it registers a brand-new top-level media type called 'haptics' 💖. Under it, different formats for haptics (touch and vibration feedback stuff 🫶) can get their own subtypes.

Abstract

Okay so the abstract is giving "we're making a whole new top-level type for touch data" 🤭. It also registers a bunch of subtypes that already exist out in the wild, so they're officially part of the family now 👑.

Status of This Memo

This is an Internet Standards Track document 💅. It's a product of the IETF and has the consensus of the IETF community, plus public review and IESG approval. If you want more on what Internet Standards are, check Section 2 of RFC 7841 🌈. The current status, errata, and feedback info live at https://www.rfc-editor.org/info/rfc9695 ✨.

Copyright (c) 2025 IETF Trust and the persons identified as the document authors. All rights reserved. 💜

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they stand on the publication date. Read those carefully, bestie, because they explain your rights 🙈. Any code pulled out of this document has to include Revised BSD License text, as described in Section 4.e of the Trust Legal Provisions, and it comes with no warranty, per the Revised BSD License 📜.

Table of Contents

   1.  Introduction
   2.  Background and Justification
     2.1.  MPEG ISOBMFF
     2.2.  Haptic Sub-Modalities
     2.3.  Another Human Sense
     2.4.  Commercial Uptake
     2.5.  Haptic Data Formats in Use
     2.6.  Haptic Subtypes (Envisioned Standards)
     2.7.  'application' Top-Level Type Not Suitable
   3.  Security Considerations
   4.  IANA Considerations
     4.1.  Definition and Encoding
     4.2.  Registration Procedure
     4.3.  Subtype Registrations
       4.3.1.  IVS Haptics Type
       4.3.2.  HJIF Haptics Type
       4.3.3.  HMPG Haptics Type
   5.  Normative References
   6.  Informative References
   Authors' Addresses

1. Introduction 🌷

Okay so "haptics" is the whole idea of making touch-y sensations happen on a device 🫶. It's everywhere, especially on phones, where your screen buzzes when you type on that virtual keyboard ⌨️💅. Haptic tech is kinda different from audio and video, because it needs some kind of actuator to actually make you feel something 😭. On phones and game controllers those are usually tiny vibrating motors 📳, and in big car touchscreens they can be specialized piezoelectric materials ⚡. Basically every modern smartphone, game controller, and VR controller has haptics now, which makes them the perfect target for upgraded media experiences 🎀.

Internet Media Types [RFC6838] are how we label content moving over Internet protocols 🏷️. This document defines a new top-level type, 'haptics', following [RFC9694] 🌈. It means "this content describes haptic data" ✨, and different haptic formats can be registered under it 🦋.

2. Background and Justification 🎀

Haptic signals add an extra layer of immersion when they're paired with audio and video 🤩. Haptic tracks can live in their own files and play back in sync with audio/video files, giving you audio, visual, AND tactile stuff all at once 💖. Even more recently, haptic tracks have been embedded in standard file formats like ISOBMFF (ISO Base Media File Format), so they can play over one or more actuators at the same time as audio and video [ISOBMFF-IS] 🎬. Haptic signals also show up in RTP media streams, like game streaming, XR, and wearables 🥽.

2.1. MPEG ISOBMFF 🎞️

Historically, there's been no registration for haptic formats 😳. But haptics got proposed as a first-order media type (same level as audio and video) in ISOBMFF back in April 2020 🌸. That proposal kept going until it became an International Standard, and it got published in January 2022 [ISOBMFF-IS] 🥳. Haptics is officially part of ISO/IEC 14496-12 (ISOBMFF), 7th Edition ✨. So honestly, there's a strong case for adding haptics to the IETF's list of recognized top-level media types 👑.

Once the top-level type 'haptics' is registered, we picture these designations for haptics in mp4 files 🧚:

  • haptics/mp4: mp4 files with just haptic tracks and no audio or video (think streaming games, or haptic files for vests, belts, gloves, and so on) 🧤
  • video/mp4: mp4 files with video, audio, and haptics (keeping it consistent with existing mp4 video files) 🎥
  • audio/mp4: mp4 files with audio and haptics (keeping it consistent with existing audio-only mp4 files, no video) 🎧

2.2. Haptic Sub-Modalities 🌈

Haptics has several sub-modalities 💅:

  • Vibrotactile (touch, vibration) 📳
  • Kinesthetic (force feedback) 🕹️
  • Surface (surface friction) ✋
  • Spatial, non-contact (ultrasound) 🦇
  • Thermal (temperature) 🔥

So making 'haptics' its own top-level type means formats for all of these can be defined way more cleanly 🧹. That would not be possible if 'haptics' got stuck under 'audio', 'video', or 'application' 🙅‍♀️.

2.3. Another Human Sense 👀

Okay so 'audio' is about hearing and 'video' is about seeing 👂👁️. It only makes sense that touch, which is just as important, gets its own top-level type 'haptics' 🫶. Putting haptics under 'audio' or 'video' would be weird, because some files and use cases need haptics and have nothing to do with sound or visuals at all 🙃.

2.4. Commercial Uptake 📱

Haptics is quickly becoming a standard feature on consumer electronics 🔥. For example:

  • iPhone (231+ million units sold in 2023 [iPhone-Sales]): built-in support for haptic-encoded data 🍎
  • Android (1.27+ billion units sold in 2023 [Android-Sales] [GlobalSmartPhone-Sales]): API support for haptic buffers 🤖
  • W3C (HTML vibration API [W3C-Vibration]): optionally supported in mobile web browsers. W3C has also defined vibration extensions for gamepads [W3C-Gamepad] 🌐
  • Game consoles (45+ million units sold in 2023 [GameConsole-Sales]): MS Xbox, Sony PlayStation, Nintendo Switch, etc. 🎮
  • XR devices (30+ million units sold in 2023 [XRDevice-Sales]): OpenXR haptic API 🥽

Haptic media is expected to get passed around between these devices 🔁. Since they make up the majority of consumer electronics (CE) devices, there's a strong case for 'haptics' as a top-level media type 💜.

2.5. Haptic Data Formats in Use 🧩

There are already several haptic data formats that will live as subtypes under the proposed 'haptics' top-level media type ✨. These subtypes have not been standardized yet, but they're everywhere on devices around the world 🌍. They even predate the standardization of haptic tracks in ISOBMFF, which is a pretty compelling reason to make 'haptics' a top-level media type 👑:

  • 'ahap': The AHAP haptic data format [AHAP] is currently the standard encoding on all iOS devices and iOS connected game peripherals 🍏. It's been picked up beyond Apple devices too, with decoders available for Android and other XR systems 🌈.
  • 'ogg': Google introduced a proprietary extension to the OGG format in the latest version of Android 11 🤖. This encoding lets haptic media get stored in OGG files 🎶.
  • 'ivs': The IVS haptic data format [MPEG-Haptics-Encoder] is in use:
    • In mobile phones from LG Electronics (specifically the V30, V40, and newest V50 models) that are sold worldwide 📱
    • In gaming phones from ASUS (specifically the ROG, ROG Phone II, and ROG Phone 3 models) that are sold worldwide 🎮
  • 'hapt': The HAPT haptic data format is currently a vendor-specific format that's in use:

Given how widely these subtypes are used, it totally makes sense for 'haptics' to be a top-level media type 💅🫶.

2.6. Haptic Subtypes (Envisioned Standards) 🔮

The MPEG ISOBMFF proposal had an informative annex listing known haptic coding formats, with proposed FourCCs (four-character codes) for each one 📝. The first two, hmpg and hjif, have already been registered per this document and standardized in ISO-IEC 23090-31: Haptics coding [MPEG-Haptics-Coding] 🎉. For the rest, once their format names are finalized and standardized, they should also be registered as subtypes of the 'haptics' top-level type 💖:

  • 'hmpg': the MPEG-I haptics streamable binary coding format described in ISO/IEC 23090-31: Haptics coding [MPEG-Haptics-Coding] 🌊
  • 'hjif': the MPEG-I haptics JSON-based interchange format described in ISO/IEC 23090-31: Haptics coding [MPEG-Haptics-Coding] 🧾
  • IEEE 1918.1.1 vibrotactile coding standard [IEEE-191811], being developed under the IEEE Tactile Internet initiative as part of the 5G Ultra-Reliable Low-Latency Communication (URLLC) profile. Format name is yet to be finalized ⏳
  • Enumerated effects haptic coding format (based on MIDI). Format name is yet to be finalized 🎹
  • Audio-to-vibe haptic coding format (automatic audio-to-vibration conversion algorithms). Format name is yet to be finalized 🎵

Part 1 of 3 done, bestie! 🌈 More parts are coming, so stay tuned 🦋

2.7. 'application' Top-Level Type Not Suitable

okay so bestie, we've been building up to this and it's giving clarity 💅✨ haptics really doesn't belong under any of the other media types. there are three main reasons why 'haptics' doesn't fit under the 'application' top-level type, and I'm going to spell them out 🎀

  • haptics is literally wired into a sensory system, touch and motion 🫶, which makes it way more specific than the super abstract 'application' type. not a vibe match at all 💀
  • 'application' has historically meant applications, aka code 🔥. and code gets treated with a LOT of care for security. 'haptics' is not code, just like 'audio' and 'video' are not code either 🌸
  • haptics is a property of a media stream. it is not an application under any normal definition 🙈 so it deserves its own type 👑

3. Security Considerations

okay this section is serious but I'm gonna make it digestible 🥹

haptics are interpreted data structures 🧚. they represent collections of different media rendering instructions that are meant to be decoded and rendered on target device hardware. haptic data can be represented as collections of signal data and/or descriptive text in XML/JSON or a similar format 💖. signal data is typically not executed by endpoint processors, so it represents minimal security risk ✨. descriptive text is typically parsed and represented in memory using standard XML data structures 🌈. that data gets used to build one or more signals that get sent to the endpoint device hardware 🦋

because of the media/rendering nature of the data path for haptic-coded data, the security profile of haptic data is expected to be largely consistent with the security profile of visual and audio media data 💜

now the scary-ish part 😭. as with any synthesized media data (audio, video, and haptics), there is a security risk tied to executing commands based on the descriptive encoding. this can happen through its inherent extensibility, or through inserting arbitrary executable data into the descriptive format itself 💀. media rendering systems are normally implemented with a mix of user and kernel space execution, since these media have to ultimately make their way to a hardware system 🔥. in theory, malicious instructions hiding in descriptive haptic media could execute arbitrary code in kernel space, which would effectively bypass system permissions structures and/or execution sandboxes 😱

haptics, audio, and video media all have widespread use, so operating system and device driver implementors need to pay careful attention 👑. they should make sure synthesis and rendering signal paths do not provide attack surfaces for malicious payloads 🛡️

thermal haptic devices (the ones that give a sensation of heat) and kinesthetic haptic devices (the ones that give force feedback) could potentially injure users if the heat or force, respectively, isn't properly controlled or accidentally goes past safety levels 🚨 implementors need to make sure adequate measures are taken to prevent that 🫶

and finally, these security considerations apply to the subtype registrations described in this document, as well as all future haptics registrations 🎀

4. IANA Considerations

so IANA has registered 'haptics' in the "Top-Level Media Types" registry defined in [RFC9694] 🌸 and also registered several subtypes 💅. IANA has additionally added 'haptics' as an alternative value of "Type Name" in the media types registration form [Media-Type-Reg] ✨

4.1. Definition and Encoding

'haptics' is the primary media content type 💖. it indicates the content identified by it requires a certain haptics subsystem, like low-level haptics APIs, which in turn will require hardware capabilities such as one or more actuators to render the haptics media 🔥. the 'haptics' media type doesn't give any specific info about the underlying data format or how the haptics info should be interpreted. that's the job of the subtypes defined within a 'haptics' tree, which name the specific haptic formats 🧚

unrecognized subtypes of 'haptics' SHOULD be treated as 'application/octet-stream'. implementations MAY still pass unrecognized subtypes to the haptics subsystem and associated rendering hardware 🦋

4.2. Registration Procedure

new haptics formats SHOULD be requested using the Application for a Media Type online form [Media-Type-Reg] 📝. [RFC6838] SHOULD be consulted on registration procedures 💜. in particular, the haptics specification SHOULD preferably be freely available 🌈

note that new subtypes may define parameters 🎀. if an implementation does not recognize a parameter sub-value in the comma-separated list, it SHOULD ignore the sub-value and continue processing the other sub-values in the list ✨

4.3. Subtype Registrations

in this section, the initial entries under the top-level 'haptics' media type are specified 💅. they also serve as examples for future registrations 🫶

4.3.1. IVS Haptics Type

Type name:  haptics

Subtype name:  ivs

Required parameters:  N/A

Optional parameters:  N/A

Encoding considerations:  8bit if UTF-8; binary if UTF-16 or UTF-32

Interoperability considerations:  The IVS format is a device-
   independent haptic effect coding based on the XML format.  It is
   designed to enable interoperability between distinct physical
   endpoints.  Not all devices may be able to render all effects
   present in an IVS file.

Security considerations:  See Section 3 of RFC 9695.

Published specification:  ISO/IEC JTC 1/SC 29/WG 2 N0072 "Encoder
   Input Format for MPEG Haptics" [MPEG-Haptics-Encoder].

Applications that use this media type:  All applications that are
   able to create, edit, or display haptic media content.

Additional information:
   File extension(s):  Haptic file extensions used for IVS files:
      .ivs
   Macintosh file type code(s):  (no code specified)
   Macintosh Universal Type Identifier code:  N/A
   Fragment Identifier:  N/A
   Deprecated Alias:  N/A

Person & email address to contact for further information:
   Yeshwant Muthusamy (yeshwant@yeshvik.com)

Change controller:  ISO/IEC JTC1/SC 29/WG 7 (MPEG 3D Graphics and
   Haptic Coding)

4.3.2. HJIF Haptics Type

Type name:  haptics

Subtype name:  hjif

Required parameters:  N/A

Optional parameters:  N/A

Encoding considerations:  8bit if UTF-8; binary if UTF-16 or UTF-32

Interoperability considerations:  The HJIF format is a human-readable
   haptic effect coding based on the JSON format.  It is designed as
   an interchange format for temporal and spatial haptic effects.
   The haptic effects may target specific parts of the human body and
   may be associated with a reference device description allowing
   haptic rendering software to adapt the effects to available
   hardware.

Security considerations:  See Section 3 of RFC 9695.

Published specification:  ISO/IEC 23090-31: Haptics coding
   [MPEG-Haptics-Coding].

Applications that use this media type:  All applications that are
   able to create, edit, or display haptic media content.

Additional information:
   File extension(s):  Haptic file extensions used for HJIF files:
      .hjif
   Macintosh file type code(s):  (no code specified)
   Macintosh Universal Type Identifier code:  N/A
   Fragment Identifier:  N/A
   Deprecated Alias:  N/A

Person & email address to contact for further information:
   Yeshwant Muthusamy (yeshwant@yeshvik.com)

Change controller:  ISO/IEC JTC1/SC 29/WG 7 (MPEG 3D Graphics and
   Haptic Coding)

4.3.3. HMPG Haptics Type

Type name:  haptics

Subtype name:  hmpg

Required parameters:  N/A

Optional parameters:  N/A

Encoding considerations:  binary

Interoperability considerations:  The HMPG format is a streamable
   binary haptic effect coding.  It is designed to enable efficient
   coding of temporal and spatial haptic effects.  The haptic effects
   may target specific parts of the human body and may be associated
   with a reference device description allowing haptic rendering
   software to adapt the effects to available hardware.

Security considerations:  See Section 3 of RFC 9695.

Published specification:  ISO/IEC 23090-31: Haptics coding
   [MPEG-Haptics-Coding].

Applications that use this media type:  All applications that are
   able to create, edit, or display haptic media content.

Additional information:
   File extension(s):  Haptic file extensions used for HMPG files:
      .hmpg
   Macintosh file type code(s):  (no code specified)
   Macintosh Universal Type Identifier code:  N/A
   Fragment Identifier:  N/A
   Deprecated Alias:  N/A

Person & email address to contact for further information:
   Yeshwant Muthusamy (yeshwant@yeshvik.com)

Change controller:  ISO/IEC JTC1/SC 29/WG 7 (MPEG 3D Graphics and
   Haptic Coding)

5. Normative References

[RFC6838] Freed, N., Klensin, J., and T. Hansen, "Media Type Specifications and Registration Procedures", BCP 13, RFC 6838, DOI 10.17487/RFC6838, January 2013, <https://www.rfc-editor.org/info/rfc6838>.

[RFC9694] Dürst, M.J., "Guidelines for the Definition of New Top- Level Media Types", BCP 13, RFC 9694, DOI 10.17487/RFC9694, March 2025, <https://www.rfc-editor.org/info/rfc9694>.

6. Informative References ✨

okay so this is the receipts drawer 🧾💅 these are the informative references, which means background reading and not requirements, so you can relax and just enjoy the sources. the list covers haptic pattern files, codecs, media formats, browser APIs, and even some sales stats, bestie 🦋

Authors' Addresses 🫶

these are the people who wrote it, so you know exactly who to thank (or ask) 🌸

Yeshwant K. Muthusamy
600 Longwood Drive
Allen, TX 75013
United States of America
Phone: +1 469-854-9836
Email: yeshwant@yeshvik.com
Chris Ullrich
311 Court Ave
Ventura, CA 93003
United States of America
Phone: +1 805-320-0774
Email: chrisullrich@gmail.com
📦 where this page comes from