RFC 6295
RTP Payload Format for MIDI
RFC 6295: RTP Payload Format for MIDI 🎀
Part 1 of 39 💖
okay bestie, let's get into it 🦋 this is RFC 6295, which is the spec for sending MIDI over RTP (basically how music gets sent across the internet in real time 🎹✨).
- Published by: the Internet Engineering Task Force (IETF) 🌸
- RFC number: 6295
- Authors: J. Lazzaro and J. Wawrzynek, both from UC Berkeley 👑
- Category: Standards Track, which means it's an official Internet standard 🔥
- Date: June 2011
- ISSN: 2070-1721
- Obsoletes: RFC 4695, so this one replaces the older version 💅
Abstract ✨
okay so this doc describes an RTP payload format for the MIDI command language 🎶. It can encode every command that's allowed to show up on a MIDI 1.0 DIN cable, which is iconic 😭. It works for interactive stuff like network musical performance and for content delivery like file streaming 🎧. You can use it over unicast and multicast UDP and TCP, and it has tools for recovering gracefully when packets get lost (no more lost notes, bestie 🫶). The stream behaviour, including the MIDI rendering method, can be customised during session setup 💜. The format also works as a mode for the mpeg4-generic format, so it supports the MPEG 4 Audio Object Types for General MIDI, Downloadable Sounds Level 2, and Structured Audio 🌈. And yes, this document obsoletes RFC 4695 💖.
Status of This Memo 🧚
This is an Internet Standards Track document ⭐.
It's a product of the Internet Engineering Task Force (IETF) and represents the consensus of the IETF community. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG) 🫶. If you want more info on Internet Standards, check out Section 2 of RFC 5741 🍓.
For the current status, any errata, and how to give feedback, go to http://www.rfc-editor.org/info/rfc6295 🦋.
Copyright Notice 💅
Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved 🔥.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents, which you can find at http://trustee.ietf.org/license-info in the version in effect on the date of publication 📜. Please read these carefully, bestie, because they spell out your rights and restrictions for this document 🙈. Code Components pulled out of this document must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as described in the Simplified BSD License 💀.
Table of Contents 🌸
1. Introduction ....................................................4
1.1. Terminology ................................................6
1.2. Bitfield Conventions .......................................6
2. Packet Format ...................................................6
2.1. RTP Header .................................................7
2.2. MIDI Payload ..............................................11
3. MIDI Command Section ...........................................13
3.1. Timestamps ................................................14
3.2. Command Coding ............................................16
4. The Recovery Journal System ....................................22
5. Recovery Journal Format ........................................24
6. Session Description Protocol ...................................28
6.1. Session Descriptions for Native Streams ...................29
6.2. Session Descriptions for mpeg4-generic Streams ............30
6.3. Parameters ................................................33
7. Extensibility ..................................................34
8. Congestion Control .............................................35
9. Security Considerations ........................................35
10. Acknowledgements ..............................................36
11. IANA Considerations ...........................................37
11.1. rtp-midi Media Type Registration .........................38
11.1.1. Repository Request for audio/rtp-midi .............40
11.2. mpeg4-generic Media Type Registration ....................42
11.2.1. Repository Request for Mode rtp-midi for
mpeg4-generic .....................................44
11.3. asc Media Type Registration ..............................46
12. Changes from RFC 4695 .........................................48
Appendix A. The Recovery Journal Channel Chapters .................52
A.1. Recovery Journal Definitions ..............................52
A.2. Chapter P: MIDI Program Change ............................56
A.3. Chapter C: MIDI Control Change ............................57
A.3.1. Log Inclusion Rules ................................58
A.3.2. Controller Log Format ..............................59
A.3.3. Log List Coding Rules ..............................61
A.3.4. The Parameter System ...............................64
A.4. Chapter M: MIDI Parameter System ..........................66
A.4.1. Log Inclusion Rules ................................68
A.4.2. Log Coding Rules ...................................69
A.4.2.1. The Value Tool ..............................71
A.4.2.2. The Count Tool ..............................74
A.5. Chapter W: MIDI Pitch Wheel ...............................74
A.6. Chapter N: MIDI NoteOff and NoteOn ........................75
A.6.1. Header Structure ...................................77
A.6.2. Note Structures ....................................78
A.7. Chapter E: MIDI Note Command Extras .......................79
A.7.1. Note Log Format ....................................80
A.7.2. Log Inclusion Rules ................................80
A.8. Chapter T: MIDI Channel Aftertouch ........................81
A.9. Chapter A: MIDI Poly Aftertouch ..........................82
Appendix B. The Recovery Journal System Chapters ..................83
B.1. System Chapter D: Simple System Commands ..................83
B.1.1. Undefined System Commands .....................84
B.2. System Chapter V: Active Sense Command ....................87
B.3. System Chapter Q: Sequencer State Commands ................87
B.3.1. Non-Compliant Sequencers ......................89
B.4. System Chapter F: MIDI Time Code Tape Position ............90
B.4.1. Partial Frames ....................................93
B.5. System Chapter X: System Exclusive ........................94
B.5.1. Chapter Format ................................94
B.5.2. Log Inclusion Semantics .......................96
B.5.3. TCOUNT and COUNT Fields .......................99
Appendix C. Session Configuration Tools ....... ..................100
C.1. Configuration Tools: Stream Subsetting ...................101
C.2. Configuration Tools: The Journalling System ..............106
C.2.1. The j_sec Parameter ...............................106
C.2.2. The j_update Parameter ............................107
C.2.2.1. The anchor Sending Policy ..................108
C.2.2.2. The closed-loop Sending Policy .............109
C.2.2.3. The open-loop Sending Policy ...............113
C.2.3. Recovery Journal Chapter Inclusion Parameters .....114
C.3. Configuration Tools: Timestamp Semantics .................119
C.3.1. The comex Algorithm ...............................120
C.3.2. The async Algorithm ...............................121
C.3.3. The buffer Algorithm ..............................122
C.4. Configuration Tools: Packet Timing Tools .................123
C.4.1. Packet Duration Tools .............................123
C.4.2. The guardtime Parameter ...........................124
C.5. Configuration Tools: Stream Description ..................125
C.6. Configuration Tools: MIDI Rendering ......................131
C.6.1. The multimode Parameter ...........................132
C.6.2. Renderer Specification ............................133
C.6.3. Renderer Initialization ...........................135
C.6.4. MIDI Channel Mapping ..............................137
C.6.4.1. The smf_info Parameter .....................138
C.6.4.2. The smf_inline, smf_url, and smf_cid
Parameters .................................140
C.6.4.3. The chanmask Parameter .....................140
C.6.5. The audio/asc Media Type ..........................141
C.7. Interoperability .........................................143
C.7.1. MIDI Content-Streaming Applications ...............144
C.7.2. MIDI Network Musical Performance Applications .....147
Appendix D. Parameter Syntax Definitions .... ....................153
Appendix E. A MIDI Overview for Networking Specialists ...........160
E.1. Commands Types ...........................................162
E.2. Running Status ...........................................163
E.3. Command Timing ...........................................163
E.4. AudioSpecificConfig Templates for MMA Renderers ..........164
References .......................................................169
Normative References ..........................................169
Informative References ........................................1701. Introduction 🎀
This document obsoletes [RFC4695].
okay so the IETF has a whole toolkit of focused tools for multimedia networking ([RFC3550] [RFC4566] [RFC3261] [RFC2326]) 💖. You can mix and match them in different ways to support a bunch of real-time apps over Internet Protocol (IP) networks. it's giving LEGO energy 🧱✨.
For example, a telephony app might use the Session Initiation Protocol (SIP, [RFC3261]) to set up a phone call 📞. Call setup would include negotiations to agree on a common audio codec [RFC3264]. Those negotiations would use the Session Description Protocol (SDP, [RFC4566]) to describe candidate codecs.
After a call is set up, the audio data flows between the parties using the Real Time Protocol (RTP, [RFC3550]) under any applicable profile (for example, the Audio/Visual Profile (AVP, [RFC3551])) 🎧. The tools from this telephony example (SIP, SDP, and RTP) can also be combined in a different way for a content-streaming app, maybe with other tools too, like the Real Time Streaming Protocol (RTSP, [RFC2326]) 🍿.
The MIDI (Musical Instrument Digital Interface) command language [MIDI] is widely used in music apps that are analogous to the examples above 🎹. On stage and in the recording studio, MIDI is used for interactive remote control of musical instruments, which is similar in spirit to telephony 🎤. On web pages, Standard MIDI Files (SMFs, [MIDI]) rendered using the General MIDI standard [MIDI] give you a low-bandwidth substitute for audio streaming 🌸.
[RFC4695] was motivated by a simple premise: if MIDI performances could be sent as RTP streams managed by IETF session tools, the MIDI and IETF application domains might hybridize 🦋.
For example, interoperable MIDI networking might foster network music performance apps, where a group of musicians in different physical locations interact over a network to perform as if they were in the same room [NMP]. As a second example, the streaming community might start using MIDI for low-bitrate audio coding, maybe alongside normative sound-synthesis methods [MPEGSA] 🎶.
Five years after [RFC4695], these apps haven't hit the mainstream yet. however, experiments in academia and industry keep going 🧪. This memo, which obsoletes [RFC4695] and fixes minor errata (see Section 12), was written in service of those experiments 💅.
To let MIDI apps use RTP, this memo defines an RTP payload format and its media type. Sections 2-5 and Appendices A and B define the RTP payload format. Section 6 and Appendices C and D define the media types that identify the payload format, the parameters needed for configuration, and how those parameters get used in SDP 🩷.
Appendix C also includes interoperability guidelines for the example apps above: network musical performance using SIP (Appendix C.7.2) and content streaming using RTSP (Appendix C.7.1) 🌈.
Another potential app area for RTP MIDI is MIDI networking for professional audio gear and electronic musical instruments. We do not offer interoperability guidelines for this app in this memo. however, RTP MIDI was designed with stage and studio apps in mind, and we expect efforts to define a stage and studio framework to rely on RTP MIDI for MIDI transport services 🎛️.
Some apps may need MIDI media delivery at a certain service quality level (latency, jitter, packet loss, etc.). RTP itself does not provide service guarantees. However, apps may use lower-layer network protocols to configure the quality of the transport services RTP uses. These protocols may reserve network resources for RTP flows [RFC2205] or may simply direct RTP traffic onto a dedicated "media network" in a local installation 🚦. Note that RTP and the MIDI payload format do provide tools that apps may use to get the best possible real-time performance at a given service level ⭐.
This memo normatively defines the syntax and semantics of the MIDI payload format. However, this memo does not define algorithms for sending and receiving packets. An ancillary document [RFC4696] provides informative guidance on algorithms. Supplemental info may be found in related conference publications [NMP] [GRAME] 📚.
Throughout this memo, the phrase "native stream" refers to a stream that uses the rtp-midi media type. The phrase "mpeg4-generic stream" refers to a stream that uses the mpeg4-generic media type (in mode rtp-midi) to operate in an MPEG 4 environment [RFC3640]. Section 6 describes this distinction in detail 👑.
1.1. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14, RFC 2119 [RFC2119] 📖.
1.2. Bitfield Conventions
Several bitfield coding idioms are used in this document. Since most of these idioms only show up in Appendices A and B, we define them in Appendix A.1 🍓.
However, a few of these idioms also appear in the main text, so here's a quick rundown:
- R flag bit. R flag bits are reserved for future use. Senders MUST set R bits to 0. Receivers MUST ignore R bit values. 🙈
- LENGTH field. All fields named
LENGTH(as distinct fromLEN) code the number of octets in the structure that contains it, including the header it resides in and all hierarchical levels below it. If a structure contains aLENGTHfield, a receiver MUST use theLENGTHfield value to advance past the structure during parsing, rather than use knowledge about the internal format of the structure. 💀
2. Packet Format 🧚
In this section, we introduce the format of RTP MIDI packets. The description includes some background on RTP for MIDI implementors who are new to IETF tools. Implementors should consult [RFC3550] for an authoritative description of RTP 💜.
This memo assumes the reader is familiar with MIDI syntax and semantics. Appendix E provides a MIDI overview at a level of detail sufficient to understand most of this memo. Implementors should consult [MIDI] for an authoritative description of MIDI 🎼.
The MIDI payload format maps a MIDI command stream (16 voice channels + systems) onto an RTP stream. An RTP media stream is a sequence of logical packets that share a common format. Each packet consists of two parts: the RTP header and the MIDI payload. Figure 1 shows this format (vertical space delineates the header and payload) ✨.
We describe RTP packets as "logical" packets to highlight the fact that RTP itself is not a network-layer protocol. Instead, RTP packets get mapped onto network protocols (such as unicast UDP, multicast UDP, or TCP) by an application [ALF] 🌊. The interleaved mode of the Real Time Streaming Protocol (RTSP, [RFC2326]) is an example of an RTP mapping to TCP transport, and so is [RFC4571] 🫶.
2.1. RTP Header ✨
[RFC3550] is the full guide to the RTP header fields 📖. In this section, we spell out what a few of those fields do for MIDI apps 🎹. Every field is coded in network byte order (big-endian) 💅.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| V |P|X| CC |M| PT | Sequence number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Timestamp |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SSRC |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MIDI command section ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Journal section ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Figure 1 -- Packet FormatThe M bit is only 1 bit, but it changes its behaviour depending on the media type of the stream 🦋. For native streams, the M bit MUST be set to 1 if the MIDI command section has a non-zero LEN field and MUST be set to 0 otherwise. For mpeg4-generic streams, the M bit MUST be set to 1 for all packets (to conform to [RFC3640]) 🎀.
In an RTP MIDI stream, the 16-bit sequence number field starts at a randomly chosen value and goes up by one (modulo 2^16) for each packet sent in the stream 🔢. A related quantity, the 32-bit extended packet sequence number, can be worked out by tracking rollovers of the 16-bit sequence number 🌈. Note that different receivers of the same stream may compute different extended packet sequence numbers, depending on when the receiver joined the session 🙈.
The 32-bit timestamp field sets the base timestamp value for the packet ⏰. The payload codes MIDI command timing relative to this value. The timestamp units are set by the clock rate parameter. For example, if the clock rate has a value of 44100 Hz, two packets whose base timestamp values differ by 2 seconds have RTP timestamp fields that differ by 88200 ✨.
Note that the clock rate parameter is NOT encoded within each RTP MIDI packet 👀. A receiver of an RTP MIDI stream finds out about the clock rate as part of the session setup process. For example, if a session management tool uses the Session Description Protocol (SDP, [RFC4566]) to describe a media session, the clock rate parameter is set using the rtpmap attribute 🧚. We show examples of session setup in Section 6 💜.
For RTP MIDI streams destined to be rendered into audio 🔊, the clock rate SHOULD be an audio sample rate of 32 KHz or higher. This recommendation is due to how sensitive human musical perception is to small timing errors in musical note sequences, and due to the timbral changes that happen when two near-simultaneous MIDI NoteOns are rendered with a different timing than the content author wanted, because of clock rate quantization 🎶. RTP MIDI streams that are not destined for audio rendering (such as MIDI streams that control stage lighting) MAY use a lower clock rate but SHOULD use a clock rate high enough to avoid timing artifacts in the application 💡.
For RTP MIDI streams destined to be rendered into audio, the clock rate SHOULD be chosen from rates in common use in professional audio applications or in consumer audio distribution 🎧. At the time of this writing, these rates include 32 KHz, 44.1 KHz, 48 KHz, 64 KHz, 88.2 KHz, 96 KHz, 176.4 KHz, and 192 KHz 🌸. If the RTP MIDI session is part of a synchronized media session that includes another (non-MIDI) RTP audio stream with a clock rate of 32 KHz or higher, the RTP MIDI stream SHOULD use a clock rate that matches the clock rate of the other audio stream 🫶. However, if the RTP MIDI stream is destined to be rendered into audio, the RTP MIDI stream SHOULD NOT use a clock rate lower than 32 KHz, even if this second stream has a clock rate lower than 32 KHz 🙅♀️.
Timestamps of consecutive packets do NOT necessarily increment at a fixed rate, because RTP MIDI packets are NOT necessarily sent at a fixed rate 🕰️. How regular the packet transmission is depends on the underlying application dynamics. Interactive apps may vary the packet-sending rate to follow the gestural rate of a human performer, whereas content-streaming apps may send packets at a fixed rate 🎹✨.
So the timestamps for two sequential RTP packets may be identical, or the second packet may have a timestamp arbitrarily larger than the first packet (modulo 2^32) 🤷♀️. Section 3 places additional restrictions on the RTP timestamps for two sequential RTP packets, as does the guardtime parameter (Appendix C.4.2) 📏.
We use the term "media time" to mean the temporal duration of the media coded by an RTP packet ⏳. The media time coded by a packet is computed by subtracting the last command timestamp in the MIDI command section from the RTP timestamp (modulo 2^32). If the MIDI list of the MIDI command section of a packet is empty, the media time coded by the packet is 0 ms. Appendix C.4.1 goes through media time issues in detail 🔍.
Now we define RTP session semantics, in the context of sessions specified using the Session Description Protocol [RFC4566] 💖. A session description media line ("m=") specifies an RTP session. An RTP session has an independent space of 2^32 synchronization sources. Synchronization source identifiers are coded in the SSRC header field of RTP session packets. The payload types that may appear in the PT header field of RTP session packets are listed at the end of the media line 📋.
Several RTP MIDI streams may appear in an RTP session 🎀. Each stream is distinguished by a unique SSRC value and has a unique sequence number and RTP timestamp space. Multiple streams in the RTP session may be sent by a single party. Multiple parties may send streams in the RTP session. An RTP MIDI stream encodes data for a single MIDI command name space (16 voice channels + systems) 🌈.
Streams in an RTP session may use different payload types, or they may use the same payload type. However, each party may send, at most, one RTP MIDI stream for each payload type mapped to an RTP MIDI payload format in an RTP session 🍓. Recall that dynamic binding of payload type numbers in [RFC4566] lets a party map many payload type numbers to the RTP MIDI payload format, so a party may send many RTP MIDI streams in a single RTP session 💅. Pairs of streams (unicast or multicast) that communicate between two parties in an RTP session and that share a payload type have the same association as a MIDI cable pair that cross-connects two devices in a MIDI 1.0 DIN network 🔌.
The RTP session architecture described above is efficient in its use of network ports, since one RTP session (using a port pair per party) supports the transport of many MIDI name spaces (16 MIDI channels + systems) 🦋. We define tools for grouping and labelling MIDI name spaces across streams and sessions in Appendix C.5 of this memo 🏷️.
The RTP header timestamps for each stream in an RTP session have separately and randomly chosen initialization values 🎲. Receivers use the timing fields encoded in the RTP Control Protocol (RTCP, [RFC3550]) sender reports to synchronize the streams sent by a party ⏱️. The SSRC values for each stream in an RTP session are also separately and randomly chosen, as described in [RFC3550]. Receivers use the CNAME field encoded in RTCP sender reports to verify that streams were sent by the same party and to detect SSRC collisions, as described in [RFC3550] 🕵️♀️.
In some apps, a receiver renders MIDI commands into audio (or into control actions, such as rewinding a tape deck or dimming stage lights) 💡. In other apps, a receiver presents a MIDI stream to software programs via an Application Programming Interface (API). Appendix C.6 defines session configuration tools to specify what receivers should do with a MIDI command stream ⚙️.
If a multimedia session uses different RTP MIDI streams to send different classes of media, the streams MUST be sent over different RTP sessions 🚫. For example, if a multimedia session uses one MIDI stream for audio and a second MIDI stream to control a lighting system, the audio and lighting streams MUST be sent over different RTP sessions, each with its own media line 🎭.
Session description tools defined in Appendix C.5 let a sending party split a single MIDI name space (16 voice channels + systems) over several RTP MIDI streams 🧩. Split transport of a MIDI command stream is a delicate task, because correct command stream reconstruction by a receiver depends on exact timing synchronization across the streams 😭.
To support split name spaces, we define the following requirements 📜:
- A party MUST NOT send several RTP MIDI streams that share a MIDI name space in the same RTP session. Instead, each stream MUST be sent from a different RTP session.
- If several RTP MIDI streams sent by a party share a MIDI name space, all streams MUST use the same SSRC value and MUST use the same randomly chosen RTP timestamp initialization value.
These rules let a receiver identify streams that share a MIDI name space (by matching SSRC values), and also let a receiver accurately reconstruct the source MIDI command stream (by using RTP timestamps to interleave commands from the two streams) 🪡. Care MUST be taken by senders to ensure that SSRC changes due to collisions are reflected in both streams. Receivers MUST regularly examine the RTCP CNAME fields associated with the linked streams to ensure that the assumed link is legitimate and not the result of an SSRC collision by another sender 🚨.
Except for the special cases described above, a party may send many RTP MIDI streams in the same session 💜. However, it is sometimes advantageous for two RTP MIDI streams to be sent over different RTP sessions. For example, two streams may need different values for RTP session-level attributes (such as the sendonly and recvonly attributes) 🌷. As a second example, two RTP sessions may be needed to send two unicast streams in a multimedia session that originate on different computers (with different IP numbers). Two RTP sessions are needed in this case because transport addresses are specified on the RTP-session or multimedia-session level, NOT on a payload type level 🧠.
On a final note, in some uses of MIDI, parties send bidirectional traffic to conduct transactions (such as file exchange) 🔁. These commands were designed to work over MIDI 1.0 DIN cable networks, and they may be configured in a multicast topology, which uses pure "party-line" signalling 📡. So, if a multimedia session ensures a multicast connection between all parties, bidirectional MIDI commands will work without additional support from the RTP MIDI payload format 🎉.
2.2. MIDI Payload 🎀
okay bestie, the payload (Figure 1) MUST begin with the MIDI command section 💖. That section codes a (possibly empty) list of timestamped MIDI commands and is literally the main event of the payload format ✨
the payload MAY also contain a journal section 🦋. The journal section is the resiliency queen 👑 because it codes the recent history of the stream. A flag in the MIDI command section codes whether a journal section is in the payload 🚩
Section 3 defines the MIDI command section 📋. Sections 4 and 5 and Appendices A and B define the recovery journal, which is the default format for the journal section 🌸. Here we're going to explain how all these payload sections work together in a stream in an RTP session, so pay attention 🧚
the journalling method for a stream gets locked in at the start of a session, and it MUST NOT be changed after that, no cap 🔒. A stream can be set to use the recovery journal, or to use an alternative journal format (none are defined in this memo), or to skip journalling entirely.
the default journalling method is inferred from the transport type 🚚. Streams on unreliable transport (like UDP) default to the recovery journal. Streams on reliable transport (like TCP) default to not using a journal. Appendix C.2.1 has the session configuration tools for overriding these defaults 🛠️. For all types of transport, a sender MUST transmit an RTP packet stream with consecutive sequence numbers (modulo 2^16) 🔢. Consecutive is the whole vibe!!
if a stream uses the recovery journal, every payload in the stream MUST include a journal section 💅. If a stream does not use journalling, a journal section MUST NOT appear in a stream payload 🙅♀️. If a stream uses an alternative journal format, the spec for that journal format defines the inclusion policy.
if a stream is sent over UDP, the Maximum Transmission Unit (MTU) of the underlying network limits how big the payload section can practically be. For example, an Ethernet MTU is 1500 octets 📦. This matters most for applications where predictable and minimal packet transmission latency is critical ⏱️. A sender SHOULD NOT create RTP MIDI UDP packets whose sizes exceed the MTU of the underlying network 😬. Instead, the sender SHOULD take steps to keep the maximum packet size under the MTU limit 💪.
there are lots of ways to do this 🌈. The default closed-loop recovery journal sending policy (defined in Appendix C.2.2.2) uses RTP Control Protocol (RTCP, [RFC3550]) feedback to manage the RTP MIDI packet size 📡. Section 3.2 and Appendix B.5.2 also give you specific tools for managing the size of packets that carry MIDI System Exclusive (0xF0) commands 🎛️. Appendix C.5 defines session configuration tools that can split a dense MIDI name space into several UDP streams (each sent in a different RTP session, per Section 2.1) so the payload fits comfortably into an MTU 🦋. Another option is to use TCP 💜. Section 4.3 of [RFC4696] provides non-normative advice for packet size management 🫶.
3. MIDI Command Section ✨
3.1. Timestamps ⏰
Figure 2 shows the format of the MIDI command section 👀.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|B|J|Z|P|LEN... | MIDI list ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 2 -- MIDI Command SectionThe MIDI command section starts with a variable-length header 📏.
the header field LEN codes the number of octets in the MIDI list that follow the header. If the header flag B is 0, the header is one octet long and LEN is a 4-bit field, so the max MIDI list length is 15 octets 🙃.
if B is 1, the header is two octets long and LEN is a 12-bit field, supporting a max MIDI list length of 4095 octets 🚀. LEN is coded in network byte order (big-endian). The 4 bits of LEN that appear in the first header octet code the most significant 4 bits of the 12-bit LEN value 🧠.
a LEN value of 0 is legal, and it codes an empty MIDI list 🫧.
if the J header bit is set to 1, a journal section MUST appear after the MIDI command section in the payload 📖. If the J header bit is set to 0, the payload MUST NOT contain a journal section 🙅.
we define the semantics of the P header bit in Section 3.2 🔜.
if the LEN header field is nonzero, the MIDI list has the structure shown in Figure 3 💗.
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Delta Time 0 (1-4 octets long, or 0 octets if Z = 0) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MIDI Command 0 (1 or more octets long) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Delta Time 1 (1-4 octets long) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MIDI Command 1 (1 or more octets long) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Delta Time N (1-4 octets long) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MIDI Command N (0 or more octets long) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 3 -- MIDI List Structureif the header flag Z is 1, the MIDI list begins with a complete MIDI command (coded in the MIDI Command 0 field in Figure 3), preceded by a delta time (coded in the Delta Time 0 field) 🎶. If Z is 0, the Delta Time 0 field isn't in the MIDI list at all, and the command in the MIDI Command 0 field has an implicit delta time of 0 🫠.
the MIDI list structure may also optionally encode a list of N additional complete MIDI commands, each coded in a MIDI Command K field 🎁. Each additional command MUST be preceded by a Delta Time K field, which codes that command's delta time ⏳. We discuss exceptions to the "command fields code complete MIDI commands" rule in Section 3.2 📝.
the final MIDI command field (i.e., the MIDI Command N field, shown in Figure 3) in the MIDI list MAY be empty 🫥. Moreover, a MIDI list MAY consist of a single delta time (encoded in the Delta Time 0 field) without an associated command (which would have been encoded in the MIDI Command 0 field). These rules enable MIDI coding features that we explain in Section 3.1. We wait to explain them because you need to understand RTP MIDI timestamps first to get them 🧚♀️.
3.1. Timestamps (the real tea) 🍵
in this section we describe how RTP MIDI encodes a timestamp for each MIDI list command ⏱️. Command timestamps use the same units as RTP packet header timestamps (described in Section 2.1 and [RFC3550]). Recall that RTP timestamps have units of seconds, and their scaling is set during session configuration (see Section 6.1 and [RFC4566]) 📐.
as shown in Figure 3, the MIDI list encodes time using a compact delta time format 🗜️. The RTP MIDI delta time syntax is a modified form of the MIDI File delta time syntax [MIDI]. RTP MIDI delta times use 1-4 octet fields to encode 32-bit unsigned integers 🔟. Figure 4 shows the encoded and decoded forms of delta times. Note that delta time values may be legally encoded in multiple formats. For example, there are four legal ways to encode the zero delta time (0x00, 0x8000, 0x808000, 0x80808000) 🤯.
RTP MIDI uses delta times to encode a timestamp for each MIDI command 🎯. The timestamp for MIDI Command K is the summation (modulo 2^32) of the RTP timestamp and decoded delta times 0 through K. This cumulative coding technique, borrowed from MIDI File delta time coding, is efficient because it cuts down on the number of multi-octet delta times 💫.
all command timestamps in a packet MUST be less than or equal to the RTP timestamp of the next packet in the stream (modulo 2^32) 🫶.
this restriction makes sure that a particular RTP MIDI packet in a stream is uniquely responsible for encoding time. It starts at the moment after the RTP timestamp in the RTP packet header and ends at the moment before the final command timestamp in the MIDI list 🕰️. The "moment before" and "moment after" qualifiers account for the "less than or equal" semantics (as opposed to "strictly less than") in the sentence above this paragraph 🤓.
note that it's possible to "pad" the end of an RTP MIDI packet with time that's guaranteed to be void of MIDI commands. You do this by setting the "Delta Time N" field of the MIDI list to the end of the void time and leaving out its matching "MIDI Command N" field (a syntactic construction the preamble of Section 3 expressly made legal) 🧁.
it's also possible to code an RTP MIDI packet to express that a period of time in the stream is void of MIDI commands 🌙. The RTP timestamp in the header codes the start of the void time. The MIDI list of this packet consists of a "Delta Time 0" field that codes the end of the void time. No other fields are present in the MIDI list (a syntactic construction the preamble of Section 3 also expressly made legal) 🫧.
by default, a command timestamp indicates the execution time for the command. The difference between two timestamps indicates the time delay between the execution of the commands. That difference may be zero, coding simultaneous execution 🎆. In this memo, we call this interpretation of timestamps "comex" (COMmand EXecution) semantics. We formally define comex semantics in Appendix C.3 📚.
the comex interpretation works well for transcoding a Standard MIDI File (SMF) into an RTP MIDI stream, since SMFs code a timestamp for each MIDI command stored in the file 🎹. To transcode an SMF that uses metric time markers, use the SMF tempo map (encoded in the SMF as meta-events) to convert metric SMF timestamp units into seconds-based RTP timestamp units 🎼.
the comex interpretation also works well for MIDI hardware controllers that code raw sensor data directly onto an RTP MIDI stream 🎛️. Note that this controller design is preferable to a design that converts raw sensor data into a MIDI 1.0 cable command stream and then transcodes that stream onto an RTP MIDI stream 💅.
the comex interpretation of timestamps is usually not the best timestamp interpretation for transcoding a MIDI source that uses implicit command timing (such as MIDI 1.0 DIN cables) into an RTP MIDI stream 🙈. Appendix C.3 defines alternatives to comex semantics and describes session configuration tools for selecting the timestamp interpretation semantics for a stream 🔧.
One-Octet Delta Time:
Encoded form: 0ddddddd
Decoded form: 00000000 00000000 00000000 0ddddddd
Two-Octet Delta Time:
Encoded form: 1ccccccc 0ddddddd
Decoded form: 00000000 00000000 00cccccc cddddddd
Three-Octet Delta Time:
Encoded form: 1bbbbbbb 1ccccccc 0ddddddd
Decoded form: 00000000 000bbbbb bbcccccc cddddddd
Four-Octet Delta Time:
Encoded form: 1aaaaaaa 1bbbbbbb 1ccccccc 0ddddddd
Decoded form: 0000aaaa aaabbbbb bbcccccc cddddddd
Figure 4 -- Decoding Delta Time Formats3.2. Command Coding 💜
Okay so the MIDI list is basically a menu of commands, and each MIDI Command field holds one command from the stuff that's allowed on a MIDI 1.0 DIN cable 🎹✨ Standard MIDI File meta-events are NOT on the menu, so they MUST NOT appear in the MIDI list 🙅♀️ Usually each field holds one complete command in the binary format from [MIDI], but there are a few plot twists, and we're about to get into them 🦋
The first MIDI channel command in the list MUST include a status octet 💖 After that, running status (from [MIDI]) MAY be used for the rest of the channel commands, which is such a bandwidth glow-up 🌸 Also, System Common and System Exclusive messages (0xF0 ... 0xF7) cancel the running status state, but System Real-Time messages (0xF8 ... 0xFF) do NOT mess with it, so they're lowkey the chillest ones 🧚 Every system command in the list MUST include a status octet too 🔥
Okay, bestie, I'm repeating myself on purpose because this part matters 🙈 The first channel command MUST include a status octet. But the original source stream might have skipped that octet because it was using running status. If the status octet of that first channel command is missing from the source data stream, the P (phantom) header bit MUST be set to 1 👻 In every other case, the P bit MUST be set to 0 ✨
Here's the tea: the P bit is about the MIDI source data stream, NOT the MIDI list encoding 🫶 So no matter what the P bit says, the MIDI list MUST include the status octet 💅
Receivers MUST be able to decode running status, so senders can totally use it to make things more efficient 🌈 Just don't be chaotic about it. Senders SHOULD NOT add timing jitter to an existing MIDI command stream by messing around with running status coding 😭 This especially matters if the receiver is going to transcode your RTP MIDI stream onto a MIDI 1.0 DIN cable [MIDI]. Both the timestamps and the command coding (running status or not) must comply with the physical limits of implicit time coding over a slow serial line 🐢
On a MIDI 1.0 DIN cable [MIDI], a System Real-Time command can hide inside another "host" MIDI command 👀 The payload format does NOT support that. A MIDI Command field in the MIDI list codes exactly one MIDI command (partially or completely) 🍓
So, to encode an embedded System Real-Time command, senders MUST pull it out of its host and code it in the MIDI list as its own separate command 💜 The host command and the System Real-Time command SHOULD land in the same MIDI list ✨ And the delta time of the System Real-Time command SHOULD give it a command timestamp that matches where it originally sat inside the host 🎀
Two methods exist for encoding MIDI System Exclusive (SysEx) commands in the MIDI list 🎶 The first is verbatim: a 0xF0 octet, then any number of data octets, then a 0xF7 octet 🦋
The second is segmentation. The command gets split into two or more SysEx command segments, and each segment gets its own MIDI Command field in the MIDI list 🧩
Segmentation is giving main character energy because it lets you encode SysEx commands that carry information in the timing of their data octets ⏱️ When you split a command into segments, each data octet can get its own delta time 🤯 It also lets a big SysEx command travel across several packets 📦
To segment a SysEx command, first split its data octet list into two or more sublists 🌷 The last sublist MAY be empty (no octets at all), but every other sublist MUST contain at least one data octet 💖 To finish the job, add the status octets from Figure 5 to the head and tail of the first, last, and any "middle" sublists 🫧 Figure 6 has some example segmentations, so go peek at those 👀
A sender MAY cancel a segmented SysEx transmission that's still in progress by sending the "cancel" sublist from Figure 5 🛑 A "cancel" sublist MAY follow a "first" or "middle" sublist, but it MUST NOT follow a "last" sublist 🙅♀️ The cancel MUST be empty, so 0xF7 0xF4 is the ONLY legal cancel sublist 💅
Why does cancelling even exist? Appendix C.1 defines configuration tools that let session parties exclude certain SysEx commands from the stream 🚫 So senders who transcode a MIDI source onto an RTP MIDI stream under those rules are responsible for leaving the undesired commands out of the RTP MIDI stream 🫶
The cancel feature lets a sender start sending a command before the MIDI source has finished sending it 🏃♀️ If the sender realizes the command in progress shouldn't be on the RTP stream, it cancels it 🙌 Without a cancel method, senders would have to store-and-forward everything first to validate each SysEx command before sending it, and that adds a lot of latency 🐌 Nobody wants that, bestie 😭
The recommended receiver reaction to a cancellation depends on what the receiver can do 🎛️ For example, a synth that parses RTP MIDI packets directly and renders them to audio knows SysEx commands can be cancelled in RTP MIDI 🧠 These receivers SHOULD detect a SysEx cancellation in the MIDI list and act as if they never received the SysEx command 🌈
Now a second example 💫 A synth might get its MIDI data from an RTP MIDI stream through a MIDI DIN cable (or a software API that pretends to be one). Here, an RTP-MIDI-aware system takes in the RTP MIDI stream and transcodes it onto that cable (or its emulation) 🔌 When the cancel sublist arrives, the transcoder might have already sent the first part of the SysEx command down the DIN cable to the receiver 😬
The annoying part is that the MIDI DIN cable protocol can't directly say "cancel SysEx in progress" 🙃 BUT, MIDI DIN cable receivers only start processing SysEx after the whole command arrives 📬 The receiver checks whether it recognizes the command (from the first few octets) and then checks whether the length is right 📏 So in practice, a transcoder can cancel a SysEx command by sending an 0xF7 to end it early 🛑 The receiver will notice the wrong length and throw the command away 🗑️ Iconic workaround, honestly 👑
Appendix C.1 also defines configuration tools that can be used to prohibit SysEx command cancellation 🔒
The order of SysEx command segments in a MIDI list MUST match the order of the sublists in the original SysEx command 📋 By default, commands other than System Real-Time MIDI commands MUST NOT appear between SysEx command segments (Appendix C.1 defines configuration tools to change this default and let other command types sit between segments) 🎀 If the segments of one SysEx command are spread across the MIDI lists of two or more RTP packets, those ordering rules apply to all the affected MIDI lists stitched together 🧵
Here's Figure 5, the segmentation status octets cheat sheet 📝
-----------------------------------------------------------
| Sublist Position | Head Status Octet | Tail Status Octet |
|-----------------------------------------------------------|
| first | 0xF0 | 0xF0 |
|-----------------------------------------------------------|
| middle | 0xF7 | 0xF0 |
|-----------------------------------------------------------|
| last | 0xF7 | 0xF7 |
|-----------------------------------------------------------|
| cancel | 0xF7 | 0xF4 |
-----------------------------------------------------------
Figure 5 -- Command Segmentation Status Octets[MIDI] lets 0xF7 octets that aren't part of a (0xF0, 0xF7) pair show up on a MIDI 1.0 DIN cable 🤷♀️ Those unpaired 0xF7 octets mean nothing in MIDI apart from cancelling running status 🫠
BUT, unpaired 0xF7 octets MUST NOT appear in the MIDI list of the MIDI Command section 🚫 This rule exists so nothing gets confused with the segmentation coding from Figure 5 🛡️
SysEx commands on a MIDI 1.0 DIN cable might use the "dropped 0xF7" trick from [MIDI] 🪄 In this method, the 0xF7 is dropped from the end of the SysEx command, and the status octet of the NEXT MIDI command both ends the SysEx command AND starts the next one 🤯 To encode that in the payload format, follow these steps:
- Figure out the right delta times for the SysEx command and for the command that comes after it.
- Insert the "dropped" 0xF7 octet at the end of the SysEx command so it has the standard SysEx syntax.
- Code both commands into the MIDI list using the rules above.
- Replace the 0xF7 octet that ends the verbatim SysEx encoding (or the last segment of the segmented SysEx encoding) with a 0xF5 octet. This substitution tells the receiver that the original coding used a "dropped 0xF7" ✨
[MIDI] reserves the undefined System Common commands 0xF4 and 0xF5, plus the undefined System Real-Time commands 0xF9 and 0xFD, for future use 🔮 By default, undefined commands MUST NOT appear in a MIDI Command field in the MIDI list, EXCEPT for the 0xF5 octets used to code the "dropped 0xF7" construction and the 0xF4 octets used by SysEx "cancel" sublists 🎀
During session configuration, a stream can be customized to carry undefined commands (Appendix C.1) 🧪 For that case, we now define how senders encode undefined commands in the MIDI list 💜
An undefined System Real-Time command MUST be coded using the System Real-Time rules 🌸
If undefined System Common commands get used in a future version of [MIDI], the command will begin with an 0xF4 or 0xF5 status octet, followed by any number of data octets (zero or more data bytes) 🔭 To encode these, senders MUST terminate the command with an 0xF7 octet and put the modified command into the MIDI Command field 💅
Unfortunately, non-compliant uses of the undefined System Common commands may show up in MIDI implementations 😤 To model these, we assume the command starts with an 0xF4 or 0xF5 status octet, then zero or more data octets, then zero or more trailing 0xF7 status octets 🫧 To encode the command, senders MUST first remove all trailing 0xF7 status octets from it. Then, senders MUST terminate the command with an 0xF7 octet and put the modified command into the MIDI Command field 🧹
Note that we include the trailing octets in our model as a precaution 🛡️ If such commands showed up in a non-compliant use of an undefined System Common command, an RTP MIDI encoding that did NOT remove the trailing octets could be mistaken for a "middle" or "last" sublist of a segmented SysEx command (Figure 5) under certain packet loss conditions 😱 Scary, right?
Here's Figure 6, the example segmentation 🎨
Original SysEx command:
0xF0 0x01 0x02 0x03 0x04 0x05 0x06 0x07 0x08 0xF7
A two-segment segmentation:
0xF0 0x01 0x02 0x03 0x04 0xF0
0xF7 0x05 0x06 0x07 0x08 0xF7A different two-segment segmentation, bestie 🎀:
0xF0 0x01 0xF0
0xF7 0x02 0x03 0x04 0x05 0x06 0x07 0x08 0xF7Ooh, a three-segment one 🌈:
0xF0 0x01 0x02 0xF0
0xF7 0x03 0x04 0xF0
0xF7 0x05 0x06 0x07 0x08 0xF7And the segmentation with the largest number of segments 👑:
0xF0 0x01 0xF0
0xF7 0x02 0xF0
0xF7 0x03 0xF0
0xF7 0x04 0xF0
0xF7 0x05 0xF0
0xF7 0x06 0xF0
0xF7 0x07 0xF0
0xF7 0x08 0xF0
0xF7 0xF7Figure 6 -- Example Segmentations
4. The Recovery Journal System ✨
The recovery journal is the default resiliency tool for unreliable transport 🛟. In this section, we normatively define the roles that senders and receivers play in the recovery journal system 💅.
MIDI is lowkey so fragile 🥺. A single lost command in a MIDI command stream may produce an artifact in the rendered performance. We normatively classify rendering artifacts into two categories:
- Transient artifacts. Transient artifacts produce immediate but short-term glitches in the performance. For example, a lost
NoteOn(0x9) command produces a transient artifact: one note fails to play, but the artifact does not extend beyond the end of that note. - Indefinite artifacts. Indefinite artifacts produce long-lasting errors in the rendered performance. For example, a lost
NoteOff(0x8) command may produce an indefinite artifact: the note that should have been ended by the lostNoteOffcommand may sustain indefinitely. As a second example, the loss of a Control Change (0xB) command for controller number 7 (Channel Volume) may produce an indefinite artifact: after the loss, all notes on the channel may play too softly or too loudly.
The purpose of the recovery journal system is to satisfy the recovery journal mandate: the MIDI performance rendered from an RTP MIDI stream sent over unreliable transport MUST NOT contain indefinite artifacts. 🚫
The recovery journal system does not use packet retransmission to satisfy this mandate. Instead, each packet includes a special section called the recovery journal 🎁.
The recovery journal codes the history of the stream back to an earlier packet called the checkpoint packet. The range of coverage for the journal is called the checkpoint history. The recovery journal codes the information necessary to recover from the loss of an arbitrary number of packets in the checkpoint history. Appendix A.1 normatively defines the checkpoint history.
When a receiver detects a packet loss, it compares its own knowledge about the history of the stream with the history information coded in the recovery journal of the packet that ends the loss event. By noting the differences in these two versions of the past, a receiver is able to transform all indefinite artifacts in the rendered performance into transient artifacts by executing MIDI commands to repair the stream 🔧✨.
We now state the normative role for senders in the recovery journal system.
Senders prepare a recovery journal for every packet in the stream. In doing so, senders choose the checkpoint packet identity for the journal. Senders make this choice by applying a sending policy. Appendix C.2.2 normatively defines three sending policies: "closed-loop", "open-loop", and "anchor".
By default, senders MUST use the closed-loop sending policy. If the session description overrides this default policy by using the parameter j_update defined in Appendix C.2.2, senders MUST use the specified policy.
After choosing the checkpoint packet identity for a packet, the sender creates the recovery journal. By default, this journal MUST conform to the normative semantics in Section 5 and Appendices A and B in this memo. In Appendix C.2.3, we define parameters that modify the normative semantics for recovery journals. If the session description uses these parameters, the journal created by the sender MUST conform to the modified semantics.
Next, we state the normative role for receivers in the recovery journal system.
A receiver MUST detect each RTP sequence number break in a stream. If the sequence number break is due to a packet loss event (as defined in [RFC3550]), the receiver MUST repair all indefinite artifacts in the rendered MIDI performance caused by the loss. If the sequence number break is due to an out-of-order packet (as defined in [RFC3550]), the receiver MUST NOT take actions that introduce indefinite artifacts (ignoring the out-of-order packet is a safe option) 👀.
Receivers take special precautions when entering or exiting a session. A receiver MUST process the first received packet in a stream as if it were a packet that ends a loss event. Upon exiting a session, a receiver MUST ensure that the rendered MIDI performance does not end with indefinite artifacts 🫶.
Receivers are under no obligation to perform indefinite artifact repairs at the moment a packet arrives. A receiver that uses a playout buffer may choose to wait until the moment of rendering before processing the recovery journal, as the "lost" packet may be a late packet that arrives in time to use ⏳.
Next, we state the normative role for the creator of the session description in the recovery journal system. The sender, the receivers, and other parties may take part in creating or approving the session description, depending on the application.
A session description that specifies the default closed-loop sending policy and the default recovery journal semantics satisfies the recovery journal mandate. However, these default behaviors may not be appropriate for all sessions. If the creators of a session description use the parameters defined in Appendix C.2 to override these defaults, the creators MUST ensure that the parameters define a system that satisfies the recovery journal mandate 📝.
Finally, we note that this memo does not specify sender or receiver recovery journal algorithms. Implementations are free to use any algorithm that conforms to the requirements in this section. The non-normative [RFC4696] discusses sender and receiver algorithm design 🌸.
5. Recovery Journal Format ✨
okay so this section is where we actually meet the recovery journal and its bitfields 🎀 Appendices A and B finish filling in the bitfield definitions, so keep them open, bestie 📚
The recovery journal has a three-level structure:
- Top-level header.
- Channel and system journal headers. These headers encode recovery information for a single voice channel (channel journal) or for all system commands (system journal).
- Chapters. Chapters describe recovery information for a single MIDI command type.
Figure 7 shows the top-level structure, and it's giving very tidy 💅 The recovery journal is a 3-octet header, then an optional system journal (labeled S-journal in Figure 7), then an optional list of channel journals. Figure 8 shows the recovery journal header format 🦋
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Recovery journal header | S-journal ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Channel journals ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 7 -- Top-Level Recovery Journal Format
0 1 2
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|Y|A|H|TOTCHAN| Checkpoint Packet Seqnum |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 8 -- Recovery Journal HeaderIf the Y header bit is set to 1, the system journal appears in the recovery journal, directly following the recovery journal header 🌸
If the A header bit is set to 1, the recovery journal ends with a list of (TOTCHAN + 1) channel journals (the 4-bit TOTCHAN header field is interpreted as an unsigned integer).
A MIDI channel MAY be represented by (at most) one channel journal in a recovery journal. Channel journals MUST appear in the recovery journal in ascending channel-number order. ✨
If A and Y are both zero, the recovery journal only contains its 3-octet header and is considered to be an "empty" journal 🫧
The S (single-packet loss) bit appears in most recovery journal structures, including the recovery journal header. It helps receivers parse the recovery journal efficiently in the common case of losing a single packet. Appendix A.1 defines S-bit semantics. Lowkey one of the most useful bits in the whole thing 🔥
The H bit indicates if MIDI channels in the stream have been configured to use the enhanced Chapter C encoding (Appendix A.3.3).
By default, the payload format does not use enhanced Chapter C encoding. In this default case, the H bit MUST be set to 0 for all packets in the stream.
If the stream has been configured so that controller numbers for one or more MIDI channels use enhanced Chapter C encoding, the H bit MUST be set to 1 in all packets in the stream. Appendix C.2.3 shows how to configure a stream to use enhanced Chapter C encoding 🧚
The 16-bit Checkpoint Packet Seqnum header field codes the sequence number of the checkpoint packet for this journal, in network byte order (big-endian). The choice of checkpoint packet sets the depth of the checkpoint history for the journal (defined in Appendix A.1). ⭐
Receivers may use the Checkpoint Packet Seqnum field of the packet that ends a loss event to verify that the journal checkpoint history covers the entire loss event. The checkpoint history covers the loss event if the Checkpoint Packet Seqnum field is less than or equal to one plus the highest RTP sequence number previously received on the stream (modulo 2^16). It's the verification check, bestie, and it's so iconic 👑
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| CHAN |H| LENGTH |P|C|M|W|N|E|T|A| Chapters ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 9 -- Channel Journal FormatFigure 9 shows a channel journal: a 3-octet header followed by a list of leaf elements called channel chapters. A channel journal encodes information about MIDI commands on the MIDI channel coded by the 4-bit CHAN header field. Note that CHAN uses the same bit encoding as the channel nibble in MIDI Channel Messages (the cccc field in Figure E.1 of Appendix E) 🎶
The 10-bit LENGTH field codes the length of the channel journal. The semantics for LENGTH fields are uniform throughout the recovery journal and are defined in Appendix A.1.
The third octet of the channel journal header is the Table of Contents (TOC) of the channel journal. The TOC is a set of bits that encode the presence of a chapter in the journal. Each chapter holds information about a certain class of MIDI channel command:
- Chapter P: MIDI Program Change (
0xC) - Chapter C: MIDI Control Change (
0xB) - Chapter M: MIDI Parameter System (part of
0xB) - Chapter W: MIDI Pitch Wheel (
0xE) - Chapter N: MIDI NoteOff (
0x8), NoteOn (0x9) - Chapter E: MIDI Note Command Extras (
0x8,0x9) - Chapter T: MIDI Channel Aftertouch (
0xD) - Chapter A: MIDI Poly Aftertouch (
0xA)
Chapters appear in a list after the header, in the order they show up in the TOC. Appendices A.2-A.9 describe the bitfield format for each chapter and define the conditions under which a chapter type MUST appear in the recovery journal. If any chapter types are required for a channel, an associated channel journal MUST appear in the recovery journal 💜
The H bit indicates if controller numbers on a MIDI channel have been configured to use the enhanced Chapter C encoding (Appendix A.3.3).
By default, controller numbers on a MIDI channel do not use enhanced Chapter C encoding. In this default case, the H bit MUST be set to 0 for all channel journal headers for the channel in the recovery journal, for all packets in the stream.
However, if at least one controller number for a MIDI channel has been configured to use the enhanced Chapter C encoding, the H bit for its channel journal MUST be set to 1, for all packets in the stream.
Appendix C.2.3 shows how to configure a controller number to use enhanced Chapter C encoding 🌈
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|D|V|Q|F|X| LENGTH | System chapters ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 10 -- System Journal FormatFigure 10 shows the system journal: a 2-octet header followed by a list of system chapters. Each chapter codes information about a specific class of MIDI system commands:
- Chapter D: Song Select (
0xF3), Tune Request (0xF6), Reset (0xFF), undefined system commands (0xF4,0xF5,0xF9,0xFD) - Chapter V: Active Sense (
0xFE) - Chapter Q: Sequencer State (
0xF2,0xF8,0xF9,0xFA,0xFB,0xFC) - Chapter F: MIDI Time Code (MTC) Tape Position (
0xF1,0xF0 0x7F 0xcc 0x01 0x01) - Chapter X: System Exclusive (all other
0xF0)
The 10-bit LENGTH field codes the size of the system journal and follows the semantics described in Appendix A.1.
The D, V, Q, F, and X header bits form a Table of Contents (TOC) for the system journal. A TOC bit set to 1 codes the presence of a chapter in the journal. Chapters appear in a list after the header, in the order they show up in the TOC 🍓
Appendix B describes the bitfield format for the system chapters and defines the conditions under which a chapter type MUST appear in the recovery journal. If any system chapter type is required to appear in the recovery journal, the system journal MUST appear in the recovery journal too. Main character energy from the system journal, no cap 💅
6. Session Description Protocol 🦋
RTP doesn't do session management on its own. Instead, it works together with session management tools, like the Session Initiation Protocol (SIP, [RFC3261]) and the Real Time Streaming Protocol (RTSP, [RFC2326]) 🎀
RTP payload formats define media type parameters for use in session management (for example, this memo defines rtp-midi as the media type for native RTP MIDI streams) ✨
In most cases, session management tools use those media type parameters through another standard, the Session Description Protocol (SDP, [RFC4566]).
SDP is a textual format for specifying session descriptions. Session descriptions specify the network transport and media encoding for RTP sessions. Session management tools coordinate the exchange of session descriptions between participants ("parties") 🫶
Some session management tools use SDP to negotiate details of media transport (network addresses, ports, etc.). We call this use of SDP "negotiated usage". One example of negotiated usage is the Offer/Answer protocol ([RFC3264] and Appendix C.7.2 in this memo) as used by SIP 🧚
Other session management tools use SDP to declare the media encoding for the session, but they use other techniques to negotiate network transport. We call this use of SDP "declarative usage". One example of declarative usage is RTSP ([RFC2326] and Appendix C.7.1 in this memo) 🌸
Below, we show session description examples for native (Section 6.1) and mpeg4-generic (Section 6.2) streams. In Section 6.3, we introduce session configuration tools that may be used to customize streams 💖
6.1. Session Descriptions for Native Streams
okay so this first one is the bare-bones native stream session description 💖 It sets up a unicast UDP RTP session (through an "m=" line) where the sole payload type, 96, gets mapped to a minimal native RTP MIDI stream 🎀
v=0
o=lazzaro 2520644554 2838152170 IN IP4 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP4 192.0.2.94
a=rtpmap:96 rtp-midi/44100The rtpmap line uses the rtp-midi media type to say "this is an RTP MIDI native stream" ✨ The clock rate on that line (44100 Hz in the example) sets the scaling for the RTP timestamp header field (see Section 2.1 and also [RFC3550]) ⏰
Heads up bestie 🙈 this doc does NOT give RTP MIDI a default clock rate. So when you're doing RTP MIDI with SDP, parties MUST use the rtpmap line to communicate the clock rate. Guidance for picking that value lives in Section 2.1 💅
We call this stream "minimal" because the session description doesn't customize it with any parameters. Without that customization, a native RTP MIDI stream has these vibes:
- If the stream uses unreliable transport (unicast UDP, multicast UDP, etc.), the recovery journal system is in use, and the RTP payload carries both the MIDI command section and the journal section. If the stream uses reliable transport (such as TCP), it skips journalling, and the payload only has the MIDI command section (Section 2.2) 🌸
- If the stream uses the recovery journal system, it runs the default sending policy and the default journal semantics (Section 4) 🧚
- In the MIDI command section of the payload, command timestamps use the default comex semantics (Section 3) ✨
- The recommended temporal duration ("media time") of an RTP packet ranges from 0 to 200 ms, and the RTP timestamp difference between sequential packets in the stream may be arbitrarily large (Section 2.1) 🦋
- If more than one minimal rtp-midi stream shows up in a session, their MIDI name spaces are independent. Channel 1 in the first stream does NOT point at the same MIDI channel as channel 1 in the second stream (see Appendix C.5 for a discussion of the independence of minimal rtp-midi streams) 👑
- The rendering method for the stream is not specified. What the receiver "does" with a minimal native MIDI stream is out of the scope of this memo 🎧 For example, in content creation environments, a user may manually configure client software to render the stream with a specific software package.
As is standard in RTP, RTP sessions managed by SIP are sendrecv by default (parties send and receive MIDI), and RTP sessions managed by RTSP are recvonly by default (server sends and client receives) 💜
In sendrecv RTP MIDI sessions for the session description above, the 16 voice channel + systems MIDI name space is unique for each sender. So in a two-party session, voice channel 0 sent by one party is totally distinct from voice channel 0 sent by the other party 🫶
This is exactly what happens when two MIDI 1.0 DIN devices are cross-connected with two MIDI cables (one cable routes MIDI Out from the first device into MIDI In of the second device, and a second cable routes MIDI In from the first device into MIDI Out of the second device). We define this "association" formally in Section 2.1 🔥
MIDI 1.0 DIN networks can be set up in a "party-line" multicast topology. For these networks, the MIDI protocol itself has tools for addressing specific devices in transactions on a multicast network, plus device discovery. So apart from providing a 1-to-many forward path and a many-to-1 reverse path, IETF protocols don't need any special support for MIDI multicast networking 🌈
6.2. Session Descriptions for mpeg4-generic Streams
an mpeg4-generic [RFC3640] RTP MIDI stream uses an MPEG 4 Audio Object Type to render MIDI into audio 🎹 Three Audio Object Types accept MIDI input:
- General MIDI (Audio Object Type ID 15), based on the General MIDI rendering standard [MIDI].
- Wavetable Synthesis (Audio Object Type ID 14), based on the Downloadable Sounds Level 2 (DLS 2) rendering standard [DLS2].
- Main Synthetic (Audio Object Type ID 13), based on Structured Audio and the programming language SAOL [MPEGSA]. The name of the language (SAOL) is an acronym that expands to Structured Audio Orechestra Language.
The main job of an mpeg4-generic stream is coding Access Units (AUs) 💅 We define the mpeg4-generic RTP MIDI AU as the MIDI payload shown in Figure 1 of Section 2.1 of this memo: a MIDI command section, optionally followed by a journal section 🌟
Exactly one RTP MIDI AU MUST be mapped to one mpeg4-generic RTP MIDI packet. The mpeg4-generic options for placing several AUs in an RTP packet MUST NOT be used with RTP MIDI, and the mpeg4-generic options for fragmenting and interleaving AUs MUST NOT be used with RTP MIDI either 🙅♀️ The mpeg4-generic RTP packet payload (Figure 1 in [RFC3640]) MUST contain empty AU Header and Auxiliary sections. These rules give you mpeg4-generic packets that are structurally identical to native RTP MIDI packets, which is an essential property for the correct operation of the payload format 🫧
These rules are non-negotiable, so no sneaking extra AUs into a packet 💀
The session description below defines a unicast UDP RTP session (through an "m=" line) whose sole payload type (96) is mapped to a minimal mpeg4-generic RTP MIDI stream. This example uses the General MIDI Audio Object Type under Synthesis Profile @ Level 2 🎀
v=0
o=lazzaro 2520644554 2838152170 IN IP6 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP6 2001:DB8::7F2E:172A:1E24
a=rtpmap:96 mpeg4-generic/44100
a=fmtp:96 streamtype=5; mode=rtp-midi; profile-level-id=12;
config=7A0A0000001A4D546864000000060000000100604D54726B0000
000600FF2F000(The a=fmtp line has been wrapped to fit the page to accommodate memo formatting restrictions; it comprises a single line in SDP.)
The fmtp attribute line codes the four parameters (streamtype, mode, profile-level-id, and config) that are required in all mpeg4-generic session descriptions [RFC3640]. For RTP MIDI streams, the streamtype parameter MUST be set to 5, the mode parameter MUST be set to rtp-midi, and the profile-level-id parameter MUST be set to the MPEG-4 Profile Level for the stream. For the Synthesis Profile, legal profile-level-id values are 11, 12, and 13, coding low (11), medium (12), or high (13) decoder computational complexity, as defined by MPEG conformance tests 🧠
In a minimal RTP MIDI session description, the config value MUST be a hexadecimal encoding [RFC3640] of the AudioSpecificConfig data block [MPEGAUDIO] for the stream. AudioSpecificConfig encodes the Audio Object Type for the stream and also encodes initialization data (SAOL programs, DLS 2 wave tables, etc.). Standard MIDI Files encoded in AudioSpecificConfig in a minimal session description MUST be ignored by the receiver 🙃
Receivers figure out the rendering algorithm for the session by reading the first 5 bits of AudioSpecificConfig as an unsigned integer that codes the Audio Object Type 🔍 In our example, those 5 bits sit inside the first two nibbles ("7A") of the config string. The Audio Object Type coded within "7A" is Audio Object Type 15 (General MIDI). In Appendix E.4, we derive the config string value in the session description shown above, and the starting point of that derivation is the MPEG bitstreams defined in [MPEGSA] and [MPEGAUDIO] ✨
We call the stream "minimal" because the session description doesn't customize it through parameters, other than the 4 required mpeg4-generic parameters described above. In Section 6.1, we describe the behavior of a minimal native stream as a numbered list of characteristics. Items 1-4 on that list also describe the minimal mpeg4-generic stream, but items 5 and 6 need restatements, as listed below 📋
- If more than one minimal mpeg4-generic stream appears in a session, each stream uses an independent instance of the Audio Object Type coded in the config parameter value.
- A minimal mpeg4-generic stream encodes the AudioSpecificConfig as an inline hexadecimal constant. If a session description is sent over UDP, it may be impossible to transport large AudioSpecificConfig blocks within the Maximum Transmission Unit (MTU) of the underlying network (for Ethernet, the MTU is 1500 octets). In some cases, the AudioSpecificConfig block may exceed the maximum size of the UDP packet itself 😭
The comments in Section 6.1 on SIP and RTSP stream directional defaults, sendrecv MIDI channel usage, and MIDI 1.0 DIN multicast networks also apply to mpeg4-generic RTP MIDI sessions 🦋
In sendrecv sessions, each party's session description MUST use identical values for the mpeg4-generic parameters (including the required streamtype, mode, profile-level-id, and config parameters). As a consequence, each party uses an identically configured MPEG 4 Audio Object Type to render MIDI commands into audio. The preamble to Appendix C discusses a way to create "virtual sendrecv" sessions that do not have this restriction 💖
6.3. Parameters ✨
okay so this part is all about parameters 💖 They're how you tune a session config for RTP MIDI streams. In a session description, a parameter tweaks what a payload type means, and they go on an fmtp attribute line. If you want to see one in action, check the session description example in Section 6.2 🎀
The parameters level up the minimal streams from Sections 6.1 and 6.2 and cover a few different services:
- Stream subsetting. By default, every MIDI command that's allowed on a MIDI 1.0 DIN cable can show up in an RTP MIDI stream 🎶 The
cm_unusedparameter flips that default by banning certain commands from the stream.cm_usedteams up withcm_unusedso you can write complicated exclusion rules without losing your mind 💅 Both are covered in Appendix C.1. - Journal customization.
j_secandj_updateset up how the journal section gets used.ch_default,ch_never, andch_anchordecide how the recovery journal chapters behave ✨ These are described in Appendix C.2, and they override default stream behaviors 1 and 2 (listed in Section 6.1 and referenced in Section 6.2). - MIDI command timestamp semantics.
tsmode,octpos,mperiod, andlineratecustomize how timestamps work in the MIDI command section ⏰ They let RTP MIDI accurately encode the implicit time coding of MIDI 1.0 DIN cables, which is giving precision 🦋 These are described in Appendix C.3 and override default stream behavior 3 (listed in Section 6.1 and referenced in Section 6.2). - Media time.
rtp_ptimeandrtp_maxptimedefine the temporal duration ("media time") of an RTP MIDI packet 🌸 Theguardtimeparameter sets the minimum sending rate of stream packets. These are described in Appendix C.4 and override default stream behavior 4 (listed in Section 6.1 and referenced in Section 6.2). - Stream description.
musicportlabels the MIDI name space of RTP streams in a multimedia session 🏷️ It's described in Appendix C.5 and overrides default stream behavior 5 (in Sections 6.1 and 6.2). - MIDI rendering. Several parameters specify the MIDI rendering method of a stream 🔥 These are described in Appendix C.6 and override default stream behavior 6 (in Sections 6.1 and 6.2).
In Appendix C.7, we spell out interoperability guidelines for two RTP MIDI application areas: content streaming using RTSP (Appendix C.7.1) and network musical performance using SIP (Appendix C.7.2) 🧚
7. Extensibility ✨
okay, so the payload format in this memo exclusively encodes all commands that are allowed to appear on a MIDI 1.0 DIN cable 🎹
Lots of worthy uses of MIDI over RTP sit outside the narrow scope of the payload format, though 🥺 For example, the payload format does not support direct transport of Standard MIDI File (SMF) meta-event and metric timing data. As a second example, it does not define transport tools for user-defined commands (apart from tools to support System Exclusive commands [MIDI]).
The payload format does not provide an extension mechanism for features like this, by design 💅 Instead, we encourage new payload formats for specialized musical applications. The IETF session management tools [RFC3264] [RFC2326] support codec negotiation, so new payload formats can be used in a backward-compatible way 🫶
That said, the payload format does come with several extensibility tools:
- Journalling. As described in Appendix C.2, new token values for the
j_secandj_updateparameters may be defined in IETF Standards-Track documents 📓 This supports the design of new journal formats and the definition of new journal sending policies. - Rendering. The payload format may be extended to support new MIDI renderers (Appendix C.6.2). Certain general aspects of the RTP MIDI rendering process may also be extended, through new token values for the
render(Appendix C.6) andsmf_info(Appendix C.6.4.1) parameters 🎨 - Undefined commands. [MIDI] reserves 4 MIDI system commands for future use (
0xF4,0xF5,0xF9,0xFD). If updates to [MIDI] define the reserved commands, IETF Standards-Track documents may be defined to provide resiliency support for them. Opaque LEGAL fields appear in System Chapter D for this purpose (Appendix B.1.1) 🔮
A final form of extensibility involves including the payload format in framework documents. Framework documents describe how to combine protocols into a platform for interoperable applications 👑 For example, a stage and studio framework might define how to use SIP [RFC3261], RTSP [RFC2326], SDP [RFC4566], and RTP [RFC3550] to support media networking for professional audio equipment and electronic musical instruments 🎤
8. Congestion Control ✨
The RTP congestion control requirements defined in [RFC3550] apply to RTP MIDI sessions, and implementors should carefully read the congestion control section in [RFC3550] 📖 As noted there, all transport protocols used on the Internet need to address congestion control in some way, and RTP is no exception 🚦
In addition, the congestion control requirements defined in [RFC3551] apply to RTP MIDI sessions run under applicable profiles. The basic one is that RTP sessions using UDP transport should monitor packet loss (via RTCP or other means) so the RTP stream competes fairly with TCP flows sharing the network 🌊
Finally, RTP MIDI has congestion control issues that are unique for an audio RTP payload format 🎵 In applications such as network musical performance [NMP], the packet rate is linked to the gestural rate of a human performer. So Senders MUST monitor the MIDI command source for patterns that result in excessive packet rates, and take actions during RTP transcoding to reduce the RTP packet rate 🙈 [RFC4696] offers implementation guidance on this issue.
9. Security Considerations ✨
🚨 This section is serious business, bestie. Read every word.
Implementors should carefully read the Security Considerations sections of the RTP [RFC3550], AVP [RFC3551], and other RTP profile documents, because the issues discussed there directly apply to RTP MIDI streams 🔐 Implementors should also review the Secure Real-time Transport Protocol (SRTP, [RFC3711]), an RTP profile that addresses the security issues discussed in [RFC3550] and [RFC3551].
Here, we discuss the security issues that are unique to the RTP MIDI payload format 💜
When using RTP MIDI, authentication of incoming RTP and RTCP packets is RECOMMENDED 🛡️ Per-packet authentication may be provided by SRTP or by other means. Without authentication, attackers could forge MIDI commands into an ongoing stream, damaging speakers and eardrums 😭 An attacker could also craft RTP and RTCP packets to exploit known bugs in the client and take effective control of a client machine. Iconic behavior from attackers, not it 💀
Session management tools (such as SIP [RFC3261]) SHOULD use authentication during transport of all session descriptions containing RTP MIDI media streams. For SIP, the Security Considerations section in [RFC3261] gives an overview of possible authentication mechanisms. RTP MIDI session descriptions should use authentication, because they may carry initialization data coded with the parameters described in Appendix C. If an attacker inserts bogus initialization data into a session description, they can corrupt the session or forge an client attack 🙅♀️
Session descriptions may also code renderer initialization data by reference, via the url (Appendix C.6.3) and smf_url (Appendix C.6.4.2) parameters. If the coded URL is spoofed, both session and client are open to attack, even if the session description itself is authenticated 😱 Therefore, URLs specified in url and smf_url parameters SHOULD use [RFC2818].
Section 2.1 allows streams sent by a party in two RTP sessions to have the same SSRC value and the same RTP timestamp initialization value, under certain circumstances. Normally these values are randomly chosen for each stream in a session, to make plaintext guessing harder if the payloads are encrypted. So yeah, Section 2.1 weakens this aspect of RTP security 🥀
10. Acknowledgements ✨
We thank the networking, media compression, and computer music community members who have commented on or contributed to the effort 🌈 That includes Kurt B, Cynthia Bruyns, Steve Casner, Paul Davis, Robin Davies, Joanne Dow, Tobias Erichsen, Roni Even, Nicolas Falquet, Adrian Farrel, Dominique Fober, Philippe Gentric, Michael Godfrey, Chris Grigg, Todd Hager, Alfred Hoenes, Russ Housley, Michel Jullian, Phil Kerr, Young-Kwon Lim, Jessica Little, Jan van der Meer, Alexey Melnikov, Colin Perkins, Charlie Richmond, Herbie Robinson, Dan Romascanu, Larry Rowe, Eric Scheirer, Dave Singer, Martijn Sipkema, Robert Sparks, William Stewart, Kent Terry, Sean Turner, Magnus Westerlund, Tom White, Jim Wright, Doug Wyatt, and Giorgio Zoia 🫶
We also thank the members of the San Francisco Bay Area music and audio community for creating the context for the work 🌴 That includes Don Buchla, Chris Chafe, Richard Duda, Dan Ellis, Adrian Freed, Ben Gold, Jaron Lanier, Roger Linn, Richard Lyon, Dana Massie, Max Mathews, Keith McMillen, Carver Mead, Nelson Morgan, Tom Oberheim, Malcolm Slaney, Dave Smith, Julius Smith, David Wessel, and Matt Wright 🎹✨
11. IANA Considerations ✨
okay bestie, this section is basically the paperwork bit 📋💅 most of it is copied word for word from Section 11 of [RFC4695] (it's a reproduction, not a rewrite), but there are a couple of things we changed first, so let's start there 🦋
Before the copied part, here's what we changed:
- every IANA reference to [RFC4695] got swapped out for a reference to this memo 🔄 and we also added a reference to this memo in the audio/mpeg4-generic MIME type registration 🎀
- in Section 11.3, we added a sentence to the Encoding Considerations for the audio/asc Media Type Registration: "Disk files that store this data object use the file extension ".acn"." 💾
The copied [RFC4695] section comes right after this, and it's the real request list 👑
This section makes a bunch of requests to IANA, and IANA has already finished registering/assigning all of them 🎉 The detailed requests live in the subsections, and every registration here is in the IETF tree and follows the rules of [RFC4288] and [RFC4855], as appropriate 🌸
Here's the lineup: Section 11.1 asks for a brand new media type, audio/rtp-midi 🎶 It comes with a request for a repository (basically a place for new values) for a bunch of its parameters, and that repository request is in Section 11.1.1 💖
Section 11.2 asks for a new value, rtp-midi, for the mode parameter of the mpeg4-generic media type ✨ mpeg4-generic is defined in [RFC3640], and [RFC3640] does define a repository for the mode parameter. BUT we think we're the first ones to ask for a mode value, which means IANA probably hasn't created the mode registry yet 🙈
Paired with that mode value request is another repository request, for new values of several parameters we defined for the rtp-midi mode value. That one is in Section 11.2.1 🧚
Section 11.3 asks for a new media type, audio/asc 🍓 No repository request goes with this one.
11.1. rtp-midi Media Type Registration
This section asks for the rtp-midi subtype of the audio media type to be registered 🎶 We want all the parameters listed in the "optional parameters" section below registered (both the "non-extensible parameters" and the "extensible parameters" lists) 💜 We also want repositories created for the "extensible parameters," and those details are in Section 11.1.1 🫶
Here's the full registration template, verbatim:
Media type name:
audio
Subtype name:
rtp-midi
Required parameters:
rate: The RTP timestamp clock rate. See Sections 2.1 and 6.1
for usage details.
Optional parameters:
Non-extensible parameters:
ch_anchor: See Appendix C.2.3 for usage details.
ch_default: See Appendix C.2.3 for usage details.
ch_never: See Appendix C.2.3 for usage details.
cm_unused: See Appendix C.1 for usage details.
cm_used: See Appendix C.1 for usage details.
chanmask: See Appendix C.6.4.3 for usage details.
cid: See Appendix C.6.3 for usage details.
guardtime: See Appendix C.4.2 for usage details.
inline: See Appendix C.6.3 for usage details.
linerate: See Appendix C.3 for usage details.
mperiod: See Appendix C.3 for usage details.
multimode: See Appendix C.6.1 for usage details.
musicport: See Appendix C.5 for usage details.
octpos: See Appendix C.3 for usage details.
rinit: See Appendix C.6.3 for usage details.
rtp_maxptime: See Appendix C.4.1 for usage details.
rtp_ptime: See Appendix C.4.1 for usage details.
smf_cid: See Appendix C.6.4.2 for usage details.
smf_inline: See Appendix C.6.4.2 for usage details.
smf_url: See Appendix C.6.4.2 for usage details.
tsmode: See Appendix C.3 for usage details.
url: See Appendix C.6.3 for usage details.
Extensible parameters:
j_sec: See Appendix C.2.1 for usage details. See
Section 11.1.1 for repository details.
j_update: See Appendix C.2.2 for usage details. See
Section 11.1.1 for repository details.
render: See Appendix C.6 for usage details. See
Section 11.1.1 for repository details.
subrender: See Appendix C.6.2 for usage details. See
Section 11.1.1 for repository details.
smf_info: See Appendix C.6.4.1 for usage details. See
Section 11.1.1 for repository details.
Encoding considerations:
The format for this type is framed and binary.
Restrictions on usage:
This type is only defined for real-time transfers of MIDI
streams via RTP. Stored-file semantics for rtp-midi may
be defined in the future.
Security considerations:
See Section 9 of this memo.
Interoperability considerations:
None.
Published specification:
This memo and [MIDI] serve as the normative specification. In
addition, references [NMP], [GRAME], and [RFC4696] provide
non-normative implementation guidance.
Applications that use this media type:
Audio content-creation hardware, such as MIDI controller piano
keyboards and MIDI audio synthesizers. Audio content-creation
software, such as music sequencers, digital audio workstations,
and soft synthesizers. Computer operating systems, for network
support of MIDI Application Programmer Interfaces. Content
distribution servers and terminals may use this media type for
low bitrate music coding.
Additional information:
None.
Person & email address to contact for further information:
John Lazzaro <lazzaro@cs.berkeley.edu>
Intended usage:
COMMON.
Author:
John Lazzaro <lazzaro@cs.berkeley.edu>
Change controller:
IETF Audio/Video Transport Working Group delegated
from the IESG.11.1.1. Repository Request for audio/rtp-midi
For the rtp-midi subtype, we're asking for repositories for extensions to the following parameters, which are the ones listed as "extensible parameters" in Section 11.1 🌈
j_sec- Registrations for this repository may only occur via an IETF Standards-Track document. Appendix C.2.1 of this memo describes the appropriate registrations for it.
- Initial values for this repository:
"none": Defined in Appendix C.2.1 of this memo."recj": Defined in Appendix C.2.1 of this memo.
j_update- Registrations for this repository may only occur via an IETF Standards-Track document. Appendix C.2.2 of this memo describes the appropriate registrations for it.
- Initial values for this repository:
"anchor": Defined in Appendix C.2.2 of this memo."open-loop": Defined in Appendix C.2.2 of this memo."closed-loop": Defined in Appendix C.2.2 of this memo.
render- Registrations for this repository MUST include a specification of how the proposed value is used. See the preamble of Appendix C.6 for details (the paragraph that begins "Other render token ...").
- Initial values for this repository:
"unknown": Defined in Appendix C.6 of this memo."synthetic": Defined in Appendix C.6 of this memo."api": Defined in Appendix C.6 of this memo."null": Defined in Appendix C.6 of this memo.
subrender- Registrations for this repository MUST include a specification of how the proposed value is used. See Appendix C.6.2 for details (the paragraph that begins "Other subrender token ...").
- Initial values for this repository:
"default": Defined in Appendix C.6.2 of this memo.
smf_info- Registrations for this repository MUST include a specification of how the proposed value is used. See Appendix C.6.4.1 for details (the paragraph that begins "Other smf_info token ...").
- Initial values for this repository:
"ignore": Defined in Appendix C.6.4.1 of this memo."sdp_start": Defined in Appendix C.6.4.1 of this memo."identity": Defined in Appendix C.6.4.1 of this memo.
11.2. mpeg4-generic Media Type Registration ✨
okay bestie, this part asks for rtp-midi to be added as a value for the mode parameter of the mpeg4-generic media type 💅 That media type is defined in [RFC3640], and [RFC3640] already has a repository for mode, so we're just adding a new value to the list 🎀 We want it so MPEG Audio codecs that use MIDI can work, per [MPEGSA] 🦋
We're also requesting registration of the parameters in the optional parameters section below (both the non-extensible and extensible lists) 🫶 For the extensible ones we want repositories created too, and the details are in Appendix 11.2.1 💜
Media type name: audio
Subtype name: mpeg4-generic
Required parameters:
The mode parameter is required by [RFC3640]. [RFC3640] sets up a repository for mode so new values can be added, and we're asking for rtp-midi to go into that mode repository 🌸
In mode rtp-midi, the mpeg4-generic parameter rate is a required parameter. Rate is the RTP timestamp clock rate. For how rate gets used in mode rtp-midi, check Sections 2.1 and 6.2 🧚
Optional parameters:
We're requesting registration of the following parameters for use in mode rtp-midi for mpeg4-generic.
Non-extensible parameters:
ch_anchor: See Appendix C.2.3 for usage details.
ch_default: See Appendix C.2.3 for usage details.
ch_never: See Appendix C.2.3 for usage details.
cm_unused: See Appendix C.1 for usage details.
cm_used: See Appendix C.1 for usage details.
chanmask: See Appendix C.6.4.3 for usage details.
cid: See Appendix C.6.3 for usage details.
guardtime: See Appendix C.4.2 for usage details.
inline: See Appendix C.6.3 for usage details.
linerate: See Appendix C.3 for usage details.
mperiod: See Appendix C.3 for usage details.
multimode: See Appendix C.6.1 for usage details.
musicport: See Appendix C.5 for usage details.
octpos: See Appendix C.3 for usage details.
rinit: See Appendix C.6.3 for usage details.
rtp_maxptime: See Appendix C.4.1 for usage details.
rtp_ptime: See Appendix C.4.1 for usage details.
smf_cid: See Appendix C.6.4.2 for usage details.
smf_inline: See Appendix C.6.4.2 for usage details.
smf_url: See Appendix C.6.4.2 for usage details.
tsmode: See Appendix C.3 for usage details.
url: See Appendix C.6.3 for usage details.Extensible parameters:
j_sec: See Appendix C.2.1 for usage details.
See Section 11.2.1 for repository details.
j_update: See Appendix C.2.2 for usage details.
See Section 11.2.1 for repository details.
render: See Appendix C.6 for usage details.
See Section 11.2.1 for repository details.
subrender: See Appendix C.6.2 for usage details.
See Section 11.2.1 for repository details.
smf_info: See Appendix C.6.4.1 for usage details.
See Section 11.2.1 for repository details.Encoding considerations: the format for this type is framed and binary 📦
Restrictions on usage: only defined for real-time transfers of audio/mpeg4-generic RTP streams with mode=rtp-midi ⏱️
Security considerations: see Section 9 of this memo 🔐
Interoperability considerations: except for the marker bit (Section 2.1), the packet formats for audio/rtp-midi and audio/mpeg4-generic (mode rtp-midi) are identical. The difference is in how they're used: audio/mpeg4-generic is for MPEG work, and audio/rtp-midi is for all other work 🎶
Published specification: this memo, [MIDI], and [MPEGSA] are the normative references. [NMP], [GRAME], and [RFC4696] give non-normative implementation guidance too 📚
Applications that use this media type: MPEG 4 servers and terminals that support [MPEGSA] 🖥️
Additional information: None.
Person & email address to contact for further information: John Lazzaro <lazzaro@cs.berkeley.edu> 💌
Intended usage: COMMON.
Author: John Lazzaro <lazzaro@cs.berkeley.edu>
Change controller: IETF Audio/Video Transport Working Group delegated from the IESG. 👑
11.2.1. Repository Request for Mode rtp-midi for mpeg4-generic 🍓
For mode rtp-midi of the mpeg4-generic subtype, we're asking for repositories for extensions to the following parameters. These are the ones listed as "extensible parameters" in Section 11.2 💅
- j_sec:
- Registrations for this repository may only occur via an IETF Standards-Track document. Appendix C.2.1 of this memo describes the appropriate registrations for it ✨
- Initial values for this repository are:
"none": Defined in Appendix C.2.1 of this memo."recj": Defined in Appendix C.2.1 of this memo.
- j_update:
- Registrations for this repository may only occur via an IETF Standards-Track document. Appendix C.2.2 of this memo describes the appropriate registrations for it 🎀
- Initial values for this repository are:
"anchor": Defined in Appendix C.2.2 of this memo."open-loop": Defined in Appendix C.2.2 of this memo."closed-loop": Defined in Appendix C.2.2 of this memo.
- render:
- Registrations for this repository MUST include a specification of how the proposed value is used. See the preamble of Appendix C.6 for details (the paragraph that begins
"Other render token ...") 🔥 - Initial values for this repository are:
"unknown": Defined in Appendix C.6 of this memo."synthetic": Defined in Appendix C.6 of this memo."null": Defined in Appendix C.6 of this memo.
- Registrations for this repository MUST include a specification of how the proposed value is used. See the preamble of Appendix C.6 for details (the paragraph that begins
- subrender:
- Registrations for this repository MUST include a specification of how the proposed value is used. See Appendix C.6.2 for details (the paragraph that begins
"Other subrender token ..."and the paragraphs after it). Heads up: Appendix C.6.2 has restrictions on subrender registrations for mpeg4-generic (the sentence that begins"Registrations for mpeg4-generic subrender values ...") 🙈 - Initial values for this repository are:
"default": Defined in Appendix C.6.2 of this memo.
- Registrations for this repository MUST include a specification of how the proposed value is used. See Appendix C.6.2 for details (the paragraph that begins
- smf_info:
- Registrations for this repository MUST include a specification of how the proposed value is used. See Appendix C.6.4.1 for details (the paragraph that begins
"Other smf_info token ...") 🌈 - Initial values for this repository are:
"ignore": Defined in Appendix C.6.4.1 of this memo."sdp_start": Defined in Appendix C.6.4.1 of this memo."identity": Defined in Appendix C.6.4.1 of this memo.
- Registrations for this repository MUST include a specification of how the proposed value is used. See Appendix C.6.4.1 for details (the paragraph that begins
11.3. asc Media Type Registration 🎀
This section registers asc as a subtype of the audio media type ✨ We're doing it to support remote transfer of the "config" parameter of the mpeg4-generic media type [RFC3640] when it's used with mpeg4-generic mode rtp-midi (registered in Appendix 11.2 above). The mechanics of using audio/asc to set the config parameter are explained in Section 6.2 and Appendix C.6.5 of this document 🧚
Quick note: this is a brand new subtype registration. It is not an addition to a repository defined by MPEG-related memos (like [RFC3640]). Also, this request for audio/asc doesn't register any parameters and doesn't ask for a repository to be created 💜
Media type name: audio
Subtype name: asc
Required parameters: None.
Optional parameters: None.
Encoding considerations: the native form of the data object is binary data, zero-padded to an octet boundary. Disk files that store this data object use the file extension ".acn" 💾
Restrictions on usage: this type is only defined for data object (stored file) transfer. The most common transports for it are HTTP and SMTP 🌐
Security considerations: see Section 9 of this memo 🔐
Interoperability considerations: None.
Published specification: the audio/asc data object is the AudioSpecificConfig binary data structure, which is normatively defined in [MPEGAUDIO] 📐
Applications that use this media type: MPEG 4 Audio servers and terminals that support audio/mpeg4-generic RTP streams for mode rtp-midi 🖥️
Additional information: None.
Person & email address to contact for further information: John Lazzaro <lazzaro@cs.berkeley.edu> 💌
Intended usage: COMMON.
Author: John Lazzaro <lazzaro@cs.berkeley.edu>
Change controller: IETF Audio/Video Transport Working Group delegated from the IESG. 👑
12. Changes from RFC 4695 ✨
okay bestie, this part is all about the errata 🧹 reviewers spotted mistakes in RFC 4695, and this doc fixes them. huge thanks to Alfred Hoenes, Roni Even, and Alexey Melnikov for catching them 🫶 as far as anyone knows, none of these errors caused interop issues, so here's the tea in order 💅
Section 3 of RFC 4695 had a bitfield format in Figure 3 that didn't match the normative text, and the normative text was the correct one 😭 Figure 3 wrongly claimed the Delta Time 0 field depends on the Z header bit. Figure 3 in this document fixes that 🎀 To our knowledge, this error didn't lead to wrong implementations of RFC 4695 💖
The rest of the mistakes live in Appendices C and D and are about session configuration parameters 🦋 The numbered list below, (1) through (11), walks through them in the order they show up in the document. As far as we know there are no interop problems here either, since implementation work has mostly happened in an application domain that doesn't use SDP for session management 🌸
(1) In Appendices C.1 and C.2.3 of RFC 4695, an ABNF rule related to System Chapter X is incorrectly defined as:
<parameter> = "__" <h-list> ["_" <h-list>] "__"The correct version of this rule is:
<parameter> = "__" <h-list> *( "_" <h-list> ) "__"(2) In Appendix C.6.3 of RFC 4695, the URIs allowed to be assigned to the url parameter weren't stated clearly 🙈 URIs assigned to url MUST specify either HTTP or HTTP over TLS transport protocols.
In Appendix C.7.1 and C.7.2 of RFC 4695, the transport interoperability requirements for the url parameter weren't clear either. For both C.7.1 and C.7.2, HTTP is REQUIRED and HTTP over TLS is OPTIONAL.
(3) In Appendix C.6.5, the filename extension ".acn" has been defined for use with AudioSpecificConfig. 🧚
(4) Both fmtp lines in both session description examples in Appendix C.7.2 of RFC 4695 had the same syntax error: a missing ";" at a line wrap after a cm_used assignment 💀
(5) In the session description examples in Appendices C.5, C.6, and C.7 of RFC 4695, this parameter assignment was wrong:
rinit="audio/asc";The correct parameter assignment is:
rinit=audio/asc;This error also shows up in the session descriptions in Figures 1 and 2 of the informative RFC 4696, so it's worth knowing about 👀 We aren't aware of any existing implementations that use the rinit parameter, so the incorrect examples in RFC 4695 and RFC 4696 should not cause interoperability problems.
(6) In Appendix D of RFC 4695, every use of "*ietf-extension" in the rules is an error and should be replaced with "ietf-extension". Same deal for "*extension", which should become "extension". This bug accidentally let the null token be assigned to the j_sec, j_update, render, smf_info, and subrender parameters 😭
(7) In Appendix D of RFC 4695, the definitions of <command-type> and <chapter-rules> allowed lowercase letters to sneak into these strings. Here are the correct definitions 💅
command-type = [A] [B] [C] [F] [G] [H] [J] [K] [M]
[N] [P] [Q] [T] [V] [W] [X] [Y] [Z]
chapter-list = [A] [B] [C] [D] [E] [F] [G] [H] [J] [K]
[M] [N] [P] [Q] [T] [V] [W] [X] [Y] [Z]
A = %x41
B = %x42
C = %x43
D = %x44
E = %x45
F = %x46
G = %x47
H = %x48
J = %x4A
K = %x4B
M = %x4D
N = %x4E
P = %x50
Q = %x51
T = %x54
V = %x56
W = %x57
X = %x58
Y = %x59
Z = %x5A(8) In Appendix D of RFC 4695, the definitions of <nonzero-four-octet>, <four-octet>, and <midi-chan> were incorrect. The correct definitions are below 🎀
nonzero-four-octet = (NZ-DIGIT 0*8(DIGIT))
/ (%x31-33 9(DIGIT))
/ ("4" %x30-31 8(DIGIT))
/ ("42" %x30-38 7(DIGIT))
/ ("429" %x30-33 6(DIGIT))
/ ("4294" %x30-38 5(DIGIT))
/ ("42949" %x30-35 4(DIGIT))
/ ("429496" %x30-36 3(DIGIT))
/ ("4294967" %x30-31 2(DIGIT))
/ ("42949672" %x30-38 (DIGIT))
/ ("429496729" %x30-34)
four-octet = "0" / nonzero-four-octet
midi-chan = DIGIT / ("1" %x30-35)
DIGIT = %x30-39
NZ-DIGIT = %x31-39(9) In Appendix D of RFC4695, the rule <hex-octet> was incorrect. The correct definition is below 🦋
hex-octet = %x30-37 U-HEXDIG
U-HEXDIG = DIGIT / A / B / C / D / E / F
; DIGIT as defined in (6) above
; A, B, C, D, E, F as defined in (5) above(10) In Appendix D, the <mime-subtype> rule now points to the <subtype-name> rule in [RFC4288] 🌈
(11) In Appendix D of RFC4695, the rules <base64-unit> and <base64-pad> were defined unclearly. Here are the rewritten versions ✨
base64-unit = 4(base64-char)
base64-pad = (2(base64-char) "==") / (3(base64-char) "=")Appendix A. The Recovery Journal Channel Chapters ✨
A.1. Recovery Journal Definitions
okay so this appendix sets up the vocab and coding idioms we use in the recovery journal bitfield descriptions 📚 that means Section 5 (the journal header structure), Appendices A.2 to A.9 (the channel journal chapters), and Appendices B.1 to B.5 (the system journal chapters) 💜
We assume the recovery journal lives in the journal section of an RTP packet with sequence number I (that's "packet I") 🌸 and that the Checkpoint Packet Seqnum field in the top-level recovery journal header points to an earlier packet with sequence number C. The one exception is the self-referential case where C = I. Unless stated otherwise, the algorithms use modulo 2^16 arithmetic for calculations on 16-bit sequence numbers, and modulo 2^32 arithmetic for calculations on 32-bit extended sequence numbers 🧮
Lots of bitfield coding idioms show up across the recovery journal system, and they always mean the same thing ✨ Most recovery journal elements start with an "S" (Single-packet loss) bit. S bits exist to help receivers parse through the recovery journal hierarchy quickly in the common case where a single packet is lost 🫶
As a rule, S bits MUST be set to 1. The one exception is when a recovery journal element in packet I encodes data about a command stored in the MIDI command section of packet I - 1. In that case, the S bit of the recovery journal element MUST be set to 0. If a recovery journal element has its S bit set to 0, then every higher-level recovery journal element that contains it MUST also have an S bit set to 0, including the top-level recovery journal header 🙈
Here are the other consistent coding idioms:
- R flag bit. R flag bits are reserved for future use. Senders MUST set R bits to 0. Receivers MUST ignore R bit values.
- LENGTH field. Every field named LENGTH (as opposed to LEN) codes the number of octets in the structure that contains it, including the header it resides in and all the hierarchical levels below it. If a structure contains a LENGTH field, a receiver MUST use the LENGTH field value to advance past the structure during parsing, rather than relying on knowledge about the structure's internal format 💅
Now for the normative terms used to describe recovery journal semantics 🔥
- Checkpoint history. The checkpoint history of a recovery journal is the concatenation of the MIDI command sections of packets C through I - 1. The final command in the MIDI command section for packet I - 1 is the most recent command, and the first command in the MIDI command section for packet C is the oldest. If command X is less recent than command Y, then X is considered to be "before Y". A checkpoint history with no commands is considered empty. The checkpoint history never contains the MIDI command section of packet I (the packet holding the recovery journal), so if C == I, the checkpoint history is empty by definition 🦋
- Session history. The session history of a recovery journal is the concatenation of MIDI command sections from the first packet of the session up to packet I - 1. Command recency and history emptiness work the same way as they do for the checkpoint history. The session history never contains the MIDI command section of packet I, so the session history of the first packet in the session is empty by definition 🌷
- Finished/unfinished commands. If all octets of a MIDI command appear in the session history, the command is finished. If some but not all of its octets appear in the session history, the command is unfinished. Unfinished commands happen when segments of a SysEx command show up across several RTP packets. For example, if a SysEx command is coded as 3 segments, with segment 1 in packet K, segment 2 in packet K + 1, and segment 3 in packet K + 2, then the session histories for packets K + 1 and K + 2 contain unfinished versions of the command. A session history contains a finished version of a cancelled SysEx command if the history contains the cancel sublist for that command 🎀
- Reset State commands. Reset State (RS) commands reset renderers to an initialized "powerup" condition. The RS commands are: System Reset (
0xFF), General MIDI System Enable (0xF0 0x7E 0xcc 0x09 0x01 0xF7), General MIDI 2 System Enable (0xF0 0x7E 0xcc 0x09 0x03 0xF7), General MIDI System Disable (0xF0 0x7E 0xcc 0x09 0x00 0xF7), Turn DLS On (0xF0 0x7E 0xcc 0x0A 0x01 0xF7), and Turn DLS Off (0xF0 0x7E 0xcc 0x0A 0x02 0xF7). Registrations of subrender parameter token values (Appendix C.6.2) and IETF Standards-Track documents MAY specify additional RS commands 🧚 - Active commands. Active commands are MIDI commands that do not appear before a Reset State command in the session history 🔥
- N-active commands. N-active commands are MIDI commands that do not appear before any of the following in the session history: MIDI Control Change numbers 123-127 (numbers with All Notes Off semantics), or 120 (All Sound Off), and any Reset State command 💜
- C-active commands. C-active commands are MIDI commands that do not appear before any of the following in the session history: MIDI Control Change number 121 (Reset All Controllers), and any Reset State command 🌸
- Oldest-first ordering rule. Several recovery journal chapters contain a list of elements, where each element is associated with a MIDI command that appears in the session history. In most cases, the chapter definition requires the list elements to be ordered according to the "oldest-first ordering rule". Here's the normative definition of that rule ✨
Elements associated with the most recent command in the session history coded in the list MUST appear at the end of the list.
Elements associated with the oldest command in the session history coded in the list MUST appear at the start of the list.
All other list elements MUST be lined up against these boundary elements, so the final list order really mirrors how recently each command showed up in the session history 💅✨
- Parameter system. A MIDI feature that gives you two sets of 16,384 parameters to stretch the 0-127 controller number space 🎀 The Registered Parameter Numbers (RPN) system and the Non-Registered Parameter Numbers (NRPN) system each provide 16,384 parameters.
- Parameter system transaction. RPN and NRPN values get changed by a series of Control Change commands that together form a parameter system transaction 🦋 A canonical transaction kicks off with two Control Change commands that set the parameter number (controller numbers 99 and 98 for NRPN parameters, controller numbers 101 and 100 for RPN parameters). Then comes an arbitrary number of Data Entry (controller numbers 6 and 38), Data Increment (controller number 96), and Data Decrement (controller number 97) Control Change commands to set the parameter value. It wraps up with a second pair of (99, 98) or (101, 100) Control Change commands that specify the null parameter (Most Significant Bit (MSB) value 0x7F, Least Significant Bit (LSB) value 0x7F).
There are a few variants of the canonical sequence, and they're honestly so fun 🙈 Most commonly, the terminal pair of (99, 98) or (101, 100) Control Change commands may specify a parameter other than the null parameter. In that case, the pair ends the first transaction and starts a second one, so the pair counts as part of both transactions. This variant is legal and recommended in [MIDI]. We refer to this variant as a "type 1 variant".
Less commonly, the MSB (99 or 101) or LSB (98 or 100) command of a (99, 98) or (101, 100) Control Change pair may be omitted.
If the MSB command is omitted, the transaction uses the MSB value of the most recent C-active Control Change command for controller number 99 or 101 that appears in the session history. We refer to this variant as a "type 2 variant".
If the LSB command is omitted, the LSB value 0x00 is assumed. We refer to this variant as a "type 3 variant". The type 2 and type 3 variants are defined as legal but are not recommended in [MIDI] 🥺
System Real-Time commands can show up at any point during a transaction, even between the octets of individual commands in that transaction 🌸 More generally, [MIDI] does not forbid unrelated MIDI commands from appearing during an open transaction. As a rule, these commands are considered to be "outside" the transaction and don't affect its status in any way. The exceptions are commands whose semantics act to terminate transactions: Reset State commands and Control Change (0xB) for controller number 121 (Reset All Controllers) [RP015].
- Initiated parameter system transaction. A canonical parameter system transaction whose initial (99, 98) or (101, 100) Control Change command pair appears in the session history counts as an initiated parameter system transaction 👑 This definition also holds for type 1 variants. For type 2 variants (dropped MSB), a transaction whose initial LSB Control Change command appears in the session history is an initiated transaction. For type 3 variants (dropped LSB), a transaction is considered initiated if at least one transaction command follows the initial MSB (99 or 101) Control Change command in the session history. Finishing a transaction does not nullify its "initiated" status, so it stays initiated, periodt ✨
- Session history reference counts. Several recovery journal chapters include a reference count field, which codes the total number of commands of a given type that appear in the session history 🧚 Examples include the Reset and Tune Request command logs (Appendix B.1, "System Chapter D") and the Active Sense command (Appendix B.2, "System Chapter V"). When a loss event is detected, reference count fields let a receiver figure out whether any instances of the command were lost, by comparing the journal reference count with its own reference count. So a reference count field makes sense even for command types where knowing the NUMBER of lost commands doesn't matter, and that's true for all of the example commands above 💜
The chapter definitions in Appendices A.2 to A.9 and B.1 to B.5 reflect the default recovery journal behavior 🍓 The ch_default, ch_never, and ch_anchor parameters modify these definitions, as described in Appendix C.2.3.
The chapter definitions specify whether data MUST be present in the journal. Senders MAY also include non-required data in the journal 🫶 This optional data MUST comply with the normative chapter definition. For example, if a chapter definition states that a field codes data from the most recent active command in the session history, the sender MUST NOT code inactive commands or older commands in that field.
Finally, a channel journal only encodes information about MIDI commands appearing on the MIDI channel the journal protects. That means all references to MIDI commands in Appendices A.2 to A.9 should be read as "MIDI commands appearing on this channel" 🌈
A.2. Chapter P: MIDI Program Change 🎀
A channel journal MUST contain Chapter P if an active Program Change (0xC) command appears in the checkpoint history. Figure A.2.1 shows the format for Chapter P.
0 1 2
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| PROGRAM |B| BANK-MSB |X| BANK-LSB |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.2.1 -- Chapter P FormatThe chapter has a fixed size of 24 bits 🧁 The PROGRAM field indicates the data value of the most recent active Program Change command in the session history. By default, the B, BANK-MSB, X, and BANK-LSB fields MUST be set to 0. Below, we define the exceptions to this default condition.
If an active Control Change (0xB) command for controller number 0 (Bank Select MSB) appears before the Program Change command in the session history, the B bit MUST be set to 1, and the BANK-MSB field MUST code the data value of the Control Change command ✨
If B is set to 1, the BANK-LSB field MUST code the data value of the most recent Control Change command for controller number 32 (Bank Select LSB) that preceded the Program Change command coded in the PROGRAM field and followed the Control Change command coded in the BANK-MSB field. If no such Control Change command exists, the BANK-LSB field MUST be set to 0.
If B is set to 1 and a Control Change command for controller number 121 (Reset All Controllers) appears in the MIDI stream between the Control Change command coded by the BANK-MSB field and the Program Change command coded by the PROGRAM field, the X bit MUST be set to 1 🔥
Note that [RP015] specifies that Reset All Controllers does not reset the values of controller numbers 0 (Bank Select MSB) and 32 (Bank Select LSB). Thus, theXbit does not affect how receivers will use theBANK-LSBandBANK-MSBvalues when recovering from a lost Program Change command. TheXbit serves to aid recovery in MIDI applications where controller numbers 0 and 32 are used in a non-standard way 💅
A.3. Chapter C: MIDI Control Change 🦋
Figure A.3.1 shows the format for Chapter C.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 8 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| LEN |S| NUMBER |A| VALUE/ALT |S| NUMBER |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|A| VALUE/ALT | .... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.3.1 -- Chapter C FormatThe chapter consists of a 1-octet header followed by a variable-length list of 2-octet controller logs 🌸 The list MUST contain at least one controller log. The 7-bit LEN field codes the number of controller logs in the list, minus one. We define the semantics of the controller log fields in Appendix A.3.2.
A channel journal MUST contain Chapter C if the rules defined in this appendix require that one or more controller logs appear in the list 💖
A.3.1. Log Inclusion Rules ✨
okay so a controller log is basically the recovery note for one Control Change command in the session history 🎀
In the default use of the payload format, list logs MUST encode information about the most recent active command in the session history for a controller number. Logs encoding earlier commands MUST NOT appear in the list. bestie, the old ones are not invited 💅
Also, as a rule, the list MUST contain a log for the most recent active command for a controller number that appears in the checkpoint history. There are some exceptions to this rule, and they're giving very specific vibes 🦋
- 14-bit controller pairs 💜 MIDI streams may transmit 14-bit controller values using paired Most Significant Byte (MSB, controller numbers 0-31, 99, 101) and Least Significant Byte (LSB, controller numbers 32-63, 98, 100) Control Change commands [MIDI].
If the most recent active Control Change command in the session history for a 14-bit controller pair uses the MSB number, Chapter C MAY omit the controller log for the most recent active Control Change command for the associated LSB number, because the command ordering makes that LSB value irrelevant. However, this exception MUST NOT be applied if the sender is not certain that the MIDI source uses 14-bit semantics for the controller number pair. Note that some MIDI sources ignore 14-bit controller semantics and use the LSB controller numbers as independent 7-bit controllers, so be careful, bestie 🙈
- Bank Select 🌸 If active Control Change commands for controller numbers 0 (Bank Select MSB) or 32 (Bank Select LSB) appear in the checkpoint history, and if the command instances are also coded in the BANK-MSB and BANK-LSB fields of the Chapter P (Appendix A.2), Chapter C MAY omit the controller logs for the commands.
- Mutually exclusive pairs 👑 Several controller number pairs are defined to be mutually exclusive. Controller numbers 124 (Omni Off) and 125 (Omni On) form one mutually exclusive pair, and controller numbers 126 (Mono) and 127 (Poly) form another.
If active Control Change commands for one or both members of a mutually exclusive pair appear in the checkpoint history, a log for the controller number of the most recent command for the pair in the checkpoint history MUST appear in the controller list. However, the list MAY omit the controller log for the most recent active command for the other number in the pair.
If active Control Change commands for one or both members of a mutually exclusive pair appear in the session history, and if a log for the controller number of the most recent command for the pair does not appear in the controller list, a log for the most recent command for the other number of the pair MUST NOT appear in the controller list. no double-dipping here 💀
- Reset All Controllers 🧚 If an active Control Change command for controller number 121 (Reset All Controllers) appears in the session history, the controller list MAY omit logs for Control Change commands that precede the Reset All Controllers command in the session history, under certain conditions.
Namely, a log MAY be omitted if the sender is certain that a command stream follows the Reset All Controllers semantics defined in [RP015], and if the log codes a controller number for which [RP015] specifies a reset value.
For example, [RP015] specifies that controller number 1 (Modulation Wheel) is reset to the value 0, so a controller log for Modulation Wheel MAY be omitted from the controller log list. In contrast, [RP015] specifies that controller number 7 (Channel Volume) is not reset, so a controller log for Channel Volume MUST NOT be omitted from the controller log list. it's giving "read the fine print" energy 📖
- MIDI Parameter System 🍓 Appendix A.3.4 defines exception rules for the MIDI Parameter System controller numbers 6, 38, and 96-101.
A.3.2. Controller Log Format
okay, the layout of a Chapter C controller log is so clean ✨ Figure A.3.2 shows the controller log structure of Chapter C:
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| NUMBER |A| VALUE/ALT |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure A.3.2 -- Chapter C Controller Log
The 7-bit NUMBER field identifies the controller number of the coded command. The 7-bit VALUE/ALT field codes recovery information for the command. The A bit sets the format of the VALUE/ALT field 🦋
A log encodes recovery information using one of the following tools: the value tool, the toggle tool, or the count tool.
A log uses the value tool if the A bit is set to 0. The value tool codes the 7-bit data value of a command in the VALUE/ALT field. It works best for controllers that code a continuous quantity, such as number 1 (Modulation Wheel) 🌈
The A bit is set to 1 to code the toggle or count tool. These tools work best for controllers that code discrete actions. Figure A.3.3 shows the controller log for these tools:
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| NUMBER |1|T| ALT |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure A.3.3 -- Controller Log for ALT Tools
A log uses the toggle tool if the T bit is set to 0. A log uses the count tool if the T bit is set to 1. Both methods use the 6-bit ALT field as an unsigned integer.
The toggle tool works best for controllers that act as on/off switches, such as 64 (Damper Pedal (Sustain)). These controllers code the "off" state with control values 0-63 and the "on" state with 64-127. 🔥
For the toggle tool, the ALT field codes the total number of toggles (off->on and on->off) due to Control Change commands in the session history, up to and including a toggle caused by the command coded by the log. The toggle count includes toggles caused by Control Change commands for controller number 121 (Reset All Controllers).
Toggle counting is performed modulo 64. The toggle count is reset at the start of a session and whenever a Reset State command (Appendix A.1) appears in the session history. When these reset events occur, the toggle count for a controller is set to 0 (for controllers whose default value is 0-63) or 1 (for controllers whose default value is 64-127).
The Damper Pedal (Sustain) controller shows off the toggle tool's glow-up over the value tool for switch controllers 👑 As often used in piano applications, the "on" state of the controller lets notes resonate, while the "off" state immediately damps notes to silence. The loss of the "off" command in an "on->off->on" sequence results in ringing notes that should have been damped silent 😭 The toggle tool lets receivers detect this lost "off" command, but the value tool does not.
The count tool is conceptually similar to the toggle tool. For the count tool, the ALT field codes the total number of Control Change commands in the session history, up to and including the command coded by the log. Command counting is performed modulo 64. The command count is set to 0 at the start of the session and is reset to 0 whenever a Reset State command (Appendix A.1) appears in the session history.
Because the count tool ignores the data value, it's a perfect match for controllers whose controller value is ignored, such as number 123 (All Notes Off) 💅 More generally, the count tool may be used to code a (modulo 64) identification number for a command.
A.3.3. Log List Coding Rules ✨
Okay bestie, this section is all about how the controller logs get organized inside the Chapter C log list 📋🎀
Each log describes one particular Control Change command from the session history 🦋 In most cases, a command SHOULD be coded by a single tool (and, thus, a single log). If a number is coded with a single tool and that tool is the count tool, then recovery Control Change commands generated by a receiver SHOULD use the default control value for the controller 💅
Sometimes a command MAY be coded by several tool types (and, thus, several logs, each using a different tool) 🌈 This can improve recovery performance for controllers with complex semantics, like controller number 84 (Portamento Control) or controller number 121 (Reset All Controllers) when it's used with a non-zero data octet (with the semantics described in [DLS2]) 😭
If a command is encoded by multiple tools, the logs MUST be placed in the list in the following order: count tool log (if any), followed by value tool log (if any), followed by toggle tool log (if any) 🔥
The Chapter C log list MUST obey the oldest-first ordering rule (defined in Appendix A.1). Note that this ordering keeps the info needed to recover 14-bit controller values, and it doesn't stop you from using MSB and LSB controller pairs as independent 7-bit controllers 🧚
In the default use of the payload format, every log that shows up in the list for a controller number encodes info about one Control Change command. That command is the most recent active one in the session history for that number 🍓
This coding scheme works really well for the standard uses of Control Change commands defined in [MIDI]. ngl though, not every MIDI app sticks to just those uses 🙈
For example, look at the common MIDI encoding of rotary encoders (the "infinite" rotation knobs) 🌸 The mixing console MIDI convention defined in [LCP] codes the position of rotary encoders as a series of Control Change commands. Each command encodes a relative change of knob position from the last update, expressed as a clockwise or counter-clockwise knob-turning angle 🎡
Since the knob position gets encoded step by step over a series of Control Change commands, the best recovery performance happens when the log list encodes all Control Change commands for encoder controller numbers that show up in the checkpoint history, not just the most recent one 👑
To support application areas that use Control Change commands this way, Chapter C can be configured to encode info about several Control Change commands for a controller number. We call this the "enhanced" encoding method, and we'll go over it below 💖
In Appendix C.2.3, we show how to configure a stream to use enhanced Chapter C encoding for specific controller numbers. In Section 5 of the main text, we show how the H bits in the recovery journal header (Figure 8) and in the channel journal header (Figure 9) indicate that enhanced Chapter C encoding is being used ✨
Here, we define how to encode a Chapter C log list that uses the enhanced encoding method 🦋
Senders that use the enhanced encoding method for a controller number MUST obey the rules below. These rules let a receiver figure out which logs in the list correspond to lost commands. Note that these rules override the exceptions listed in Appendix A.3.1 💅
- If N commands for a controller number are encoded in the list, the commands MUST be the N most recent commands for the controller number in the session history. For example, for N = 2, the sender MUST encode the most recent command and the second most recent command, not the most recent command and the third most recent command 🎀
- If a controller number uses enhanced encoding, the encoding of the least recent command for the controller number in the log list MUST include a count tool log. In addition, if commands are encoded for the controller number whose logs have S bits set to 0, the encoding of the least recent command with S = 0 logs MUST include a count tool log.
The count tool is OPTIONAL for the other commands for the controller number encoded in the list, because a receiver can efficiently deduce the count tool value for these commands for both single-packet and multi-packet loss events 🧠
- The use of the value and toggle tools MUST be identical for all commands for a controller number encoded in the list. For example, either a value tool log MUST appear for all commands for the controller number coded in the list or, alternatively, value tool logs for the controller number MUST NOT appear in the list. Likewise, either a toggle tool log MUST appear for all commands for the controller number coded in the list or, alternatively, toggle tool logs for the controller number MUST NOT appear in the list. 🌈
- If a command is encoded by multiple tools, the logs MUST be placed in the list in the following order: count tool log (if any), followed by value tool log (if any), followed by toggle tool log (if any).
These rules let a receiver recovering from a packet loss use the count tool log to match the commands in the list with its own history of the stream, as we describe below. Note that the text below describes a non-normative algorithm, and receivers are free to use any algorithm they want to match their history with the log list 🦋
In a typical implementation of the enhanced encoding method, a receiver computes and stores count, value, and toggle tool data field values for the most recent Control Change command it has received for a controller number 📦
After a loss event, a receiver parses the Chapter C list and processes the list logs for a controller number that uses enhanced encoding like this 🔍
The receiver compares the count tool ALT field for the least recent command for the controller number in the list against its stored count data for that controller number, to figure out whether recovery is needed for the command coded in the list. The value and toggle tool logs (if any) that directly follow the count tool log are associated with this least recent command 💜
To check more recent commands for the controller, the receiver detects any additional value and/or toggle tool logs for the controller number in the list, and infers count tool data for the command coded by those logs. This inferred data is then used to decide whether recovery is needed for the command coded by the value and/or toggle tool logs 🫶
This way, a receiver can execute only the lost commands, without executing a command twice 👑 While recovering from a single packet loss, a receiver may skip through S = 1 logs in the list, since the first S = 0 log for an enhanced controller number is always a count tool log 🌸
Note that the requirements in Appendix C.2.2.2 for protective sender and receiver actions during session startup for multicast operation are super important for enhanced encoding. Receivers need to initialize their count tool data structures with recovery journal data so they can match commands correctly after a loss event 🚨
Finally, in some applications of rotary encoders, a good user experience may be possible without enhanced encoding. These apps are distinguished by visual feedback of encoding position that is driven by the post-recovery rotary encoding stream, plus relatively low packet loss. In these domains, recovery performance may be acceptable for rotary encoders if the log list encodes only the most recent command and if both count and value logs appear for that command 🍰
A.3.4. The Parameter System 🎛️
Before you read this section, you might want to look over the Appendix A.1 definitions of "parameter system", "parameter system transaction", and "initiated parameter system transaction" 📖
Parameter system transactions update a MIDI Registered Parameter Numbers (RPN) or Non-Registered Parameter Numbers (NRPN) value. A parameter system transaction is a sequence of Control Change commands that may use the following controller numbers:
- Data Entry MSB (6)
- Data Entry LSB (38)
- Data Increment (96)
- Data Decrement (97)
- Non-Registered Parameter Number (NRPN) LSB (98)
- Non-Registered Parameter Number (NRPN) MSB (99)
- Registered Parameter Numbers (RPN) LSB (100)
- Registered Parameter Numbers (RPN) MSB (101)
Control Change commands that are part of a parameter system transaction MUST NOT be coded in Chapter C controller logs. Instead, these commands get coded in Chapter M, the MIDI Parameter chapter defined in Appendix A.4 🙅♀️
However, Control Change commands that use the listed controllers as general-purpose controllers (i.e., outside of a parameter system transaction) MUST NOT be coded in Chapter M 💫
Instead, those controllers get coded in Chapter C controller logs. The controller logs follow the coding rules stated in Appendix A.3.2 and A.3.3. The rules for coding paired LSB and MSB controllers, as defined in Appendix A.3.1, apply to the pairs (6, 38), (99, 98), and (101, 100) when they're coded in Chapter C 🌷
If active Control Change commands for controller numbers 6, 38, or 96-101 appear in the checkpoint history, and these commands are used as general-purpose controllers, the most recent general-purpose command instance for these controller numbers MUST appear as entries in the Chapter C controller list 🍓
MIDI syntax lets a source use controllers 6, 38, 96, and 97 as parameter-system controllers and as general-purpose controllers in the same stream 🤯 An RTP MIDI sender MUST deduce the role of each Control Change command for these controller numbers by noting where the command sits in the stream, and MUST use this info to code the command in Chapter C or Chapter M, as appropriate 🧩
Specifically, active Control Change commands for controllers 6, 38, 96, and 97 act in a general-purpose way when:
- no active Control Change commands that set an RPN or NRPN parameter number appear in the session history, or
- the most recent active Control Change commands in the session history that set an RPN or NRPN parameter number code the null parameter (MSB value 0x7F, LSB value 0x7F), or
- a Control Change command for controller number 121 (Reset All Controllers) appears more recently in the session history than all active Control Change commands that set an RPN or NRPN parameter number (see [RP015] for details) 🎉
Finally, a MIDI source that follows the recommendations of [MIDI] exclusively uses numbers 98-101 as parameter system controllers. Alternatively, a MIDI source may exclusively use 98-101 as general-purpose controllers, and then it loses the ability to perform parameter system transactions in a stream 😬
In the language of [MIDI], the general-purpose use of controllers 98-101 is a non-standard controller assignment. Since most real-world MIDI sources use the standard controller assignment for controller numbers 98-101, an RTP MIDI sender SHOULD assume these controllers act as parameter system controllers, unless it knows that a MIDI source uses controller numbers 98-101 in a general-purpose way 🎹💖
A.4. Chapter M: MIDI Parameter System ✨
okay bestie, before we dive in, go re-read the Appendix A.1 definitions for "C-active commands", "parameter system", "parameter system transaction", and "initiated parameter system transaction" 🧚. Chapter M is the one that protects RPN and NRPN parameter system transactions 💅.
Chapter M's top-level layout looks like this 🎀:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|P|E|U|W|Z| LENGTH |Q| PENDING | Log list ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.4.1 -- Top-Level Chapter M FormatChapter M starts with a 2-octet header 🌸. If the P header bit is set to 1, a 1-octet field tags along after the header, carrying the 7-bit PENDING value and its buddy, the Q bit 🫶.
The 10-bit LENGTH field tells you how big Chapter M is, and it follows the semantics from Appendix A.1 🔥.
Chapter M wraps up with a list of zero or more variable-length parameter logs 💜. Appendix A.4.2 explains the bitfield format of a parameter log, and Appendix A.4.1 explains when logs show up in the list 🦋.
A channel journal MUST contain Chapter M if the rules in Appendix A.4.1 say one or more parameter logs have to be in the list. No skipping it, no cap 🙈.
A channel journal also MUST contain Chapter M if the most recent C-active Control Change command involved in a parameter system transaction in the checkpoint history is:
- an RPN MSB (101) or NRPN MSB (99) controller, or
- an RPN LSB (100) or NRPN LSB (98) controller that completes the coding of the null parameter (MSB value 0x7F, LSB value 0x7F).
this rule is literally loss protection for partially sent parameter numbers and for the null parameter numbers 🛡️ iconic.
If the most recent C-active Control Change command involved in a parameter system transaction in the session history is for the RPN MSB or NRPN MSB controller, the P header bit MUST be set to 1, and the PENDING field (and its associated Q bit) MUST follow the Chapter M header. Otherwise, the P header bit MUST be set to 0, and the PENDING field and Q bit MUST NOT appear in Chapter M 💖.
If PENDING codes an NRPN MSB controller, the Q bit MUST be set to 1. If PENDING codes an RPN MSB controller, the Q bit MUST be set to 0 ✨.
The E header bit codes the current transaction state of the MIDI stream. If E = 1, an initiated transaction is in progress. Here are the rules for setting the E header bit 🧚:
- If no C-active parameter system transaction Control Change commands appear in the session history, the E bit MUST be set to 0.
- If the P header bit is set to 1, the E bit MUST be set to 0.
- If the most recent C-active parameter system transaction Control Change command in the session history is for the NRPN LSB or RPN LSB controller number and if this command acts to complete the coding of the null parameter (MSB value 0x7F, LSB value 0x7F), the E bit MUST be set to 0.
- Otherwise, an initiated transaction is in progress, and the E bit MUST be set to 1.
The U, W, and Z header bits code properties that are shared by all parameter logs in the list. If these properties are set, parameter logs can be coded with improved efficiency (we explain how in A.4.2) 🌈.
By default, the U, W, and Z bits MUST be set to 0. If all parameter logs in the list code RPN parameters, the U bit MAY be set to 1. If all parameter logs in the list code NRPN parameters, the W bit MAY be set to 1. If the parameter numbers of all RPN and NRPN logs in the list lie in the range 0-127 (and thus have an MSB value of 0), the Z bit MAY be set to 1 🍓.
Quick aside: the C-active stuff shows up in the paragraphs above because [RP015] says pending Parameter System transactions get closed by a Control Change command for controller number 121 (Reset All Controllers). That's the whole reason it matters 😭.
A.4.1. Log Inclusion Rules
Parameter logs code recovery information for one specific RPN or NRPN parameter 💅.
A parameter log MUST appear in the list if an active Control Change command that forms a part of an initiated transaction for the parameter appears in the checkpoint history 🔥.
There's an exception to that rule. It applies if the checkpoint history only has transaction Control Change commands for controller numbers 98-101 that act to terminate the transaction. In that case, a log for the parameter MAY be left out of the list 🙈.
A log MAY appear in the list if an active Control Change command that forms a part of an initiated transaction for the parameter appears in the session history. Otherwise, a log for the parameter MUST NOT appear in the list 🦋.
Multiple logs for the same RPN or NRPN parameter MUST NOT appear in the log list 🚫.
The parameter log list MUST obey the oldest-first ordering rule (defined in Appendix A.1), with the phrase "parameter transaction" replacing the word "command" in the rule definition 🍓.
Parameter logs tied to the RPN or NRPN null parameter (LSB = 0x7F, MSB = 0x7F) MUST NOT appear in the log list. Chapter M uses the E header bit (Figure A.4.1) and the log list ordering rules to code null parameter semantics 🧚.
Note that "active" semantics (rather than "C-active" semantics) show up in the paragraphs above because [RP015] says pending Parameter System transactions are NOT reset by a Control Change command for controller number 121 (Reset All Controllers). Ah, the plot twist. However, the rule that follows uses C-active semantics, because it's about protecting the transaction system itself, and [RP015] says Reset All Controllers acts to close a transaction in progress 🎀.
In most cases, parameter logs for RPN and NRPN parameters assigned to the ch_never parameter (Appendix C.2.3) MAY be left out of the list. The exception is if:
- the log codes the most recent initiated transaction in the session history, and
- a C-active command that forms a part of the transaction appears in the checkpoint history, and
- the E header bit for the top-level Chapter M header (Figure A.4.1) is set to 1.
In this case, a log for the parameter MUST appear in the list. This log tells receivers recovering from a loss that a transaction is in progress, so they can correctly interpret RPN or NRPN Control Change commands that follow the loss event 🫶.
A.4.2. Log Coding Rules
Figure A.4.2 shows the parameter log structure of Chapter M 👑.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 8 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| PNUM-LSB |Q| PNUM-MSB |J|K|L|M|N|T|V|R| Fields ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.4.2 -- Parameter Log FormatThe log starts with a header. Its default size (as shown in Figure A.4.2) is 3 octets. If the Q header bit is set to 0, the log encodes an RPN parameter. If Q = 1, the log encodes an NRPN parameter. The 7-bit PNUM-MSB and PNUM-LSB fields code the parameter number, and they reflect the Control Change command data values for controllers 99 and 98 (for NRPN parameters) or 101 and 100 (for RPN parameters) 🌸.
The J, K, L, M, and N header bits form a Table of Contents (TOC) for the log. They also signal the presence of fixed-size fields that follow the header. A header bit set to 1 codes the presence of a field in the log. The fields in the log follow the ordering of the header bits in the TOC. Appendices A.4.2.1 and A.4.2.2 define the fields for each TOC header bit 📖.
The T and V header bits code information about the parameter log but are NOT part of the TOC. A set T or V bit does not signal the presence of any parameter log field 🙈.
If the rules in Appendix A.4.1 say a log for a given parameter MUST appear in Chapter M, the log MUST code sufficient information to protect the parameter from the loss of active parameter transaction Control Change commands in the checkpoint history 🛡️.
This rule does NOT apply if the parameter coded by the log is assigned to the ch_never parameter (Appendix C.2.3). In this case, senders MAY choose to set the J, K, L, M, and N TOC bits to 0, coding a parameter log with no fields 🍃.
Logs that protect parameters assigned to ch_never are REQUIRED under certain conditions (see Appendix A.4.1). The point of the log is to tell receivers recovering from a loss that a transaction is in progress, so they can correctly interpret RPN or NRPN Control Change commands that follow the loss event ✨.
Parameter logs give you two tools for parameter protection: the value tool and the count tool. Depending on the semantics of the parameter, senders may use either tool, both tools, or neither tool to protect a given parameter 🧰.
The value tool codes information a receiver can use to figure out the current value of an RPN or NRPN parameter. If a parameter log uses the value tool, the V header bit MUST be set to 1, and the semantics defined in Appendix A.4.2.1 for setting the J, K, L, and M TOC bits MUST be followed. If a parameter log does NOT use the value tool, the V bit MUST be set to 0, and the J, K, L, and M TOC bits MUST also be set to 0 💜.
The count tool codes the number of transactions for an RPN or NRPN parameter. If a parameter log uses the count tool, the T header bit MUST be set to 1, and the semantics defined in Appendix A.4.2.2 for setting the N TOC bit MUST be followed. If a parameter log does NOT use the count tool, the T bit and the N TOC bit MUST be set to 0 🔢.
Note that V and T are set if the sender uses the value (V) or count (T) tool for the log on an ongoing basis. So V may be set even if J = K = L = M = 0, and T may be set even if N = 0 🌈.
In many cases, all parameters coded in the log list are one type (RPN parameters or NRPN parameters), and all parameter numbers lie in the range 0-127. As described in Appendix A.4, senders MAY signal this condition by setting the top-level Chapter M header bit Z to 1 (to code the restricted range) and by setting the U or W bit to 1 (to code the parameter type) 🎀.
If the top-level Chapter M header codes Z = 1 and either U = 1 or W = 1, all logs in the parameter log list MUST use a modified header format. This modification deletes bits 8-15 of the bitfield shown in Figure A.4.2 to yield a 2-octet header. The values of the deleted PNUM-MSB and Q fields can be inferred from the U, W, and Z bit values 🧠.
A.4.2.1. The Value Tool 🎀
Okay so the value tool is how we keep track of the value of an RPN or NRPN parameter 💖 It uses a few fields to do it.
The J TOC bit codes the presence of the octet shown in Figure A.4.3 in the field list. ✨
0
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|X| ENTRY-MSB |
+-+-+-+-+-+-+-+-+
Figure A.4.3 -- ENTRY-MSB FieldThe 7-bit ENTRY-MSB field stores the data value of the most recent active Control Change command for controller number 6 (Data Entry MSB) in the session history that shows up in a transaction for the log parameter. 🦋
The X bit MUST be set to 1 if the command coded by ENTRY-MSB comes before the most recent Control Change command for controller 121 (Reset All Controllers) in the session history. Otherwise, the X bit MUST be set to 0. 🫶
A parameter log that uses the value tool MUST include the ENTRY-MSB field if an active Control Change command for controller number 6 appears in the checkpoint history. 💅
Note that [RP015] specifies that Control Change commands for controller 121 (Reset All Controllers) do not reset RPN and NRPN values. So the X bit wouldn't do any recovery work for MIDI systems that comply with [RP015]. Lowkey iconic that it still exists for everyone else 🙈
However, certain renderers (such as DLS 2 [DLS2]) specify that certain RPN values are reset for some uses of Reset All Controllers. The X bit (and other bitfield features of this nature in this appendix) plays a role in recovery for renderers of this type. 👑
The K TOC bit codes the presence of the octet shown in Figure A.4.4 in the field list. 🌸
0
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|X| ENTRY-LSB |
+-+-+-+-+-+-+-+-+
Figure A.4.4 -- ENTRY-LSB FieldThe 7-bit ENTRY-LSB field stores the data value of the most recent active Control Change command for controller number 38 (Data Entry LSB) in the session history that shows up in a transaction for the log parameter. 🧚
The X bit MUST be set to 1 if the command coded by ENTRY-LSB comes before the most recent Control Change command for controller 121 (Reset All Controllers) in the session history. Otherwise, the X bit MUST be set to 0. 💜
As a rule, a parameter log that uses the value tool MUST include the ENTRY-LSB field if an active Control Change command for controller number 38 appears in the checkpoint history. However, the ENTRY-LSB field MUST NOT appear in a parameter log if the Control Change command associated with the ENTRY-LSB comes before a Control Change command for controller number 6 (Data Entry MSB) that shows up in a transaction for the log parameter in the session history. It's giving "the newer one wins" energy 🍓
The L TOC bit codes the presence of the octets shown in Figure A.4.5 in the field list. ✨
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|G|X| A-BUTTON |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.4.5 -- A-BUTTON FieldThe 14-bit A-BUTTON field codes a count of the number of active Control Change commands for controller numbers 96 and 97 (Data Increment and Data Decrement) in the session history that show up in a transaction for the log parameter. 🔥
The M TOC bit codes the presence of the octets shown in Figure A.4.6 in the field list. 🦋
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|G|R| C-BUTTON |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.4.6 -- C-BUTTON FieldThe 14-bit C-BUTTON field has semantics identical to A-BUTTON, except that Data Increment and Data Decrement Control Change commands that come before the most recent Control Change command for controller 121 (Reset All Controllers) in the session history are not counted. 🌈
For both A-BUTTON and C-BUTTON, Data Increment and Data Decrement Control Change commands are not counted if they come before Control Change commands for controller numbers 6 (Data Entry MSB) or 38 (Data Entry LSB) that show up in a transaction for the log parameter in the session history. 💅
The A-BUTTON and C-BUTTON fields are read as unsigned integers, and the G bit attached to each field codes the sign of the integer (G = 0 for positive or zero, G = 1 for negative). ✨
To compute and code the count value, start the count value at 0, add 1 for each qualifying Data Increment command, and subtract 1 for each qualifying Data Decrement command. After each addition or subtraction, limit the count magnitude to 16383. The G bit codes the sign of the count, and the A-BUTTON or C-BUTTON field codes the count magnitude. 🧮💖
For the A-BUTTON field, if the most recent qualified Data Increment or Data Decrement command comes before the most recent Control Change command for controller 121 (Reset All Controllers) in the session history, the X bit associated with the A-BUTTON field MUST be set to 1. Otherwise, the X bit MUST be set to 0. 🫶
A parameter log that uses the value tool MUST include the A-BUTTON and C-BUTTON fields if an active Control Change command for controller numbers 96 or 97 appears in the checkpoint history. However, to improve coding efficiency, this rule has several exceptions:
- If the log includes the A-BUTTON field, and if the X bit of the A-BUTTON field is set to 1, the C-BUTTON field (and its associated R and G bits) MAY be omitted from the log.
- If the log includes the A-BUTTON field, and if the A-BUTTON and C-BUTTON fields (and their associated G bits) code identical values, the C-BUTTON field (and its associated R and G bits) MAY be omitted from the log.
Those exceptions are basically the "don't pay for what you don't need" clause, and I respect it 👑
A.4.2.2. The Count Tool 🌸
The count tool tracks the number of transactions for an RPN or NRPN parameter. The N TOC bit codes the presence of the octet shown in Figure A.4.7 in the field list. 🧚
0
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|X| COUNT |
+-+-+-+-+-+-+-+-+
Figure A.4.7 -- COUNT FieldThe 7-bit COUNT codes the number of initiated transactions for the log parameter that appear in the session history. Initiated transactions are counted if they contain one or more active Control Change commands, including commands for controllers 98-101 that initiate the parameter transaction. 💜
If the most recent counted transaction comes before the most recent Control Change command for controller 121 (Reset All Controllers) in the session history, the X bit associated with the COUNT field MUST be set to 1. Otherwise, the X bit MUST be set to 0. 🍓
Transaction counting is performed modulo 128. The transaction count is set to 0 at the start of a session and is reset to 0 whenever a Reset State command (Appendix A.1) appears in the session history. ✨
A parameter log that uses the count tool MUST include the COUNT field if an active command that increments the transaction count (modulo 128) appears in the checkpoint history. 💅
A.5. Chapter W: MIDI Pitch Wheel 🦋
A channel journal MUST contain Chapter W if a C-active MIDI Pitch Wheel (0xE) command appears in the checkpoint history. Figure A.5.1 shows the format for Chapter W. 🌈
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| FIRST |R| SECOND |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.5.1 -- Chapter W FormatThe chapter has a fixed size of 16 bits. The FIRST and SECOND fields are the 7-bit values of the first and second data octets of the most recent active Pitch Wheel command in the session history. 🎀
Note that Chapter W encodes C-active commands, so it does not encode active commands that are not C-active. (See the second-to-last paragraph of Appendix A.1 for an explanation of chapter inclusion text in this regard.) 🙈
Chapter W does not encode "active but not C-active" commands because [RP015] declares that Control Change commands for controller number 121 (Reset All Controllers) act to reset the Pitch Wheel value to 0. If Chapter W encoded "active but not C-active" commands, a repair operation following a Reset All Controllers command could incorrectly repair the stream with a stale Pitch Wheel value. Stale data is a no-go, bestie 😭
A.6. Chapter N: MIDI NoteOff and NoteOn 🎶
In this appendix, we consider NoteOn commands with zero velocity to be NoteOff commands. Readers may wish to review the Appendix A.1 definition of "N-active commands" before reading this appendix. 💖
Chapter N completely protects note commands in streams that alternate between NoteOn and NoteOff commands for a particular note number. However, in rare applications, multiple overlapping NoteOn commands may appear for a note number. Chapter E, described in Appendix A.7, augments Chapter N to completely protect these streams. ✨
A channel journal MUST contain Chapter N if an N-active MIDI NoteOn (0x9) or NoteOff (0x8) command appears in the checkpoint history. Figure A.6.1 shows the format for Chapter N. 🫶
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 8 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|B| LEN | LOW | HIGH |S| NOTENUM |Y| VELOCITY |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| NOTENUM |Y| VELOCITY | .... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OFFBITS | OFFBITS | .... | OFFBITS |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.6.1 -- Chapter N FormatChapter N is made of a 2-octet header followed by at least one of the following data structures:
- A list of note logs to code NoteOn commands.
- A NoteOff bitfield structure to code NoteOff commands.
We define the header bitfield semantics in Appendix A.6.1. We define the note log semantics and the NoteOff bitfield semantics in Appendix A.6.2. 🌸
If one or more N-active NoteOn or NoteOff commands in the checkpoint history reference a note number, the note number MUST be coded in either the note log list or the NoteOff bitfield structure. 🧚
The note log list MUST contain an entry for all note numbers whose most recent checkpoint history appearance is in an N-active NoteOn command. The NoteOff bitfield structure MUST contain a set bit for all note numbers whose most recent checkpoint history appearance is in an N-active NoteOff command. 💜
A note number MUST NOT be coded in both structures. 🙅♀️
All note logs and NoteOff bitfield set bits MUST code the most recent N-active NoteOn or NoteOff reference to a note number in the session history. 🍓
The note log list MUST obey the oldest-first ordering rule (defined in Appendix A.1). 👑
A.6.1. Header Structure 🎀
okay bestie, the header for Chapter N is where it all starts ✨ it tells you how big the note list and the bitfield structures are. peep Figure A.6.2:
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|B| LEN | LOW | HIGH |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.6.2 -- Chapter N Headerthe LEN field is a 7-bit integer and it's the count of 2-octet note logs in the note list 🎶 zero is totally valid for LEN and just means the note list is empty 🫶
the 4-bit LOW and HIGH fields are the ones counting the OFFBITS octets that come after the note log list 💅 both are unsigned integers. when LOW <= HIGH, you get (HIGH - LOW + 1) OFFBITS octets in the chapter. the pairs (LOW = 15, HIGH = 0) and (LOW = 15, HIGH = 1) both mean an empty NoteOff bitfield structure, so no OFFBITS octets at all 🙈 and any other pair where LOW > HIGH MUST NOT appear in the header. not allowed, periodt 💀
the B bit is basically S-bit energy (see Appendix A.1) but for the NoteOff bitfield structure 🦋 by default the B bit MUST be set to 1. BUT if the MIDI command section of the previous packet (packet I - 1, with I as defined in Appendix A.1) includes a NoteOff command for the channel, then the B bit MUST be set to 0 instead. and when the B bit is 0, the higher-level recovery journal elements that contain Chapter N MUST have their S bits set to 0, top-level journal header included 😭
the LEN = 127 case is the tricky one, so read this part slowly 👀
the LEN value of 127 codes a note list length of 127 or 128 note logs, depending on LOW and HIGH. if LEN = 127, LOW = 15, and HIGH = 0, the note list holds 128 note logs and the NoteOff bitfield structure is empty ✨ for other LOW and HIGH values, LEN = 127 means the note list has 127 note logs. in that case the chapter has (HIGH - LOW + 1) NoteOff OFFBITS octets if LOW <= HIGH, and no OFFBITS octets if LOW = 15 and HIGH = 1.
A.6.2. Note Structures 🌸
Figure A.6.3 shows the 2-octet note log structure, and it's giving clean layout:
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| NOTENUM |Y| VELOCITY |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.6.3 -- Chapter N Note Logthe 7-bit NOTENUM field codes the note number for the log. a note number MUST NOT be represented by multiple note logs in the note list. one note, one log, no drama 🎀
the 7-bit VELOCITY field codes the velocity value for the most recent N-active NoteOn command for that note number in the session history. multiple overlapping NoteOns for a given note number may be coded using Chapter E, as discussed in Appendix A.7 🧚
VELOCITY is never zero. NoteOn commands with zero velocity get coded as NoteOff commands in the NoteOff bitfield structure, which is a whole mood 😌
the note log doesn't code the execution time of the NoteOn command, BUT the Y bit gives you a hint from the sender about it 🫣 the Y bit codes a recommendation to play (Y = 1) or skip (Y = 0) the NoteOn command recovered from the note log. see Section 4.2 of [RFC4696] for non-normative guidance on using the Y bit.
Figure A.6.1 shows the NoteOff bitfield structure as the list of OFFBITS octets at the end of the chapter. each NoteOff OFFBITS octet codes NoteOff info for eight consecutive MIDI note numbers, with the most significant bit representing the lowest note number 🌈 the most significant bit of the first OFFBITS octet codes note number 8*LOW, and the most significant bit of the last OFFBITS octet codes note number 8*HIGH.
a set bit codes a NoteOff command for that note number 💜 in the most efficient coding for the NoteOff bitfield structure, the first and last octets each contain at least one set bit. also, Chapter N does not code NoteOff velocity data, so don't go looking for it there 🙅♀️
ok real talk about the pedal 🎹 in the general case, the recovery journal does NOT code the relative placement of a NoteOff command and a Change Control command for controller 64 (Damper Pedal (Sustain)). in lots of cases, a receiver handling a loss event can figure out that placement from the history of the stream, and then tell if a NoteOff note is sustained by the pedal. if that isn't possible, receivers SHOULD err on the side of silencing pedal sustains, because erroneously sustained notes may produce unpleasant (albeit transient) artifacts 😭
A.7. Chapter E: MIDI Note Command Extras 🧚
readers may wish to review the Appendix A.1 definition of "N-active commands" before reading this appendix 💅 in this appendix, a NoteOn command with a velocity of 0 is considered to be a NoteOff command with a release velocity value of 64.
Chapter E encodes recovery info about MIDI NoteOn (0x9) and NoteOff (0x8) command features that rarely show up in MIDI streams 🌸 receivers use Chapter E to reduce transient artifacts for streams where several NoteOn commands appear for a note number without an intervening NoteOff. receivers also use it for streams that use NoteOff release velocity. Chapter E supplements the note info coded in Chapter N (Appendix A.6).
Figure A.7.1 shows the format for Chapter E:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 8 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| LEN |S| NOTENUM |V| COUNT/VEL |S| NOTENUM |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V| COUNT/VEL | .... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.7.1 -- Chapter E Formatthe chapter is a 1-octet header followed by a variable-length list of 2-octet note logs 🎀 Appendix A.7.1 defines the bitfield format for a note log.
the log list MUST contain at least one note log 🙌 the 7-bit LEN header field codes the number of note logs in the list, minus one. a channel journal MUST contain Chapter E if the rules in this appendix require that one or more note logs appear in the list. and the note log list MUST obey the oldest-first ordering rule (defined in Appendix A.1). no shuffling allowed 🕰️
A.7.1. Note Log Format 💖
Figure A.7.2 reproduces the note log structure of Chapter E:
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| NOTENUM |V| COUNT/VEL |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.7.2 -- Chapter E Note Loga note log codes info about the MIDI note number in the 7-bit NOTENUM field 🍓 what the info actually is depends on the V flag bit.
if the V bit is set to 1, the COUNT/VEL field codes the release velocity value for the most recent N-active NoteOff command for that note number in the session history.
if the V bit is set to 0, the COUNT/VEL field codes a reference count of the NoteOn and NoteOff commands for that note number in the session history 🧮
the reference count starts at 0 at the start of the session. NoteOn commands increment the count by 1. NoteOff commands decrement it by 1. however, a decrement that would make the count negative is not performed 🙅♀️
if the reference count is in the range 0-126, the 7-bit COUNT/VEL field codes an unsigned integer version of the count. if the count is greater than or equal to 127, COUNT/VEL is set to 127 ✨
by default, the count gets reset to 0 whenever a Reset State command (Appendix A.1) shows up in the session history, and also whenever MIDI Control Change commands for controller numbers 123-127 (numbers with All Notes Off semantics) or 120 (All Sound Off) show up in the session history 🔄
A.7.2. Log Inclusion Rules 🦋
if the most recent N-active NoteOn or NoteOff command for a note number in the checkpoint history is a NoteOff command with a release velocity value other than 64, a note log whose V bit is set to 1 MUST appear in Chapter E for that note number 💅
if the most recent N-active NoteOn or NoteOff command for a note number in the checkpoint history is a NoteOff command, AND the reference count for that note number is greater than 0, a note log whose V bit is set to 0 MUST appear in Chapter E for that note number 🫶
if the most recent N-active NoteOn or NoteOff command for a note number in the checkpoint history is a NoteOn command, AND the reference count for that note number is greater than 1, a note log whose V bit is set to 0 MUST appear in Chapter E for that note number 🎶
at most, two note logs MAY appear in Chapter E for a note number: one log whose V bit is set to 0 and one log whose V bit is set to 1 🌸
Chapter E codes a maximum of 128 note logs. if the log inclusion rules yield more than 128 REQUIRED logs, note logs whose V bit is set to 1 MUST be dropped from Chapter E to reach the 128-log limit. note logs whose V bit is set to 0 MUST NOT be dropped. those are the ones we protect 🛡️
most MIDI streams don't use NoteOn and NoteOff commands in ways that trigger the log inclusion rules 😌 for these streams, Chapter E would never be REQUIRED to appear in a channel journal. easy life.
the ch_never parameter (Appendix C.2.3) may be used to configure the log inclusion rules for Chapter E 🎛️
A.8. Chapter T: MIDI Channel Aftertouch 💜
a channel journal MUST contain Chapter T if an N-active and C-active MIDI Channel Aftertouch (0xD) command appears in the checkpoint history 🌈 Figure A.8.1 shows the format for Chapter T:
0
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|S| PRESSURE |
+-+-+-+-+-+-+-+-+
Figure A.8.1 -- Chapter T Formatthe chapter has a fixed size of 8 bits. the 7-bit PRESSURE field holds the pressure value of the most recent N-active and C-active Channel Aftertouch command in the session history 🧁
Chapter T only encodes commands that are C-active and N-active, and we're being picky on purpose 🧐 we define a C-active restriction because [RP015] declares that a Control Change command for controller 121 (Reset All Controllers) acts to reset the channel pressure to 0 (see the discussion at the end of Appendix A.5 for a more complete rationale).
we define an N-active restriction on the assumption that aftertouch commands are linked to note activity. so Channel Aftertouch commands that are not N-active are stale and should not be used to repair a stream 🫶 this is the kind of detail that keeps the whole stream from getting messy, iykyk 🦋
A.9. Chapter A: MIDI Poly Aftertouch ✨
okay bestie, a channel journal MUST contain Chapter A if a C-active Poly Aftertouch (0xA) command shows up in the checkpoint history 💅 Figure A.9.1 shows what the format looks like:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 8 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| LEN |S| NOTENUM |X| PRESSURE |S| NOTENUM |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|X| PRESSURE | .... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.9.1 -- Chapter A FormatSo the chapter is a 1-octet header, then a variable-length list of 2-octet note logs 🎀 A note log MUST appear for a note number if a C-active Poly Aftertouch command for that note number shows up in the checkpoint history. A note number MUST NOT be represented by multiple note logs in the note list, and the note log list MUST obey the oldest-first ordering rule (defined in Appendix A.1). Very organised, very iconic 🧚
The 7-bit LEN field tells you how many note logs are in the list, minus one 💖 And here's the structure of one note log, Figure A.9.2:
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S| NOTENUM |X| PRESSURE |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure A.9.2 -- Chapter A Note LogThe 7-bit PRESSURE field holds the pressure value from the most recent C-active Poly Aftertouch command in the session history for the MIDI note number sitting in the 7-bit NOTENUM field 🦋
Now the X bit, which is giving drama. As a rule, the X bit MUST be set to 0. BUT the X bit MUST be set to 1 if the command coded by the log appears before one of these commands in the session history: MIDI Control Change numbers 123-127 (the ones with All Notes Off semantics) or 120 (All Sound Off) 🙈
We set these C-active restrictions for Chapter A because [RP015] says that a Control Change command for controller 121 (Reset All Controllers) resets the polyphonic pressure to 0. If you want the full reasoning, check the discussion at the end of Appendix A.5 🌸
Appendix B. The Recovery Journal System Chapters 👑
B.1. System Chapter D: Simple System Commands 🌈
The system journal MUST contain Chapter D if an active MIDI Reset (0xFF), MIDI Tune Request (0xF6), MIDI Song Select (0xF3), undefined MIDI System Common (0xF4 and 0xF5), or undefined MIDI System Real-Time (0xF9 and 0xFD) command appears in the checkpoint history. Iykyk, these are the system commands that really matter ✨
Figure B.1.1 is the variable-length format for Chapter D:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|B|G|H|J|K|Y|Z| Command logs ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure B.1.1 -- System Chapter D FormatThe chapter is a 1-octet header followed by one or more command logs. The header flag bits tell you which command logs are there: Reset (B = 1), Tune Request (G = 1), Song Select (H = 1), undefined System Common 0xF4 (J = 1), undefined System Common 0xF5 (K = 1), undefined System Real-Time 0xF9 (Y = 1), or undefined System Real-Time 0xFD (Z = 1) 🎀
The command logs show up in a list after the header, in the same order the flag bits appear in the header. Very ordered, very slay 💅
Figure B.1.2 is the 1-octet command log format for Reset and Tune Request:
0
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|S| COUNT |
+-+-+-+-+-+-+-+-+
Figure B.1.2 -- Command Log for Reset and Tune RequestChapter D MUST contain the Reset command log if an active Reset command appears in the checkpoint history. The 7-bit COUNT field codes the total number of Reset commands (modulo 128) in the session history 🍓
Chapter D MUST contain the Tune Request command log if an active Tune Request command appears in the checkpoint history. The 7-bit COUNT field codes the total number of Tune Request commands (modulo 128) in the session history 🦋
For these commands, the COUNT field acts as a reference count. For more on that, look up the definition of "session history reference counts" in Appendix A.1 🌸
Figure B.1.3 is the 1-octet command log format for Song Select:
0
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|S| VALUE |
+-+-+-+-+-+-+-+-+
Figure B.1.3 -- Song Select Command Log FormatChapter D MUST contain the Song Select command log if an active Song Select command appears in the checkpoint history. The 7-bit VALUE field codes the song number of the most recent active Song Select command in the session history 🎵
B.1.1. Undefined System Commands 🙈
Here we define the Chapter D command logs for the undefined system commands. [MIDI] reserves the undefined system commands 0xF4, 0xF5, 0xF9, and 0xFD for future use. At the time of writing, any MIDI command stream that uses these commands is non-compliant with [MIDI]. However, future versions of [MIDI] may define these commands, and a few products do use them in a non-compliant manner. Chaotic, but that's the vibe 💀
Figure B.1.4 shows the variable-length command log format for the undefined System Common commands (0xF4 and 0xF5):
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|C|V|L|DSZ| LENGTH | COUNT | VALUE ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| LEGAL ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure B.1.4 -- Undefined System Common Command Log FormatThe command log codes a single command type (0xF4 or 0xF5, not both). Chapter D MUST contain a command log if an active 0xF4 command appears in the checkpoint history, and MUST contain an independent command log if an active 0xF5 command appears in the checkpoint history 🫶
A Chapter D Undefined System Common command log is a two-octet header followed by a variable number of data fields. The header flag bits tell you if the COUNT field (C = 1), the VALUE field (V = 1), or the LEGAL field (L = 1) is there. The 10-bit LENGTH field codes the size of the command log and follows the semantics in Appendix A.1 ✨
The 2-bit DSZ field codes how many data octets the most recent command instance in the session history had. If DSZ = 0-2, the command has 0-2 data octets. If DSZ = 3, the command has 3 or more command data octets 🍓
Now the default rules for COUNT, VALUE, and LEGAL. The session configuration tools defined in Appendix C.2.3 can override this behaviour if needed 🔥
By default, if the DSZ field is set to 0, the command log MUST include the COUNT field. The 8-bit COUNT field codes the total number of commands of the type coded by the log (0xF4 or 0xF5) in the session history, modulo 256 💜
By default, if the DSZ field is set to 1-3, the command log MUST include the VALUE field. The variable-length VALUE field is a verbatim copy of the data octets from the most recent use of the command type coded by the log (0xF4 or 0xF5) in the session history. The most significant bit of the final data octet MUST be set to 1, and the most significant bit of all other data octets MUST be set to 0 🌈
The LEGAL field is reserved for future use. If an update to [MIDI] defines the 0xF4 or 0xF5 command, an IETF Standards-Track document may define the LEGAL field. Until that document shows up, senders MUST NOT use the LEGAL field, and receivers MUST use the LENGTH field to skip over it. The IETF would define LEGAL only if the semantics of the new 0xF4 or 0xF5 command couldn't be protected from packet loss using the COUNT and VALUE fields 👑
Figure B.1.5 shows the variable-length command log format for the undefined System Real-Time commands (0xF9 and 0xFD):
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|C|L| LENGTH | COUNT | LEGAL ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure B.1.5 -- Undefined System Real-Time Command Log FormatThe command log codes a single command type (0xF9 or 0xFD, not both). Chapter D MUST contain a command log if an active 0xF9 command appears in the checkpoint history, and MUST contain an independent command log if an active 0xFD command appears in the checkpoint history 🦋
A Chapter D Undefined System Real-Time command log is a one-octet header followed by a variable number of data fields. The header flag bits tell you if the COUNT field (C = 1) or the LEGAL field (L = 1) is there. The 5-bit LENGTH field codes the size of the command log and follows the semantics in Appendix A.1 💖
Default rules for COUNT and LEGAL are next. The session configuration tools defined in Appendix C.2.3 can override this behaviour too 🧚
The 8-bit COUNT field codes the total number of commands of the type coded by the log in the session history, modulo 256. By default, the COUNT field MUST be present in the command log 🌸
The LEGAL field is reserved for future use. If an update to [MIDI] defines the 0xF9 or 0xFD command, an IETF Standards-Track document may define the LEGAL field to protect the command. Until that document appears, senders MUST NOT use the LEGAL field, and receivers MUST use the LENGTH field to skip over it. The IETF would define LEGAL only if the semantics of the new 0xF9 or 0xFD command couldn't be protected from packet loss using the COUNT field ⭐
Heads up, bestie: some non-standard uses of the undefined System Real-Time commands act to implement non-compliant variants of the MIDI sequencer system. Appendix B.3.1 describes resiliency tools for the MIDI sequencer system that give some protection in this case 🫶
B.2. System Chapter V: Active Sense Command ✨
okay so the system journal MUST contain Chapter V if an active MIDI Active Sense (0xFE) command shows up in the checkpoint history 👀 Figure B.2.1 shows what Chapter V looks like 💅
0
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|S| COUNT |
+-+-+-+-+-+-+-+-+
Figure B.2.1 -- System Chapter V Formatthe 7-bit COUNT field tracks the total number of Active Sense commands (modulo 128) in the session history 🔢 it works like a reference count, and if you want the full deal on "session history reference counts," check Appendix A.1 🧚
B.3. System Chapter Q: Sequencer State Commands 🎀
this appendix is all about Chapter Q, the system chapter for the MIDI sequencer commands 🎶
the system journal MUST contain Chapter Q if an active MIDI Song Position Pointer (0xF2), MIDI Clock (0xF8), MIDI Start (0xFA), MIDI Continue (0xFB), or MIDI Stop (0xFC) command appears in the checkpoint history, and also if the rules in this appendix need a change to the Chapter Q bitfield contents because of that command showing up 🔥 Figure B.3.1 shows the variable-length format for Chapter Q.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|N|D|C|T| TOP | CLOCK | TIMETOOLS ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure B.3.1 -- System Chapter Q FormatChapter Q is a 1-octet header followed by a few optional fields, in the order shown in Figure B.3.1 ✨ The header flag bits tell you whether the 16-bit CLOCK field is there (C = 1) and whether the 24-bit TIMETOOLS field is there (T = 1). The 3-bit TOP header field is read as an unsigned integer, same as CLOCK and TIMETOOLS. We'll get into TIMETOOLS in Appendix B.3.1 🦋
Chapter Q stores the most recent state of the sequencer system 💜 receivers use it to resync the sequencer after a packet loss episode. The chapter fields encode the on/off state of the sequencer, the current position in the song, and the downbeat. It's giving full status update, bestie 💅
The N header bit encodes the relative order of the Start, Stop, and Continue commands in the session history 🌸 If an active Start or Continue command showed up most recently, N MUST be set to 1. If an active Stop showed up most recently, or if no active Start, Stop, or Continue commands are in the session history, N MUST be set to 0.
The C header flag, the TOP header field, and the CLOCK field work together to code the current position in the sequence 🧚
- If
C= 1, the 3-bitTOPheader field and the 16-bitCLOCKfield combine into the 19-bit unsigned quantity65536*TOP + CLOCK. This value encodes the song position in units of MIDI Clocks (24 clocks per quarter note), modulo 524288. Quick aside: the biggest song position the Song Position Pointer command can code is 98303 clocks (that fits in 17 bits), and MIDI-coded songs are usually built so they don't run longer than that 🙈 The 19-bit size can still be handy for real-time stuff, like a drum machine's MIDI output sending clock commands for a long time. - If
C= 0, the song position is the start of the song. TheC= 0 position is identical to the one coded byC= 1,TOP= 0, andCLOCK= 0, for the case where the song position is less than 524288 MIDI clocks. In certain situations (defined later in this section), normative text may require theC= 0 or theC= 1,TOP= 0,CLOCK= 0 encoding of the start of the song 🎀
The C, TOP, and CLOCK fields MUST be set to code the current song position, for both N = 0 and N = 1 conditions. If C = 0, the TOP field MUST be set to 0. See [MIDI] for a precise definition of a song position 📖
The D header bit encodes info about the downbeat, and it qualifies the song position coded by the C, TOP, and CLOCK fields 🥁
If the D bit is set to 1, the song position is the most recent position in the sequence that has already played ✨ If D = 1, the next Clock command (if N = 1) or the next (Continue, Clock) pair (if N = 0) increments the song position by one clock and plays the updated position.
If the D bit is set to 0, the song position is a position in the sequence that hasn't played yet 🌈 If D = 0, the next Clock command (if N = 1) or the next (Continue, Clock) pair (if N = 0) plays the point in the song coded by the song position. The song position doesn't get incremented here.
An example of a stream using D = 0 coding is one whose most recent sequence command is a Start or Song Position Pointer command (both N = 1 conditions). It's also possible to build examples where D = 0 and N = 0 💀 A Start command immediately followed by a Stop command is coded in Chapter Q by setting C = 0, D = 0, N = 0, TOP = 0.
If N = 1 (coding Start or Continue), D = 0 (coding that the downbeat hasn't been played yet), and the song position is at the start of the song, then the C = 0 song position encoding MUST be used if a Start command occurs more recently than a Continue command in the session history. The C = 1, TOP = 0, CLOCK = 0 song position encoding MUST be used if a Continue command occurs more recently than a Start command in the session history 👑
B.3.1. Non-Compliant Sequencers 🙈
The Chapter Q description in this appendix assumes the sequencer system counts off time with Clock commands, as [MIDI] requires. But a few non-compliant products don't use Clock commands to count off time and do their own non-standard thing 💅
Chapter Q uses the TIMETOOLS field to add resiliency support for those non-standard products ✨ By default, the TIMETOOLS field MUST NOT appear in Chapter Q, and the T header bit MUST be set to 0. The session configuration tools described in Appendix C.2.3 can be used to select TIMETOOLS coding 🫶
Figure B.3.2 shows the format of the 24-bit TIMETOOLS field.
0 1 2
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| TIME |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure B.3.2 -- TIMETOOLS FormatThe TIME field is a 24-bit unsigned integer in units of milliseconds ⏱️ TIME codes an additive correction term for the song position coded by the TOP, CLOCK, and C fields. TIME is coded in network byte order (big-endian) 🌸
A receiver works out the correct song position by converting TIME into units of MIDI clocks and adding it to 65536*TOP + CLOCK (assuming C = 1). Or, a receiver can convert 65536*TOP + CLOCK into milliseconds (assuming C = 1) and add it to TIME 🦋 The downbeat (D header bit) semantics defined in Appendix B.3 apply to the corrected song position, so the downbeat stays iconic even after the correction 👑
B.4. System Chapter F: MIDI Time Code Tape Position 🎀
okay bestie, this appendix is all about Chapter F, the system chapter for the MIDI Time Code (MTC) commands 🧚 before you dive in, peek at the Appendix A.1 definition of "finished/unfinished commands" because it's the tea and you'll need it 🫖
The system journal MUST contain Chapter F if an active System Common Quarter Frame command (0xF1) or an active finished System Exclusive (Universal Real Time) MTC Full Frame command (F0 7F cc 01 01 hr mn sc fr F7) appears in the checkpoint history. Otherwise, the system journal MUST NOT contain Chapter F. 💅
Figure B.4.1 shows the variable-length format for Chapter F, so here's the layout 🦋
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|C|P|Q|D|POINT| COMPLETE ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | PARTIAL ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+
Figure B.4.1 -- System Chapter F FormatChapter F holds information about recent MTC tape positions coded in the session history. Receivers use Chapter F to resynchronize the MTC system after a packet loss episode, so it's basically the comeback story 🌸
Chapter F is a 1-octet header followed by several optional fields, in the order shown in Figure B.4.1. The C and P header bits form a Table of Contents (TOC) and signal the presence of the 32-bit COMPLETE field (C = 1) and the 32-bit PARTIAL field (P = 1). It's giving organised 🗂️✨
The Q header bit codes information about the COMPLETE field format. If Chapter F does not contain a COMPLETE field, Q MUST be set to 0. 🎀
The D header bit codes the tape movement direction. If the tape is moving forward, or if the tape direction is indeterminate, the D bit MUST be set to 0. If the tape is moving in the reverse direction, the D bit MUST be set to 1. In most cases, the ordering of commands in the session history clearly defines the tape direction. However, a few command sequences have an indeterminate direction (such as a session history consisting of one Full Frame command). Not me getting confused by the one-Full-Frame plot twist 🙈
The 3-bit POINT header field is interpreted as an unsigned integer. Appendix B.4.1 defines how the POINT field codes information about the contents of the PARTIAL field. If Chapter F does not contain a PARTIAL field, POINT MUST be set to 7 (if D = 0) or 0 (if D = 1). 🦋
Chapter F MUST include the COMPLETE field if an active finished Full Frame command appears in the checkpoint history or if an active Quarter Frame command that completes the encoding of a frame value appears in the checkpoint history. 👑
The COMPLETE field encodes the most recent active complete MTC frame value that appears in the session history. This frame value may take the form of a series of 8 active Quarter Frame commands (0xF1 0x0n through 0xF1 0x7n for forward tape movement, 0xF1 0x7n through 0xF1 0x0n for reverse tape movement) or may take the form of an active finished Full Frame command. So many options, all iconic 🌈
If the COMPLETE field encodes a Quarter Frame command series, the Q header bit MUST be set to 1, and the COMPLETE field MUST have the format shown in Figure B.4.2. The 4-bit fields MT0 through MT7 code the data (lower) nibble for the Quarter Frame commands for Message Type 0 through Message Type 7 [MIDI]. These nibbles encode a complete frame value, in addition to fields reserved for future use by [MIDI]. Here's the picture 📸
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MT0 | MT1 | MT2 | MT3 | MT4 | MT5 | MT6 | MT7 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure B.4.2 -- COMPLETE Field Format, Q = 1In this usage, the frame value encoded in the COMPLETE field MUST be offset by 2 frames (relative to the frame value encoded in the Quarter Frame commands) if the frame value codes a 0xF1 0x0n through 0xF1 0x7n command sequence. This offset compensates for the two-frame latency of the Quarter Frame encoding for forward tape movement. No offset is applied if the frame value codes a 0xF1 0x7n through 0xF1 0x0n Quarter Frame command sequence. Offset drama, but make it make sense 💖
The most recent active complete MTC frame value may alternatively be encoded by an active finished Full Frame command. In this case, the Q header bit MUST be set to 0, and the COMPLETE field MUST have the format shown in Figure B.4.3. The HR, MN, SC, and FR fields correspond to the hr, mn, sc, and fr data octets of the Full Frame command. 🫶
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| HR | MN | SC | FR |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure B.4.3 -- COMPLETE Field Format, Q = 0B.4.1. Partial Frames 🌙
The most recent active session history command that encodes MTC frame value data may be a Quarter Frame command other than a forward-moving 0xF1 0x7n command (which completes a frame value for forward tape movement) or a reverse-moving 0xF1 0x1n command (which completes a frame value for reverse tape movement). Plot twist, the last command doesn't always finish the frame 😭
We consider this type of Quarter Frame command to be associated with a partial frame value. The Quarter Frame sequence that defines a partial frame value MUST either start at Message Type 0 and increment contiguously to an intermediate Message Type less than 7 or start at Message Type 7 and decrement contiguously to an intermediate Message type greater than 0. A Quarter Frame command sequence that does not follow this pattern is not associated with a partial frame value. Pattern check, bestie 🔍
Chapter F MUST include a PARTIAL field if the most recent active command in the checkpoint history that encodes MTC frame value data is a Quarter Frame command that is associated with a partial frame value. Otherwise, Chapter F MUST NOT include a PARTIAL field. 🍓
The partial frame value consists of the data (lower) nibbles of the Quarter Frame command sequence. The PARTIAL field codes the partial frame value, using the format shown in Figure B.4.2. Message Type fields that are not associated with a Quarter Frame command MUST be set to 0. Zeros are the cleanest little placeholders 🧁
The POINT header field identifies the Message Type fields in the PARTIAL field that code valid data. If P = 1, the POINT field MUST encode the unsigned integer value formed by the lower 3 bits of the upper nibble of the data value of the most recent active Quarter Frame command in the session history. If D = 0 and P = 1, POINT MUST take on a value in the range 0-6. If D = 1 and P = 1, POINT MUST take on a value in the range 1-7. Ranges, but make it precise 🎯
If D = 0, MT fields (Figure B.4.2) in the inclusive range from 0 up to and including the POINT value encode the partial frame value. If D = 1, MT fields in the inclusive range from 7 down to and including the POINT value encode the partial frame value. Note that, unlike the COMPLETE field encoding, senders MUST NOT add a 2-frame offset to the partial frame value encoded in PARTIAL. Important: no offset here, periodt 💅⚠️
For the default semantics, if a recovery journal contains Chapter F and if the session history codes a legal [MIDI] series of Quarter Frame and Full Frame commands, the chapter always contains a COMPLETE or a PARTIAL field (and may contain both fields). Thus, a one-octet Chapter F (C = P = 0) always codes the presence of an illegal command sequence in the session history (under some conditions, the C = 1, P = 0 condition may also code the presence of an illegal command sequence). The illegal command sequence conditions are transient in nature and usually indicate that a Quarter Frame command sequence began with an intermediate Message Type. 😳
B.5. System Chapter X: System Exclusive 🦄
This appendix describes Chapter X, the system chapter for MIDI System Exclusive (SysEx) commands (0xF0). Readers may wish to review the Appendix A.1 definition of "finished/unfinished commands" before reading this appendix. Same tea as before, lowkey essential 🫖
Chapter X consists of a list of one or more command logs. Each log in the list codes information about a specific finished or unfinished SysEx command that appears in the session history. The system journal MUST contain Chapter X if the rules defined in Appendix B.5.2 require that one or more logs appear in the list. ✨
The log list is not preceded by a header. Instead, each log implicitly encodes its own length. Given the length of the N'th list log, the presence of the (N+1)'th list log may be inferred from the LENGTH field of the system journal header (Figure 10 in Section 5 of the main text). The log list MUST obey the oldest-first ordering rule (defined in Appendix A.1). No header, no problem, the logs basically carry their own lengths 🧚💜
B.5.1. Chapter Format 🎀
Okay bestie, Figure B.5.1 shows the bitfield format for the Chapter X command logs 💖
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|T|C|F|D|L|STA| TCOUNT | COUNT | FIRST ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DATA ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure B.5.1 -- Chapter X Command Log FormatSo a Chapter X command log is a 1-octet header, followed by the optional TCOUNT, COUNT, FIRST, and DATA fields ✨
The T, C, F, and D header bits act as a Table of Contents (TOC) for the log 📚 If T is set to 1, the 1-octet TCOUNT field shows up in the log. If C is set to 1, the 1-octet COUNT field shows up. If F is set to 1, the variable-length FIRST field shows up, and if D is set to 1, the variable-length DATA field shows up. Main character energy 👑
The L header bit sets the coding tool for the log 🦋 We define the log coding tools in Appendix B.5.2.
The STA field codes the status of the command coded by the log 💅 The 2-bit STA value is read as an unsigned integer. If STA is 0, the log codes an unfinished command. Non-zero STA values code different classes of finished commands: an STA value of 1 codes a cancelled command, an STA value of 2 codes a command that uses the "dropped 0xF7" construction, and an STA value of 3 codes all other finished commands. Section 3.2 in the main text describes cancelled and "dropped 0xF7" commands 🌸
The S bit (Appendix A.1) of the first log in the list acts as the S bit for Chapter X. For the other logs in the list, the S bit refers to the log itself. The value of the "phantom" S bit tied to the first log is defined by these rules 🧚
- If the list codes one log, the phantom S-bit value is the same as the Chapter X S-bit value.
- If the list codes multiple logs, the phantom S-bit value is the logical OR of the S-bit value of the first and second command logs in the list.
In all other respects, the S bit follows the semantics defined in Appendix A.1 🫶
The FIRST field (present if F = 1) encodes a variable-length unsigned integer value that sets the coverage of the DATA field 💜
The FIRST field (present if F = 1) encodes a variable-length unsigned integer value that specifies which SysEx data bytes are encoded in the DATA field of the log. It's giving scavenger hunt 🍓 The FIRST field is made of an octet whose most significant bit is set to 0, optionally preceded by one or more octets whose most significant bit is set to 1. The algorithm in Figure B.5.2 decodes this format into an unsigned integer to get the value dec(FIRST). FIRST uses a variable-length encoding because dec(FIRST) points at a data octet in a SysEx command, and a SysEx command may contain an arbitrary number of data octets 😭
One-Octet FIRST value:
Encoded form: 0ddddddd
Decoded form: 00000000 00000000 00000000 0ddddddd
Two-Octet FIRST value:
Encoded form: 1ccccccc 0ddddddd
Decoded form: 00000000 00000000 00cccccc cddddddd
Three-Octet FIRST value:
Encoded form: 1bbbbbbb 1ccccccc 0ddddddd
Decoded form: 00000000 000bbbbb bbcccccc cddddddd
Four-Octet FIRST value:
Encoded form: 1aaaaaaa 1bbbbbbb 1ccccccc 0ddddddd
Decoded form: 0000aaaa aaabbbbb bbcccccc cddddddd
Figure B.5.2 -- Decoding FIRST Field FormatsThe DATA field (present if D = 1) encodes a modified version of the data octets of the SysEx command coded by the log. Status octets MUST NOT be coded in the DATA field 🚫
If F = 0, the DATA field begins with the first data octet of the SysEx command and includes all subsequent data octets for the command that appear in the session history. If F = 1, the DATA field begins with the (dec(FIRST) + 1)'th data octet of the SysEx command and includes all subsequent data octets for the command that appear in the session history. Quick note: the word "command" in these descriptions means the original SysEx command as it appears in the source MIDI data stream, not a particular MIDI list SysEx command segment 🌈
The length of the DATA field is coded implicitly, using the most significant bit of each octet. The most significant bit of the final octet of the DATA field MUST be set to 1. The most significant bit of all other DATA octets MUST be set to 0. This works because the most significant bit of a MIDI data octet is 0 by definition 🧠 Apart from this length-coding tweak, the DATA field encodes a verbatim copy of all data octets it encodes ✨
B.5.2. Log Inclusion Semantics 🦋
Chapter X gives you two tools to protect SysEx commands: the "recency" tool and the "list" tool. The tool definitions use the concept of the "SysEx type" of a command, which we're about to define 👑
Each SysEx command instance in a session, except MTC Full Frame commands, is said to have a "SysEx type". Types are used in equality comparisons: two SysEx commands in a session are said to have "the same SysEx type" or "different SysEx types" 💅
If efficiency is not a concern, a sender may follow a simple typing rule: every SysEx command in the session history has a different SysEx type, so no two commands in the session have the same type 🌸
To improve efficiency, senders MAY implement exceptions to this rule. These exceptions declare that certain sets of SysEx command instances have the same SysEx type. Any command not covered by an exception follows the simple rule. The exceptions are listed below 🧚
- All commands with identical data octet fields (same number of data octets, same value for each data octet) have the same type. This rule MUST be applied to all SysEx commands in the session or not at all. Note that this exception needs no sender knowledge of the format and semantics of the SysEx commands in the stream, just the ability to count and compare octets 🙈
- Two instances of the same command whose semantics set or report the value of the same "parameter" have the same type. This exception needs specific knowledge of the format and semantics of SysEx commands. In practice, a sender implementation chooses to support this exception for certain classes of commands (such as the Universal System Exclusive commands defined in [MIDI]). If a sender supports this exception for a particular command in a class (for example, the Universal Real Time System Exclusive message for Master Volume,
F0 F7 cc 04 01 vv vv F7, defined in [MIDI]), it MUST support the exception to all instances of this particular command in the session 💖
We now use this definition of "SysEx type" to define the "recency" tool and the "list" tool for Chapter X 🫶
By default, the Chapter X log list MUST code enough information to protect the rendered MIDI performance from indefinite artifacts caused by the loss of all finished or unfinished active SysEx commands that appear in the checkpoint history (excluding finished MTC Full Frame commands, which are coded in Chapter F (Appendix B.4)) 🔥
To protect a command of a specific SysEx type with the recency tool, senders MUST code a log in the log list for the most recent finished active instance of the SysEx type that appears in the checkpoint history. Additionally, if an unfinished active instance of the SysEx type appears in the checkpoint history, senders MUST code a log in the log list for the unfinished command instance. The L header bit of both command logs MUST be set to 0 💜
To protect a command of a specific SysEx type with the list tool, senders MUST code a log in the Chapter X log list for each finished or unfinished active instance of the SysEx type that appears in the checkpoint history. The L header bit of list tool command logs MUST be set to 1 🍓
As a rule, a log REQUIRED by the list or recency tool MUST include a DATA field that codes all data octets that appear in the checkpoint history for the SysEx command instance associated with the log. The FIRST field MAY be used to configure a DATA field that minimally meets this requirement 🧚
There's an exception to this rule for cancelled commands (defined in Section 3.2). REQUIRED command logs associated with cancelled commands MAY be coded with no DATA field. However, if DATA appears in the log, DATA MUST code all data octets that appear in the checkpoint history for the command associated with the log 🙈
As defined by the preceding text in this section, by default all finished or unfinished active SysEx commands that appear in the checkpoint history (excluding finished MTC Full Frame commands) MUST be protected by the list tool or the recency tool 🌈
For some MIDI source streams, this default gives you a Chapter X that's too large 😭 For example, imagine a sender that starts transcoding a SysEx command with 10,000 data octets onto a UDP RTP stream "on the fly", sending SysEx command segments as soon as the MIDI source delivers data octets. After 1000 octets have been sent, expanding Chapter X yields an RTP packet that is too large to fit in the Maximum Transmission Unit (MTU) for the stream 💀
In this situation, if a sender uses the closed-loop sending policy for SysEx commands, the RTP packet size may always be capped by stalling the stream. In a stream stall, once the packet reaches a maximum size, the sender refrains from sending new packets with non-empty MIDI Command Sections until receiver feedback permits the trimming of Chapter X. If the stream permits arbitrary commands to appear between SysEx segments (selectable during configuration using the tools defined in Appendix C.1), the sender may stall the SysEx segment stream but continue to code other commands in the MIDI list 🫧
Stalls are a workable but suboptimal solution to Chapter X size issues 🙄 As an alternative to stalls, senders SHOULD take preemptive action during session configuration to reduce the anticipated size of Chapter X, using the methods described below 💅
- Partitioned transport. Appendix C.5 provides tools for sending a MIDI name space over several RTP streams. Senders may use these tools to map a MIDI source into a low-latency UDP RTP stream (for channel commands and short SysEx commands) and a reliable [RFC4571] TCP stream (for bulk-data SysEx commands). The cm_unused and cm_used parameters (Appendix C.1) may be used to communicate the nature of the SysEx command partition. As TCP is reliable, the RTP MIDI TCP stream would not use the recovery journal. To minimize transmission latency for short SysEx commands, senders may begin segmental transmission for all SysEx commands over the UDP stream and then cancel the UDP transmission of long commands (using tools described in Section 3.2) and resend the commands over the TCP stream 🦋
- Selective protection. Journal protection may not be necessary for all SysEx commands in a stream. The ch_never parameter (Appendix C.2) may be used to communicate which SysEx commands are excluded from Chapter X 🌸
B.5.3. TCOUNT and COUNT Fields ✨
okay so, if the T header bit is set to 1, the 8-bit TCOUNT field shows up in the command log 🧾. same deal for the C header bit and the 8-bit COUNT field 💅. both of them get read as unsigned integers, so no negatives allowed, bestie 🔢
TCOUNT is the total number of SysEx commands of the SysEx type that the log codes, counted up to the moment right after the command the log is describing enters the session history 🦋. and yes, it counts the command whether it's finished or unfinished 🙈
COUNT is kinda the same vibe but bigger 👑. it's the total number of SysEx commands sitting in the session history, except the ones that get kicked out of Chapter X through the ch_never parameter (Appendix C.2) 🚫. it's also counted at that same "right after it enters the session history" moment 💖
counting for both of these uses modulo-256 arithmetic, so the numbers wrap around once they hit 256 🌀. MTC Full Frame command instances (Appendix B.4) get counted too, but only if the TCOUNT and COUNT definitions say they should be 🎀. cancelled commands (Section 3.2) are included as well 🍓
now the main point: senders use TCOUNT and COUNT to keep track of which command instance is which, and for TCOUNT specifically, where it sits in the sequence 🧚. and if identity or sequence info is needed to protect the command type coded by the log, senders MUST use TCOUNT or COUNT 🔒 this one's non-negotiable, periodt 😭
one more rule, and it's a big one 🫶. if a sender uses COUNT in a session, then the final command log in every Chapter X in the stream MUST code the COUNT field ✨. the whole point is that receivers can resynchronize the COUNT value after packet loss, so they're not stuck with a wrong number forever 🌈
Appendix C. Session Configuration Tools 🎀
okay now we're in the appendix era 💅 the main text showed off session descriptions for minimal native and mpeg4-generic RTP MIDI streams in Sections 6.1 and 6.2 🌸. but minimal streams are kinda basic and can't do everything, so this appendix is all about customizing stream behavior with the payload format parameters 🧁
the appendix has six sections, each one handling a different part of how the stream acts 🦋
- Appendix C.1 describes the stream subsetting system (
cm_unusedandcm_used). - Appendix C.2 describes the journalling system (
ch_anchor,ch_default,ch_never,j_sec,j_update). - Appendix C.3 describes MIDI command timestamp semantics (
linerate,mperiod,octpos,tsmode). - Appendix C.4 describes the temporal duration ("media time") of an RTP MIDI packet (
guardtime,rtp_maxptime,rtp_ptime). - Appendix C.5 concerns stream description (
musicport). - Appendix C.6 describes MIDI rendering (
chanmask,cid,inline,multimode,render,rinit,subrender,smf_cid,smf_info,smf_inline,smf_url,url).
these parameters are totally optional to include in session descriptions of RTP MIDI streams 🫧. if you do use them in an SDP session description, they go on an fmtp attribute line, and that line applies to the payload type it's tied to 💜
they add extra functionality ("features") to minimal RTP MIDI streams ✨. Appendix C.7 shows how to use those features for two kinds of apps: content streaming with RTSP (Appendix C.7.1) and network musical performance with SIP (Appendix C.7.2) 🎶
the participants in a multimedia session MUST share a common view of all the RTP MIDI streams in an RTP session, as defined by a single media (m=) line. this is a must-follow rule, no skipping 🚨
and the "common view" rule can be a real headache for sendrecv streams (where everyone sends and receives) 😩. every party has its own needs, right? like, in a two-party network musical performance, one host might want its renderer tuned to match its own CPU performance, and that's a totally valid vibe [NMP] 🖥️
so the fix is to use two RTP MIDI streams, one sendonly and one recvonly, instead of one sendrecv stream 🦋. the data flows in the two streams go in opposite directions, so each control receiver can use a different renderer 💅. the third example in Appendix C.5 shows how the musicport parameter can define virtual sendrecv streams, so you get the effect without the headache 🌈
now, the general rule is that RTP MIDI doesn't handle parameter changes mid-session very well 🙈. the parameters describe heavy, stateful config that's not easy to change once the session is up and running 🍓. so parties SHOULD NOT expect that parameter change requests during a session will be accepted by other parties 😬. BUT implementors SHOULD support in-session parameter changes that are easy to handle (like the guardtime parameter from Appendix C.4) 🫶, and SHOULD be capable of accepting requests for changes to those parameters, as received by their session management protocol (for example, re-offers in SIP [RFC3264]) 📞
Appendix D defines the Augmented Backus-Naur Form (ABNF, [RFC5234]) syntax for the payload parameters 📝. Section 11 gives the Internet Assigned Numbers Authority (IANA) info on the media types and parameters this document defines 🗂️
last but not least, Appendix C.6.5 defines the media type audio/asc, which is a stored object for initializing mpeg4-generic renderers 🎧. as Appendix C.6 explains, the audio/asc media type gets assigned to the rinit parameter to specify an initialization data object for the default mpeg4-generic renderer ✨. note that RTP stream semantics are not defined for audio/asc 🙅♀️. so the asc subtype MUST NOT appear on the rtpmap line of a session description 💀 don't do it, bestie 🫶
C.1. Configuration Tools: Stream Subsetting ✨
Okay so, remember Section 3.2 in the main text? 💖 That's where we learned that the MIDI list of an RTP MIDI packet can carry any MIDI command that's allowed to show up on a MIDI 1.0 DIN cable. Literally anything legal 🎹
In this appendix we get to play with two parameters, cm_unused and cm_used, that tweak that default by kicking certain types of MIDI commands out of the MIDI list of every packet in a stream 🫶 For example, if a multimedia session splits a MIDI name space into two RTP MIDI streams, these parameters decide which commands go in each stream. Iconic divide-and-conquer behavior 👑
We also get a little language for spelling out MIDI command types. If a command type is assigned to cm_unused, the commands coded by the string MUST NOT appear in the MIDI list. If a command type is assigned to cm_used, the commands coded by the string MAY appear in the MIDI list. 🦋
The parameter list can code multiple assignments to cm_used and cm_unused. Assignments stack up cumulatively and get applied in the order they show up in the parameter list. A later assignment of a command type to the same parameter expands the scope of the earlier one. A later assignment of a command type to the opposite parameter cancels the earlier effect, either partially or completely. The cancelling is giving main character energy 💅
To set up the stream subsetting system, "implicit" assignments to cm_unused and cm_used get processed before the actual assignments in the parameter list. The System Common undefined commands (0xF4, 0xF5) and the System Real-Time Undefined commands (0xF9, 0xFD) are implicitly assigned to cm_unused. Every other command type is implicitly assigned to cm_used. 🌸
Note that the implicit assignments code the default behavior of an RTP MIDI stream, as defined in Section 3.2 in the main text. Basically, every command that's legal on a MIDI 1.0 DIN cable may appear in the stream. Also, assignments of the System Common undefined commands (0xF4, 0xF5) apply to the use of those commands in the MIDI source command stream. They do NOT touch the special use of 0xF4 and 0xF5 in SysEx segment encoding, which is defined in Section 3.2 in the main text. Not the same thing at all, bestie 🙈
As a rule, parameter assignments follow this syntax (see Appendix D for the ABNF):
<parameter> = [channel list]<command-type list>[field list]The command-type list is mandatory. The channel list and field list are optional. 🧚
The command-type list says which MIDI command types the parameter applies to. It's a concatenated sequence of one or more of the letters ABCFGHJKMNPQTVWXYZ. Each letter codes these command types:
A: Poly Aftertouch (0xA)B: System Reset (0xFF)C: Control Change (0xB)F: System Time Code (0xF1)G: System Tune Request (0xF6)H: System Song Select (0xF3)J: System Common Undefined (0xF4)K: System Common Undefined (0xF5)N: NoteOff (0x8), NoteOn (0x9)P: Program Change (0xC)Q: System Sequencer (0xF2,0xF8,0xFA,0xFB,0xFC)T: Channel Aftertouch (0xD)V: System Active Sense (0xFE)W: Pitch Wheel (0xE)X: SysEx (0xF0,0xF7)Y: System Real-Time Undefined (0xF9)Z: System Real-Time Undefined (0xFD)
On top of those letters, the letter M can also show up in the command-type list. M refers to the MIDI parameter system (see the definition in Appendix A.1 and in [MIDI]). Assigning M to cm_unused means no RPN or NRPN transactions may appear in the MIDI list. 🎀
Heads up: if cm_unused gets the letter M, Control Change (0xB) commands for the controller numbers in the standard controller assignment might still appear in the MIDI list. For the reasoning, check out Appendix A.3.4, which talks about the "general-purpose" use of parameter system controller numbers.
In the text below, rules that apply to "MIDI voice channel commands" also apply to the letter M.
The letters in the command-type list MUST be uppercase and MUST appear in alphabetical order. Letters other than ABCFGHJKMNPQTVWXYZ that show up in the list MUST be ignored. 🔥
For MIDI voice channel commands, the channel list says which MIDI channels the parameter applies to. If there's no channel list, the parameter applies to all MIDI channels (0-15). The channel list is a list of channel numbers (0 through 15) and dash-separated channel number ranges (like 0-5, 8-12, etc.). Dots (the "." characters) separate the elements in the channel list. 💜
Quick reminder: system commands don't have a MIDI channel attached. So for the command-type letters that code system commands (B, F, G, H, J, K, Q, V, Y, and Z), the channel list gets ignored. 🌈
For the command-type letter X, certain numbers in the channel list code special semantics:
- The digit
0codes that SysEx "cancel" sublists (Section 3.2 in the main text) MUST NOT appear in the MIDI list. - The digit
1codes that cancel sublists MAY appear in the MIDI list (the default condition). - The digit
2codes that commands other than System Real-Time MIDI commands MUST NOT appear between SysEx command segments in the MIDI list (the default condition). - The digit
3codes that any MIDI command type may appear between SysEx command segments in the MIDI list, with one exception: the segmented encoding of a second SysEx command. Verbatim SysEx commands are OK though 🫶
For command-type X, the channel list MUST NOT contain both digits 0 and 1, and it MUST NOT contain both digits 2 and 3. For command-type X, channel list numbers other than the ones defined above get ignored. If X doesn't have a channel list, the semantics marked "the default condition" in the list above apply. 💀
The syntax for field lists in a parameter assignment follows the syntax for channel lists. If no field list is provided, the parameter applies to all controller or note numbers. ✨
For command-type C (Control Change), the field list codes the controller numbers (0-255) that the parameter applies to.
For command-type M (Parameter System), the field list codes the RPN and NRPN controller numbers that the parameter applies to. The number range 0-16383 specifies RPN controllers. The number range 16384-32767 specifies NRPN controllers, where 16384 corresponds to NRPN controller number 0 and 32767 corresponds to NRPN controller number 16383. 🧚
For command-types N (NoteOn and NoteOff) and A (Poly Aftertouch), the field list codes the note numbers that the parameter applies to.
For command-types J and K (System Common Undefined), the field list is a single digit. That digit specifies how many data octets follow the command octet.
For command-type X (SysEx), the field list codes how many data octets may appear in a SysEx command. So the field list 0-255 specifies SysEx commands with 255 or fewer data octets. The field list 256-4294967295 specifies SysEx commands with more than 255 data octets, and it excludes commands with 255 or fewer. The field list 0 excludes all commands. 🌸
There's a secondary parameter-assignment syntax for customizing command-type X (see Appendix D for the complete ABNF):
<parameter> = "__" <h-list> *( "_" <h-list> ) "__"This assignment defines the class of SysEx commands that obeys the semantics of the assigned parameter. You specify the class by listing the permitted values of the first N data octets that follow the SysEx 0xF0 command octet. Any SysEx command whose first N data octets match the list is a member of the class. 🎀
Each <h-list> defines one data octet of the command as a dot-separated (".") list of one or more hexadecimal constants (like 7F) or dash-separated hexadecimal ranges (like 01-1F). Underscores ("_") separate each <h-list>. Double underscores ("__") delineate the data octet list. 🦋
With this syntax, each assignment specifies a single SysEx command class. Session descriptions can use several assignments to cm_used and cm_unused to set up complex behaviors. 👑
Here's an example session description showing how the stream subsetting parameters get used:
v=0
o=lazzaro 2520644554 2838152170 IN IP6 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP6 2001:DB8::7F2E:172A:1E24
a=rtpmap:96 rtp-midi/44100
a=fmtp:96 cm_unused=ACGHJKNMPTVWXYZ; cm_used=__7F_00-7F_01_01__This session description sets the stream up for clock applications. All voice channels are unused, and so are all system commands except the ones for MIDI Time Code (command-type F, plus the Full Frame SysEx command matched by the string assigned to cm_used), the System Sequencer commands (command-type Q), and System Reset (command-type B). Clock vibes all the way ⏰🌈
C.2. Configuration Tools: The Journalling System 🎀
In this appendix we define the payload format parameters that configure stream journalling and the recovery journal system. 🌸
The j_sec parameter (Appendix C.2.1) sets the journalling method for the stream. The j_update parameter (Appendix C.2.2) sets the recovery journal sending policy for the stream. Appendix C.2.2 also defines the sending policies of the recovery journal system. 💖
Appendix C.2.3 defines several parameters that modify the recovery journal semantics. These parameters change the default recovery journal semantics as defined in Section 5 and Appendices A and B. 🧚
The journalling method for a stream is set at the start of a session and MUST NOT be changed thereafter. This requirement forbids changes to the j_sec parameter once a session has begun. 🔥
A related requirement, defined in the appendices below, forbids accepting parameter values that would violate the recovery journal mandate. In many cases, changing one of the parameters defined in this appendix during an ongoing session would result in a violation of the recovery journal mandate for an implementation. In that case, the parameter change MUST NOT be accepted. 🙈
C.2.1. The j_sec Parameter
okay so Section 2.2 is where the default journalling method for a stream gets decided 💅 streams on unreliable transport (like UDP) default to using the recovery journal 🦋, and streams on reliable transport (like TCP) default to not using a journal 🫶
the j_sec parameter can override that default ✨ this memo defines two symbolic values for j_sec: "none", which means all stream payloads MUST NOT contain a journal section, and "recj", which means all stream payloads MUST contain a journal section that uses the recovery journal format 🎀
for example, j_sec might be set to "none" for a UDP stream that travels between two hosts on a local network that is known to provide reliable datagram delivery 🌸
this session description configures a UDP stream that does not use the recovery journal:
v=0
o=lazzaro 2520644554 2838152170 IN IP4 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP4 192.0.2.94
a=rtpmap:96 rtp-midi/44100
a=fmtp:96 j_sec=noneother IETF Standards-Track documents may define alternative journal formats 🧚 but these documents MUST define new symbolic values for the j_sec parameter to signal the use of the format. the vibe is: no mystery formats allowed 🙅♀️
parties MUST NOT accept a j_sec value that violates the recovery journal mandate (see Section 4 for details) 🚫 and if a session description uses a j_sec value the recipient doesn't know, the recipient MUST NOT accept the description. no guessing, bestie 💀
now the tricky part 😭 some session management tools (like RTSP, [RFC2326]) use SDP for "declarative usage" purposes (see the preamble of Section 6 for details). with those tools, SDP does not code transport details (such as UDP or TCP) for the session. instead, the server and client negotiate transport details some other way (for RTSP, that's the SETUP method) 🎬
in this scenario, using j_sec may be ill-advised, because the creator of the session description may not know the transport type for the session yet 🤷♀️ so in this case the session description SHOULD configure the journalling system using the parameters defined in the rest of Appendix C.2, but it SHOULD NOT use j_sec to set the journalling status. and remember, if j_sec does not appear in the session description, the default method for choosing the journalling method is in effect (no journal for reliable transport, recovery journal for unreliable transport) 🧠
BUT, in declarative usage situations where the creator of the session description knows that journalling is always required or never required, the session description SHOULD use the j_sec parameter ✨ knowing is everything, periodt 👑
C.2.2. The j_update Parameter
in Section 4, we use the term "sending policy" to describe the method a sender uses to choose the checkpoint packet identity for each recovery journal in a stream 🍓 in the subsections that follow, we normatively define three sending policies: anchor, closed-loop, and open-loop 🌈
as Section 4 says, the default sending policy for a stream is the closed-loop policy 🔁 the j_update parameter can override that default 💜
we define three symbolic values for j_update: "anchor", meaning the stream uses the anchor sending policy; "open-loop", meaning the stream uses the open-loop sending policy; and "closed-loop", meaning the stream uses the closed-loop sending policy. see Appendix C.2.3 for examples of session descriptions that use the j_update parameter 📖
parties MUST NOT accept a j_update value that violates the recovery journal mandate (Section 4). no exceptions, no cap 🚫
other IETF Standards-Track documents may define additional sending policies for the recovery journal system 🦋 those documents MUST define new symbolic values for the j_update parameter to signal the use of the new policy. and if a session description uses a j_update value unknown to the recipient, the recipient MUST NOT accept the description 🙈
C.2.2.1. The anchor Sending Policy
in the anchor policy, the sender uses the first packet in the stream as the checkpoint packet for all packets in the stream 🌟 this policy satisfies the recovery journal mandate (Section 4), because the checkpoint history always covers the entire stream. iconic behaviour, honestly 👑
the anchor policy does not require the use of the RTP Control Protocol (RTCP, [RFC3550]) or other feedback from receiver to sender 🎀 senders don't need to do anything special to make sure received streams start up free of artifacts, since the recovery journal always covers the entire history of the stream 🫶 and receivers are off the hook for tracking the changing identity of the checkpoint packet, because the checkpoint packet never changes. main character energy for the receiver 💅
the main drawback of the anchor policy is bandwidth efficiency 😭 because the checkpoint history covers the entire stream, the recovery journals this policy produces usually end up bigger than the journals from other policies. for single-channel MIDI data streams, that bandwidth overhead is often acceptable (see Appendix A.4 of [NMP]) 🎶 but for dense streams, the closed-loop or open-loop policies may be more appropriate ✨
C.2.2.2. The closed-loop Sending Policy 💜
okay so the closed-loop policy is the default policy of the recovery journal system 🫶 for each packet in the stream, it lets senders pick the smallest possible checkpoint history that still meets the recovery journal mandate ✨ smaller checkpoint histories usually give smaller recovery journals, so closed-loop cuts the bandwidth of a stream compared to the anchor policy 📉 iconic efficiency, bestie
the closed-loop policy relies on feedback going from receiver back to sender 🔁 it assumes a receiver regularly tells the sender the highest sequence number it has seen so far in the stream, coded in the 32-bit extension format from [RFC3550] 🧠 for RTCP, receivers send that in the Extended Highest Sequence Number Received (EHSNR) field of Receiver Reports. RTCP Sender or Receiver Reports MUST be sent by any participant in a session with the closed-loop sending policy, unless another feedback mechanism has been agreed upon 📣
the sender may safely use that receiver sequence number feedback to guide checkpoint history management, because Section 4 requires that receivers repair indefinite artifacts whenever a packet loss event occurs 🛠️ that's the whole plan, no cap
okay now for the official rules, normatively speaking 📜 when a sender prepares an RTP packet for transmission, the sender is aware of R >= 0 receivers for the stream. senders may become aware of a receiver via RTCP traffic from that receiver, via RTP packets from a paired stream sent by the receiver to the sender, via messages from a session management tool, or by other means. as receivers join and leave a session, the value of R changes 🔄
each known receiver k (1 <= k <= R) is linked to a 32-bit extended packet sequence number M(k), where the extension reflects the sequence number rollover count of the sender 🧮
if the sender has received at least one feedback report from receiver k, M(k) is the most recent report of the highest RTP packet sequence number seen by the receiver, normalized to reflect the rollover count of the sender 📬
if the sender has not received a feedback report from the receiver, M(k) is the extended sequence number of the last packet the sender transmitted before it became aware of the receiver. if the sender became aware of this receiver before it sent the first packet in the stream, M(k) is the extended sequence number of the first packet in the stream 🥹
now that we've got M(k) locked in, here's the closed-loop policy itself ✨ when preparing a new packet for transmission, a sender MUST choose a checkpoint packet with extended sequence number N, such that M(k) >= (N - 1) for all k, 1 <= k <= R, where R >= 1. the policy doesn't restrict sender behavior in the R == 0 (no known receivers) case 🫠
under the closed-loop policy as written above, a sender may transmit packets whose checkpoint history is shorter than the session history (as defined in Appendix A.1). in that situation, a new receiver that joins the stream may experience indefinite artifacts 😬 yikes
for example, say a Control Change (0xB) command for Channel Volume (controller number 7) was sent early in a stream, and later a new receiver joins the session. the closed-loop policy may permit all packets sent to the new receiver to use a checkpoint history that doesn't include that Channel Volume Control Change command. so the new receiver gets stuck with an indefinite artifact and plays every note on that channel too loudly or too softly 🔊🔉
to fix this, the closed-loop policy says that whenever a sender becomes aware of a new receiver, the sender MUST determine if that receiver would be subject to indefinite artifacts under the closed-loop policy. if so, the sender MUST ensure that the receiver starts the session free of indefinite artifacts 🛡️ for the Channel Volume issue from before, the sender may code the current state of the Channel Volume controller numbers in recovery journal Chapter C, until it receives the first RTCP RR report that signals a packet containing this Chapter C has been received 💌
in meeting this requirement, senders MAY infer the initial MIDI state of the receiver from the session description. for example, the stream example in Section 6.2 has the initial state defined in [MIDI] for General MIDI 🎹
in a unicast RTP session, a receiver may safely assume that the sender knows it's there as a receiver from the first packet sent in the RTP stream 💁♀️ but in other kinds of RTP sessions (multicast, conference focus, RTP translator/mixer), a receiver often can't tell whether the sender is initially aware of its presence 🤷♀️
so the closed-loop policy adds this: if a receiver takes part in a session where it may have access to a stream whose sender isn't aware of it, the receiver MUST take actions to ensure that its rendered MIDI performance does not contain indefinite artifacts 🧯 these protections will necessarily be incomplete, sorry bestie. for example, a receiver may monitor the Checkpoint Packet Seqnum for uncovered loss events and "err on the side of caution" about stuck notes caused by lost MIDI NoteOff commands 🎵 but it can't make up for the missing Channel Volume initialization data in the recovery journal 😭
the receiver MUST NOT discontinue these protective actions until it is certain that the sender is aware of its presence. if a receiver can't work out whether the sender is aware, the receiver MUST continue these protective actions for the duration of the session 🫡
Note: in a multicast session where everyone is expected to send and receive, getting RTCP receiver reports from the sender about the RTP stream the receiver is multicasting back is evidence that the sender knows the receiver is monitoring its RTP stream 👀
receivers may also get sender-awareness evidence from session management tools, or by other means. in practice, watching the Checkpoint Packet Seqnum over time to see if the sender is taking steps to prevent loss events for a receiver is a good sign of sender awareness 💫 so is the sudden appearance of recovery journal chapters with lots of Control Change controller data that wasn't set up by recent commands in the MIDI list shortly after sending an RTCP RR 🪄
the last set of normative closed-loop requirements is about how senders and receivers handle unplanned disruptions of RTCP feedback from a receiver to a sender. by "unplanned", we mean disruptions that aren't due to a signalled termination of an RTP stream, via an RTCP BYE or via session management tools 🚫
as defined earlier in this section, the closed-loop policy says a sender MUST choose a checkpoint packet with extended sequence number N, such that M(k) >= (N - 1) for all k, 1 <= k <= R, where R >= 1. if the sender has received at least one feedback report from receiver k, M(k) is the most recent report of the highest RTP packet sequence number seen by the receiver, normalized to reflect the rollover count of the sender 🔁
if this receiver k stops sending feedback, the M(k) value the sender uses reflects the last feedback report from that receiver. as time goes on without feedback from receiver k, that fixed M(k) value forces the sender to grow the checkpoint history, and that increases the bandwidth of the stream 📈 not ideal
at some point, the sender may need to act to limit the bandwidth of the stream. in most envisioned uses of RTP MIDI, long before that point, the SSRC time-out mechanism defined in [RFC3550] removes the uncooperative receiver from the session 👋 (the closed-loop policy doesn't suggest or require any special sender behavior on an SSRC time-out, other than the sender actions related to changing R, described earlier in this section 🙃)
but in rare situations, the bandwidth of the stream (because of a lack of feedback reports) may get too large to keep sending it to the receiver before the SSRC time-out happens for that receiver 😰 in this case, the closed-loop policy says the sender should invoke the SSRC time-out for the receiver early ⏰
now let's talk receiver responsibilities when unplanned disruptions of RTCP feedback from receiver to sender happen 🧑💻
in the unicast case, if a sender invokes the SSRC time-out mechanism for a receiver, the receiver stops getting packets from the sender. the sender behavior set by the guardtime parameter (Appendix C.4.2) lets the receiver conclude that an SSRC time-out has happened within a reasonable time period ⌛
in this time-out case, a receiver MUST keep sending RTCP feedback, so it can re-establish the RTP flow from the sender 📡 unless the receiver expects a prompt recovery of the RTP flow, the receiver MUST take actions to make sure the rendered MIDI performance doesn't exhibit "very long transient artifacts" (for example, by silencing NoteOns to prevent stuck notes) while it waits for the flow to reconnect 🤫
in the multicast case, if a sender invokes the SSRC time-out mechanism for a receiver, the receiver may keep receiving packets, but the sender won't be using the M(k) feedback from that receiver to pick each checkpoint packet anymore 🫣 if the receiver doesn't have extra information that rules out an SSRC time-out (like RTCP Receiver Reports from the sender about an RTP stream the receiver is multicasting back to the sender), the receiver MUST monitor the Checkpoint Packet Seqnum to detect an SSRC time-out. if an SSRC time-out is detected, the receiver MUST follow the instructions for SSRC time-outs described for the unicast case above 🕵️♀️
last but not least, the closed-loop policy works for RTP/RTCP sessions that use multicast transport 🌈 however, some parts of it don't scale well to sessions with lots of participants 📉 the sender state grows linearly with the number of receivers, since the sender has to track the identity and M(k) value for each receiver k. the average recovery journal size isn't independent of the number of receivers, because the RTCP reporting interval backoff slows down how often each M(k) value gets a full update 🐢
the backoff algorithm may also increase the amount of ancillary state used by implementations of the normative sender and receiver behaviors defined in Section 4 🧠💅
C.2.2.3. The open-loop Sending Policy ✨
okay bestie, the open-loop policy is for sessions that can't do the receiver-to-sender feedback that the closed-loop policy needs 😭, and that also can't use the anchor policy because of bandwidth limits. it's the scrappy little sibling who still has to keep the vibe together 💅🦋
the open-loop policy doesn't put any rules on how a sender picks the checkpoint packet for each packet in the stream. so without those rules, a receiver might find that the recovery journal in the packet that ends a loss event has a checkpoint history that doesn't cover the whole loss event. we call these loss events uncovered loss events 🙈 (not me having to explain the same thing twice, lol)
so that uncovered loss events don't wreck the recovery journal mandate 🫶, the open-loop policy hands out specific recovery jobs to senders, receivers, and the people who create session descriptions. the core idea is that the indefinite artifacts you get during an uncovered loss event fall into two classes 🌈
the first class is recoverable indefinite artifacts 💖. receivers can fix these during an uncovered loss event without the sender having to step in, even though it might cost some unpleasant transient artifacts 🎀
for example, after an uncovered loss event, receivers can fix indefinite artifacts caused by NoteOff (0x8) commands that might have happened during the loss event. they do this by running NoteOff commands for all active NoteOns commands 🍓. yes it causes a sudden silent gap in the performance, but it makes sure no stuck notes ring out forever 💀✨. we call MIDI commands that can be fixed this way recoverable MIDI commands ⭐
the second class is unrecoverable indefinite artifacts 😭. if one of these shows up during an uncovered loss event, the receiver just can't fix the stream 💔
for example, after an uncovered loss event, receivers can't fix indefinite artifacts caused by Control Change (0xB) Channel Volume (controller number 7) commands that happened during the loss event. it's literally impossible because the receiver has no way to figure out the data value of a lost Channel Volume command 🥺. we call MIDI commands that are this fragile unrecoverable MIDI commands 🌸
the open-loop policy doesn't say how to split the MIDI command set into recoverable and unrecoverable commands. instead, it assumes the creators of the session descriptions can agree on a recoverable/unrecoverable MIDI command split that works for their application 🤝💜
okay now that we've got the definitions down, here are the normative requirements for the open-loop policy 👑
in the open-loop policy, the creators of the session description MUST use the ch_anchor parameter (defined in Appendix C.2.3) to protect all unrecoverable MIDI command types from indefinite artifacts, or they MUST use the cm_unused parameter (defined in Appendix C.1) to exclude those command types from the stream. either option shields those command types from artifacts during an uncovered loss event 🛡️🎀
in the open-loop policy, receivers MUST examine the Checkpoint Packet Seqnum field of the recovery journal header after every loss event, so they can check whether that loss event is an uncovered loss event 🔍. Section 5 shows how to do that check 🧚. if an uncovered loss event has happened, a receiver MUST perform indefinite artifact recovery for all MIDI command types that aren't shielded by ch_anchor and cm_unused parameter assignments in the session description 💅
the open-loop policy doesn't put specific constraints on the sender 🌈. however, it works best when the sender manages the size of the checkpoint history so that uncovered losses happen infrequently, and that means taking the delay and loss characteristics of the network into account ⭐. also, every checkpoint packet change carries a risk of an uncovered loss, so senders should only move the checkpoint if it reduces the size of the journal 🍓🫶
C.2.3. Recovery Journal Chapter Inclusion Parameters ✨
Okay so the recovery journal chapter definitions (Appendices A and B) spell out when a chapter MUST appear in the recovery journal. Usually it goes like this: if a certain command shows up in the checkpoint history, then a certain chapter type MUST appear in the recovery journal to protect that command 🛡️
In this section we get into the chapter inclusion parameters 💅 These tweak the conditions under which a chapter lands in the journal. They're essential for the open-loop policy (Appendix C.2.2.3), and they can also simplify things for the closed-loop (Appendix C.2.2.2) and anchor (Appendix C.2.2.1) policies. The syntax for assigning them comes later in this section.
Each parameter is its own flavour of chapter inclusion semantics, and an assignment to a parameter declares which chapters (or chapter subsets) follow that flavour. Let's get into the rules, bestie 🍓
- A party MUST NOT accept chapter inclusion parameter values that violate the recovery journal mandate (Section 4). 🚫
- All assignments of the subsetting parameters (
cm_usedandcm_unused) MUST precede the first assignment of a chapter inclusion parameter in the parameter list. Order matters, periodt 🎀
Here's the normative definition of each semantic. We're describing what the parameters do to complete chapters. If a parameter gets assigned a subset of a chapter, the definition only applies to that subset 🧚
ch_never. A chapter assigned to thech_neverparameter MUST NOT appear in the recovery journal (Appendices A.4.1 and A.4.2 define exceptions to this rule for Chapter M). To signal that a chapter is excluded from the journal, an assignment toch_neverMUST be made, even if the commands coded by the chapter are assigned tocm_unused. This rule simplifies the handling of commands types that may be coded in several chapters. Main character energy, no notes 💜ch_default. A chapter assigned to thech_defaultparameter MUST follow the default semantics for the chapter, as defined in Appendices A and B. Totally standard, totally iconic ✨ch_anchor. A chapter assigned to thech_anchorMUST obey a modified version of the default chapter semantics. In the modified version, all references to the checkpoint history are replaced with references to the session history, and all references to the checkpoint packet are replaced with references to the first packet sent in the stream. It's giving "start from the very beginning" 🌸
Parameter assignments follow this syntax (see Appendix D for the ABNF) 🦋
<parameter> = [channel list]<chapter list>[field list]The chapter list is mandatory, and the channel and field lists are optional. Multiple assignments to parameters stack up cumulatively and get applied in the order they show up in a media description 🌈
To figure out what a list of chapter inclusion parameter assignments actually means, we start by assuming every channel and system chapter is implicitly assigned to ch_default. That comes with the default channel list and field list for each chapter, which we define below. Then we layer on the real assignments using the rules in this section 💖
A later assignment of a chapter to the same parameter expands the scope of the earlier one. In most cases though, a later assignment of a chapter to a different parameter cancels out (partially or completely) the effect of the earlier assignment. Plot twist 😭
The chapter list tells you which channel or system chapters the parameter applies to. It's a concatenated sequence of one or more of the letters for the chapter types, which are ACDEFMNPQTVWX. The list can also include one or more of the letters for the subchapter types of System Chapter D, which are BGHJKYZ. 🎀
The letters in a chapter list MUST be uppercase, and they MUST appear in alphabetical order. Any letters that aren't in ABCDEFGHJKMNPQTVWXYZ and show up in the chapter list MUST be ignored 🫶
The channel list tells you which channel journals the parameter applies to. If there's no channel list, the parameter applies to all channel journals. The channel list is a list of channel numbers (0 through 15) and dash-separated channel number ranges (like 0-5, 8-12, and so on). Dots (the "." characters) separate the elements in the channel list 🧚♀️
Several system chapters can be configured to have special semantics. You configure them by specifying a channel list for the system channel, using the coding described below. Heads up: MIDI system commands don't have a "channel", so the original purpose of the channel list doesn't really apply to system chapters 🙈 In the text below, "the digit N" means including N as a "channel" in the channel list for a system chapter.
- For the J and K Chapter D subchapters (undefined System Common), the digit 0 says the parameter applies to the LEGAL field of the associated command log (Figure B.1.4 of Appendix B.1). The digit 1 says it applies to the VALUE field of the command log, and the digit 2 says it applies to the COUNT field of the command log.
- For the Y and Z Chapter D subchapters (undefined System Real-Time), the digit 0 says the parameter applies to the LEGAL field of the associated command log (Figure B.1.5 of Appendix B.1), and the digit 1 says it applies to the COUNT field of the command log.
- For Chapter Q (Sequencer State Commands), the digit 0 says the parameter applies to the default Chapter Q definition, which forbids the TIME field. The digit 1 says it applies to the optional Chapter Q definition, which supports the TIME field. 🕰️
The syntax for field lists follows the syntax for channel lists. If there's no field list, the parameter applies to all controller or note numbers. For Chapter C, if there's no field list, the controller numbers do not use the enhanced Chapter C encoding (Appendix A.3.3) 🍭
For Chapter C, the field list can take values in the range 0 to 255. A field value X in the range 0-127 refers to controller number X and says that the controller number does not use enhanced Chapter C encoding. A field value X in the range 128-255 refers to controller number "X minus 128" and says that the controller number does use the enhanced Chapter C encoding. ✨
Assignments that configure the Chapter C encoding method for a controller number MUST be made to the ch_default or ch_anchor parameters. Assignments to ch_never exclude the number from the recovery journal, so the encoding method wouldn't matter anyway 🙅♀️
A Chapter C field list MUST NOT encode conflicting information about the enhanced encoding status of a particular controller number. For example, values 0 and 128 MUST NOT both be coded by a field list. Absolutely not, that's a contradiction 💀
For Chapter M, the field list codes the RPN and NRPN controller numbers that the parameter applies to. The number range 0-16383 specifies RPN controller numbers. The number range 16384-32767 specifies NRPN controller numbers, where 16384 corresponds to NRPN controller number 0 and 32767 corresponds to NRPN controller number 16383. 🎹
For Chapters N and A, the field list codes the note numbers that the parameter applies to. The note number range specified for Chapter N also applies to Chapter E 🎶
For Chapter E, the digit 0 says the parameter applies to Chapter E note logs whose V bit is set to 0, and the digit 1 says it applies to note logs whose V bit is set to 1. 🦋
For Chapter X, the field list codes the number of data octets that can appear in a SysEx command coded in the chapter. So the field list 0-255 covers SysEx commands with 255 or fewer data octets. The field list 256-4294967295 covers SysEx commands with more than 255 data octets, and it excludes commands with 255 or fewer. The field list 0 excludes all commands. Okay the range math is kinda intense, but we love a precise bestie 🧮
A secondary parameter assignment syntax customizes Chapter X (see Appendix D for the complete ABNF) 💅
<parameter> = "__" <h-list> *( "_" <h-list> ) "__"Each assignment defines a class of SysEx commands whose Chapter X coding follows the semantics of the assigned parameter. You specify the command class by listing the permitted values of the first N data octets that follow the SysEx 0xF0 command octet. Any SysEx command whose first N data octets match the list is a member of the class 🫧
Each <h-list> defines one data octet of the command as a dot-separated (".") list of one or more hexadecimal constants (such as 7F), or dash-separated hexadecimal ranges (such as 01-1F). Underscores (_) separate each <h-list>, and double-underscores (__) delineate the data octet list 🌸
Each assignment in this syntax specifies one SysEx command class. Session descriptions can use several assignments to the same (or different) parameters to set up complex Chapter X behaviours. The ordering of multiple assignments follows the guidelines for chapter parameter assignments we covered earlier in this section 🔥
The example session description below shows how the chapter inclusion parameters work together 👑
v=0
o=lazzaro 2520644554 2838152170 IN IP6 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP6 2001:DB8::7F2E:172A:1E24
a=rtpmap:96 rtp-midi/44100
a=fmtp:96 j_update=open-loop; cm_unused=ABCFGHJKMQTVWXYZ;
cm_used=__7E_00-7F_09_01.02.03__;
cm_used=__7F_00-7F_04_01.02__; cm_used=C7.64;
ch_never=ABCDEFGHJKMQTVWXYZ; ch_never=4.11-13N;
ch_anchor=P; ch_anchor=C7.64;
ch_anchor=__7E_00-7F_09_01.02.03__;
ch_anchor=__7F_00-7F_04_01.02__(The a=fmtp line has been wrapped to fit the page to accommodate memo formatting restrictions; it comprises a single line in SDP.) 📎
The j_update parameter says the stream uses the open-loop policy. Most MIDI command types are assigned to cm_unused, so they don't show up in the stream at all. That's why the assignments to the first ch_never parameter reflect that most chapters aren't in use 😌
Chapter N for several MIDI channels is assigned to ch_never. Chapter N for MIDI channels other than 4, 11, 12, and 13 may appear in the recovery journal, using the (default) ch_default semantics. In practice, this pattern would reflect knowledge about a resilient rendering method in use for the excluded channels. Smart and intentional 🧠
The MIDI Program Change command and several MIDI Control Change controller numbers are assigned to ch_anchor. Notice that the ch_anchor Chapter C assignment comes after the ch_never command, which overrides the ch_never assignment for the listed controller numbers (7 and 64). Order really is everything here 🦋
The assignment of command type X to cm_unused excludes most SysEx commands from the stream. The exceptions are General MIDI System On/Off commands and the Master Volume and Balance commands, which get in through the secondary assignment syntax. The cm_used assignment codes the exception, and the ch_anchor assignment codes how these commands are protected in Chapter X. Protected and in the spotlight 🌟
C.3. Configuration Tools: Timestamp Semantics ⏰
Okay so every MIDI command in the payload comes with its own timestamp 🎀 and the way you read those timestamps is totally configurable. This section is all about the parameters that set that up during session configuration, and honestly it's giving main character energy 💅
The tsmode parameter sets the timestamp semantics for a stream. It takes one of three token values: "comex", "async", or "buffer". 🧚
The default "comex" value means timestamps code the execution time for a command (Appendix C.3.1), and it supports accurate transcoding of Standard MIDI Files (SMFs, [MIDI]) 🎶 It's also RECOMMENDED for new MIDI user-interface controller designs ✨ The "async" value is an asynchronous timestamp sampling algorithm for time-of-arrival sources (Appendix C.3.2) 🦋 And "buffer" is a synchronous timestamp sampling algorithm (Appendix C.3.3), also for time-of-arrival sources 🌸
Ancillary parameters MAY follow tsmode in a media description. We define those in Appendices C.3.2 and C.3.3 💖
C.3.1. The comex Algorithm
Okay the default "comex" (COMmand EXecution) value is the execution time for the command, bestie 🎀 With comex, the gap between two timestamps tells you the time delay between when the commands execute. That gap can be zero, which means simultaneous execution 🫶
Comex is lowkey perfect for transcoding a Standard MIDI File (SMF, [MIDI]) into an RTP MIDI stream, because SMFs already store a timestamp for every MIDI command in the file 😭 If you have an SMF that uses metric time markers, you use the SMF tempo map (encoded in the SMF as meta-events) to convert metric SMF timestamp units into seconds-based RTP timestamp units 🧠
New MIDI controller designs (piano keyboards, drum pads, etc.) that support RTP MIDI and have direct access to sensor data SHOULD use comex interpretation for timestamps. That way simultaneous gestural events can be coded accurately by RTP MIDI 🔥
Now the tea ☕ Comex is a poor choice for transcoding MIDI 1.0 DIN cables [MIDI], and here's why. A MIDI DIN cable is an asynchronous serial protocol (320 microseconds per MIDI byte) 🙈 MIDI commands on a DIN cable aren't tagged with timestamps. Instead, MIDI DIN receivers infer command timing from when the bytes arrive. So two two-byte MIDI commands that occur at a source simultaneously end up encoded on a MIDI 1.0 DIN cable with a 640 microsecond time offset 💀 A MIDI DIN receiver can't tell whether that offset existed in the source performance or is just an artifact of the cable's serial speed. Meanwhile, the RTP MIDI comex interpretation says that a timestamp offset between two commands reflects the timing of the source performance. So it's a semantic mismatch 😩
That mismatch is exactly why comex is a poor choice for MIDI DIN cables. And choosing the RTP timestamp rate (Sections 6.1 and 6.2 in the main text) can't fix this inaccuracy, sadly 🥲 The next sections describe two alternative timestamp interpretations, "async" and "buffer", that match MIDI 1.0 DIN cable timing and other MIDI time-of-arrival sources way better 🦋
The octpos, linerate, and mperiod ancillary parameters (defined below) SHOULD NOT be used with comex. Don't do it, periodt 🙅♀️
C.3.2. The async Algorithm
The "async" tsmode value is the asynchronous sampling of a MIDI time-of-arrival source. With asynchronous sampling, the moment an octet is received from a source, it gets labelled with a wall-clock time value 🕰️ That time value has RTP timestamp units ✨
The octpos ancillary parameter defines how RTP command timestamps are derived from octet time values 🎀 If octpos has the token value "first", a timestamp codes the time value of the first octet of the command. If it has the token value "last", a timestamp codes the time value of the last octet of the command. And if the octpos parameter doesn't appear in the media description, the sender doesn't know which octet of the command the timestamp references. For example, the sender might be relying on an operating system service that doesn't specify that info 🤷♀️
Okay, this part is important, so listen up 👑 The octpos semantics refer to the first or last octet of a command as it appears on a time-of-arrival MIDI source, not as it appears in an RTP MIDI packet. That distinction matters, because the RTP coding might contain octets that weren't present in the source. For instance, the status octet of the first MIDI command in a packet may have been added to the MIDI stream during transcoding to follow the RTP MIDI running status requirements (Section 3.2) 🧩
The linerate ancillary parameter defines the timespan of one MIDI octet on the transmission medium of the MIDI source being sampled (like a MIDI 1.0 DIN cable). It has units of nanoseconds and takes on integral values. For MIDI 1.0 DIN cables, the correct linerate value is 320000, which is also the default value for the parameter ⚡
Let's do an example for async 💅 Imagine a sender transcoding a MIDI 1.0 DIN cable source into RTP. The sender runs on a computing platform that assigns time values to every incoming octet of the source, and it uses those time values to label the first octet of each command in the RTP packet. Here's the session description for that transcoding:
v=0
o=lazzaro 2520644554 2838152170 IN IP4 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP4 192.0.2.94
a=rtpmap:96 rtp-midi/44100
a=sendonly
a=fmtp:96 tsmode=async; linerate=320000; octpos=firstC.3.3. The buffer Algorithm
The "buffer" tsmode value is the synchronous sampling of a MIDI time-of-arrival source 🪣
With synchronous sampling, octets from a source get placed in a holding buffer as they arrive. At periodic intervals, the RTP sender checks the buffer. The sender removes complete commands from the buffer and codes them into an RTP packet. The command timestamp codes the moment of buffer examination, expressed in RTP timestamp units. Several commands can share the same timestamp value, and that's totally normal 🥰
The mperiod ancillary parameter defines the nominal periodic sampling interval. It takes on positive integral values and has RTP timestamp units ⏱️
The octpos ancillary parameter, which we defined in Appendix C.3.2 for asynchronous sampling, plays a different role in synchronous sampling 🔄 Here it specifies the timestamp semantics of a command whose octets span several sampling periods.
If octpos has the token value "first", the timestamp reflects the arrival period of the first octet of the command. If it has the token value "last", the timestamp reflects the arrival period of the last octet of the command. Again, the octpos semantics refer to the first or last octet as it appears on a time-of-arrival source, not as it appears in the RTP packet 🧠
If the octpos parameter doesn't appear in the media description, the timestamp MAY reflect the arrival period of any octet of the command. Senders use this option to signal that they don't know the timing details of the buffering process at subcommand granularity. It's giving honesty 🫶
Here's a buffer example 🎀 Imagine a sender transcoding a MIDI 1.0 DIN cable source into RTP. The sender runs on a computing platform that places source data into a buffer as soon as it's received. The sender polls the buffer 1000 times a second, pulls out all complete commands, and places them in an RTP packet. Here's the session description for that:
v=0
o=lazzaro 2520644554 2838152170 IN IP6 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP6 2001:DB8::7F2E:172A:1E24
a=rtpmap:96 rtp-midi/44100
a=sendonly
a=fmtp:96 tsmode=buffer; linerate=320000; octpos=last; mperiod=44The mperiod value of 44 comes from dividing the clock rate in the rtpmap attribute (44100 Hz) by the 1000 Hz buffer sampling rate, then rounding to the nearest integer 🧮 Heads up, though: command timestamps might not increment by exact multiples of 44, because the actual sampling period might not precisely match the nominal mperiod value 🙃
C.4. Configuration Tools: Packet Timing Tools ⏳
Okay in this appendix, we're covering the session configuration tools for customizing the temporal behavior of MIDI stream packets 🌈
C.4.1. Packet Duration Tools
Senders control the granularity of a stream by setting the temporal duration ("media time") of the packets in that stream 🍓 Short media times (20 ms or less) often mean an interactive session. Longer media times (100 ms or more) usually mean a content-streaming session. The RTP AVP profile [RFC3551] recommends audio packet media times between 0 and 200 ms ✨
By default, an RTP receiver dynamically senses the media time of packets in a stream and picks the length of its playout buffer to match. A receiver usually sizes its playout buffer to fit several audio packets, and it adjusts the buffer length to reflect the network jitter and the sender timing fidelity 🎧
Alternatively, the packet media time can be set statically during session configuration. Session descriptions MAY use the RTP MIDI parameter rtp_ptime to set the recommended media time for a packet. Session descriptions MAY also use the RTP MIDI parameter rtp_maxptime to set the maximum media time for a packet permitted in a stream. And both parameters MAY be used together to configure a stream 💜
The values for rtp_ptime and rtp_maxptime have the units of the RTP timestamp for the stream, as set by the rtpmap attribute (see Section 6.1). So if rtpmap sets the clock rate of a stream to 44100 Hz, a maximum packet media time of 10 ms is coded by setting rtp_maxptime=441. As the Appendix C preamble says, the senders and receivers of a stream MUST agree on common values for rtp_ptime and rtp_maxptime if the parameters appear in the media description for the stream. No drama, just agreement 🤝
0 ms is a reasonable media time for MIDI packets, and it's often used in low-latency interactive applications ⚡ In a packet with a 0 ms media time, every command executes at the instant it's coded by the packet timestamp. The session description below configures every packet in the stream to have a 0 ms media time:
v=0
o=lazzaro 2520644554 2838152170 IN IP4 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP4 192.0.2.94
a=rtpmap:96 rtp-midi/44100
a=fmtp:96 rtp_ptime=0; rtp_maxptime=0The session attributes ptime and maxptime [RFC4566] MUST NOT be used to configure an RTP MIDI stream. Sessions MUST use rtp_ptime in lieu of ptime, and MUST use rtp_maxptime in lieu of maxptime. RTP MIDI defines its own media time parameters because 0 ms values for ptime and maxptime are forbidden by [RFC3264], but they're essential for certain RTP MIDI applications 🙅♀️💖
See the Appendix C.7 examples for more about using rtp_ptime and rtp_maxptime for session configuration 📖✨
C.4.2. The guardtime Parameter ✨
okay so RTP lets a sender just stop sending audio packets for as long as it wants during a session 🦋 When it starts up again, the RTP sequence number keeps counting right where it left off, no gaps in the numbering, and the RTP timestamp shows how much media time the silence took up. ⏳
this feature comes from telephony, but it's also a perfect match for interactive MIDI sessions, because players can go quiet for several seconds during a song or between songs 🎹
certain MIDI apps get a little upgrade on top of this. in interactive apps, receivers can use online network models to help them decide what to do with lost and late RTP packets 🧠 those models can struggle if a sender goes silent on the network for long stretches 😭
so session descriptions can use the parameter guardtime to set a minimum sending rate for a media session 💅 its value says the maximum amount of time allowed between two sequential packets, written in RTP timestamp units.
typical guardtime values are 500-2000 ms. this range is not a normative bound, and parties SHOULD be prepared to process values outside this range 🌈
the congestion control rules for senders (from Section 8 and [RFC3550]) win over the guardtime parameter, no contest 👑 so if guardtime asks for a minimum sending rate, but sending at that rate would break the congestion control rules, senders MUST ignore the guardtime parameter value. in that case, senders SHOULD use the lowest minimum sending rate that still follows the congestion control rules 🫶
here's a session description that uses the guardtime parameter 🍓
v=0
o=lazzaro 2520644554 2838152170 IN IP6 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP6 2001:DB8::7F2E:172A:1E24
a=rtpmap:96 rtp-midi/44100
a=fmtp:96 guardtime=44100; rtp_ptime=0; rtp_maxptime=0C.5. Configuration Tools: Stream Description 🎀
as we talked about in Section 2.1, a party can send several RTP MIDI streams in the same RTP session, and several RTP sessions carrying MIDI can show up in one multimedia session 🧚
by default, the MIDI name space (16 channels + systems) of each RTP stream a party sends in a multimedia session is independent 💜 and by "independent" we mean three distinct things:
- if a party sends two RTP MIDI streams (A and B), MIDI voice channel 0 in stream A is a different "channel 0" than MIDI voice channel 0 in stream B 🙈
- MIDI voice channel 0 in stream B is not considered to be "channel 16" of a 32-channel MIDI voice channel space whose "channel 0" is channel 0 of stream A 💀
- streams sent by different parties over different RTP sessions, or over the same RTP session but with different payload type numbers, do not share the association that a MIDI cable pair shares when it cross-connects two devices in a MIDI 1.0 DIN network 🔥 by default, this association is only held by streams sent by different parties in the same RTP session that use the same payload type number
in this appendix, we show how to say that specific RTP MIDI streams in a multimedia session are not independent, but are related in one of the three ways defined above ✨ we use two tools to express these relations:
- the
musicportparameter. this parameter is assigned a non-negative integer value between 0 and 4294967295. it appears in the fmtp lines of payload types 🦋 - the FID grouping attribute [RFC5888] signals that several RTP sessions in a multimedia session are using the
musicportparameter to express an inter-session relationship 🫶
if a multimedia session has several payload types whose musicport parameters are assigned the same integer value, the streams using those payload types share an "identity relationship" (this includes streams that use the same payload type) 👑 streams in an identity relationship share two properties:
- identity relationship streams sent by the same party target the same MIDI name space. so if streams A and B share an identity relationship, voice channel 0 in stream A is the same "channel 0" as voice channel 0 in stream B 🌸
- pairs of identity relationship streams sent by different parties share the association that a MIDI cable pair shares when it cross-connects two devices in a MIDI 1.0 DIN network 🌈
a party MUST NOT send two RTP MIDI streams that share an identity relationship in the same RTP session. instead, each stream MUST be in a separate RTP session 🚫 as explained in Section 2.1, this restriction is needed to support the RTP MIDI method for synchronizing streams that share a MIDI name space 🔥
if a multimedia session has several payload types whose musicport parameters are assigned sequential values (i.e., i, i+1, ... i+k), the streams using those payload types share an "ordered relationship" 🎹 for example, if payload type A assigns 2 to musicport and payload type B assigns 3 to musicport, then A and B are in an ordered relationship 🍓
streams in an ordered relationship that are sent by the same party are considered by renderers to form a single larger MIDI space 🧠 for example, if stream A has a musicport value of 2 and stream B has a musicport value of 3, MIDI voice channel 0 in stream B is considered to be voice channel 16 in the larger MIDI space formed by the relationship. also, streams can take part in both an identity relationship and an ordered relationship at the same time 💖
now we state several rules for using musicport 📋
- if streams from several RTP sessions in a multimedia session use the
musicportparameter, the RTP sessions MUST be grouped using the FID grouping attribute defined in [RFC5888] 🎀 - an ordered or identity relationship MUST NOT contain both native RTP MIDI streams and mpeg4-generic RTP MIDI streams. there's an exception if a relationship consists of sendonly and recvonly (but not sendrecv) streams. in that case, the sendonly streams MUST NOT contain both types of streams, and the recvonly streams MUST NOT contain both types of streams 🙅♀️
- it's possible to build identity relationships that break the recovery journal mandate (for example, sending NoteOns for a voice channel on stream A and NoteOffs for that same voice channel on stream B) 😭 parties MUST NOT generate (or accept) session descriptions that show this flaw
- other payload formats MAY define
musicportmedia type parameters 🦋 formats would define these parameters so their sessions could be bundled into RTP MIDI name spaces. the parameter definitions MUST be compatible with themusicportsemantics defined in this appendix 💅
as a rule, at most one payload type in a relationship may specify a MIDI renderer 👑 the exception is relationships that contain sendonly and recvonly streams but no sendrecv streams. in that case, one sendonly session and one recvonly session may each define a renderer 🌈
renderer specification in a relationship may be done using the tools described in Appendix C.6 💜 these tools work for both native streams and mpeg4-generic streams. an mpeg4-generic stream that uses the Appendix C.6 tools MUST set all "config" parameters to the empty string ("").
alternatively, for mpeg4-generic streams, renderer specification may be done by setting one "config" parameter in the relationship to the renderer configuration string, and setting all other "config" parameters to the empty string ("") 🍓
now we define the sender and receiver rules that apply when a party sends several streams that target the same MIDI name space 🧚
senders MAY use the subsetting parameters (Appendix C.1) to predefine how commands are partitioned between streams, or they MAY use a dynamic partitioning strategy 🫶
receivers that merge identity relationship streams into a single MIDI command stream MUST maintain the structural integrity of the MIDI commands coded in each stream during the merge. they do this the same way software that merges traditional MIDI 1.0 DIN cable flows is responsible for creating a merged command flow compatible with [MIDI] ✨
senders MUST partition the name space so that the rendered MIDI performance does not contain indefinite artifacts (as defined in Section 4) 😱 this responsibility holds even if all streams are sent over reliable transport, because different stream latencies may still produce indefinite artifacts. for example, stuck notes may happen in a performance split over two TCP streams, if NoteOn commands are sent on one stream and NoteOff commands are sent on the other 💀
senders MUST NOT split a Registered Parameter Numbers (RPN) or Non-Registered Parameter Numbers (NRPN) transaction appearing on a MIDI channel across multiple identity relationship sessions. receivers MUST assume that RPN/NRPN transactions appearing on different identity relationship sessions are independent, and MUST preserve transactional integrity during the MIDI merge 🔒
a simple way to safely partition voice channel commands is to put all MIDI commands for a particular voice channel into the same session 🌸 safely partitioning MIDI system commands can be trickier for sessions that use System Exclusive a lot 🙈
now we show several session description examples that use the musicport parameter 🎀
our first example shows two RTP MIDI streams that drive the same General MIDI decoder. the sender partitions MIDI commands between the streams dynamically. the musicport values show that the streams share an identity relationship 🦋
v=0
o=lazzaro 2520644554 2838152170 IN IP4 first.example.net
s=Example
t=0 0
a=group:FID 1 2
c=IN IP4 192.0.2.94
m=audio 5004 RTP/AVP 96
a=rtpmap:96 mpeg4-generic/44100
a=mid:1
a=fmtp:96 streamtype=5; mode=rtp-midi; profile-level-id=12;
config=7A0A0000001A4D546864000000060000000100604D54726B0
000000600FF2F000; musicport=12
m=audio 5006 RTP/AVP 96
a=rtpmap:96 mpeg4-generic/44100
a=mid:2
a=fmtp:96 streamtype=5; mode=rtp-midi; config="";
profile-level-id=12; musicport=12(The a=fmtp lines have been wrapped to fit the page to accommodate memo formatting restrictions; they comprise single lines in SDP.) 📎
remember that Section 2.1 sets out rules for streams that target the same MIDI name space 💖 those rules, which the example above follows, require that each stream sits in a separate RTP session and that the grouping mechanisms defined in [RFC5888] signal the inter-session relationship. the "group" and "mid" attribute lines do this grouping 🫶
a variant on this example (not shown as a session description) would use two streams in an identity relationship driving the same MIDI renderer, each with a different transport type 🌈 one stream would use UDP and be dedicated to real-time messages. the second stream would use TCP [RFC4571] and be used for SysEx bulk data messages 🧠
in the next example, two mpeg4-generic streams form an ordered relationship to drive a Structured Audio decoder with 32 MIDI voice channels 🎹 both streams reside in the same RTP session 👑
v=0
o=lazzaro 2520644554 2838152170 IN IP6 first.example.net
s=Example
t=0 0
m=audio 5006 RTP/AVP 96 97
c=IN IP6 2001:DB8::7F2E:172A:1E24
a=rtpmap:96 mpeg4-generic/44100
a=fmtp:96 streamtype=5; mode=rtp-midi; config="";
profile-level-id=13; musicport=5
a=rtpmap:97 mpeg4-generic/44100
a=fmtp:97 streamtype=5; mode=rtp-midi; config="";
profile-level-id=13; musicport=6; render=synthetic;
rinit=audio/asc;
url="http://example.com/cardinal.asc";
cid="azsldkaslkdjqpwojdkmsldkfpe"(The a=fmtp lines have been wrapped to fit the page to accommodate memo formatting restrictions; they comprise single lines in SDP.)
okay so the musicport values here go 5 then 6, and that ordered little pattern is how the relationship gets established 🦋 musicport=5 maps to Structured Audio extended channels range 0-15, and musicport=6 maps to range 16-31 💖
Both config strings are empty 🫶 The actual configuration data lives in the parameters on the fmtp line of the second media description, and we define this whole configuration method in Appendix C.6 ✨
Next up, we've got two RTP MIDI streams, one recvonly and one sendonly, that together make a "virtual sendrecv" session 🎀 Each stream lives in its own RTP session, and that's required because sendonly and recvonly are RTP session attributes 💅
v=0
o=lazzaro 2520644554 2838152170 IN IP4 first.example.net
s=Example
t=0 0
a=group:FID 1 2
c=IN IP4 192.0.2.94
m=audio 5004 RTP/AVP 96
a=sendonly
a=rtpmap:96 mpeg4-generic/44100
a=mid:1
a=fmtp:96 streamtype=5; mode=rtp-midi; profile-level-id=12;
config=7A0A0000001A4D546864000000060000000100604D54726B0
000000600FF2F000; musicport=12
m=audio 5006 RTP/AVP 96
a=recvonly
a=rtpmap:96 mpeg4-generic/44100
a=mid:2
a=fmtp:96 streamtype=5; mode=rtp-midi; profile-level-id=12;
config=7A0A0000001A4D546864000000060000000100604D54726B0
000000600FF2F000; musicport=12(The a=fmtp lines have been wrapped to fit the page to accommodate memo formatting restrictions; they comprise single lines in SDP.)
To flag the "virtual sendrecv" vibe, both streams give musicport the same value, 12 🌸 Earlier in this section we covered how identity relationship streams sent by different parties share the same association a MIDI cable pair has when it cross-connects two devices in a MIDI 1.0 network 🔌 We call it "virtual sendrecv" because streams from different parties in a true sendrecv session have this exact same property too 🌈
As we talked about in the preamble to Appendix C, the big perk of virtual sendrecv is that each party gets to customize the stream it receives 🙈 In the example above, each stream has its own "config" string, which could tweak the rendering algorithm for each party. (The strings shown here happen to be identical, because General MIDI is not a configurable MPEG 4 renderer 💀)
C.6. Configuration Tools: MIDI Rendering 🎀
This appendix defines the session configuration tools for rendering ✨
The render parameter specifies a rendering method for a stream 💖 It gets a token value that signals the top-level rendering class. This memo defines four token values for render: "unknown", "synthetic", "api", and "null" 🧚
- An
"unknown"renderer is a renderer whose nature is unspecified 🌫️ It's the default renderer for native RTP MIDI streams. - A
"synthetic"renderer transforms the MIDI stream into audio output (or sometimes into stage lighting changes or other actions) 🔊 It's the default renderer for mpeg4-generic RTP MIDI streams. - An
"api"renderer presents the command stream to applications through an Application Programming Interface (API) 💻 - The
"null"renderer discards the MIDI stream 🗑️
The "null" render value has some special jobs during Offer/Answer negotiations [RFC3264] 🤝 A party uses the "null" value in an answer to reject an offered renderer. Heads up though, rejecting a renderer is its own thing, separate from rejecting a payload type (which is coded by removing the payload type from a media line) and from rejecting a media stream (coded by zeroing the port of a media line that uses the renderer) 🙅♀️
Other render token values MAY be registered with IANA 🌈 The token value MUST adhere to the ABNF for render tokens defined in Appendix D. Registrations MUST include a complete specification of parameter value usage, similar in depth to the specifications that appear throughout Appendix C.6 for "synthetic" and "api" render values 📚 If a party is offered a session description that uses a render token value the party doesn't know, the party MUST NOT accept the renderer. The options here are rejecting the renderer (using the "null" value), the payload type, the media stream, or the session description 😭
Other parameters MAY follow a render parameter in a parameter list ✨ Those extra parameters help define the exact nature of the renderer. For example, the subrender parameter (defined in Appendix C.6.2) specifies the exact nature of the renderer 🦋
Special rules apply when you use the render parameter in an mpeg4-generic stream 👑 We define those rules in Appendix C.6.5 💅
C.6.1. The multimode Parameter
A media description MAY contain several render parameters 🌸 By default, if a parameter list includes several render parameters, a receiver MUST choose exactly one renderer from the list to render the stream. The multimode parameter can be used to override that default 🔥 We define two token values for multimode: "one" and "all" ✨
- The default
"one"value requests rendering by exactly one of the listed renderers 1️⃣ - The
"all"value requests the synchronized rendering of the RTP MIDI stream by all listed renderers, if possible 🎶
If the multimode parameter appears in a parameter list, it MUST appear before the first render parameter assignment 🧵
Render parameters appear in the parameter list in order of decreasing priority 📋 A receiver MAY use that priority ordering to decide which renderer(s) to keep in a session 🫶
If the "offer" in an Offer/Answer-style negotiation [RFC3264] contains a parameter list with one or more render parameters, the "answer" MUST set the render parameters of all unchosen renderers to "null" 🙌
C.6.2. Renderer Specification
The render parameter (Appendix C.6 preamble) specifies, in a broad sense, what a renderer does with a MIDI stream 🎧 In this appendix, we describe the subrender parameter. The token value assigned to subrender defines the exact nature of the renderer. So render and subrender team up to define a renderer, kind of like how MIME types and MIME subtypes team up to define a type of media [RFC2045] 🍓
If the subrender parameter is used for a renderer definition, it MUST appear immediately after the render parameter in the parameter list 📍 A renderer definition can have at most one subrender parameter, so no double-dipping 🙅♀️
This document defines one value for subrender: the value "default" 💜 The "default" token specifies the use of the default renderer for the stream type (native or mpeg4-generic). The default renderer for native RTP MIDI streams is a renderer whose nature is unspecified (see point 6 in Section 6.1 for details). The default renderer for mpeg4-generic RTP MIDI streams is an MPEG 4 Audio Object Type whose ID number is 13, 14, or 15 (see Section 6.2 for details) 🎹
If a renderer definition doesn't use the subrender parameter, the value "default" gets assumed for subrender 🌷
Other subrender token values may be registered with IANA 🧁 Next up, we've got some guidelines for registering them.
A subrender value is registered for a specific stream type (native or mpeg4-generic) and a specific render value (excluding "null" and "unknown") 📝 Registrations for mpeg4-generic subrender values are restricted to new MPEG 4 Audio Object Types that accept MIDI input 🎛️ The syntax of the token MUST adhere to the token definition in Appendix D.
For "render=synthetic" renderers, a subrender value registration specifies an exact method for transforming the MIDI stream into audio (or sometimes into video or control actions, such as stage lighting) 💡 For standardized renderers, this is usually a pointer to a standards document, maybe with some RTP-MIDI-specific info on top. For commercial products and open-source projects, it usually takes the form of instructions for interfacing the RTP MIDI stream with the product or project software 🛠️ A "render=synthetic" registration MAY specify additional Reset State commands for the renderer (Appendix A.1) 🔄
A "render=api" subrender value registration specifies how an RTP MIDI stream interfaces with an API 🔌 This is usually a pointer to the API's programmer documentation, maybe with RTP-MIDI-specific details added in 📖
A subrender registration MAY specify an initialization file (referred to in this document as an initialization data object) for the stream 🗂️ The initialization data object MAY be encoded in the parameter list (verbatim or by reference) using the coding tools defined in Appendix C.6.3 🧩 An initialization data object MUST have a registered [RFC4288] media type and subtype [RFC2045] 🏷️
For "render=synthetic" renderers, the data object usually encodes initialization data for the renderer (sample files, synthesis patch parameters, reverberation room impulse responses, and so on) 🎚️
For "render=api" renderers, the data object usually encodes data about the stream that the API uses. For example, for an RTP MIDI stream generated by a piano keyboard controller, that could be the manufacturer and model number of the keyboard, for use in GUI presentation 🎹✨
Usually, only one initialization object gets encoded for a renderer 🙂 If a renderer uses multiple data objects, the correct receiver interpretation of those multiple data objects MUST be defined in the subrender registration 📑
A subrender value registration may also specify additional parameters, which appear in the parameter list immediately after subrender 🧁 Those parameter names MUST begin with the subrender value followed by an underscore ("_") to avoid name space collisions with future RTP MIDI parameter names (for example, a parameter "foo_bar" defined for subrender value "foo") 🫶
Now let's talk about how to interpret the subrender parameter during session configuration 🧠
If a party is offered a session description that uses a renderer whose subrender value the party doesn't know, the party MUST NOT accept the renderer. The options are rejecting the renderer (using the "null" value), the payload type, the media stream, or the session description 🚫
Receivers MUST be aware of the Reset State commands (Appendix A.1) for the renderer specified by the subrender parameter, and MUST insure that the renderer does not experience indefinite artifacts due to the presence (or the loss) of a Reset State command 🛡️💖
C.6.3. Renderer Initialization 🎀
Okay so, if a renderer for a stream uses an initialization data object, you MUST put an rinit parameter in the parameter list right after the subrender parameter 💅. If the renderer's parameter list doesn't have a subrender parameter (remember what "default" means from Appendix C.6.2 🧚), then the rinit parameter MUST go right after the render parameter.
The value for rinit MUST be the media type/subtype [RFC2045] of the initialization data object. If that object type is registered with a bunch of media types and one of them is audio, the rinit value MUST use the audio media type 🎧.
RTP MIDI has a few parameters for encoding initialization data objects for renderers: inline, url, and cid 🌸.
If a renderer uses any of inline, url, or cid, those parameters MUST come right after rinit 🫶.
If a url parameter shows up for a renderer, an inline parameter MUST NOT show up too. And if inline shows up, url MUST NOT show up. Honestly, neither url nor inline is required to appear, which is so chill 😌. But if neither of them follows rinit, then cid MUST follow rinit 💖.
The inline parameter is how you encode the data object right in the message. Its value is a double-quoted Base64 [RFC2045] encoding of the binary data object, with no line breaks ✨. Appendix E.4 has an example that builds an inline value, so go check it out 🍓.
The url parameter takes a double-quoted string that's a Uniform Resource Locator (URL) for the data object 🔗. That string MUST point to either an HyperText Transport Protocol URI (HTTP, [RFC2616]) or an HTTP over TLS URI (HTTPS, [RFC2818]) 🔐. The media type/subtype for the data object SHOULD be given in the matching HTTP or HTTPS transport header 📨.
The cid parameter is all about caching data objects 🗂️. Its value is a double-quoted string that encodes a globally unique identifier for the data object 🆔.
A cid parameter MAY come right after an inline parameter. When it does, the cid identifier value MUST be tied to that inline data object 💜.
If a url parameter is there, and the data object for that URL is expected to stay the same for the whole life of the URL, then a cid parameter MAY come right after the url parameter 🦋. The cid identifier value MUST be tied to the data object for that URL. A cid parameter with that same identifier value SHOULD be given after the data object's type/subtype in the matching HTTP transport header 🌈.
If a url parameter is there, and the data object is expected to change during the life of the URL, then a cid parameter MUST NOT come after the url parameter 🙈. A receiver reads a cid parameter as a sign that it's safe to use a cached copy of the url data object 👍. No cid parameter means it's NOT safe to use a cached copy, because it may change 😬.
Finally, the cid parameter MAY be used all by itself, without inline or url. In that case, the identifier points to a local or distributed catalog of data objects 📚.
In most cases, each renderer only has one data object coded in the parameter list. For example, the default renderer for mpeg4-generic streams uses a single data object (see Appendix C.6.5 for example usage) 🎀.
BUT, a subrender registration MAY allow multiple data objects for one renderer 🌟. If a renderer has multiple data objects encoded, each object's encoding starts with an rinit parameter, followed by inline, url, and/or cid parameters 🧩.
Initialization data objects MAY encapsulate a Standard MIDI File (SMF) 🎶. By default, a receiver MUST ignore any SMFs that are encapsulated in a data object. We define parameters to override that default in Appendix C.6.4 🔓.
To wrap up this section, here are some guidelines for registering media types for initialization data objects 📝. They're on top of what's already in [RFC4288] 💅.
Some initialization data objects can also encode MIDI note information, which means they can hold whole audio performances 🎹. Those objects SHOULD be registered with the audio media type (so they can also be used for store-and-forward rendering) AND the "application" media type (so editing tools can use them) 🛠️. Initialization objects without note storage, or initialization objects for non-audio renderers, SHOULD be registered only with an "application" media type 📱.
C.6.4. MIDI Channel Mapping 🎛️
In this appendix, we spell out how to map MIDI name spaces (16 voice channels + systems) onto a renderer 🗺️.
In the general case:
- A session can define an ordered relationship (Appendix C.5) that presents more than one MIDI name space to a renderer 🧁.
- A renderer can accept any number of MIDI name spaces, or it can expect a specific number of them 🔢.
A session description SHOULD give each renderer in the session a compatible MIDI name space 🎀. If a receiver notices that a session description has too many or too few MIDI name spaces for a renderer, then MIDI data from the extra stream name spaces MUST be discarded, and extra renderer name spaces MUST NOT be driven with MIDI data (except as described in Appendix C.6.4.1) 🚫.
If a parameter list defines several renderers and gives the "all" token value to the multimode parameter, then every renderer gets the same name space 🪞. The chanmask parameter can be used to mask out selected voice channels for each renderer 🎭. We define chanmask and the other MIDI management parameters in the subsections below 🔜.
C.6.4.1. The smf_info Parameter 🧾
The smf_info parameter defines how to use the SMFs encapsulated in renderer data objects (if there are any) ✨. It also defines how to use SMFs coded in the smf_inline, smf_url, and smf_cid parameters (defined in Appendix C.6.4.2) 💫.
The smf_info parameter describes the render parameter that came right before it in the parameter list 🔗. The smf_info parameter MUST NOT appear in parameter lists that don't use the render parameter, and it MUST NOT appear before the first use of render in the parameter list 🙅♀️.
We define three token values for smf_info: "ignore", "sdp_start", and "identity" 🌷:
- The
"ignore"value means the SMFs MUST be discarded. This is the default SMF-rendering behavior 🗑️. - The
"sdp_start"value says the SMFs MUST be rendered, and the rendering MUST begin as soon as the session description is accepted 🎬. If a receiver is offered a session description with a renderer whosesmf_infoparameter is set to"sdp_start", and the receiver doesn't support rendering SMFs, the receiver MUST NOT accept that renderer 😱. Its options include rejecting the renderer (by setting therenderparameter to"null"), the payload type, the media stream, or the entire session description 🙏. - The
"identity"value means the SMFs code the identity of the renderer 🪪. This value is meant for use with the "unknown" renderer (see the Appendix C.6 preamble) 🕵️. The MIDI commands coded in the SMF are informational, so they MUST NOT be presented to a renderer for audio presentation 🤫. Typically, the SMF uses SysEx Identity Reply commands (F0 7E nn 06 02, as defined in [MIDI]) to identify devices, and device-specific SysEx commands to describe the devices' current state (patch memory contents, etc.) 🖥️.
Other smf_info token values MAY be registered with IANA 📋. A token value MUST follow the ABNF for render tokens defined in Appendix D 📐. Registrations MUST include a complete specification of parameter usage, about as deep as the specs for "sdp_start" and "identity" in this appendix 🧠.
If a party is offered a session description that uses an smf_info parameter value it doesn't know, that party MUST NOT accept the renderer tied to the smf_info parameter 🚧. Its options include rejecting the renderer, the payload type, the media stream, or the entire session description 💌.
Now let's get into the rendering semantics for the "sdp_start" token value in detail 🔍.
The SMFs and RTP MIDI streams in a session description share the same MIDI name space(s) 🤝. In the simple case of one RTP MIDI stream and one SMF, the SMF MIDI commands and the RTP MIDI commands get merged into one name space and presented to the renderer 🌀. The indefinite artifact responsibilities for merged MIDI streams, defined in Appendix C.5, also apply to merging RTP and SMF MIDI data 🧵.
If a payload type codes multiple SMFs, the SMF name spaces are presented to the renderer as an ordered entity 📑. To figure out the order of SMFs for a renderer (which SMF is "first", which is "second", and so on), use these rules:
- If the renderer uses a single data object, the order the SMFs appear in the object's internal structure sets the order (the earliest SMF in the object is "first", the next one is "second", and so on) 🥇.
- If multiple data objects are encoded for a renderer, the position of each data object in the parameter list sets the relative order of the SMFs encoded in it (SMFs in parameters that appear earlier in the list go before SMFs in parameters that appear later) 📜.
- If SMFs are encoded in data object parameters AND in the parameters defined in Appendix C.6.4.2, the relative order of those parameters in the list sets the relative order of the SMFs (SMFs in parameters that appear earlier in the list go before SMFs in parameters that appear later) 🪄.
With that ordering in place, here's how SMFs map onto renderer name spaces 🗺️. The SMF that appears first for a renderer maps to the first renderer name space, the second SMF maps to the second renderer name space, and so on 🔢. If the associated RTP MIDI streams also form an ordered relationship, the first SMF is merged with the first name space of that relationship, the second SMF is merged with the second name space, and so on 💞.
Unless the streams and the SMFs both use MIDI Time Code, the time offset between SMF data and stream data is unspecified ⏱️. This restriction limits SMFs to applications where synchronization isn't critical, like transporting System Exclusive commands for renderer initialization or human-SMF interactivity 🧑💻.
Last but not least, each SMF in the "sdp_start" discussion above encodes exactly one MIDI name space (16 voice channels + systems) 🎯. So using the Device Name SMF meta event to specify several MIDI name spaces in one SMF is NOT supported for "sdp_start" 🙅♀️💀.
C.6.4.2. The smf_inline, smf_url, and smf_cid Parameters ✨
Okay so, sometimes the renderer data object doesn't encapsulate SMFs 🎶, but the app still wants to use SMFs the way Appendix C.6.4.1 describes. That's the whole reason these three exist 💖
The smf_inline, smf_url, and smf_cid parameters handle exactly that. They use the same syntax and semantics as the inline, url, and cid parameters from Appendix C.6.3, except the encoded data object is an SMF 🎀
Here's the bestie-level rule: these parameters belong to whichever render parameter came most recently before them in the session description 🦋 They MUST NOT show up in parameter lists that don't use render, and they MUST NOT appear before the first render in the list 🙅♀️ If you've got several smf_inline, smf_url, or smf_cid parameters for one renderer, the order you list them in sets the SMF name space ordering 📋
C.6.4.3. The chanmask Parameter 🎹
The chanmask parameter tells the renderer to ignore all MIDI voice commands on certain channel numbers 🙈 Its value is a string made of "1" and "0" digits. Each position in the string lines up with a MIDI voice channel number (system channels can't be masked, sorry system channels 😭). A "1" means "process this voice channel," and a "0" means "ignore it" 💅
The string length of the chanmask value MUST be 16 (for a single stream or an identity relationship) or a multiple of 16 (for an ordered relationship) 🌈
chanmask describes the render parameter that most recently precedes it in the session description. It MUST NOT appear in parameter lists that don't use render, and it MUST NOT appear before the first use of render in the list 🫶
The chanmask parameter describes the final MIDI name spaces that get presented to the renderer ✨ The SMF and stream parts of the MIDI name spaces can't be masked independently of each other, so no sneaky splitting 🙃
If a receiver gets offered a session description with a renderer that useschanmask, and it doesn't implement whatchanmaskmeans, the receiver MUST NOT accept that renderer unless thechanmaskvalue contains only "1"s. Not optional, bestie 💀
C.6.5. The audio/asc Media Type 🎧
In Appendix 11.3, we register the audio/asc media type 🍓 Its data object is a binary encoding of the AudioSpecificConfig data block that gets used to initialize mpeg4-generic streams (see Section 6.2 and [MPEGAUDIO]). Disk files holding this data object use the file extension ".acn" 📁
An mpeg4-generic parameter list MAY use the render, subrender, and rinit parameters with the audio/asc media type for renderer configuration 🧚 There are a few restrictions though, so listen up:
- An mpeg4-generic media description that uses the
renderparameter MUST assign the empty string ("") to the mpeg4-generic "config" parameter. Thestreamtype,mode, andprofile-level-idparameters MUST follow the normative text in Section 6.2 🎀 - Sessions that use identity or ordered relationships MUST follow the mpeg4-generic configuration restrictions in Appendix C.5 💜
- The
renderparameter MUST be assigned the value "synthetic", "unknown", "null", or a render value that's been added to the IANA repository for use with mpeg4-generic RTP MIDI streams. The "api" token value forrenderMUST NOT be used 🙅♀️ - If a
subrenderparameter is present, it MUST immediately follow therenderparameter, and it MUST be assigned the token value "default" or a subrender value added to the IANA repository for use with mpeg4-generic RTP MIDI streams. You can leave thesubrenderassignment out of the renderer configuration, and then the implied value is the default value of "default" 🌸 - If
renderis "synthetic" andsubrenderis "default" (assigned or implied), therinitparameter MUST be assigned the value audio/asc, and an AudioSpecificConfig data object MUST be encoded using the mechanisms defined in Appendices C.6.2 and C.6.3. That AudioSpecificConfig data MUST encode one of the MPEG 4 Audio Object Types defined for use with mpeg4-generic in Section 6.2. Ifsubrenderis anything other than "default," check the subrender registration for how audio/asc works with that renderer 🔥 - If
renderis "null" or "unknown," the data object MAY be omitted 🫧
There are also some general restrictions on using the audio/asc media type in RTP MIDI:
- A native stream MUST NOT assign audio/asc to
rinit. The audio/asc media type isn't meant to be a general-purpose container for rendering systems outside of MPEG usage 🙈 - The audio/asc media type defines a stored object type, not semantics for RTP streams. So audio/asc MUST NOT appear on an rtpmap line of a session description 🚫
Here are some session description examples for audio/asc 👑 The first one uses the inline parameter to code the AudioSpecificConfig block for an mpeg4-generic General MIDI stream. We work out the value that gets assigned to inline in Appendix E.4 🧮 The "default" subrender token is implied here, since the subrender parameter just isn't in the list:
v=0
o=lazzaro 2520644554 2838152170 IN IP4 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP4 192.0.2.94
a=rtpmap:96 mpeg4-generic/44100
a=fmtp:96 streamtype=5; mode=rtp-midi; config="";
profile-level-id=12; render=synthetic; rinit=audio/asc;
inline="egoAAAAaTVRoZAAAAAYAAAABAGBNVHJrAAAABgD/LwAA"(Heads up: the a=fmtp line got wrapped so it fits the page with the memo formatting rules 📄 In real SDP it's one single line.)
The next one uses the url parameter to code the AudioSpecificConfig block for that same General MIDI stream 🎶
v=0
o=lazzaro 2520644554 2838152170 IN IP4 first.example.net
s=Example
t=0 0
m=audio 5004 RTP/AVP 96
c=IN IP4 192.0.2.94
a=rtpmap:96 mpeg4-generic/44100
a=fmtp:96 streamtype=5; mode=rtp-midi; config="";
profile-level-id=12; render=synthetic; rinit=audio/asc;
url="http://example.net/oski.asc";
cid="xjflsoeiurvpa09itnvlduihgnvet98pa3w9utnuighbuk"(Same deal: the a=fmtp line is wrapped for the page, but it's a single line in SDP 💖)
C.7. Interoperability 🌍
In this appendix we lay out interoperability guidelines for two application areas 🦋
- MIDI content-streaming applications. RTP MIDI gets added to RTSP-based content-streaming servers so viewers can experience MIDI performances (made by a specified client-side renderer) in sync with other streams like video and audio 📺
- Long-distance network musical performance applications. RTP MIDI gets added to SIP-based voice chat or videoconferencing programs, either as an alternative or as an addition to audio and/or video RTP streams 🎤
For each application, we define a core set of functionalities that all implementations MUST implement 🫶
The applications covered in this section aren't a complete list of RTP MIDI uses 🌸 We expect framework documents for other applications to show up, either inside the IETF or in other organizations 🧚 We talk about other potential application areas for RTP MIDI in Section 1 of the main text of this memo 💜
C.7.1. MIDI Content-Streaming Applications ✨
okay so content streaming is when you press play and a session just shows up. 🎧 Like, you click a link on a web page for an Internet Radio channel, and that click launches an RTSP client, which uses the link's RTSP URL to find the RTSP server hosting the station. Main character energy, honestly 💅
The content can be pre-recorded (think on-demand replay of yesterday's football game) or "live" (think football game coverage as it happens) 🏈 but either way, you're usually an "audience member" and not a "participant" like you'd be on a phone call 📞
Also, these examples are about sending audio to an audience member. The interoperability stuff in this appendix is for RTP MIDI apps of that kind, NOT stuff like raw MIDI command streams used in a professional environment (recording studio, performance stage, etc.) 🎛️ Different vibe, different rules, bestie 🙈
In an RTSP session, the client gets a session description that the server "declares". That can happen through the RTSP DESCRIBE method, or through other means like HTTP or email 📨 The session description is the story of the session from the client's point of view. For example, if a media line has a non-zero port number, that's the server basically saying "hey, I'd prefer you receive RTP and RTCP on these ports" 🦋 Once media starts flowing, the server sends an RTP MIDI stream to the client, and the client renders it for presentation, maybe in sync with video or other audio streams 🎬✨
Okay, now the interoperability text for content-streaming RTSP apps. 🫶
Most of the time, server responsibilities are described as limits on the "reference" session description that a server gives out for a performance when it doesn't know anything about the client's capabilities. The reference session is a "lowest common denominator" session, meaning it's built to maximize the odds that any client can view it 🌈 If the server DOES know what the client can do, it's free to send a session description customized for that client in the DESCRIBE reply 💖
Clients MUST support unicast UDP RTP MIDI streams that use the recovery journal with the closed-loop or the anchor sending policies. Clients MUST be able to interpret stream subsetting and chapter inclusion parameters in the session description that qualify the sending policies. Client support of enhanced Chapter C encoding is OPTIONAL. 🍓
The reference session description a server offers MUST send all RTP MIDI UDP streams as unicast streams that use the recovery journal and the closed-loop or anchor sending policies. Servers SHOULD use the stream subsetting and chapter inclusion parameters in the reference session description to simplify the rendering task of the client. Server support of enhanced Chapter C encoding is OPTIONAL. 🧚
Clients and servers MUST support the use of RTSP interleaved mode (a method for interleaving RTP onto the RTSP TCP transport) 🔥
Clients MUST be able to interpret the timestamp semantics signalled by the "comex" value of the tsmode parameter (that's the timestamp semantics of Standard MIDI Files [MIDI]) 🕰️ Servers MUST use the "comex" value for the tsmode parameter in the reference session description.
Clients MUST be able to process an RTP MIDI stream whose packets encode an arbitrary temporal duration ("media time"). So, in practice, clients MUST implement a MIDI playout buffer 🧠 Clients MUST NOT depend on the presence of rtp_ptime, rtp_maxtime, and guardtime parameters in the session description in order to process packets, but they SHOULD be able to use these parameters to improve packet processing. ✨
Servers SHOULD strive to send RTP MIDI streams the same way media servers send conventional audio streams: a sequence of packets that all code either the same temporal duration (non-normative example: 50 ms packets) or one of an integral number of temporal durations (non-normative example: 50 ms, 100 ms, 250 ms, or 500 ms packets) 🎀 Servers SHOULD encode information about the packetization method in the rtp_ptime and rtp_maxtime parameters in the session description.
Clients MUST be able to examine the render and subrender parameter to figure out whether a multimedia session uses a renderer they support. Clients MUST be able to interpret the default "one" value of the multimode parameter to identify supported renderers from a list of renderer descriptions. Clients MUST be able to interpret the musicport parameter to the degree that it's relevant to the renderers they support. Clients MUST be able to interpret the chanmask parameter. 🎹
Clients supporting renderers whose data object (as encoded by a parameter value for inline) could exceed 300 octets in size MUST support the url and cid parameters, and thus must implement the HTTP protocol in addition to RTSP. HTTP over TLS [RFC2818] support for data objects is OPTIONAL. 🌸
Servers MUST specify complete rendering systems for RTP MIDI streams. Note that a minimal RTP MIDI native stream does NOT meet this requirement (Section 6.1), because the rendering method for those streams is "not specified" 🙅♀️
At the time of writing this memo, the ONLY way for servers to specify a complete rendering system is to specify an mpeg4-generic RTP MIDI stream in mode rtp-midi (Section 6.2 and Appendix C.6.5). So the only rendering systems that may be used right now are General MIDI [MIDI], DLS 2 [DLS2], or Structured Audio [MPEGSA] 👑 Note that the maximum inline value for General MIDI is well under 300 octets (so clients don't need to support the url parameter), and the maximum inline values for DLS 2 and Structured Audio may be much larger than 300 octets (so clients MUST support the url parameter) 💀
We expect the owners of rendering systems (both standardized and proprietary) to register subrender parameters for their renderers. Once registration happens, native RTP MIDI sessions may use render and subrender (Appendix C.6.2) to specify complete rendering systems for RTSP content-streaming multimedia sessions 🌈
Servers MUST NOT use the sdp_start value for the smf_info parameter in the reference session description, because that use would require clients to be able to parse and render Standard MIDI Files 🙈
Clients MUST support mpeg4-generic mode rtp-midi General MIDI (GM) sessions, at a polyphony limited by the hardware capabilities of the client. This requirement gives content providers a "lowest common denominator" rendering system to target 🎯 Note that this requirement does NOT force implementors of a non-GM renderer (such as DLS 2 or Structured Audio) to add a second rendering engine. Instead, a client may satisfy the requirement by including a set of voice patches that implement the GM instrument set and using that emulation for mpeg4-generic GM sessions 🧚♀️
It is RECOMMENDED that servers use General MIDI as the renderer for the reference session description, because clients are REQUIRED to support it. We don't require General MIDI as the reference renderer, though, because it's an inappropriate choice for normative applications 🙃
Servers using General MIDI as a "lowest common denominator" renderer SHOULD use Universal Real-Time SysEx Maximum Instantaneous Polyphony (MIP) messages [SPMIDI] to communicate the priority of voices to polyphony-limited clients. 👑
C.7.2. MIDI Network Musical Performance Applications 🎶
In Internet telephony and videoconferencing, people interact over an IP network the same way they would face-to-face. Good user experiences need low end-to-end audio latency and tight audiovisual sync (for "lip-sync") 👄 The Session Initiation Protocol (SIP, [RFC3261]) handles session management.
In this appendix section, we define interoperability guidelines for using RTP MIDI streams in interactive SIP apps. Our main focus is Network Musical Performances (NMPs), where musicians in different locations play together over the network as if they were in the same room 🎸 See [NMP] for background on NMP, and see [RFC4696] for a discussion of low-latency RTP MIDI implementation techniques for NMP ✨
The goal of NMP apps is telepresence: the parties should hear audio that's close to what they'd hear if they were in the same room 🫶 The interoperability guidelines in this appendix address RTP MIDI apps of this nature, NOT things like raw MIDI command streams used in a professional environment (recording studio, performance stage, etc.) 🎙️
We focus on session management for two-party unicast sessions that specify a renderer for RTP MIDI streams. Within that limited scope, the guidelines here are enough to let apps interoperate 💅 We define the REQUIRED capabilities of RTP MIDI senders and receivers in NMP sessions, and we define how the session descriptions exchanged are used to set up network musical performance sessions 🦋
SIP lets parties negotiate session details using the Offer/Answer protocol [RFC3264]. BUT, RTP MIDI has SO many parameters that "blind" negotiations between two parties might not end up with a common session configuration 😭
So, we now define a set of capabilities that NMP parties MUST support. Session description offers whose options lie outside the envelope of REQUIRED party behavior risk negotiation failure 💀 We also define session description idioms that the RTP MIDI part of an offer MUST follow, so the offer is easier to analyze 🧠
We use "offerer" for the party making a SIP offer and "answerer" for the party answering it. Also, unless a statement is qualified by the adjective "sender" or "receiver", a statement that a party MUST support X means it MUST support X for both sending AND receiving 🔁
If an offerer wants a "sendrecv" RTP MIDI stream, it may use a true sendrecv session, or the "virtual sendrecv" construction described in the preamble to Appendix C and in Appendix C.5. A true sendrecv session means the offerer wants to join a session where both parties use identically configured renderers. A virtual sendrecv session means the offerer is willing to join a session where the two parties may be using different renderer configurations 🌈 So parties MUST be prepared to see both real and virtual sendrecv sessions in an offer 👑
Parties MUST support unicast UDP transport of RTP MIDI streams. These streams MUST use the recovery journal with the closed-loop or anchor sending policies. These streams MUST use the stream subsetting and chapter inclusion parameters to declare the types of MIDI commands that will be sent on the stream (for sendonly streams) or will be processed (for recvonly streams), including the size limits on System Exclusive commands. Support of enhanced Chapter C encoding is OPTIONAL. 🎀
Note that both TCP and multicast UDP support are OPTIONAL. We make TCP OPTIONAL because we expect NMP renderers to rely on data objects (signalled by rinit and associated parameters) for initialization at the start of the session, and to only use System Exclusive commands for interactive control during the session 🎛️ These interactive commands are small enough to be protected via the recovery journal mechanism of RTP MIDI UDP streams 🫶
Okay so, bestie, timestamps and packet timing are where things get spicy 🌶️ The tsmode parameter decides what command timestamps mean inside the MIDI list of RTP packets, so keep that in your head as we go 🧠💖
Parties MUST support clock rates of 44.1 kHz, 48 kHz, 88.2 kHz, and 96 kHz 🎶 They MUST also support streams using the "comex", "async", and "buffer" tsmode values ✨ And recvonly offers MUST offer the default "comex" 🎀
Parties MUST support a wide range of packet temporal durations, from rtp_ptime and rtp_maxptime values of 0 all the way up to values that code 100 ms 🦋 That means receivers MUST be able to implement a playout buffer, and yes, it's giving main character energy 💅
Offers and answers MUST present rtp_ptime, rtp_maxptime, and guardtime values that support the latency users would expect in the app, subject to bandwidth constraints 🌸 Senders MUST abide by whatever values are set for these in a session description, so a receiver SHOULD use those values to size its playout buffer and get the lowest reliable latency for the session 🫶 If you're building something latency-sensitive, [RFC4696] has info on packet-sending algorithms, so go check it out 🔥 And parties MUST be able to implement the semantics of the guardtime parameter for times from 5 ms to 5000 ms ⏱️
Now, the render parameter! 🌈 Sessions MUST specify complete rendering systems for all RTP MIDI streams 👑 A minimal RTP MIDI native stream does NOT meet this, ngl, because its rendering method is "not specified" (Section 6.1) 💀
Right now, the only way to specify a complete rendering system is an mpeg4-generic RTP MIDI stream in mode rtp-midi (Section 6.2 and Appendix C.6.5) 🍓 The owners of rendering systems, both standardized and proprietary, are expected to register subrender values for their renderers 🧚 Once IANA registration happens, native RTP MIDI sessions can use render and subrender (Appendix C.6.2) to specify complete rendering systems for SIP network musical performance multimedia sessions 🎀
All parties MUST support General MIDI (GM) sessions at a polyphony limited by the party's hardware 🎹 This is the "lowest common denominator" rendering system, and honestly, without it, practical interoperability would be a total mess 😭 When using GM, parties SHOULD use Universal Real-Time SysEx MIP messages [SPMIDI] to communicate voice priority to polyphony-limited clients 💜
Also, this requirement doesn't force a non-GM renderer (for mpeg4-generic sessions, DLS 2, or Structured Audio) to build a second rendering engine 🙈 A client can instead include a set of voice patches that implement the GM instrument set and use that emulation for mpeg4-generic GM sessions ✨ We require GM support so an offerer who wants max interoperability can offer GM if the answerer doesn't accept its preferred renderer 🫶
Offerers MUST NOT present several renderers as options by listing several payload types on a media line 🚫 That's because Section 2.1 uses that construct to let a party send several RTP MIDI streams in the same RTP session 🌸
Instead, an offerer who wants to present rendering options SHOULD offer a single payload type that offers several renderers 🦋 In this construct, the parameter list codes a list of render parameters, each followed by its support parameters 🎀 As Appendix C.6.1 explains, the order of renderers in the list declares the offerer's preference 👑 The "unknown" and "null" values MUST NOT appear in the offer 💅 The answer MUST set all render values except the desired renderer to "null" ✨ So "unknown" MUST NOT appear in the answer either 🙅♀️
We use SHOULD instead of MUST in the first sentence of the paragraph above because this technique doesn't work in every situation 🥲 For example, if an offerer wants to offer both mpeg4-generic renderers and native RTP MIDI renderers as options, it can't do it that way 💀 In that case, the offerer MUST present a series of session descriptions, each offering a single renderer, until the answerer accepts one 🌈
Parties MUST support the musicport, chanmask, subrender, rinit, and inline parameters 🧡 Parties supporting renderers whose data object (as encoded by a parameter value for inline) could exceed 300 octets in size MUST support the url and cid parameters, and thus must implement the HTTP protocol 🌐 HTTP over TLS [RFC2818] support for data objects is OPTIONAL 🔐 Note that in mpeg4-generic, General MIDI data objects can't exceed 300 octets, but DLS 2 and Structured Audio data objects can 🎶 Support for the other rendering parameters (smf_cif, smf_info, smf_inline, smf_url) is OPTIONAL 💖
Up to now, we've assumed the only MIDI flows driving a renderer are the network flows in the session description 🦋 In NMP apps, that assumption would need two rendering engines: one for local use by a party and a second for the remote party 😵💫
In practice though, apps may want both parties to share a single rendering engine 🤝 In that case, the session description MUST use a virtual sendrecv session, and MUST use the stream subsetting and chapter inclusion parameters to allocate which MIDI channels are meant for a party 🎀 If two parties share a MIDI channel, the application MUST ensure appropriate MIDI merging happens at the input to the renderer 🔥
Next up: (non-MIDI) audio streams in the session 🎤 Audio streams can serve two purposes: as a "talkback" channel so parties can converse, or as a way to conduct a performance that includes MIDI and audio channels ✨ In the latter case, offers MUST use sample rates and packet temporal durations for the audio and MIDI streams that support low-latency synchronized rendering ⏱️💜
Now for the example, an offer/answer exchange in a network musical performance app 🌸 First up is an offer that follows the interoperability text in this appendix section 🎀
v=0
o=first 2520644554 2838152170 IN IP4 first.example.net
s=Example
t=0 0
a=group:FID 1 2
c=IN IP4 192.0.2.94
m=audio 16112 RTP/AVP 96
a=recvonly
a=mid:1
a=rtpmap:96 mpeg4-generic/44100
a=fmtp:96 streamtype=5; mode=rtp-midi; config="";
profile-level-id=12; cm_unused=ABCFGHJKMNPQTVWXYZ; cm_used=2NPTW;
cm_used=2C0.1.7.10.11.64.121.123; cm_used=2M0.1.2;
cm_used=X0-16; ch_never=ABCDEFGHJKMNPQTVWXYZ;
ch_default=2NPTW; ch_default=2C0.1.7.10.11.64.121.123;
ch_default=2M0.1.2; cm_default=X0-16;
rtp_ptime=0; rtp_maxptime=0; guardtime=44100;
musicport=1; render=synthetic; rinit=audio/asc;
inline="egoAAAAaTVRoZAAAAAYAAAABAGBNVHJrAAAABgD/LwAA"
m=audio 16114 RTP/AVP 96
a=sendonly
a=mid:2
a=rtpmap:96 mpeg4-generic/44100
a=fmtp:96 streamtype=5; mode=rtp-midi; config="";
profile-level-id=12; cm_unused=ABCFGHJKMNPQTVWXYZ; cm_used=1NPTW;
cm_used=1C0.1.7.10.11.64.121.123; cm_used=1M0.1.2;
cm_used=X0-16; ch_never=ABCDEFGHJKMNPQTVWXYZ;
ch_default=1NPTW; ch_default=1C0.1.7.10.11.64.121.123;
ch_default=1M0.1.2; cm_default=X0-16;
rtp_ptime=0; rtp_maxptime=0; guardtime=44100;
musicport=1; render=synthetic; rinit=audio/asc;
inline="egoAAAAaTVRoZAAAAAYAAAABAGBNVHJrAAAABgD/LwAA"(The a=fmtp lines have been wrapped to fit the page to accommodate memo formatting restrictions; it comprises a single line in SDP.)
The owner line (o=) identifies the session owner as "first" 🧑💻
This session description defines two MIDI streams: a recvonly stream where "first" receives a performance, and a sendonly stream that "first" uses to send one 🎹 The recvonly port number encodes the ports where "first" wants to receive RTP (16112) and RTCP (16113) media at IP4 address 192.0.2.94 📡 The sendonly port number encodes the port where "first" wants to receive RTCP for that stream (16115) 🎀
The musicport parameters code that the two streams share an identity relationship, so they form a virtual sendrecv stream 🤝
Both streams are mpeg4-generic RTP MIDI streams that specify a General MIDI renderer 🎶 The stream subsetting parameters code that the recvonly stream uses MIDI channel 1 exclusively for voice commands, and that the sendonly stream uses MIDI channel 2 exclusively for voice commands 🌈 This mapping lets the app share a single renderer for local and remote performers, which is iconic 👑
Now the answer to the offer 💌
v=0
o=second 2520644554 2838152170 IN IP4 second.example.net
s=Example
t=0 0
a=group:FID 1 2
c=IN IP4 192.0.2.105
m=audio 5004 RTP/AVP 96
a=sendonly
a=mid:1
a=rtpmap:96 mpeg4-generic/44100
a=fmtp:96 streamtype=5; mode=rtp-midi; config="";
profile-level-id=12; cm_unused=ABCFGHJKMNPQTVWXYZ; cm_used=2NPTW;
cm_used=2C0.1.7.10.11.64.121.123; cm_used=2M0.1.2;
cm_used=X0-16; ch_never=ABCDEFGHJKMNPQTVWXYZ;
ch_default=2NPTW; ch_default=2C0.1.7.10.11.64.121.123;
ch_default=2M0.1.2; cm_default=X0-16;
rtp_ptime=0; rtp_maxptime=882; guardtime=44100;
musicport=1; render=synthetic; rinit=audio/asc;
inline="egoAAAAaTVRoZAAAAAYAAAABAGBNVHJrAAAABgD/LwAA"
m=audio 5006 RTP/AVP 96
a=recvonly
a=mid:2
a=rtpmap:96 mpeg4-generic/44100
a=fmtp:96 streamtype=5; mode=rtp-midi; config="";
profile-level-id=12; cm_unused=ABCFGHJKMNPQTVWXYZ; cm_used=1NPTW;
cm_used=1C0.1.7.10.11.64.121.123; cm_used=1M0.1.2;
cm_used=X0-16; ch_never=ABCDEFGHJKMNPQTVWXYZ;
ch_default=1NPTW; ch_default=1C0.1.7.10.11.64.121.123;
ch_default=1M0.1.2; cm_default=X0-16;
rtp_ptime=0; rtp_maxptime=0; guardtime=88200;
musicport=1; render=synthetic; rinit=audio/asc;
inline="egoAAAAaTVRoZAAAAAYAAAABAGBNVHJrAAAABgD/LwAA"(The a=fmtp lines have been wrapped to fit the page to accommodate memo formatting restrictions; they comprise single lines in SDP.)
The owner line (o=) identifies the session owner as "second" 🙋♀️
Both media port numbers are non-zero, so "second" has accepted the session description 🎉 The stream marked "sendonly" in the offer is marked "recvonly" in the answer, and vice versa, which codes the different view of the session held by "session" 🦋 The IP4 number (192.0.2.105), RTP (5004 and 5006), and RTCP (5005 and 5007) have been changed by "second" to match its transport wishes 🌸
"Second" also made several parameter changes 💅 The rtp_maxptime for the sendonly stream was changed to code 2 ms (441 in clock units), and the guardtime for the recvonly stream was doubled ⏱️ Since these changes request capabilities that interoperable parties are REQUIRED to implement, "second" can make them with confidence that "first" can abide by them 🫶✨
Appendix D. Parameter Syntax Definitions 🎀
okay bestie, this appendix is where we write down the exact grammar for every RTP MIDI parameter we've been talking about 💅 We use ABNF ([RFC5234]) so there's zero wiggle room. When these parameters go into SDP, every single one MUST live on one fmtp attribute line for an RTP MIDI media description 🦋 And if you're using mpeg4-generic RTP MIDI streams, that same line MUST also carry the mpeg4-generic parameters (Section 6.2 explains those). The fmtp line shape comes from [RFC3640], so it looks like this:
;
; SDP fmtp line definition
;
fmtp = "a=fmtp:" token SP param-assign 0*(";" SP param-assign) CRLFHere token is where the RTP payload type goes. Also, NO white space is allowed between "a=fmtp:" and the payload type. Non-negotiable 🙅♀️
Next up, we define the syntax for every parameter from Appendix C 🌈 The trick is that each rule builds on param-assign one piece at a time, like stacking pastel blocks 🧱 For the mpeg4-generic parameter syntax, check [RFC3640] and Section 6.2.
;
;
; top-level definition for all parameters
;
;
;
; Parameters defined in Appendix C.1
param-assign = ("cm_unused=" (([channel-list] command-type
[f-list]) / sysex-data))
param-assign =/ ("cm_used=" (([channel-list] command-type
[f-list]) / sysex-data))
;
; Parameters defined in Appendix C.2
param-assign =/ ("j_sec=" ("none" / "recj" / ietf-extension))
param-assign =/ ("j_update=" ("anchor" / "closed-loop" /
"open-loop" / ietf-extension))
param-assign =/ ("ch_default=" (([channel-list] chapter-list
[f-list]) / sysex-data))
param-assign =/ ("ch_never=" (([channel-list] chapter-list
[f-list]) / sysex-data))
param-assign =/ ("ch_anchor=" (([channel-list] chapter-list
[f-list]) / sysex-data))
;
; Parameters defined in Appendix C.3
param-assign =/ ("tsmode=" ("comex" / "async" / "buffer"))
param-assign =/ ("linerate=" nonzero-four-octet)
param-assign =/ ("octpos=" ("first" / "last"))
param-assign =/ ("mperiod=" nonzero-four-octet)
;
; Parameter defined in Appendix C.4
param-assign =/ ("guardtime=" nonzero-four-octet)
param-assign =/ ("rtp_ptime=" four-octet)
param-assign =/ ("rtp_maxptime=" four-octet)
;
; Parameters defined in Appendix C.5
param-assign =/ ("musicport=" four-octet)
;
; Parameters defined in Appendix C.6
param-assign =/ ("chanmask=" 1*( 16(BIT) ))
param-assign =/ ("cid=" DQUOTE cid-block DQUOTE)
param-assign =/ ("inline=" DQUOTE base-64-block DQUOTE)
param-assign =/ ("multimode=" ("all" / "one"))
param-assign =/ ("render=" ("synthetic" / "api" / "null" /
"unknown" / extension))
param-assign =/ ("rinit=" mime-type "/" mime-subtype)
param-assign =/ ("smf_cid=" DQUOTE cid-block DQUOTE)
param-assign =/ ("smf_info=" ("ignore" / "identity" /
"sdp_start" / extension))
param-assign =/ ("smf_inline=" DQUOTE base-64-block DQUOTE)
param-assign =/ ("smf_url=" DQUOTE uri-element DQUOTE)
param-assign =/ ("subrender=" ("default" / extension))
param-assign =/ ("url=" DQUOTE uri-element DQUOTE)
;
; list definitions for the cm_ command-type
;
command-type = [A] [B] [C] [F] [G] [H] [J] [K] [M]
[N] [P] [Q] [T] [V] [W] [X] [Y] [Z]
;
; list definitions for the ch_ chapter-list
;
chapter-list = [A] [B] [C] [D] [E] [F] [G] [H] [J] [K]
[M] [N] [P] [Q] [T] [V] [W] [X] [Y] [Z]
;
; list definitions for the channel-list (used in ch_* / cm_* params)
;
channel-list = midi-chan-element *("." midi-chan-element)
midi-chan-element = midi-chan / midi-chan-range
midi-chan-range = midi-chan "-" midi-chan
;
; Decimal value of left midi-chan
; MUST be strictly less than
; decimal value of right midi-chan.
midi-chan = DIGIT / ("1" %x30-35) ; "0" .. "15"
;
; list definitions for the ch_ field list (f-list)
;
f-list = midi-field-element *("." midi-field-element)
midi-field-element = midi-field / midi-field-range
midi-field-range = midi-field "-" midi-field
;
; Decimal value of left midi-field
; MUST be strictly less than
; decimal value of right midi-field.
midi-field = four-octet
;
; Large range accommodates Chapter M
; RPN (0-16383), NRPN (16384-32767)
; parameters, and Chapter X octet sizes.
;
; definitions for ch_ sysex-data
;
sysex-data = "__" h-list *("_" h-list) "__"
h-list = hex-field-element *("." hex-field-element)
hex-field-element = hex-octet / hex-field-range
hex-field-range = hex-octet "-" hex-octet
;
; Hexadecimal value of left hex-octet
; MUST be strictly less than hexadecimal
; value of right hex-octet.
hex-octet = %x30-37 U-HEXDIG
;
; Rewritten special case of hex-octet in
; [RFC2045] (page 23).
; Note that a-f are not permitted, only A-F.
; hex-octet values MUST NOT exceed 0x7F.
;
; definitions for rinit parameter
;
mime-type = "audio" / "application"
mime-subtype = subtype-name
;
; See Appendix C.6.2 for registration
; requirements for rinit type/subtypes.
;
; subtype-name is defined in [RFC4288],
; Section 4.2.
;
; Definitions for base64 encoding
; copied from [RFC4566]
; changes from [RFC4566] to improve automatic syntax checking.
;
base-64-block = *base64-unit [base64-pad]
base64-unit = 4(base64-char)
base64-pad = (2(base64-char) "==") / (3(base64-char) "=")
base64-char = %x41-5A / %x61-7A / %x30-39 / "+" / "/"
; A-Z, a-z, 0-9, "+" and "/"
;
; generic rules
;
ietf-extension = token
;
; may only be defined in Standards-Track RFCs
extension = token
;
; may be defined
; by filing a registration with IANA
nonzero-four-octet = (NZ-DIGIT 0*8(DIGIT))
/ (%x31-33 9(DIGIT))
/ ("4" %x30-31 8(DIGIT))
/ ("42" %x30-38 7(DIGIT))
/ ("429" %x30-33 6(DIGIT))
/ ("4294" %x30-38 5(DIGIT))
/ ("42949" %x30-35 4(DIGIT))
/ ("429496" %x30-36 3(DIGIT))
/ ("4294967" %x30-31 2(DIGIT))
/ ("42949672" %x30-38 (DIGIT))
/ ("429496729" %x30-34)
;
; unsigned encoding of non-zero 32-bit value:
; 1 .. 4294967295
four-octet = "0" / nonzero-four-octet
;
; unsigned encoding of 32-bit value:
; 0 .. 4294967295
uri-element = URI-reference
; as defined in [RFC3986]
token = 1*token-char
; copied from [RFC4566]
token-char = %x21 / %x23-27 / %x2A-2B / %x2D-2E /
%x30-39 / %x41-5A / %x5E-7E
; copied from [RFC4566]
cid-block = 1*cid-char
cid-char = token-char
cid-char =/ "@"
cid-char =/ ","
cid-char =/ ";"
cid-char =/ ":"
cid-char =/ "\"
cid-char =/ "/"
cid-char =/ "["
cid-char =/ "]"
cid-char =/ "?"
cid-char =/ "="
;
; - Add back in the tspecials [RFC2045], except
; for DQUOTE and the non-email safe ( ) < >.
; - Note that the definitions above ensure that
; cid-block is always enclosed with DQUOTEs.
A = %x41 ; Uppercase-only letters used above.
B = %x42
C = %x43
D = %x44
E = %x45
F = %x46
G = %x47
H = %x48
J = %x4A
K = %x4B
M = %x4D
N = %x4E
P = %x50
Q = %x51
T = %x54
V = %x56
W = %x57
X = %x58
Y = %x59
Z = %x5A
NZ-DIGIT = %x31-39 ; non-zero decimal digit
U-HEXDIG = DIGIT / A / B / C / D / E / F
; variant of HEXDIG [RFC5234] :
; hexadecimal digit using uppercase A-F only
; The rules below are from the Core Rules from [RFC5234].
BIT = "0" / "1"
DQUOTE = %x22 ; " (Double Quote)
DIGIT = %x30-39 ; 0-9
; external references
; URI-reference: from [RFC3986]
; subtype-name: from [RFC4288]
;
; End of ABNFThe mpeg4-generic RTP payload [RFC3640] has a mode parameter that says what kind of MPEG stream you're working with 🎶 For RTP MIDI we add one brand-new value, rtp-midi, and it's defined by this little rule:
;
; mpeg4-generic mode parameter extension
;
mode =/ "rtp-midi"
; as described in Section 6.2 of this memoAppendix E. A MIDI Overview for Networking Specialists ✨
Okay bestie, this appendix is a MIDI crash course for my networking people who are new to music tech 🎹🫶. If you're actually implementing anything, go read [MIDI] for the official normative version. This is the vibe check, not the legal doc 💅.
Musicians make music by pulling off a controlled sequence of physical moves. A pianist, for example, is coordinating key presses, key releases, and pedal actions all at once 🎹. MIDI encodes all those physical gestures as a sequence of MIDI commands. It's super compact but kinda fragile, ngl 💀: losing just one command can be catastrophic for the performance.
MIDI commands are giving microprocessor machine instructions energy 🤖. They're defined as binary elements, and their bitfields have a regular structure with a specialized purpose. The upper nibble of the first command octet (the opcode field) tells you the command type 🔥. A MIDI command can be any number of complete octets, but most are 1, 2, or 3 octets long.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Channel Voice Messages | Bitfield Pattern |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| NoteOff (end a note) | 1000cccc 0nnnnnnn 0vvvvvvv |
|-------------------------------------------------------------|
| NoteOn (start a note) | 1001cccc 0nnnnnnn 0vvvvvvv |
|-------------------------------------------------------------|
| PTouch (Polyphonic Aftertouch) | 1010cccc 0nnnnnnn 0aaaaaaa |
|-------------------------------------------------------------|
| CControl (Controller Change) | 1011cccc 0xxxxxxx 0yyyyyyy |
|-------------------------------------------------------------|
| PChange (Program Change) | 1100cccc 0ppppppp |
|-------------------------------------------------------------|
| CTouch (Channel Aftertouch) | 1101cccc 0aaaaaaa |
|-------------------------------------------------------------|
| PWheel (Pitch Wheel) | 1110cccc 0xxxxxxx 0yyyyyyy |
-------------------------------------------------------------Figure E.1 -- MIDI Channel Messages
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| System Common Messages | Bitfield Pattern |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| System Exclusive | 11110000, followed by a |
| | list of 0xxxxxx octets, |
| | followed by 11110111 |
|-------------------------------------------------------------|
| MIDI Time Code Quarter Frame | 11110001 0xxxxxxx |
|-------------------------------------------------------------|
| Song Position Pointer | 11110010 0xxxxxxx 0yyyyyyy |
|-------------------------------------------------------------|
| Song Select | 11110011 0xxxxxxx |
|-------------------------------------------------------------|
| Undefined | 11110100 |
|-------------------------------------------------------------|
| Undefined | 11110101 |
|-------------------------------------------------------------|
| Tune Request | 11110110 |
|-------------------------------------------------------------|
| System Exclusive End Marker | 11110111 |
-------------------------------------------------------------
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| System Real-Time Messages | Bitfield Pattern |
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Clock | 11111000 |
|-------------------------------------------------------------|
| Undefined | 11111001 |
|-------------------------------------------------------------|
| Start | 11111010 |
|-------------------------------------------------------------|
| Continue | 11111011 |
|-------------------------------------------------------------|
| Stop | 11111100 |
|-------------------------------------------------------------|
| Undefined | 11111101 |
|-------------------------------------------------------------|
| Active Sense | 11111110 |
|-------------------------------------------------------------|
| System Reset | 11111111 |
-------------------------------------------------------------Figure E.2 -- MIDI System Messages
Figures E.1 and E.2 show the whole MIDI command family 🦋. There are three major classes: voice commands (opcode field values in the range 0x8 through 0xE), System Common commands (opcode field 0xF, commands 0xF0 through 0xF7), and System Real-Time commands (opcode field 0xF, commands 0xF8 through 0xFF). Voice commands handle the musical gestures for each timbre in a composition. System commands do global stuff that usually hits all the voice channels, like System Reset (0xFF) 🧹.
E.1. Commands Types
A voice command runs on one of 16 MIDI channels, coded by its 4-bit channel field (field cccc in Figure E.1) 🎶. In most apps, notes for different timbres get assigned to different channels. To support apps that need more than 16 channels, MIDI systems run several MIDI command streams in parallel to get 32, 48, or 64 MIDI channels. Iconic multitasking 💖.
Here's an example voice command: a NoteOn (opcode 0x9), with binary encoding 1001cccc 0nnnnnnn 0aaaaaaa. It signals the start of a musical note on MIDI channel cccc. The pitch is coded by the note number nnnnnnn, and the onset amplitude is coded by the note velocity aaaaaaa ✨.
Other voice commands signal the end of notes (NoteOff, opcode 0x8), map a specific timbre to a MIDI channel (PChange, opcode 0xC), or set the value of parameters that modulate the timbral quality (all the other voice commands) 🌸. The exact meaning of most voice channel commands depends on the rendering algorithms the MIDI receiver uses to generate sound. In most apps, a MIDI sender has a model (in some sense) of the rendering method the receiver uses 🧚.
System commands do a bunch of global tasks in the stream 🎛️. That includes "sequencer" playback control of pre-recorded MIDI commands (the Song Position Pointer, Song Select, Clock, Start, Continue, and Stop messages), SMPTE time code (the MIDI Time Code Quarter Frame command), and the communication of device-specific data (the System Exclusive messages).
E.2. Running Status
All MIDI command bitfields share a special structure: the leading bit of the first octet is set to 1, and the leading bit of all subsequent octets is set to 0 🔢. This structure supports a data compression system called running status [MIDI], which improves the coding efficiency of MIDI. So smart, honestly 🧠.
In running status coding, the first octet of a MIDI voice command may be dropped if it's identical to the first octet of the previous MIDI voice command. Combine that with the convention of treating a NoteOn with a null third octet as a NoteOff, and you can code note sequences using just two octets per command 🎹✨.
Running status coding is only used for voice commands. If a System Common message shows up in the stream, it cancels running status mode for the next voice command. System Real-Time messages, though? They do not cancel running status mode 🙅♀️.
E.3. Command Timing
The bitfield formats in Figures E.1 and E.2 don't encode the execution time for a command. Timing info isn't part of the MIDI command syntax itself. Different applications of the MIDI command language use different methods to encode timing ⏰.
Take MIDI 1.0 DIN cables, for example. The MIDI command set acts as the transport layer for them [MIDI]. These cables are short asynchronous serial lines that facilitate remote operation of musical instruments and audio gear 🎚️. Timestamps are not sent over a MIDI 1.0 DIN cable. Instead, the standard uses an implicit "time of arrival" code, and receivers execute MIDI commands the moment they arrive 🏃♀️.
Standard MIDI Files (SMFs, [MIDI]), on the other hand, are a file format for representing complete musical performances 🎼. They add an explicit timestamp to each MIDI command, using a delta encoding scheme that's optimized for the statistics of musical performance. SMF timestamps usually code timing using the metric notation of a musical score. SMF meta-events add a tempo map to the file, so score beats can be accurately converted into seconds during rendering. Okay, the timing glow-up is real 👑🥹.
E.4. AudioSpecificConfig Templates for MMA Renderers ✨
okay so, bestie, remember how Section 6.2 and Appendix C.6.5 say session descriptions carry an AudioSpecificConfig data block to pick a MIDI rendering algorithm for mpeg4-generic RTP MIDI streams? this section is the cute little cheat sheet for that 🧚
The bitfield format of AudioSpecificConfig is defined in [MPEGAUDIO]. The StructuredAudioSpecificConfig, which is the key data structure coded inside AudioSpecificConfig, is defined in [MPEGSA].
If you want to build Structured Audio renderers, you need to be obsessed with [MPEGSA] and [MPEGAUDIO] 💀 no shortcuts. But most people are just gonna use one of the two MIDI Manufacturers Association (MMA) renderers that can be named in AudioSpecificConfig: General MIDI (GM, [MIDI]) and Downloadable Sounds 2 (DLS 2, [DLS2]). Honestly the main character renderers 👑
To help those people out, we put the AudioSpecificConfig bitfield formats for a GM renderer and a DLS 2 renderer right here. We double-checked them really carefully and we believe they're correct 🫶 BUT the stuff below is informative, and [MPEGAUDIO] and [MPEGSA] are the normative definitions for AudioSpecificConfig. so if the two ever disagree, the spec wins, no cap.
As described in Section 6.2, a minimal mpeg4-generic session description encodes the AudioSpecificConfig binary bitfield as a hexadecimal string (the format of which is defined in [RFC3640]) and assigns it to the "config" parameter. As described in Appendix C.6.3, a session description that uses the render parameter encodes the AudioSpecificConfig binary bitfield as a Base64-encoded string assigned to the inline parameter, or as the body of an HTTP URL assigned to the url parameter. 🌸
Here's a simplified binary AudioSpecificConfig bitfield format that works for sending and receiving GM and DLS 2 data:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| AOTYPE |FREQIDX|CHANNEL|SACNK| FILE_BLK 1 (required) ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1|SACNK| FILE_BLK 2 (optional) ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |1|SACNK| FILE_BLK N (optional) ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0|0| (first "0" bit terminates FILE_BLK list)
+-+-+
Figure E.3 -- Simplified AudioSpecificConfigok the 5-bit AOTYPE field is the Audio Object Type as an unsigned integer 🦋 The legal values for mpeg4-generic RTP MIDI streams are "15" (General MIDI), "14" (DLS 2), and "13" (Structured Audio). So a receiver that doesn't support all three mpeg4-generic renderers can parse the first 5 bits of an AudioSpecificConfig coded in a session description and reject any session that asks for a renderer it doesn't support. that's giving efficient 💅
The 4-bit FREQIDX field is the sampling rate of the renderer. Here's how FREQIDX values map to sampling rates:
FREQIDX Sampling Frequency
0x0 96000
0x1 88200
0x2 64000
0x3 48000
0x4 44100
0x5 32000
0x6 24000
0x7 22050
0x8 16000
0x9 12000
0xa 11025
0xb 8000
0xc reserved
0xd reserved
0xe reserved
0xf escape value
Figure E.4 -- FreqIdx EncodingSenders MUST specify a sampling frequency that matches the RTP clock rate, if possible; if not, senders MUST specify the escape value. Receivers MUST consult the RTP clock parameter for the true sampling rate if the escape value is specified. this is the rule that keeps everyone on the same page 🎀
The 4-bit CHANNEL field is the number of audio channels for the renderer. The values 0x1 to 0x5 mean 1 to 5 audio channels, 0x6 means 5+1 surround sound, and 0x7 means 7+1 surround sound. If the rtpmap line in the session description specifies one of these formats, CHANNEL MUST be set to the matching value. Otherwise, CHANNEL MUST be set to 0x0. 🌈
The CHANNEL field is followed by a list of one or more binary file data blocks. The 3-bit SACNK field (the chunk_type field in class StructuredAudioSpecificConfig, defined in [MPEGSA]) says what type each data block is.
For General MIDI, only Standard MIDI Files may show up in the list (SACNK field value 2). For DLS 2, only Standard MIDI Files and DLS 2 RIFF files (SACNK field value 4) may show up. For both of these file types, the FILE_BLK field has the format shown in Figure E.5: a 32-bit unsigned integer value (FILE_LEN) coding the number of bytes in the SMF or RIFF file, followed by FILE_LEN bytes coding the file data. 🍓
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| FILE_LEN (32-bit, a byte count SMF file or RIFF file) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| FILE_DATA (file contents, a list of FILE_LEN bytes) ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure E.5 -- The FILE_BLK Field FormatNote that several files can follow the CHANNEL field. The "1" constant fields in Figure E.3 code the presence of another file, and the "0" constant field codes the end of the list. The final "0" bit in Figure E.3 codes the absence of special coding tools (see [MPEGAUDIO] for details). Senders that aren't using these tools MUST append this "0" bit, and receivers that don't understand these coding tools MUST ignore all data following a "1" in this position. the bit-level stuff is wild but it's so precise 🤓
The StructuredAudioSpecificConfig bitfield structure requires the presence of one FILE_BLK. For mpeg4-generic RTP MIDI use of DLS 2, FILE_BLKs MUST code RIFF files or SMF files. For mpeg4-generic RTP MIDI use of General MIDI, FILE_BLKs MUST code SMF files. By default, this SMF will be ignored (Appendix C.6.4.1). In that default case, a GM StructuredAudioSpecificConfig bitfield SHOULD code a FILE_BLK whose FILE_LEN is 0 and whose FILE_DATA is empty. ✨
To wrap up this appendix, we work out the StructuredAudioSpecificConfig we use in the General MIDI session examples in this memo. Looking at Figure E.3, for GM, AOTYPE = 15. Our examples use a 44,100 Hz sample rate (FREQIDX = 4) and are in mono (CHANNEL = 1). For GM, a single SMF is encoded (SACNK = 2), using the SMF shown in Figure E.6 (a 26 byte file). 🎶
--------------------------------------------
| MIDI File = <Header Chunk> <Track Chunk> |
--------------------------------------------
<Header Chunk> = <chunk type> <length> <format> <ntrks> <divsn>
4D 54 68 64 00 00 00 06 00 00 00 01 00 60
<Track Chunk> = <chunk type> <length> <delta-time> <end-event>
4D 54 72 6B 00 00 00 04 00 FF 2F 00
Figure E.6 -- SMF File Encoded in the ExamplePlacing these constants in binary format into the data structure shown in Figure E.3 gives us the constant shown in Figure E.7. this is the moment where it all clicks 🦋
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0 1 1 1 1|0 1 0 0|0 0 0 1|0 1 0|0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0 0 0 0 0 0 0 0 0 0 0 1 1 0 1 0|0 1 0 0|1 1 0 1|0 1 0 1|0 1 0 0|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0 1 1 0|1 0 0 0|0 1 1 0|0 1 0 0|0 0 0 0|0 0 0 0|0 0 0 0|0 0 0 0|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0 0 0 0|0 0 0 0|0 0 0 0|0 1 1 0|0 0 0 0|0 0 0 0|0 0 0 0|0 0 0 0|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0 0 0 0|0 0 0 0|0 0 0 1|0 0 0 0|0 0 0 0|0 1 1 0|0 0 0 0|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0 1 0 0|1 1 0 1|0 1 0 1|0 1 0 0|0 1 1 1|0 0 1 0|0 1 1 0|1 0 1 1|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0 0 0 0|0 0 0 0|0 0 0 0|0 0 0 0|0 0 0 0|0 0 0 0|0 0 0 0|0 1 1 0|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0 0 0 0|0 0 0 0|1 1 1 1|1 1 1 1|0 0 1 0|1 1 1 1|0 0 0 0|0 0 0 0|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0|0|
+-+-+
Figure E.7 -- AudioSpecificConfig Used in GM ExamplesWriting this bitfield out as an ASCII hexadecimal string gives:
7A0A0000001A4D546864000000060000000100604D54726B0000000600FF2F000That string gets assigned to the "config" parameter in the minimal mpeg4-generic General MIDI examples in this memo (such as the example in Section 6.2). Writing this string in Base64 [RFC2045] gives:
egoAAAAaTVRoZAAAAAYAAAABAGBNVHJrAAAABgD/LwAAThat string gets assigned to the inline parameter in the General MIDI example shown in Appendix C.6.5. 💖
References 📚
Normative References ✨
Okay bestie, here's the normative pile 📚💖 These are the docs you need to have read to get this spec, and yes, every single one is listed exactly as written.
[MIDI]MIDI Manufacturers Association. "The Complete MIDI 1.0 Detailed Specification", 1996.[RFC3550]Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, July 2003.[RFC3551]Schulzrinne, H. and S. Casner, "RTP Profile for Audio and Video Conferences with Minimal Control", STD 65, RFC 3551, July 2003.[RFC3640]van der Meer, J., Mackie, D., Swaminathan, V., Singer, D., and P. Gentric, "RTP Payload Format for Transport of MPEG-4 Elementary Streams", RFC 3640, November 2003.[MPEGSA]International Standards Organization. "ISO/IEC 14496 MPEG-4", Part 3 (Audio), Subpart 5 (Structured Audio), 2001.[RFC4566]Handley, M., Jacobson, V., and C. Perkins, "SDP: Session Description Protocol", RFC 4566, July 2006.[MPEGAUDIO]International Standards Organization. "ISO 14496 MPEG-4", Part 3 (Audio), 2001.[RFC2045]Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies", RFC 2045, November 1996.[DLS2]MIDI Manufacturers Association. "The MIDI Downloadable Sounds Specification", v98.2, 1998.[RFC5234]Crocker, D., Ed., and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, January 2008.[RFC2119]Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.[RFC3711]Baugher, M., McGrew, D., Naslund, M., Carrara, E., and K. Norrman, "The Secure Real-time Transport Protocol (SRTP)", RFC 3711, March 2004.[RFC3264]Rosenberg, J. and H. Schulzrinne, "An Offer/Answer Model with Session Description Protocol (SDP)", RFC 3264, June 2002.[RFC3986]Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform Resource Identifier (URI): Generic Syntax", STD 66, RFC 3986, January 2005.[RFC2616]Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.[RFC5888]Camarillo, G. and H. Schulzrinne, "The Session Description Protocol (SDP) Grouping Framework", RFC 5888, June 2010.[RFC2818]Rescorla, E., "HTTP Over TLS", RFC 2818, May 2000.[RP015]MIDI Manufacturers Association. "Recommended Practice 015 (RP-015): Response to Reset All Controllers", 11/98.[RFC4288]Freed, N. and J. Klensin, "Media Type Specifications and Registration Procedures", BCP 13, RFC 4288, December 2005.[RFC4855]Casner, S., "Media Type Registration of RTP Payload Formats", RFC 4855, February 2007.
Informative References 🧚
These ones are the extra-credit reading: not strictly required, but SO good for background 🌸
[NMP]Lazzaro, J. and J. Wawrzynek. "A Case for Network Musical Performance", 11th International Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV 2001) June 25-26, 2001, Port Jefferson, New York.[GRAME]Fober, D., Orlarey, Y., and S. Letz. "Real Time Musical Events Streaming over Internet", Proceedings of the International Conference on WEB Delivering of Music 2001, pages 147-154.[RFC3261]Rosenberg, J., Schulzrinne, H., Camarillo, G., Johnston, A., Peterson, J., Sparks, R., Handley, M., and E. Schooler, "SIP: Session Initiation Protocol", RFC 3261, June 2002.[RFC2326]Schulzrinne, H., Rao, A., and R. Lanphier, "Real Time Streaming Protocol (RTSP)", RFC 2326, April 1998.[ALF]Clark, D.D. and D.L. Tennenhouse. "Architectural Considerations for a New Generation of Protocols", SIGCOMM Symposium on Communications Architectures and Protocols, (Philadelphia, Pennsylvania), pp. 200-208, ACM, Sept. 1990.[RFC4695]Lazzaro, J. and J. Wawrzynek, "RTP Payload Format for MIDI", RFC 4695, November 2006.[RFC4696]Lazzaro, J. and J. Wawrzynek, "An Implementation Guide for RTP MIDI", RFC 4696, November 2006.[RFC2205]Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 Functional Specification", RFC 2205, September 1997.[RFC4571]Lazzaro, J., "Framing Real-time Transport Protocol (RTP) and RTP Control Protocol (RTCP) Packets over Connection-Oriented Transport", RFC 4571, July 2006.[SPMIDI]MIDI Manufacturers Association. "Scalable Polyphony MIDI, Specification and Device Profiles", Document Version 1.0a, 2002.[LCP]Apple Computer. "Logic 7 Dedicated Control Surface Support", Appendix B. Product manual available from www.apple.com.
Authors' Addresses 💌
Meet the brains behind it all 🧠✨ Here's how to reach them, exactly as written:
John Lazzaro (corresponding author)
UC Berkeley
CS Division
315 Soda Hall
Berkeley, CA 94720-1776
EMail: lazzaro@cs.berkeley.edu
John Wawrzynek
UC Berkeley
CS Division
631 Soda Hall
Berkeley, CA 94720-1776
EMail: johnw@cs.berkeley.eduAnd that's a wrap on RFC 6295, periodt! 🌈👑💅
📦 where this page comes from
- ietf: rfc6295 2011-06
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms