RTP Payload Format for MIDI
obsoletes
- rfc-4695 — RTP Payload Format for MIDI
Extracted elements (31)
The payload format intentionally provides no extension mechanism for new MIDI features beyond encoding MIDI 1.0 DIN cable commands. The design encourages development of entirely new payload formats for specialized uses, relying on IETF codec negotiation tools (RFC 3264, RFC 2326) for backward-compatible deployment of new formats.
The recovery journal uses stream history coding rather than packet retransmission to satisfy the no-indefinite-artifacts mandate. This avoids the latency penalty of retransmission-based schemes, making the format suitable for interactive real-time musical performance where retransmitted packets arrive too late to be useful.
The SysEx cancellation sublist (0xF7 0xF4) allows senders to begin transmitting a SysEx command before the MIDI source has finished sending it, then abort mid-stream if the command must be excluded. Without cancellation, senders would be forced to use high-latency store-and-forward transcoding to validate each SysEx command before transmission begins.
In mpeg4-generic sendrecv sessions, each party's session description MUST use identical values for all mpeg4-generic parameters (streamtype, mode, profile-level-id, and config). This constraint ensures all parties instantiate an identically configured MPEG 4 Audio Object Type for MIDI-to-audio rendering.
A receiver MUST detect each RTP sequence number break. If due to packet loss, the receiver MUST repair all indefinite artifacts in the rendered MIDI performance. A receiver MUST treat the first received packet as ending a loss event, and upon exiting a session MUST ensure no indefinite artifacts remain.
All command timestamps in a packet MUST be less than or equal to the RTP timestamp of the next packet in the stream (modulo 2^32). This rule ensures each RTP MIDI packet is uniquely responsible for encoding a distinct, non-overlapping time interval.
By default, senders MUST use the closed-loop sending policy when creating recovery journals. If the session description overrides this default via the j_update parameter, senders MUST use the specified policy (anchor or open-loop).
For mpeg4-generic streams, exactly one RTP MIDI Access Unit MUST be mapped to one RTP packet. The mpeg4-generic options for placing multiple AUs in a packet, fragmenting AUs, and interleaving AUs MUST NOT be used. The AU Header and Auxiliary sections MUST be empty.
For native streams, the RTP M bit MUST be set to 1 if the MIDI command section has a non-zero LEN field, and MUST be set to 0 otherwise. For mpeg4-generic streams, the M bit MUST be set to 1 for all packets to conform to RFC 3640.
For RTP MIDI streams destined for audio rendering, the clock rate SHOULD be an audio sample rate of 32 KHz or higher (e.g., 32, 44.1, 48, 96 KHz). If the MIDI stream is part of a synchronized session with another RTP audio stream at 32 KHz or higher, the MIDI stream SHOULD match that stream's clock rate.
If several RTP MIDI streams are classified as sending different classes of media (e.g., one for audio and one for lighting control) within a multimedia session, they MUST be sent over different RTP sessions, each with its own media line in the session description.
MIDI System Real-Time commands embedded inside a host MIDI command on a DIN cable MUST be extracted and coded as separate commands in the MIDI list. The host command and the extracted System Real-Time command SHOULD appear in the same MIDI list, with the System Real-Time command timestamped at its original embedded position.
RTP MIDI streams sharing a MIDI name space MUST NOT be sent in the same RTP session; each MUST be sent from a different RTP session. All such streams MUST use the same SSRC value and the same randomly chosen RTP timestamp initialization value, enabling receivers to reconstruct the interleaved command stream.
Standard MIDI File meta-events MUST NOT appear in the MIDI list. Unpaired 0xF7 octets MUST NOT appear in the MIDI list, as they would interfere with SysEx segmentation coding. By default, undefined MIDI system commands (0xF4, 0xF5, 0xF9, 0xFD) MUST NOT appear in a MIDI Command field.
The first MIDI channel command in the MIDI list MUST include a status octet. Running status MAY be used for all subsequent channel commands. All system commands MUST include a status octet. The P (phantom) bit MUST be set to 1 when the first channel command's status octet was not physically present in the source data stream.
The journalling method for a stream is set at the start of a session and MUST NOT be changed thereafter. Streams over unreliable transport default to using the recovery journal; streams over reliable transport default to no journal. If using the recovery journal, every payload MUST include a journal section; otherwise a journal section MUST NOT appear.
The recovery journal mandate states that the MIDI performance rendered from an RTP MIDI stream over unreliable transport MUST NOT contain indefinite artifacts (long-lasting rendering errors such as a sustained note from a lost NoteOff). Senders prepare a journal for every packet; receivers MUST repair indefinite artifacts on loss detection.
This document does not specify a default clock rate for RTP MIDI. When RTP MIDI is used with SDP, parties MUST use the rtpmap attribute line to communicate the clock rate; no default value may be assumed.
Recovery journal channel chapters encode per-channel MIDI state needed for loss recovery: Chapter P (Program Change 0xC), C (Control Change 0xB), M (Parameter System, part of 0xB), W (Pitch Wheel 0xE), N (NoteOff/NoteOn 0x8/0x9), E (Note Extras), T (Channel Aftertouch 0xD), A (Poly Aftertouch 0xA).
Recovery journal system chapters encode system-level MIDI state: Chapter D (Song Select 0xF3, Tune Request 0xF6, Reset 0xFF, undefined commands), V (Active Sense 0xFE), Q (Sequencer State 0xF2/0xF8/0xFA/0xFB/0xFC), F (MTC Tape Position 0xF1), X (System Exclusive). The system journal appears whenever any system chapter is required.
SysEx commands may be encoded verbatim or segmented into first/middle/last/cancel sublists, distinguished by specific head and tail status octets (first: 0xF0…0xF0; middle: 0xF7…0xF0; last: 0xF7…0xF7; cancel: 0xF7…0xF4). Segmentation allows per-octet timing and spanning large SysEx commands across multiple packets.
The MIDI list encodes an ordered sequence of (delta-time, MIDI-command) pairs. When Z=0 the first command has an implicit delta time of zero. Each command timestamp is the cumulative sum (mod 2^32) of the RTP timestamp plus all preceding delta times. The final command field may be empty, enabling time-padding without a command.
This document registers mode 'rtp-midi' for the mpeg4-generic media type, enabling MPEG 4 Audio Object Types (General MIDI ID 15, Wavetable Synthesis ID 14, Main Synthetic ID 13) to transport MIDI commands via RTP with the same payload structure as native streams.
This document registers the audio/asc media type for standalone AudioSpecificConfig data blocks, used as initialization data for mpeg4-generic RTP MIDI streams (SAOL programs, DLS 2 wave tables, etc.).
This document registers the audio/rtp-midi media type for native RTP MIDI streams. It is used in SDP rtpmap attributes to identify the RTP MIDI payload format and carries an optional fmtp attribute for stream configuration parameters.
Section 9 addresses security considerations for RTP MIDI. As an RTP payload format, standard RTP security mechanisms apply; SRTP should be used for confidentiality, integrity, and replay protection. MIDI commands can directly control hardware devices and instruments, making integrity protection particularly important to prevent unauthorized command injection.
A channel journal has a 3-octet header: S bit, 4-bit CHAN field (MIDI channel number), H bit (enhanced Chapter C configured), 10-bit LENGTH field, and an 8-bit Table of Contents with bits P, C, M, W, N, E, T, A indicating presence of each chapter type.
Delta times in the MIDI list use a 1–4 octet variable-length encoding derived from MIDI File format. Each continuation octet has MSB=1; the final octet has MSB=0. The four-byte form decodes to a 28-bit unsigned integer. There are four legal encodings of zero (0x00, 0x8000, 0x808000, 0x80808000).
The MIDI command section begins with a variable-length header containing flags B (big LEN), J (journal present), Z (first delta time present), P (phantom status), plus a LEN field. If B=0, the header is 1 octet and LEN is 4 bits (max 15-octet MIDI list); if B=1, the header is 2 octets and LEN is 12 bits (max 4095-octet MIDI list), coded big-endian.
The recovery journal has a 3-octet header with fields S (single-packet loss hint), Y (system journal present), A (channel journals present), H (enhanced Chapter C in use), TOTCHAN (4-bit count of channel journals minus one), and a 16-bit Checkpoint Packet Seqnum in network byte order.
The system journal has a 2-octet header: S bit, followed by TOC bits D, V, Q, F, X indicating presence of system chapters for simple system commands, Active Sense, Sequencer State, MIDI Time Code, and System Exclusive respectively, plus a 10-bit LENGTH field.