RTP Payload Format for Versatile Video Coding (VVC)
Extracted elements (26)
Conventional CTU-based slices (as in HEVC/AVC) were removed from VVC because slice-based error concealment became practically impossible with modern prediction tools, network conditions have greatly improved, and transport-level error resilience (retransmission, FEC, IRAP insertion) is now the standard approach for low-delay error resilience applications.
The Z bit (nuh_reserved_zero_bit) is not overloaded for local extensions: overloading the F bit is sufficient for local signaling, and preserving Z maintains compatibility with possible future versions of the VVC specification that may assign semantics to this reserved bit.
This payload format consciously shares its basic structure with AVC (RFC 6184), SVC (RFC 6190), and HEVC (RFC 7798) payload formats so that implementers already familiar with those widely deployed formats can apply existing knowledge, keeping implementation complexity and learning curve low.
When an FU is lost, the receiver SHOULD discard all subsequent fragmentation units for the same fragmented NAL unit in transmission order, unless the decoder is known to handle incomplete NAL units gracefully. A receiver MAY instead aggregate the first n-1 fragments into an incomplete NAL unit and set the forbidden_zero_bit to 1 to signal a syntax violation.
A receiver MUST ignore any parameter unspecified in the media type registration. When profile-id is not present in an SDP description, a value of 1 (Main 10 profile) MUST be inferred.
An AP MUST NOT contain NAL units from more than one access unit, MUST carry at least two aggregation units, MUST NOT contain FUs, and MUST NOT be nested (an AP cannot contain another AP). The AP Type field MUST be 28; LayerId MUST equal the lowest LayerId of all aggregated NAL units; TID MUST equal the lowest TID of all aggregated NAL units.
During de-packetization, NAL-unit-like structures with NAL unit type values 28 to 31, inclusive (the AP and FU type identifiers), MUST NOT be passed to the decoder; only values 0 to 27 may be passed.
Fragments of the same NAL unit in FUs MUST be sent in consecutive order with ascending RTP sequence numbers, with no other RTP packets of the same stream interleaved. APs MUST NOT be fragmented; FUs MUST NOT be nested. The FU payload header F, LayerId, and TID MUST equal those of the fragmented NAL unit; FuType MUST equal the NAL unit Type.
If sprop-max-don-diff > 0, the DONL field MUST be present in single NAL unit packets and in the first aggregation unit of an AP, with DON derived directly from DONL; subsequent AP aggregation units derive DON as preceding DON plus 1 modulo 65536. If sprop-max-don-diff = 0, the DONL field MUST NOT be present.
In an FU header, the S (start) and E (end) bits MUST NOT both be set to 1 in the same FU, because a non-fragmented NAL unit must not be transmitted as a single FU. The FU payload MUST NOT be empty.
In SDP offer/answer when the answer does not include recv-ols-id selecting a sub-OLS, the parameters tier-flag, profile-id, sub-profile-id, and interop-constraints MUST be used symmetrically: the value in the offer MUST be the same as in the answer, either explicitly or implicitly.
Receivers MUST use the RTP timestamp for the display process, even when the bitstream contains picture timing SEI messages or decoding unit information SEI messages as specified in VVC.
The RTP Marker bit (M) MUST be set for the last packet in transmission order among each set of packets containing NAL units of one access unit, consistent with the standard use of M in video formats for efficient playout buffer handling.
The RTP timestamp MUST use a 90 kHz clock rate. For NAL units without their own timing (e.g., parameter sets, SEI NAL units), the RTP timestamp MUST be set to the RTP timestamp of the coded pictures of the access unit in which the NAL unit is included.
When sprop-max-don-diff = 0, the transmission order of NAL units in the RTP stream MUST be the same as the NAL unit decoding order. When sprop-max-don-diff > 0, transmission order MAY differ from decoding order, and the DON mechanism is used to reconstruct decoding order at the receiver.
The Decoding Order Number (DON) mechanism uses a 16-bit DONL field and derives an absolute decoding order number AbsDon per NAL unit using modular arithmetic on the 16-bit DON values. AbsDon enables receivers to reorder NAL units from transmission order to decoding order when sprop-max-don-diff > 0.
The sprop-max-don-diff parameter specifies the maximum absolute difference between AbsDon values of any two consecutive NAL units in decoding order within the stream. When 0, no de-packetization buffer is needed; when > 0, its value determines the required de-packetization buffer depth.
The TID field in the RTP payload header indicates relative packet importance: a lower TID value (higher temporal sublayer importance) means the NAL units are needed for decoding lower sublayers. MANEs can use TID to preferentially protect or forward higher-importance packets; more important NAL units MAY be better protected against transmission losses.
Three payload structure types are defined: Single NAL Unit Packets (exactly one NAL unit, Type 0-27), Aggregation Packets (AP, Type=28, multiple NAL units from one access unit), and Fragmentation Units (FU, Type=29, subsets of a single NAL unit). The receiver identifies the type via the Type field in the 2-byte payload header.
This document registers the 'video/H266' media subtype with IANA for RTP transport of VVC. Optional parameters include profile-id, tier-flag, sub-profile-id, interop-constraints, level-id, sprop-sublayer-id, sprop-ols-id, recv-sublayer-id, recv-ols-id, max-recv-level-id, sprop-dci, sprop-vps, sprop-sps, sprop-pps, sprop-sei, max-lsr, max-fps, sprop-max-don-diff, sprop-depack-buf-bytes, and depack-buf-cap.
The VVC RTP payload format shares the security design philosophy of AVC (RFC 6184) and HEVC (RFC 7798) payload formats. Implementers must guard against malformed bitstreams that could exploit decoder vulnerabilities. MANEs that operate on SRTP streams must be trusted parties; after dropping packets, they must rewrite RTCP to reflect the changes.
The de-packetization buffer operates in two states: initial buffering (active from reception initialization) and buffering-while-playing. The transition occurs when the difference between the greatest and smallest AbsDon in the buffer reaches or exceeds sprop-max-don-diff; subsequently, NAL units with the smallest AbsDon are released to the decoder whenever the difference again meets or exceeds that threshold.
A Fragmentation Unit (FU) consists of a 2-byte PayloadHdr (Type=29), a 1-byte FU header with fields S (1-bit start), E (1-bit end), P (1-bit last-VCL-NAL indicator), and FuType (5 bits), followed by a conditional 16-bit DONL field (present in first FU when sprop-max-don-diff > 0), and the FU payload.
A Single NAL Unit Packet consists of a 2-byte PayloadHdr (same field layout as the NAL unit header), a conditional 16-bit DONL field in network byte order (present only when sprop-max-don-diff > 0), followed by the NAL unit payload data (the NAL unit excluding its header).
An Aggregation Packet (AP) begins with a 2-byte PayloadHdr (Type=28), followed by two or more aggregation units. Each aggregation unit contains an optional conditional 16-bit DONL, a 16-bit unsigned NALU size in network byte order (not counting those 2 octets but including the NAL unit header), and the NAL unit including its header.
The VVC NAL unit header is 2 bytes: F (1 bit, forbidden_zero_bit), Z (1 bit, reserved), LayerId (6 bits, nuh_layer_id), Type (5 bits, nal_unit_type), and TID (3 bits, nuh_temporal_id_plus1). TID=0 is illegal to ensure at least one header bit is 1.