RTP Payload Format for High Efficiency Video Coding (HEVC)
Extracted elements (27)
PACI cannot be fragmented because losing any non-first fragment would make the PACI control information inaccessible. PACI cannot be aggregated because multiple NAL units would redundantly replicate shared PACI fields; instead, NAL units should be aggregated inside a PACI payload using an AP.
The basic payload header is intentionally limited to the 16-bit NAL unit header to minimize packetization overhead. The PACI mechanism was introduced as an optional, extensible wrapper to carry additional per-packet control information (such as TSCI) only when needed, avoiding overhead for deployments that do not require it.
The payload format deliberately shares its basic design with RFC 6184 (H.264 RTP) and RFC 6190 (scalable H.264 RTP) because RFC 6184 is widely deployed and familiar to the implementer community. Differences are highlighted rather than repeating well-known specifications, reducing the learning curve.
This payload format is designed to be consistent with RFC 6184 (H.264 RTP) and RFC 6190 (scalable H.264 RTP), using the same three basic packet structures (single NAL unit, aggregation, fragmentation). HEVC-specific extensions are the 2-byte NAL unit header format and the new PACI/TSCI mechanism for temporal scalability control.
A non-fragmented NAL unit MUST NOT be transmitted in a single FU (S and E bits must not both be 1 in the same FU header). An FU payload MUST NOT be empty.
An AP MUST carry at least two aggregation units, MUST NOT contain FUs, and MUST NOT be nested (an AP must not contain another AP). The total data MUST fit into an IP packet, and size SHOULD be chosen to remain below MTU to avoid IP-layer fragmentation.
Fragments of the same NAL unit MUST be sent in consecutive ascending RTP sequence number order with no intervening packets from the same RTP stream. APs MUST NOT be fragmented; FUs MUST NOT be nested.
If an FU is lost, the receiver SHOULD discard all following fragmentation units in transmission order for the same fragmented NAL unit, unless the decoder is known to gracefully handle incomplete NAL units. A receiver or MANE MAY aggregate the first n-1 fragments with the forbidden_zero_bit set to 1 to indicate a syntax violation.
PACI F1 and F2 bits MUST be 0 in this version; receivers MUST ignore F1=1 and/or F2=1 and any associated PHES data. The Y bit MUST be 0 and receivers MUST ignore Y=1 and any associated PHES data.
PACIs MUST NOT be fragmented (an FU must not carry a PACI) and MUST NOT be aggregated (an AP must not contain a PACI). The PACI payload MAY be a single NAL unit, an FU, or an AP.
Receivers MUST support all three transmission modes: SRST, MRST, and MRMT. When tx-mode equals 'SRST', SRST MUST be used; when 'MRST', MRST MUST be used; otherwise MRMT MUST be used.
Receivers MUST use the RTP timestamp for the display process, even when the bitstream contains picture timing SEI messages or decoding unit information SEI messages as specified in HEVC.
SRST or MRST SHOULD be used for point-to-point unicast scenarios, whereas MRMT SHOULD be used for point-to-multipoint multicast scenarios where different receivers require different operation points of the same HEVC bitstream.
The DONL field MUST be present in single NAL unit packets and the first aggregation unit of an AP if sprop-max-don-diff > 0 for any RTP stream; otherwise it MUST NOT be present. The DOND field is governed by the same condition for non-first aggregation units in an AP.
The RTP marker bit (M) MUST be set for the last packet of the access unit in the current RTP stream. In MRST or MRMT, if an access unit appears in multiple RTP streams, the marker bit is set on each stream's last packet of that access unit.
The RTP timestamp MUST use a 90 kHz clock rate. For NAL units with no timing properties of their own (e.g., parameter sets and SEI NAL units), the RTP timestamp MUST be set to the RTP timestamp of the coded picture of the access unit in which the NAL unit is included.
Four RTP payload structures are defined, distinguished by the Type field in the payload header: single NAL unit packet (Type 1–47), Aggregation Packet (AP, Type=48), Fragmentation Unit (FU, Type=49), and PACI carrying RTP packet (Type=50). A receiver identifies the structure type from the Type field.
The Decoding Order Number (DON) mechanism uses a 16-bit DONL field in the first packet of an interleaved sequence and an 8-bit DOND difference field in subsequent AP units to allow receivers to reconstruct NAL unit decoding order when it differs from transmission order (interleaved mode). This is activated by the SDP parameter sprop-max-don-diff.
Three transmission modes are defined: SRST (single RTP stream on single transport), MRST (multiple RTP streams on single transport), and MRMT (multiple RTP streams on multiple transports). In MRST/MRMT, different SSRCs identify each sub-layer's stream, and receivers must correctly associate SSRCs belonging to the same bitstream.
IANA registered the media subtype 'video/H265' for HEVC RTP payload. The registration defines SDP parameters including tx-mode, sprop-vps, sprop-sps, sprop-pps, sprop-sei, sprop-max-don-diff, sprop-depack-buf-nalus, sprop-depack-buf-bytes, and others governing session negotiation and stream configuration.
The HEVC RTP payload inherits the security considerations of RFC 6184. Confidentiality and integrity of the media stream should be provided using SRTP. MANEs that drop or modify RTP packets must rewrite RTCP packets to match as specified in RFC 3550 Section 7, and must be trusted entities within SRTP deployments.
A Fragmentation Unit (FU, Type=49) contains a 2-byte PayloadHdr, a 1-byte FU header (S bit for start, E bit for end, 6-bit FuType equal to the fragmented NAL unit's Type), an optional 16-bit DONL (present only in first fragment when sprop-max-don-diff > 0), and the FU payload.
A PACI packet (Type=50) contains a 2-byte PayloadHdr, a 16-bit control word with A (1 bit, copy of payload NAL unit F bit), cType (6 bits), PHSsize (5 bits, max 32 octets), F0/F1/F2 (3 bits for extensions), and Y (1 bit for further extension), followed by the variable-length PHES and the PACI payload (NAL unit, FU, or AP, omitting its first 2 bytes).
A single NAL unit packet contains a 2-byte PayloadHdr (copy of the NAL unit header), a conditional 16-bit DONL field (present only when sprop-max-don-diff > 0), and the NAL unit payload data excluding the NAL unit header.
An Aggregation Packet (AP, Type=48) begins with a 2-byte PayloadHdr, followed by two or more aggregation units. The first aggregation unit contains an optional 16-bit DONL, a 16-bit NALU size, and the full NAL unit. Each subsequent unit contains an optional 8-bit DOND, a 16-bit NALU size, and the full NAL unit.
The HEVC NAL unit header is 2 bytes with fields F (1 bit, forbidden_zero_bit, must be 0), Type (6 bits, nal_unit_type), LayerId (6 bits, nuh_layer_id, must be 0 for single-layer), and TID (3 bits, nuh_temporal_id_plus1, value 0 is illegal). This same header structure serves as the RTP payload header for all packet types.
The Temporal Scalability Control Information (TSCI), signaled by F0=1 in PACI, occupies 4 bytes in the PHES: TL0PICIDX (8 bits), IrapPicID (8 bits), S bit (1 bit, start of sub-layer access unit), E bit (1 bit, end of sub-layer access unit), and RES (6 reserved bits).