RTP Payload Format for VP8 Video
Extracted elements (30)
The PictureID field was included primarily to enable use of RPSI and SLI feedback messages. Combined with an alternating leapfrog update pattern for golden and altref frames, it enables efficient error recovery without requiring retransmission of the full reference frame.
The sender may choose either 7-bit or 15-bit PictureID and may switch from 7-bit to 15-bit after wrap rather than resetting; the receiver must not assume a fixed size. This design allows flexibility for both short-lived and long-running sessions without pre-negotiation of PictureID width.
VP8 frames are divided into partitions (mode/motion vectors in the first partition, DCT/WHT coefficients in remaining partitions) to enable loss robustness: the first partition is decodable alone, and subsequent partitions provide value even with partial frame loss. This motivated the PID, S, and 1st-partition-size fields.
Implementations that splice VP8 streams must ensure the incrementing rules for TL0PICIDX and KEYIDX are obeyed across the splice point, which will likely require rewriting these field values after the splice.
A receiver SHOULD NOT decode an interframe if it has not received and decoded a key frame with the same KEYIDX after the last KEYIDX wraparound.
Congestion control for RTP SHALL be used in accordance with RFC 3550 and applicable RTP profiles. Media-aware network elements MAY use the VP8 payload descriptor to identify non-reference frames and discard them to reduce congestion, but only when the stream is unencrypted.
KEYIDX MUST wrap to 0 after reaching the maximum value of 31. The sender MUST increment KEYIDX for key frames that convey parameter updates critical to subsequent frames, and SHOULD leave it unchanged for non-critical key frames.
The first VP8 partition (containing modes and motion vectors) MUST be labeled with PID=0. PID SHOULD be incremented by 1 for each subsequent partition, but MAY be kept at 0 for all packets. PID cannot be larger than 7.
The payload of an RTP packet MUST contain data from only one video frame. Partitions MUST be aggregated in decoding order. The last (or only) packet carrying data for a frame MUST have the M bit set.
The PictureID MUST increase by 1 for each subsequent frame and MUST wrap to 0 after reaching the maximum ID (all bits set). It MAY start at a random value. The receiver MUST NOT assume the number of bits in PictureID stays the same through the session.
The receiver MUST ignore any fmtp parameter unspecified in this memo. If the implementation is willing to receive media, both max-fr and max-fs parameters MUST be provided and MUST NOT be used for any purpose other than signaling receiver capabilities.
The RTP marker bit (M) MUST be set for the very last packet of each encoded VP8 frame, enabling a decoder to finish decoding without waiting for the next packet to confirm frame completion.
The RTP timestamp for VP8 uses a 90 kHz clock granularity, representing the time when the frame was sampled. A delta of 1 represents 1/90,000 of a second.
The S (start of partition) bit MUST be set to 1 for the first packet of each encoded frame. It SHOULD be set to 1 when the first payload octet begins a new VP8 partition, and MUST NOT be 1 otherwise.
The SDP clock rate in the a=rtpmap line MUST be 90000. The media name MUST be video, and the encoding name MUST be VP8. The max-fs and max-fr parameters MUST be included in the a=fmtp line when SDP is used to declare receiver capabilities.
The Y (layer sync) bit MUST be set to 0 if the current frame depends on any frame other than the base layer (TID=0) frame with the matching TL0PICIDX. It SHOULD be set to 1 only when the frame depends solely on that base layer frame.
TL0PICIDX MUST be incremented when TID is 0 (base layer frames), and MUST wrap to 0 after reaching 255. It is RECOMMENDED that TID be used whenever TL0PICIDX is used.
Usage of RTP/AVPF (RFC 4585) is RECOMMENDED for handling network losses. The sender SHOULD packetize each VP8 data partition in a separate packet or packets for improved loss robustness.
When the X bit in the payload descriptor is set to 1, the extension octet MUST be provided immediately after the mandatory first octet. If X is 0, all optional fields (I, L, T, K) MUST be omitted.
RPSI (Reference Picture Selection Indication) feedback for VP8 uses the PictureID as the native RPSI bit string. It enables positive acknowledgment of correctly decoded golden or altref frames for improved error recovery in unicast scenarios.
SLI (Slice Loss Indication) for VP8 uses the six least significant bits of the current frame's PictureID in the PictureID field of the SLI message. If the current frame has no PictureID, the receiver MAY send the last received PictureID+1.
The N (non-reference frame) bit in the VP8 payload descriptor indicates that when set to 1, the frame can be discarded without affecting any other future or past frames. If reference status is unknown, N SHOULD be set to 0.
IANA registered the media type video/VP8 as described in Section 6.1, with optional parameters max-fr (maximum frame rate in frames/second) and max-fs (maximum frame size in macroblocks). The clock rate is fixed at 90000 Hz.
Discarding of non-reference frames based on the N bit in the VP8 payload descriptor cannot be done when the stream is encrypted, because the non-reference marker is encrypted along with the payload.
VP8 RTP packets are subject to the security considerations of RFC 3550 and applicable RTP profiles (RTP/AVP, RTP/AVPF, RTP/SAVP, RTP/SAVPF). Applications SHOULD use one or more appropriate strong security mechanisms for confidentiality, integrity, and source authenticity.
The PictureID extension field uses either 7 bits (M=0) or 15 bits (M=1) to carry a running frame index. The M bit in the first octet of the PictureID field signals which size is in use; receivers MUST NOT assume the size stays constant through the session.
The TID/Y/KEYIDX octet contains a 2-bit temporal-layer index (TID), a 1-bit layer sync bit (Y), and a 5-bit temporal key frame index (KEYIDX, wrapping at 31). This octet MUST be present when either T or K is set to 1 in the extension control bits.
The TL0PICIDX field is 8 bits and carries a running index for temporal base layer (TID=0) frames, wrapping to 0 after 255. It indicates which base-layer frame a higher-layer frame depends on.
The VP8 payload descriptor is the first octets after the RTP header. The mandatory first octet contains X (extended control bits present), R (reserved), N (non-reference frame), S (start of VP8 partition), R (reserved), and PID (3-bit partition index). Optional extension octets follow when X=1.
The VP8 payload header (uncompressed data chunk from RFC 6386 Section 9.1) appears only in the first packet of a frame (S=1, PID=0). It is 3 octets for interframes and 10 octets for key frames; the P bit (bit 0) is the inverse key frame flag (0=key frame, 1=interframe).