RTP Payload Format for VP9 Video
Extracted elements (29)
The non-flexible mode's Picture Group (PG) pre-declares the full inter-picture dependency structure in the Scalability Structure, avoiding per-packet reference overhead for streams with a fixed, recurring dependency pattern. Flexible mode trades this compactness for dynamic, per-packet control of temporal-layer hierarchies.
The Z bit allows a receiver targeting a higher spatial layer to discard a lower-layer packet without processing it and without waiting for the D bit in the higher-layer frame, providing an early-discard signal that reduces unnecessary decode work in selective forwarding scenarios.
VP9 treats quality layers (same resolution, different quality) as spatial layers without resolution changes. This unifies the spatial/quality layer concept, simplifying the payload descriptor by using a single SID field for both.
Media-aware network elements MAY use the VP9 payload descriptor to identify non-reference frames and discard them to reduce congestion, but this is not possible when the stream is encrypted because the non-reference marker is then encrypted.
The receiver MUST NOT assume that the number of bits in the PID (7 or 15) stays the same through the session. Transitions are handled by zero-extension (7→15 bits) or truncation (15→7 bits), and the switch may only occur on the first packet of a key picture.
All frames of the same picture MUST have the same PID value. The PID SHOULD start on a random number and MUST wrap after reaching the maximum (0x7f for 7-bit, 0x7fff for 15-bit). The receiver MUST NOT assume the number of bits in the PID remains constant throughout the session.
Applications using the VP9 RTP payload profile MUST assign a dynamic payload type number and provide a mechanism to indicate the mapping, per the policy in RFC 3551 Section 3.
For scalable streams, a VP9 frame that follows a frame that might be removed (in full decode order) MUST be encoded with 'error_resilient_mode' set to true, to prevent dependency on frames that could legitimately be absent.
In Offer/Answer negotiation, profile-id MUST be used symmetrically: the answerer MUST either maintain the profile-id or remove the media format entirely. An answer MUST NOT contain the payload type number from the offer unless profile-id is exactly the same.
In SDP, the media name MUST be 'video', the encoding name MUST be 'VP9', and the clock rate MUST be 90000. If profile-id is absent, a value of 0 (Profile 0) MUST be inferred.
Layers MUST be encoded such that if any layer and all higher layers are removed from the bitstream along either the spatial or temporal dimension, the remaining bitstream is still correctly decodable.
The B bit MUST be set to one if the first payload octet is the beginning of a new VP9 frame; otherwise it MUST NOT be one. The E bit MUST be set to one for the final RTP packet of a VP9 frame; otherwise it is zero.
The D bit MUST be set to one if and only if the current spatial-layer SID frame depends on the spatial-layer SID-1 frame of the same picture; otherwise it MUST be zero. For the base-layer frame (SID=0), D MUST be zero.
The F bit (flexible mode) MUST only change on the first packet of a key picture — a picture whose base spatial-layer frame is a keyframe (P=0, SID=0 or L=0, B=1). The F bit also MUST only be set to one if the I bit is also set to one.
The max-fr and max-fs SDP parameters MUST both be provided if the implementation is willing to receive media. A sender SHOULD NOT send media exceeding the receiver's signaled max-fr and max-fs values, though MAY do so in scenarios such as pre-encoded media or selective forwarding.
The RTP Marker bit (M) MUST be set to one for the final packet of the highest spatial-layer frame (the final packet of the picture); otherwise it is zero. If a stream is rewritten to remove higher spatial layers, M MUST be set to one for the target spatial-layer frame.
The RTP timestamp MUST use a clock rate of 90 kHz. All frames of the same picture MUST have the same timestamp. Every picture containing a frame with show_frame==1 MUST have a unique timestamp modulo 2^32.
To handle network losses, usage of RTP/AVPF (RFC 4585) is RECOMMENDED. All integer fields in the specification are encoded as unsigned integers in network octet order.
Upon receipt of a Full Intra Request (FIR), a VP9 sender MUST send a picture with a keyframe for its spatial-layer 0 frame and then send frames without inter-picture prediction (P=0) for any higher-layer frames.
When P and F are both set to one (non-keyframe in flexible mode), at least one reference index (P_DIFF) MUST be specified. A P_DIFF value of zero is invalid. Up to three total reference indices may be chained using the N bit.
Two dependency-signaling modes are defined: flexible mode (F=1), where each packet carries up to three explicit P_DIFF reference indices relative to the current PID; and non-flexible mode (F=0), where dependencies are pre-declared in the Scalability Structure's Picture Group (PG) and indexed by PID modulo N_G.
IANA registered the media type 'video/vp9' with clock rate 90000 Hz, and added it to the 'RTP Payload Format Media Types' registry under the 'Real-Time Transport Protocol (RTP) Parameters' registry group.
Applications using this payload format SHOULD use one or more appropriate strong security mechanisms (e.g., SRTP). Responsibility for confidentiality, integrity, and source authenticity lies with the application, not with the payload format.
Decoders MUST be robust against malicious or malformed VP9 payloads; a memory overrun in the decoder could lead to arbitrary code execution. The same robustness applies to encoders, though encoder vulnerabilities are typically rarer.
Layer indices occupy one octet (flexible mode) or two octets (non-flexible mode). The octet encodes TID (3 bits, temporal-layer ID), U (1 bit, switching up point), SID (3 bits, spatial-layer ID), and D (1 bit, inter-layer dependency used). Non-flexible mode adds an 8-bit TL0PICIDX field.
The Layer Refresh Request (LRR) index for VP9 uses two octets: a 5-bit RES field followed by a 3-bit TID in the first octet, and a 5-bit RES field followed by a 3-bit SID in the second octet. Both RES fields MUST be zero on transmission and ignored on reception.
The mandatory first octet of the VP9 payload descriptor encodes eight 1-bit flags: I (PID present), P (inter-picture predicted), L (layer indices present), F (flexible mode), B (start of frame), E (end of frame), V (scalability structure present), Z (not a reference for upper spatial layers).
The Picture ID (PID) field is 7 or 15 bits depending on the M extension bit in the first PID octet. The PID is present only when the I bit is set. In flexible mode, reference indices (P_DIFF, 7 bits each, up to 3) follow when both P and F are set.
The Scalability Structure (SS) starts with an octet encoding N_S (3 bits, number of spatial layers minus 1), Y (1 bit, resolution present), G (1 bit, PG description present), and 3 reserved bits. When Y=1, each spatial layer has WIDTH and HEIGHT fields (2 octets each). When G=1, N_G (1 octet) precedes per-picture TID/U/R entries with R P_DIFF fields each.