Using Codec Control Messages in the RTP Audio-Visual Profile with Feedback with Layered Codecs
updates
- rfc-5104 — Codec Control Messages in the RTP Audio-Visual Profile with Feedback (AVPF)
Extracted elements (16)
RFC 5104's FIR specification was underspecified for layered codecs when more than one RTP stream is used: it was unclear whether a FIR targeting one stream required a decoder refresh for all layers or only the targeted layer. This document resolves the ambiguity by requiring a full decoder refresh across all layers.
The requirement to react to FIR commands received on enhancement-layer RTCP streams (not just the base layer) is retained for robustness and backward compatibility. While the normative preference is to send FIR targeting the base layer, the receiver must handle FIR on any layer to interoperate with legacy implementations.
The updated decoder refresh point definition was aligned with the corresponding definition used in H.264/SVC (Scalable Video Coding), which is a widely deployed layered codec format. This alignment ensures implementation consistency with existing codec standards rather than introducing a divergent protocol definition.
Congestion control requirements from RFC 5104 Sections 3.5.1 and 5 apply to the forward RTP streams both in isolation and combined when a decoder refresh point is sent in response to a FIR on a layered bitstream. The multi-stream nature of layered transport does not relax these requirements.
For layered codec implementations receiving TSTR messages, it is suggested to be liberal in accepting them and respond with TSTN indicating 'no change'. New implementations should not send TSTR messages except when operating in SRST (Single RTP stream on a Single media Transport) mode, as defined in RFC 7656.
Identifying whether a layered bitstream with inter-layer prediction (pyramid-shaped) is in use is non-trivial. A reliable indicator in SDP environments is that RTP streams are grouped using the SDP Grouping Framework (RFC 5888) combined with layer dependency signaling (RFC 5583).
The RTP payload format negotiated for RTP streams alone is not a reliable indicator of layered bitstream use with inter-layer prediction. H.264/SVC can form decodable sub-bitstreams from enhancement layers only without inter-layer prediction, making blind application of the full-refresh FIR rule potentially suboptimal for such exotic bitstreams.
When a FIR command is received in the RTCP stream associated with any RTP stream carrying any layer of a layered bitstream, the designated media sender MUST send a decoder refresh point at its earliest opportunity.
When sending a FIR command for a layered bitstream, the sender MUST target the RTP stream that carries the base layer by setting the SSRC field in the FCI to the SSRC of the forward RTP stream carrying the base layer.
A Decoder Refresh Point is a bit string, packetized in one or more RTP packets, that completely resets the decoder to a known state. Examples include IDR pictures in H.264/H.265, keyframes in VP8/VP9, and Intra pictures in H.261/MPEG-1/2/4. It also contains all header information above the picture layer conveyed in-band, such as parameter set NAL units in H.264.
Picture Loss Indication (PLI, RFC 4585 Section 6.3.1) requires no specification update for layered codecs. Upon receipt for an enhancement layer, the encoder should repair that layer and all dependent enhancement layers; reference layers do not need repair and the encoder can determine dependent layers autonomously.
Reference Picture Selection Indication (RPSI, RFC 4585 Section 6.3.3) requires no specification update for layered codecs. Reception of RPSI indicating a missing reference picture on any layer forces the encoder to handle that missing reference in the indicated layer and all dependent layers. No implementations with layered codecs are known.
Slice Loss Indication (SLI, RFC 4585 Section 6.3.2) requires no specification update for layered codecs. The prudent response is to repair the affected spatial area of the targeted enhancement layer and all dependent layers; reference layers do not need repair. SLI has seen very little implementation and none known with layered systems.
Temporal-Spatial Trade-Off Request/Notification (TSTR/TSTN, RFC 5104 Sections 4.3.2 and 4.3.3) includes an SSRC target that may refer to any layer's RTP stream, and the semantics apply to layers present in the targeted RTP stream. Per-layer TSTR/TSTN is considered potentially counterproductive in layered codec systems because adjusting spatio/temporal quality balance may involve more than just the addressed stream.
When a layered codec is in use, a Decoder Refresh Point must include hard or gradual single-layer decoder refresh points for all layers, thereby completely resetting the entire decoder — not just one layer — to a known state. This extends the RFC 5104 Section 2.2 definition to cover pyramid-structured layered bitstreams.
The security considerations of AVPF (RFC 4585, as updated by RFC 5506) and Codec Control Messages (RFC 5104) apply to this document in their entirety. The clarified FIR response behavior for layered codecs does not introduce any additional security considerations beyond those already documented in those RFCs.