Codec Control Messages in the RTP Audio-Visual Profile with Feedback (AVPF)
updated by
Extracted elements (28)
A two-way handshake (request + notification) is introduced only for messages where acknowledgement is required by nature and cannot be derived from the media bit stream. FIR instead uses command repetition with sequence numbers because decoder refresh points are directly identifiable in the stream.
Codec control messages are carried on the media path via RTCP rather than through SIP signaling because MCU media and signaling components are often physically separated, SIP proxies introduce architectural delay not optimized for latency, and these messages have strict real-time constraints that make a SIP re-INVITE (requiring at least two additional round trips) impractical.
FMT=2 in the RTPFB space is reserved rather than reused because early AVPF implementations may have already deployed this value per an expired draft. Sufficient numbering space exists to avoid reassignment and the potential interoperability problems it would cause.
TMMBR expresses limits as a (max total media bit rate, per-packet overhead) tuple rather than net media bit rate alone because endpoints observe overhead at different protocol layers. Calculations carried out in terms of packet rate and net media rate have the same value at both sender and receiver, making the algorithm layer-agnostic.
H.271 message-type equivalence table: type 0 (reference picture positive ACK) maps loosely to AVPF RPSI but is not multicast-safe; types 1 (picture loss) and 2 (partial loss) map to PLI and SLI respectively; type 5 (refresh point) maps to CCM FIR. Types 3 and 4 (CRC checks) have no AVPF equivalent and are multicast-safe.
In multicast with FIR, a receiver that has already sent a decoder refresh point should await a second FIR request before responding again if the first request arrives within 2×RTT of the previous refresh point delivery, to avoid unnecessary intra frames caused by in-flight requests from participants unaware of the recent refresh.
A FIR sender shall not have more than one outstanding FIR (with a distinct request sequence number) per media sender at any time. When a new FIR is required after a decoder refresh point is received, the sequence number must be incremented.
A media sender that raises its bit rate after issuing a TMMBN must delay increases by at least 2×RTT plus the AVPF-calculated minimum TMMBR wait time, to allow other participants to report lower limits before the sender actually increases throughput.
FIR MUST NOT be used to recover from media errors. The Picture Loss Indication (PLI) message defined in AVPF [RFC4585] is the correct mechanism for reporting lost pictures.
H.271 Video Back Channel Messages MUST NOT be used in multicast environments unless all multicast safety issues are fully understood and accounted for by all endpoints. They MUST NOT be used in centralized multipoint unless both mixer and endpoints handle all known aggregation issues.
Implementations using H.271 feedback messages MUST remain compliant with congestion control principles, and SHOULD use IETF-native AVPF/CCM messages instead of H.271 equivalents where applicable (as documented in the equivalence table).
TMMBR SHALL NOT be used as a substitute for congestion control. It conveys known local limitations and provides no guarantees about the full path, nor the continuous feedback loop required for real congestion control.
Full Intra Request (FIR) is a command sent to a media sender requiring delivery of a Decoder Refresh Point at the earliest opportunity. It is used by mixers when switching sources or adding new video sources, and must not be used for error recovery (PLI serves that purpose).
H.271 Video Back Channel Message (VBCM) transparently conveys ITU-T Rec. H.271 video back channel payloads from media receiver to media sender. Usage in multicast or centralized multipoint is restricted due to non-multicast-safe H.271 code points and the difficulty of correct state aggregation.
Temporal-Spatial Trade-off Notification (TSTN) is the mandatory response to a TSTR, conveying the encoder's chosen trade-off index. It provides an explicit acknowledgement because trade-off changes cannot be directly identified from the media bit stream.
Temporal-Spatial Trade-off Request (TSTR) instructs a video encoder to adjust its trade-off between temporal and spatial resolution. Index values 0–31 indicate monotonically increasing preference for higher frame rate: 0 = highest spatial quality at low frame rate, 31 = highest frame rate at potentially low spatial quality.
Temporary Maximum Media Stream Bit Rate Notification (TMMBN) is sent by a media sender to report the current bounding set of TMMBR-defined limits and identify the owners of each tuple. Non-owning receivers use this to suppress redundant TMMBR messages.
Temporary Maximum Media Stream Bit Rate Request (TMMBR, pronounced 'timber') allows a receiver, translator, or mixer to request a sender to limit the maximum total media bit rate for a given stream. Each request is expressed as a tuple of (maximum total media bit rate, measured per-packet overhead), enabling multi-receiver constraint aggregation via a bounding-set algorithm.
The bounding-set algorithm selects the minimal subset of TMMBR tuples that define the feasible region for a media sender. Each tuple is modeled as a straight line in (packet rate, net media bit rate) space; the lower envelope of these lines forms a polygon whose interior is the feasible region. The algorithm sorts tuples by overhead and iteratively builds the bounding set.
Payload-Specific Feedback Message (PSFB, PT=206) FMT values assigned: FMT=4 for FIR, FMT=5 for TSTR, FMT=6 for TSTN, FMT=7 for VBCM. Registered with IANA under the AVPF feedback message namespace.
SDP rtcp-fb attribute extended with a new feedback type value 'ccm' (Codec Control Messages), with sub-parameters 'fir', 'tmmbr', 'tstr', and 'vbcm' enabling per-session negotiation of each message type via SDP offer/answer.
Transport Layer Feedback Message (RTPFB, PT=205) FMT assignments added by this memo: FMT=2 reserved (legacy early-AVPF conflict avoidance), FMT=3 assigned to TMMBR, FMT=4 assigned to TMMBN. FMT=0 and FMT=5–30 remain available.
An attacker able to inject RTCP packets could send spurious FIR commands forcing excessive bandwidth-intensive intra frames, fabricate TMMBR messages throttling legitimate streams below their actual path capacity, or inject VBCM payloads to manipulate encoder reference-picture state. The document recommends SRTP/SRTCP or equivalent lower-layer security to authenticate RTCP feedback messages.
TMMBR owner/non-owner states: a receiver is an 'owner' if its (bit rate, overhead) tuple appears in the media sender's TMMBN bounding set, and a 'non-owner' otherwise. Owners send TMMBR when their values change; non-owners apply the bounding-set algorithm and send TMMBR only if their tuple would enter the bounding set. A receiver checks each TMMBN to determine its current status.
FIR FCI entry (64 bits): SSRC (32 bits) of the requested media sender; Seq nr (8 bits) for deduplication; 24 reserved bits. Multiple FCI entries may appear in one FIR message to simultaneously request decoder refreshes from multiple senders.
TMMBR FCI entry (64 bits): SSRC (32 bits) identifying the target media sender; MxTBR Exp (6 bits) and MxTBR Mantissa (17 bits) encoding maximum total media bit rate as a floating-point value; Measured Overhead (9 bits) for average per-packet overhead in bytes.
TSTR/TSTN FCI entry (64 bits): SSRC (32 bits) of the media sender; Seq nr (8 bits); 19 reserved bits; Index (5 bits, value 0–31) for the requested or chosen temporal/spatial trade-off. TSTN uses the identical format with the Index reflecting the encoder's actual decision.
VBCM FCI entry (variable length): SSRC (32 bits) of the media sender; Seq nr (8 bits); 8 reserved bits; PayloadType (7 bits); 1 padding bit; Length (16 bits); variable-length VBCM Octet String containing the H.271 payload.