RTP Payload Format for JPEG-compressed Video
obsoletes
- rfc-2035 — RTP Payload Format for JPEG-compressed Video
Extracted elements (27)
Frame-level fragmentation is handled at the RTP layer rather than the IP layer to enable rate control, partial delivery on loss, and to avoid exceeding maximum network reassembly sizes, as motivated by 'Fragmentation Considered Harmful'.
RTP/JPEG uses abbreviated JPEG format (omitting table headers from the bit stream) because table-specification data rarely changes from frame to frame, reducing overhead. An explicit RTP/JPEG header carries the information that would otherwise appear in JPEG frame and scan headers.
The Fragment Offset field is included in the RTP/JPEG header (in addition to RTP sequence numbers) to enable efficient single-copy reassembly in the presence of misordered packets, and to detect a complete frame even if the previous frame's last packet (with the marker bit) was dropped.
The Restart Marker header was moved from the Type field (as in RFC 2035) to a dedicated bit in the Type field plus an optional header, so the type-specific field is free for interlace and other type-specific purposes and new types can be added without encoding restart status in the type number.
Type numbers 2–5 are reserved and SHOULD NOT be used. Applications based on RFC 2035 should be updated to use type 64 or 65 (with Restart Marker header) instead of the older restart encoding conventions.
All fields in the main JPEG header except for the Fragment Offset field MUST remain the same in all packets that correspond to the same JPEG frame.
All packets in a frame MUST have the same RTP timestamp, the RTP marker bit MUST be set on the last packet, and packet fragments MUST NOT overlap one another.
All packets MUST contain a 'stuffed' 0x00 byte following any true 0xFF byte generated by the entropy coder. Restart markers MUST NOT appear anywhere other than on MCU boundaries at the defined interval.
For Q values 128–254, the Q value to quantization table mapping MUST be static; a Q value of 255 denotes a dynamic mapping and decoders MUST NOT depend on any previous version of those tables. Packets MUST NOT contain Q=255 and Length=0.
For Q values 128–255, the Quantization Table header MUST be present in the first packet of every frame. When tables are specified out-of-band, the Length field may be set to 0 to omit the table data inline.
Frames SHOULD be fragmented into packets to avoid lower-layer fragmentation, and SHOULD be fragmented so each packet contains an integral number of restart intervals to support partial frame decoding.
If the Length field in a received Quantization Table header is larger than the remaining number of bytes in the packet, the packet MUST be discarded.
The RTP marker bit MUST be set in the last packet of a frame.
The RTP timestamp for JPEG payload MUST be in units of 90000Hz, and the same timestamp MUST appear in each fragment of a given frame.
When restart intervals are not guaranteed to align with packet boundaries, the F and L bits MUST be set to 1 and Restart Count MUST be set to 0x3FFF, indicating the receiver MUST reassemble the entire frame before decoding.
For Q values 1–99, quantization tables are computed using the IJG formula: S = 5000/Q for Q<=50, or S = 200-2*Q for Q 51–99, applied to scale JPEG standard Tables K.1 and K.2, with values saturated to 8 bits.
The Fragment Offset field is a 24-bit value (network byte order) giving the byte offset of the current packet's payload within the JPEG frame. The Fragment Offset plus payload length MUST NOT exceed 2^24 bytes.
The Q field (8 bits) specifies the quantization tables: values 0–127 use an algorithm determined by the Type field; values 128–255 indicate a Quantization Table header is present in the first packet of the frame.
The Type field defines the abbreviated table-specification and JFIF-style parameters. Types 0–63 are fixed well-known mappings; types 64–127 are the same but indicate restart markers are present; types 128–255 are dynamically defined by a session setup protocol.
The Type-specific field encodes interlace information for types 0 and 1: 0=progressive, 1=odd field, 2=even field, 3=single field displayed full-frame (each line doubled).
Types 0 and 1 define two YUV color space modes: type 0 uses 4:2:2 chrominance subsampling (horizontal only), type 1 uses 4:2:0 (horizontal and vertical). Both use Baseline DCT sequential mode, 8-bit samples, standard Huffman tables, and a single interleaved scan.
Width and Height fields each encode image dimensions in 8-pixel multiples (e.g., width=40 means 320 pixels wide), with a maximum of 2040 pixels in each dimension. For interlaced video, Height encodes the height of a single field.
A denial-of-service threat exists around the fragmentation mechanism: receivers should limit the total amount of data buffered for frame reassembly to avoid resource exhaustion from maliciously crafted or excessive fragments.
Confidentiality of RTP/JPEG media streams is achieved by encryption applied after compression (end-to-end), so there is no conflict between compression and encryption. General RTP security considerations from RFC 1889 apply.
The main JPEG header is 8 bytes: Type-specific (8 bits), Fragment Offset (24 bits, network byte order), Type (8 bits), Q (8 bits), Width (8 bits, in 8-pixel multiples), Height (8 bits, in 8-pixel multiples).
The Quantization Table header consists of MBZ (8 bits), Precision (8 bits), and Length (16 bits), followed by quantization table data. Each table is 64 values in zig-zag order; the Precision field indicates 8-bit (0) or 16-bit (1) coefficients per table.
The Restart Marker header (4 bytes) appears immediately after the main JPEG header for types 64–127: Restart Interval (16 bits), F bit (1), L bit (1), Restart Count (14 bits). The Restart Interval MUST NOT be zero.