RTP Payload Format for JPEG 2000 Streaming with Sub-Codestream Latency
Extracted elements (27)
Only ORDH values 4 (PCRL) and 6 (PRCL) support sub-codestream latency streaming because these progression orders emit image data in position-first order, allowing RTP packets to be sent as the top of the frame is encoded before the rest is available. The PRCL order defined in JPEG 2000-2 (T.801) is particularly useful for subframe latency operations.
PTSTAMP provides a 12-bit temporal offset relative to the first RTP packet with the same timestamp, allowing receivers to recover the sender clock more rapidly than is possible using only the 90 kHz RTP timestamp. PTSTAMP is only applicable when consecutive RTP packets with the same timestamp are transmitted within 45 ms of each other.
Sub-codestream latency is achieved by avoiding JPEG 2000 PLM and PLT marker segments, which can only be written after the entire codestream is complete. Instead, the payload header fields ORDH, ORDB, POS, and PID indicate resynchronization points so receivers can restart codestream processing mid-stream without needing those markers.
The extended sequence number (ESEQ field providing high-order bits beyond the standard 16-bit RTP sequence number) is borrowed from RFC 4175 to prevent ambiguity at high data rates. At 1 Gbps with 1000-octet RTP packets, the standard 16-bit sequence number rolls over in 0.5 seconds, which is shorter than typical round-trip times.
This format differs from RFC 5371 (the earlier JPEG 2000 RTP payload format) by not relying on PLM/PLT marker segments, thus supporting sub-codestream latency. It also adds code-block caching, progressive segmented frame (PsF) support, explicit colorspace signaling (PRIMS/TRANS/MAT/RANGE), and an extended sequence number.
A single RTP packet payload MUST NOT contain bytes from more than one JPEG 2000 codestream. The Body Packet payload MUST NOT contain any bytes of the codestream Extended Header; the Main Packet payload MUST contain at least one byte of the Extended Header.
If P=1, the transmission time offset TOFF for two consecutive RTP packets with identical timestamp fields MUST NOT differ by more than 4095 (approximately 45 ms at 90 kHz). The sender SHOULD set P=1 but only when it can generate PTSTAMP accurately.
ORDH MUST be 0 if the codestream consists of more than one tile; similarly, ORDB MUST be 0 for multi-tile codestreams. Only ORDH values 4 (PCRL) and 6 (PRCL) allow sub-codestream latency streaming.
The SDP mapping of the video/jpeg2000-scl media type and its parameters MUST be done according to Section 3 of RFC 4855. The media type is restricted to RTP framing as specified in RFC 3550.
The sender MUST emit either a single Main Packet with MH=3 (sole Main Packet for the codestream), or one or more Main Packets with MH=1 followed by exactly one Main Packet with MH=2. Main Packet Payload Header fields MUST be identical across all Main Packets of a codestream except for MH, ESEQ, and PTSTAMP.
The sender MUST set XTRAC to 0 and MUST set all unassigned (reserved) values to 0. A sender MUST NOT use extension values. Receivers MUST accept XTRAC ≠ 0 and MUST ignore XTRAB; receivers MUST ignore unassigned values; receivers MUST discard any RTP packet containing an extension value.
When C=1, a receiver MUST replace each empty code-block in the codestream with a matching code-block from its cache (if one exists), and MUST update the cache by replacing any matching cached code-block with a non-empty code-block received from the codestream.
When C=1 (code-block caching), R MUST equal 1. When S=0, the fields PRIMS, TRANS, MAT, and RANGE MUST all be zero.
When concatenated, the JPEG 2000 codestream fragments across all RTP packets MUST form a valid sequence of JPEG 2000 codestreams, where successive codestreams MAY be separated by padding bytes. Each padding byte MUST be set to zero by the sender and MUST be ignored by the receiver.
When R=1, all Main Packets in the RTP stream MUST have identical Main Packet Payload Headers (except TP, MH, ESEQ, and PTSTAMP), MUST contain the same codestream main header information (except SOT, COM, and pointer marker segments), and MUST NOT contain non-Extended-Header bytes.
A resync point is the first byte of JPEG 2000 packet header data for a precinct where PID < 2^24. Resync points allow a receiver to begin processing a codestream even if earlier RTP packets were lost, corrupted, or discarded by an intermediate system. Resync points cannot be signaled when the codestream has more than one tile.
Each RTP packet is either a Main Packet (carrying the JPEG 2000 codestream Extended Header plus optional body data) or a Body Packet (carrying only codestream body data). A Main Packet includes a Main Packet Payload Header; a Body Packet includes a Body Packet Payload Header. Both header types share the MH, TP, PTSTAMP, and ESEQ fields.
The JPEG 2000 codestream Extended Header is defined as all bytes from and including the SOC marker through and including the first SOD marker. This is a superset of the JPEG 2000 main header (SOC to first SOT) and is specific to this payload format. Body Packet payloads must not contain any Extended Header bytes.
The QUAL field (3 bits in the Body Packet Payload Header) identifies the lowest quality layer present in the payload: QUAL=0 means the payload can contribute to all quality layers; otherwise the payload contributes only to quality layer index QUAL or above. This allows intermediate systems or receivers to discard packets by quality layer without parsing the codestream.
When C=1 (code-block caching enabled), the sender improves bandwidth efficiency by transmitting empty code-blocks for static image regions; the receiver maintains a cache of previously received code-blocks to substitute for empty ones. Two code-blocks match if their spatial coordinates, resolution level, component, sub-band, and parent packet TP field are all identical.
IANA has registered the media type video/jpeg2000-scl as defined in Section 9.2. Optional parameters include: pixel (pixel format URI), sample (sample format URI), width, height, signal (scanning structure URI), caps (capability constraint URI list), and cache (boolean enabling code-block caching).
JPEG 2000's flexible syntax permits image dimensions up to 2^32−1 pixels, which can cause memory structure sizes to vary enormously. Implementations SHOULD take care when processing input that influences memory allocation and SHOULD fail gracefully when resource constraints are exceeded, to avoid denial-of-service from pathological inputs.
This payload format does not exhibit significant non-uniformity in receiver-side computational complexity for RTP packet processing and contains no active content, so it is unlikely to pose a denial-of-service threat from pathological data. Applications SHOULD use one or more appropriate strong security mechanisms (e.g., SRTP) as per RFC 7201.
The Body Packet Payload Header (Figure 3) is exactly 8 octets (two 32-bit words). Word 1 contains: MH (2 bits), TP (3 bits), RES (3 bits), ORDB (1 bit), QUAL (3 bits), PTSTAMP (12 bits), ESEQ (8 bits). Word 2 contains: POS (12 bits) and PID (20 bits). All fields are unsigned binary integers in network order.
The extended sequence number is a 24-bit value computed as: ESEQ_field × 65536 + RTP_sequence_number_field. ESEQ (8 bits) carries the high-order bits in the payload header; the standard 16-bit RTP Fixed Header sequence number carries the low-order bits.
The Main Packet Payload Header (Figure 2) occupies a minimum of 8 octets (two 32-bit words) plus an optional variable-length XTRAB extension. Word 1 contains: MH (2 bits), TP (3 bits), ORDH (3 bits), P (1 bit), XTRAC (3 bits), PTSTAMP (12 bits), ESEQ (8 bits). Word 2 contains: R (1 bit), S (1 bit), C (1 bit), RSVD (4 bits), RANGE (1 bit), PRIMS (8 bits), TRANS (8 bits), MAT (8 bits).
The RTP timestamp clock rate is 90 kHz. For interlaced frames, Field 2's timestamp MUST be offset from Field 1 by half the instantaneous frame period. For progressive segmented frames, both segments MUST carry the same timestamp. All RTP packets of a given image MUST share the same timestamp.