Framework for Telepresence Multi-Streams
Extracted elements (28)
Each Capture Scene uses its own distinct coordinate system unrelated to other Scenes. This design supports MCUs that aggregate Streams from multiple Endpoints, each with potentially different and unknown physical scales; the 'No Scale' value is intended specifically for this MCU use case where scale can vary per Capture.
Encoding Groups were introduced to express system-level bandwidth constraints across multiple Individual Encodings. This addresses cases where devices cannot deal with multiple Encoding options simultaneously even when simultaneous transmission would otherwise be sensible (e.g., multicast to multiple Endpoints).
Individual Encoding parameters are represented in SDP rather than in CLUE messages, because SDP is the existing negotiation mechanism for codec parameters. Only the bandwidth parameter has special significance in CLUE, as it can be further constrained by the Encoding Group maximum.
The CLUE channel (a separate data channel) is used in addition to SIP/SDP because aspects of telepresence control—such as switching active speaker, layout changes, or changing Capture source within an existing Encoding—change dynamically during a call and should not require heavyweight SIP re-invites.
A CLUE-capable device interoperates with a device that does not support CLUE. The CLUE-capable device determines support by the result of the initial SIP offer/answer exchange; if the other device does not use CLUE, the CLUE-capable device falls back to behavior that does not require CLUE.
The Advertisement and Configure exchange typically occurs during call initiation after CLUE is enabled, but MAY also happen at any time throughout the call. Some mid-call CLUE updates (e.g., switching Capture source) do not require a SIP re-INVITE, while others that change bandwidth allocation do require a new offer/answer exchange.
A Provider MUST be compliant to RFC 6350 when supplying xCard Person Information and MUST supply a VERSION and a FN property. The Provider SHOULD use a URI to point to any LOGO, PHOTO, or SOUND in the xCard rather than embedding the data inline in CLUE messages.
Captures within the same CSV MUST be of the same Media type. The Provider MUST be capable of encoding and sending all Captures (that have an Encoding Group) in a single CSV simultaneously.
If a Provider advertises mutually exclusive Simultaneous Transmission Sets, the Consumer MUST ensure that it chooses Media Captures that lie wholly within one of those Simultaneous Transmission Sets for each Media type.
If an Advertisement does not include Simultaneous Transmission Sets, the Provider MUST be able to simultaneously provide all Captures from any one CSV of each Media type from each Capture Scene, and from any particular Global View if a Global View list is present.
It is a protocol conformance requirement that Encoding Groups MUST allow all the Captures in a particular CSV to be used simultaneously.
Person Type attribute values MUST NOT be repeated within the attribute. A Provider associates a type only when it can do so with some certainty.
Simultaneous Transmission Sets MUST allow all Media Captures in any particular CSV to be used simultaneously, and MUST reflect the simultaneity expressed by any Global View in the Advertisement.
Systems that know their physical dimensions MUST provide real-world measurements (in millimeters) to enable the best user experience. Systems that know relative but not absolute distances MUST use 'Unknown Scale'.
The four points of the Area of Capture attribute MUST be co-planar, forming a quadrilateral that defines the Plane of Interest. For a switched Capture, the Area of Capture MUST use coordinates for the larger potential area.
The Point of Capture of a static Capture MUST NOT move for the life of the CLUE session.
The Provider MUST use the same coordinate system with the same scale and origin for all coordinates within the same Capture Scene. The coordinate system is right-handed Cartesian X, Y, Z.
When the Allow Subset Choice MCC attribute is true and the MCC references other Captures, the Consumer MAY select a specific subset of those Captures in a Configure message, and the Provider MUST then include only that subset. If the attribute is false, the Consumer MUST NOT select a subset.
When two CLUE-capable devices have established a CLUE channel, they MUST adhere to the CLUE protocol suite as outlined in this document and companion documents.
A Capture Scene is a structure representing a spatial region captured by one or more Capture Devices. It contains one or more Capture Scene Views (CSVs), each being a list of Media Captures of the same type that together form one representation of the entire Scene. Spatial relationships exist only within a single Capture Scene.
A Configure message is a CLUE message a Media Consumer sends to a Media Provider specifying which content and Media Streams it wants to receive, based on the information in a corresponding Advertisement message.
A Global View List is an optional list of Global Views in an Advertisement. Each Global View is a set of references to CSVs (of the same Media type) across Scenes, representing a Provider suggestion for one coherent set of Captures. The Provider MUST be capable of encoding and sending all Captures within any given Global View simultaneously.
A Media Capture (MC) is a source of Media from one or more Capture Devices or constructed from other Media Streams. Each MC is of a single Media type, has an Advertisement-unique identity, exactly one set of spatial information, and can be the source of at most one Capture Encoding.
A Multiple Content Capture (MCC) is a Capture that mixes and/or switches other Captures of a single type. It MAY reference Captures from within or across Capture Scenes, MAY contain no references (indicating unknown sources), and has MCC-specific attributes such as MaxCaptures, Policy, SynchronizationID, and Allow Subset Choice.
A Simultaneous Transmission Set is a set of Media Captures that a Provider can transmit simultaneously. Sets are Media-type-specific, apply across all Capture Scenes in an Advertisement, and constrain which Captures a Consumer may request together. An MCU Provider may advertise multiple mutually exclusive sets when a device (e.g., a zoom camera) cannot be used in two modes at once.
An Advertisement is a CLUE message a Media Provider sends to a Media Consumer describing specific aspects of the content of the Media and any restrictions on simultaneously providing certain Streams. It contains Capture Scenes, Simultaneous Transmission Sets, Global View List, and Encoding Groups.
An Encoding Group is a set of one or more Individual Encodings plus a maximum combined bitrate for the group. It allows a Provider to signal bandwidth constraints across multiple Capture Encodings. Each Capture Encoding bitrate MUST NOT exceed its Individual Encoding maximum, and the sum of all Capture Encoding bitrates MUST NOT exceed the Encoding Group maximum.
Section 14 addresses security considerations for the CLUE framework. Because CLUE operates over a data channel established via SIP and SDP, it inherits the security properties of those protocols; implementors must ensure the CLUE channel is protected (e.g., via DTLS) and that Advertisement/Configure messages cannot be spoofed or tampered with to redirect media streams.