The Congestion Manager
Extracted elements (29)
Concurrent independent streams each implementing their own congestion control are more aggressive in aggregate than a single TCP connection under standard algorithms. The CM addresses this by sharing congestion state across streams in a macroflow, reducing the ensemble's aggressiveness.
The basic CM API deliberately omits a buffered congestion-controlled transmission mode because this can be built using the callback-based primitive, as demonstrated by the congestion-controlled UDP socket example in Section 5.1.2.
The CM API convention terms (MUST, SHOULD, etc.) apply only to implementations offering the CM API, not to congestion control implementations in general. This scoping is explicit because IETF does not treat APIs as standards-track by convention.
The CM does not buffer any application data. This allows streams to adapt and repacketize data at the last possible instant before transmission when network conditions change, which is infeasible once data has been buffered in the CM.
The default macroflow aggregation groups all streams to the same destination IP address into one macroflow. This is a simple heuristic; cm_getmacroflow()/cm_setmacroflow() allow applications to override it where needed.
The CM framework does not address networks with reservations or service differentiation, nor paths where different streams between the same host pair traverse links with differing characteristics. Extensions for these cases are left to future documents.
When a group of receivers are behind a NAT device, the sender sees them all as one address and may aggregate them into a single macroflow. If those hosts are on different bottleneck links, some could experience worse performance than without the CM. Detection via delay and loss estimates is possible but out of scope.
A cmapp_send() grant is valid only until an expiration time equal to max(RTT, implementation threshold). The application MUST NOT send data after this expiration. If the application decides not to send, it SHOULD call cm_notify(stream_info, 0) to allow other streams to transmit.
All applications MUST call cm_open(stream_info) before using any other CM API function. A returned cm_streamid of -1 indicates failure; that stream cannot use the CM.
All applications using the CM MUST incorporate feedback from the receiver, typically once or twice per RTT, and MUST use cm_update() to inform the CM of the resulting congestion information.
cm_notify(i32 cm_streamid, u32 nsent) MUST be called whenever data is transmitted from the host so the CM can update its estimate of outstanding bytes for the macroflow and stream.
Systems MAY choose to use the Congestion Manager, but if they do, they MUST follow the specification completely.
The CM congestion controller MUST be robust to applications forgetting to invoke cm_notify() correctly, or applications that crash or disappear after making a cm_request() call.
The CM MUST export the cmapp_update(u64 newrate, u32 srtt, u32 rttdev) callback for synchronous senders. In response, the stream MUST adapt its packet size or timer interval to not exceed the reported rate.
The CM MUST implement cm_query(i32 cm_streamid, u64* rate, u32* srtt, u32* rttdev). If no valid estimates exist for the macroflow, the CM fills in negative values for rate, srtt, and rttdev.
The CM MUST invoke the cmapp_send() callback after a cm_request() call, granting the stream permission to send up to PMTU bytes. Each cm_request() is an implicit request for up to one PMTU-sized transmission.
The CM MUST provide cm_thresh(float rate_downthresh, float rate_upthresh, float rtt_downthresh, float rtt_upthresh) so streams can specify thresholds for cmapp_update() callbacks. The CM SHOULD invoke the callback only when rates or RTTs change by the requisite factors.
The congestion control module MUST call the associated scheduler's schedule() function when the current congestion state allows an MTU-sized packet to be sent. The Scheduler MUST implement schedule(u32 num_bytes), query_share(i32 cm_streamid), and notify(i32 cm_streamid, u32 nsent) interfaces.
The congestion controller MUST implement a TCP-friendly congestion control algorithm. The congestion control module MUST implement the abstract query(), notify(), and update() interfaces to allow modular interoperability with separately developed schedulers.
When a stream receives receiver feedback, it MUST call cm_update(i32 cm_streamid, u32 nrecd, u32 nlost, u8 lossmode, i32 rtt) to inform the CM of receptions, losses, loss type, and RTT samples. The lossmode parameter MUST be reported as a bit-vector.
When a stream terminates, the application SHOULD invoke cm_close(cm_streamid) to inform the CM. PMTU information used by cm_mtu() SHOULD be obtained via path MTU discovery and MAY be statically configured if that mechanism is unavailable.
A STREAM is defined by source/destination IP address, IP type-of-service, transport protocol, and source/destination port numbers. A MACROFLOW is a group of CM-enabled streams sharing the same congestion management and scheduling algorithms; streams to the same receiver experiencing identical congestion SHOULD belong to the same macroflow.
cm_getmacroflow(i32 cm_streamid) returns the macroflow identifier for a stream. cm_setmacroflow(i32 cm_macroflowid, i32 cm_streamid) assigns a stream to a macroflow; passing -1 as cm_macroflowid creates a new macroflow. Each call overrides any prior macroflow association.
The callback-based transmission API uses cm_request(i32 cm_streamid) to request a send grant; the CM later invokes cmapp_send() authorizing up to PMTU bytes. An optional cm_request(int k) variant grants up to k PMTU-sized segments. This is the recommended API for ALF-based asynchronous streams.
The lossmode field in cm_update() is a bit-vector with four values: CM_NO_FEEDBACK (timeout, no feedback received), CM_LOSS_FEEDBACK (duplicate-ACK or data-driven loss detection), CM_EXPLICIT_CONGESTION (ECN echo), and CM_NO_CONGESTION (non-congestion loss).
A stream on the same host can arbitrarily slow down other streams in the same macroflow by reporting artificially high congestion estimates, a form of intra-host denial of service. The CM does not enforce correct behavior; a separate host-level policer or router machinery is required for enforcement.
An application that reports a low congestion estimate to the CM will cause the CM to become overly aggressive, allowing data to be sent far faster than sound congestion control policy permits. This is the more dangerous attack vector.
Because the CM API exposes shared congestion state, a poorly designed, compromised, or malicious application can misuse cm_update() to cause starvation of other streams in the macroflow or induce too-aggressive behavior, constituting a denial-of-service against co-located applications.
The AIMD_CC example congestion controller maintains cwnd, ownd, ssthresh, and srtt. On CM_NO_FEEDBACK loss, cwnd is set to 1 MTU; on CM_LOSS_FEEDBACK or CM_EXPLICIT_CONGESTION, cwnd is halved (min 1 MTU) and ssthresh = cwnd/2. Without loss, cwnd grows by nsent when cwnd < ssthresh (slow start) or by nsent*MTU/cwnd when cwnd >= ssthresh (linear).