Careful Resume: Convergence of Congestion Control from Retained State
Extracted elements (28)
A current RTT more than ten times saved_rtt is treated as indicating a path change. The factor of ten accommodates both buffering-induced latency increases and RTT sample variation without prematurely aborting Careful Resume on transient spikes.
jump_cwnd is limited to half of saved_cwnd to avoid starving flows that started or increased their capacity since the last observation. A direct reuse of saved_cwnd at full value would risk a disproportionate share of the bottleneck, displacing existing flows.
Only one connection at a time may use a specific set of saved CC parameters. Allowing multiple simultaneous connections to each jump by saved_cwnd/2 would create an excessive aggregate load at the bottleneck. Implementations must coordinate (e.g., via hash table or distributed lock) when load balancing may route concurrent connections from the same client to different server processes.
Proportional Rate Reduction (PRR) is explicitly not used in the Safe Retreat Phase because PRR assumes it is safe to reduce rate gradually during congestion avoidance. When Careful Resume produces significant overshoot, gradual reduction would maintain an excessively high rate and further starve competing flows.
The default Beta value of 0.5 was chosen to reduce the probability of inducing a second round of congestion after Safe Retreat. CUBIC defines Beta_cubic=0.7, which may be used instead when running CUBIC.
The Lifetime for saved CC parameters is intentionally configurable rather than fixed. For dynamic or shared paths a short Lifetime (minutes) is recommended to reduce risk; for stable, lightly shared paths a longer Lifetime (hours) is justified because Careful Resume's overshoot mitigations reduce the safety risk compared to RFC 7661's fixed 5-minute NVP.
The Reconnaissance Phase uses the standard IW (no more than recommended maximum initial window per RFC 6928 / RFC 9002) for the first RTT rather than jumping immediately to saved_cwnd. This ensures any path with persistent congestion is detected before the large CWND jump, and establishes the current minimum RTT needed to validate the saved parameters.
The RTT lower bound for path confirmation is saved_rtt/2, because jump_cwnd is set to saved_cwnd/2 and the resulting sending rate should not exceed the previously observed rate. A factor of two arises directly from the relationship between CWND, RTT, and throughput.
Unvalidated packets are paced using the current RTT rather than the saved_rtt. This avoids a smaller saved_rtt inflating the sending rate beyond what was previously observed, providing a safety margin proportional to actual path conditions.
A TCP sender is limited by the receiver-advertised window (rwnd). Unless the receiver configures a sufficiently large rwnd, it will constrain the rate of increase and reduce the benefit of Careful Resume, since the CWND increase enabled by the method may not be usable.
For BBR, Careful Resume is restricted to operation during BBR's Startup state only. The 'carefully-resuming' flag gates unvalidated packet transmission and triggers entry into BBR's Drain state (Safe Retreat) on congestion. The flag is cleared two rounds after being set to allow measured delivery rates to reflect actual path capacity.
Network mechanisms such as ECMP, anycast routing, and NAT can silently change the forwarding path between connections. Servers using anycast addresses cannot guarantee that the same physical path is used across connections; operators must ensure alternative paths have equivalent capacity or avoid using Careful Resume in such deployments.
QUIC includes flow control and anti-amplification mechanisms that can constrain sending rate independently of CWND. A QUIC receiver may need to issue proactive MAX_DATA frames to increase flow control limits before or during Careful Resume to realise the expected throughput benefit.
A sender MUST NOT retain more than one set of CC parameters for a Remote Endpoint, though the set SHOULD be updated or replaced after a later observation of the same path.
All packets sent in the Unvalidated Phase MUST use pacing based on the current RTT. The recommended inter-packet transmission time is ITT = (current_RTT × MPS) / jump_cwnd.
If congestion (packet loss or ECN-CE) is detected in the Reconnaissance Phase, the sender MUST continue using normal CC to react to the detected congestion and MUST NOT proceed with Careful Resume.
In the Reconnaissance Phase, the sender MUST stop using Careful Resume and return to normal CC if: another connection is already using the saved_cwnd; the Remote Endpoint does not match any saved_remote_endpoint; the Lifetime of saved CC parameters is exceeded; or the minimum RTT measured in the Reconnaissance Phase is less than or equal to saved_rtt/2.
In the Unvalidated Phase exit check, if flight_size is less than IW or less than or equal to PipeSize, CWND is reset to PipeSize and normal CC resumes; otherwise CWND MUST be set to flight_size before entering the Validating Phase.
In the Unvalidated Phase, if a detected path change or congestion (packet loss or ECN-CE) occurs, the Safe Retreat Phase MUST be entered immediately.
On entry to the Safe Retreat Phase, the CWND MUST be reduced to no more than PipeSize/2 to avoid persistent starvation of competing flows. The CWND MUST NOT be increased while in the Safe Retreat Phase.
On entry to the Unvalidated Phase, the jump_cwnd MUST be no more than half of the saved_cwnd: jump_cwnd ≤ min(max_jump, saved_cwnd/2). CWND is set to jump_cwnd.
On leaving the Safe Retreat Phase, ssthresh MUST be set to no larger than the most recently measured PipeSize multiplied by Beta (default Beta=0.5). Saved CC parameters are deleted on Safe Retreat entry to prevent reuse.
When using BBR, delivery rates measured in the Drain state MUST be discarded if the 'carefully-resuming' flag is True, because packets sent during the attempt may still be queued at the bottleneck and could have displaced competing flows' packets.
jump_cwnd is the tentative CWND used in the Unvalidated Phase, set to min(max_jump, saved_cwnd/2). PipeSize tracks the validated portion of capacity, initialised to flight_size on entry to the Unvalidated Phase and incremented per acknowledged byte. Beta is a scaling factor (0.5–1, default 0.5) applied to PipeSize when setting ssthresh on Safe Retreat exit.
saved_cwnd records the utilised capacity (bytes sent during one RTT, or volume acknowledged in one RTT) observed on a previous connection. saved_rtt records the minimum RTT at the time of observation. Together with a Lifetime timestamp and a saved_remote_endpoint, these form the complete set of saved CC parameters.
The Remote Endpoint identifies the observed network path and consists of an identifier for the sending interface (e.g., globally assigned address/prefix) and an identifier for the destination (e.g., unicast or anycast IP address). It may also include a DSCP value. Saved CC parameters are only reused when the Remote Endpoint has not changed.
The security considerations for Careful Resume are the same as for other sender-based congestion control methods. These methods rely on the receiver appropriately acknowledging receipt of data; an on-path or off-path attacker's ability to influence CC depends on the security properties of the underlying transport protocol.
Careful Resume defines five phases: Observing (normal CC, saving parameters), Reconnaissance (confirm path using IW), Unvalidated (tentative jump to saved_cwnd/2 with pacing), Validating (check unvalidated packets for congestion), and Safe Retreat (drain path, reduce CWND to PipeSize/2 on detected congestion). Transitions back to normal CC can occur from any phase.