7. Congestion Control
This document specifies a sender-side congestion controller for QUIC similar to TCP NewReno [RFC6582].
The signals QUIC provides for congestion control are generic and are designed to support different sender-side algorithms. A sender can unilaterally choose a different algorithm to use, such as CUBIC [RFC8312].
If a sender uses a different controller than that specified in this document, the chosen controller MUST conform to the congestion control guidelines specified in Section 3.1 of [RFC8085].
Similar to TCP, packets containing only ACK frames do not count toward bytes in flight and are not congestion controlled. Unlike TCP, QUIC can detect the loss of these packets and MAY use that information to adjust the congestion controller or the rate of ACK-only packets being sent, but this document does not describe a mechanism for doing so.
The congestion controller is per path, so packets sent on other paths do not alter the current path's congestion controller, as described in Section 9.4 of [QUIC-TRANSPORT].
The algorithm in this document specifies and uses the controller's congestion window in bytes.
An endpoint MUST NOT send a packet if it would cause bytes_in_flight (see Appendix B.2) to be larger than the congestion window, unless the packet is sent on a PTO timer expiration (see Section 6.2) or when entering recovery (see Section 7.3.2).
7.1. Explicit Congestion Notification
If a path has been validated to support Explicit Congestion Notification (ECN) [RFC3168] [RFC8311], QUIC treats a Congestion Experienced (CE) codepoint in the IP header as a signal of congestion. This document specifies an endpoint's response when the peer-reported ECN-CE count increases; see Section 13.4.2 of [QUIC-TRANSPORT].
7.2. Initial and Minimum Congestion Window
QUIC begins every connection in slow start with the congestion window set to an initial value. Endpoints SHOULD use an initial congestion window of ten times the maximum datagram size (max_datagram_size), while limiting the window to the larger of 14,720 bytes or twice the maximum datagram size. This follows the analysis and recommendations in [RFC6928], increasing the byte limit to account for the smaller 8-byte overhead of UDP compared to the 20-byte overhead for TCP.
If the maximum datagram size changes during the connection, the initial congestion window SHOULD be recalculated with the new size. If the maximum datagram size is decreased in order to complete the handshake, the congestion window SHOULD be set to the new initial congestion window.
Prior to validating the client's address, the server can be further limited by the anti-amplification limit as specified in Section 8.1 of [QUIC-TRANSPORT]. Though the anti-amplification limit can prevent the congestion window from being fully utilized and therefore slow down the increase in congestion window, it does not directly affect the congestion window.
The minimum congestion window is the smallest value the congestion window can attain in response to loss, an increase in the peer-reported ECN-CE count, or persistent congestion. The RECOMMENDED value is 2 * max_datagram_size.
7.3. Congestion Control States
The NewReno congestion controller described in this document has three distinct states, as shown in Figure 1.
New path or +------------+
persistent congestion | Slow |
(O)----------------------> | Start |
+------------+
|
Loss or
ECN-CE increase
|
v
+------------+ Loss or +------------+
| Congestion | ECN-CE increase | Recovery |
| Avoidance |------------------>| Period |
+------------+ +------------+
^ |
| |
+----------------------------+
Acknowledgment of packet
sent during recovery
Figure 1: Congestion Control States and Transitions
These states and the transitions between them are described in subsequent sections.
7.3.1. Slow Start
A NewReno sender is in slow start any time the congestion window is below the slow start threshold. A sender begins in slow start because the slow start threshold is initialized to an infinite value.
While a sender is in slow start, the congestion window increases by the number of bytes acknowledged when each acknowledgment is processed. This results in exponential growth of the congestion window.
The sender MUST exit slow start and enter a recovery period when a packet is lost or when the ECN-CE count reported by its peer increases.
A sender reenters slow start any time the congestion window is less than the slow start threshold, which only occurs after persistent congestion is declared.
7.3.2. Recovery
A NewReno sender enters a recovery period when it detects the loss of a packet or when the ECN-CE count reported by its peer increases. A sender that is already in a recovery period stays in it and does not reenter it.
On entering a recovery period, a sender MUST set the slow start threshold to half the value of the congestion window when loss is detected. The congestion window MUST be set to the reduced value of the slow start threshold before exiting the recovery period.
Implementations MAY reduce the congestion window immediately upon entering a recovery period or use other mechanisms, such as Proportional Rate Reduction [PRR], to reduce the congestion window more gradually. If the congestion window is reduced immediately, a single packet can be sent prior to reduction. This speeds up loss recovery if the data in the lost packet is retransmitted and is similar to TCP as described in Section 5 of [RFC6675].
The recovery period aims to limit congestion window reduction to once per round trip. Therefore, during a recovery period, the congestion window does not change in response to new losses or increases in the ECN-CE count.
A recovery period ends and the sender enters congestion avoidance when a packet sent during the recovery period is acknowledged. This is slightly different from TCP's definition of recovery, which ends when the lost segment that started recovery is acknowledged [RFC5681].
7.3.3. Congestion Avoidance
A NewReno sender is in congestion avoidance any time the congestion window is at or above the slow start threshold and not in a recovery period.
A sender in congestion avoidance uses an Additive Increase Multiplicative Decrease (AIMD) approach that MUST limit the increase to the congestion window to at most one maximum datagram size for each congestion window that is acknowledged.
The sender exits congestion avoidance and enters a recovery period when a packet is lost or when the ECN-CE count reported by its peer increases.
7.4. Ignoring Loss of Undecryptable Packets
During the handshake, some packet protection keys might not be available when a packet arrives, and the receiver can choose to drop the packet. In particular, Handshake and 0-RTT packets cannot be processed until the Initial packets arrive, and 1-RTT packets cannot be processed until the handshake completes. Endpoints MAY ignore the loss of Handshake, 0-RTT, and 1-RTT packets that might have arrived before the peer had packet protection keys to process those packets. Endpoints MUST NOT ignore the loss of packets that were sent after the earliest acknowledged packet in a given packet number space.
7.5. Probe Timeout
Probe packets MUST NOT be blocked by the congestion controller. A sender MUST however count these packets as being additionally in flight, since these packets add network load without establishing packet loss. Note that sending probe packets might cause the sender's bytes in flight to exceed the congestion window until an acknowledgment is received that establishes loss or delivery of packets.
7.6. Persistent Congestion
When a sender establishes loss of all packets sent over a long enough duration, the network is considered to be experiencing persistent congestion.
7.6.1. Duration
The persistent congestion duration is computed as follows:
(smoothed_rtt + max(4*rttvar, kGranularity) + max_ack_delay) * kPersistentCongestionThreshold
Unlike the PTO computation in Section 6.2, this duration includes the max_ack_delay irrespective of the packet number spaces in which losses are established.
This duration allows a sender to send as many packets before establishing persistent congestion, including some in response to PTO expiration, as TCP does with Tail Loss Probes [RFC8985] and an RTO [RFC5681].
Larger values of kPersistentCongestionThreshold cause the sender to become less responsive to persistent congestion in the network, which can result in aggressive sending into a congested network. Too small a value can result in a sender declaring persistent congestion unnecessarily, resulting in reduced throughput for the sender.
The RECOMMENDED value for kPersistentCongestionThreshold is 3, which results in behavior that is approximately equivalent to a TCP sender declaring an RTO after two TLPs.
This design does not use consecutive PTO events to establish persistent congestion, since application patterns impact PTO expiration. For example, a sender that sends small amounts of data with silence periods between them restarts the PTO timer every time it sends, potentially preventing the PTO timer from expiring for a long period of time, even when no acknowledgments are being received. The use of a duration enables a sender to establish persistent congestion without depending on PTO expiration.
7.6.2. Establishing Persistent Congestion
A sender establishes persistent congestion after the receipt of an acknowledgment if two packets that are ack-eliciting are declared lost, and:
-
across all packet number spaces, none of the packets sent between the send times of these two packets are acknowledged;
-
the duration between the send times of these two packets exceeds the persistent congestion duration (Section 7.6.1); and
-
a prior RTT sample existed when these two packets were sent.
These two packets MUST be ack-eliciting, since a receiver is required to acknowledge only ack-eliciting packets within its maximum acknowledgment delay; see Section 13.2 of [QUIC-TRANSPORT].
The persistent congestion period SHOULD NOT start until there is at least one RTT sample. Before the first RTT sample, a sender arms its PTO timer based on the initial RTT (Section 6.2.2), which could be substantially larger than the actual RTT. Requiring a prior RTT sample prevents a sender from establishing persistent congestion with potentially too few probes.
Since network congestion is not affected by packet number spaces, persistent congestion SHOULD consider packets sent across packet number spaces. A sender that does not have state for all packet number spaces or an implementation that cannot compare send times across packet number spaces MAY use state for just the packet number space that was acknowledged. This might result in erroneously declaring persistent congestion, but it will not lead to a failure to detect persistent congestion.
When persistent congestion is declared, the sender's congestion window MUST be reduced to the minimum congestion window (kMinimumWindow), similar to a TCP sender's response on an RTO [RFC5681].
7.6.3. Example
The following example illustrates how a sender might establish persistent congestion. Assume:
smoothed_rtt + max(4*rttvar, kGranularity) + max_ack_delay = 2
kPersistentCongestionThreshold = 3
Consider the following sequence of events:
| Time | Action |
|---|---|
| t=0 | Send packet #1 (application data) |
| t=1 | Send packet #2 (application data) |
| t=1.2 | Receive acknowledgment of #1 |
| t=2 | Send packet #3 (application data) |
| t=3 | Send packet #4 (application data) |
| t=4 | Send packet #5 (application data) |
| t=5 | Send packet #6 (application data) |
| t=6 | Send packet #7 (application data) |
| t=8 | Send packet #8 (PTO 1) |
| t=12 | Send packet #9 (PTO 2) |
| t=12.2 | Receive acknowledgment of #9 |
Packets 2 through 8 are declared lost when the acknowledgment for packet 9 is received at t = 12.2.
The congestion period is calculated as the time between the oldest and newest lost packets: 8 - 1 = 7. The persistent congestion duration is 2 * 3 = 6. Because the threshold was reached and because none of the packets between the oldest and the newest lost packets were acknowledged, the network is considered to have experienced persistent congestion.
While this example shows PTO expiration, they are not required for persistent congestion to be established.
7.7. Pacing
A sender SHOULD pace sending of all in-flight packets based on input from the congestion controller.
Sending multiple packets into the network without any delay between them creates a packet burst that might cause short-term congestion and losses. Senders MUST either use pacing or limit such bursts. Senders SHOULD limit bursts to the initial congestion window; see Section 7.2. A sender with knowledge that the network path to the receiver can absorb larger bursts MAY use a higher limit.
An implementation should take care to architect its congestion controller to work well with a pacer. For instance, a pacer might wrap the congestion controller and control the availability of the congestion window, or a pacer might pace out packets handed to it by the congestion controller.
Timely delivery of ACK frames is important for efficient loss recovery. To avoid delaying their delivery to the peer, packets containing only ACK frames SHOULD therefore not be paced.
Endpoints can implement pacing as they choose. A perfectly paced sender spreads packets exactly evenly over time. For a window-based congestion controller, such as the one in this document, that rate can be computed by averaging the congestion window over the RTT. Expressed as a rate in units of bytes per time, where congestion_window is in bytes:
rate = N * congestion_window / smoothed_rtt
Or expressed as an inter-packet interval in units of time:
interval = (smoothed_rtt * packet_size / congestion_window) / N
Using a value for N that is small, but at least 1 (for example, 1.25) ensures that variations in RTT do not result in underutilization of the congestion window.
Practical considerations, such as packetization, scheduling delays, and computational efficiency, can cause a sender to deviate from this rate over time periods that are much shorter than an RTT.
One possible implementation strategy for pacing uses a leaky bucket algorithm, where the capacity of the "bucket" is limited to the maximum burst size and the rate the "bucket" fills is determined by the above function.
7.8. Underutilizing the Congestion Window
When bytes in flight is smaller than the congestion window and sending is not pacing limited, the congestion window is underutilized. This can happen due to insufficient application data or flow control limits. When this occurs, the congestion window SHOULD NOT be increased in either slow start or congestion avoidance.
A sender that paces packets (see Section 7.7) might delay sending packets and not fully utilize the congestion window due to this delay. A sender SHOULD NOT consider itself application limited if it would have fully utilized the congestion window without pacing delay.
A sender MAY implement alternative mechanisms to update its congestion window after periods of underutilization, such as those proposed for TCP in [RFC7661].