Minimum RTT Estimation Under Low ACK Frequency

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in .

specifies a simple delayed ACK mechanism that a receiver can send an ACK for every other packet, and for every packet when reordering is observed, or when the max_ack_delay timer expires. However, this ACK mechanism may not match the number of ACKs to the transport's required intensity under different network conditions. For example, when the data throughput of a WLAN transport is extremely high, QUIC will generate a large number of ACKs. In this case, minimizing the ACK intensity of QUIC is not only a win for data throughput improvement but also a win for energy and CPU efficiency. and were two core functionality standards that introduced delayed ACK, which was the default acknowledgment mechanism in most Linux distributions. and described an acknowledgment congestion control mechanism in which the minimum ACK frequency allowed is twice per send window. discussed the imperfection and variability of TCP's acknowledgment mechanism because of asymmetric effects and recommended scaling ACK frequency as a mitigation to these effects. These RFCs reveal that the dependence on frequent ACKs is an artifact of current transport protocol designs rather than a fundamental requirement. Based on this insight, some work-in-progress IETF drafts have paid great attention to ACK scaling technologies in both TCP and QUIC working groups. First of all, proposed the TCP ACK pull mechanism, which allows a sender to request the ACK for a data segment to be sent without additional delay by the receiver. This helps in some cases when the delayed ACKs degrade transport performance. Instead of pulling more ACKs, recommended reducing the ACK frequency by sending an ACK for at least every 10 received packets and recommended an ACK frequency of four ACKs every round-trip time (RTT), aiming to reduce link transmission costs for asymmetric paths. Different from using an empirical value of ACK frequency, instead, Li tried to improve the scalability by proposing a novel ACK mechanism named Tame ACK (TACK), whose frequency is a function of the bandwidth-delay product of network connections. The detailed TCP-based implementation (i.e., TCP-TACK) details and evaluation results in industrial applications have been shown in . The recent work has been adopted in QUIC working group, it specifies an extension to QUIC that enables an endpoint to request its peer change its behavior when acknowledging at a low ACK frequency. Based on the observations, low ACK frequency not only addresses an industry-wide need but also demonstrates broad applicability across diverse operational environments.

In this document, the minRTT is the minimum RTT samples observed at the sender for a given network path during a period of time. The minRTT is a fundamental state variable in transmission control, and its accuracy is critical for achieving both high throughput and low latency. For example, some congestion controllers depend on minRTT to estimate the congestion window . minRTT is also used by QUIC loss detection to reject implausibly small RTT samples . minRTT is further used to update the ACK frequency in . An RTT estimation system contains a sender and a receiver. Ideally, when sending an ACK for every data packet, the minimum RTT sample can be computed by monitoring the per-packet RTT samples. However, the sender can hardly generate per-packet RTT samples in the case of sending fewer ACKs, which is the root cause of the minimum RTT estimation biases. When multiple packets carrying departure timestamps are transported between endpoints via the same path, an RTT of this path can be sampled at the sender upon receiving an ACK frame. However, when sending fewer ACK frames, more data packets might be received during the ACK interval, generating only one RTT sample among multiple packets is likely to result in biases. For example, a larger minimum RTT estimate. In general, the higher the throughput, the larger the biases. Experiments in show that the default way of RTT sampling suffers 8% ∼ 18% larger minRTT estimates. One alternative way to reduce biases can be that, each ACK frame carries multiple timestamps for the sender to generate more RTT samples. For example, the recent draft has been adopted in QUIC working group, this document defines an extension to the QUIC transport protocol which supports reporting multiple packet receive timestamps in a best-effort way. However, (1) the overhead might be too high to be acceptable for high-bandwidth transport. Also, (2) the number of data packets might be far more than the maximum number of timestamps that an ACK frame is capable of carrying. This issue is unremarkable for the basic RTT estimation (e.g., smoothed RTT) when throughput is low. However, the minRTT might be overestimated because that the sender can hardly generate per-packet RTT samples when the throughput is hign.

Since the receiver is capable of monitoring per-packet state, the one-way delay (OWD) of each packet can be easily computed according to the departure timestamps (carried in the TIMESTAMP frame) and the arrival timestamps of each packet. In this case, QUIC SHOULD adopt the OWD-based minRTT estimation. The rationale is that the variation of OWD reflects the variation of RTT over near-symmetric links. The OWD-based minRTT estimation requires the sender to record the departure timestamp in each ack-eliciting packet. Meanwhile, at the receiver, the per-packet OWD samples SHOULD be computed upon packet arrivals and a function of computing the minimum OWD (minOWD) SHOULD be newly added. In this document, minOWD is the minimum OWD samples observed on the same network path during a period of time. The receiver then generates an ACK frame for the sender, in which the ACK delay and departure timestamp for the packet that achieves the minimum OWD is reported. The ACK delay is defined as the delay incurred between when the packet is received and when the ACK frame is sent. Based on the information reported by the incoming ACK frames and the ACK arrival timestamps, the sender can generate RTT samples and then compute minRTT accordingly.

Generally, the minRTT is calculated at the sender. Before estimating the minRTT, the RTT samples should be computed based on the ACK frames collected during a period. Assume that a packet is sent by the sender at time t_1 and arrives at time t_3, and the ACK frame is sent at time t_4. The ACK delay can be computed at the receiver. For example, the receiver computes the ACK delay delta_t = t_4 - t_3, and syncs the ACK delay to the sender via an ACK frame. The ACK delay can also be computed at the sender. For example, the receiver directly syncs an ACK frame carrying t_4 and t_3 to the sender, the sender then computes the ACK delay delta_t = t_4 - t_3. The sender therefore computes an RTT sample according to delta_t, t_1, and the arrival time (t_2) of the ACK frame, i.e., RTT_sample = t_2 − t_1 − delta_t. Measuring delta_t at the receiver assures an explicit correction for a more accurate RTT estimate. RTT samples SHOULD be smoothed using exponentially weighted moving average (EWMA) as specified in . The sender then computes the minRTT according to these RTT samples during a period. The sender SHOULD insert a TIMESTAMP frame into each packet for measuring OWD at the receiver.

As specified in , by default the QUIC receiver reports ACK delays for only the largest acknowledged packet in an ACK frame, hence an RTT sample is generated using only the largest acknowledged packet in the received ACK frame. For a more accurate minRTT estimate when sending fewer ACK frames, this document requires the QUIC receiver to filter the departure timestamp for the packet who achieves the minOWD during the interval between two ACK frames and report the ACK delay of this packet. Hence, to achieve the OWD-based minRTT estimation, the QUIC receiver is RECOMMENDED to send an MINOWD-ACK frame periodically to report ACK delay and the departure timestamp of the packet who achieves the minOWD. Upon packet arrivals, the receiver is capable of generating per-packet OWD samples according to the difference between the packet departure timestamp and packet arrival timestamp. The receiver then computes the minOWD by comparing the per-packet OWD samples. The OWD estimation does not require clock synchronization here because the relative values are adopted. Afterwards, based on the ACK delay and the departure timestamp corresponding to the packet that achieves the minOWD, the sender calculates the RTT of this packet as a minimum RTT sample. Ultimately, the minRTT is computed according to these minimum RTT samples. The MINOWD-ACK frame is RECOMMENDED to send at most once an RTT to limit the acknowledgment frequency.

A new field named timestamp-support should be added for negotiation between both parties on whether to sync packet departure timestamps in QUIC connection. The endpoints send this parameter during handshakes. Only when both parties agree, packet departure timestamp synchronization can be adopted. timestamp-support (0x XX): This parameter has two values (0 or 1) specifying whether the sending endpoint is willing to sync packet departure timestamps. When the value is set as 1, it means that the sending endpoint wants to sync packet departure timestamps during connection. When the value is set as 0, it means that the sending endpoint does not support this function.

Instead of the invasive way of adding a new field in the QUIC public packet header, it is RECOMMENDED that a new frame be added for exchanging the departure timestamp of each packet. It is worth noting that has proposed an extension of TIMESTAMP frame for QUIC, which can be reused for minRTT estimation in this document. Based on , a TIMESTAMP frame is shown in .

A TIMESTAMP frame contains the following fields: Departure Timestamp: An integer indicating the departure time of a packet. QUIC SHOULD carry the TIMESTAMP Frame in each packet.

Instead of the invasive way of redefining the ACK Delay field in the QUIC ACK frame, it is RECOMMENDED that a new MINOWD-ACK frame be added for minRTT estimation. A MINOWD-ACK frame is shown in .

A MINOWD-ACK frame contains the following fields: MinOWD ACK Delay: An integer indicating the MINOWD-ACK Delay. When generating the MINOWD-ACK frame, QUIC SHOULD change the way of computing MINOWD-ACK Delay according to the arrival timestamp of the packet with minimum OWD instead of the arrival timestamp of the largest acknowledged packet. MinOWD Departure Timestamp: An integer indicating the departure time of the packet who achieves the mimOWD.

TBD

The values for the timestamp-support transport parameter, TIMESTAMP frame, and MINOWD-ACK frame should be allocated.

Acknowledgements Thanks to Kai Zheng, Kun Tan, Rahul Arvind Jadhav, Jiao Kang, Keith Winstein, Marco Munizaga, Marten Seemann, and Christian Huitema for their reviews and suggestions.