| rfc9438.original | rfc9438.txt | |||
|---|---|---|---|---|
| TCPM L. Xu | Internet Engineering Task Force (IETF) L. Xu | |||
| Internet-Draft UNL | Request for Comments: 9438 UNL | |||
| Obsoletes: 8312 (if approved) S. Ha | Obsoletes: 8312 S. Ha | |||
| Updates: 5681 (if approved) Colorado | Updates: 5681 Colorado | |||
| Intended status: Standards Track I. Rhee | Category: Standards Track I. Rhee | |||
| Expires: 4 August 2023 Bowery | ISSN: 2070-1721 Bowery | |||
| V. Goel | V. Goel | |||
| Apple Inc. | Apple Inc. | |||
| L. Eggert, Ed. | L. Eggert, Ed. | |||
| NetApp | NetApp | |||
| 31 January 2023 | August 2023 | |||
| CUBIC for Fast and Long-Distance Networks | CUBIC for Fast and Long-Distance Networks | |||
| draft-ietf-tcpm-rfc8312bis-15 | ||||
| Abstract | Abstract | |||
| CUBIC is a standard TCP congestion control algorithm that uses a | CUBIC is a standard TCP congestion control algorithm that uses a | |||
| cubic function instead of a linear congestion window increase | cubic function instead of a linear congestion window increase | |||
| function to improve scalability and stability over fast and long- | function to improve scalability and stability over fast and long- | |||
| distance networks. CUBIC has been adopted as the default TCP | distance networks. CUBIC has been adopted as the default TCP | |||
| congestion control algorithm by the Linux, Windows, and Apple stacks. | congestion control algorithm by the Linux, Windows, and Apple stacks. | |||
| This document updates the specification of CUBIC to include | This document updates the specification of CUBIC to include | |||
| algorithmic improvements based on these implementations and recent | algorithmic improvements based on these implementations and recent | |||
| academic work. Based on the extensive deployment experience with | academic work. Based on the extensive deployment experience with | |||
| CUBIC, it also moves the specification to the Standards Track, | CUBIC, this document also moves the specification to the Standards | |||
| obsoleting RFC 8312. This also requires updating RFC 5681, to allow | Track and obsoletes RFC 8312. This document also updates RFC 5681, | |||
| for CUBIC's occasionally more aggressive sending behavior. | to allow for CUBIC's occasionally more aggressive sending behavior. | |||
| About This Document | ||||
| This note is to be removed before publishing as an RFC. | ||||
| Status information for this document may be found at | ||||
| https://datatracker.ietf.org/doc/draft-ietf-tcpm-rfc8312bis/. | ||||
| Discussion of this document takes place on the TCPM Working Group | ||||
| mailing list (mailto:tcpm@ietf.org), which is archived at | ||||
| https://mailarchive.ietf.org/arch/browse/tcpm/. Subscribe at | ||||
| https://www.ietf.org/mailman/listinfo/tcpm/. | ||||
| Source for this draft and an issue tracker can be found at | ||||
| https://github.com/NTAP/rfc8312bis. | ||||
| Note to the RFC Editor | ||||
| xml2rfc currently renders <em></em> in the XML by surrounding the | ||||
| corresponding text with underscores. This is highly distracting; | ||||
| please manually remove the underscores when doing the final edits to | ||||
| the text version of this document. | ||||
| (There is an issue open against xml2rfc to stop doing this in the | ||||
| future: https://github.com/ietf-tools/xml2rfc/issues/596) | ||||
| Also, please manually change "Figure" to "Equation" for all artwork | ||||
| with anchors beginning with "eq" - xml2rfc doesn't seem to be able to | ||||
| do this. | ||||
| Status of This Memo | Status of This Memo | |||
| This Internet-Draft is submitted in full conformance with the | This is an Internet Standards Track document. | |||
| provisions of BCP 78 and BCP 79. | ||||
| Internet-Drafts are working documents of the Internet Engineering | ||||
| Task Force (IETF). Note that other groups may also distribute | ||||
| working documents as Internet-Drafts. The list of current Internet- | ||||
| Drafts is at https://datatracker.ietf.org/drafts/current/. | ||||
| Internet-Drafts are draft documents valid for a maximum of six months | This document is a product of the Internet Engineering Task Force | |||
| and may be updated, replaced, or obsoleted by other documents at any | (IETF). It represents the consensus of the IETF community. It has | |||
| time. It is inappropriate to use Internet-Drafts as reference | received public review and has been approved for publication by the | |||
| material or to cite them other than as "work in progress." | Internet Engineering Steering Group (IESG). Further information on | |||
| Internet Standards is available in Section 2 of RFC 7841. | ||||
| This Internet-Draft will expire on 4 August 2023. | Information about the current status of this document, any errata, | |||
| and how to provide feedback on it may be obtained at | ||||
| https://www.rfc-editor.org/info/rfc9438. | ||||
| Copyright Notice | Copyright Notice | |||
| Copyright (c) 2023 IETF Trust and the persons identified as the | Copyright (c) 2023 IETF Trust and the persons identified as the | |||
| document authors. All rights reserved. | document authors. All rights reserved. | |||
| This document is subject to BCP 78 and the IETF Trust's Legal | This document is subject to BCP 78 and the IETF Trust's Legal | |||
| Provisions Relating to IETF Documents (https://trustee.ietf.org/ | Provisions Relating to IETF Documents | |||
| license-info) in effect on the date of publication of this document. | (https://trustee.ietf.org/license-info) in effect on the date of | |||
| Please review these documents carefully, as they describe your rights | publication of this document. Please review these documents | |||
| and restrictions with respect to this document. Code Components | carefully, as they describe your rights and restrictions with respect | |||
| extracted from this document must include Revised BSD License text as | to this document. Code Components extracted from this document must | |||
| described in Section 4.e of the Trust Legal Provisions and are | include Revised BSD License text as described in Section 4.e of the | |||
| provided without warranty as described in the Revised BSD License. | Trust Legal Provisions and are provided without warranty as described | |||
| in the Revised BSD License. | ||||
| Table of Contents | Table of Contents | |||
| 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 4 | 1. Introduction | |||
| 2. Conventions . . . . . . . . . . . . . . . . . . . . . . . . . 6 | 2. Conventions | |||
| 3. Design Principles of CUBIC . . . . . . . . . . . . . . . . . 6 | 3. Design Principles of CUBIC | |||
| 3.1. Principle 1 for the CUBIC Increase Function . . . . . . . 7 | 3.1. Principle 1 for the CUBIC Increase Function | |||
| 3.2. Principle 2 for Reno-Friendliness . . . . . . . . . . . . 7 | 3.2. Principle 2 for Reno-Friendliness | |||
| 3.3. Principle 3 for RTT Fairness . . . . . . . . . . . . . . 8 | 3.3. Principle 3 for RTT-Fairness | |||
| 3.4. Principle 4 for the CUBIC Decrease Factor . . . . . . . . 8 | 3.4. Principle 4 for the CUBIC Decrease Factor | |||
| 4. CUBIC Congestion Control . . . . . . . . . . . . . . . . . . 9 | 4. CUBIC Congestion Control | |||
| 4.1. Definitions . . . . . . . . . . . . . . . . . . . . . . . 9 | 4.1. Definitions | |||
| 4.1.1. Constants of Interest . . . . . . . . . . . . . . . . 9 | 4.1.1. Constants of Interest | |||
| 4.1.2. Variables of Interest . . . . . . . . . . . . . . . . 10 | 4.1.2. Variables of Interest | |||
| 4.2. Window Increase Function . . . . . . . . . . . . . . . . 11 | 4.2. Window Increase Function | |||
| 4.3. Reno-Friendly Region . . . . . . . . . . . . . . . . . . 12 | 4.3. Reno-Friendly Region | |||
| 4.4. Concave Region . . . . . . . . . . . . . . . . . . . . . 14 | 4.4. Concave Region | |||
| 4.5. Convex Region . . . . . . . . . . . . . . . . . . . . . . 14 | 4.5. Convex Region | |||
| 4.6. Multiplicative Decrease . . . . . . . . . . . . . . . . . 15 | 4.6. Multiplicative Decrease | |||
| 4.7. Fast Convergence . . . . . . . . . . . . . . . . . . . . 16 | 4.7. Fast Convergence | |||
| 4.8. Timeout . . . . . . . . . . . . . . . . . . . . . . . . . 17 | 4.8. Timeout | |||
| 4.9. Spurious Congestion Events . . . . . . . . . . . . . . . 17 | 4.9. Spurious Congestion Events | |||
| 4.9.1. Spurious timeout . . . . . . . . . . . . . . . . . . 18 | 4.9.1. Spurious Timeouts | |||
| 4.9.2. Spurious loss detected by acknowledgments . . . . . . 18 | 4.9.2. Spurious Fast Retransmits | |||
| 4.10. Slow Start . . . . . . . . . . . . . . . . . . . . . . . 19 | 4.10. Slow Start | |||
| 5. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 20 | 5. Discussion | |||
| 5.1. Fairness to Reno . . . . . . . . . . . . . . . . . . . . 21 | 5.1. Fairness to Reno | |||
| 5.2. Using Spare Capacity . . . . . . . . . . . . . . . . . . 23 | 5.2. Using Spare Capacity | |||
| 5.3. Difficult Environments . . . . . . . . . . . . . . . . . 24 | 5.3. Difficult Environments | |||
| 5.4. Investigating a Range of Environments . . . . . . . . . . 24 | 5.4. Investigating a Range of Environments | |||
| 5.5. Protection against Congestion Collapse . . . . . . . . . 24 | 5.5. Protection against Congestion Collapse | |||
| 5.6. Fairness within the Alternative Congestion Control | 5.6. Fairness within the Alternative Congestion Control | |||
| Algorithm . . . . . . . . . . . . . . . . . . . . . . . 25 | Algorithm | |||
| 5.7. Performance with Misbehaving Nodes and Outside | 5.7. Performance with Misbehaving Nodes and Outside Attackers | |||
| Attackers . . . . . . . . . . . . . . . . . . . . . . . 25 | 5.8. Behavior for Application-Limited Flows | |||
| 5.8. Behavior for Application-Limited Flows . . . . . . . . . 25 | 5.9. Responses to Sudden or Transient Events | |||
| 5.9. Responses to Sudden or Transient Events . . . . . . . . . 25 | 5.10. Incremental Deployment | |||
| 5.10. Incremental Deployment . . . . . . . . . . . . . . . . . 26 | 6. Security Considerations | |||
| 6. Security Considerations . . . . . . . . . . . . . . . . . . . 26 | 7. IANA Considerations | |||
| 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 26 | 8. References | |||
| 8. References . . . . . . . . . . . . . . . . . . . . . . . . . 26 | 8.1. Normative References | |||
| 8.1. Normative References . . . . . . . . . . . . . . . . . . 26 | 8.2. Informative References | |||
| 8.2. Informative References . . . . . . . . . . . . . . . . . 28 | Appendix A. Evolution of CUBIC since the Original Paper | |||
| Appendix A. Acknowledgments . . . . . . . . . . . . . . . . . . 31 | Appendix B. Proof of the Average CUBIC Window Size | |||
| Appendix B. Evolution of CUBIC . . . . . . . . . . . . . . . . . 32 | Acknowledgments | |||
| B.1. Since draft-ietf-tcpm-rfc8312bis-14 . . . . . . . . . . . 32 | Authors' Addresses | |||
| B.2. Since draft-ietf-tcpm-rfc8312bis-13 . . . . . . . . . . . 32 | ||||
| B.3. Since draft-ietf-tcpm-rfc8312bis-12 . . . . . . . . . . . 32 | ||||
| B.4. Since draft-ietf-tcpm-rfc8312bis-11 . . . . . . . . . . . 32 | ||||
| B.5. Since draft-ietf-tcpm-rfc8312bis-10 . . . . . . . . . . . 32 | ||||
| B.6. Since draft-ietf-tcpm-rfc8312bis-09 . . . . . . . . . . . 32 | ||||
| B.7. Since draft-ietf-tcpm-rfc8312bis-08 . . . . . . . . . . . 32 | ||||
| B.8. Since draft-ietf-tcpm-rfc8312bis-07 . . . . . . . . . . . 33 | ||||
| B.9. Since draft-ietf-tcpm-rfc8312bis-06 . . . . . . . . . . . 33 | ||||
| B.10. Since draft-ietf-tcpm-rfc8312bis-05 . . . . . . . . . . . 33 | ||||
| B.11. Since draft-ietf-tcpm-rfc8312bis-04 . . . . . . . . . . . 33 | ||||
| B.12. Since draft-ietf-tcpm-rfc8312bis-03 . . . . . . . . . . . 34 | ||||
| B.13. Since draft-ietf-tcpm-rfc8312bis-02 . . . . . . . . . . . 34 | ||||
| B.14. Since draft-ietf-tcpm-rfc8312bis-01 . . . . . . . . . . . 35 | ||||
| B.15. Since draft-ietf-tcpm-rfc8312bis-00 . . . . . . . . . . . 35 | ||||
| B.16. Since draft-eggert-tcpm-rfc8312bis-03 . . . . . . . . . . 35 | ||||
| B.17. Since draft-eggert-tcpm-rfc8312bis-02 . . . . . . . . . . 35 | ||||
| B.18. Since draft-eggert-tcpm-rfc8312bis-01 . . . . . . . . . . 35 | ||||
| B.19. Since draft-eggert-tcpm-rfc8312bis-00 . . . . . . . . . . 36 | ||||
| B.20. Since RFC8312 . . . . . . . . . . . . . . . . . . . . . . 36 | ||||
| B.21. Since the Original Paper . . . . . . . . . . . . . . . . 37 | ||||
| Appendix C. Proof of the Average CUBIC Window Size . . . . . . . 37 | ||||
| Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 40 | ||||
| 1. Introduction | 1. Introduction | |||
| CUBIC has been adopted as the default TCP congestion control | CUBIC has been adopted as the default TCP congestion control | |||
| algorithm in the Linux, Windows, and Apple stacks, and has been used | algorithm in the Linux, Windows, and Apple stacks, and has been used | |||
| and deployed globally. Extensive, decade-long deployment experience | and deployed globally. Extensive, decade-long deployment experience | |||
| in vastly different Internet scenarios has convincingly demonstrated | in vastly different Internet scenarios has convincingly demonstrated | |||
| that CUBIC is safe for deployment on the global Internet and delivers | that CUBIC is safe for deployment on the global Internet and delivers | |||
| substantial benefits over classical Reno congestion control | substantial benefits over classical Reno congestion control | |||
| [RFC5681]. It is therefore to be regarded as the currently most | [RFC5681]. It is therefore to be regarded as the currently most | |||
| widely deployed standard for TCP congestion control. CUBIC can also | widely deployed standard for TCP congestion control. CUBIC can also | |||
| be used for other transport protocols such as QUIC [RFC9000] and SCTP | be used for other transport protocols such as QUIC [RFC9000] and the | |||
| [RFC9260] as a default congestion controller. | Stream Control Transmission Protocol (SCTP) [RFC9260] as a default | |||
| congestion controller. | ||||
| The design of CUBIC was motivated by the well-documented problem | The design of CUBIC was motivated by the well-documented problem | |||
| classical Reno TCP has with low utilization over fast and long- | classical Reno TCP has with low utilization over fast and long- | |||
| distance networks [K03][RFC3649]. This problem arises from a slow | distance networks [K03] [RFC3649]. This problem arises from a slow | |||
| increase of the congestion window following a congestion event in a | increase of the congestion window (cwnd) following a congestion event | |||
| network with a large bandwidth-delay product (BDP). [HLRX07] | in a network with a large bandwidth-delay product (BDP). [HLRX07] | |||
| indicates that this problem is frequently observed even in the range | indicates that this problem is frequently observed even in the range | |||
| of congestion window sizes over several hundreds of packets. This | of congestion window sizes over several hundreds of packets. This | |||
| problem is equally applicable to all Reno-style standards and their | problem is equally applicable to all Reno-style standards and their | |||
| variants, including TCP-Reno [RFC5681], TCP-NewReno | variants, including TCP-Reno [RFC5681], TCP-NewReno [RFC6582] | |||
| [RFC6582][RFC6675], SCTP [RFC9260], TFRC [RFC5348], and QUIC | [RFC6675], SCTP [RFC9260], TCP Friendly Rate Control (TFRC) | |||
| congestion control [RFC9002], which use the same linear increase | [RFC5348], and QUIC congestion control [RFC9002], which use the same | |||
| function for window growth. All Reno-style standards and their | linear increase function for window growth. All Reno-style standards | |||
| variants are collectively referred to as "Reno" in this document. | and their variants are collectively referred to as "Reno" in this | |||
| document. | ||||
| CUBIC, originally proposed in [HRX08], is a modification to the | CUBIC, originally proposed in [HRX08], is a modification to the | |||
| congestion control algorithm of classical Reno to remedy this | congestion control algorithm of classical Reno to remedy this | |||
| problem. Specifically, CUBIC uses a cubic function instead of the | problem. Specifically, CUBIC uses a cubic function instead of the | |||
| linear window increase function of Reno to improve scalability and | linear window increase function of Reno to improve scalability and | |||
| stability under fast and long-distance networks. | stability under fast and long-distance networks. | |||
| This document updates the specification of CUBIC to include | This document updates the specification of CUBIC to include | |||
| algorithmic improvements based on the Linux, Windows, and Apple | algorithmic improvements based on the Linux, Windows, and Apple | |||
| implementations and recent academic work. Based on the extensive | implementations and recent academic work. Based on the extensive | |||
| deployment experience with CUBIC, it also moves the specification to | deployment experience with CUBIC, it also moves the specification to | |||
| the Standards Track, obsoleting [RFC8312]. This requires an update | the Standards Track, obsoleting [RFC8312]. This requires an update | |||
| to Section 3 of [RFC5681], which limits the aggressiveness of Reno | to Section 3 of [RFC5681], which limits the aggressiveness of Reno | |||
| TCP implementations. Since CUBIC is occasionally more aggressive | TCP implementations. Since CUBIC is occasionally more aggressive | |||
| than the [RFC5681] algorithms, this document updates the first | than the algorithms defined in [RFC5681], this document updates the | |||
| paragraph of Section 3 of [RFC5681], replacing it with a normative | first paragraph of Section 3 of [RFC5681], replacing it with a | |||
| reference to guideline (1) in Section 3 of [RFC5033], which allows | normative reference to guideline (1) in Section 3 of [RFC5033], which | |||
| for CUBIC's behavior as defined in this document. | allows for CUBIC's behavior as defined in this document. | |||
| Specifically, CUBIC may increase the congestion window more | Specifically, CUBIC may increase the congestion window more | |||
| aggressively than Reno during the congestion avoidance phase. | aggressively than Reno during the congestion avoidance phase. | |||
| According to [RFC5681], during congestion avoidance, the sender must | According to [RFC5681], during congestion avoidance, the sender must | |||
| not increment cwnd by more than SMSS bytes once per RTT, whereas | not increment cwnd by more than Sender Maximum Segment Size (SMSS) | |||
| CUBIC may increase cwnd much more aggressively. Additionally, CUBIC | bytes once per round-trip time (RTT), whereas CUBIC may increase cwnd | |||
| recommends the HyStart++ algorithm [I-D.ietf-tcpm-hystartplusplus] | much more aggressively. Additionally, CUBIC recommends the HyStart++ | |||
| for slow start, which allows for cwnd increases of more than SMSS | algorithm [RFC9406] for slow start, which allows for cwnd increases | |||
| bytes for incoming acknowledgments during slow start, while this | of more than SMSS bytes for incoming acknowledgments during slow | |||
| behavior is not allowed as part of [RFC5681] standard. | start, while this behavior is not allowed as part of the standard | |||
| behavior prescribed by [RFC5681]. | ||||
| Binary Increase Congestion Control (BIC-TCP) [XHR04], a predecessor | Binary Increase Congestion Control (BIC-TCP) [XHR04], a predecessor | |||
| of CUBIC, was selected as the default TCP congestion control | of CUBIC, was selected as the default TCP congestion control | |||
| algorithm by Linux in the year 2005 and had been used for several | algorithm by Linux in the year 2005 and had been used for several | |||
| years by the Internet community at large. | years by the Internet community at large. | |||
| CUBIC uses a similar window increase function as BIC-TCP and is | CUBIC uses a window increase function similar to BIC-TCP and is | |||
| designed to be less aggressive and fairer to Reno in bandwidth usage | designed to be less aggressive and fairer to Reno in bandwidth usage | |||
| than BIC-TCP while maintaining the strengths of BIC-TCP such as | than BIC-TCP while maintaining the strengths of BIC-TCP such as | |||
| stability, window scalability, and round-trip time (RTT) fairness. | stability, window scalability, and RTT-fairness. | |||
| [RFC5033] documents the IETF's best current practices for specifying | [RFC5033] documents the IETF's best current practices for specifying | |||
| new congestion control algorithms, specifically, ones that differ | new congestion control algorithms, specifically those that differ | |||
| from the general congestion control principles outlined in [RFC2914]. | from the general congestion control principles outlined in [RFC2914]. | |||
| It describes what type of evaluation is expected by the IETF to | It describes what type of evaluation is expected by the IETF to | |||
| understand the suitability of a new congestion control algorithm and | understand the suitability of a new congestion control algorithm and | |||
| the process to enable a specification to be approved for widespread | the process of enabling a specification to be approved for widespread | |||
| deployment in the global Internet. | deployment in the global Internet. | |||
| There are areas in which CUBIC differs from the congestion control | There are areas in which CUBIC differs from the congestion control | |||
| algorithms previously published in standards-track RFCs; those | algorithms previously published in Standards Track RFCs; those | |||
| changes are specified in this document. However, it is not obvious | changes are specified in this document. However, it is not obvious | |||
| that these changes go beyond the general congestion control | that these changes go beyond the general congestion control | |||
| principles outlined in [RFC2914], so the process in [RFC5033] may not | principles outlined in [RFC2914], so the process documented in | |||
| apply. | [RFC5033] may not apply. | |||
| Also, the wide deployment of CUBIC on the Internet was driven by | Also, the wide deployment of CUBIC on the Internet was driven by | |||
| direct adoption in most of the popular operating systems, and did not | direct adoption in most of the popular operating systems and did not | |||
| follow the practices documented in [RFC5033]. However, due to the | follow the practices documented in [RFC5033]. However, due to the | |||
| resulting Internet-scale deployment experience over a long period of | resulting Internet-scale deployment experience over a long period of | |||
| time, the IETF has determined that CUBIC may be published as a | time, the IETF determined that CUBIC could be published as a | |||
| standards-track specification. This decision by the IETF does not | Standards Track specification. This decision by the IETF does not | |||
| alter the general guidance in [RFC2914]. | alter the general guidance provided in [RFC2914]. | |||
| The following sections first briefly explain the design principles of | The following sections | |||
| CUBIC, provide the exact specification of CUBIC, and finally discuss | ||||
| the safety features of CUBIC following the guidelines specified in | 1. briefly explain the design principles of CUBIC, | |||
| [RFC5033]. | ||||
| 2. provide the exact specification of CUBIC, and | ||||
| 3. discuss the safety features of CUBIC, following the guidelines | ||||
| specified in [RFC5033]. | ||||
| 2. Conventions | 2. Conventions | |||
| The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", | The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", | |||
| "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and | "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and | |||
| "OPTIONAL" in this document are to be interpreted as described in | "OPTIONAL" in this document are to be interpreted as described in | |||
| BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all | BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all | |||
| capitals, as shown here. | capitals, as shown here. | |||
| 3. Design Principles of CUBIC | 3. Design Principles of CUBIC | |||
| skipping to change at page 6, line 43 ¶ | skipping to change at line 231 ¶ | |||
| convex function. | convex function. | |||
| Principle 2: To be Reno-friendly, CUBIC is designed to behave like | Principle 2: To be Reno-friendly, CUBIC is designed to behave like | |||
| Reno in networks with short RTTs and small bandwidth where Reno | Reno in networks with short RTTs and small bandwidth where Reno | |||
| performs well. | performs well. | |||
| Principle 3: For RTT-fairness, CUBIC is designed to achieve linear | Principle 3: For RTT-fairness, CUBIC is designed to achieve linear | |||
| bandwidth sharing among flows with different RTTs. | bandwidth sharing among flows with different RTTs. | |||
| Principle 4: CUBIC appropriately sets its multiplicative window | Principle 4: CUBIC appropriately sets its multiplicative window | |||
| decrease factor in order to balance between the scalability and | decrease factor in order to achieve a balance between scalability | |||
| convergence speed. | and convergence speed. | |||
| 3.1. Principle 1 for the CUBIC Increase Function | 3.1. Principle 1 for the CUBIC Increase Function | |||
| For better network utilization and stability, CUBIC [HRX08] uses a | For better network utilization and stability, CUBIC [HRX08] uses a | |||
| cubic window increase function in terms of the elapsed time from the | cubic window increase function in terms of the elapsed time from the | |||
| last congestion event. While most alternative congestion control | last congestion event. While most congestion control algorithms that | |||
| algorithms to Reno increase the congestion window using convex | provide alternatives to Reno increase the congestion window using | |||
| functions, CUBIC uses both the concave and convex profiles of a cubic | convex functions, CUBIC uses both the concave and convex profiles of | |||
| function for window growth. | a cubic function for window growth. | |||
| After a window reduction in response to a congestion event detected | After a window reduction in response to a congestion event detected | |||
| by duplicate ACKs, Explicit Congestion Notification-Echo (ECN-Echo, | by duplicate acknowledgments (ACKs), Explicit Congestion | |||
| ECE) ACKs [RFC3168], TCP RACK [RFC8985] or QUIC loss detection | Notification-Echo (ECN-Echo (ECE)) ACKs [RFC3168], RACK-TLP for TCP | |||
| [RFC9002], CUBIC remembers the congestion window size at which it | [RFC8985], or QUIC loss detection [RFC9002], CUBIC remembers the | |||
| received the congestion event and performs a multiplicative decrease | congestion window size at which it received the congestion event and | |||
| of the congestion window. When CUBIC enters into congestion | performs a multiplicative decrease of the congestion window. When | |||
| avoidance, it starts to increase the congestion window using the | CUBIC enters into congestion avoidance, it starts to increase the | |||
| concave profile of the cubic function. The cubic function is set to | congestion window using the concave profile of the cubic function. | |||
| have its plateau at the remembered congestion window size, so that | The cubic function is set to have its plateau at the remembered | |||
| the concave window increase continues until then. After that, the | congestion window size, so that the concave window increase continues | |||
| cubic function turns into a convex profile and the convex window | until then. After that, the cubic function turns into a convex | |||
| increase begins. | profile and the convex window increase begins. | |||
| This style of window adjustment (concave and then convex) improves | This style of window adjustment (concave and then convex) improves | |||
| the algorithm stability while maintaining high network utilization | algorithm stability while maintaining high network utilization | |||
| [CEHRX09]. This is because the window size remains almost constant, | [CEHRX09]. This is because the window size remains almost constant, | |||
| forming a plateau around the remembered congestion window size of the | forming a plateau around the remembered congestion window size of the | |||
| last congestion event, where network utilization is deemed highest. | last congestion event, where network utilization is deemed highest. | |||
| Under steady state, most window size samples of CUBIC are close to | Under steady state, most window size samples of CUBIC are close to | |||
| that remembered congestion window size, thus promoting high network | that remembered congestion window size, thus promoting high network | |||
| utilization and stability. | utilization and stability. | |||
| Note that congestion control algorithms that only use convex | Note that congestion control algorithms that only use convex | |||
| functions to increase the congestion window size have their maximum | functions to increase the congestion window size have their maximum | |||
| increments around the remembered congestion window size of the last | increments around the remembered congestion window size of the last | |||
| congestion event, and thus introduce many packet bursts around the | congestion event and thus introduce many packet bursts around the | |||
| saturation point of the network, likely causing frequent global loss | saturation point of the network, likely causing frequent global loss | |||
| synchronizations. | synchronizations. | |||
| 3.2. Principle 2 for Reno-Friendliness | 3.2. Principle 2 for Reno-Friendliness | |||
| CUBIC promotes per-flow fairness to Reno. Note that Reno performs | CUBIC promotes per-flow fairness to Reno. Note that Reno performs | |||
| well over paths with small bandwidth-delay products, and only | well over paths with small BDPs and only experiences problems when | |||
| experiences problems when attempting to increase bandwidth | attempting to increase bandwidth utilization on paths with large | |||
| utilization on paths with large bandwidth-delay products. | BDPs. | |||
| A congestion control algorithm designed to be friendly to Reno on a | A congestion control algorithm designed to be friendly to Reno on a | |||
| per-flow basis must increase its congestion window less aggressively | per-flow basis must increase its congestion window less aggressively | |||
| in small-BDP networks than in large-BDP networks. | in small-BDP networks than in large-BDP networks. | |||
| The aggressiveness of CUBIC mainly depends on the maximum window size | The aggressiveness of CUBIC mainly depends on the maximum window size | |||
| before a window reduction, which is smaller in small-BDP networks | before a window reduction, which is smaller in small-BDP networks | |||
| than in large-BDP networks. Thus, CUBIC increases its congestion | than in large-BDP networks. Thus, CUBIC increases its congestion | |||
| window less aggressively in small-BDP networks than in large-BDP | window less aggressively in small-BDP networks than in large-BDP | |||
| networks. | networks. | |||
| Furthermore, in cases when the cubic function of CUBIC would increase | Furthermore, in cases when the cubic function of CUBIC would increase | |||
| the congestion window less aggressively than Reno, CUBIC simply | the congestion window less aggressively than Reno, CUBIC simply | |||
| follows the window size of Reno to ensure that CUBIC achieves at | follows the window size of Reno to ensure that CUBIC achieves at | |||
| least the same throughput as Reno in small-BDP networks. The region | least the same throughput as Reno in small-BDP networks. The region | |||
| where CUBIC behaves like Reno is called the "Reno-friendly region". | where CUBIC behaves like Reno is called the "Reno-friendly region". | |||
| 3.3. Principle 3 for RTT Fairness | 3.3. Principle 3 for RTT-Fairness | |||
| Two CUBIC flows with different RTTs have a throughput ratio that is | Two CUBIC flows with different RTTs have a throughput ratio that is | |||
| linearly proportional to the inverse of their RTT ratio, where the | linearly proportional to the inverse of their RTT ratio, where the | |||
| throughput of a flow is approximately the size of its congestion | throughput of a flow is approximately the size of its congestion | |||
| window divided by its RTT. | window divided by its RTT. | |||
| Specifically, CUBIC maintains a window increase rate independent of | Specifically, CUBIC maintains a window increase rate that is | |||
| RTTs outside the Reno-friendly region, and thus flows with different | independent of RTTs outside the Reno-friendly region, and thus flows | |||
| RTTs have similar congestion window sizes under steady state when | with different RTTs have similar congestion window sizes under steady | |||
| they operate outside the Reno-friendly region. | state when they operate outside the Reno-friendly region. | |||
| This notion of a linear throughput ratio is similar to that of Reno | This notion of a linear throughput ratio is similar to that of Reno | |||
| under an asynchronous loss model, where flows with different RTTs | under an asynchronous loss model, where flows with different RTTs | |||
| have the same packet loss rate but experience loss events at | have the same packet loss rate but experience loss events at | |||
| different times. However, under a synchronous loss model, where | different times. However, under a synchronous loss model, where | |||
| flows with different RTTs experience loss events at the same time but | flows with different RTTs experience loss events at the same time but | |||
| have different packet loss rates, the throughput ratio of Reno flows | have different packet loss rates, the throughput ratio of Reno flows | |||
| with different RTTs is quadratically proportional to the inverse of | with different RTTs is quadratically proportional to the inverse of | |||
| their RTT ratio [XHR04]. | their RTT ratio [XHR04]. | |||
| CUBIC always ensures a linear throughput ratio independent of the | CUBIC always ensures a linear throughput ratio that is independent of | |||
| loss environment. This is an improvement over Reno. While there is | the loss environment. This is an improvement over Reno. While there | |||
| no consensus on the optimal throughput ratio for different RTT flows, | is no consensus on the optimal throughput ratio for different RTT | |||
| over wired Internet paths, use of a linear throughput ratio seems | flows, over wired Internet paths, use of a linear throughput ratio | |||
| more reasonable than equal throughputs (i.e., the same throughput for | seems more reasonable than equal throughputs (i.e., the same | |||
| flows with different RTTs) or a higher-order throughput ratio (e.g., | throughput for flows with different RTTs) or a higher-order | |||
| a quadratical throughput ratio of Reno in synchronous loss | throughput ratio (e.g., a quadratic throughput ratio of Reno in | |||
| environments). | synchronous loss environments). | |||
| 3.4. Principle 4 for the CUBIC Decrease Factor | 3.4. Principle 4 for the CUBIC Decrease Factor | |||
| To balance between scalability and convergence speed, CUBIC sets the | To achieve a balance between scalability and convergence speed, CUBIC | |||
| multiplicative window decrease factor to 0.7, whereas Reno uses 0.5. | sets the multiplicative window decrease factor to 0.7, whereas Reno | |||
| uses 0.5. | ||||
| While this improves the scalability of CUBIC, a side effect of this | While this improves the scalability of CUBIC, a side effect of this | |||
| decision is slower convergence, especially under low statistical | decision is slower convergence, especially under low statistical | |||
| multiplexing. This design choice is following the observation that | multiplexing. This design choice is following the observation that | |||
| HighSpeed TCP (HSTCP) [RFC3649] and other approaches (e.g., [GV02]) | HighSpeed TCP (HSTCP) [RFC3649] and other approaches (e.g., [GV02]) | |||
| made: the current Internet becomes more asynchronous with less | made: the current Internet becomes more asynchronous with less | |||
| frequent loss synchronizations under high statistical multiplexing. | frequent loss synchronizations under high statistical multiplexing. | |||
| In such environments, even strict Multiplicative-Increase | In such environments, even strict Multiplicative-Increase | |||
| Multiplicative-Decrease (MIMD) can converge. CUBIC flows with the | Multiplicative-Decrease (MIMD) can converge. CUBIC flows with the | |||
| same RTT always converge to the same throughput independent of | same RTT always converge to the same throughput independently of | |||
| statistical multiplexing, thus achieving intra-algorithm fairness. | statistical multiplexing, thus achieving intra-algorithm fairness. | |||
| In environments with sufficient statistical multiplexing, the | In environments with sufficient statistical multiplexing, the | |||
| convergence speed of CUBIC is reasonable. | convergence speed of CUBIC is reasonable. | |||
| 4. CUBIC Congestion Control | 4. CUBIC Congestion Control | |||
| This section discusses how the congestion window is updated during | This section discusses how the congestion window is updated during | |||
| the different stages of the CUBIC congestion controller. | the different stages of the CUBIC congestion controller. | |||
| 4.1. Definitions | 4.1. Definitions | |||
| The unit of all window sizes in this document is segments of the | The unit of all window sizes in this document is segments of the | |||
| maximum segment size (MSS), and the unit of all times is seconds. | SMSS, and the unit of all times is seconds. Implementations can use | |||
| Implementations can use bytes to express window sizes, which would | bytes to express window sizes, which would require factoring in the | |||
| require factoring in the maximum segment size wherever necessary and | SMSS wherever necessary and replacing _segments_acked_ (Figure 4) | |||
| replacing _segments_acked_ with the number of bytes acknowledged in | with the number of acknowledged bytes. | |||
| Figure 4. | ||||
| 4.1.1. Constants of Interest | 4.1.1. Constants of Interest | |||
| β__cubic_: CUBIC multiplicative decrease factor as described in | * β__cubic_: CUBIC multiplicative decrease factor as described in | |||
| Section 4.6. | Section 4.6. | |||
| α__cubic_: CUBIC additive increase factor used in Reno-friendly | * α__cubic_: CUBIC additive increase factor used in the Reno- | |||
| region as described in Section 4.3. | friendly region as described in Section 4.3. | |||
| _C_: constant that determines the aggressiveness of CUBIC in | * _C_: Constant that determines the aggressiveness of CUBIC in | |||
| competing with other congestion control algorithms in high-BDP | competing with other congestion control algorithms in high-BDP | |||
| networks. Please see Section 5 for more explanation on how it is | networks. Please see Section 5 for more explanation on how it is | |||
| set. The unit for _C_ is | set. The unit for _C_ is | |||
| segment | segment | |||
| ─────── | ─────── | |||
| 3 | 3 | |||
| second | second | |||
| 4.1.2. Variables of Interest | 4.1.2. Variables of Interest | |||
| This section defines the variables required to implement CUBIC: | This section defines the variables required to implement CUBIC: | |||
| _RTT_: Smoothed round-trip time in seconds, calculated as described | * _RTT_: Smoothed round-trip time in seconds, calculated as | |||
| in [RFC6298]. | described in [RFC6298]. | |||
| _cwnd_: Current congestion window in segments. | * _cwnd_: Current congestion window in segments. | |||
| _ssthresh_: Current slow start threshold in segments. | * _ssthresh_: Current slow start threshold in segments. | |||
| _cwnd_prior_: Size of _cwnd_ in segments at the time of setting | * _cwnd_prior_: Size of _cwnd_ in segments at the time of setting | |||
| _ssthresh_ most recently, either upon exiting the first slow start, | _ssthresh_ most recently, either upon exiting the first slow start | |||
| or just before _cwnd_ was reduced in the last congestion event. | or just before _cwnd_ was reduced in the last congestion event. | |||
| _W_max_: Size of _cwnd_ in segments just before _cwnd_ was reduced in | * _W_max_: Size of _cwnd_ in segments just before _cwnd_ was reduced | |||
| the last congestion event when fast convergence is disabled (same as | in the last congestion event when fast convergence is disabled | |||
| _cwnd_prior_ on a congestion event). However, if fast convergence is | (same as _cwnd_prior_ on a congestion event). However, if fast | |||
| enabled, _W_max_ may be further reduced based on the current | convergence is enabled, _W_max_ may be further reduced based on | |||
| saturation point. | the current saturation point. | |||
| _K_: The time period in seconds it takes to increase the congestion | * _K_: The time period in seconds it takes to increase the | |||
| window size at the beginning of the current congestion avoidance | congestion window size at the beginning of the current congestion | |||
| stage to _W_max_. | avoidance stage to _W_max_. | |||
| _t_current_: Current time of the system in seconds. | * _t_current_: Current time of the system in seconds. | |||
| _t_epoch_: The time in seconds at which the current congestion | * _t_epoch_: The time in seconds at which the current congestion | |||
| avoidance stage started. | avoidance stage started. | |||
| _cwnd_epoch_: The _cwnd_ at the beginning of the current congestion | * _cwnd_epoch_: The _cwnd_ at the beginning of the current | |||
| avoidance stage, i.e., at time _t_epoch_. | congestion avoidance stage, i.e., at time _t_epoch_. | |||
| W_cubic(_t_): The congestion window in segments at time _t_ in | * W_cubic(_t_): The congestion window in segments at time _t_ in | |||
| seconds based on the cubic increase function, as described in | seconds based on the cubic increase function, as described in | |||
| Section 4.2. | Section 4.2. | |||
| _target_: Target value of congestion window in segments after the | * _target_: Target value of the congestion window in segments after | |||
| next RTT, that is, W_cubic(_t_ + _RTT_), as described in Section 4.2. | the next RTT -- that is, W_cubic(_t_ + _RTT_), as described in | |||
| Section 4.2. | ||||
| _W_est_: An estimate for the congestion window in segments in the | * _W_est_: An estimate for the congestion window in segments in the | |||
| Reno-friendly region, that is, an estimate for the congestion window | Reno-friendly region -- that is, an estimate for the congestion | |||
| of Reno. | window of Reno. | |||
| _segments_acked_: Number of MSS-sized segments acked when a "new ACK" | * _segments_acked_: Number of SMSS-sized segments acked when a "new | |||
| is received, i.e., an ACK that cumulatively acknowledges the delivery | ACK" is received, i.e., an ACK that cumulatively acknowledges the | |||
| of new data. This number will be a decimal value when a new ACK | delivery of previously unacknowledged data. This number will be a | |||
| acknowledges an amount of data that is not MSS-sized. Specifically, | decimal value when a new ACK acknowledges an amount of data that | |||
| it can be less than 1 when a new ACK acknowledges a segment smaller | is not SMSS-sized. Specifically, it can be less than 1 when a new | |||
| than the MSS. | ACK acknowledges a segment smaller than the SMSS. | |||
| 4.2. Window Increase Function | 4.2. Window Increase Function | |||
| CUBIC maintains the acknowledgment (ACK) clocking of Reno by | CUBIC maintains the ACK clocking of Reno by increasing the congestion | |||
| increasing the congestion window only at the reception of a new ACK. | window only at the reception of a new ACK. It does not make any | |||
| It does not make any changes to the TCP Fast Recovery and Fast | changes to the TCP Fast Recovery and Fast Retransmit algorithms | |||
| Retransmit algorithms [RFC6582][RFC6675]. | [RFC6582] [RFC6675]. | |||
| During congestion avoidance, after a congestion event is detected by | During congestion avoidance, after a congestion event is detected as | |||
| mechanisms described in Section 3.1, CUBIC uses a window increase | described in Section 3.1, CUBIC uses a window increase function | |||
| function different from Reno. | different from Reno. | |||
| CUBIC uses the following window increase function: | CUBIC uses the following window increase function: | |||
| 3 | 3 | |||
| W (t) = C * (t - K) + W | W (t) = C * (t - K) + W | |||
| cubic max | cubic max | |||
| Figure 1 | Figure 1 | |||
| where _t_ is the elapsed time in seconds from the beginning of the | where _t_ is the elapsed time in seconds from the beginning of the | |||
| current congestion avoidance stage, that is, | current congestion avoidance stage -- that is, | |||
| t = t - t | t = t - t | |||
| current epoch | current epoch | |||
| and where _t_epoch_ is the time at which the current congestion | and where _t_epoch_ is the time at which the current congestion | |||
| avoidance stage starts. _K_ is the time period that the above | avoidance stage starts. _K_ is the time period that the above | |||
| function takes to increase the congestion window size at the | function takes to increase the congestion window size at the | |||
| beginning of the current congestion avoidance stage to _W_max_ if | beginning of the current congestion avoidance stage to _W_max_ if | |||
| there are no further congestion events and is calculated using the | there are no further congestion events. _K_ is calculated using the | |||
| following equation: | following equation: | |||
| ┌────────────────┐ | ┌────────────────┐ | |||
| 3 │W - cwnd | 3 │W - cwnd | |||
| ╲ │ max epoch | ╲ │ max epoch | |||
| K = ╲ │──────────────── | K = ╲ │──────────────── | |||
| ╲│ C | ╲│ C | |||
| Figure 2 | Figure 2 | |||
| skipping to change at page 12, line 48 ¶ | skipping to change at line 511 ¶ | |||
| To summarize, CUBIC computes both W_cubic(_t_) and _W_est_ (see | To summarize, CUBIC computes both W_cubic(_t_) and _W_est_ (see | |||
| Section 4.3) on receiving a new ACK in congestion avoidance and | Section 4.3) on receiving a new ACK in congestion avoidance and | |||
| chooses the larger of the two values. | chooses the larger of the two values. | |||
| The next sections describe the exact actions taken by CUBIC in each | The next sections describe the exact actions taken by CUBIC in each | |||
| region. | region. | |||
| 4.3. Reno-Friendly Region | 4.3. Reno-Friendly Region | |||
| Reno performs well in certain types of networks, for example, under | Reno performs well in certain types of networks -- for example, under | |||
| short RTTs and small bandwidths (or small BDPs). In these networks, | short RTTs and small bandwidths (or small BDPs). In these networks, | |||
| CUBIC remains in the Reno-friendly region to achieve at least the | CUBIC remains in the Reno-friendly region to achieve at least the | |||
| same throughput as Reno. | same throughput as Reno. | |||
| The Reno-friendly region is designed according to the analysis in | The Reno-friendly region is designed according to the analysis | |||
| [FHP00], which studies the performance of an AIMD algorithm with an | discussed in [FHP00], which studies the performance of an AIMD | |||
| additive factor of α (segments per _RTT_) and a multiplicative factor | algorithm with an additive factor of α (segments per _RTT_) and a | |||
| of β, denoted by AIMD(α, β). _p_ is the packet loss rate. | multiplicative factor of β, denoted by AIMD(α, β). _p_ is the packet | |||
| Specifically, the average congestion window size of AIMD(α, β) can be | loss rate. Specifically, the average congestion window size of | |||
| calculated using Figure 3. | AIMD(α, β) can be calculated using Figure 3. | |||
| ┌───────────────┐ | ┌───────────────┐ | |||
| │ α * (1 + β) | │ α * (1 + β) | |||
| AVG_AIMD(α, β) = ╲ │─────────────── | AVG_AIMD(α, β) = ╲ │─────────────── | |||
| ╲│2 * (1 - β) * p | ╲│2 * (1 - β) * p | |||
| Figure 3 | Figure 3 | |||
| By the same analysis, to achieve a similar average window size as | By the same analysis, to achieve an average window size similar to | |||
| Reno that uses AIMD(1, 0.5), α must be equal to, | Reno that uses AIMD(1, 0.5), α must be equal to | |||
| 1 - β | 1 - β | |||
| 3 * ───── | 3 * ───── | |||
| 1 + β | 1 + β | |||
| Thus, CUBIC uses Figure 4 to estimate the window size _W_est_ in the | Thus, CUBIC uses Figure 4 to estimate the window size _W_est_ in the | |||
| Reno-friendly region with | Reno-friendly region with | |||
| 1 - β | 1 - β | |||
| cubic | cubic | |||
| α = 3 * ────────── | α = 3 * ────────── | |||
| cubic 1 + β | cubic 1 + β | |||
| cubic | cubic | |||
| which achieves approximately the same average window size as Reno in | which achieves approximately the same average window size as Reno in | |||
| many cases. The model used to calculate α__cubic_ is not absolutely | many cases. The model used to calculate α__cubic_ is not absolutely | |||
| precise, but analysis and simulation in [AIMD-friendliness], as well | precise, but analysis and simulation as discussed in | |||
| as over a decade of experience with CUBIC in the public Internet, | [AIMD-friendliness], as well as over a decade of experience with | |||
| show that this approach produces acceptable levels of rate fairness | CUBIC in the public Internet, show that this approach produces | |||
| between CUBIC and Reno flows. Also, no significant drawbacks of the | acceptable levels of rate fairness between CUBIC and Reno flows. | |||
| model have been reported. However, it would be beneficial to see | Also, no significant drawbacks of the model have been reported. | |||
| continued detailed analysis on it. When receiving a new ACK in | However, continued detailed analysis of this approach would be | |||
| congestion avoidance (where _cwnd_ could be greater than or less than | beneficial. When receiving a new ACK in congestion avoidance (where | |||
| _W_max_), CUBIC checks whether W_cubic(_t_) is less than _W_est_. If | _cwnd_ could be greater than or less than _W_max_), CUBIC checks | |||
| so, CUBIC is in the Reno-friendly region and _cwnd_ SHOULD be set to | whether W_cubic(_t_) is less than _W_est_. If so, CUBIC is in the | |||
| _W_est_ at each reception of a new ACK. | Reno-friendly region and _cwnd_ SHOULD be set to _W_est_ at each | |||
| reception of a new ACK. | ||||
| _W_est_ is set equal to _cwnd_epoch_ at the start of the congestion | _W_est_ is set equal to _cwnd_epoch_ at the start of the congestion | |||
| avoidance stage. After that, on every new ACK, _W_est_ is updated | avoidance stage. After that, on every new ACK, _W_est_ is updated | |||
| using Figure 4. Note that this equation uses _segments_acked_ and | using Figure 4. Note that this equation uses _segments_acked_ and | |||
| _cwnd_ is measured in segments. An implementation that measures | _cwnd_ is measured in segments. An implementation that measures | |||
| _cwnd_ in bytes should adjust the equation accordingly using number | _cwnd_ in bytes should adjust the equation accordingly using the | |||
| of acknowledged bytes and MSS. Also note that this equation works | number of acknowledged bytes and the SMSS. Also note that this | |||
| for connections with enabled or disabled Delayed ACKs [RFC5681], as | equation works for connections with enabled or disabled delayed ACKs | |||
| _segments_acked_ will be different based on the segments actually | [RFC5681], as _segments_acked_ will be different based on the | |||
| acknowledged by a new ACK. | segments actually acknowledged by a new ACK. | |||
| segments_acked | segments_acked | |||
| W = W + α * ────────────── | W = W + α * ────────────── | |||
| est est cubic cwnd | est est cubic cwnd | |||
| Figure 4 | Figure 4 | |||
| Once _W_est_ has grown to reach the _cwnd_ at the time of most | Once _W_est_ has grown to reach the _cwnd_ at the time of most | |||
| recently setting _ssthresh_, that is, _W_est_ >= _cwnd_prior_, the | recently setting _ssthresh_ -- that is, _W_est_ >= _cwnd_prior_ -- | |||
| sender SHOULD set α__cubic_ to 1 to ensure that it can achieve the | the sender SHOULD set α__cubic_ to 1 to ensure that it can achieve | |||
| same congestion window increment rate as Reno, which uses AIMD(1, | the same congestion window increment rate as Reno, which uses AIMD(1, | |||
| 0.5). | 0.5). | |||
| The next two sections assume that CUBIC is not in the Reno-friendly | The next two sections assume that CUBIC is not in the Reno-friendly | |||
| region and uses the window increase function described in | region and uses the window increase function described in | |||
| Section 4.2. Although _cwnd_ is incremented in the same way for both | Section 4.2. Although _cwnd_ is incremented in the same way for both | |||
| concave and convex regions, they are discussed separately to analyze | concave and convex regions, they are discussed separately to analyze | |||
| and understand the difference between the two regions. | and understand the difference between the two regions. | |||
| 4.4. Concave Region | 4.4. Concave Region | |||
| skipping to change at page 15, line 5 ¶ | skipping to change at line 614 ¶ | |||
| When receiving a new ACK in congestion avoidance, if CUBIC is not in | When receiving a new ACK in congestion avoidance, if CUBIC is not in | |||
| the Reno-friendly region and _cwnd_ is larger than or equal to | the Reno-friendly region and _cwnd_ is larger than or equal to | |||
| _W_max_, then CUBIC is in the convex region. | _W_max_, then CUBIC is in the convex region. | |||
| The convex region indicates that the network conditions might have | The convex region indicates that the network conditions might have | |||
| changed since the last congestion event, possibly implying more | changed since the last congestion event, possibly implying more | |||
| available bandwidth after some flow departures. Since the Internet | available bandwidth after some flow departures. Since the Internet | |||
| is highly asynchronous, some amount of perturbation is always | is highly asynchronous, some amount of perturbation is always | |||
| possible without causing a major change in available bandwidth. | possible without causing a major change in available bandwidth. | |||
| Unless it is overridden by the AIMD window increase, CUBIC is very | Unless the cwnd is overridden by the AIMD window increase, CUBIC will | |||
| careful in this region. The convex profile aims to increase the | behave cautiously when operating in this region. The convex profile | |||
| window very slowly at the beginning when _cwnd_ is around _W_max_ and | aims to increase the window very slowly at the beginning when _cwnd_ | |||
| then gradually increases its rate of increase. This region is also | is around _W_max_ and then gradually increases its rate of increase. | |||
| called the "maximum probing phase", since CUBIC is searching for a | This region is also called the "maximum probing phase", since CUBIC | |||
| new _W_max_. In this region, _cwnd_ MUST be incremented by | is searching for a new _W_max_. In this region, _cwnd_ MUST be | |||
| incremented by | ||||
| target - cwnd | target - cwnd | |||
| ───────────── | ───────────── | |||
| cwnd | cwnd | |||
| for each received new ACK, where _target_ is calculated as described | for each received new ACK, where _target_ is calculated as described | |||
| in Section 4.2. | in Section 4.2. | |||
| 4.6. Multiplicative Decrease | 4.6. Multiplicative Decrease | |||
| When a congestion event is detected by mechanisms described in | When a congestion event is detected by the mechanisms described in | |||
| Section 3.1, CUBIC updates _W_max_ and reduces _cwnd_ and _ssthresh_ | Section 3.1, CUBIC updates _W_max_ and reduces _cwnd_ and _ssthresh_ | |||
| immediately as described below. In case of packet loss, the sender | immediately, as described below. In the case of packet loss, the | |||
| MUST reduce _cwnd_ and _ssthresh_ immediately upon entering loss | sender MUST reduce _cwnd_ and _ssthresh_ immediately upon entering | |||
| recovery, similar to [RFC5681] (and [RFC6675]). Note that other | loss recovery, similar to [RFC5681] (and [RFC6675]). Note that other | |||
| mechanisms, such as Proportional Rate Reduction [RFC6937], can be | mechanisms, such as Proportional Rate Reduction [RFC6937], can be | |||
| used to reduce the sending rate during loss recovery more gradually. | used to reduce the sending rate during loss recovery more gradually. | |||
| The parameter β__cubic_ SHOULD be set to 0.7, which is different from | The parameter β__cubic_ SHOULD be set to 0.7, which is different from | |||
| the multiplicative decrease factor used in [RFC5681] (and [RFC6675]) | the multiplicative decrease factor used in [RFC5681] (and [RFC6675]) | |||
| during fast recovery. | during fast recovery. | |||
| In Figure 5, _flight_size_ is the amount of outstanding | In Figure 5, _flight_size_ is the amount of outstanding | |||
| (unacknowledged) data in the network, as defined in [RFC5681]. Note | (unacknowledged) data in the network, as defined in [RFC5681]. Note | |||
| that a rate-limited application with idle periods or periods when | that a rate-limited application with idle periods or periods when | |||
| unable to send at the full rate permitted by _cwnd_ could easily | unable to send at the full rate permitted by _cwnd_ could easily | |||
| encounter notable variations in the volume of data sent from one RTT | encounter notable variations in the volume of data sent from one RTT | |||
| to another, resulting in _flight_size_ that is significantly less | to another, resulting in _flight_size_ that is significantly less | |||
| than _cwnd_ when there is a congestion event. The congestion | than _cwnd_ when there is a congestion event. The congestion | |||
| response would therefore decrease _cwnd_ to a much lower value than | response would therefore decrease _cwnd_ to a much lower value than | |||
| necessary. To avoid such suboptimal performance, the mechanisms | necessary. To avoid such suboptimal performance, the mechanisms | |||
| described in [RFC7661] can be used. These describe how to manage and | described in [RFC7661] can be used. [RFC7661] describes how to | |||
| use _cwnd_ and _ssthresh_ during a rate-limited Interval, and how to | manage and use _cwnd_ and _ssthresh_ during a rate-limited interval, | |||
| update _cwnd_ and _ssthresh_ after congestion has been detected. The | and how to update _cwnd_ and _ssthresh_ after congestion has been | |||
| mechanism defined in [RFC7661] is safe to use even when _cwnd_ is | detected. The mechanisms defined in [RFC7661] are safe to use even | |||
| greater than the receive window, because it validates _cwnd_ based on | when _cwnd_ is greater than the receive window, because they validate | |||
| the amount of data acknowledged by the network in an RTT, which | _cwnd_ based on the amount of data acknowledged by the network in an | |||
| implicitly accounts for the allowed receive window. | RTT, which implicitly accounts for the allowed receive window. | |||
| Some implementations of CUBIC currently use _cwnd_ instead of | Some implementations of CUBIC currently use _cwnd_ instead of | |||
| _flight_size_ when calculating a new _ssthresh_. Implementations that | _flight_size_ when calculating a new _ssthresh_. Implementations | |||
| use _cwnd_ MUST use other measures to prevent _cwnd_ from growing | that use _cwnd_ MUST use other measures to prevent _cwnd_ from | |||
| when the volume of bytes in flight is smaller than _cwnd_. This also | growing when the volume of bytes in flight is smaller than | |||
| effectively avoids _cwnd_ from growing beyond the receive window. | _cwnd_. This also effectively prevents _cwnd_ from growing beyond | |||
| Such measures are important to prevent a CUBIC sender from using an | the receive window. Such measures are important for preventing a | |||
| arbitrarily high cwnd _value_ when calculating new values for | CUBIC sender from using an arbitrarily high cwnd _value_ when | |||
| _ssthresh_ and _cwnd_ when congestion is detected. This might not be | calculating new values for _ssthresh_ and _cwnd_ when congestion is | |||
| as robust as the mechanisms described in [RFC7661]. | detected. This might not be as robust as the mechanisms described in | |||
| [RFC7661]. | ||||
| A QUIC sender that uses _cwnd_ to calculate new values for _cwnd_ and | A QUIC sender that uses a _cwnd_ _value_ to calculate new values for | |||
| _ssthresh_ after detecting a congestion event is REQUIRED to apply | _cwnd_ and _ssthresh_ after detecting a congestion event is REQUIRED | |||
| similar mechanisms [RFC9002]. | to apply similar mechanisms [RFC9002]. | |||
| ssthresh = flight_size * β new ssthresh | ssthresh = flight_size * β new ssthresh | |||
| cubic | cubic | |||
| cwnd = cwnd save cwnd | cwnd = cwnd save cwnd | |||
| prior | prior | |||
| ⎧ | ⎧max(ssthresh, 2) reduction on loss, cwnd >= 2 SMSS | |||
| ⎨max(ssthresh, 2) reduction on loss, cwnd is at least 2 MSS | cwnd = ⎨max(ssthresh, 1) reduction on ECE, cwnd >= 1 SMSS | |||
| cwnd = ⎩max(ssthresh, 1) reduction on ECE, cwnd is at least 1 MSS | ⎩ | |||
| ssthresh = max(ssthresh, 2) ssthresh is at least 2 MSS | ssthresh = max(ssthresh, 2) ssthresh >= 2 SMSS | |||
| Figure 5 | Figure 5 | |||
| A side effect of setting β__cubic_ to a value bigger than 0.5 is that | A side effect of setting β__cubic_ to a value bigger than 0.5 is that | |||
| packet loss can happen for more than one round-trip in certain cases, | packet loss can happen for more than one RTT in certain cases, but it | |||
| but it can work efficiently in other cases, for example, when | can work efficiently in other cases -- for example, when HyStart++ | |||
| HyStart++ is used along with CUBIC or when the sending rate is | [RFC9406] is used along with CUBIC or when the sending rate is | |||
| limited by the application. While a more adaptive setting of | limited by the application. While a more adaptive setting of | |||
| β__cubic_ could help limit packet loss to a single round, it would | β__cubic_ could help limit packet loss to a single round, it would | |||
| require detailed analyses and large-scale evaluations to validate | require detailed analyses and large-scale evaluations to validate | |||
| such algorithms. | such algorithms. | |||
| Note that CUBIC MUST continue to reduce _cwnd_ in response to | Note that CUBIC MUST continue to reduce _cwnd_ in response to | |||
| congestion events due to ECN-Echo ACKs until it reaches a value of 1 | congestion events detected by ECN-Echo ACKs until it reaches a value | |||
| MSS. If congestion events indicated by ECN-Echo ACKs persist, a | of 1 SMSS. If congestion events indicated by ECN-Echo ACKs persist, | |||
| sender with a _cwnd_ of 1 MSS MUST reduce its sending rate even | a sender with a _cwnd_ of 1 SMSS MUST reduce its sending rate even | |||
| further. It can achieve that by using a retransmission timer with | further. This can be achieved by using a retransmission timer with | |||
| exponential backoff, as described in [RFC3168]. | exponential backoff, as described in [RFC3168]. | |||
| 4.7. Fast Convergence | 4.7. Fast Convergence | |||
| To improve convergence speed, CUBIC uses a heuristic. When a new | To improve convergence speed, CUBIC uses a heuristic. When a new | |||
| flow joins the network, existing flows need to give up some of their | flow joins the network, existing flows need to give up some of their | |||
| bandwidth to allow the new flow some room for growth, if the existing | bandwidth to allow the new flow some room for growth if the existing | |||
| flows have been using all the network bandwidth. To speed up this | flows have been using all the network bandwidth. To speed up this | |||
| bandwidth release by existing flows, the following "Fast Convergence" | bandwidth release by existing flows, the following fast convergence | |||
| mechanism SHOULD be implemented. | mechanism SHOULD be implemented. | |||
| With Fast Convergence, when a congestion event occurs, _W_max_ is | With fast convergence, when a congestion event occurs, _W_max_ is | |||
| updated as follows, before the window reduction described in | updated as follows, before the window reduction described in | |||
| Section 4.6. | Section 4.6. | |||
| ⎧ 1 + β | ⎧ 1 + β | |||
| ⎪ cubic | ⎪ cubic | |||
| ⎪cwnd * ────────── if cwnd < W and fast convergence enabled, | ⎪cwnd * ────────── if cwnd < W and fast convergence enabled, | |||
| W = ⎨ 2 max | W = ⎨ 2 max | |||
| max ⎪ further reduce W | max ⎪ further reduce W | |||
| ⎪ max | ⎪ max | |||
| ⎩cwnd otherwise, remember cwnd before reduction | ⎩cwnd otherwise, remember cwnd before reduction | |||
| At a congestion event, if the current _cwnd_ is less than _W_max_, | During a congestion event, if the current _cwnd_ is less than | |||
| this indicates that the saturation point experienced by this flow is | _W_max_, this indicates that the saturation point experienced by this | |||
| getting reduced because of a change in available bandwidth. This | flow is getting reduced because of a change in available bandwidth. | |||
| flow can then release more bandwidth by reducing _W_max_ further. | This flow can then release more bandwidth by reducing _W_max_ | |||
| This action effectively lengthens the time for this flow to increase | further. This action effectively lengthens the time for this flow to | |||
| its congestion window, because the reduced _W_max_ forces the flow to | increase its congestion window, because the reduced _W_max_ forces | |||
| plateau earlier. This allows more time for the new flow to catch up | the flow to plateau earlier. This allows more time for the new flow | |||
| to its congestion window size. | to catch up to its congestion window size. | |||
| Fast Convergence is designed for network environments with multiple | Fast convergence is designed for network environments with multiple | |||
| CUBIC flows. In network environments with only a single CUBIC flow | CUBIC flows. In network environments with only a single CUBIC flow | |||
| and without any other traffic, Fast Convergence SHOULD be disabled. | and without any other traffic, fast convergence SHOULD be disabled. | |||
| 4.8. Timeout | 4.8. Timeout | |||
| In case of a timeout, CUBIC follows Reno to reduce _cwnd_ [RFC5681], | In the case of a timeout, CUBIC follows Reno to reduce _cwnd_ | |||
| but sets _ssthresh_ using β__cubic_ (same as in Section 4.6) in a way | [RFC5681] but sets _ssthresh_ using β__cubic_ (same as in | |||
| that is different from Reno TCP [RFC5681]. | Section 4.6) in a way that is different from Reno TCP [RFC5681]. | |||
| During the first congestion avoidance stage after a timeout, CUBIC | During the first congestion avoidance stage after a timeout, CUBIC | |||
| increases its congestion window size using Figure 1, where _t_ is the | increases its congestion window size using Figure 1, where _t_ is the | |||
| elapsed time since the beginning of the current congestion avoidance, | elapsed time since the beginning of the current congestion avoidance | |||
| _K_ is set to 0, and _W_max_ is set to the congestion window size at | stage, _K_ is set to 0, and _W_max_ is set to the congestion window | |||
| the beginning of the current congestion avoidance stage. In | size at the beginning of the current congestion avoidance stage. In | |||
| addition, for the Reno-friendly region, _W_est_ SHOULD be set to the | addition, for the Reno-friendly region, _W_est_ SHOULD be set to the | |||
| congestion window size at the beginning of the current congestion | congestion window size at the beginning of the current congestion | |||
| avoidance. | avoidance stage. | |||
| 4.9. Spurious Congestion Events | 4.9. Spurious Congestion Events | |||
| In cases where CUBIC reduces its congestion window in response to | In cases where CUBIC reduces its congestion window in response to | |||
| having detected packet loss via duplicate ACKs or timeouts, there is | having detected packet loss via duplicate ACKs or timeouts, it is | |||
| a possibility that the missing ACK would arrive after the congestion | possible that the missing ACK could arrive after the congestion | |||
| window reduction and a corresponding packet retransmission. For | window reduction and a corresponding packet retransmission. For | |||
| example, packet reordering could trigger this behavior. A high | example, packet reordering could trigger this behavior. A high | |||
| degree of packet reordering could cause multiple congestion window | degree of packet reordering could cause multiple congestion window | |||
| reduction events, where spurious losses are incorrectly interpreted | reduction events, where spurious losses are incorrectly interpreted | |||
| as congestion signals, thus degrading CUBIC's performance | as congestion signals, thus degrading CUBIC's performance | |||
| significantly. | significantly. | |||
| For TCP, there are two types of spurious events - spurious timeouts | For TCP, there are two types of spurious events: spurious timeouts | |||
| and spurious fast retransmits. In case of QUIC, there are no | and spurious fast retransmits. In the case of QUIC, there are no | |||
| spurious timeouts as the loss is only detected after receiving an | spurious timeouts, as the loss is only detected after receiving an | |||
| ACK. | ACK. | |||
| 4.9.1. Spurious timeout | 4.9.1. Spurious Timeouts | |||
| An implementation MAY detect spurious timeouts based on the | An implementation MAY detect spurious timeouts based on the | |||
| mechanisms described in Forward RTO-Recovery [RFC5682]. Experimental | mechanisms described in Forward RTO-Recovery [RFC5682]. Experimental | |||
| alternatives include Eifel [RFC3522]. When a spurious timeout is | alternatives include the Eifel detection algorithm [RFC3522]. When a | |||
| detected, a TCP implementation MAY follow the response algorithm | spurious timeout is detected, a TCP implementation MAY follow the | |||
| described in [RFC4015] to restore the congestion control state and | response algorithm described in [RFC4015] to restore the congestion | |||
| adapt the retransmission timer to avoid further spurious timeouts. | control state and adapt the retransmission timer to avoid further | |||
| spurious timeouts. | ||||
| 4.9.2. Spurious loss detected by acknowledgments | 4.9.2. Spurious Fast Retransmits | |||
| Upon receiving an ACK, a TCP implementation MAY detect spurious | Upon receiving an ACK, a TCP implementation MAY detect spurious fast | |||
| losses either using TCP Timestamps or via D-SACK[RFC2883]. | retransmits either using TCP Timestamps or via D-SACK [RFC2883]. As | |||
| Experimental alternatives include Eifel detection algorithm [RFC3522] | noted above, experimental alternatives include the Eifel detection | |||
| which uses TCP Timestamps and DSACK based detection [RFC3708] which | algorithm [RFC3522], which uses TCP Timestamps; and DSACK-based | |||
| uses DSACK information. A QUIC implementation can easily determine a | detection [RFC3708], which uses DSACK information. A QUIC | |||
| spurious loss if a QUIC packet is acknowledged after it has been | implementation can easily determine a spurious fast retransmit if a | |||
| marked as lost and the original data has been retransmitted with a | QUIC packet is acknowledged after it has been marked as lost and the | |||
| new QUIC packet. | original data has been retransmitted with a new QUIC packet. | |||
| This section specifies a simple response algorithm when a spurious | This section specifies a simple response algorithm when a spurious | |||
| loss is detected by acknowledgments. Implementations would need to | fast retransmit is detected by acknowledgments. Implementations | |||
| carefully evaluate the impact of using this algorithm in different | would need to carefully evaluate the impact of using this algorithm | |||
| environments that may experience sudden change in available capacity | in different environments that may experience a sudden change in | |||
| (e.g., due to variable radio capacity, a routing change, or a | available capacity (e.g., due to variable radio capacity, a routing | |||
| mobility event). | change, or a mobility event). | |||
| When a packet loss is detected via acknowledgments, a CUBIC | When packet loss is detected via acknowledgments, a CUBIC | |||
| implementation MAY save the current value of the following variables | implementation MAY save the current value of the following variables | |||
| before the congestion window is reduced. | before the congestion window is reduced. | |||
| undo_cwnd = cwnd | undo_cwnd = cwnd | |||
| undo_cwnd = cwnd | undo_cwnd = cwnd | |||
| prior prior | prior prior | |||
| undo_ssthresh = ssthresh | undo_ssthresh = ssthresh | |||
| undo_W = W | undo_W = W | |||
| max max | max max | |||
| undo_K = K | undo_K = K | |||
| undo_t = t | undo_t = t | |||
| epoch epoch | epoch epoch | |||
| undo_W = W | undo_W = W | |||
| est est | est est | |||
| Once the previously declared packet loss is confirmed to be spurious, | Once the previously declared packet loss is confirmed to be spurious, | |||
| CUBIC MAY restore the original values of the above-mentioned | CUBIC MAY restore the original values of the above-mentioned | |||
| variables as follows if the current _cwnd_ is lower than | variables as follows if the current _cwnd_ is lower than | |||
| _cwnd_prior_. Restoring the original values ensures that CUBIC's | _cwnd_prior_. Restoring the original values ensures that CUBIC's | |||
| performance is similar to what it would be without spurious losses. | performance is similar to what it would be without spurious losses. | |||
| cwnd = undo_cwnd ⎫ | cwnd = undo_cwnd ⎫ | |||
| cwnd = undo_cwnd ⎮ | cwnd = undo_cwnd ⎮ | |||
| prior prior ⎮ | prior prior ⎮ | |||
| ssthresh = undo_ssthresh ⎮ | ssthresh = undo_ssthresh ⎮ | |||
| W = undo_W ⎮ | W = undo_W ⎮ | |||
| max max ⎬if cwnd < cwnd | max max ⎬if cwnd < cwnd | |||
| K = undo_K ⎮ prior | K = undo_K ⎮ prior | |||
| t = undo_t ⎮ | t = undo_t ⎮ | |||
| epoch epoch ⎮ | epoch epoch ⎮ | |||
| W = undo_W ⎮ | W = undo_W ⎮ | |||
| est est ⎭ | est est ⎭ | |||
| In rare cases, when the detection happens long after a spurious loss | In rare cases, when the detection happens long after a spurious fast | |||
| event and the current _cwnd_ is already higher than _cwnd_prior_, | retransmit event and the current _cwnd_ is already higher than | |||
| CUBIC SHOULD continue to use the current and the most recent values | _cwnd_prior_, CUBIC SHOULD continue to use the current and the most | |||
| of these variables. | recent values of these variables. | |||
| 4.10. Slow Start | 4.10. Slow Start | |||
| CUBIC MUST employ a slow-start algorithm, when _cwnd_ is no more than | When _cwnd_ is no more than _ssthresh_, CUBIC MUST employ a slow | |||
| _ssthresh_. In general, CUBIC SHOULD use the HyStart++ slow start | start algorithm. In general, CUBIC SHOULD use the HyStart++ slow | |||
| algorithm [I-D.ietf-tcpm-hystartplusplus], or MAY use the Reno TCP | start algorithm [RFC9406] or MAY use the Reno TCP slow start | |||
| slow start algorithm [RFC5681] in the rare cases when HyStart++ is | algorithm [RFC5681] in the rare cases when HyStart++ is not suitable. | |||
| not suitable. Experimental alternatives include hybrid slow start | Experimental alternatives include hybrid slow start [HR11], a | |||
| [HR11], a predecessor to HyStart++ that some CUBIC implementations | predecessor to HyStart++ that some CUBIC implementations have used as | |||
| have used as the default for the last decade, and limited slow start | the default for the last decade, and limited slow start [RFC3742]. | |||
| [RFC3742]. Whichever start-up algorithm is used, work might be | Whichever startup algorithm is used, work might be needed to ensure | |||
| needed to ensure that the end of slow start and the first | that the end of slow start and the first multiplicative decrease of | |||
| multiplicative decrease of congestion avoidance work well together. | congestion avoidance work well together. | |||
| When CUBIC uses HyStart++ [I-D.ietf-tcpm-hystartplusplus], it may | When CUBIC uses HyStart++ [RFC9406], it may exit the first slow start | |||
| exit the first slow start without incurring any packet loss and thus | without incurring any packet loss and thus _W_max_ is undefined. In | |||
| _W_max_ is undefined. In this special case, CUBIC sets _cwnd_prior = | this special case, CUBIC sets _cwnd_prior = cwnd_ and switches to | |||
| cwnd_ and switches to congestion avoidance. It then increases its | congestion avoidance. It then increases its congestion window size | |||
| congestion window size using Figure 1, where _t_ is the elapsed time | using Figure 1, where _t_ is the elapsed time since the beginning of | |||
| since the beginning of the current congestion avoidance, _K_ is set | the current congestion avoidance stage, _K_ is set to 0, and _W_max_ | |||
| to 0, and _W_max_ is set to the congestion window size at the | is set to the congestion window size at the beginning of the current | |||
| beginning of the current congestion avoidance stage. | congestion avoidance stage. | |||
| 5. Discussion | 5. Discussion | |||
| This section further discusses the safety features of CUBIC following | This section further discusses the safety features of CUBIC, | |||
| the guidelines specified in [RFC5033]. | following the guidelines specified in [RFC5033]. | |||
| With a deterministic loss model where the number of packets between | With a deterministic loss model where the number of packets between | |||
| two successive packet losses is always _1/p_, CUBIC always operates | two successive packet losses is always _1/p_, CUBIC always operates | |||
| with the concave window profile, which greatly simplifies the | with the concave window profile, which greatly simplifies the | |||
| performance analysis of CUBIC. The average window size of CUBIC (see | performance analysis of CUBIC. The average window size of CUBIC (see | |||
| Appendix C) can be obtained by the following function: | Appendix B) can be obtained via the following function: | |||
| ┌────────────────┐ 4 ┌────┐ | ┌────────────────┐ 4 ┌────┐ | |||
| │C * (3 + β ) ╲ │ 3 | │C * (3 + β ) ╲ │ 3 | |||
| 4 │ cubic ╲│RTT | 4 │ cubic ╲│RTT | |||
| AVG_W = ╲ │──────────────── * ──────── | AVG_W = ╲ │──────────────── * ──────── | |||
| cubic ╲ │4 * (1 - β ) 4 ┌──┐ | cubic ╲ │4 * (1 - β ) 4 ┌──┐ | |||
| ╲│ cubic ╲ │ 3 | ╲│ cubic ╲ │ 3 | |||
| ╲│p | ╲│p | |||
| Figure 6 | Figure 6 | |||
| With β__cubic_ set to 0.7, the above formula reduces to: | With β__cubic_ set to 0.7, the above formula reduces to | |||
| 4 ┌────┐ | 4 ┌────┐ | |||
| ┌───────┐ ╲ │ 3 | ┌───────┐ ╲ │ 3 | |||
| 4 │C * 3.7 ╲│RTT | 4 │C * 3.7 ╲│RTT | |||
| AVG_W = ╲ │─────── * ──────── | AVG_W = ╲ │─────── * ──────── | |||
| cubic ╲│ 1.2 4 ┌──┐ | cubic ╲│ 1.2 4 ┌──┐ | |||
| ╲ │ 3 | ╲ │ 3 | |||
| ╲│p | ╲│p | |||
| Figure 7 | Figure 7 | |||
| skipping to change at page 21, line 14 ¶ | skipping to change at line 901 ¶ | |||
| 5.1. Fairness to Reno | 5.1. Fairness to Reno | |||
| In environments where Reno is able to make reasonable use of the | In environments where Reno is able to make reasonable use of the | |||
| available bandwidth, CUBIC does not significantly change this state. | available bandwidth, CUBIC does not significantly change this state. | |||
| Reno performs well in the following two types of networks: | Reno performs well in the following two types of networks: | |||
| 1. networks with a small bandwidth-delay product (BDP) | 1. networks with a small bandwidth-delay product (BDP) | |||
| 2. networks with a short RTTs, but not necessarily a small BDP | 2. networks with short RTTs, but not necessarily a small BDP | |||
| CUBIC is designed to behave very similarly to Reno in the above two | CUBIC is designed to behave very similarly to Reno in the above two | |||
| types of networks. The following two tables show the average window | types of networks. The following two tables show the average window | |||
| sizes of Reno TCP, HSTCP, and CUBIC TCP. The average window sizes of | sizes of Reno TCP, HSTCP, and CUBIC TCP. The average window sizes of | |||
| Reno TCP and HSTCP are from [RFC3649]. The average window size of | Reno TCP and HSTCP are from [RFC3649]. The average window size of | |||
| CUBIC is calculated using Figure 7 and the CUBIC Reno-friendly region | CUBIC is calculated using Figure 7 and the CUBIC Reno-friendly region | |||
| for three different values of _C_. | for three different values of _C_. | |||
| +=============+=======+========+================+=========+========+ | +=============+=======+========+================+=========+========+ | |||
| | Loss Rate P | Reno | HSTCP | CUBIC (C=0.04) | CUBIC | CUBIC | | | Loss Rate P | Reno | HSTCP | CUBIC (C=0.04) | CUBIC | CUBIC | | |||
| skipping to change at page 21, line 42 ¶ | skipping to change at line 929 ¶ | |||
| +-------------+-------+--------+----------------+---------+--------+ | +-------------+-------+--------+----------------+---------+--------+ | |||
| | 1.0e-05 | 379 | 1795 | 593 | 1054 | 1874 | | | 1.0e-05 | 379 | 1795 | 593 | 1054 | 1874 | | |||
| +-------------+-------+--------+----------------+---------+--------+ | +-------------+-------+--------+----------------+---------+--------+ | |||
| | 1.0e-06 | 1200 | 12280 | 3332 | 5926 | 10538 | | | 1.0e-06 | 1200 | 12280 | 3332 | 5926 | 10538 | | |||
| +-------------+-------+--------+----------------+---------+--------+ | +-------------+-------+--------+----------------+---------+--------+ | |||
| | 1.0e-07 | 3795 | 83981 | 18740 | 33325 | 59261 | | | 1.0e-07 | 3795 | 83981 | 18740 | 33325 | 59261 | | |||
| +-------------+-------+--------+----------------+---------+--------+ | +-------------+-------+--------+----------------+---------+--------+ | |||
| | 1.0e-08 | 12000 | 574356 | 105383 | 187400 | 333250 | | | 1.0e-08 | 12000 | 574356 | 105383 | 187400 | 333250 | | |||
| +-------------+-------+--------+----------------+---------+--------+ | +-------------+-------+--------+----------------+---------+--------+ | |||
| Table 1: Reno TCP, HSTCP, and CUBIC with RTT = 0.1 seconds | Table 1: Reno TCP, HSTCP, and CUBIC with RTT = 0.1 Seconds | |||
| Table 1 describes the response function of Reno TCP, HSTCP, and CUBIC | Table 1 describes the response function of Reno TCP, HSTCP, and CUBIC | |||
| in networks with _RTT_ = 0.1 seconds. The average window size is in | in networks with _RTT_ = 0.1 seconds. The average window size is in | |||
| MSS-sized segments. | SMSS-sized segments. | |||
| +=============+=======+========+================+=========+=======+ | +=============+=======+========+================+=========+=======+ | |||
| | Loss Rate P | Reno | HSTCP | CUBIC (C=0.04) | CUBIC | CUBIC | | | Loss Rate P | Reno | HSTCP | CUBIC (C=0.04) | CUBIC | CUBIC | | |||
| | | | | | (C=0.4) | (C=4) | | | | | | | (C=0.4) | (C=4) | | |||
| +=============+=======+========+================+=========+=======+ | +=============+=======+========+================+=========+=======+ | |||
| | 1.0e-02 | 12 | 12 | 12 | 12 | 12 | | | 1.0e-02 | 12 | 12 | 12 | 12 | 12 | | |||
| +-------------+-------+--------+----------------+---------+-------+ | +-------------+-------+--------+----------------+---------+-------+ | |||
| | 1.0e-03 | 38 | 38 | 38 | 38 | 38 | | | 1.0e-03 | 38 | 38 | 38 | 38 | 38 | | |||
| +-------------+-------+--------+----------------+---------+-------+ | +-------------+-------+--------+----------------+---------+-------+ | |||
| | 1.0e-04 | 120 | 263 | 120 | 120 | 120 | | | 1.0e-04 | 120 | 263 | 120 | 120 | 120 | | |||
| +-------------+-------+--------+----------------+---------+-------+ | +-------------+-------+--------+----------------+---------+-------+ | |||
| | 1.0e-05 | 379 | 1795 | 379 | 379 | 379 | | | 1.0e-05 | 379 | 1795 | 379 | 379 | 379 | | |||
| +-------------+-------+--------+----------------+---------+-------+ | +-------------+-------+--------+----------------+---------+-------+ | |||
| | 1.0e-06 | 1200 | 12280 | 1200 | 1200 | 1874 | | | 1.0e-06 | 1200 | 12280 | 1200 | 1200 | 1874 | | |||
| +-------------+-------+--------+----------------+---------+-------+ | +-------------+-------+--------+----------------+---------+-------+ | |||
| | 1.0e-07 | 3795 | 83981 | 3795 | 5926 | 10538 | | | 1.0e-07 | 3795 | 83981 | 3795 | 5926 | 10538 | | |||
| +-------------+-------+--------+----------------+---------+-------+ | +-------------+-------+--------+----------------+---------+-------+ | |||
| | 1.0e-08 | 12000 | 574356 | 18740 | 33325 | 59261 | | | 1.0e-08 | 12000 | 574356 | 18740 | 33325 | 59261 | | |||
| +-------------+-------+--------+----------------+---------+-------+ | +-------------+-------+--------+----------------+---------+-------+ | |||
| Table 2: Reno TCP, HSTCP, and CUBIC with RTT = 0.01 seconds | Table 2: Reno TCP, HSTCP, and CUBIC with RTT = 0.01 Seconds | |||
| Table 2 describes the response function of Reno TCP, HSTCP, and CUBIC | Table 2 describes the response function of Reno TCP, HSTCP, and CUBIC | |||
| in networks with _RTT_ = 0.01 seconds. The average window size is in | in networks with _RTT_ = 0.01 seconds. The average window size is in | |||
| MSS-sized segments. | SMSS-sized segments. | |||
| Both tables show that CUBIC with any of these three _C_ values is | Both tables show that CUBIC with any of these three _C_ values is | |||
| more friendly to Reno TCP than HSTCP, especially in networks with a | more friendly to Reno TCP than HSTCP, especially in networks with a | |||
| short _RTT_ where Reno TCP performs reasonably well. For example, in | short _RTT_ where Reno TCP performs reasonably well. For example, in | |||
| a network with _RTT_ = 0.01 seconds and p=10^-6, Reno TCP has an | a network with _RTT_ = 0.01 seconds and p=10^-6, Reno TCP has an | |||
| average window of 1200 packets. If the packet size is 1500 bytes, | average window of 1200 packets. If the packet size is 1500 bytes, | |||
| then Reno TCP can achieve an average rate of 1.44 Gbps. In this | then Reno TCP can achieve an average rate of 1.44 Gbps. In this | |||
| case, CUBIC with _C_=0.04 or _C_=0.4 achieves exactly the same rate | case, CUBIC with _C_=0.04 or _C_=0.4 achieves exactly the same rate | |||
| as Reno TCP, whereas HSTCP is about ten times more aggressive than | as Reno TCP, whereas HSTCP is about ten times more aggressive than | |||
| Reno TCP. | Reno TCP. | |||
| _C_ determines the aggressiveness of CUBIC in competing with other | _C_ determines the aggressiveness of CUBIC in competing with other | |||
| congestion control algorithms for bandwidth. CUBIC is more friendly | congestion control algorithms for bandwidth. CUBIC is more friendly | |||
| to Reno TCP, if the value of _C_ is lower. However, it is NOT | to Reno TCP if the value of _C_ is lower. However, it is NOT | |||
| RECOMMENDED to set _C_ to a very low value like 0.04, since CUBIC | RECOMMENDED to set _C_ to a very low value like 0.04, since CUBIC | |||
| with a low _C_ cannot efficiently use the bandwidth in fast and long- | with a low _C_ cannot efficiently use the bandwidth in fast and long- | |||
| distance networks. Based on these observations and extensive | distance networks. Based on these observations and extensive | |||
| deployment experience, _C_=0.4 seems to give a good balance between | deployment experience, _C_=0.4 seems to provide a good balance | |||
| Reno-friendliness and aggressiveness of window increase. Therefore, | between Reno-friendliness and aggressiveness of window increase. | |||
| _C_ SHOULD be set to 0.4. With _C_ set to 0.4, Figure 7 is reduced | Therefore, _C_ SHOULD be set to 0.4. With _C_ set to 0.4, Figure 7 | |||
| to: | is reduced to | |||
| 4 ┌────┐ | 4 ┌────┐ | |||
| ╲ │ 3 | ╲ │ 3 | |||
| ╲│RTT | ╲│RTT | |||
| AVG_W = 1.054 * ──────── | AVG_W = 1.054 * ──────── | |||
| cubic 4 ┌──┐ | cubic 4 ┌──┐ | |||
| ╲ │ 3 | ╲ │ 3 | |||
| ╲│p | ╲│p | |||
| Figure 8 | Figure 8 | |||
| Figure 8 is then used in the next subsection to show the scalability | Figure 8 is then used in the next subsection to show the scalability | |||
| of CUBIC. | of CUBIC. | |||
| 5.2. Using Spare Capacity | 5.2. Using Spare Capacity | |||
| CUBIC uses a more aggressive window increase function than Reno for | CUBIC uses a more aggressive window increase function than Reno for | |||
| fast and long-distance networks. | fast and long-distance networks. | |||
| The following table shows that to achieve the 10 Gbps rate, Reno TCP | Table 3 shows that to achieve the 10 Gbps rate, Reno TCP requires a | |||
| requires a packet loss rate of 2.0e-10, while CUBIC TCP requires a | packet loss rate of 2.0e-10, while CUBIC TCP requires a packet loss | |||
| packet loss rate of 2.9e-8. | rate of 2.9e-8. | |||
| +===================+===========+=========+=========+=========+ | +===================+===========+=========+=========+=========+ | |||
| | Throughput (Mbps) | Average W | Reno P | HSTCP P | CUBIC P | | | Throughput (Mbps) | Average W | Reno P | HSTCP P | CUBIC P | | |||
| +===================+===========+=========+=========+=========+ | +===================+===========+=========+=========+=========+ | |||
| | 1 | 8.3 | 2.0e-2 | 2.0e-2 | 2.0e-2 | | | 1 | 8.3 | 2.0e-2 | 2.0e-2 | 2.0e-2 | | |||
| +-------------------+-----------+---------+---------+---------+ | +-------------------+-----------+---------+---------+---------+ | |||
| | 10 | 83.3 | 2.0e-4 | 3.9e-4 | 2.9e-4 | | | 10 | 83.3 | 2.0e-4 | 3.9e-4 | 2.9e-4 | | |||
| +-------------------+-----------+---------+---------+---------+ | +-------------------+-----------+---------+---------+---------+ | |||
| | 100 | 833.3 | 2.0e-6 | 2.5e-5 | 1.4e-5 | | | 100 | 833.3 | 2.0e-6 | 2.5e-5 | 1.4e-5 | | |||
| +-------------------+-----------+---------+---------+---------+ | +-------------------+-----------+---------+---------+---------+ | |||
| | 1000 | 8333.3 | 2.0e-8 | 1.5e-6 | 6.3e-7 | | | 1000 | 8333.3 | 2.0e-8 | 1.5e-6 | 6.3e-7 | | |||
| +-------------------+-----------+---------+---------+---------+ | +-------------------+-----------+---------+---------+---------+ | |||
| | 10000 | 83333.3 | 2.0e-10 | 1.0e-7 | 2.9e-8 | | | 10000 | 83333.3 | 2.0e-10 | 1.0e-7 | 2.9e-8 | | |||
| +-------------------+-----------+---------+---------+---------+ | +-------------------+-----------+---------+---------+---------+ | |||
| Table 3: Required packet loss rate for Reno TCP, HSTCP, and | Table 3: Required Packet Loss Rate for Reno TCP, HSTCP, and | |||
| CUBIC to achieve a certain throughput | CUBIC to Achieve a Certain Throughput | |||
| Table 3 describes the required packet loss rate for Reno TCP, HSTCP, | Table 3 describes the required packet loss rate for Reno TCP, HSTCP, | |||
| and CUBIC to achieve a certain throughput, with 1500-byte packets and | and CUBIC to achieve a certain throughput, with 1500-byte packets and | |||
| an _RTT_ of 0.1 seconds. | an _RTT_ of 0.1 seconds. | |||
| The test results in [HLRX07] indicate that, in typical cases with a | The test results provided in [HLRX07] indicate that, in typical cases | |||
| degree of background traffic, CUBIC uses the spare bandwidth left | with a degree of background traffic, CUBIC uses the spare bandwidth | |||
| unused by existing Reno TCP flows in the same bottleneck link without | left unused by existing Reno TCP flows in the same bottleneck link | |||
| taking away much bandwidth from the existing flows. | without taking away much bandwidth from the existing flows. | |||
| 5.3. Difficult Environments | 5.3. Difficult Environments | |||
| CUBIC is designed to remedy the poor performance of Reno in fast and | CUBIC is designed to remedy the poor performance of Reno in fast and | |||
| long-distance networks. | long-distance networks. | |||
| 5.4. Investigating a Range of Environments | 5.4. Investigating a Range of Environments | |||
| CUBIC has been extensively studied using simulations, testbed | CUBIC has been extensively studied using simulations, testbed | |||
| emulations, Internet experiments, and Internet measurements, covering | emulations, Internet experiments, and Internet measurements, covering | |||
| a wide range of network environments | a wide range of network environments [HLRX07] [H16] [CEHRX09] [HR11] | |||
| [HLRX07][H16][CEHRX09][HR11][BSCLU13][LBEWK16]. They have | [BSCLU13] [LBEWK16]. They have convincingly demonstrated that CUBIC | |||
| convincingly demonstrated that CUBIC delivers substantial benefits | delivers substantial benefits over classical Reno congestion control | |||
| over classical Reno congestion control [RFC5681]. | [RFC5681]. | |||
| Same as Reno, CUBIC is a loss-based congestion control algorithm. | Same as Reno, CUBIC is a loss-based congestion control algorithm. | |||
| Because CUBIC is designed to be more aggressive (due to a faster | Because CUBIC is designed to be more aggressive (due to a faster | |||
| window increase function and bigger multiplicative decrease factor) | window increase function and bigger multiplicative decrease factor) | |||
| than Reno in fast and long-distance networks, it can fill large drop- | than Reno in fast and long-distance networks, it can fill large drop- | |||
| tail buffers more quickly than Reno and increases the risk of a | tail buffers more quickly than Reno and increases the risk of a | |||
| standing queue [RFC8511]. In this case, proper queue sizing and | standing queue [RFC8511]. In this case, proper queue sizing and | |||
| management [RFC7567] could be used to mitigate the risk to some | management [RFC7567] could be used to mitigate the risk to some | |||
| extent and reduce the packet queuing delay. Also, in large-BDP | extent and reduce the packet queuing delay. Also, in large-BDP | |||
| networks after a congestion event, CUBIC, due its cubic window | networks after a congestion event, CUBIC, due to its cubic window | |||
| increase function, recovers quickly to the highest link utilization | increase function, recovers quickly to the highest link utilization | |||
| point. This means that link utilization is less sensitive to an | point. This means that link utilization is less sensitive to an | |||
| active queue management (AQM) target that is lower than the amplitude | active queue management (AQM) target that is lower than the amplitude | |||
| of the whole sawtooth. | of the whole sawtooth. | |||
| Similar to Reno, the performance of CUBIC as a loss-based congestion | Similar to Reno, the performance of CUBIC as a loss-based congestion | |||
| control algorithm suffers in networks where a packet loss is not a | control algorithm suffers in networks where packet loss is not a good | |||
| good indication of bandwidth utilization, such as wireless or mobile | indication of bandwidth utilization, such as wireless or mobile | |||
| networks [LIU16]. | networks [LIU16]. | |||
| 5.5. Protection against Congestion Collapse | 5.5. Protection against Congestion Collapse | |||
| With regard to the potential of causing congestion collapse, CUBIC | With regard to the potential of causing congestion collapse, CUBIC | |||
| behaves like Reno, since CUBIC modifies only the window adjustment | behaves like Reno, since CUBIC modifies only the window adjustment | |||
| algorithm of Reno. Thus, it does not modify the ACK clocking and | algorithm of Reno. Thus, it does not modify the ACK clocking and | |||
| timeout behaviors of Reno. | timeout behaviors of Reno. | |||
| CUBIC also satisfies the "full backoff" requirement as described in | CUBIC also satisfies the "full backoff" requirement as described in | |||
| [RFC5033]. After reducing the sending rate to one packet per RTT in | [RFC5033]. After reducing the sending rate to one packet per RTT in | |||
| response to congestion events due to ECN-Echo ACKs, CUBIC then | response to congestion events detected by ECN-Echo ACKs, CUBIC then | |||
| exponentially increases the transmission timer for each packet | exponentially increases the transmission timer for each packet | |||
| retransmission while congestion persists. | retransmission while congestion persists. | |||
| 5.6. Fairness within the Alternative Congestion Control Algorithm | 5.6. Fairness within the Alternative Congestion Control Algorithm | |||
| CUBIC ensures convergence of competing CUBIC flows with the same RTT | CUBIC ensures convergence of competing CUBIC flows with the same RTT | |||
| in the same bottleneck links to an equal throughput. When competing | in the same bottleneck links to an equal throughput. When competing | |||
| flows have different RTT values, their throughput ratio is linearly | flows have different RTT values, their throughput ratio is linearly | |||
| proportional to the inverse of their RTT ratios. This is true | proportional to the inverse of their RTT ratios. This is true and is | |||
| independently of the level of statistical multiplexing on the link. | independent of the level of statistical multiplexing on the link. | |||
| The convergence time depends on the network environments (e.g., | The convergence time depends on the network environments (e.g., | |||
| bandwidth, RTT) and the level of statistical multiplexing, as | bandwidth, RTT) and the level of statistical multiplexing, as | |||
| mentioned in Section 3.4. | mentioned in Section 3.4. | |||
| 5.7. Performance with Misbehaving Nodes and Outside Attackers | 5.7. Performance with Misbehaving Nodes and Outside Attackers | |||
| CUBIC does not introduce new entities or signals, so its | CUBIC does not introduce new entities or signals, so its | |||
| vulnerability to misbehaving nodes or attackers is unchanged from | vulnerability to misbehaving nodes or attackers is unchanged from | |||
| Reno. | Reno. | |||
| 5.8. Behavior for Application-Limited Flows | 5.8. Behavior for Application-Limited Flows | |||
| A flow is application-limited if it is currently sending less than | A flow is application limited if it is currently sending less than | |||
| what is allowed by the congestion window. This can happen if the | what is allowed by the congestion window. This can happen if the | |||
| flow is limited by either the sender application or the receiver | flow is limited by either the sender application or the receiver | |||
| application (via the receiver advertised window) and thus sends less | application (via the receiver's advertised window) and thus sends | |||
| data than what is allowed by the sender's congestion window. | less data than what is allowed by the sender's congestion window. | |||
| CUBIC does not increase its congestion window if a flow is | CUBIC does not increase its congestion window if a flow is | |||
| application-limited. Section 4.2 requires that _t_ in Figure 1 does | application limited. Per Section 4.2, it is required that _t_ in | |||
| not include application-limited periods, such as idle periods, | Figure 1 not include application-limited periods, such as idle | |||
| otherwise W_cubic(_t_) might be very high after restarting from these | periods; otherwise, W_cubic(_t_) might be very high after restarting | |||
| periods. | from these periods. | |||
| 5.9. Responses to Sudden or Transient Events | 5.9. Responses to Sudden or Transient Events | |||
| If there is a sudden increase in capacity, e.g., due to variable | If there is a sudden increase in capacity, e.g., due to variable | |||
| radio capacity, a routing change, or a mobility event, CUBIC is | radio capacity, a routing change, or a mobility event, CUBIC is | |||
| designed to utilize the newly available capacity faster than Reno. | designed to utilize the newly available capacity more quickly than | |||
| Reno. | ||||
| On the other hand, if there is a sudden decrease in capacity, CUBIC | On the other hand, if there is a sudden decrease in capacity, CUBIC | |||
| reduces more slowly than Reno. This remains true regardless of | reduces more slowly than Reno. This remains true regardless of | |||
| whether CUBIC is in Reno-friendly mode and regardless of whether fast | whether CUBIC is in Reno-friendly mode and regardless of whether fast | |||
| convergence is enabled. | convergence is enabled. | |||
| 5.10. Incremental Deployment | 5.10. Incremental Deployment | |||
| CUBIC requires only changes to the congestion control at the sender, | CUBIC requires only changes to congestion control at the sender, and | |||
| and it does not require any changes at receivers. That is, a CUBIC | it does not require any changes at receivers. That is, a CUBIC | |||
| sender works correctly with Reno receivers. In addition, CUBIC does | sender works correctly with Reno receivers. In addition, CUBIC does | |||
| not require any changes to routers and does not require any | not require any changes to routers and does not require any | |||
| assistance from routers. | assistance from routers. | |||
| 6. Security Considerations | 6. Security Considerations | |||
| CUBIC makes no changes to the underlying security of a transport | CUBIC makes no changes to the underlying security of a transport | |||
| protocol and inherits the general security concerns described in | protocol and inherits the general security concerns described in | |||
| [RFC5681]. Specifically, changing the window computation on the | [RFC5681]. Specifically, changing the window computation on the | |||
| sender may allow an attacker, through dropping or injecting ACKs (as | sender may allow an attacker, through dropping or injecting ACKs (as | |||
| described in [RFC5681]), to either force the CUBIC implementation to | described in [RFC5681]), to either force the CUBIC implementation to | |||
| reduce its bandwidth, or to convince it that there is no congestion | reduce its bandwidth or convince it that there is no congestion when | |||
| when congestion does exist, and use the CUBIC implementation as an | congestion does exist, and to use the CUBIC implementation as an | |||
| attack vector against other hosts. These attacks are not new to | attack vector against other hosts. These attacks are not new to | |||
| CUBIC and are inherently part of any transport protocol like TCP. | CUBIC and are inherently part of any transport protocol like TCP. | |||
| 7. IANA Considerations | 7. IANA Considerations | |||
| This document does not require any IANA actions. | This document does not require any IANA actions. | |||
| 8. References | 8. References | |||
| 8.1. Normative References | 8.1. Normative References | |||
| [I-D.ietf-tcpm-hystartplusplus] | ||||
| Balasubramanian, P., Huang, Y., and M. Olson, "HyStart++: | ||||
| Modified Slow Start for TCP", Work in Progress, Internet- | ||||
| Draft, draft-ietf-tcpm-hystartplusplus-13, 30 January | ||||
| 2023, <https://datatracker.ietf.org/doc/html/draft-ietf- | ||||
| tcpm-hystartplusplus-13>. | ||||
| [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate | [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate | |||
| Requirement Levels", BCP 14, RFC 2119, | Requirement Levels", BCP 14, RFC 2119, | |||
| DOI 10.17487/RFC2119, March 1997, | DOI 10.17487/RFC2119, March 1997, | |||
| <https://www.rfc-editor.org/rfc/rfc2119>. | <https://www.rfc-editor.org/info/rfc2119>. | |||
| [RFC2883] Floyd, S., Mahdavi, J., Mathis, M., and M. Podolsky, "An | [RFC2883] Floyd, S., Mahdavi, J., Mathis, M., and M. Podolsky, "An | |||
| Extension to the Selective Acknowledgement (SACK) Option | Extension to the Selective Acknowledgement (SACK) Option | |||
| for TCP", RFC 2883, DOI 10.17487/RFC2883, July 2000, | for TCP", RFC 2883, DOI 10.17487/RFC2883, July 2000, | |||
| <https://www.rfc-editor.org/rfc/rfc2883>. | <https://www.rfc-editor.org/info/rfc2883>. | |||
| [RFC2914] Floyd, S., "Congestion Control Principles", BCP 41, | [RFC2914] Floyd, S., "Congestion Control Principles", BCP 41, | |||
| RFC 2914, DOI 10.17487/RFC2914, September 2000, | RFC 2914, DOI 10.17487/RFC2914, September 2000, | |||
| <https://www.rfc-editor.org/rfc/rfc2914>. | <https://www.rfc-editor.org/info/rfc2914>. | |||
| [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition | [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition | |||
| of Explicit Congestion Notification (ECN) to IP", | of Explicit Congestion Notification (ECN) to IP", | |||
| RFC 3168, DOI 10.17487/RFC3168, September 2001, | RFC 3168, DOI 10.17487/RFC3168, September 2001, | |||
| <https://www.rfc-editor.org/rfc/rfc3168>. | <https://www.rfc-editor.org/info/rfc3168>. | |||
| [RFC4015] Ludwig, R. and A. Gurtov, "The Eifel Response Algorithm | [RFC4015] Ludwig, R. and A. Gurtov, "The Eifel Response Algorithm | |||
| for TCP", RFC 4015, DOI 10.17487/RFC4015, February 2005, | for TCP", RFC 4015, DOI 10.17487/RFC4015, February 2005, | |||
| <https://www.rfc-editor.org/rfc/rfc4015>. | <https://www.rfc-editor.org/info/rfc4015>. | |||
| [RFC5033] Floyd, S. and M. Allman, "Specifying New Congestion | [RFC5033] Floyd, S. and M. Allman, "Specifying New Congestion | |||
| Control Algorithms", BCP 133, RFC 5033, | Control Algorithms", BCP 133, RFC 5033, | |||
| DOI 10.17487/RFC5033, August 2007, | DOI 10.17487/RFC5033, August 2007, | |||
| <https://www.rfc-editor.org/rfc/rfc5033>. | <https://www.rfc-editor.org/info/rfc5033>. | |||
| [RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP | [RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP | |||
| Friendly Rate Control (TFRC): Protocol Specification", | Friendly Rate Control (TFRC): Protocol Specification", | |||
| RFC 5348, DOI 10.17487/RFC5348, September 2008, | RFC 5348, DOI 10.17487/RFC5348, September 2008, | |||
| <https://www.rfc-editor.org/rfc/rfc5348>. | <https://www.rfc-editor.org/info/rfc5348>. | |||
| [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion | [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion | |||
| Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, | Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, | |||
| <https://www.rfc-editor.org/rfc/rfc5681>. | <https://www.rfc-editor.org/info/rfc5681>. | |||
| [RFC5682] Sarolahti, P., Kojo, M., Yamamoto, K., and M. Hata, | [RFC5682] Sarolahti, P., Kojo, M., Yamamoto, K., and M. Hata, | |||
| "Forward RTO-Recovery (F-RTO): An Algorithm for Detecting | "Forward RTO-Recovery (F-RTO): An Algorithm for Detecting | |||
| Spurious Retransmission Timeouts with TCP", RFC 5682, | Spurious Retransmission Timeouts with TCP", RFC 5682, | |||
| DOI 10.17487/RFC5682, September 2009, | DOI 10.17487/RFC5682, September 2009, | |||
| <https://www.rfc-editor.org/rfc/rfc5682>. | <https://www.rfc-editor.org/info/rfc5682>. | |||
| [RFC6298] Paxson, V., Allman, M., Chu, J., and M. Sargent, | [RFC6298] Paxson, V., Allman, M., Chu, J., and M. Sargent, | |||
| "Computing TCP's Retransmission Timer", RFC 6298, | "Computing TCP's Retransmission Timer", RFC 6298, | |||
| DOI 10.17487/RFC6298, June 2011, | DOI 10.17487/RFC6298, June 2011, | |||
| <https://www.rfc-editor.org/rfc/rfc6298>. | <https://www.rfc-editor.org/info/rfc6298>. | |||
| [RFC6582] Henderson, T., Floyd, S., Gurtov, A., and Y. Nishida, "The | [RFC6582] Henderson, T., Floyd, S., Gurtov, A., and Y. Nishida, "The | |||
| NewReno Modification to TCP's Fast Recovery Algorithm", | NewReno Modification to TCP's Fast Recovery Algorithm", | |||
| RFC 6582, DOI 10.17487/RFC6582, April 2012, | RFC 6582, DOI 10.17487/RFC6582, April 2012, | |||
| <https://www.rfc-editor.org/rfc/rfc6582>. | <https://www.rfc-editor.org/info/rfc6582>. | |||
| [RFC6675] Blanton, E., Allman, M., Wang, L., Jarvinen, I., Kojo, M., | [RFC6675] Blanton, E., Allman, M., Wang, L., Jarvinen, I., Kojo, M., | |||
| and Y. Nishida, "A Conservative Loss Recovery Algorithm | and Y. Nishida, "A Conservative Loss Recovery Algorithm | |||
| Based on Selective Acknowledgment (SACK) for TCP", | Based on Selective Acknowledgment (SACK) for TCP", | |||
| RFC 6675, DOI 10.17487/RFC6675, August 2012, | RFC 6675, DOI 10.17487/RFC6675, August 2012, | |||
| <https://www.rfc-editor.org/rfc/rfc6675>. | <https://www.rfc-editor.org/info/rfc6675>. | |||
| [RFC7567] Baker, F., Ed. and G. Fairhurst, Ed., "IETF | [RFC7567] Baker, F., Ed. and G. Fairhurst, Ed., "IETF | |||
| Recommendations Regarding Active Queue Management", | Recommendations Regarding Active Queue Management", | |||
| BCP 197, RFC 7567, DOI 10.17487/RFC7567, July 2015, | BCP 197, RFC 7567, DOI 10.17487/RFC7567, July 2015, | |||
| <https://www.rfc-editor.org/rfc/rfc7567>. | <https://www.rfc-editor.org/info/rfc7567>. | |||
| [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC | [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC | |||
| 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, | 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, | |||
| May 2017, <https://www.rfc-editor.org/rfc/rfc8174>. | May 2017, <https://www.rfc-editor.org/info/rfc8174>. | |||
| [RFC8985] Cheng, Y., Cardwell, N., Dukkipati, N., and P. Jha, "The | [RFC8985] Cheng, Y., Cardwell, N., Dukkipati, N., and P. Jha, "The | |||
| RACK-TLP Loss Detection Algorithm for TCP", RFC 8985, | RACK-TLP Loss Detection Algorithm for TCP", RFC 8985, | |||
| DOI 10.17487/RFC8985, February 2021, | DOI 10.17487/RFC8985, February 2021, | |||
| <https://www.rfc-editor.org/rfc/rfc8985>. | <https://www.rfc-editor.org/info/rfc8985>. | |||
| [RFC9002] Iyengar, J., Ed. and I. Swett, Ed., "QUIC Loss Detection | [RFC9002] Iyengar, J., Ed. and I. Swett, Ed., "QUIC Loss Detection | |||
| and Congestion Control", RFC 9002, DOI 10.17487/RFC9002, | and Congestion Control", RFC 9002, DOI 10.17487/RFC9002, | |||
| May 2021, <https://www.rfc-editor.org/rfc/rfc9002>. | May 2021, <https://www.rfc-editor.org/info/rfc9002>. | |||
| [RFC9406] Balasubramanian, P., Huang, Y., and M. Olson, "HyStart++: | ||||
| Modified Slow Start for TCP", RFC 9406, | ||||
| DOI 10.17487/RFC9406, May 2023, | ||||
| <https://www.rfc-editor.org/info/rfc9406>. | ||||
| 8.2. Informative References | 8.2. Informative References | |||
| [AIMD-friendliness] | [AIMD-friendliness] | |||
| Briscoe, B. and O. Albisser, "Friendliness between AIMD | Briscoe, B. and O. Albisser, "Friendliness between AIMD | |||
| Algorithms", RFC Editor, please replace this URL with the | Algorithms", DOI 10.48550/arXiv.2305.10581, May 2023, | |||
| permanent arXiv one , 8 August 2022, | <https://arxiv.org/abs/2305.10581>. | |||
| <https://raw.githubusercontent.com/bbriscoe/cubic- | ||||
| reno/main/creno_tr.pdf>. | ||||
| [BSCLU13] Belhareth, S., Sassatelli, L., Collange, D., Lopez- | [BSCLU13] Belhareth, S., Sassatelli, L., Collange, D., Lopez- | |||
| Pacheco, D., and G. Urvoy-Keller, "Understanding TCP cubic | Pacheco, D., and G. Urvoy-Keller, "Understanding TCP cubic | |||
| performance in the cloud: A mean-field approach", 2013 | performance in the cloud: A mean-field approach", 2013 | |||
| IEEE 2nd International Conference on Cloud | IEEE 2nd International Conference on Cloud Networking | |||
| Networking (CloudNet), DOI 10.1109/cloudnet.2013.6710576, | (CloudNet), DOI 10.1109/cloudnet.2013.6710576, November | |||
| November 2013, | 2013, <https://doi.org/10.1109/cloudnet.2013.6710576>. | |||
| <https://doi.org/10.1109/cloudnet.2013.6710576>. | ||||
| [CEHRX09] Cai, H., Eun, D., Ha, S., Rhee, I., and L. Xu, "Stochastic | [CEHRX09] Cai, H., Eun, D., Ha, S., Rhee, I., and L. Xu, "Stochastic | |||
| convex ordering for multiplicative decrease internet | convex ordering for multiplicative decrease internet | |||
| congestion control", Computer Networks vol. 53, no. 3, pp. | congestion control", Computer Networks, vol. 53, no. 3, | |||
| 365-381, DOI 10.1016/j.comnet.2008.10.012, February 2009, | pp. 365-381, DOI 10.1016/j.comnet.2008.10.012, February | |||
| <https://doi.org/10.1016/j.comnet.2008.10.012>. | 2009, <https://doi.org/10.1016/j.comnet.2008.10.012>. | |||
| [FHP00] Floyd, S., Handley, M., and J. Padhye, "A Comparison of | [FHP00] Floyd, S., Handley, M., and J. Padhye, "A Comparison of | |||
| Equation-Based and AIMD Congestion Control", May 2000, | Equation-Based and AIMD Congestion Control", May 2000, | |||
| <https://www.icir.org/tfrc/aimd.pdf>. | <https://www.icir.org/tfrc/aimd.pdf>. | |||
| [GV02] Gorinsky, S. and H. Vin, "Extended Analysis of Binary | [GV02] Gorinsky, S. and H. Vin, "Extended Analysis of Binary | |||
| Adjustment Algorithms", Technical Report TR2002-29, | Adjustment Algorithms", Technical Report TR2002-39, | |||
| Department of Computer Sciences, The University of | Department of Computer Sciences, The University of Texas | |||
| Texas at Austin, 11 August 2002, | at Austin, August 2002, <https://citeseerx.ist.psu.edu/doc | |||
| <https://www.cs.utexas.edu/ftp/techreports/tr02-39.ps.gz>. | ument?repid=rep1&type=pdf&doi=1828bdcef118b02d3996b8e00b8a | |||
| aa92b50abb0f>. | ||||
| [H16] Ha, S., "Simulation, Testbed, and Deployment Testing | [H16] Ha, S., "Deployment, Testbed, and Simulation Results for | |||
| Results of CUBIC", 3 November 2016, | CUBIC", Wayback Machine archive, 3 November 2016, | |||
| <https://web.archive.org/web/20161118125842/ | <https://web.archive.org/web/20161118125842/ | |||
| http://netsrv.csc.ncsu.edu/wiki/index.php/TCP_Testing>. | http://netsrv.csc.ncsu.edu/wiki/index.php/TCP_Testing>. | |||
| [HLRX07] Ha, S., Le, L., Rhee, I., and L. Xu, "Impact of background | [HLRX07] Ha, S., Le, L., Rhee, I., and L. Xu, "Impact of background | |||
| traffic on performance of high-speed TCP variant | traffic on performance of high-speed TCP variant | |||
| protocols", Computer Networks vol. 51, no. 7, pp. | protocols", Computer Networks, vol. 51, no. 7, pp. | |||
| 1748-1762, DOI 10.1016/j.comnet.2006.11.005, May 2007, | 1748-1762, DOI 10.1016/j.comnet.2006.11.005, May 2007, | |||
| <https://doi.org/10.1016/j.comnet.2006.11.005>. | <https://doi.org/10.1016/j.comnet.2006.11.005>. | |||
| [HR11] Ha, S. and I. Rhee, "Taming the elephants: New TCP slow | [HR11] Ha, S. and I. Rhee, "Taming the elephants: New TCP slow | |||
| start", Computer Networks vol. 55, no. 9, pp. 2092-2110, | start", Computer Networks, vol. 55, no. 9, pp. 2092-2110, | |||
| DOI 10.1016/j.comnet.2011.01.014, June 2011, | DOI 10.1016/j.comnet.2011.01.014, June 2011, | |||
| <https://doi.org/10.1016/j.comnet.2011.01.014>. | <https://doi.org/10.1016/j.comnet.2011.01.014>. | |||
| [HRX08] Ha, S., Rhee, I., and L. Xu, "CUBIC: a new TCP-friendly | [HRX08] Ha, S., Rhee, I., and L. Xu, "CUBIC: a new TCP-friendly | |||
| high-speed TCP variant", ACM SIGOPS Operating Systems | high-speed TCP variant", ACM SIGOPS Operating Systems | |||
| Review vol. 42, no. 5, pp. 64-74, | Review, vol. 42, no. 5, pp. 64-74, | |||
| DOI 10.1145/1400097.1400105, July 2008, | DOI 10.1145/1400097.1400105, July 2008, | |||
| <https://doi.org/10.1145/1400097.1400105>. | <https://doi.org/10.1145/1400097.1400105>. | |||
| [K03] Kelly, T., "Scalable TCP: improving performance in | [K03] Kelly, T., "Scalable TCP: improving performance in | |||
| highspeed wide area networks", ACM SIGCOMM Computer | highspeed wide area networks", ACM SIGCOMM Computer | |||
| Communication Review vol. 33, no. 2, pp. 83-91, | Communication Review, vol. 33, no. 2, pp. 83-91, | |||
| DOI 10.1145/956981.956989, April 2003, | DOI 10.1145/956981.956989, April 2003, | |||
| <https://doi.org/10.1145/956981.956989>. | <https://doi.org/10.1145/956981.956989>. | |||
| [LBEWK16] Lukaseder, T., Bradatsch, L., Erb, B., Van Der Heijden, | [LBEWK16] Lukaseder, T., Bradatsch, L., Erb, B., Van Der Heijden, | |||
| R., and F. Kargl, "A Comparison of TCP Congestion Control | R., and F. Kargl, "A Comparison of TCP Congestion Control | |||
| Algorithms in 10G Networks", 2016 IEEE 41st Conference on | Algorithms in 10G Networks", 2016 IEEE 41st Conference on | |||
| Local Computer Networks (LCN), DOI 10.1109/lcn.2016.121, | Local Computer Networks (LCN), DOI 10.1109/lcn.2016.121, | |||
| November 2016, <https://doi.org/10.1109/lcn.2016.121>. | November 2016, <https://doi.org/10.1109/lcn.2016.121>. | |||
| [LIU16] Liu, K. and J. Lee, "On Improving TCP Performance over | [LIU16] Liu, K. and J. Lee, "On Improving TCP Performance over | |||
| Mobile Data Networks", IEEE Transactions on Mobile | Mobile Data Networks", IEEE Transactions on Mobile | |||
| Computing vol. 15, no. 10, pp. 2522-2536, | Computing, vol. 15, no. 10, pp. 2522-2536, | |||
| DOI 10.1109/tmc.2015.2500227, October 2016, | DOI 10.1109/tmc.2015.2500227, October 2016, | |||
| <https://doi.org/10.1109/tmc.2015.2500227>. | <https://doi.org/10.1109/tmc.2015.2500227>. | |||
| [RFC3522] Ludwig, R. and M. Meyer, "The Eifel Detection Algorithm | [RFC3522] Ludwig, R. and M. Meyer, "The Eifel Detection Algorithm | |||
| for TCP", RFC 3522, DOI 10.17487/RFC3522, April 2003, | for TCP", RFC 3522, DOI 10.17487/RFC3522, April 2003, | |||
| <https://www.rfc-editor.org/rfc/rfc3522>. | <https://www.rfc-editor.org/info/rfc3522>. | |||
| [RFC3649] Floyd, S., "HighSpeed TCP for Large Congestion Windows", | [RFC3649] Floyd, S., "HighSpeed TCP for Large Congestion Windows", | |||
| RFC 3649, DOI 10.17487/RFC3649, December 2003, | RFC 3649, DOI 10.17487/RFC3649, December 2003, | |||
| <https://www.rfc-editor.org/rfc/rfc3649>. | <https://www.rfc-editor.org/info/rfc3649>. | |||
| [RFC3708] Blanton, E. and M. Allman, "Using TCP Duplicate Selective | [RFC3708] Blanton, E. and M. Allman, "Using TCP Duplicate Selective | |||
| Acknowledgement (DSACKs) and Stream Control Transmission | Acknowledgement (DSACKs) and Stream Control Transmission | |||
| Protocol (SCTP) Duplicate Transmission Sequence Numbers | Protocol (SCTP) Duplicate Transmission Sequence Numbers | |||
| (TSNs) to Detect Spurious Retransmissions", RFC 3708, | (TSNs) to Detect Spurious Retransmissions", RFC 3708, | |||
| DOI 10.17487/RFC3708, February 2004, | DOI 10.17487/RFC3708, February 2004, | |||
| <https://www.rfc-editor.org/rfc/rfc3708>. | <https://www.rfc-editor.org/info/rfc3708>. | |||
| [RFC3742] Floyd, S., "Limited Slow-Start for TCP with Large | [RFC3742] Floyd, S., "Limited Slow-Start for TCP with Large | |||
| Congestion Windows", RFC 3742, DOI 10.17487/RFC3742, March | Congestion Windows", RFC 3742, DOI 10.17487/RFC3742, March | |||
| 2004, <https://www.rfc-editor.org/rfc/rfc3742>. | 2004, <https://www.rfc-editor.org/info/rfc3742>. | |||
| [RFC6937] Mathis, M., Dukkipati, N., and Y. Cheng, "Proportional | [RFC6937] Mathis, M., Dukkipati, N., and Y. Cheng, "Proportional | |||
| Rate Reduction for TCP", RFC 6937, DOI 10.17487/RFC6937, | Rate Reduction for TCP", RFC 6937, DOI 10.17487/RFC6937, | |||
| May 2013, <https://www.rfc-editor.org/rfc/rfc6937>. | May 2013, <https://www.rfc-editor.org/info/rfc6937>. | |||
| [RFC7661] Fairhurst, G., Sathiaseelan, A., and R. Secchi, "Updating | [RFC7661] Fairhurst, G., Sathiaseelan, A., and R. Secchi, "Updating | |||
| TCP to Support Rate-Limited Traffic", RFC 7661, | TCP to Support Rate-Limited Traffic", RFC 7661, | |||
| DOI 10.17487/RFC7661, October 2015, | DOI 10.17487/RFC7661, October 2015, | |||
| <https://www.rfc-editor.org/rfc/rfc7661>. | <https://www.rfc-editor.org/info/rfc7661>. | |||
| [RFC8312] Rhee, I., Xu, L., Ha, S., Zimmermann, A., Eggert, L., and | [RFC8312] Rhee, I., Xu, L., Ha, S., Zimmermann, A., Eggert, L., and | |||
| R. Scheffenegger, "CUBIC for Fast Long-Distance Networks", | R. Scheffenegger, "CUBIC for Fast Long-Distance Networks", | |||
| RFC 8312, DOI 10.17487/RFC8312, February 2018, | RFC 8312, DOI 10.17487/RFC8312, February 2018, | |||
| <https://www.rfc-editor.org/rfc/rfc8312>. | <https://www.rfc-editor.org/info/rfc8312>. | |||
| [RFC8511] Khademi, N., Welzl, M., Armitage, G., and G. Fairhurst, | [RFC8511] Khademi, N., Welzl, M., Armitage, G., and G. Fairhurst, | |||
| "TCP Alternative Backoff with ECN (ABE)", RFC 8511, | "TCP Alternative Backoff with ECN (ABE)", RFC 8511, | |||
| DOI 10.17487/RFC8511, December 2018, | DOI 10.17487/RFC8511, December 2018, | |||
| <https://www.rfc-editor.org/rfc/rfc8511>. | <https://www.rfc-editor.org/info/rfc8511>. | |||
| [RFC9000] Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based | [RFC9000] Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based | |||
| Multiplexed and Secure Transport", RFC 9000, | Multiplexed and Secure Transport", RFC 9000, | |||
| DOI 10.17487/RFC9000, May 2021, | DOI 10.17487/RFC9000, May 2021, | |||
| <https://www.rfc-editor.org/rfc/rfc9000>. | <https://www.rfc-editor.org/info/rfc9000>. | |||
| [RFC9260] Stewart, R., Tüxen, M., and K. Nielsen, "Stream Control | [RFC9260] Stewart, R., Tüxen, M., and K. Nielsen, "Stream Control | |||
| Transmission Protocol", RFC 9260, DOI 10.17487/RFC9260, | Transmission Protocol", RFC 9260, DOI 10.17487/RFC9260, | |||
| June 2022, <https://www.rfc-editor.org/rfc/rfc9260>. | June 2022, <https://www.rfc-editor.org/info/rfc9260>. | |||
| [SXEZ19] Sun, W., Xu, L., Elbaum, S., and D. Zhao, "Model-Agnostic | [SXEZ19] Sun, W., Xu, L., Elbaum, S., and D. Zhao, "Model-Agnostic | |||
| and Efficient Exploration of Numerical Congestion Control | and Efficient Exploration of Numerical Congestion Control | |||
| State Space of Real-World TCP Implementations", IEEE/ACM | State Space of Real-World TCP Implementations", IEEE/ACM | |||
| Transactions on Networking vol. 29, no. 5, pp. 1990-2004, | Transactions on Networking, vol. 29, no. 5, pp. 1990-2004, | |||
| DOI 10.1109/tnet.2021.3078161, October 2021, | DOI 10.1109/tnet.2021.3078161, October 2021, | |||
| <https://doi.org/10.1109/tnet.2021.3078161>. | <https://doi.org/10.1109/tnet.2021.3078161>. | |||
| [XHR04] Xu, L., Harfoush, K., and I. Rhee, "Binary increase | [XHR04] Xu, L., Harfoush, K., and I. Rhee, "Binary increase | |||
| congestion control (BIC) for fast long-distance networks", | congestion control (BIC) for fast long-distance networks", | |||
| IEEE INFOCOM 2004, DOI 10.1109/infcom.2004.1354672, | IEEE INFOCOM 2004, DOI 10.1109/infcom.2004.1354672, March | |||
| February 2005, | 2004, <https://doi.org/10.1109/infcom.2004.1354672>. | |||
| <https://doi.org/10.1109/infcom.2004.1354672>. | ||||
| Appendix A. Acknowledgments | ||||
| Richard Scheffenegger and Alexander Zimmermann originally co-authored | ||||
| [RFC8312]. | ||||
| These individuals suggested improvements to this document: | ||||
| * Bob Briscoe | ||||
| * Christian Huitema | ||||
| * Gorry Fairhurst | ||||
| * Jonathan Morton | ||||
| * Juhamatti Kuusisaari | ||||
| * Junho Choi | ||||
| * Markku Kojo | ||||
| * Martin Duke | ||||
| * Martin Thomson | ||||
| * Matt Mathis | ||||
| * Matt Olson | ||||
| * Michael Welzl | ||||
| * Mirja Kühlewind | ||||
| * Mohit P. Tahiliani | ||||
| * Neal Cardwell | ||||
| * Praveen Balasubramanian | ||||
| * Randall Stewart | ||||
| * Richard Scheffenegger | ||||
| * Rod Grimes | ||||
| * Spencer Dawkins | ||||
| * Tom Henderson | ||||
| * Tom Petch | ||||
| * Wesley Rosenblum | ||||
| * Yoav Nir | ||||
| * Yoshifumi Nishida | ||||
| * Yuchung Cheng | ||||
| Appendix B. Evolution of CUBIC | ||||
| B.1. Since draft-ietf-tcpm-rfc8312bis-14 | ||||
| * Specify how security considerations of TCP applies to CUBIC. | ||||
| * Elaborate differences between RFC 5681 and Cubic. | ||||
| * Tweak math for better plaintext rendering. (#164 | ||||
| (https://github.com/NTAP/rfc8312bis/pull/164)) | ||||
| B.2. Since draft-ietf-tcpm-rfc8312bis-13 | ||||
| * Add contents of https://cse.unl.edu/~xu/avg_cubic_cwnd.pdf to | ||||
| Appendix C. | ||||
| * Multiple comments from Martin, define synchronized/asynchronized | ||||
| loss model, clean up 3465 reference, clarficiation for when Cubic | ||||
| is not in Reno-friendly region, referring to proof of avg Cubic | ||||
| window, better text for misbehaving nodes and fix typo in | ||||
| _cwnd_epoch_. (#158 (https://github.com/NTAP/rfc8312bis/pull/158)) | ||||
| B.3. Since draft-ietf-tcpm-rfc8312bis-12 | ||||
| * Fix plaintext version of Figure 5. | ||||
| B.4. Since draft-ietf-tcpm-rfc8312bis-11 | ||||
| * Fix various nits. (#157 (https://github.com/NTAP/rfc8312bis/ | ||||
| pull/157)) | ||||
| B.5. Since draft-ietf-tcpm-rfc8312bis-10 | ||||
| * Improve text related to [RFC7661]. (#149 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/149)) | ||||
| * Made variable naming a bit more consistent. (#156 | ||||
| (https://github.com/NTAP/rfc8312bis/pull/156)) | ||||
| B.6. Since draft-ietf-tcpm-rfc8312bis-09 | ||||
| * Improve text for Reno friendliness, multiplicative decrease and | ||||
| reference to HLRX07. (#152 (https://github.com/NTAP/rfc8312bis/ | ||||
| pull/152)) | ||||
| B.7. Since draft-ietf-tcpm-rfc8312bis-08 | ||||
| * Fix the text specifying when alpha_cubic SHOULD be set to 1 to | ||||
| indicate this should happen when cwnd >= cwnd_prior rather than | ||||
| cwnd >= W_max, since these are different in the fast convergence | ||||
| case (#146 (https://github.com/NTAP/rfc8312bis/pull/146)) | ||||
| * Restrict use of _cwnd_ directly on a congestion event (#148 | ||||
| (https://github.com/NTAP/rfc8312bis/pull/148)) | ||||
| B.8. Since draft-ietf-tcpm-rfc8312bis-07 | ||||
| * Document the WG discussion and decision around [RFC5033] and | ||||
| [RFC2914] (#145 (https://github.com/NTAP/rfc8312bis/pull/145)) | ||||
| B.9. Since draft-ietf-tcpm-rfc8312bis-06 | ||||
| * RFC7661 is safe even when cwnd grows beyond rwnd (#143 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/143)) | ||||
| B.10. Since draft-ietf-tcpm-rfc8312bis-05 | ||||
| * Clarify meaning of "application-limited" in Section 5.8 (#137 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/137)) | ||||
| * Create new subsections for spurious timeouts and spurious loss via | ||||
| ACK (#90 (https://github.com/NTAP/rfc8312bis/issues/90)) | ||||
| * Brief discussion of convergence in Section 5.6 (#96 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/96)) | ||||
| * Add more test results to Section 5 and update some references (#91 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/91)) | ||||
| * Change wording around setting ssthresh (#131 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/131)) | ||||
| B.11. Since draft-ietf-tcpm-rfc8312bis-04 | ||||
| * Fix incorrect math (#106 (https://github.com/NTAP/rfc8312bis/ | ||||
| issues/106)) | ||||
| * Update RFC5681 (#99 (https://github.com/NTAP/rfc8312bis/ | ||||
| issues/99)) | ||||
| * Rephrase text around algorithmic alternatives, add HyStart++ (#85 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/85), #86 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/86), #90 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/90)) | ||||
| * Clarify what we mean by "new ACK" and use it in the text in more | ||||
| places. (#101 (https://github.com/NTAP/rfc8312bis/issues/101)) | ||||
| * Rewrite the Responses to Sudden or Transient Events section (#98 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/98)) | ||||
| * Remove confusing text about _cwnd_epoch_ in Section 4.2 (#100 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/100)) | ||||
| * Change terminology from "AIMD" to "Reno" (#108 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/108)) | ||||
| * Moved MUST NOT from app-limited section to main cubic AI section | ||||
| (#97 (https://github.com/NTAP/rfc8312bis/issues/97)) | ||||
| * Clarify cwnd decrease during multiplicative decrease (#102 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/102)) | ||||
| * Clarify text around queuing and slow adaptation of CUBIC in | ||||
| wireless environments (#94 (https://github.com/NTAP/rfc8312bis/ | ||||
| issues/94)) | ||||
| * Set lower bound of cwnd to 1 MSS and use retransmit timer | ||||
| thereafter (#83 (https://github.com/NTAP/rfc8312bis/issues/83)) | ||||
| * Use FlightSize instead of cwnd to update ssthresh (#114 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/114)) | ||||
| B.12. Since draft-ietf-tcpm-rfc8312bis-03 | ||||
| * Remove reference from abstract (#82 | ||||
| (https://github.com/NTAP/rfc8312bis/pull/82)) | ||||
| B.13. Since draft-ietf-tcpm-rfc8312bis-02 | ||||
| * Description of packet loss rate _p_ (#65 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/65)) | ||||
| * Clarification of TCP Friendly Equation for ABC and Delayed ACK | ||||
| (#66 (https://github.com/NTAP/rfc8312bis/issues/66)) | ||||
| * add applicability to QUIC and SCTP (#61 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/61)) | ||||
| * clarity on setting alpha__aimd_ to 1 (#68 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/68)) | ||||
| * introduce alpha__cubic_ (#64 (https://github.com/NTAP/rfc8312bis/ | ||||
| issues/64)) | ||||
| * clarify _cwnd_ growth in convex region (#69 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/69)) | ||||
| * add guidance for using bytes and mention that segments count is | ||||
| decimal (#67 (https://github.com/NTAP/rfc8312bis/issues/67)) | ||||
| * add loss events detected by RACK and QUIC loss detection (#62 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/62)) | ||||
| B.14. Since draft-ietf-tcpm-rfc8312bis-01 | ||||
| * address Michael Scharf's editorial suggestions. (#59 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/59)) | ||||
| * add "Note to the RFC Editor" about removing underscores | ||||
| B.15. Since draft-ietf-tcpm-rfc8312bis-00 | ||||
| * use updated xml2rfc with better text rendering of subscripts | ||||
| B.16. Since draft-eggert-tcpm-rfc8312bis-03 | ||||
| * fix spelling nits | ||||
| * rename to draft-ietf | ||||
| * define _W_max_ more clearly | ||||
| B.17. Since draft-eggert-tcpm-rfc8312bis-02 | ||||
| * add definition for segments_acked and alpha__aimd_. (#47 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/47)) | ||||
| * fix a mistake in _W_max_ calculation in the fast convergence | ||||
| section. (#51 (https://github.com/NTAP/rfc8312bis/issues/51)) | ||||
| * clarity on setting _ssthresh_ and _cwnd_epoch_ during | ||||
| multiplicative decrease. (#53 (https://github.com/NTAP/rfc8312bis/ | ||||
| issues/53)) | ||||
| B.18. Since draft-eggert-tcpm-rfc8312bis-01 | ||||
| * rename TCP-Friendly to AIMD-Friendly and rename Standard TCP to | ||||
| AIMD TCP to avoid confusion as CUBIC has been widely used on the | ||||
| Internet. (#38 (https://github.com/NTAP/rfc8312bis/issues/38)) | ||||
| * change introductory text to reflect the significant broader | ||||
| deployment of CUBIC on the Internet. (#39 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/39)) | ||||
| * rephrase introduction to avoid referring to variables that have | ||||
| not been defined yet. | ||||
| B.19. Since draft-eggert-tcpm-rfc8312bis-00 | ||||
| * acknowledge former co-authors (#15 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/15)) | ||||
| * prevent _cwnd_ from becoming less than two (#7 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/7)) | ||||
| * add list of variables and constants (#5 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/5), #6 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/6)) | ||||
| * update _K_'s definition and add bounds for CUBIC _target_ _cwnd_ | ||||
| [SXEZ19] (#1 (https://github.com/NTAP/rfc8312bis/issues/1), #14 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/14)) | ||||
| * update _W_est_ to use AIMD approach (#20 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/20)) | ||||
| * set alpha__aimd_ to 1 once _W_est_ reaches _W_max_ (#2 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/2)) | ||||
| * add Vidhi as co-author (#17 (https://github.com/NTAP/rfc8312bis/ | ||||
| issues/17)) | ||||
| * note for Fast Recovery during _cwnd_ decrease due to congestion | ||||
| event (#11 (https://github.com/NTAP/rfc8312bis/issues/11)) | ||||
| * add section for spurious congestion events (#23 | ||||
| (https://github.com/NTAP/rfc8312bis/issues/23)) | ||||
| * initialize _W_est_ after timeout and remove variable | ||||
| _W_(last_max)_ (#28 (https://github.com/NTAP/rfc8312bis/ | ||||
| issues/28)) | ||||
| B.20. Since RFC8312 | ||||
| * converted to Markdown and xml2rfc v3 | ||||
| * updated references (as part of the conversion) | ||||
| * updated author information | ||||
| * various formatting changes | ||||
| * move to Standards Track | ||||
| B.21. Since the Original Paper | Appendix A. Evolution of CUBIC since the Original Paper | |||
| CUBIC has gone through a few changes since the initial release | CUBIC has gone through a few changes since the initial release | |||
| [HRX08] of its algorithm and implementation. This section highlights | [HRX08] of its algorithm and implementation. This appendix | |||
| the differences between the original paper and [RFC8312]. | highlights the differences between the original paper and [RFC8312]. | |||
| * The original paper [HRX08] includes the pseudocode of CUBIC | * The original paper [HRX08] includes the pseudocode of CUBIC | |||
| implementation using Linux's pluggable congestion control | implementation using Linux's pluggable congestion control | |||
| framework, which excludes system-specific optimizations. The | framework, which excludes system-specific optimizations. The | |||
| simplified pseudocode might be a good source to start with and | simplified pseudocode might be a good starting point for learning | |||
| understand CUBIC. | about CUBIC. | |||
| * [HRX08] also includes experimental results showing its performance | * [HRX08] also includes experimental results showing its performance | |||
| and fairness. | and fairness. | |||
| * The definition of beta__cubic_ constant was changed in [RFC8312]. | * The definition of the β__cubic_ constant was changed in [RFC8312]. | |||
| For example, beta__cubic_ in the original paper was the window | For example, β__cubic_ in the original paper was referred to as | |||
| decrease constant while [RFC8312] changed it to CUBIC | the window decrease constant, while [RFC8312] changed it to "CUBIC | |||
| multiplication decrease factor. With this change, the current | multiplicative decrease factor". With this change, the current | |||
| congestion window size after a congestion event in [RFC8312] was | congestion window size after a congestion event as listed in | |||
| beta__cubic_ * _W_max_ while it was (1-beta__cubic_) * _W_max_ in | [RFC8312] was β__cubic_ * _W_max_, while it was (1-β__cubic_) * | |||
| the original paper. | _W_max_ in the original paper. | |||
| * Its pseudocode used _W_(last_max)_ while [RFC8312] used _W_max_. | * Its pseudocode used _W_(last_max)_, while [RFC8312] used _W_max_. | |||
| * Its AIMD-friendly window was _W_tcp_ while [RFC8312] used _W_est_. | * Its AIMD-friendly window was _W_tcp_, while [RFC8312] used | |||
| _W_est_. | ||||
| Appendix C. Proof of the Average CUBIC Window Size | Appendix B. Proof of the Average CUBIC Window Size | |||
| This appendix contains a proof for the average CUBIC window size | This appendix contains a proof for the average CUBIC window size | |||
| _AVG_W_cubic_ in Figure 6. | _AVG_W_cubic_ in Figure 6. | |||
| We find _AVG_W_cubic_ under a deterministic loss model, where the | We find _AVG_W_cubic_ under a deterministic loss model, where the | |||
| number of packets between two successive packet losses is 1/_p_. With | number of packets between two successive packet losses is | |||
| this model, CUBIC always operates with the concave window profile and | 1/_p_. With this model, CUBIC always operates with the concave | |||
| the time period between two successive packet losses is _K_. | window profile and the time period between two successive packet | |||
| losses is _K_. | ||||
| The average window size _AVG_W_cubic_ is defined as follows, where | The average window size _AVG_W_cubic_ is defined as follows, where | |||
| the numerator 1/_p_ is the total number of packets between two | the numerator 1/_p_ is the total number of packets between two | |||
| successive packet losses, and the denominator _K_/_RTT_ is the total | successive packet losses and the denominator _K_/_RTT_ is the total | |||
| number of RTTs between two successive packet losses. | number of RTTs between two successive packet losses. | |||
| 1 | 1 | |||
| ─ | ─ | |||
| p | p | |||
| AVG_W = ─── | AVG_W = ─── | |||
| cubic K | cubic K | |||
| ─── | ─── | |||
| RTT | RTT | |||
| Figure 9 | Figure 9 | |||
| Below, we find _K_ as a function of CUBIC parameters β__cubic_ and | Below, we find _K_ as a function of CUBIC parameters β__cubic_ and | |||
| _C_, and network parameters _p_ and _RTT_. According to the | _C_, and network parameters _p_ and _RTT_. According to the | |||
| definition of _K_ in Figure 2, we have | definition of _K_ in Figure 2, we have | |||
| ┌────────────────────┐ | ┌────────────────────┐ | |||
| 3 │W - W * β | 3 │W - W * β | |||
| ╲ │ max max cubic | ╲ │ max max cubic | |||
| K = ╲ │──────────────────── | K = ╲ │──────────────────── | |||
| ╲│ C | ╲│ C | |||
| Figure 10 | Figure 10 | |||
| The total number of packets between two successive packet losses can | The total number of packets between two successive packet losses can | |||
| also be obtained as follows using the window increase function in | also be obtained as follows, using the window increase function in | |||
| Figure 1. Specifically, the window size in the first RTT (i.e., | Figure 1. Specifically, the window size in the first RTT (i.e., | |||
| _n_=1 or equivalently _t_=0) is _C_(-_K_)^3+_W_max_ and the window | _n_=1, or equivalently, _t_=0) is _C_(-_K_)^3+_W_max_ and the window | |||
| size in the last RTT (i.e., _n_=_K_/_RTT_ or equivalently _t_=_K_- | size in the last RTT (i.e., _n_=_K_/_RTT_, or equivalently, _t_=_K_- | |||
| _RTT_) is _C_(-_RTT_)^3+_W_max_. | _RTT_) is _C_(-_RTT_)^3+_W_max_. | |||
| K | K | |||
| ─── | ─── | |||
| RTT | RTT | |||
| ⎯⎯ | ⎯⎯ | |||
| 1 ╲ ⎛ 3 ⎞ | 1 ╲ ⎛ 3 ⎞ | |||
| ─ = ╱ ⎜C((n-1) * RTT-K) + W ⎟ | ─ = ╱ ⎜C((n-1) * RTT-K) + W ⎟ | |||
| p ⎺⎺ ⎝ max⎠ | p ⎺⎺ ⎝ max⎠ | |||
| n=1 | n=1 | |||
| skipping to change at page 39, line 40 ¶ | skipping to change at line 1466 ¶ | |||
| 3 1 ⎛ K ⎞ K | 3 1 ⎛ K ⎞ K | |||
| ≈ -C * RTT * ─ *⎜───⎟ + W * ─── | ≈ -C * RTT * ─ *⎜───⎟ + W * ─── | |||
| 4 ⎝RTT⎠ max RTT | 4 ⎝RTT⎠ max RTT | |||
| 4 | 4 | |||
| 1 K K | 1 K K | |||
| = -C * ─ * ─── + W * ─── | = -C * ─ * ─── + W * ─── | |||
| 4 RTT max RTT | 4 RTT max RTT | |||
| Figure 11 | Figure 11 | |||
| After solving Figure 10 and Figure 11 for _K_ and _W_max_, we have | After solving the equations in Figures 10 and 11 for _K_ and _W_max_, | |||
| we have | ||||
| ┌──────────────────────┐ | ┌──────────────────────┐ | |||
| │ 4 * ⎛1-β ⎞ | │ 4 * ⎛1-β ⎞ | |||
| 4 │ ⎝ cubic⎠ RTT | 4 │ ⎝ cubic⎠ RTT | |||
| K = ╲ │──────────────── * ─── | K = ╲ │──────────────── * ─── | |||
| ╲ │C * ⎛3 + β ⎞ p | ╲ │C * ⎛3 + β ⎞ p | |||
| ╲│ ⎝ cubic⎠ | ╲│ ⎝ cubic⎠ | |||
| Figure 12 | Figure 12 | |||
| The average CUBIC window size _AVG_W_cubic_ can be obtained by | The average CUBIC window size _AVG_W_cubic_ can be obtained by | |||
| substituting _K_ with Figure 12 in Figure 9 | substituting _K_ from Figure 12 in Figure 9. | |||
| 1 ┌───────────────────────┐ | 1 ┌───────────────────────┐ | |||
| ─ │C * ⎛3 + β ⎞ 3 | ─ │C * ⎛3 + β ⎞ 3 | |||
| p 4 │ ⎝ cubic⎠ RTT | p 4 │ ⎝ cubic⎠ RTT | |||
| AVG_W = ─── = ╲ │──────────────── * ──── | AVG_W = ─── = ╲ │──────────────── * ──── | |||
| cubic K ╲ │ 4 * ⎛1-β ⎞ 3 | cubic K ╲ │ 4 * ⎛1-β ⎞ 3 | |||
| ─── ╲│ ⎝ cubic⎠ p | ─── ╲│ ⎝ cubic⎠ p | |||
| RTT | RTT | |||
| Acknowledgments | ||||
| Richard Scheffenegger and Alexander Zimmermann originally coauthored | ||||
| [RFC8312]. | ||||
| These individuals suggested improvements to this document: | ||||
| * Bob Briscoe | ||||
| * Christian Huitema | ||||
| * Gorry Fairhurst | ||||
| * Jonathan Morton | ||||
| * Juhamatti Kuusisaari | ||||
| * Junho Choi | ||||
| * Markku Kojo | ||||
| * Martin Duke | ||||
| * Martin Thomson | ||||
| * Matt Mathis | ||||
| * Matt Olson | ||||
| * Michael Welzl | ||||
| * Mirja Kühlewind | ||||
| * Mohit P. Tahiliani | ||||
| * Neal Cardwell | ||||
| * Praveen Balasubramanian | ||||
| * Randall Stewart | ||||
| * Richard Scheffenegger | ||||
| * Rod Grimes | ||||
| * Spencer Dawkins | ||||
| * Tom Henderson | ||||
| * Tom Petch | ||||
| * Wesley Rosenblum | ||||
| * Yoav Nir | ||||
| * Yoshifumi Nishida | ||||
| * Yuchung Cheng | ||||
| Authors' Addresses | Authors' Addresses | |||
| Lisong Xu | Lisong Xu | |||
| University of Nebraska-Lincoln | University of Nebraska-Lincoln | |||
| Department of Computer Science and Engineering | Department of Computer Science and Engineering | |||
| Lincoln, NE 68588-0115 | Lincoln, NE 68588-0115 | |||
| United States of America | United States of America | |||
| Email: xu@unl.edu | Email: xu@unl.edu | |||
| URI: https://cse.unl.edu/~xu/ | URI: https://cse.unl.edu/~xu/ | |||
| Sangtae Ha | Sangtae Ha | |||
| University of Colorado at Boulder | University of Colorado at Boulder | |||
| Department of Computer Science | Department of Computer Science | |||
| Boulder, CO 80309-0430 | Boulder, CO 80309-0430 | |||
| United States of America | United States of America | |||
| Email: sangtae.ha@colorado.edu | Email: sangtae.ha@colorado.edu | |||
| URI: https://netstech.org/sangtaeha/ | URI: https://netstech.org/sangtaeha/ | |||
| Injong Rhee | Injong Rhee | |||
| Bowery Farming | Bowery Farming | |||
| 151 W 26TH Street, 12TH Floor | 151 W 26th Street, 12th Floor | |||
| New York, NY 10001 | New York, NY 10001 | |||
| United States of America | United States of America | |||
| Email: injongrhee@gmail.com | Email: injongrhee@gmail.com | |||
| Vidhi Goel | Vidhi Goel | |||
| Apple Inc. | Apple Inc. | |||
| One Apple Park Way | One Apple Park Way | |||
| Cupertino, California 95014 | Cupertino, CA 95014 | |||
| United States of America | United States of America | |||
| Email: vidhi_goel@apple.com | Email: vidhi_goel@apple.com | |||
| Lars Eggert (editor) | Lars Eggert (editor) | |||
| NetApp | NetApp | |||
| Stenbergintie 12 B | Stenbergintie 12 B | |||
| FI-02700 Kauniainen | FI-02700 Kauniainen | |||
| Finland | Finland | |||
| Email: lars@eggert.org | Email: lars@eggert.org | |||
| URI: https://eggert.org/ | URI: https://eggert.org/ | |||
| End of changes. 175 change blocks. | ||||
| 799 lines changed or deleted | 516 lines changed or added | |||
This html diff was produced by rfcdiff 1.48. | ||||