Ayush Mishra
Xiangpeng Sun
Atishya Jain
Raj Joshi
Ben Leong
Abstract
In 2016, Google proposed and deployed a new TCP variant called BBR. BBR represents a major departure from traditional congestion-window-based congestion control. Instead of using loss as a congestion signal, BBR uses estimates of the bandwidth and round-trip delays to regulate its sending rate. The last major study on the distribution of TCP variants on the Internet was done in 2011, so it is timely to conduct a new census given the recent developments around BBR. To this end, we designed and implemented Gordon, a tool that allows us to measure the exact congestion window (cwnd) corresponding to each successive RTT in the TCP connection response of a congestion control algorithm. To compare a measured flow to the known variants, we created a localized bottleneck where we can introduce a variety of network changes like loss events, bandwidth change, and increased delay, and normalize all measurements by RTT. An offline classifier is used to identify the TCP variant based on the cwnd trace over time. Our results suggest that CUBIC is currently the dominant TCP variant on the Internet, and it is deployed on about 36% of the websites in the Alexa Top 20,000 list. While BBR and its variant BBR G1.1 are currently in second place with a 22% share by website count, their present share of total Internet traffic volume is estimated to be larger than 40%. We also found that Akamai has deployed a unique loss-agnostic rate-based TCP variant on some 6% of the Alexa Top 20,000 websites and there are likely other undocumented variants. The traditional assumption that TCP variants "in the wild" will come from a small known set is not likely to be true anymore. We predict that some variant of BBR seems poised to replace CUBIC as the next dominant TCP variant on the Internet.
- Andrea Baiocchi, Angelo P Castellani, and Francesco Vacirca. 2007. YeAH-TCP: yet another highspeed TCP. In Proceedings of PFLDnet .Google Scholar
- Lawrence S. Brakmo, Sean W. O'Malley, and Larry L. Peterson. 1994. TCP Vegas: New Techniques for Congestion Detection and Avoidance. In Proceedings of SIGCOMM .Google Scholar
- Neal Cardwell, Yuchung Cheng, C. Stephen Gunn, Soheil Hassas Yeganeh, and Van Jacobson. 2017a. BBR: Congestion-based Congestion Control . CACM , Vol. 60, 2 (2017), 58--66.Google ScholarDigital Library
- Neal Cardwell, Yuchung Cheng, Soheil Hassas Yeganeh, and Van Jacobson. 2017b. BBR Congestion Control . IETF Draft. (2017). https://datatracker.ietf.org/doc/html/draft-cardwell-iccrg-bbr-congestion-control-00Google Scholar
- Neal Cardwell, Yuchung Cheng, Soheil Hassas Yeganeh, Ian Swett, Victor Vasiliev, Priyaranjan Jha, Yousuk Seung, Matt Mathis, and Van Jacobson. 2019. BBR v2 - A Model-based Congestion Control . ICCRG at IETF 104. (2019). https://bit.ly/2HgGOuQGoogle Scholar
- Claudio Casetti, Mario Gerla, Saverio Mascolo, Medy Y Sanadidi, and Ren Wang. 2002. TCP Westwood: end-to-end congestion control for wired/wireless networks. Wireless Networks , Vol. 8, 5 (2002), 467--479.Google ScholarDigital Library
- Xiaoyu Chen, Shugong Xu, Xudong Chen, Shan Cao, Shunqing Zhang, and Yanzan Sun. 2019. Passive TCP Identification for Wired and Wireless Networks: A Long-Short Term Memory Approach . arXiv preprint:1904.04430 (2019).Google Scholar
- Yuchung Cheng, Neal Cardwell, Nandita Dukkipati, and Priyaranjan Jha. 2019. RACK: a time-based fast loss detection algorithm for TCP . IETF Draft. (2019). https://datatracker.ietf.org/doc/html/draft-ietf-tcpm-rack-05Google Scholar
- Douglas E. Comer and John C. Lin. 1994. Probing TCP Implementations. In Proceedings of USTC .Google Scholar
- DARPA. 1981. Internet Protocol. RFC 791. (1981).Google Scholar
- Sally Floyd. 2003. HighSpeed TCP for Large Congestion Windows . RFC 3649. (2003).Google Scholar
- Cheng Peng Fu and S. C. Liew. 2006. TCP Veno: TCP Enhancement for Transmission over Wireless Access Networks . IEEE JSAC , Vol. 21, 2 (2006), 216--228.Google ScholarDigital Library
- Sangtae Ha, Injong Rhee, and Lisong Xu. 2008. CUBIC: A New TCP-friendly High-speed TCP Variant . SIGOPS Operating Systems Review , Vol. 42, 5 (2008), 64--74.Google ScholarDigital Library
- Desta H. Hagos, Paal E. Engelstad, Anis Yazidi, and Øivind Kure. 2018. General TCP State Inference Model From Passive Measurements Using Machine Learning Techniques. IEEE Access , Vol. 6 (2018), 28372--28387.Google ScholarCross Ref
- Mario Hock, Roland Bless, and Martina Zitterbart. 2017. Experimental Evaluation of BBR Congestion Control. Proceedings of ICNP.Google ScholarCross Ref
- Alexa Internet Inc. 2018. The Top 500 websites on the Internet. (2018). https://www.alexa.com/topsitesGoogle Scholar
- Yuchung Cheng Jerry Chu, Nandita Dukkipati and Matt Mathis. 2013. Increasing TCP's Initial Window . RFC 6928. (2013).Google Scholar
- Tom Kelly. 2003. Scalable TCP: Improving Performance in Highspeed Wide Area Networks. SIGCOMM CCR , Vol. 33, 2 (2003), 83--91.Google ScholarDigital Library
- Douglas Leith, R Shorten, and Y Lee. 2005. H-TCP: A framework for congestion control in high-speed and long-distance networks. In Proceedings of PFLDnet .Google Scholar
- Shao Liu, Tamer Bacsar, and R. Srikant. 2006. TCP-Illinois: A Loss and Delay-based Congestion Control Algorithm for High-speed Networks. In Proceedings of VALUETOOLS .Google Scholar
- Ralf Lübben and Markus Fidler. 2016. On characteristic features of the application level delay distribution of TCP congestion avoidance. In Proceedings of ICC .Google ScholarCross Ref
- Alberto Medina, Mark Allman, and Sally Floyd. 2005. Measuring the Evolution of Transport Protocols in the Internet. SIGCOMM CCR , Vol. 35, 2 (2005), 37--52.Google ScholarDigital Library
- Ravi Netravali, Anirudh Sivaraman, Somak Das, Ameesh Goyal, Keith Winstein, James Mickens, and Hari Balakrishnan. 2015. Mahimahi: Accurate Record-and-Replay for HTTP . Proceedings of ATC .Google Scholar
- Netfilter Organization. 2019. libnetfilter_queue. (2019). https://bit.ly/2HimY17Google Scholar
- Jitendra Padhye and Sally Floyd. 2001. On Inferring TCP Behavior. In Proceedings of SIGCOMM .Google Scholar
- Vern Paxson and Mark Allman. 2009. TCP Congestion Control . RFC 5681. (2009).Google Scholar
- Brien Posey. 2019. Explore the Cubic congestion control provider for Windows. (2019). https://bit.ly/2VfhxoAGoogle Scholar
- GNU Project. 2019. wget. (2019). https://www.gnu.org/software/wgetGoogle Scholar
- Jan Rüth, Christian Bormann, and Oliver Hohlfeld. 2017. Large-scale scanning of TCP's initial window. In Proceedings of IMC .Google ScholarDigital Library
- Canada Sandvine Inc. Waterloo, ON. 2018. The 2018 Global Internet Phenomena Report. (2018). https://www.sandvine.com/phenomenaGoogle Scholar
- W. Sun, L. Xu, and S. Elbaum. 2018. Scalably Testing Congestion Control Algorithms of Real-World TCP Implementations. In Proceedings of ICC .Google Scholar
- Kun Tan, Jingmin Song, Qian Zhang, and Murad Sridharan. 2006. A compound TCP approach for high-speed and long distance networks. In Proceedings of INFOCOM .Google ScholarCross Ref
- Ranysha Ware, Matthew K. Mukerjee, Srinivasan Seshan, and Justine Sherry. 2019. Modeling BBR's Interactions with Loss-Based Congestion Control. In Proceedings of IMC .Google ScholarDigital Library
- David X Wei, Cheng Jin, Steven H Low, and Sanjay Hegde. 2007. FAST TCP. IEEE/ACM Transactions on Networking.Google Scholar
- Lisong Xu, K. Harfoush, and Injong Rhee. 2004. Binary increase congestion control (BIC) for fast long-distance networks. In Proceedings of INFOCOM .Google Scholar
- Peng Yang, Juan Shao, Wen Luo, Lisong Xu, Jitendra Deogun, and Ying Lu. 2011. TCP Congestion Avoidance Algorithm Identification. IEEE/ACM Transactions on Networking , Vol. 22, 4 (2011), 1311--1324.Google ScholarDigital Library
- Peng Yang and Lisong Xu. 2011. A survey of deployment information of delay-based TCP congestion avoidance algorithm for transmitting multimedia data. In Proceedings of GLOBECOM Workshops .Google ScholarCross Ref
Index Terms
- The Great Internet TCP Congestion Control Census
Recommendations
The Great Internet TCP Congestion Control Census
SIGMETRICS '20: Abstracts of the 2020 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer SystemsIn 2016, Google proposed and deployed a new TCP variant called BBR. BBR represents a major departure from traditional congestion control as it uses estimates of bandwidth and round-trip delays to regulate its sending rate. BBR has since been introduced ...
The Great Internet TCP Congestion Control Census
In 2016, Google proposed and deployed a new TCP variant called BBR. BBR represents a major departure from traditional congestion control as it uses estimates of bandwidth and round-trip delays to regulate its sending rate. BBR has since been introduced ...
TCP congestion control algorithm for heterogeneous Internet
The available bandwidth, round trip time (RTT) and packet loss rate can vary over many orders of magnitude, which characterizes the heterogeneity of the Internet. To cope with the heterogeneity in Internet congestion control, we propose a new TCP ...
Comments