Pravein Govindan Kannan
Raj Joshi
Mun Choon Chan
ABSTRACT
Current implementations of time synchronization protocols (e.g. PTP) in standard industry-grade switches handle the protocol stack in the slow-path (control-plane). With new use cases of in-network computing using programmable switching ASICs, global time-synchronization in the data-plane is very much necessary for supporting distributed applications. In this paper, we explore the possibility of using programmable switching ASICs to design and implement a time synchronization protocol, DPTP, with the core logic running in the data-plane. We perform comprehensive measurement studies on the variable delay characteristics in the switches and NICs under different traffic conditions. Based on the measurement insights, we design and implement DPTP on a Barefoot Tofino switch using the P4 programming language. Our evaluation on a multi-switch testbed shows that DPTP can achieve median and 99th percentile synchronization error of 19 ns and 47 ns between 2 switches, 4-hops apart, in the presence of clock drifts and under heavy network load.
- D. L. Mills. Internet time synchronization: the network time protocol. IEEE Transactions on Communications, 39(10):1482--1493, 1991.Google Scholar
Cross Ref
- IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems. IEEE Std 1588-2008 (Revision of IEEE Std 1588-2002), pages 1--300, 2008.Google Scholar
- Ki Suh Lee, Han Wang, Vishal Shrivastav, and Hakim Weatherspoon. Globally synchronized time via datacenter networks. In Proceedings of SIGCOMM, 2016. Google ScholarDigital Library
- R. Zarick, M. Hagen, and R. BartoÅą. Transparent clocks vs. enterprise ethernet switches. In Proceedings of IEEE International Symposium on Precision Clock Synchronization for Measurement, Control and Communication, 2011.Google ScholarCross Ref
- Yilong Geng, Shiyu Liu, Zi Yin, Ashish Naik, Balaji Prabhakar, Mendel Rosenblum, and Amin Vahdat. Exploiting a natural network effect for scalable, fine-grained clock synchronization. In Proceedings of NSDI, 2018. Google ScholarDigital Library
- Huynh Tu Dang, Daniele Sciascia, Marco Canini, Fernando Pedone, and Robert Soulé. NetPaxos: Consensus at Network Speed. In Proceedings of SOSR, 2015. Google ScholarDigital Library
- Huynh Tu Dang, Marco Canini, Fernando Pedone, and Robert Soulé. Paxos made switch-y. SIGCOMM CCR, 2016. Google ScholarDigital Library
- Xin Jin, Xiaozhou Li, Haoyu Zhang, Nate Foster, Jeongkeun Lee, Robert Soulé, Changhoon Kim, and Ion Stoica. Netchain: Scale-free sub-rtt coordination. In Proceedings of NSDI, 2018. Google ScholarDigital Library
- Xin Jin, Xiaozhou Li, Haoyu Zhang, Robert Soulé, Jeongkeun Lee, Nate Foster, Changhoon Kim, and Ion Stoica. Netcache: Balancing key-value stores with fast in-network caching. In Proceedings of SOSP, 2017. Google ScholarDigital Library
- Ming Liu, Liang Luo, Jacob Nelson, Luis Ceze, Arvind Krishnamurthy, and Kishore Atreya. Incbricks: Toward in-network computation with an in-network cache. In Proceedings of ASPLOS, 2017. Google ScholarDigital Library
- Jialin Li, Ellis Michael, and Dan R. K. Ports. Eris: Coordination-Free Consistent Transactions Using In-Network Concurrency Control. In Proceedings of SOSP, 2017. Google ScholarDigital Library
- Amin Tootoonchian, Aurojit Panda, Aida Nematzadeh, and Scoot Shenkar. Distributed shared memory for machine learning. In Proceedings of SysML, 2018.Google Scholar
- Thomas Kohler, Ruben Mayer, Frank Dürr, Marius Maaß, Sukanya Bhowmik, and Kurt Rothermel. P4CEP: Towards In-Network Complex Event Processing. In Proceedings of NetCompute, 2018. Google ScholarDigital Library
- Nofel Yaseen, John Sonchack, and Vincent Liu. Synchronized network snapshots. In Proceedings of SIGCOMM, 2018. Google ScholarDigital Library
- The world's fastest and most programmable networks. https://www.barefootnetworks.com/resources/worlds-fastest-most-programmable-networks/.Google Scholar
- C. D. Murta, P. R. Torres Jr., and P. Mohapatra. Qrpp1-4: Characterizing quality of time and topology in a time synchronization network. In Proceedings of Globecom, 2006.Google ScholarCross Ref
- Meinberg PTP Client. https://www.meinbergglobal.com/english/products/ptp-client-software-win-linux.htm.Google Scholar
- IEEE 1588 PTP and Analytics on Cisco Nexus 3548 Switch. https://www.cisco.com/c/en/us/products/collateral/switches/nexus-3000-series-switches/white-paper-c11-731501.html.Google Scholar
- FSMLabs Timekeeper Appliance. https://www.fsmlabs.com/timekeeper/enterprise-appliance/.Google Scholar
- James C. Corbett, Jeffrey Dean, Michael Epstein, Andrew Fikes, Christopher Frost, JJ Furman, Sanjay Ghemawat, Andrey Gubarev, Christopher Heiser, Peter Hochschild, Wilson Hsieh, Sebastian Kanthak, Eugene Kogan, Hongyi Li, Alexander Lloyd, Sergey Melnik, David Mwaura, David Nagle, Sean Quinlan, Rajesh Rao, Lindsay Rolig, Yasushi Saito, Michal Szymaniak, Christopher Taylor, Ruth Wang, and Dale Woodford. Spanner: Google's globally-distributed database. In Proceedings of OSDI, 2012. Google ScholarDigital Library
- W. Lewandowski, J. Azoubib, and W. J. Klepczynski. Gps: primary tool for time transfer. Proceedings of the IEEE, 1999.Google ScholarCross Ref
- Cavium. Xpliant ethernet switch product family, 2018.Google Scholar
- Barefoot Networks. Tofino, 2018.Google Scholar
- Intel. Flexpipe, 2018.Google Scholar
- Cisco UADP. https://blogs.cisco.com/enterprise/new-frontiers-anti-aging-treatment-for-your-network.Google Scholar
- Broadcom Trident 3. https://packetpushers.net/broadcom-trident3-programmable-varied-volume/.Google Scholar
- Xilinx SDNet. https://www.xilinx.com/products/design-tools/software-zone/sdnet.html.Google Scholar
- Portable Switch Architecture. https://p4.org/p4-spec/docs/PSA-v1.0.0.pdf.Google Scholar
- In-band Network Telemetry. https://p4.org/assets/INT-current-spec.pdf.Google Scholar
- Pravein Govindan Kannan, Ahmad Soltani, Mun Choon Chan, and Ee-Chien Chang. BNV: Enabling scalable network experimentation through bare-metal network virtualization. In Proceedings of USENIX Workshop on Cyber Security Experimentation and Test (CSET), 2018. Google ScholarDigital Library
- Pat Bosshart, Glen Gibb, Hun-Seok Kim, George Varghese, Nick McKeown, Martin Izzard, Fernando Mujica, and Mark Horowitz. Forwarding Metamorphosis: Fast Programmable Match-action Processing in Hardware for SDN. In Proceedings of SIGCOMM, 2013. Google ScholarDigital Library
- Junda Liu, Baohua Yan, Scott Shenker, and Michael Schapira. Data-driven network connectivity. HotNets-X, 2011. Google ScholarDigital Library
- Paul Emmerich, Sebastian Gallenmüller, Daniel Raumer, Florian Wohl-fart, and Georg Carle. MoonGen: A Scriptable High-Speed Packet Generator. In Proceedings of IMC, 2015. Google ScholarDigital Library
- P4 Language Consortium. 2018. Baseline switch.p4. https://github.com/p4lang/switch/blob/master/p4src/switch.p4.Google Scholar
- Dan R. K. Ports, Jialin Li, Vincent Liu, Naveen Kr. Sharma, and Arvind Krishnamurthy. Designing Distributed Systems Using Approximate Synchrony in Data Center Networks. In Proceedings of NSDI, 2015. Google ScholarDigital Library
- T. Mizrahi and Y. Moses. The case for Data Plane Timestamping in SDN. In Proceedings of IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), 2016.Google ScholarCross Ref
Index Terms
- Precise Time-synchronization in the Data-Plane using Programmable Switching ASICs
Recommendations
Offloading Virtual Evolved Packet Gateway User Plane Functions to a Programmable ASIC
ENCP '19: Proceedings of the 1st ACM CoNEXT Workshop on Emerging in-Network Computing ParadigmsRoll-outs of 5G Mobile Packet Core (MPC) rely on principles and technologies of Software-Defined Networking (SDN) and Network Function Virtualization (NFV). While the benefits of SDN and NFV in terms of flexibility are well-known, how to guarantee data ...
Delay Analysis in Programmable Data Plane Using Jitter Calculations
IC3-2023: Proceedings of the 2023 Fifteenth International Conference on Contemporary ComputingThe second generation of Software Defined Networks (SDN) made the network more flexible and automated with the introduction of programmable forwarding devices. This programmability has reduced the delay and processing time taken by a traditional router ...
Precise relative clock synchronization for distributed control using TSC registers
Precise clock synchronization is essential in emerging time-critical distributed control systems operating over computer networks where the clock synchronization requirements are mostly focused on relative clock synchronization and high synchronization ...
Comments