Jiaqi Gao
ABSTRACT
Programmable data plane has been moving towards deployments in data centers as mainstream vendors of switching ASICs enable programmability in their newly launched products, such as Broadcom's Trident-4, Intel/Barefoot's Tofino, and Cisco's Silicon One. However, current data plane programs are written in low-level, chip-specific languages (e.g., P4 and NPL) and thus tightly coupled to the chip-specific architecture. As a result, it is arduous and error-prone to develop, maintain, and composite data plane programs in production networks. This paper presents Lyra, the first cross-platform, high-level language & compiler system that aids the programmers in programming data planes efficiently. Lyra offers a one-big-pipeline abstraction that allows programmers to use simple statements to express their intent, without laboriously taking care of the details in hardware; Lyra also proposes a set of synthesis and optimization techniques to automatically compile this "big-pipeline" program into multiple pieces of runnable chip-specific code that can be launched directly on the individual programmable switches of the target network. We built and evaluated Lyra. Lyra not only generates runnable real-world programs (in both P4 and NPL), but also uses up to 87.5% fewer hardware resources and up to 78% fewer lines of code than human-written programs.
Supplemental Material
- 2019. Broadcom's new Trident 4 and Jericho 2 switch devices offer programmability at scale. https://www.broadcom.com/blog/trident4-and-jericho2-offer-programmability-at-scale.Google Scholar
- 2019. In-band Network Telemetry (INT) Dataplane Specification. https://github.com/p4lang/p4-applications/blob/master/docs/INT.pdf.Google Scholar
- 2019. NPL 1.3 Specification. https://github.com/nplang/NPL-Spec.Google Scholar
- 2019. ONE Silicon, ONE Experience, MULTIPLE Roles. https://blogs.cisco.com/sp/one-silicon-one-experience-multiple-roles.Google Scholar
- 2019. p4c, a reference compiler for P4 programming language. https://github.com/p4lang/p4c.Google Scholar
- 2020. Barefoot Tofino. https://www.barefootnetworks.com/products/brief-tofino.Google Scholar
- 2020. Barefoot Tofino's 32Q-model and 64Q-model. https://www.arista.com/en/products/7170-series.Google Scholar
- 2020. Netcope P4 - Flexible FPGA Programming. https://www.netcope.com/en/products/netcopep4.Google Scholar
- 2020. Netronome P4. https://www.netronome.com/technology/p4/.Google Scholar
- 2020. P4Runtime. https://p4.org/p4-runtime/.Google Scholar
- 2020. Pensando Expands What SmartNIC Offloads Can Do. https://pivotnine.com/2020/05/18/pensando-expands-what-smartnic-offloads-can-do/.Google Scholar
- Mohammad Alizadeh, Tom Edsall, Sarang Dharmapurikar, Ramanan Vaidyanathan, Kevin Chu, Andy Fingerhut, Vinh The Lam, Francis Matus, Rong Pan, Navindra Yadav, and George Varghese. 2014. CONGA: distributed congestion-aware load balancing for datacenters. In ACM SIGCOMM (SIGCOMM).Google Scholar
- Carolyn Jane Anderson, Nate Foster, Arjun Guha, Jean-Baptiste Jeannin, Dexter Kozen, Cole Schlesinger, and David Walker. 2014. NetKAT: Semantic foundations for networks. In 41st Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL).Google ScholarDigital Library
- Mina Tahmasbi Arashloo, Yaron Koral, Michael Greenberg, Jennifer Rexford, and David Walker. 2016. SNAP: Stateful network-wide abstractions for packet processing. In ACM SIGCOMM (SIGCOMM).Google ScholarDigital Library
- Pat Bosshart, Dan Daly, Glen Gibb, Martin Izzard, Nick McKeown, Jennifer Rexford, Cole Schlesinger, Dan Talayco, Amin Vahdat, George Varghese, and David Walker. 2014. P4: programming protocol-independent packet processors. Computer Communication Review 44, 3 (2014), 87--95.Google ScholarDigital Library
- Pat Bosshart, Glen Gibb, Hun-Seok Kim, George Varghese, Nick McKeown, Martin Izzard, Fernando A. Mujica, and Mark Horowitz. 2013. Forwarding metamorphosis: fast programmable match-action processing in hardware for SDN. In ACM SIGCOMM (SIGCOMM).Google Scholar
- Pat Bosshart, Glen Gibb, Hun-Seok Kim, George Varghese, Nick McKeown, Martin Izzard, Fernando Mujica, and Mark Horowitz. 2013. Forwarding metamorphosis: Fast programmable match-action processing in hardware for SDN. ACM SIGCOMM Computer Communication Review 43, 4 (2013).Google ScholarDigital Library
- Leonardo Mendonça de Moura and Nikolaj Bjørner. 2008. Z3: An efficient SMT solver. In 14th Tools and Algorithms for the Construction and Analysis of Systems (TACAS).Google Scholar
- Nate Foster, Rob Harrison, Michael J. Freedman, Christopher Monsanto, Jennifer Rexford, Alec Story, and David Walker. 2011. Frenetic: a network programming language. In 16th ACM SIGPLAN international conference on Functional Programming (ICFP).Google ScholarDigital Library
- Xiangyu Gao, Taegyun Kim, Aatish Kishan Varma, Anirudh Sivaraman, and Srinivas Narayana. 2019. Autogenerating fast packet-processing code using program synthesis. In 18th ACM Workshop on Hot Topics in Networks (HotNets).Google ScholarDigital Library
- Xiangyu Gao, Taegyun Kim, Michael D. Wong, Divya Raghunathan, Aatish Kishan Varma, Pravein Govindan Kannan, Anirudh Sivaraman, Srinivas Narayana, and Aarti Gupta. 2020. Switch code generation using program synthesis. In ACM SIGCOMM (SIGCOMM).Google Scholar
- Arpit Gupta, Rob Harrison, Marco Canini, Nick Feamster, Jennifer Rexford, and Walter Willinger. 2018. Sonata: query-driven streaming network telemetry. In ACM SIGCOMM (SIGCOMM).Google Scholar
- David Hancock and Jacobus E. van der Merwe. 2016. HyPer4: Using P4 to virtualize the programmable data plane. In 12th International Conference on emerging Networking EXperiments and Technologies (CoNEXT).Google Scholar
- Xin Jin, Xiaozhou Li, Haoyu Zhang, Nate Foster, Jeongkeun Lee, Robert Soulé, Changhoon Kim, and Ion Stoica. 2018. NetChain: Scale-free sub-RTT coordination. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI).Google Scholar
- Xin Jin, Xiaozhou Li, Haoyu Zhang, Robert Soulé, Jeongkeun Lee, Nate Foster, Changhoon Kim, and Ion Stoica. 2017. NetCache: Balancing key-value stores with fast in-network caching. In 26th Symposium on Operating Systems Principles (SOSP).Google ScholarDigital Library
- Lavanya Jose, Lisa Yan, George Varghese, and Nick McKeown. 2015. Compiling packet programs to reconfigurable switches. In 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI).Google Scholar
- Naga Praveen Katta, Mukesh Hira, Changhoon Kim, Anirudh Sivaraman, and Jennifer Rexford. 2016. HULA: Scalable load balancing using programmable data planes. In Symposium on SDN Research (SOSR).Google ScholarDigital Library
- Jialin Li, Ellis Michael, Naveen Kr. Sharma, Adriana Szekeres, and Dan R. K. Ports. 2016. Just say NO to Paxos overhead: Replacing consensus with network ordering. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI).Google Scholar
- Yuliang Li, Rui Miao, Hongqiang Harry Liu, Yan Zhuang, Fei Feng, Lingbo Tang, Zheng Cao, Ming Zhang, Frank Kelly, Mohammad Alizadeh, and Minlan Yu. 2019. HPCC: High precision congestion control. In ACM SIGCOMM (SIGCOMM).Google Scholar
- Hongqiang Harry Liu, Yibo Zhu, Jitu Padhye, Jiaxin Cao, Sri Tallapragada, Nuno P. Lopes, Andrey Rybalchenko, Guohan Lu, and Lihua Yuan. 2017. CrystalNet: Faithfully emulating large production networks. In 26th Symposium on Operating Systems Principles (SOSP).Google ScholarDigital Library
- Jedidiah McClurg, Hossein Hojjat, Nate Foster, and Pavol Cerný. 2016. Event-driven network programming. In 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI).Google ScholarDigital Library
- Rui Miao, Hongyi Zeng, Changhoon Kim, Jeongkeun Lee, and Minlan Yu. 2017. SilkRoad: Making stateful layer-4 load balancing fast and cheap using switching ASICs. In ACM SIGCOMM (SIGCOMM).Google Scholar
- Christopher Monsanto, Nate Foster, Rob Harrison, and David Walker. 2012. A compiler and run-time system for network programming languages. In 39th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL).Google ScholarDigital Library
- Christopher Monsanto, Joshua Reich, Nate Foster, Jennifer Rexford, and David Walker. 2013. Composing software defined networks. In 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI).Google Scholar
- Srinivas Narayana, Anirudh Sivaraman, Vikram Nathan, Prateesh Goyal, Venkat Arun, Mohammad Alizadeh, Vimalkumar Jeyakumar, and Changhoon Kim. 2017. Language-directed hardware design for network performance monitoring. In ACM SIGCOMM (SIGCOMM).Google Scholar
- Arjun Singh, Joon Ong, Amit Agarwal, Glen Anderson, Ashby Armistead, Roy Bannon, Seb Boving, Gaurav Desai, Bob Felderman, Paulie Germano, Anand Kanagala, Jeff Provost, Jason Simmons, Eiichi Tanda, Jim Wanderer, Urs Hölzle, Stephen Stuart, and Amin Vahdat. 2015. Jupiter rising: A decade of Clos topologies and centralized control in Google's datacenter network. In ACM SIGCOMM (SIGCOMM).Google Scholar
- Anirudh Sivaraman, Alvin Cheung, Mihai Budiu, Changhoon Kim, Mohammad Alizadeh, Hari Balakrishnan, George Varghese, Nick McKeown, and Steve Licking. 2016. Packet transactions: High-level programming for line-rate switches. In ACM SIGCOMM (SIGCOMM).Google Scholar
- Anirudh Sivaraman, Thomas Mason, Aurojit Panda, Ravi Netravali, and Sai Anirudh Kondaveeti. 2020. Network architecture in the age of programmability. Computer Communication Review 50, 1 (2020).Google ScholarDigital Library
- Hardik Soni, Myriana Rifai, Praveen Kumar, Ryan Doenges, and Nate Foster. 2020. Composing dataplane programs with μP4. In ACM SIGCOMM (SIGCOMM).Google Scholar
- Dingming Wu, Ang Chen, T. S. Eugene Ng, Guohui Wang, and Haiyong Wang. 2019. Accelerated service chaining on a singles witch ASIC. In 18th ACM Workshop on Hot Topics in Networks (HotNets).Google ScholarDigital Library
- Yang Richard Yang, Kai Gao, Kerim Gokarslan, Dong Guo, and Christopher Leet. 2019. Magellan: Toward high-level programming and analysis of SDN using flow algebra. In ACM SIGCOMM Workshopon Networking and Programming Languages (NetPL).Google Scholar
- Cheng Zhang, Jun Bi, Yu Zhou, Adbul Basit Dogar, and Jianping Wu. 2017. HyperV: A high performance hypervisor for virtualization of the programmable data plane. In 26th International Conference on Computer Communication and Networks (ICCCN).Google ScholarCross Ref
- Cheng Zhang, Jun Bi, Yu Zhou, and Jianping Wu. 2019. HyperVDP: High-Performance virtualization of the programmable data plane. IEEE J. Sel. Areas Commun. 37, 3 (2019), 556--569.Google ScholarCross Ref
- Peng Zheng, Theophilus Benson, and Chengchen Hu. 2018. P4Visor: Lightweight virtualization and composition primitives for building and testing modular programs. In 14th International Conference on emerging Networking EXperiments and Technologies (CoNEXT).Google ScholarDigital Library
Index Terms
- Lyra: A Cross-Platform Language and Compiler for Data Plane Programming on Heterogeneous ASICs
Recommendations
P4-Compatible High-Level Synthesis of Low Latency 100 Gb/s Streaming Packet Parsers in FPGAs
FPGA '18: Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate ArraysPacket parsing is a key step in SDN-aware devices. Packet parsers in SDN networks need to be both reconfigurable and fast, to support the evolving network protocols and the increasing multi-gigabit data rates. The combination of packet processing ...
Part-compilation in high-level languages
AbstractMany programming languages include the ability to divide large programs into smaller segments, which are compiled separately. When a small modification is made to a large program, then the affected segment only has to be re-compiled.
This paper ...
Babel and SOAP applications of extensible compilers
This paper describes a method of writing compilers which can easily be extended or altered. Two applications of the method are described; the first is a conventional programming language called Babel and the second is a program called SOAP which ...
Comments