Cheng-Chun Tu
ABSTRACT
As the performance overhead associated with CPU and memory virtualization becomes largely negligible, research efforts are directed toward reducing the I/O virtualization overhead, which mainly comes from two sources: DMA set-up and payload copy, and interrupt delivery. The advent of SRIOV and MRIOV effectively reduces the DMA-related virtualization overhead to a minimum. Therefore, the last battleground for minimizing virtualization overhead is how to directly deliver every interrupt to its target VM without involving the hypervisor.
This paper describes the design, implementation, and evaluation of a KVM-based direct interrupt delivery system called DID. DID delivers interrupts from SRIOV devices, virtual devices, and timers to their target VMs directly, completely avoiding VM exits. Moreover, DID does not require any modifications to the VM's operating system and preserves the correct priority among interrupts in all cases. We demonstrate that DID reduces the number of VM exits by a factor of 100 for I/O-intensive workloads, decreases the interrupt invocation latency by 80%, and improves the throughput of a VM running Memcached by a factor of 3.
- Enabling Optimized Interrupt/APIC Virtualization in KVM. KVM Forum 2012.Google Scholar
- Fio - Flexible I/O Tester. http://freecode.com/projects/fio.Google Scholar
- Introduction of AMD Advanced Virtual Interrupt Controller. XenSummit 2012.Google Scholar
- Jailhouse Partitioning Hypervisor. https://github.com/siemens/jailhouse.Google Scholar
- Memcached: memory object caching system. http://memcached.org/.Google Scholar
- Secure virtual machine architecture reference manual. AMD.Google Scholar
- Single-Root I/O Virtualization and Sharing Specification, Revision 1.0, PCI-SIG.Google Scholar
- SIPp: traffic generator for the SIP protocol. http://sipp.sourceforge.net/.Google Scholar
- Sippy B2BUA. http://www.b2bua.org/.Google Scholar
- virtio- and vhost-net need for speed performance challenges. KVM Forum 2010.Google Scholar
- Intel 64 Architecture x2APIC Specification, Intel Corporation, 2008.Google Scholar
- Single-Root I/O Virtualization and Sharing Specification, Revision 1.0, PCI-SIG, 2008.Google Scholar
- Keith Adams and Ole Agesen. A comparison of software and hardware techniques for x86 virtualization. In ACM ASPLOS'06. Google ScholarDigital Library
- Ole Agesen, Jim Mattson, Radu Rugina, and Jeffrey Sheldon. Software techniques for avoiding hardware virtualization exits. In USENIX Annual Technical Conference, pages 373--385, 2012. Google ScholarDigital Library
- Muli Ben-Yehuda, Michael D Day, Zvi Dubitzky, Michael Factor, Nadav Har'El, Abel Gordon, Anthony Liguori, Orit Wasserman, and Ben-Ami Yassour. The turtles project: Design and implementation of nested virtualization. In OSDI, volume 10, pages 423--436, 2010. Google ScholarDigital Library
- Christoffer Dall and Jason Nieh. Kvm/arm: Experiences building the linux arm hypervisor. 2013.Google Scholar
- Michael Ferdman, Almutaz Adileh, Onur Kocberber, Stavros Volos, Mohammad Alisafaee, Djordje Jevdjic, Cansu Kaynak, Adrian Daniel Popescu, Anastasia Ailamaki, and Babak Falsafi. Clearing the Clouds: A Study of Emerging Scale-out Workloads on Modern Hardware. volume 40, pages 37--48. ACM, 2012. Google ScholarDigital Library
- Abel Gordon, Nadav Amit, Nadav Har'El, Muli Ben-Yehuda, Alex Landau, Assaf Schuster, and Dan Tsafrir. Eli: bare-metal performance for i/o virtualization. ACM SIGARCH Computer Architecture News, 40(1):411--422, 2012. Google ScholarDigital Library
- Nadav Har'El, Abel Gordon, Alex Landau, Muli Ben-Yehuda, Avishay Traeger, and Razya Ladelsky. Efficient and scalable paravirtual i/o system. In USENIX Annual Technical Conference, pages 231--242, 2013. Google ScholarDigital Library
- R. Hiremane. Intel Virtualization Technology for Directed I/O (Intel VT-d). Technology@ Intel Magazine, 2007.Google Scholar
- DPDK Intel. Intel data plane development kit.Google Scholar
- Eric Keller, Jakub Szefer, Jennifer Rexford, and Ruby B Lee. Nohype: virtualized cloud infrastructure without the virtualization. In ACM SIGARCH Computer Architecture News, volume 38, pages 350--361. ACM, 2010. Google ScholarDigital Library
- Jan Kiszka. Towards linux as a real-time hypervisor. RTLWS11, 2009.Google Scholar
- Avi Kivity, Yaniv Kamay, Dor Laor, Uri Lublin, and Anthony Liguori. kvm: the linux virtual machine monitor. In Proceedings of the Linux Symposium, volume 1, pages 225--230, 2007.Google Scholar
- Kaushik Kumar Ram, Jose Renato Santos, Yoshio Turner, Alan L Cox, and Scott Rixner. Achieving 10 gb/s using safe and transparent network interface virtualization. In Proceedings of the 2009 ACM SIGPLAN/SIGOPS international conference on Virtual execution environments, pages 61--70. ACM, 2009. Google ScholarDigital Library
- Rusty Russell. virtio: towards a de-facto standard for virtual i/o devices. SIGOPS Oper. Syst. Rev. 2008. Google ScholarDigital Library
- Jose Renato Santos, Yoshio Turner, G John Janakiraman, and Ian Pratt. Bridging the gap between software and hardware techniques for i/o virtualization. In USENIX Annual Technical Conference, pages 29--42, 2008. Google ScholarDigital Library
- Quinn O Snell, Armin R Mikler, and John L Gustafson. Netpipe: A network protocol independent performance evaluator. In IASTED International Conference on Intelligent Information Management and Systems, volume 6. Washington, DC, USA), 1996.Google Scholar
- Jakub Szefer, Eric Keller, Ruby B Lee, and Jennifer Rexford. Eliminating the hypervisor attack surface for a more secure cloud. In Proceedings of the 18th ACM conference on Computer and communications security, pages 401--412. ACM, 2011. Google ScholarDigital Library
- Ajay Tirumala, Feng Qin, Jon Dugan, Jim Ferguson, and Kevin Gibbs. Iperf: The tcp/udp bandwidth measurement tool. http://dast.nlanr.net/Projects, 2005.Google Scholar
- Hitachi Tomoki Sekiyama, Yokohama Research Lab. Improvement of real-time performance of kvm.Google Scholar
- Cheng-Chun Tu, Chao-tang Lee, and Tzi-cker Chiueh. Secure i/o device sharing among virtual machines on multiple hosts. In Proceedings of the 40th Annual International Symposium on Computer Architecture, pages 108--119. ACM, 2013. Google ScholarDigital Library
- Rich Uhlig, Gil Neiger, Dion Rodgers, Amy L Santoni, Fernando CM Martins, Andrew V Anderson, Steven M Bennett, Alain Kagi, Felix H Leung, and Larry Smith. Intel virtualization technology. Computer, 38(5):48--56, 2005. Google ScholarDigital Library
- Rafal Wojtczuk and Joanna Rutkowska. Following the white rabbit: Software attacks against intel vt-d technology.Google Scholar
Index Terms
- A Comprehensive Implementation and Evaluation of Direct Interrupt Delivery
Recommendations
A Comprehensive Implementation and Evaluation of Direct Interrupt Delivery
VEE '15As the performance overhead associated with CPU and memory virtualization becomes largely negligible, research efforts are directed toward reducing the I/O virtualization overhead, which mainly comes from two sources: DMA set-up and payload copy, and ...
ELI: bare-metal performance for I/O virtualization
ASPLOS XVII: Proceedings of the seventeenth international conference on Architectural Support for Programming Languages and Operating SystemsDirect device assignment enhances the performance of guest virtual machines by allowing them to communicate with I/O devices without host involvement. But even with device assignment, guests are still unable to approach bare-metal performance, because ...
ELI: bare-metal performance for I/O virtualization
ASPLOS '12Direct device assignment enhances the performance of guest virtual machines by allowing them to communicate with I/O devices without host involvement. But even with device assignment, guests are still unable to approach bare-metal performance, because ...
Comments