Application-Motivated, Holistic Benchmarking of a Full Quantum Computing Stack

Daniel Mills; Seyon Sivarajah; Travis L. Scholten; Ross Duncan

doi:10.22331/q-2021-03-22-415

Application-Motivated, Holistic Benchmarking of a Full Quantum Computing Stack

Daniel Mills^1,2, Seyon Sivarajah², Travis L. Scholten³, and Ross Duncan^2,4

¹University of Edinburgh, 10 Crichton Street, Edinburgh EH8 9AB, UK
²Cambridge Quantum Computing Ltd, 9a Bridge Street, Cambridge, CB2 1UB, UK
³IBM T.J. Watson Research Center, Yorktown Heights, NY 10598, USA
⁴University of Strathclyde, 26 Richmond Street, Glasgow, G1 1XH, UK

Published:	2021-03-22, volume 5, page 415
Eprint:	arXiv:2006.01273v3
Doi:	https://doi.org/10.22331/q-2021-03-22-415
Citation:	Quantum 5, 415 (2021).

Find this paper interesting or want to discuss? Scite or leave a comment on SciRate.

Abstract

Quantum computing systems need to be benchmarked in terms of practical tasks they would be expected to do. Here, we propose 3 "application-motivated" circuit classes for benchmarking: deep (relevant for state preparation in the variational quantum eigensolver algorithm), shallow (inspired by IQP-type circuits that might be useful for near-term quantum machine learning), and square (inspired by the quantum volume benchmark). We quantify the performance of a quantum computing system in running circuits from these classes using several figures of merit, all of which require exponential classical computing resources and a polynomial number of classical samples (bitstrings) from the system. We study how performance varies with the compilation strategy used and the device on which the circuit is run. Using systems made available by IBM Quantum, we examine their performance, showing that noise-aware compilation strategies may be beneficial, and that device connectivity and noise levels play a crucial role in the performance of the system according to our benchmarks.

Featured image: An example comparison of compilation strategies (fixed colours) when used on application-motivated circuits run on a real device.

Popular summary

Benchmarking of quantum computing devices is necessary to measure their performance, and to guide their use and future development. As quantum computing systems become more complex, they need to be benchmarked in terms of practical applications they would be expected to do. This paper sets out and demonstrates an application-motivated benchmarking framework for full-stack quantum computational systems. The framework is used to benchmark the performance of several quantum devices made available by IBM Quantum, which are combined with different compiler strategies (enabled by CQC’s tket and IBM’s Qiskit) to produce the full-stack. By considering three different classes of circuits, motivated by a variety of applications, the framework assesses the strengths and weaknesses of quantum computational systems when performing relevant tasks. This work also considers the effect of different compilation strategies, which are used to transform and optimise a circuit. Doing so can inform compiler development for a given application or device.

► BibTeX data

@article{Mills2021application,
  doi = {10.22331/q-2021-03-22-415},
  url = {https://doi.org/10.22331/q-2021-03-22-415},
  title = {Application-{M}otivated, {H}olistic {B}enchmarking of a {F}ull {Q}uantum {C}omputing {S}tack},
  author = {Mills, Daniel and Sivarajah, Seyon and Scholten, Travis L. and Duncan, Ross},
  journal = {{Quantum}},
  issn = {2521-327X},
  publisher = {{Verein zur F{\"{o}}rderung des Open Access Publizierens in den Quantenwissenschaften}},
  volume = {5},
  pages = {415},
  month = mar,
  year = {2021}
}

► References

[1] Gadi Aleksandrowicz, Thomas Alexander, Panagiotis Barkoutsos, Luciano Bello, Yael Ben-Haim, David Bucher, Francisco Jose Cabrera-Hernández, Jorge Carballo-Franquis, Adrian Chen, Chun-Fu Chen, Jerry M. Chow, Antonio D. Córcoles-Gonzales, Abigail J. Cross, Andrew Cross, Juan Cruz-Benito, Chris Culver, Salvador De La Puente González, Enrique De La Torre, Delton Ding, Eugene Dumitrescu, Ivan Duran, Pieter Eendebak, Mark Everitt, Ismael Faro Sertage, Albert Frisch, Andreas Fuhrer, Jay Gambetta, Borja Godoy Gago, Juan Gomez-Mosquera, Donny Greenberg, Ikko Hamamura, Vojtech Havlicek, Joe Hellmers, Åukasz Herok, Hiroshi Horii, Shaohan Hu, Takashi Imamichi, Toshinari Itoko, Ali Javadi-Abhari, Naoki Kanazawa, Anton Karazeev, Kevin Krsulich, Peng Liu, Yang Luh, Yunho Maeng, Manoel Marques, Francisco Jose Martín-Ferández, Douglas T. McClure, David McKay, Srujan Meesala, Antonio Mezzacapo, Nikolaj Moll, Diego Moreda Rodríguez, Giacomo Nannicini, Paul Nation, Pauline Ollitrault, Lee James O'Riordan, Hanhee Paik, Jesús Pérez, Anna Phan, Marco Pistoia, Viktor Prutyanov, Max Reuter, Julia Rice, Abdón Rodríguez Davila, Raymond Harry Putra Rudy, Mingi Ryu, Ninad Sathaye, Chris Schnabel, Eddie Schoute, Kanav Setia, Yunong Shi, Adenilton Silva, Yukio Siraichi, Seyon Sivarajah, John A. Smolin, Mathias Soeken, Hitomi Takahashi, Ivano Tavernelli, Charles Taylor, Pete Taylour, Kenso Trabing, Matthew Treinish, Wes Turner, Desiree Vogt-Lee, Christophe Vuillot, Jonathan A. Wildstrom, Jessica Wilson, Erick Winston, Christopher Wood, Stephen Wood, Stefan Wörner, Ismail Yunus Akhalwaya, and Christa Zoufal, ``Qiskit: An Open-source Framework for Quantum Computing'' (2019).
https://doi.org/10.5281/zenodo.2562111

[2] Peter J Karalekas, Nikolas A Tezak, Eric C Peterson, Colm A Ryan, Marcus P da Silva, and Robert S Smith, ``A quantum-classical cloud platform optimized for variational hybrid algorithms'' Quantum Science and Technology 5, 024003 (2020).
https://doi.org/10.1088/2058-9565/ab7559

[3] Frank Arute, Kunal Arya, Ryan Babbush, Dave Bacon, Joseph C. Bardin, Rami Barends, Rupak Biswas, Sergio Boixo, Fernando G. S. L. Brandao, David A. Buell, Brian Burkett, Yu Chen, Zijun Chen, Ben Chiaro, Roberto Collins, William Courtney, Andrew Dunsworth, Edward Farhi, Brooks Foxen, Austin Fowler, Craig Gidney, Marissa Giustina, Rob Graff, Keith Guerin, Steve Habegger, Matthew P. Harrigan, Michael J. Hartmann, Alan Ho, Markus Hoffmann, Trent Huang, Travis S. Humble, Sergei V. Isakov, Evan Jeffrey, Zhang Jiang, Dvir Kafri, Kostyantyn Kechedzhi, Julian Kelly, Paul V. Klimov, Sergey Knysh, Alexander Korotkov, Fedor Kostritsa, David Landhuis, Mike Lindmark, Erik Lucero, Dmitry Lyakh, Salvatore Mandrà, Jarrod R. McClean, Matthew McEwen, Anthony Megrant, Xiao Mi, Kristel Michielsen, Masoud Mohseni, Josh Mutus, Ofer Naaman, Matthew Neeley, Charles Neill, Murphy Yuezhen Niu, Eric Ostby, Andre Petukhov, John C. Platt, Chris Quintana, Eleanor G. Rieffel, Pedram Roushan, Nicholas C. Rubin, Daniel Sank, Kevin J. Satzinger, Vadim Smelyanskiy, Kevin J. Sung, Matthew D. Trevithick, Amit Vainsencher, Benjamin Villalonga, Theodore White, Z. Jamie Yao, Ping Yeh, Adam Zalcman, Hartmut Neven, and John M. Martinis, ``Quantum supremacy using a programmable superconducting processor'' Nature 574, 505–510 (2019).
https://doi.org/10.1038/s41586-019-1666-5

[4] Simon J. Devitt ``Performing quantum computing experiments in the cloud'' Phys. Rev. A 94, 032329 (2016).
https://doi.org/10.1103/PhysRevA.94.032329

[5] S. Debnath, N. M. Linke, C. Figgatt, K. A. Landsman, K. Wright, and C. Monroe, ``Demonstration of a small programmable quantum computer with atomic qubits'' Nature 536, 63–66 (2016).
https://doi.org/10.1038/nature18648

[6] Prakash Murali, Norbert Matthias Linke, Margaret Martonosi, Ali Javadi Abhari, Nhung Hong Nguyen, and Cinthia Huerta Alderete, ``Full-Stack, Real-System Quantum Computer Studies: Architectural Comparisons and Design Insights'' Proceedings of the 46th International Symposium on Computer Architecture 527â540 (2019).
https://doi.org/10.1145/3307650.3322273

[7] A Petitet, R C Whaley, J Dongarra, and A Cleary, ``HPL - A Portable Implementation of the High-Performance Linpack Benchmark for Distributed-Memory Computers'' http://www.netlib.org/benchmark/hpl/.
http://www.netlib.org/benchmark/hpl/

[8] Jack J. Dongarra, Piotr Luszczek, and Antoine Petitet, ``The LINPACK Benchmark: past, present and future'' Concurrency and Computation: Practice and Experience 15, 803–820 (2003).
https://doi.org/10.1002/cpe.728

[9] Jack Dongarraand Piotr Luszczek ``TOP500'' Springer US (2011).
https://doi.org/10.1007/978-0-387-09766-4_157

[10] Norbert M. Linke, Dmitri Maslov, Martin Roetteler, Shantanu Debnath, Caroline Figgatt, Kevin A. Landsman, Kenneth Wright, and Christopher Monroe, ``Experimental comparison of two quantum computing architectures'' Proceedings of the National Academy of Sciences 114, 3305–3310 (2017).
https://doi.org/10.1073/pnas.1618020114
https://www.pnas.org/content/114/13/3305

[11] Robin Blume-Kohoutand Kevin C. Young ``A volumetric framework for quantum computer benchmarks'' (2020).
https://doi.org/10.22331/q-2020-11-15-362

[12] Sam McArdle, Suguru Endo, Alán Aspuru-Guzik, Simon C. Benjamin, and Xiao Yuan, ``Quantum computational chemistry'' Rev. Mod. Phys. 92, 015003 (2020).
https://doi.org/10.1103/RevModPhys.92.015003

[13] Alexander J. McCaskey, Zachary P. Parks, Jacek Jakowski, Shirley V. Moore, Titus D. Morris, Travis S. Humble, and Raphael C. Pooser, ``Quantum chemistry as a benchmark for near-term quantum computers'' npj Quantum Information 5, 99 (2019).
https://doi.org/10.1038/s41534-019-0209-0

[14] Pierre-Luc Dallaire-Demers, Michał Stęchły, Jerome F. Gonthier, Ntwali Toussaint Bashige, Jonathan Romero, and Yudong Cao, ``An application benchmark for fermionic quantum simulations'' (2020).
arXiv:2003.01862

[15] E. F. Dumitrescu, A. J. McCaskey, G. Hagen, G. R. Jansen, T. D. Morris, T. Papenbrock, R. C. Pooser, D. J. Dean, and P. Lougovski, ``Cloud Quantum Computing of an Atomic Nucleus'' Phys. Rev. Lett. 120, 210501 (2018).
https://doi.org/10.1103/PhysRevLett.120.210501

[16] Frank Arute, Kunal Arya, Ryan Babbush, Dave Bacon, Joseph C. Bardin, Rami Barends, Sergio Boixo, Michael Broughton, Bob B. Buckley, David A. Buell, Brian Burkett, Nicholas Bushnell, Yu Chen, Zijun Chen, Benjamin Chiaro, Roberto Collins, William Courtney, Sean Demura, Andrew Dunsworth, Edward Farhi, Austin Fowler, Brooks Foxen, Craig Gidney, Marissa Giustina, Rob Graff, Steve Habegger, Matthew P. Harrigan, Alan Ho, Sabrina Hong, Trent Huang, William J. Huggins, Lev Ioffe, Sergei V. Isakov, Evan Jeffrey, Zhang Jiang, Cody Jones, Dvir Kafri, Kostyantyn Kechedzhi, Julian Kelly, Seon Kim, Paul V. Klimov, Alexander Korotkov, Fedor Kostritsa, David Landhuis, Pavel Laptev, Mike Lindmark, Erik Lucero, Orion Martin, John M. Martinis, Jarrod R. McClean, Matt McEwen, Anthony Megrant, Xiao Mi, Masoud Mohseni, Wojciech Mruczkiewicz, Josh Mutus, Ofer Naaman, Matthew Neeley, Charles Neill, Hartmut Neven, Murphy Yuezhen Niu, Thomas E. O’Brien, Eric Ostby, Andre Petukhov, Harald Putterman, Chris Quintana, Pedram Roushan, Nicholas C. Rubin, Daniel Sank, Kevin J. Satzinger, Vadim Smelyanskiy, Doug Strain, Kevin J. Sung, Marco Szalay, Tyler Y. Takeshita, Amit Vainsencher, Theodore White, Nathan Wiebe, Z. Jamie Yao, Ping Yeh, and Adam Zalcman, ``Hartree-Fock on a superconducting qubit quantum computer'' (2020).
https://doi.org/10.1126/science.abb9811
https://science.sciencemag.org/content/369/6507/1084

[17] Vedran Dunjkoand Hans J Briegel ``Machine learning & artificial intelligence in the quantum domain: a review of recent progress'' Reports on Progress in Physics 81, 074001 (2018).
https://doi.org/10.1088/1361-6633/aab406

[18] Marcello Benedetti, Delfina Garcia-Pintos, Oscar Perdomo, Vicente Leyton-Ortega, Yunseong Nam, and Alejandro Perdomo-Ortiz, ``A generative modeling approach for benchmarking and training shallow quantum circuits'' npj Quantum Information 5, 45 (2019).
https://doi.org/10.1038/s41534-019-0157-8

[19] Kathleen E. Hamiltonand Raphael C. Pooser ``Error-mitigated data-driven circuit learning on noisy quantum hardware'' Quantum Machine Intelligence 2, 10 (2020).
https://doi.org/10.1007/s42484-020-00021-x

[20] Kathleen E. Hamilton, Eugene F. Dumitrescu, and Raphael C. Pooser, ``Generative model benchmarks for superconducting qubits'' Phys. Rev. A 99, 062323 (2019).
https://doi.org/10.1103/PhysRevA.99.062323

[21] Matthew P. Harrigan, Kevin J. Sung, Matthew Neeley, Kevin J. Satzinger, Frank Arute, Kunal Arya, Juan Atalaya, Joseph C. Bardin, Rami Barends, Sergio Boixo, Michael Broughton, Bob B. Buckley, David A. Buell, Brian Burkett, Nicholas Bushnell, Yu Chen, Zijun Chen, Ben Chiaro, Roberto Collins, William Courtney, Sean Demura, Andrew Dunsworth, Daniel Eppens, Austin Fowler, Brooks Foxen, Craig Gidney, Marissa Giustina, Rob Graff, Steve Habegger, Alan Ho, Sabrina Hong, Trent Huang, L. B. Ioffe, Sergei V. Isakov, Evan Jeffrey, Zhang Jiang, Cody Jones, Dvir Kafri, Kostyantyn Kechedzhi, Julian Kelly, Seon Kim, Paul V. Klimov, Alexander N. Korotkov, Fedor Kostritsa, David Landhuis, Pavel Laptev, Mike Lindmark, Martin Leib, Orion Martin, John M. Martinis, Jarrod R. McClean, Matt McEwen, Anthony Megrant, Xiao Mi, Masoud Mohseni, Wojciech Mruczkiewicz, Josh Mutus, Ofer Naaman, Charles Neill, Florian Neukart, Murphy Yuezhen Niu, Thomas E. O'Brien, Bryan O'Gorman, Eric Ostby, Andre Petukhov, Harald Putterman, Chris Quintana, Pedram Roushan, Nicholas C. Rubin, Daniel Sank, Andrea Skolik, Vadim Smelyanskiy, Doug Strain, Michael Streif, Marco Szalay, Amit Vainsencher, Theodore White, Z. Jamie Yao, Ping Yeh, Adam Zalcman, Leo Zhou, Hartmut Neven, Dave Bacon, Erik Lucero, Edward Farhi, and Ryan Babbush, ``Quantum approximate optimization of non-planar graph problems on a planar superconducting processor'' (2021).
https://doi.org/10.1038/s41567-020-01105-y

[22] Madita Willsch, Dennis Willsch, Fengping Jin, Hans De Raedt, and Kristel Michielsen, ``Benchmarking the quantum approximate optimization algorithm'' (2020).
https://doi.org/10.1007/s11128-020-02692-8

[23] Andreas Bengtsson, Pontus Vikstål, Christopher Warren, Marika Svensson, Xiu Gu, Anton Frisk Kockum, Philip Krantz, Christian Kri\ifmmode \checkz\else Križan, Daryoush Shiri, Ida-Maria Svensson, Giovanna Tancredi, Göran Johansson, Per Delsing, Giulia Ferrini, and Jonas Bylander, ``Improved Success Probability with Greater Circuit Depth for the Quantum Approximate Optimization Algorithm'' (2020).
https://doi.org/10.1103/PhysRevApplied.14.034010

[24] Guido Pagano, Aniruddha Bapat, Patrick Becker, Katherine S. Collins, Arinjoy De, Paul W. Hess, Harvey B. Kaplan, Antonis Kyprianidis, Wen Lin Tan, Christopher Baldwin, Lucas T. Brady, Abhinav Deshpande, Fangli Liu, Stephen Jordan, Alexey V. Gorshkov, and Christopher Monroe, ``Quantum approximate optimization of the long-range Ising model with a trapped-ion quantum simulator'' (2020).
https://doi.org/10.1073/pnas.2006373117
https://www.pnas.org/content/117/41/25396

[25] John Preskill ``Quantum computing and the entanglement frontier'' (2012).
arXiv:1203.5813

[26] Aram W. Harrowand Ashley Montanaro ``Quantum computational supremacy'' Nature 549, 203â209 (2017).
https://doi.org/10.1038/nature23458

[27] Sergio Boixo, Sergei V. Isakov, Vadim N. Smelyanskiy, Ryan Babbush, Nan Ding, Zhang Jiang, Michael J. Bremner, John M. Martinis, and Hartmut Neven, ``Characterizing quantum supremacy in near-term devices'' Nature Physics 14, 595–600 (2018).
https://doi.org/10.1038/s41567-018-0124-x

[28] Sukin Sim, Peter D. Johnson, and Alán Aspuru-Guzik, ``Expressibility and Entangling Capability of Parameterized Quantum Circuits for Hybrid Quantum-Classical Algorithms'' Advanced Quantum Technologies 2, 1900070 (2019).
https://doi.org/10.1002/qute.201900070

[29] Abhinav Kandala, Antonio Mezzacapo, Kristan Temme, Maika Takita, Markus Brink, Jerry M. Chow, and Jay M. Gambetta, ``Hardware-efficient variational quantum eigensolver for small molecules and quantum magnets'' Nature 549, 242–246 (2017).
https://doi.org/10.1038/nature23879

[30] Brian Coyle, Daniel Mills, Vincent Danos, and Elham Kashefi, ``The Born supremacy: quantum advantage and training of an Ising Born machine'' npj Quantum Information 6, 60 (2020).
https://doi.org/10.1038/s41534-020-00288-9

[31] Yuxuan Du, Min-Hsiu Hsieh, Tongliang Liu, and Dacheng Tao, ``Expressive power of parametrized quantum circuits'' (2020).
https://doi.org/10.1103/PhysRevResearch.2.033125

[32] Thomas Hubregtsen, Josef Pichlmeier, and Koen Bertels, ``Evaluation of Parameterized Quantum Circuits: on the design, and the relation between classification accuracy, expressibility and entangling capability'' (2020).
arXiv:2003.09887

[33] Andrew W. Cross, Lev S. Bishop, Sarah Sheldon, Paul D. Nation, and Jay M. Gambetta, ``Validating quantum computers using randomized model circuits'' Phys. Rev. A 100, 032328 (2019).
https://doi.org/10.1103/PhysRevA.100.032328

[34] Dominic W. Berry, Graeme Ahokas, Richard Cleve, and Barry C. Sanders, ``Efficient Quantum Algorithms for Simulating Sparse Hamiltonians'' Communications in Mathematical Physics 270, 359–371 (2007).
https://doi.org/10.1007/s00220-006-0150-x

[35] Alberto Peruzzo, Jarrod McClean, Peter Shadbolt, Man-Hong Yung, Xiao-Qi Zhou, Peter J. Love, Alán Aspuru-Guzik, and Jeremy L. O'Brien, ``A variational eigenvalue solver on a photonic quantum processor'' Nature Communications 5, 4213 (2014).
https://doi.org/10.1038/ncomms5213

[36] Jonathan Romero, Ryan Babbush, Jarrod R McClean, Cornelius Hempel, Peter J Love, and Alán Aspuru-Guzik, ``Strategies for quantum computing molecular energies using the unitary coupled cluster ansatz'' Quantum Science and Technology 4, 014008 (2018).
https://doi.org/10.1088/2058-9565/aad3e4

[37] Alexandru Gheorghiu, Theodoros Kapourniotis, and Elham Kashefi, ``Verification of Quantum Computation: An Overview of Existing Approaches'' Theor. Comp. Sys. 63, 715–808 (2019).
https://doi.org/10.1007/s00224-018-9872-3

[38] U. Mahadev ``Classical Verification of Quantum Computations'' 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS) 259–267 (2018).
https://doi.org/10.1109/FOCS.2018.00033

[39] Scott Aaronsonand Lijie Chen ``Complexity-Theoretic Foundations of Quantum Supremacy Experiments'' (2016).
arXiv:1612.05903

[40] Nathan Wiebe, Christopher Granade, and D G Cory, ``Quantum bootstrapping via compressed quantum Hamiltonian learning'' New Journal of Physics 17, 022005 (2015).
https://doi.org/10.1088/1367-2630/17/2/022005

[41] C. E. Porterand R. G. Thomas ``Fluctuations of Nuclear Reaction Widths'' Phys. Rev. 104, 483–491 (1956).
https://doi.org/10.1103/PhysRev.104.483

[42] C. Neill, P. Roushan, K. Kechedzhi, S. Boixo, S. V. Isakov, V. Smelyanskiy, A. Megrant, B. Chiaro, A. Dunsworth, K. Arya, R. Barends, B. Burkett, Y. Chen, Z. Chen, A. Fowler, B. Foxen, M. Giustina, R. Graff, E. Jeffrey, T. Huang, J. Kelly, P. Klimov, E. Lucero, J. Mutus, M. Neeley, C. Quintana, D. Sank, A. Vainsencher, J. Wenner, T. C. White, H. Neven, and J. M. Martinis, ``A blueprint for demonstrating quantum supremacy with superconducting qubits'' Science 360, 195–199 (2018).
https://doi.org/10.1126/science.aao4309
https://science.sciencemag.org/content/360/6385/195

[43] Sergio Boixo, Vadim N. Smelyanskiy, and Hartmut Neven, ``Fourier analysis of sampling from noisy chaotic quantum circuits'' (2017).
arXiv:1708.01875

[44] Adam Bouland, Bill Fefferman, Chinmay Nirkhe, and Umesh Vazirani, ``On the complexity and verification of quantum random circuit sampling'' Nature Physics 15, 159–163 (2019).
https://doi.org/10.1038/s41567-018-0318-2

[45] Ramis Movassagh ``Efficient unitary paths and quantum computational supremacy: A proof of average-case hardness of Random Circuit Sampling'' (2018).
arXiv:1810.04681

[46] Scott Aaronsonand Alex Arkhipov ``The Computational Complexity of Linear Optics'' Proceedings of the Forty-Third Annual ACM Symposium on Theory of Computing 333â342 (2011).
https://doi.org/10.1145/1993636.1993682

[47] Michael J. Bremner, Ashley Montanaro, and Dan J. Shepherd, ``Average-Case Complexity Versus Approximate Simulation of Commuting Quantum Computations'' Phys. Rev. Lett. 117, 080501 (2016).
https://doi.org/10.1103/PhysRevLett.117.080501

[48] Michael J. Bremner, Richard Jozsa, and Dan J. Shepherd, ``Classical simulation of commuting quantum computations implies collapse of the polynomial hierarchy'' Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 467, 459–472 (2011).
https://doi.org/10.1098/rspa.2010.0301

[49] Benjamin Villalonga, Sergio Boixo, Bron Nelson, Christopher Henze, Eleanor Rieffel, Rupak Biswas, and Salvatore Mandrà, ``A flexible high-performance simulator for verifying and benchmarking quantum circuits implemented on real hardware'' npj Quantum Information 5, 86 (2019).
https://doi.org/10.1038/s41534-019-0196-1

[50] Benjamin Villalonga, Dmitry Lyakh, Sergio Boixo, Hartmut Neven, Travis S Humble, Rupak Biswas, Eleanor G Rieffel, Alan Ho, and Salvatore Mandrà, ``Establishing the quantum supremacy frontier with a 281 Pflop/s simulation'' Quantum Science and Technology 5, 034003 (2020).
https://doi.org/10.1088/2058-9565/ab7eeb

[51] Dan Shepherd and Michael J. Bremner ``Temporally unstructured quantum computation'' Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 465, 1413–1439 (2009).
https://doi.org/10.1098/rspa.2008.0443

[52] Michael J. Bremner, Ashley Montanaro, and Dan J. Shepherd, ``Achieving quantum supremacy with sparse and noisy commuting quantum computations'' Quantum 1, 8 (2017).
https://doi.org/10.22331/q-2017-04-25-8

[53] Vojtěch Havlíček, Antonio D. Córcoles, Kristan Temme, Aram W. Harrow, Abhinav Kandala, Jerry M. Chow, and Jay M. Gambetta, ``Supervised learning with quantum-enhanced feature spaces'' Nature 567, 209–212 (2019).
https://doi.org/10.1038/s41586-019-0980-2

[54] Daniel Mills, Anna Pappa, Theodoros Kapourniotis, and Elham Kashefi, ``Information Theoretically Secure Hypothesis Test for Temporally Unstructured Quantum Computation (Extended Abstract)'' Electronic Proceedings in Theoretical Computer Science 266, 209â221 (2018).
https://doi.org/10.4204/eptcs.266.14

[55] Richard M. Karpand Richard J. Lipton ``Some Connections between Nonuniform and Uniform Complexity Classes'' Proceedings of the Twelfth Annual ACM Symposium on Theory of Computing 302â309 (1980).
https://doi.org/10.1145/800141.804678

[56] J. Misraand David Gries ``A constructive proof of Vizing's theorem'' Information Processing Letters 41, 131 –133 (1992).
https://doi.org/10.1016/0020-0190(92)90041-S
http://www.sciencedirect.com/science/article/pii/002001909290041S

[57] Juan Bermejo-Vega, Dominik Hangleiter, Martin Schwarz, Robert Raussendorf, and Jens Eisert, ``Architectures for Quantum Simulation Showing a Quantum Speedup'' Phys. Rev. X 8, 021010 (2018).
https://doi.org/10.1103/PhysRevX.8.021010

[58] D Hangleiter, M Kliesch, M Schwarz, and J Eisert, ``Direct certification of a class of quantum simulations'' Quantum Science and Technology 2, 015004 (2017).
https://doi.org/10.1088/2058-9565/2/1/015004

[59] Michael A. Nielsenand Isaac L. Chuang ``Quantum Computation and Quantum Information: 10th Anniversary Edition'' (2010).
https://doi.org/10.1017/CBO9780511976667

[60] Alexander Cowtan, Silas Dilkes, Ross Duncan, Will Simmons, and Seyon Sivarajah, ``Phase Gadget Synthesis for Shallow Circuits'' Electronic Proceedings in Theoretical Computer Science 318, 214â229 (2020).
https://doi.org/10.4204/eptcs.318.13

[61] Panagiotis Kl. Barkoutsos, Jerome F. Gonthier, Igor Sokolov, Nikolaj Moll, Gian Salis, Andreas Fuhrer, Marc Ganzhorn, Daniel J. Egger, Matthias Troyer, Antonio Mezzacapo, Stefan Filipp, and Ivano Tavernelli, ``Quantum algorithms for electronic structure calculations: Particle-hole Hamiltonian and optimized wave-function expansions'' Phys. Rev. A 98, 022322 (2018).
https://doi.org/10.1103/PhysRevA.98.022322

[62] Seyon Sivarajah, Silas Dilkes, Alexander Cowtan, Will Simmons, Alec Edgington, and Ross Duncan, ``t|ket〉: A retargetable compiler for NISQ devices'' Quantum Science and Technology (2020).
https://doi.org/10.1088/2058-9565/ab8e92

[63] ``pytket documentation'' https://cqcl.github.io/pytket/build/html/index.html.
https://cqcl.github.io/pytket/build/html/index.html

[64] ``qiskit documentation'' https://qiskit.org/documentation/.
https://qiskit.org/documentation/

[65] Sumeet Khatri, Ryan LaRose, Alexander Poremba, Lukasz Cincio, Andrew T. Sornborger, and Patrick J. Coles, ``Quantum-assisted quantum compiling'' Quantum 3, 140 (2019).
https://doi.org/10.22331/q-2019-05-13-140

[66] Aram W. Harrow, Benjamin Recht, and Isaac L. Chuang, ``Efficient discrete approximations of quantum gates'' Journal of Mathematical Physics 43, 4445–4451 (2002).
https://doi.org/10.1063/1.1495899

[67] Joel J. Wallmanand Joseph Emerson ``Noise tailoring for scalable quantum computation via randomized compiling'' Phys. Rev. A 94, 052325 (2016).
https://doi.org/10.1103/PhysRevA.94.052325

[68] Jeff Heckey, Shruti Patil, Ali JavadiAbhari, Adam Holmes, Daniel Kudrow, Kenneth R. Brown, Diana Franklin, Frederic T. Chong, and Margaret Martonosi, ``Compiler Management of Communication and Parallelism for Quantum Computation'' Proceedings of the Twentieth International Conference on Architectural Support for Programming Languages and Operating Systems 445-456 (2015).
https://doi.org/10.1145/2694344.2694357

[69] Prakash Murali, Jonathan M. Baker, Ali Javadi-Abhari, Frederic T. Chong, and Margaret Martonosi, ``Noise-Adaptive Compiler Mappings for Noisy Intermediate-Scale Quantum Computers'' Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems 1015â1029 (2019).
https://doi.org/10.1145/3297858.3304075

[70] Ali JavadiAbhari, Shruti Patil, Daniel Kudrow, Jeff Heckey, Alexey Lvov, Frederic T. Chong, and Margaret Martonosi, ``ScaffCC: Scalable compilation and analysis of quantum programs'' report (2015) Computing Frontiers 2014: Best Paper.
https://doi.org/10.1016/j.parco.2014.12.001
https://www.sciencedirect.com/science/article/pii/S0167819114001422

[71] Alexander J McCaskey, Dmitry I Lyakh, Eugene F Dumitrescu, Sarah S Powers, and Travis S Humble, ``XACC: a system-level software infrastructure for heterogeneous quantum–classical computing'' Quantum Science and Technology 5, 024002 (2020).
https://doi.org/10.1088/2058-9565/ab6bf6

[72] Alexander Cowtan, Silas Dilkes, Ross Duncan, Alexandre Krajenbrink, Will Simmons, and Seyon Sivarajah, ``On the Qubit Routing Problem'' 14th Conference on the Theory of Quantum Computation, Communication and Cryptography (TQC 2019) 135, 5:1–5:32 (2019).
https://doi.org/10.4230/LIPIcs.TQC.2019.5
http://drops.dagstuhl.de/opus/volltexte/2019/10397

[73] Prakash Murali, David C. Mckay, Margaret Martonosi, and Ali Javadi-Abhari, ``Software Mitigation of Crosstalk on Noisy Intermediate-Scale Quantum Computers'' Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems 1001â1016 (2020).
https://doi.org/10.1145/3373376.3378477

[74] IBM Quantum Experience User Guide ``Advanced Single-Qubit Gates'' (2019) https://quantum-computing.ibm.com/support/guides/user-guide.
https://quantum-computing.ibm.com/support/guides/user-guide

[75] Christopher Chamberland, Guanyu Zhu, Theodore J. Yoder, Jared B. Hertzberg, and Andrew W. Cross, ``Topological and Subsystem Codes on Low-Degree Graphs with Flag Qubits'' Phys. Rev. X 10, 011022 (2020).
https://doi.org/10.1103/PhysRevX.10.011022

[76] Iskren Vankov, Daniel Mills, Petros Wallden, and Elham Kashefi, ``Methods for classically simulating noisy networked quantum architectures'' Quantum Science and Technology 5, 014001 (2019).
https://doi.org/10.1088/2058-9565/ab54a4

[77] J. M. Pino, J. M. Dreiling, C. Figgatt, J. P. Gaebler, S. A. Moses, M. S. Allman, C. H. Baldwin, M. Foss-Feig, D. Hayes, K. Mayer, C. Ryan-Anderson, and B. Neyenhuis, ``Demonstration of the QCCD trapped-ion quantum computer architecture'' (2020).
arXiv:2003.01293

[78] Daniel Mills, Seyon Sivarajah, Travis L. Scholten, and Ross Duncan, ``Application-Motivated, Holistic Benchmarking of a Full Quantum Computing Stack: Experimental Data'' (2020).
https://doi.org/10.5281/zenodo.3832121

[79] Joseph Emerson, Yaakov S. Weinstein, Marcos Saraceno, Seth Lloyd, and David G. Cory, ``Pseudo-Random Unitary Operators for Quantum Information Processing'' Science 302, 2098–2100 (2003).
https://doi.org/10.1126/science.1090790
https://science.sciencemag.org/content/302/5653/2098

[80] Joseph Emerson, Etera Livine, and Seth Lloyd, ``Convergence conditions for random quantum circuits'' Phys. Rev. A 72, 060302 (2005).
https://doi.org/10.1103/PhysRevA.72.060302

[81] Fernando G. S. L. Brandão, Aram W. Harrow, and Michał Horodecki, ``Local Random Quantum Circuits are Approximate Polynomial-Designs'' Communications in Mathematical Physics 346, 397–434 (2016).
https://doi.org/10.1007/s00220-016-2706-8

[82] Andrew Faganand Ross Duncan ``Optimising Clifford Circuits with Quantomatic'' Electronic Proceedings in Theoretical Computer Science 287, 85â105 (2019).
https://doi.org/10.4204/eptcs.287.5

[83] M Blaauboerand R L de Visser ``An analytical decomposition protocol for optimal implementation of two-qubit entangling gates'' Journal of Physics A: Mathematical and Theoretical 41, 395307 (2008).
https://doi.org/10.1088/1751-8113/41/39/395307

[84] Easwar Magesanand Jay M. Gambetta ``Effective Hamiltonian models of the cross-resonance gate'' Phys. Rev. A 101, 052308 (2020).
https://doi.org/10.1103/PhysRevA.101.052308

[85] Timothy Proctor, Kenneth Rudinger, Kevin Young, Mohan Sarovar, and Robin Blume-Kohout, ``What Randomized Benchmarking Actually Measures'' Phys. Rev. Lett. 119, 130502 (2017).
https://doi.org/10.1103/PhysRevLett.119.130502

[86] E. Knill, D. Leibfried, R. Reichle, J. Britton, R. B. Blakestad, J. D. Jost, C. Langer, R. Ozeri, S. Seidelin, and D. J. Wineland, ``Randomized benchmarking of quantum gates'' Phys. Rev. A 77, 012307 (2008).
https://doi.org/10.1103/PhysRevA.77.012307

[87] Arnaud Carignan-Dugas, Kristine Boone, Joel J Wallman, and Joseph Emerson, ``From randomized benchmarking experiments to gate-set circuit fidelity: how to interpret randomized benchmarking decay parameters'' New Journal of Physics 20, 092001 (2018).
https://doi.org/10.1088/1367-2630/aadcc7

[88] Timothy J. Proctor, Arnaud Carignan-Dugas, Kenneth Rudinger, Erik Nielsen, Robin Blume-Kohout, and Kevin Young, ``Direct Randomized Benchmarking for Multiqubit Devices'' Phys. Rev. Lett. 123, 030503 (2019).
https://doi.org/10.1103/PhysRevLett.123.030503

[89] Jay M. Gambetta, A. D. Córcoles, S. T. Merkel, B. R. Johnson, John A. Smolin, Jerry M. Chow, Colm A. Ryan, Chad Rigetti, S. Poletto, Thomas A. Ohki, Mark B. Ketchen, and M. Steffen, ``Characterization of Addressability by Simultaneous Randomized Benchmarking'' Phys. Rev. Lett. 109, 240504 (2012).
https://doi.org/10.1103/PhysRevLett.109.240504

[90] David C. McKay, Andrew W. Cross, Christopher J. Wood, and Jay M. Gambetta, ``Correlated Randomized Benchmarking'' (2020).
arXiv:2003.02354

[91] Easwar Magesan, J. M. Gambetta, and Joseph Emerson, ``Scalable and Robust Randomized Benchmarking of Quantum Processes'' Phys. Rev. Lett. 106, 180504 (2011).
https://doi.org/10.1103/PhysRevLett.106.180504

[92] Easwar Magesan, Jay M. Gambetta, and Joseph Emerson, ``Characterizing quantum gates via randomized benchmarking'' Phys. Rev. A 85, 042311 (2012).
https://doi.org/10.1103/PhysRevA.85.042311

[93] A. D. Córcoles, Jay M. Gambetta, Jerry M. Chow, John A. Smolin, Matthew Ware, Joel Strand, B. L. T. Plourde, and M. Steffen, ``Process verification of two-qubit quantum gates by randomized benchmarking'' Phys. Rev. A 87, 030301 (2013).
https://doi.org/10.1103/PhysRevA.87.030301

[94] David C. McKay, Sarah Sheldon, John A. Smolin, Jerry M. Chow, and Jay M. Gambetta, ``Three-Qubit Randomized Benchmarking'' Phys. Rev. Lett. 122, 200502 (2019).
https://doi.org/10.1103/PhysRevLett.122.200502

Cited by

[1] Medina Bandic, Carmen G. Almudever, and Sebastian Feld, "Interaction graph-based characterization of quantum benchmarks for improving quantum circuit mapping techniques", Quantum Machine Intelligence 5 2, 40 (2023).

[2] Hamza Reffad, Adel Alti, and Ahmed Almuhirat, "A Dynamic Adaptive Bio-Inspired Multi-Agent System for Healthcare Task Deployment", Engineering, Technology & Applied Science Research 13 1, 10192 (2023).

[3] Elijah Pelofske, Andreas Bärtschi, and Stephan Eidenbenz, "Quantum Volume in Practice: What Users Can Expect From NISQ Devices", IEEE Transactions on Quantum Engineering 3, 1 (2022).

[4] Dewansh Khandelwal, Nimish Vyas, Priyanshi Skaktawat, Vaidehi Anwekar, C.U. Om Kumar, and D. Jeyakumar, Quantum Computing in Cybersecurity 275 (2023) ISBN:9781394166336.

[5] Cristina Cirstoiu, Silas Dilkes, Daniel Mills, Seyon Sivarajah, and Ross Duncan, "Volumetric Benchmarking of Error Mitigation with Qermit", Quantum 7, 1059 (2023).

[6] Michele Amoretti, "An Effective Framework for Full-Stack Benchmarking of Quantum Computers", Quantum Views 5, 52 (2021).

[7] Kentaro Yamamoto, David Zsolt Manrique, Irfan T. Khan, Hideaki Sawada, and David Muñoz Ramo, "Quantum hardware calculations of periodic systems with partition-measurement symmetry verification: Simplified models of hydrogen chain and iron crystals", Physical Review Research 4 3, 033110 (2022).

[8] Davide Ferrari and Michele Amoretti, Proceedings of the 19th ACM International Conference on Computing Frontiers 237 (2022) ISBN:9781450393386.

[9] Marie Salm, Johanna Barzen, Frank Leymann, Benjamin Weder, and Karoline Wild, Communications in Computer and Information Science 1429, 64 (2021) ISBN:978-3-030-87567-1.

[10] Thomas Lubinski, Sonika Johri, Paul Varosy, Jeremiah Coleman, Luning Zhao, Jason Necaise, Charles H. Baldwin, Karl Mayer, and Timothy Proctor, "Application-Oriented Performance Benchmarks for Quantum Computing", IEEE Transactions on Quantum Engineering 4, 1 (2023).

[11] Mohammad Kordzanganeh, Markus Buchberger, Basil Kyriacou, Maxim Povolotskii, Wilhelm Fischer, Andrii Kurkin, Wilfrid Somogyi, Asel Sagingalieva, Markus Pflitsch, and Alexey Melnikov, "Benchmarking Simulated and Physical Quantum Processing Units Using Quantum and Hybrid Algorithms", Advanced Quantum Technologies 6 8, 2300043 (2023).

[12] Adel Alti, Abderrahim Lakehal, and Philippe Roose, "A decentralized agent‐based semantic service control and self‐adaptation in smart health mobile applications", Concurrency and Computation: Practice and Experience 34 5, e6697 (2022).

[13] Hoa T. Nguyen, Muhammad Usman, and Rajkumar Buyya, "iQuantum: A toolkit for modeling and simulation of quantum computing environments", Software: Practice and Experience spe.3331 (2024).

[14] Jernej Rudi Finzgar, Philipp Ross, Leonhard Holscher, Johannes Klepsch, and Andre Luckow, 2022 IEEE International Conference on Quantum Computing and Engineering (QCE) 226 (2022) ISBN:978-1-6654-9113-6.

[15] Eduard Alarcón, Sergi Abadal, Fabio Sebastiano, Masoud Babaie, Edoardo Charbon, Peter Haring Bolívar, Maurizio Palesi, Elena Blokhina, Dirk Leipold, Bogdan Staszewski, Artur Garcia-Sáez, and Carmen G. Almudever, 2023 IEEE International Symposium on Circuits and Systems (ISCAS) 1 (2023) ISBN:978-1-6654-5109-3.

[16] Nils Herrmann, Daanish Arya, Marcus W. Doherty, Angus Mingare, Jason C. Pillay, Florian Preis, and Stefan Prestel, 2023 IEEE International Conference on Quantum Software (QSW) 162 (2023) ISBN:979-8-3503-0479-4.

[17] Junchao Wang, Guoping Guo, and Zheng Shan, "SoK: Benchmarking the Performance of a Quantum Computer", Entropy 24 10, 1467 (2022).

[18] Nils Quetschlich, Lukas Burgholzer, and Robert Wille, 2023 IEEE International Conference on Quantum Software (QSW) 43 (2023) ISBN:979-8-3503-0479-4.

[19] Manuel A. Serrano, José A. Cruz-Lemus, Ricardo Perez-Castillo, and Mario Piattini, "Quantum Software Components and Platforms: Overview and Quality Assessment", ACM Computing Surveys 55 8, 1 (2023).

[20] Florian J. Kiwit, Maximilian A. Wolf, Marwa Marso, Philipp Ross, Jeanette M. Lorenz, Carlos A. Riofrío, and Andre Luckow, "Benchmarking Quantum Generative Learning: A Study on Scalability and Noise Resilience using QUARK", arXiv:2403.18662, (2024).

[21] Elijah Pelofske, Andreas Bärtschi, and Stephan Eidenbenz, "Quantum Volume in Practice: What Users Can Expect from NISQ Devices", arXiv:2203.03816, (2022).

[22] Medina Bandić, Carmen G. Almudever, and Sebastian Feld, "Interaction graph-based characterization of quantum benchmarks for improving quantum circuit mapping techniques", arXiv:2212.06640, (2022).

[23] Pablo Andres-Martinez, Tim Forrer, Daniel Mills, Jun-Yi Wu, Luciana Henaut, Kentaro Yamamoto, Mio Murao, and Ross Duncan, "Distributing circuits over heterogeneous, modular quantum computing network architectures", arXiv:2305.14148, (2023).

[24] Medina Bandic, Sebastian Feld, and Carmen G. Almudever, "Full-stack quantum computing systems in the NISQ era: algorithm-driven and hardware-aware compilation techniques", arXiv:2204.06369, (2022).

The above citations are from Crossref's cited-by service (last updated successfully 2024-04-25 10:54:08) and SAO/NASA ADS (last updated successfully 2024-04-25 10:54:09). The list may be incomplete as not all publishers provide suitable and complete citation data.

This Paper is published in Quantum under the Creative Commons Attribution 4.0 International (CC BY 4.0) license. Copyright remains with the original copyright holders such as the authors or their institutions.

Application-Motivated, Holistic Benchmarking of a Full Quantum Computing Stack

Abstract

Popular summary

► BibTeX data

► References

Cited by

1 thought on “Application-Motivated, Holistic Benchmarking of a Full Quantum Computing Stack”