Eric Schkufza
Alex Aiken
Abstract
We formulate the loop-free binary superoptimization task as a stochastic search problem. The competing constraints of transformation correctness and performance improvement are encoded as terms in a cost function, and a Markov Chain Monte Carlo sampler is used to rapidly explore the space of all possible programs to find one that is an optimization of a given target program. Although our method sacrifices completeness, the scope of programs we are able to consider, and the resulting quality of the programs that we produce, far exceed those of existing superoptimizers. Beginning from binaries compiled by llvm -O0 for 64-bit x86, our prototype implementation, STOKE, is able to produce programs which either match or outperform the code produced by gcc -O3, icc -O3, and in some cases, expert handwritten assembly.
- C. Andrieu, N. de Freitas, A. Doucet, and M. I. Jordan. An introduction to MCMC for machine learning. Machine Learning, 50(1-2):5--43, 2003.Google Scholar
Cross Ref
- S. Bansal and A. Aiken. Automatic generation of peephole superoptimizers. In ASPLOS, pages 394--403, 2006. Google ScholarDigital Library
- L. S. Blackford, J. Demmel, J. Dongarra, I. Duff, S. Hammarling, G. Henry, M. Heroux, L. Kaufman, A. Lumsdaine, A. Petitet, R. Pozo, K. Remington, and R. C. Whaley. An updated set of basic linear algebra subprograms (BLAS). ACM Transactions on Mathematical Software, 28:135--151, 2001. Google ScholarDigital Library
- C. Cadar, D. Dunbar, and D. R. Engler. Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs. In OSDI, pages 209--224, 2008. Google ScholarDigital Library
- S. Chenney and D. A. Forsyth. Sampling plausible solutions to multi-body constraint problems. In SIGGRAPH, pages 219--228, 2000. Google ScholarDigital Library
- P. Diaconis. The markov chain monte carlo revolution. Bulletin of the American Mathematical Society, 46(2):179--205, Nov. 2008.Google ScholarCross Ref
- V. Ganesh and D. L. Dill. A decision procedure for bit-vectors and arrays. In CAV, pages 519--531, 2007. Google ScholarDigital Library
- W. R. Gilks. Markov Chain Monte Carlo in Practice. Chapman and Hall/CRC, 1999.Google Scholar
- S. Gulwani, S. Jha, A. Tiwari, and R. Venkatesan. Synthesis of loop-free programs. In PLDI, pages 62--73, 2011. Google ScholarDigital Library
- W. K. Hastings. Monte carlo sampling methods using markov chains and their applications. Biometrika, 57(1):97--109, Apr. 1970.Google ScholarCross Ref
- R. Joshi, G. Nelson, and K. H. Randall. Denali: A goal-directed superoptimizer. In PLDI, pages 304--314, 2002. Google ScholarDigital Library
- P. Liang, M. I. Jordan, and D. Klein. Learning programs: A hierarchical bayesian approach. In ICML, pages 639--646, 2010.Google Scholar
- C.-K. Luk, R. S. Cohn, R. Muth, H. Patil, A. Klauser, P. G. Lowney, S. Wallace, V. J. Reddi, and K. M. Hazelwood. Pin: Building customized program analysis tools with dynamic instrumentation. In PLDI, pages 190--200, 2005. Google ScholarDigital Library
- H. Massalin. Superoptimizer - a look at the smallest program. In ASPLOS, pages 122--126, 1987. Google ScholarDigital Library
- N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller. Equation of state calculations by fast computing machines. The Journal of Chemical Physics, 21(6):1087--1092, 1953.Google ScholarCross Ref
- A. F. Neuwald, J. S. Liu, D. J. Lipman, and C. E. Lawrence. Extracting protein alignment models from the sequence database. Nucleic Acids Research, 25:1665--1677, 1997.Google ScholarCross Ref
- A. Solar-Lezama, L. Tancau, R. Bodık, S. A. Seshia, and V. A. Saraswat. Combinatorial sketching for finite programs. In ASPLOS, pages 404--415, 2006. Google ScholarDigital Library
- R. Tate, M. Stepp, Z. Tatlock, and S. Lerner. Equality saturation: a new approach to optimization. In POPL, pages 264--276, 2009. Google ScholarDigital Library
- E. Veach and L. J. Guibas. Metropolis light transport. In SIGGRAPH, pages 65--76, 1997. Google ScholarDigital Library
- H. S. Warren. Hacker's Delight. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 2002. Google ScholarDigital Library
Index Terms
- Stochastic superoptimization
Recommendations
Stochastic superoptimization
ASPLOS '13We formulate the loop-free binary superoptimization task as a stochastic search problem. The competing constraints of transformation correctness and performance improvement are encoded as terms in a cost function, and a Markov Chain Monte Carlo sampler ...
Stochastic superoptimization
ASPLOS '13: Proceedings of the eighteenth international conference on Architectural support for programming languages and operating systemsWe formulate the loop-free binary superoptimization task as a stochastic search problem. The competing constraints of transformation correctness and performance improvement are encoded as terms in a cost function, and a Markov Chain Monte Carlo sampler ...
Stochastic optimization of floating-point programs with tunable precision
PLDI '14: Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and ImplementationThe aggressive optimization of floating-point computations is an important problem in high-performance computing. Unfortunately, floating-point instruction sets have complicated semantics that often force compilers to preserve programs as written. We ...
Comments