Cristian Cadar
Vijay Ganesh
Skip Abstract Section
Abstract
This article presents EXE, an effective bug-finding tool that automatically generates inputs that crash real code. Instead of running code on manually or randomly constructed input, EXE runs it on symbolic input initially allowed to be anything. As checked code runs, EXE tracks the constraints on each symbolic (i.e., input-derived) memory location. If a statement uses a symbolic value, EXE does not run it, but instead adds it as an input-constraint; all other statements run as usual. If code conditionally checks a symbolic expression, EXE forks execution, constraining the expression to be true on the true branch and false on the other. Because EXE reasons about all possible values on a path, it has much more power than a traditional runtime tool: (1) it can force execution down any feasible program path and (2) at dangerous operations (e.g., a pointer dereference), it detects if the current path constraints allow any value that causes a bug. When a path terminates or hits a bug, EXE automatically generates a test case by solving the current path constraints to find concrete values using its own co-designed constraint solver, STP. Because EXE’s constraints have no approximations, feeding this concrete input to an uninstrumented version of the checked code will cause it to follow the same path and hit the same bug (assuming deterministic code).
EXE works well on real code, finding bugs along with inputs that trigger them in: the BSD and Linux packet filter implementations, the dhcpd DHCP server, the pcre regular expression library, and three Linux file systems.
- Ball, T. 2004. A theory of predicate-complete test coverage and generation. In Proceedings of the 3rd International Symposium on Formal Methods for Components and Objects (FMCO’04). Google Scholar
Digital Library
- Ball, T. and Jones, R. B., eds. 2006. Proceedings of the 18th International Conference on Computer Aided Verification (CAV’06), Seattle, WA. Lecture Notes in Computer Science, vol. 4144. Springer. Google ScholarDigital Library
- Ball, T., Majumdar, R., Millstein, T., and Rajamani, S. K. 2001. Automatic predicate abstraction of C programs. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI’01). ACM Press, 203--213. Google ScholarDigital Library
- Ball, T. and Rajamani, S. 2001. Automatically validating temporal safety properties of interfaces. In Proceedings of the Workshop on Model Checking of Software (SPIN’01). Google ScholarDigital Library
- Barrett, C. and Berezin, S. 2004. CVC Lite: A new implementation of the cooperating validity checker. In Proceedings of the 18th International Conference on Computer Aided Verification (CAV’04), R. Alur and D. A. Peled Eds. Lecture Notes in Computer Science. Springer. Google ScholarDigital Library
- Barrett, C., Berezin, S., Shikanian, I., Chechik, M., Gurfinkel, A., and Dill, D. L. 2004. A practical approach to partial functions in CVC Lite. In Proceedings of the 2nd Workshop on Pragmatics of Decision Procedures in Automated Reasoning (PDPAR’04), Cork, Ireland.Google Scholar
- Boyer, R. S., Elspas, B., and Levitt, K. N. 1975. Select -- A formal system for testing and debugging programs by symbolic execution. ACM SIGPLAN Notices 10, 6, 234--245. Google ScholarDigital Library
- Brat, G., Havelund, K., Park, S., and Visser, W. 2000. Model checking programs. In Proceedings of the IEEE International Conference on Automated Software Engineering (ASE’00). Google ScholarDigital Library
- Brumley, D., Newsome, J., Song, D., Wang, H., and Jha, S. 2006. Towards automatic generation of vulnerability-based signatures. In Proceedings of the 2006 IEEE Symposium on Security and Privacy (SSP’06). Google ScholarDigital Library
- Bryant, R. E., Lahiri, S. K., and Seshia, S. A. 2002. Modeling and verifying systems using a logic of counter arithmetic with lambda expressions and uninterpreted functions. In Proceedings of the Conference on Computer-Aided Verification (CAV’02), E. Brinksma and K. G. Larsen Eds. Springer-Verlag, 78--92. Google ScholarDigital Library
- Bush, W., Pincus, J., and Sielaff, D. 2000. A static analyzer for finding dynamic programming errors. Softw. Pract. Exp. 30, 7, 775--802. Google ScholarDigital Library
- Cadar, C. and Engler, D. 2005. Execution generated test cases: How to make systems code crash itself. In Proceedings of the 12th International SPIN Workshop on Model Checking of Software (SPIN’05). A longer version of this article appeared as Tech. rep. CSTR-2005-04, Computer Systems Laboratory, Stanford University. Google ScholarDigital Library
- Cadar, C., Ganesh, V., Pawlowski, P., Dill, D., and Engler, D. 2006. EXE: Automatically generating inputs of death. In Proceedings of the 13th ACM Conference on Computer and
Communications Security (CCS’06). Google ScholarDigital Library
- Clarke, E. and Kroening, D. 2003. Hardware verification using ANSI-C programs as a reference. In Proceedings of Asia and South Pacific Design Automation Conference (ASP-DAC’03. IEEE Computer Society Press, 308--311. Google ScholarDigital Library
- Cook, B., Kroening, D., and Sharygina, N. 2005. Cogent: Accurate theorem proving for program verification. In Proceedings of the Conference on Computer-Aided Verification (CAV’05), K. Etessami and S. K. Rajamani Eds. Lecture Notes in Computer Science, vol. 3576. Springer Verlag, 296--300. Google ScholarDigital Library
- Corbett, J., Dwyer, M., Hatcliff, J., Laubach, S., Pasareanu, C., Robby, and Zheng, H. 2000. Bandera: Extracting finite-state models from Java source code. In Proceedings of the 22nd International Conference on Software Engineering (ICSE’00). Google ScholarDigital Library
- Cormen, T. H., Leiserson, C. E., Rivest, R. L., and Stein, C. 2001. Introduction to Algorithms. The MIT Electrical Engineering and Computer Science Series. MIT Press/McGraw Hill. Google ScholarDigital Library
- Coverity. SWAT: the Coverity software analysis toolset. http://coverity.com.Google Scholar
- Das, M., Lerner, S., and Seigle, M. 2002. Path-sensitive program verification in polynomial time. In Proceedings of the ACM SIGPLAN 2002 Conference on Programming Language Design and Implementation (PLDI’02). Berlin, Germany. Google ScholarDigital Library
- DeLine, R. and Fähndrich, M. 2001. Enforcing high-level protocols in low-level software. In Proceedings of the ACM SIGPLAN 2001 Conference on Programming Language Design and Implementation (PLDI’01). Google ScholarDigital Library
- Een, N. and Sorensson, N. 2003. An extensible SAT-solver. In Proceedings of the 6th International Conference on Theory and Applications of Satisfiability Testing (SAT’03). 78--92.Google Scholar
- Ferguson, R. and Korel, B. 1996. The chaining approach for software test data generation. Trans. Softw. Eng. Methodol. 5, 1, 63--86. Google ScholarDigital Library
- Flanagan, C. and Freund, S. N. 2000. Type-based race detection for Java. In Proceedings of the SIGPLAN Conference on Programming Language Design and Implementation (PLDI’00). 219--232. Google ScholarDigital Library
- Flanagan, C., Leino, K., Lillibridge, M., Nelson, G., Saxe, J., and Stata, R. 2002. Extended static checking for Java. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI’02). ACM Press. Google ScholarDigital Library
- Foster, J., Terauchi, T., and Aiken, A. 2002. Flow-sensitive type qualifiers. In Proceedings of the ACM SIGPLAN 2002 Conference on Programming Language Design and Implementation (PLDI’02). Google ScholarDigital Library
- Godefroid, P. 1997. Model Checking for Programming Languages using VeriSoft. In Proceedings of the 24th ACM Symposium on Principles of Programming Languages (POPL’97). Google ScholarDigital Library
- Godefroid, P., Klarlund, N., and Sen, K. 2005. DART: Directed automated random testing. In Proceedings of the Conference on Programming Language Design and Implementation (PLDI’05). ACM Press. Google ScholarDigital Library
- Gotlieb, A., Botella, B., and Rueher, M. 1998. Automatic test data generation using constraint solving techniques. In Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’98). ACM Press, 53--62. Google ScholarDigital Library
- Gupta, N., Mathur, A. P., and Soffa, M. L. 1998. Automated test data generation using an iterative relaxation method. In Proceedings of the 6th ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE’98). ACM Press, 231--244. Google ScholarDigital Library
- Hastings, R. and Joyce, B. 1992. Purify: Fast detection of memory leaks and access errors. In Proceedings of the Winter USENIX Conference (USENIX’92).Google Scholar
- Holzmann, G. J. 1997. The model checker SPIN. Softw. Eng. 23, 5, 279--295. Google ScholarDigital Library
- Holzmann, G. J. 2001. From code to models. In Proceedings of the 2nd International Conference on Applications of Concurrency to System Design. (ACSD’01), 3--10. Google ScholarDigital Library
- Khurshid, S., Pasareanu, C. S., and Visser, W. 2003. Generalized symbolic execution for model checking and testing. In Proceedings of the 9th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS’03). Google ScholarDigital Library
- Larson, E. and Austin, T. 2003. High coverage detection of input-related security faults. In Proceedings of the 12th USENIX Security Symposium (SECURITY’03). Google ScholarDigital Library
- Miller, B. P., Fredriksen, L., and So, B. 1990. An empirical study of the reliability of UNIX utilities. Comm. ACM 33, 12, 32--44. Google ScholarDigital Library
- Necula, G. C., McPeak, S., Rahul, S., and Weimer, W. 2002. CIL: Intermediate language and tools for analysis and transformation of C programs. In Proceedings of the International Conference on Compiler Construction (CC’02). Google ScholarDigital Library
- Nelson, G. and Oppen, D. 1979. Simplification by cooperating decision procedures. ACM Trans. Program. Lang. Syst. 1, 2, 245--57. Google ScholarDigital Library
- Nethercote, N. and Seward, J. 2003. Valgrind: A program supervision framework. Electron. Notes Theor. Comput. Sci. 89, 2.Google ScholarCross Ref
- PCRE. PCRE - Perl Compatible Regular Expressions. http://www.pcre.org/.Google Scholar
- PCRE - CERT 2005. PCRE Regular Expression Heap Overflow. US-CERT Cyber Security Bulletin SB05-334. http://www.us-cert.gov/cas/bulletins/SB05-334.html#pcre.Google Scholar
- Ruwase, O. and Lam, M. S. 2004. A practical dynamic buffer overflow detector. In Proceedings of the 11th Annual Network and Distributed System Security Symposium (NDSS’04). 159--169.Google Scholar
- Sen, K., Marinov, D., and Agha, G. 2005. CUTE: A concolic unit testing engine for C. In Proceedings of the 5th Joint Meeting of the European Software Engineering Conference and ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE’05). Google ScholarDigital Library
- SMTLIB 2006. SMTLIB competition. http://www.csl.sri.com/users/demoura/smt-comp.Google Scholar
- Wagner, D., Foster, J., Brewer, E., and Aiken, A. 2000. A first step towards automated detection of buffer overrun vulnerabilities. In Proceedings of the Network and Distributed Systems Security Conference (NDSS’00). San Diego, CA.Google Scholar
- Xie, Y. and Aiken, A. 2005. Scalable error detection using Boolean satisfiability. In Proceedings of the 32nd Annual Symposium on Principles of Programming Languages (POPL’05). Google ScholarDigital Library
- Xie, Y. and Aiken, A. 2005. Saturn: A SAT-based tool for bug detection. In Proceedings of the Conference on Computer-Aided Verification (CAV’05), K. Etessami and S. K. Rajamani Eds. Lecture Notes in Computer Science, vol. 3576. Springer, 139--143. Google ScholarDigital Library
- Yang, J., Sar, C., Twohey, P., Cadar, C., and Engler, D. 2006. Automatically generating malicious disks using symbolic execution. In Proceedings of the IEEE Symposium on Security and Privacy (SSP’06). Google ScholarDigital Library
- Yang, J., Twohey, P., Engler, D., and Musuvathi, M. 2004. Using model checking to find serious file system errors. In Proceedings of the Symposium on Operating Systems Design and Implementation (OSDI’04). Google ScholarDigital Library
Index Terms
- EXE: Automatically Generating Inputs of Death
Recommendations
EXE: automatically generating inputs of death
CCS '06: Proceedings of the 13th ACM conference on Computer and communications securityThis paper presents EXE, an effective bug-finding tool that automatically generates inputs that crash real code. Instead of running code on manually or randomly constructed input, EXE runs it on symbolic input initially allowed to be "anything." As ...
User-defined backtracking criteria for symbolic execution
Symbolic execution is a path-sensitive program analysis technique that aids users with program verification. To avoid exploring infeasible paths, symbolic execution checks the prefix of a current path for feasibility by adding a branch constraint to the ...
Multiplex symbolic execution: exploring multiple paths by solving once
ASE '20: Proceedings of the 35th IEEE/ACM International Conference on Automated Software EngineeringPath explosion and constraint solving are two challenges to symbolic execution's scalability. Symbolic execution explores the program's path space with a searching strategy and invokes the underlying constraint solver in a black-box manner to check the ...
Comments