Lorenzo Martignoni
ABSTRACT
A CPU emulator is a software that simulates a hardware CPU. Emulators are widely used by computer scientists for various kind of activities (e.g., debugging, profiling, and malware analysis). Although no theoretical limitation prevents to develop an emulator that faithfully emulates a physical CPU, writing a fully featured emulator is a very challenging and error-prone task. Modern CISC architectures have a very rich instruction set, some instructions lack proper specifications, and others may have undefined effects in corner-cases. This paper presents a testing methodology specific for CPU emulators, based on fuzzing. The emulator is "stressed" with specially crafted test-cases, to verify whether the CPU is properly emulated or not. Improper behaviours of the emulator are detected by running the same test-case concurrently on the emulated and on the physical CPUs and by comparing the state of the two after the execution. Differences in the final state testify defects in the code of the emulator. We implemented this methodology in a prototype (codenamed EmuFuzzer), analysed four state-of-the-art IA-32 emulators (QEMU, Valgrind, Pin and BOCHS), and found several defects in each of them, some of which can prevent the proper execution of programs.
- U. Bayer, C. Kruegel, and E. Kirda. TTAnalyze: A Tool for Analyzing Malware. In 15th European Institute for Computer Antivirus Research Annual Conference (EICAR 2006), 2006.Google Scholar
- F. Bellard. QEMU, a fast and portable dynamic translator. In Proceedings of the annual conference on USENIX Annual Technical Conference (ATEC), Berkeley, CA, USA, 2005. USENIX Association. Google ScholarDigital Library
- C. Cadar, V. Ganesh, P. M. Pawlowski, D. L. Dill, and D. R. Engler. EXE: Automatically Generating Inputs of Death. In Proceedings of the 13th ACM conference on Computer and communications security, 2006. Google ScholarDigital Library
- B. Daniel, D. Dig, K. Garcia, and D. Marinov. Automated testing of refactoring engines. In Proceedings of the 6th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, Sept. 2007. Google ScholarDigital Library
- J. DeMott. The Evolving Art of Fuzzing. http://www.vdalabs.com/tools/The_Evolving_Art_of_Fuzzing.pdf.Google Scholar
- A. Dinaburg, P. Royal, M. Sharif, and W. Lee. Ether: Malware Analysis via Hardware Virtualization Extensions. In Proceedings of the 15th ACM conference on Computer and communications security, 2008. Google ScholarDigital Library
- P. Ferrie. Attacks on Virtual Machine Emulators. Technical report, Symantec Advanced Threat Research, 2006.Google Scholar
- T. Garfinkel and M. Rosenblum. A Virtual Machine Introspection Based Architecture for Intrusion Detection. In Proceedings of Network and Distributed Systems Security Symposium, NDSS, San Diego, California, USA. The Internet Society, Feb. 2003.Google Scholar
- P. Godefroid, M. Y. Levin, and D. Molnar. Automated Whitebox Fuzz Testing. In Proceedings of the Network and Distributed System Security Symposium, 2008.Google Scholar
- Google Inc. Android emulator. http://code.google.com/android/reference/emulator.html.Google Scholar
- Intel. Intel 64 and IA-32 Architectures Software Developer's Manual, Nov. 2008. Instruction Set Reference.Google Scholar
- R. Kaksonen. A Functional Method for Assessing Protocol Implementation Security. Technical report, VTT Electronics, 2001.Google Scholar
- J. Koziol, D. Litchfield, D. Aitel, C. Anley, S. Eren, N. Mehta, and R. Hassell. The Shellcoder's Handbook: Discovering and Exploiting Security Holes. John Wiley&Sons, 2004. Google ScholarDigital Library
- K. P. Lawton. Bochs: A Portable PC Emulator for Unix/X. Linux Journal, Sept. 1996. Google ScholarDigital Library
- H. A. Lichstein. When Should You Emulate? Datamation, 1969.Google Scholar
- C. Luk, R. Cohn, R. Muth, H. Patil, A. Klauser, G. Lowney, S. Wallace, V. J. Reddi, and K. Hazelwood. Pin: building customized program analysis tools with dynamic instrumentation. In Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation (PLDI), 2005. Google ScholarDigital Library
- P. S. Magnusson, M. Christensson, J. Eskilson, D. Forsgren, G. Hallberg, J. Högberg, F. Larsson, A. Moestedt, and B. Werner. Simics: A Full System Simulation Platform. Computer, 35(2), 2002. Google ScholarDigital Library
- R. Majumdar and K. Sen. Hybrid Concolic Testing. In Proceedings of the 29th international conference on Software Engineering (ICSE'07), 2007. Google ScholarDigital Library
- L. Martignoni, E. Stinson, M. Fredrikson, S. Jha, and J. C. Mitchell. A Layered Architecture for Detecting Malicious Behaviors. In Proceedings of the International Symposium on Recent Advances in Intrusion Detection (RAID), Lecture Notes in Computer Science. Springer, Sept. 2008. Google ScholarDigital Library
- W. M. McKeeman. Differential Testing for Software. Digital Technical Journal, 10(1), 1998.Google Scholar
- B. P. Miller, L. Fredrikson, and B. So. An Empirical Study of the Reliability of UNIX Utilities. Communications of the ACM, 33(12), December 1990. Google ScholarDigital Library
- G. J. Myers. The Art of Software Testing. John Wiley&Sons, 1978. Google ScholarDigital Library
- NetBSD/amd64. http://www.netbsd.org/ports/amd64/.Google Scholar
- N. Nethercote. Dynamic Binary Analysis and Instrumentation. PhD thesis, Computer Laboratory, University of Cambridge, United Kingdom, Nov. 2004.Google Scholar
- T. Ormandy. An Empirical Study into the Security Exposure to Host of Hostile Virtualized Environments. In Proceedings of CanSecWest Applied Security Conference, 2007.Google Scholar
- D. Quist and V. Smith. Detecting the Presence of Virtual Machines Using the Local Data Table. http://www.offensivecomputing.net/files/active/0/vm.pdf.Google Scholar
- T. Raffetseder, C. Kruegel, and E. Kirda. Detecting System Emulators. In Proceedings of Information Security Conference (ISC 2007). Springer-Verlag, 2007. Google ScholarDigital Library
- J. S. Robin and C. E. Irvine. Analysis of the intel pentium's ability to support a secure virtual machine monitor. In Proceedings of the 9th conference on USENIX Security Symposium (SSYMM'00), Berkeley, CA, USA, 2000. USENIX Association. Google ScholarDigital Library
- J. Rutkowska. Red Pill?or how to detect VMM using (almost) one CPU instruction. http://invisiblethings.org/papers/redpill.html.Google Scholar
- K. Sen, D. Marinov, and G. Agha. CUTE: a concolic unit testing engine for c. In Proceedings of the 10th European software engineering conference, 2005. Google ScholarDigital Library
- Sun Microsystem. VirtualBox. http://www.virtualbox.org.Google Scholar
- M. Sutton, A. Greene, and P. Amini. Fuzzing: Brute Force Vulnerability Discovery. Addison-Wesley Professional, 2007. Google ScholarDigital Library
Index Terms
- Testing CPU emulators
Recommendations
A methodology for testing CPU emulators
Testing, debugging, and error handling, formal methods, lifecycle concerns, evolution and maintenanceA CPU emulator is a software system that simulates a hardware CPU. Emulators are widely used by computer scientists for various kind of activities (e.g., debugging, profiling, and malware analysis). Although no theoretical limitation prevents developing ...
Icicle: A Re-designed Emulator for Grey-Box Firmware Fuzzing
ISSTA 2023: Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and AnalysisEmulation-based fuzzers enable testing binaries without source code and facilitate testing embedded applications where automated execution on the target hardware architecture is difficult and slow. The instrumentation techniques added to extract ...
Methods for Emulation of Multi-core CPU Performance
HPCC '11: Proceedings of the 2011 IEEE International Conference on High Performance Computing and CommunicationsWhen validating or evaluating real distributed applications, it is useful to be able to test the application in a large range of environments. In that context, emulation of CPU performance enables researchers to investigate how the performance of the ...
Comments