Wei Jin
Alessandro Orso
ABSTRACT
Reproducing and debugging field failures--failures that occur on user machines after release--are challenging tasks for developers. To help the first task, in previous work we have proposed BugRedux, a technique for reproducing, in-house, failures observed in the field. Although BugRedux can help developers reproduce field failures, it does not provide any specific support for debugging such failures. To address this limitation, in this paper we present F3, a novel technique that builds on BugRedux and extends it with support for fault localization. Specifically, in F3 we extend our previous technique in two main ways: first, we modify BugRedux so that it generates multiple failing and passing executions "similar" to the observed field failure; second, we add to BugRedux debugging capabilities by combining it with a customized fault-localization technique. The results of our empirical evaluation, performed on a set of real-world programs and field failures, are promising: for all the failures considered, F3 was able to (1) synthesize passing and failing executions and (2) successfully use the synthesized executions to perform fault localization and, ultimately, help debugging.
- Software-artifact Infrastructure Repository. http://sir.unl.edu/, Apr. 2012.Google Scholar
- Technical Note TN2123: CrashReporter. http://developer. apple.com/technotes/tn2004/tn2123.html, Apr. 2012.Google Scholar
- Windows Error Reporting: Getting Started. http://www. microsoft.com/whdc/maintain/StartWER.mspx, 2012.Google Scholar
- R. Abreu, P. Zoeteweij, and A. J. C. v. Gemund. An Evaluation of Similarity Coefficients for Software Fault Localization. In Proc. of the 12th Pacific Rim International Symposium on Dependable Computing, pages 39–46, 2006. Google ScholarDigital Library
- R. Abreu, P. Zoeteweij, and A. J. C. van Gemund. An Observation-based Model for Fault Localization. In Proc. of the 2008 International Workshop on Dynamic Analysis, pages 64–70, 2008. Google ScholarDigital Library
- S. Artzi, J. Dolby, F. Tip, and M. Pistoia. Directed Test Generation for Effective Fault Localization. In Proc. of the 19th International Symposium on Software Testing and Analysis, pages 49–60, 2010. Google ScholarDigital Library
- S. Artzi, J. Dolby, F. Tip, and M. Pistoia. Fault Localization for Dynamic Web Applications. IEEE Transactions on Software Engineering, 38(2):314 –335, March-April 2012. Google ScholarDigital Library
- T. Avgerinos, S. K. Cha, B. L. T. Hao, and D. Brumley. AEG: Automatic Exploit Generation. In Proc. of the 18th Network and Distributed System Security Symposium, Feb. 2011.Google Scholar
- G. K. Baah, A. Podgurski, and M. J. Harrold. Mitigating the Confounding Effects of Program Dependences for Effective Fault Localization. In Proc. of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering, pages 146–156, 2011. Google ScholarDigital Library
- J. Bailey and P. J. Stuckey. Discovery of Minimal Unsatisfiable Subsets of Constraints Using Hitting Set Dualization. In Proc. of the 7th International Conference on Practical Aspects of Declarative Languages, pages 174–186, 2005. Google ScholarDigital Library
- C. Cadar, D. Dunbar, and D. Engler. KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs. In Proc. of the 8th USENIX Conference on Operating Systems Design and Implementation, pages 209–224, 2008. Google ScholarDigital Library
- T. M. Chilimbi, B. Liblit, K. Mehra, A. V. Nori, and K. Vaswani. H OLMES : Effective Statistical Debugging via Efficient Path Profiling. In Proc. of the 31st International Conference on Software Engineering, pages 34–44, 2009. Google ScholarDigital Library
- J. Christ, E. Ermis, M. Schäf, and T. Wies. Flow-sensitive Fault Localization. In Proc. of the 14th International Conference on Verification, Model Checking, and Abstract Interpretation, 2013.Google ScholarDigital Library
- J. Clause and A. Orso. A Technique for Enabling and Supporting Debugging of Field Failures. In ICSE 2007, pages 261–270, 2007. Google ScholarDigital Library
- J. Clause and A. Orso. P ENUMBRA : Automatically Identifying Failure-Relevant Inputs Using Dynamic Tainting. In Proc. of the 2009 International Symposium on Software Testing and Analysis, pages 249–260, 2009. Google ScholarDigital Library
- H. Cleve and A. Zeller. Locating Causes of Program Failures. In Proc. of the 27th International Conference on Software Engineering, pages 342–351, 2005. Google ScholarDigital Library
- S. Elbaum and M. Diep. Profiling Deployed Software: Assessing Strategies and Testing Opportunities. IEEE Transactions on Software Engineering, 31(4):312–327, 2005. Google ScholarDigital Library
- W. Eric Wong, V. Debroy, and B. Choi. A Family of Code Coverage-based Heuristics for Effective Fault Localization. Journal of System and Software, 83(2):188–208, Feb. 2010. Google ScholarDigital Library
- E. Ermis, M. Schäf, and T. Wies. Error Invariants. In Proc. of the 18th International Symposium on Formal Methods, pages 187–201, 2012.Google Scholar
- A. Groce. Error Explanation with Distance Metrics. In Tools and Algorithms for the Construction and Analysis of Systems, pages 108–122, 2004.Google ScholarCross Ref
- D. M. Hilbert and D. F. Redmiles. Extracting Usability Information from User Interface Events. ACM Computing Surveys, 32(4):384–421, Dec. 2000. Google ScholarDigital Library
- D. Jeffrey, N. Gupta, and R. Gupta. Fault Localization Using Value Replacement. In Proc. of the 2008 International Symposium on Software Testing and Analysis, pages 167–178, 2008. Google ScholarDigital Library
- W. Jin and A. Orso. BugRedux: Reproducing Field Failures for In-house Debugging. In Proc. of the 34th International Conference on Software Engineering, pages 474–484, 2012. Google ScholarDigital Library
- J. A. Jones, M. J. Harrold, and J. Stasko. Visualization of Test Information to Assist Fault Localization. In Proc. of the 24th International Conference on Software Engineering, pages 467–477, 2002. Google ScholarDigital Library
- M. Jose and R. Majumdar. Cause Clue Clauses: Error Localization Using Maximum Satisfiability. In Proc. of the 2011 ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 437–446, 2011. Google ScholarDigital Library
- B. Liblit, A. Aiken, A. X. Zheng, and M. I. Jordan. Bug isolation via remote program sampling. In Proc. of the 2003 ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 141–154, 2003. Google ScholarDigital Library
- B. Liblit, M. Naik, A. X. Zheng, A. Aiken, and M. I. Jordan. Scalable Statistical Bug Isolation. In Proc. of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 15–26, 2005. Google ScholarDigital Library
- C. Liu, X. Yan, L. Fei, J. Han, and S. P. Midkiff. SOBER: Statistical Model-based Bug Localization. In Proc. of the 10th European Software Engineering Conference and 13th ACM SIGSOFT Symposium on the Foundations of Software Engineering, pages 286–295, 2005. Google ScholarDigital Library
- S. Lu, Z. Li, F. Qin, L. Tan, P. Zhou, and Y. Zhou. BugBench: Benchmarks for Evaluating Bug Detection Tools. In Workshop on the Evaluation of Software Defect Detection Tools, 2005.Google Scholar
- L. Naish, H. J. Lee, and K. Ramamohanarao. A Model for Spectra-based Software Diagnosis. ACM Transactions on Software Engineering Methodology, 20(3):11:1–11:32, August 2011. Google ScholarDigital Library
- C. Parnin and A. Orso. Are Automated Debugging Techniques Actually Helping Programmers? In Proc. of the 2011 International Symposium on Software Testing and Analysis, pages 199–209, July 2011. Google ScholarDigital Library
- C. Pavlopoulou and M. Young. Residual Test Coverage Monitoring. In Proc. of the 21st International Conference on Software Engineering, pages 277–284, 1999. Google ScholarDigital Library
- J. Röβler, G. Fraser, A. Zeller, and A. Orso. Isolating Failure Causes Through Test Case Generation. In Proc. of the 2012 International Symposium on Software Testing and Analysis, pages 309–319, 2012. Google ScholarDigital Library
- R. Santelices, J. A. Jones, Y. Yu, and M. J. Harrold. Lightweight Fault-localization Using Multiple Coverage Types. In Proc. of the 31st International Conference on Software Engineering, pages 56–66, 2009. Google ScholarDigital Library
- W. N. Sumner, T. Bao, and X. Zhang. Selecting Peers for Execution Comparison. In Proc. of the 2011 International Symposium on Software Testing and Analysis, pages 309–319, 2011. Google ScholarDigital Library
- S. Yoo, M. Harman, and D. Clark. Fault Localization Prioritization: Comparing Information Theoretic and Coverage Based Approaches. ACM Transactions on Software Engineering Methodology, to appear.Google Scholar
- C. Zamfir and G. Candea. Execution Synthesis: A Technique for Automated Software Debugging. In Proc. of the 5th European Conference on Computer Systems, pages 321–334, 2010. Google ScholarDigital Library
- A. Zeller. Yesterday, my program worked. Today, it does not. Why? In Proc. of the 7th European Software Engineering Conference Held Jointly With the 7th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pages 253–267, 1999. Google ScholarDigital Library
- A. Zeller. Isolating Cause-effect Chains From Computer Programs. In Proc. of the 10th ACM SIGSOFT Symposium on Foundations of Software Engineering, pages 1–10, 2002. Google ScholarDigital Library
- A. Zeller and R. Hildebrandt. Simplifying and Isolating Failure-Inducing Input. IEEE Transactions on Software Engineering, 28(2):183–200, Feb. 2002. Google ScholarDigital Library
- X. Zhang, N. Gupta, and R. Gupta. Locating Faults Through Automated Predicate Switching. In Proc. of the 28th International Conference on Software Engineering, pages 272–281, 2006. Google ScholarDigital Library
- Z. Zhang, W. K. Chan, T. H. Tse, B. Jiang, and X. Wang. Capturing Propagation of Infected Program States. In The 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, pages 43–52, 2009. Google ScholarDigital Library
- T. Zimmermann, R. Premraj, N. Bettenburg, S. Just, A. Schröter, and C. Weiss. What Makes a Good Bug Report? IEEE Transactions on Software Engineering, 36(5):618–643, Sept. 2010. Google ScholarDigital Library
Index Terms
- F3: fault localization for field failures
Recommendations
Automated Support for Reproducing and Debugging Field Failures
Special Issue on ISSTA 2013As confirmed by a recent survey conducted among developers of the Apache, Eclipse, and Mozilla projects, two extremely challenging tasks during maintenance are reproducing and debugging field failures—failures that occur on user machines after release. ...
Interactive fault localization using test information
Special section on trust and reputation management in future computing systmes and applicationsDebugging is a time-consuming task in software development. Although various automated approches have been proposed, they are not effective enough. On the other hand, in manual debugging, developers have difficulty in choosing breakpoints. To address ...
Fault localization for build code errors in makefiles
ICSE Companion 2014: Companion Proceedings of the 36th International Conference on Software EngineeringBuilding is an important process in software development. In large software projects, build code has a high level of complexity, churn rate, and defect proneness. While several automated approaches exist to help developers in localizing faults in ...
Comments