Abstract
We present an application of learning-based testing to the problem of automated test case generation (ATCG) for numerical software. Our approach uses n-dimensional polynomial models as an algorithmically learned abstraction of the SUT which supports n-wise testing. Test cases are iteratively generated by applying a satisfiability algorithm to first-order program specifications over real closed fields and iteratively refined piecewise polynomial models.
We benchmark the performance of our iterative ATCG algorithm against iterative random testing, and empirically analyse its performance in finding injected errors in numerical codes. Our results show that for software with small errors, or long mean time to failure, learning-based testing is increasingly more efficient than iterative random testing.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Caviness, B.F., Johnson, J.R.: Quantifier Elimination and Cylindrical Algebraic Decomposition. Springer, Heidelberg (1998)
Chauhan, P., Clarke, E.M., Kukula, J.H., Sapra, S., Veith, H., Wang, D.: Automated abstraction refinement for model checking large state spaces using sat based conflict analysis. In: Aagaard, M.D., O’Leary, J.W. (eds.) FMCAD 2002. LNCS, vol. 2517, Springer, Heidelberg (2002)
Clarke, E., Gupta, A., Kukula, J., Strichman, O.: Sat-based abstraction refinement using ilp and machine learning. In: Brinksma, E., Larsen, K.G. (eds.) CAV 2002. LNCS, vol. 2404, Springer, Heidelberg (2002)
Cox, M.G., Harris, P.M., Johnson, E.G., Kenward, P.D., Parkin, G.I.: Testing the numerical correctness of software. Technical Report CMSC 34/04, National Physical Laboratory, Teddington (January 2004)
Groce, A., Peled, D., Yannakakis, M.: Adaptive model checking. Logic Journal of the IGPL 14(5), 729–744 (2006)
Hatton, L., Roberts, A.: How accurate is scientific software? ACM Transactions on Software Engineering 20(10), 786–797 (1994)
Hatton, L.: The chimera of software quality. Computer 40(8), 104, 102–103 (2007)
Knupp, P., Salari, K.: Verification of Computer Codes in Computational Science and Engineering. CRC Press, Boca Raton (2002)
Loeckx, J., Sieber, K.: The foundations of program verification, 2nd edn. John Wiley & Sons, Inc., New York (1987)
Meinke, K.: Automated black-box testing of functional correctness using function approximation. In: ISSTA 2004: Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis, pp. 143–153. ACM, New York (2004)
Poston, R.M.: Automating Specification-Based Software Testing. IEEE Computer Society Press, Los Alamitos (1997)
Reimer, M.: Multivariate Polynomial Approximation. Birkhäuser, Basel (October 2003)
Roache, P.J.: Building pde codes to be verifiable and validatable. Computing in Science and Engineering, 30–38 (September/October 2004)
Tarski, A.: Decision Method for Elementary Algebra and Geometry. Univ. of California Press, Berkeley (1951)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 IFIP International Federation for Information Processing
About this paper
Cite this paper
Meinke, K., Niu, F. (2010). A Learning-Based Approach to Unit Testing of Numerical Software. In: Petrenko, A., Simão, A., Maldonado, J.C. (eds) Testing Software and Systems. ICTSS 2010. Lecture Notes in Computer Science, vol 6435. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-16573-3_16
Download citation
DOI: https://doi.org/10.1007/978-3-642-16573-3_16
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-16572-6
Online ISBN: 978-3-642-16573-3
eBook Packages: Computer ScienceComputer Science (R0)