ABSTRACT
Automation of acceptance test suites becomes necessary in the context of agile software development practices, which require rapid feedback on the quality of code changes. To this end, companies try to automate their acceptance tests as much as possible. Unfortunately, the growth of the automated test suites, by several automation testers, gives rise to potential test smells, i.e., poorly designed test code, being introduced in the test code base, which in turn may increase the cost of maintaining the code and creating new one. In this paper, we investigate this problem in the context of our industrial partner, BGL BNP Paribas, and introduce Ukwikora, an automated tool that statically analyzes acceptance test suites, enabling the continuous inspection of the test code base. Ukwikora targets code written in the Robot Framework syntax, a popular framework for writing Keyword-Driven tests. Ukwikora has been successfully deployed at BGL BNP Paribas, detecting issues otherwise unknown to the automation testers, such as the presence of duplicated test code, dead test code and dependency issues among the tests. The success of our case study reinforces the need for additional research and tooling for acceptance test suites.
- Emil Alégroth, Robert Feldt, and Pirjo Kolström. 2016. Maintenance of Automated Test Suites in Industry: An Empirical study on Visual GUI Testing. Information and Software Technology 73 (feb 2016), 66–80. 01.012 arXiv: 1602.01226 Google ScholarDigital Library
- Gabriele Bavota, Abdallah Qusef, Rocco Oliveto, Andrea De Lucia, and Dave Binkley. 2015. Are test smells really harmful? An empirical study. Empirical Software Engineering 20, 4 (aug 2015), 1052–1094. Google ScholarDigital Library
- Thierry Lavoie, Mathieu Mérineau, Ettore Merlo, and Pascal Potvin. 2017. A case study of TTCN-3 test scripts clone analysis in an industrial telecommunication setting. Information and Software Technology 87 (jul 2017), 32–45. Google ScholarDigital Library
- Maurizio Leotta, Diego Clerissi, Filippo Ricca, and Paolo Tonella. 2013. Capturereplay vs. programmable web testing: An empirical assessment during test case evolution. In 2013 20th Working Conference on Reverse Engineering (WCRE). IEEE, 272–281.Google ScholarCross Ref
- Robot RobotFramework. 2019. Introduction. http://robotframework.org/Google Scholar
- Renaud Rwemalika, Marinos Kintis, Mike Papadakis, Yves Le Traon, and Pierre Lorrach. 2019. On the Evolution of Keyword-Driven Test Suites. In 12th IEEE Conference on Software Testing, Validation and Verification (ICST). IEEE Computer Society, Xi’an, 335–345.Google Scholar
Index Terms
- Ukwikora: continuous inspection for keyword-driven testing
Recommendations
The secret life of test smells - an empirical study on test smell evolution and maintenance
AbstractIn recent years, researchers and practitioners have been studying the impact of test smells on test maintenance. However, there is still limited empirical evidence on why developers remove test smells in software maintenance and the mechanism ...
How Students Unit Test: Perceptions, Practices, and Pitfalls
ITiCSE '21: Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1Unit testing is reported as one of the skills that graduating students lack, yet it is an essential skill for professional software developers. Understanding the challenges students face during testing can help inform practices for software testing ...
Unit Test Smells and Accuracy of Software Engineering Student Test Suites
ITiCSE '21: Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1With an increasing emphasis on unit testing in computer science curricula, we examined students' work on testing assignments to investigate their adoption of test smells---practices that indicate potential problems in unit tests. We discovered three ...
Comments