Abstract
Companies are facing constant pressure towards shorter release cycles while still maintaining a high level of quality. Agile development, continuous integration and testing are commonly used quality assurance techniques applied in industry. Increasing the level of test automation is a key ingredient to address the short release cycles. Testing at the graphical user interface (GUI) level is challenging to automate, and therefore many companies still do this manually. To help find solutions for better GUI test automation, academics are researching scriptless GUI testing to complement the script-based approach. In order to better match industrial problems with academic results, more academia-industry collaborations for case-based evaluations are needed. This paper describes such an initiative to improve, transfer and integrate an academic scriptless GUI testing tool TESTAR into the CI pipeline of a Spanish company Prodevelop. The paper describes the steps taken, the outcome, the challenges, and some lessons learned for successful industry-academia collaboration.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Aho, P., Vos, T.: Challenges in automated testing through graphical user interface. In: 2018 IEEE International Conference on Software Testing. Verification and Validation Workshops (ICSTW), pp. 118–121. IEEE Computer Society, Los Alamitos, April 2018
Aho, P., Vos, T.E.J., Ahonen, S., Piirainen, T., Moilanen, P., Ricos, F.P.: Continuous piloting of an open source test automation tool in an industrial environment. Jornadas de Ingeniería del Software y Bases de Datos (JISBD) 1–4 (2019)
Bauersfeld, S., de Rojas, A., Vos, T.E.J.: Evaluating rogue user testing in industry: an experience report. In: 2014 IEEE Eighth International Conference on Research Challenges in Information Science (RCIS), pp. 1–10, May 2014
Bauersfeld, S., Vos, T.E.J., Condori-Fernández, N., Bagnato, A., Brosse, E.: Evaluating the TESTAR tool in an industrial case study. In: 2014 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, ESEM 2014, Torino, Italy, 18–19 September 2014, p. 4 (2014)
Beckman, K., Coulter, N., Khajenoori, S., Mead, N.R.: Collaborations: closing the industry-academia gap. IEEE Softw. 14(6), 49–57 (1997)
Coppola, R., Ardito, L., Torchiano, M.: Fragility of layout-based and visual GUI test scripts: an assessment study on a hybrid mobile application. In: Proceedings of the 10th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation, A-TEST 2019, pp. 28–34. ACM, New York (2019)
Chahim, H., Duran, M., Vos, T.E.J., Aho, P., Condori Fernandez, N.: Scriptless testing at the GUI level in an industrial setting. In: Dalpiaz, F., Zdravkovic, J., Loucopoulos, P. (eds.) RCIS 2020. LNBIP, vol. 385, pp. 267–284. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-50316-1_16
Fowler, M.: Continuous integration (2006). https://www.martinfowler.com/articles/continuousIntegration.html. Accessed 12 Dec 2019
Martinez, M., Esparcia, A.I., Rueda, U., Vos, T.E.J., Ortega, C.: Automated localisation testing in industry with test\(^*\). In: Wotawa, F., Nica, M., Kushik, N. (eds.) ICTSS 2016. LNCS, vol. 9976, pp. 241–248. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-47443-4_17
Meyer, M.: Continuous integration and its tools. Softw. IEEE 31, 14–16 (2014)
O’Connor, R.V., Elger, P., Clarke, P.M.: Continuous software engineering: a microservices architecture perspective. J. Softw.: Evol. Process. 29(11), e1866 (2017)
Rovegard, P., et al.: The success factors powering industry-academia collaboration. IEEE Softw. 29(02), 67–73 (2012)
Sandberg, A., Pareto, L., Arts, T.: Agile collaborative research: action principles for industry-academia collaboration. IEEE Softw. 28(4), 74–83 (2011)
Vos, T.E.J., Kruse, P.M., Condori-Fernández, N., Bauersfeld, S., Wegener, J.: TESTAR: tool support for test automation at the user interface level. Int. J. Inf. Syst. Model. Des. 6(3), 46–83 (2015)
Wieringa, R., Daneva, M.: Six strategies for generalizing software engineering theories. Sci. Comput. Program. 101, 136–152 (2015). Towards general theories of software engineering
Acknowledgment
This work has been funded through the ITEA3 TESTOMAT project (www.testomatproject.eu), the EU H2020 DECODER project (www.decoder-project.eu), the EU H2020 iv4XR project (iv4xr-project.eu) and the ITEA3 IVVES project (ivves.weebly.com).
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Ricós, F.P., Aho, P., Vos, T., Boigues, I.T., Blasco, E.C., Martínez, H.M. (2020). Deploying TESTAR to Enable Remote Testing in an Industrial CI Pipeline: A Case-Based Evaluation. In: Margaria, T., Steffen, B. (eds) Leveraging Applications of Formal Methods, Verification and Validation: Verification Principles. ISoLA 2020. Lecture Notes in Computer Science(), vol 12476. Springer, Cham. https://doi.org/10.1007/978-3-030-61362-4_31
Download citation
DOI: https://doi.org/10.1007/978-3-030-61362-4_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-61361-7
Online ISBN: 978-3-030-61362-4
eBook Packages: Computer ScienceComputer Science (R0)