Helson L. Jakubovski Filho
ABSTRACT
The selection of products for the variability testing of Feature Models (FMs) is a complex task impacted by many factors. To solve this problem, Multi-Objective Evolutionary Algorithms (MOEAs) have been successfully used in the field known as Search-Based Software Engineering (SBSE). However, the design of a search-based approach is not an easy task for the software engineer, who can find some difficulties such as: the choice and configuration of the best MOEAs, the choice of the best search operators to be implemented, and so on. In addition to this, existing approaches are dependent on the problem domain and do not allow reuse. In this way the use of Hyper-Heuristic (HH) can help to obtain more generic and reusable search-based approaches, and because of this is considered a trend in the SBSE field. Following this trend and to contribute to reduce the software engineer's efforts, this work explores the use of a hyper-heuristic for automatic generation of MOEAs to select test products from the FM, considering three factors: pairwise coverage, mutation score and cost, given by the number of products. The HH is based on a grammar that represents the elements, parameters and components of existing MOEAs and implements evolutionary operators, such as crossover and mutation, suitable for selection problems. In this way, it can be reused for other similar software engineering problems. Evaluation results show that the proposed approach obtains results that are better or statistically equivalent than similar approaches found in the literature.
- P. Arcaini, A. Gargantini, and P. Vavassori. 2015. Generating Tests for Detecting Faults in Feature Models. In Proceedings of the 8th International Conference on Software Testing, Verification and Validation (ICST'15). IEEE, Graz, Austria, 1--10.Google Scholar
- M. P. Basgalupp, R. C. Barros, T. S. da Silva, and A. C. P. L. F. de Carvalho. 2013. Software Effort Prediction: A Hyper-heuristic Decision-tree Based Approach. In Proceedings of the 28th Annual Symposium on Applied Computing (SAC '13). ACM, Coimbra, Portugal, 1109--1116. Google ScholarDigital Library
- D. Benavides, S. Segura, and A. Ruiz-Cortés. 2010. Automated analysis of feature models 20 years later: A literature review. Information Systems 35, 6 (2010), 615--636. Google ScholarDigital Library
- E. K. Burke, M. Gendreau, M. Hyde, G. Kendall, G. Ochoa, E. Özcan, and R. Qu. 2013. Hyper-heuristics: a survey of the state of the art. Journal of the Operational Research Society 64, 12 (Dec. 2013), 1695.Google ScholarCross Ref
- E. K. Burke, M. Hyde, G. Kendall, G. Ochoa, E. Özcan, and J. R. Woodward. 2010. A Classification of Hyper-heuristic Approaches. In Handbook of Metaheuristics. Springer, Boston, MA, 449--468.Google Scholar
- D. M. Cohen, S. R. Dalai, J. Parelius, and G. C. Patton. 1996. The combinatorial design approach to automatic test generation. IEEE Software 13, 5 (1996), 83--88. Google ScholarDigital Library
- M. B. Cohen, M. B. Dwyer, and J. Shi. 2006. Coverage and Adequacy in Software Product Line Testing. In Proceedings of the Workshop on Role of Software Architecture for Testing and Analysis (ROSATEA'06). ACM, Portland, USA, 53--63. Google ScholarDigital Library
- K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation 6, 2 (2002), 182--197. Google ScholarDigital Library
- F. Ensan, E. Bagheri, and D. Gašević. 2012. Evolutionary Search-based Test Generation for Software Product Line Feature Models. In Proceedings of the 24th International Conference on Advanced Information Systems Engineering (CAiSE'12). Springer, Gdansk, Poland, 613--628. Google ScholarDigital Library
- J. M. Ferreira, S. R. Vergilio, and M. A. Quináia. 2013. A Mutation Approach to Feature Testing of Software Product Lines. In Proceedings of the 25th International Conference on Software Engineering and Knowledge Engineering (SEKE' 13). Knowledge Systems Institute Graduate School, Boston, MA, USA, 232--237.Google Scholar
- T. N. Ferreira, J. N. Kuk, A. T. R. Pozo, and S. R. Vergilio. 2016. Product Selection Based on Upper Confidence Bound MOEA/D-DRA for Testing Software Product Lines. In Proceedings of the Congress on Evolutionary Computation (CEC '16). IEEE, Vancouver, Canada, 4135--4142.Google Scholar
- T. N. Ferreira, J. A. Prado Lima, A. Strickler, J. N. Kuk, S. R. Vergilio, and A. Pozo. 2017. Hyper-heuristic Based Product Selection for Software Product Line Testing. IEEE Computational Intelligence Magazine 12, 2 (May 2017), 34--45. Google ScholarDigital Library
- Á. Fialho, L. Da Costa, M. Schoenauer, and M. Sebag. 2010. Analyzing bandit-based adaptive operator selection mechanisms. Annals of Mathematics and Artificial Intelligence 60, 1-2 (Oct. 2010), 25--64. Google ScholarDigital Library
- M. Friedman. 1940. A comparison of alternative tests of significance for the problem of m rankings. The Annals of Mathematical Statistics 11, 1 (1940), 86--92.Google ScholarCross Ref
- G. Guizzo, G. M. Fritsche, S. R. Vergilio, and A. T. R. Pozo. 2015. A Hyper-Heuristic for the Multi-Objective Integration and Test Order Problem. In Proceedings of the Annual Conference on Genetic and Evolutionary Computation (GECCO '15). ACM, Madrid, Spain, 1343--1350. Google ScholarDigital Library
- M. Harman, E. Burke, J. Clark, and X. Yao. 2012. Dynamic adaptive Search Based Software Engineering. In Proceedings of the International Symposium on Empirical Software Engineering and Measurement (ESEM'12). ACM, Lund, Sweden, 1--8. Google ScholarDigital Library
- M. Harman, Y. Jia, J. Krinke, W. B. Langdon, J. Petke, and Y. Zhang. 2014. Search Based Software Engineering for Software Product Line Engineering: A Survey and Directions for Future Work. In Proceedings of the 18th International Software Product Line Conference (SPLC'14), Vol. 1. ACM, Florence, Italy, 5--18. Google ScholarDigital Library
- C. Henard, M. Papadakis, M. Harman, and Y. Le Traon. 2015. Combining Multi-Objective Search and Constraint Solving for Configuring Large Software Product Lines. In International Conference on Software Engineering (ICSE), Vol. 1. 517--528. Google ScholarDigital Library
- C. Henard, M. Papadakis, and Y. Le Traon. 2014. Mutation-based generation of software product line test configurations. In International Symposium on Search Based Software Engineering (SSBSE'14). Springer, Fortaleza, Brazil, 92--106.Google Scholar
- C. Henard, M. Papadakis, G. Perrouin, J. Klein, P. Heymans, and Y. Le Traon. 2014. Bypassing the combinatorial explosion: Using similarity to generate and prioritize t-wise test configurations for software product lines. IEEE Transactions on Software Engineering 40, 7 (2014), 650--670. Google ScholarDigital Library
- C. Henard, M. Papadakis, G. Perrouin, J. Klein, and Y. Le Traon. 2013. Assessing software product line testing via model-based mutation: An application to similarity testing. In Proceedings of the 6th International Conference on Software Testing, Verification and Validation (ICSTW'13). IEEE, Luxembourg, Luxembourg, 188--197. Google ScholarDigital Library
- C. Henard, M. Papadakis, G. Perrouin, J. Klein, and Y. Le Traon. 2013. Multi-objective Test Generation for Software Product Lines. In International Software Product Line Conference (SPLC'13). 62--71. Google ScholarDigital Library
- Y. Jia, M. B Cohen, M. Harman, and J. Petke. 2015. Learning combinatorial interaction test generation strategies using hyperheuristic search. In International Conference on Software Engineering (ICSE'15), Vol. 1. 540--550. Google ScholarDigital Library
- K. C. Kang, J. Lee, and P. Donohoe. 2002. Feature-Oriented Project Line Engineering. IEEE Software 19, 4 (July 2002), 58--65. Google ScholarDigital Library
- W. H. Kruskal and W. A. Wallis. 1952. Use of Ranks in One-Criterion Variance Analysis. J. Amer. Statist. Assoc. 47, 260 (Dec. 1952), 583--621.Google ScholarCross Ref
- A. C. Kumari, K. Srinivas, and M. P. Gupta. 2013. Software module clustering using a hyper-heuristic based multi-objective genetic algorithm. In Proceedings of the 3rd International Advance Computing Conference (IACC'13). 813--818.Google Scholar
- H. Lackner and M. Schmidt. 2014. Towards the Assessment of Software Product Line Tests: A Mutation System for Variable Systems. In International Software Product Line Conference (SPLC'14), Vol. 2. 62--69. Google ScholarDigital Library
- B. P. Lamancha and M. P. Usaola. 2010. Testing product generation in software product lines using pairwise for features coverage. In Proceedings of the 22nd IFIP International Conference on Testing Software and Systems (ICTSS'10). Springer, Natal, Brazil, 111--125. Google ScholarDigital Library
- K. Li, A. Fialho, S. Kwong, and Q. Zhang. 2014. Adaptive Operator Selection With Bandits for a Multiobjective Evolutionary Algorithm Based on Decomposition. IEEE Transactions on Evolutionary Computation 18, 1 (Feb. 2014), 114--130.Google ScholarCross Ref
- R. E. Lopez-Herrejon, F. Chicano, J. Ferrer, A. Egyed, and E. Alba. 2013. Multi-objective optimal test suite computation for software product line pairwise testing. In International Conference on Software Maintenance (ICSM'13). 404--407. Google ScholarDigital Library
- T. Mariani, G. Guizzo, S. R. Vergilio, and A. T. R. Pozo. 2016. Grammatical Evolution for the Multi-Objective Integration and Test Order Problem. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO'16). 1069--1076. Google ScholarDigital Library
- R. A. Matnei Filho and S. R. Vergilio. 2015. A Mutation and Multi-objective Test Data Generation Approach for Feature Testing of Software Product Lines. In Proceedings of the 29th Brazilian Symposium on Software Engineering (SBES'15). IEEE Computer Society, Belo Horizonte - MG, Brazil, 21--30. Google ScholarDigital Library
- R. A. Matnei Filho and S. R. Vergilio. 2016. A multi-objective test data generation approach for mutation testing of feature models. Journal of Software Engineering Research and Development 4, 1 (2016).Google ScholarCross Ref
- J. McGregor. 2001. Testing a Software Product Line. Technical Report CMU/SEI-2001-TR-022. Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA.Google Scholar
- S. Oster, M. Zink, M. Lochau, and M. Grechanik. 2011. Pairwise feature-interaction testing for SPLs: potentials and limitations. In Proceedings of the 15th International Software Product Line Conference (SPLC'11), Vol. 2. ACM, Munich, Germany, 1--8. Google ScholarDigital Library
- G. Perrouin, S. Sen, J. Klein, B. Baudry, and Y. le Traon. 2010. Automated and Scalable T-wise Test Case Generation Strategies for Software Product Lines. In Proceedings of the 3rd International Conference on Software Testing, Verification and Validation (ICST'10). IEEE, Paris, France, 459--468. Google ScholarDigital Library
- K. Pohl, G. Böckle, and F.J. van Der Linden. 2005. Software product line engineering: foundations, principles and techniques (1st ed.). Springer Science & Business Media, Secaucus, NJ, USA. Google ScholarDigital Library
- D. Reuling, J. Bürdek, S. Rotärmel, M. Lochau, and U. Kelter. 2015. Fault-based Product-line Testing: Effective Sample Generation Based on Feature-diagram Mutation. In Proceedings of the 19th International Software Product Line Conference (SPLC'15). Nashville, USA, 131--140. Google ScholarDigital Library
- C. Ryan, J. J. Collins, and M. O. Neill. 1998. Grammatical evolution: Evolving programs for an arbitrary language. Springer, Paris, France, 83--96. Google ScholarDigital Library
- N. R. Sabar, M. Ayob, G. Kendall, and R. Qu. 2013. Grammatical Evolution Hyper-Heuristic for Combinatorial Optimization Problems. IEEE Transactions on Evolutionary Computation 17, 6 (Dec. 2013), 840--861. Google ScholarDigital Library
- A. Strickler, J. A. Prado Lima, S. R. Vergilio, and A. T. R. Pozo. 2016. Deriving products for variability test of Feature Models with a hyper-heuristic approach. Applied Soft Computing 49 (Dec. 2016), 1232--1242. Google ScholarDigital Library
- P. Trinidad, D. Benavides, A. Ruiz-Cortés, S. Segura, and A. Jimenez. 2008. FaMa framework. In Proceedings of the 12 International Software Product Line Conference (SPLC'08). IEEE, Limerick, Ireland, 359--359. Google ScholarDigital Library
- E. Uzuncaova, S. Khurshid, and D. Batory. 2010. Incremental test generation for software product lines. IEEE Transactions on Software Engineering 36, 3 (2010), 309--322. Google ScholarDigital Library
- S. Wang, S. Ali, and A. Gotlieb. 2013. Minimizing Test Suites in Software Product Lines Using Weight-based Genetic Algorithms. In Proceedings of the Annual Conference on Genetic and Evolutionary Computation (GECCO'13). ACM, Amsterdam, The Netherlands, 1493--1500. Google ScholarDigital Library
- S. Wang, D. Buchmann, S. Ali, A. Gotlieb, D. Pradhan, and M. Liaaen. 2014. Multi-objective Test Prioritization in Software Product Line Testing: An Industrial Case Study. In International Software Product Line Conference (SPLC'14), Vol. 1. 32--41. Google ScholarDigital Library
- E. Zitzler. 1999. Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications. Ph.D. Dissertation. ETH Zurich, Switzerland.Google Scholar
- E. Zitzler and S. Künzli. 2004. Indicator-Based Selection in Multiobjective Search. In Proceedings of the 8th International Conference on Parallel Problem Solving from Nature (PPSN VIII). Springer, Birmingham, UK, 832--842.Google Scholar
- E. Zitzler, M. Laumanns, and L. Thiele. 2001. SPEA2: Improving the strength Pareto evolutionary algorithm for multiobjective optimization. In Evolutionary Methods for Design Optimization and Control with Applications to Industrial Problems. International Center for Numerical Methods in Engineering, Athens, Greece, 95--100.Google Scholar
Index Terms
- Automatic Generation of Search-Based Algorithms Applied to the Feature Testing of Software Product Lines
Recommendations
Multiple objective test set selection for software product line testing: evaluating different preference-based algorithms
SBES '18: Proceedings of the XXXII Brazilian Symposium on Software EngineeringThe selection of optimal test sets for Software Product Lines (SPLs) is a complex task impacted by many factors and that needs to consider the tester's preferences. To help in this task, Preference-based Evolutionary Multi-objective Algorithms (PEMOAs) ...
Preference based multi-objective algorithms applied to the variability testing of software product lines
Highlights- Preference-Based Evolutionary Multi-objective Algorithms applied to SPL testing.
AbstractEvolutionary Multi-Objective Algorithms (EMOAs) have been applied to derive products for the variability testing of Software Product Lines (SPLs), which is a complex task impacted by many factors, such as the number of products to be ...
Dynamic analysis of shared execution in software product line testing
SPLC '16: Proceedings of the 20th International Systems and Software Product Line ConferenceSoftware product line (SPL), a family-based software development process, has proven to be a more effective technology than single software systems. Testing SPL products individually is redundant for product lines testing. Meanwhile, the complexity of ...
Comments