research-article

An Empirical Study of Meta- and Hyper-Heuristic Search for Multi-Objective Release Planning

Authors:
Yuanyuan Zhang

CREST, University College London, Gower Street, London, UK

CREST, University College London, Gower Street, London, UK
View Profile

,
Mark Harman

CREST, University College London, Gower Street, London, UK

CREST, University College London, Gower Street, London, UK
View Profile

,
Gabriela Ochoa

University of Stirling, Stirling, UK

University of Stirling, Stirling, UK
View Profile

,
Guenther Ruhe

University of Calgary, Alberta, Canada

University of Calgary, Alberta, Canada
View Profile

,
Sjaak Brinkkemper

Utrecht University, CC Utrecht, The Netherlands

Utrecht University, CC Utrecht, The Netherlands
View Profile

ACM Transactions on Software Engineering and Methodology Volume 27 Issue 1Article No.: 3pp 1–32https://doi.org/10.1145/3196831

Published:05 June 2018Publication History

ACM Transactions on Software Engineering and Methodology

Abstract

A variety of meta-heuristic search algorithms have been introduced for optimising software release planning. However, there has been no comprehensive empirical study of different search algorithms across multiple different real-world datasets. In this article, we present an empirical study of global, local, and hybrid meta- and hyper-heuristic search-based algorithms on 10 real-world datasets. We find that the hyper-heuristics are particularly effective. For example, the hyper-heuristic genetic algorithm significantly outperformed the other six approaches (and with high effect size) for solution quality 85% of the time, and was also faster than all others 70% of the time. Furthermore, correlation analysis reveals that it scales well as the number of requirements increases.

References

Philip Achimugu, Ali Selamat, Roliana Ibrahim, and Mohd Naz’ri Mahrin. 2014. A systematic literature review of software requirements prioritization research. Inform. Softw. Technol. 56, 6 (June 2014), 568--585. Google ScholarDigital Library
Ahmed Al-Emran, Dietmar Pfahl, and Günther Ruhe. 2010. A Hybrid Method for Advanced Decision Support in Strategic Product Release Planning. Technical Report 088/2010. University of Calgary.Google Scholar
Thamer AlBourae, Günther Ruhe, and Mahmood Moussavi. 2006. Lightweight replanning of software product releases. In Proceedings of the 1st International Workshop on Software Product Management (IWSPM’06). IEEE, Los Alamitos, CA, 27--34. Google ScholarDigital Library
A. Amandeep, Günther Ruhe, and Mark Stanford. 2004. Intelligent support for software release planning. In Product Focused Software Process Improvement. Lecture Notes in Computer Science, Vol. 3009. Springer, 248--262.Google Scholar
Allysson Allex Araújo, Matheus Paixao, Italo Yeltsin, Altino Dantas, and Jerffeson Souza. 2017. An architecture based on interactive optimization and machine learning applied to the next release problem. Autom. Softw. Eng. 24, 3 (September 2017), 623--671. Google ScholarDigital Library
Andrea Arcuri and Lionel Briand. 2011. A practical guide for using statistical tests to assess randomized algorithms in software engineering. In Proceedings of the 33rd International Conference on Software Engineering (ICSE’11). ACM, New York, NY, 1--10. Google ScholarDigital Library
A. J. Bagnall, V. J. Rayward-Smith, and I. M. Whittley. 2001. The next release problem. Inform. Softw. Technol. 43, 14 (December 2001), 883--890.Google ScholarCross Ref
Yoav Bejamini and Yosef Hochberg. 1995. Controlling the false discovery rate: A practical and powerful approach to multiple testing. J. Royal Stat. Soc. (Series B) 57, 1 (1995), 289--300.Google Scholar
Marcia Maria Albuquerque Brasil, Thiago Gomes Nepomuceno da Silva, Fabricio Gomes de Freitas, Jerffeson Teixeira de Souza, and Mariela Ines Cortes. 2012. A multiobjective optimization approach to the software release planning with undefined number of releases and interdependent requirements. In Enterprise Information Systems, Vol. 102. Springer, 300--314.Google ScholarCross Ref
Edmund Burke and Graham Kendall. 2005. Search Methodologies. Introductory Tutorials in Optimization and Decision Support Techniques. Springer. Google ScholarDigital Library
Edmund K. Burke, Michel Gendreau, Matthew Hyde, Graham Kendall, Ender Özcan Gabriela Ochoa, and Rong Qu. 2013. Hyper-heuristics: A survey of the state of the art. J. Op. Res. Soc. 64, 12 (2013), 1695--1724.Google ScholarCross Ref
Edmund K. Burke, Matthew Hyde, Graham Kendall, Gabriela Ochoa, Ender Özcan, and John R. Woodward. 2010. A classification of hyper-heuristic approaches. In Handbook of Metaheuristics. International Series in Operations Research 8 Management Science, Vol. 146. Springer, 449--468.Google Scholar
Edmund K. Burke, J. Dario Landa Silva, and Eric Soubeiga. 2005. Multi-objective hyper-heuristic approaches for space allocation and timetabling. In Metaheuristics: Progress as Real Problem Solvers, T. Ibaraki, K. Nonobe, and M. Yagiura (Eds.). Operations Research/Computer Science Interfaces Series, Vol. 32. Springer, 129--158..Google Scholar
Xinye Cai and Ou Wei. 2013. A hybrid of decomposition and domination based evolutionary algorithm for multi-objective software next release problem. In Proceedings of the 10th IEEE International Conference on Control and Automation (ICCA’13).Google ScholarCross Ref
Xinye Cai, Ou Wei, and Zhiqiu Huang. 2012. Evolutionary approaches for multi-objective next release problem. Comput. Inform. 31 (2012), 847--875.Google Scholar
José M. Chaves-González and Miguel A. Pérez-Toledano. 2015. Differential evolution with Pareto tournament for the multi-objective next release problem. Appl. Math. Comput. 252 (February 2015), 1--13. Google ScholarDigital Library
Norman Cliff. 1996. Ordinal Methods for Behavioral Data Analysis. Lawrence Erlbaum Associates, Mahwah, NJ.Google Scholar
Felipe Colares, Jerffeson Teixeira de Souza, Rafael Augusto Ferreira do Carmo, Clarindo Pádua, and Geraldo Robson Mateus. 2009. A new approach to the software release planning. In Proceedings of the 23rd Brazilian Symposium on Software Engineering (SBES’09). IEEE, Los Alamitos, CA, 207--215. Google ScholarDigital Library
Matej Crepinsek, Shih-Hsi Liu, and Marjan Mernik. 2013. Exploration and exploitation in evolutionary algorithms: A survey. Comput. Surv. 45, 3 (June 2013), 35:1--35:33. Google ScholarDigital Library
Jerffeson Teixeira de Souza, Camila Loiola Brito Maia, Thiago Ferreira, Rafael Augusto Ferreira do Carmo, and Marcia Brasil. 2011. An ant colony optimization approach to the software release planning with dependent requirements. In Search Based Software Engineering. Lecture Notes in Computer Science, Vol. 6956. Springer, 142--157. Google ScholarDigital Library
Kalyanmony Deb. 2001. Multi-Objective Optimization Using Evolutionary Algorithms. John Wiley 8 Sons. Google ScholarDigital Library
Isabel María del Águila and José Del Sagrado. 2016. Three steps multiobjective decision process for software release planning. Complexity 21, S1 (September/October 2016), 250--262.Google ScholarCross Ref
José Del Sagrado and Isabel María Del Águila. 2009. Ant colony optimization for requirement selection in incremental software development. In Proceedings of the 1st International Symposium on Search Based Software Engineering (SSBSE’09). IEEE, Los Alamitos, CA.Google Scholar
José Del Sagrado, Isabel María Del Águila, and Francisco Javier Orellana. 2010. Ant colony optimization for the next release problem—a comparative study. In Proceedings of the 2nd International Symposium on Search Based Software Engineering (SSBSE’10). IEEE, Los Alamitos, CA, 67--76. Google ScholarDigital Library
José del Sagrado, Isabel María del Águila, and Francisco Javier Orellana. 2015. Multi-objective ant colony optimization for requirements selection. Empir. Softw. Eng. 20, 3 (June 2015), 577--610. Google ScholarDigital Library
Thiago do Nascimento Ferreira and Jerffeson Teixeira de Souza. 2012. An ACO approach for the next release problem with dependency among requirements. In Proceedings of the 3rd Brazilian Workshop on Search-Based Software Engineering (WESB’12).Google Scholar
Olive Jean Dunn. 1961. Multiple comparisons among means. J. Amer. Statist. Assoc. 56, 293, 52--64.Google ScholarCross Ref
Juan J. Durillo, Yuanyuan Zhang, Enrique Alba, Mark Harman, and Antonio J. Nebro. 2011. A study of the bi-objective next release problem. Empir. Softw. Eng. 16, 1 (February 2011), 29--60. Google ScholarDigital Library
Juan J. Durillo, Yuanyuan Zhang, Enrique Alba, and Antonio J. Nebro. 2009. A study of the multi-objective next release problem. In Proceedings of the 1st International Symposium on Search Based Software Engineering (SSBSE’09). IEEE, Los Alamitos, CA, 49--58. Google ScholarDigital Library
Martin S. Feather, Steven L. Cornford, James D. Kiper, and Tim Menzies. 2006. Experiences using visualization techniques to present requirements, risks to them, and options for risk mitigation. In Proceedings of the International Workshop on Requirements Engineering Visualization (REV’06). IEEE, Los Alamitos, CA, 10. Google ScholarDigital Library
Martin S. Feather, James D. Kiper, and Selcuk Kalafat. 2004. Combining heuristic search, visualization and data mining for exploration of system design space. In The International Council on Systems Engineering (INCOSE’04)—Proceedings of the 14th Annual International Symposium.Google Scholar
Martin S. Feather and Tim Menzies. 2002. Converging on the optimal attainment of requirements. In Proceedings of the 10th IEEE International Conference on Requirements Engineering (RE’02). IEEE, Los Alamitos, CA, 263--270. Google ScholarDigital Library
George Andrew Ferguson. 1965. Nonparametric Trend Analysis: A Practical Guide for Research Workers. McGill University Press, Montréal, Canada.Google Scholar
A. Fialho, L. Da Costa, M. Schoenauer, and M. Sebag. 2008. Extreme value based adaptive operator selection. In Parallel Problem Solving from Nature PPSN X. Lecture Notes in Computer Science, Vol. 5199. Springer, 175--184.Google Scholar
Alvaro Fialho, Luis Da Costa, Marc Schoenauer, and Michèle Sebag. 2010. Analyzing bandit-based adaptive operator selection mechanisms. Ann. Math. Artif. Intell. 60, 1-2 (2010), 25--64. Google ScholarDigital Library
Anthony Finkelstein, Mark Harman, S. Afshin Mansouri, Jian Ren, and Yuanyuan Zhang. 2008. “Fairness analysis” in requirements assignments. In Proceedings of the 16th IEEE International Requirements Engineering Conference (RE’08). IEEE, Los Alamitos, CA, 115--124. Google ScholarDigital Library
Anthony Finkelstein, Mark Harman, S. Afshin Mansouri, Jian Ren, and Yuanyuan Zhang. 2009. A search based approach to fairness analysis in requirement assignments to aid negotiation, mediation and decision making. Requir. Eng. J. 14, 4 (December 2009), 231--245. Google ScholarDigital Library
Wei Fu, Tim Menzies, and Xipeng Shen. 2016. Tuning for software analytics: Is it really necessary? Inform. Softw. Technol. 76 (August 2016), 135--146. Google ScholarDigital Library
Francis Galton. 1889. Natural Inheritance. Macmillan and Co., London, UK.Google Scholar
Tom Gilb. 2005. Competitive Engineering: A Handbook for Systems Engineering, Requirements Engineering, and Software Engineering Using Planguage. Butterworth-Heinemann. Google ScholarDigital Library
Mark Harman, Edmund Burke, John A. Clark, and Xin Yao. 2012a. Dynamic adaptive search based software engineering. In Proceedings of the 6th IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM’12). 1--8. Google ScholarDigital Library
Mark Harman and Bryan F. Jones. 2001. Search-based software engineering. Inform. and Softw. Technol. 43, 14 (December 2001), 833--839.Google ScholarCross Ref
Mark Harman, S. Afshin Mansouri, and Yuanyuan Zhang. 2012b. Search-based software engineering: Trends, techniques and applications. ACM Comput. Surv. 45, 1 (November 2012), Article 11. Google ScholarDigital Library
Mark Harman, Phil McMinn, Jerffeson Teixeira de Souza, and Shin Yoo. 2012c. Search based software engineering: Techniques, taxonomy, tutorial. In Empirical Software Engineering and Verification: LASER 2009-2010. Lecture Notes in Computer Science, Vol. 7007. Springer, 1--59. Google ScholarDigital Library
Yosef Hochberg. 1988. A sharper Bonferroni procedure for multiple tests of significance. Biometrika 75, 4 (1988), 800--802.Google ScholarCross Ref
John H. Holland. 1975. Adaption in Natural and Artificial Systems. MIT Press, Ann Arbor, MI.Google Scholar
He Jiang, Jifeng Xuan, and Zhilei Ren. 2010a. Approximate backbone based multilevel algorithm for next release problem. In Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation (GECCO’10). ACM, New York, NY, 1333--1340. Google ScholarDigital Library
He Jiang, Jingyuan Zhang, Jifeng Xuan, Zhilei Re, and Yan Hu. 2010b. A hybrid ACO algorithm for the next release problem. In Proceedings of the 2nd International Conference on Software Engineering and Data Mining (SEDM’10). IEEE, Los Alamitos, CA, 166--171.Google Scholar
Muhammad Rezaul Karim and Guenther Ruhe. 2014. Bi-objective genetic search for release planning in support of themes. In Search-Based Software Engineering.Lecture Notes in Computer Science, Vol. 8636. Springer, 123--137.Google Scholar
J. Karlsson and K. Ryan. 1997. A cost-value approach for prioritizing requirements. IEEE Softw. 14, 5 (1997), 67--74. Google ScholarDigital Library
Maurice Kendall. 1948. Rank Correlation Methods. Charles Griffin 8 Company Ltd., London, UK.Google Scholar
William Henry Kruskal and Wilson Allen Wallis. 1952. Use of ranks in one-criterion variance analysis. J. Am. Statist. Assoc. 47, 260 (1952), 583--621.Google ScholarCross Ref
A. Charan Kumari, K. Srinivas, and M. P. Gupta. 2012. Software requirements selection using quantum-inspired elitist multi-objective evolutionary algorithm. In Proceedings of International Conference on Advances in Engineering, Science and Management (ICAESM’12). IEEE, Los Alamitos, CA, 782--787.Google Scholar
Chen Li, Marjan Van den Akker, Sjaak Brinkkemper, and Guido Diepen. 2010. An integrated approach for requirement selection and scheduling in software release planning. Requir. Eng. 15, 4 (November 2010), 375--396. Google ScholarDigital Library
Lingbo Li, Mark Harman, Emmanuel Letier, and Yuanyuan Zhang. 2014. Robust next release problem: Handling uncertainty during optimization. In Proceedings of the 2014 Conference on Genetic and Evolutionary Computation (GECCO’14). ACM, New York, NY, 1247--1254. Google ScholarDigital Library
Yan Li, Tao Yue, Shaukat Ali, and Li Zhang. 2017. Zen-reqoptimizer: A search-based approach for requirements assignment optimization. Emp. Softw. Eng. 22, 1 (February 2017), 175--234. Google ScholarDigital Library
Soo Ling Lim. 2010. Social Networks and Collaborative Filtering for Large-Scale Requirements Elicitation. Ph.D. Dissertation. School of Computer Science and Engineering, University of New South Wales.Google Scholar
Henry Berthold Mann and Donald Ransom Whitney. 1947. On a test of whether one of two random variables is stochastically larger than the other. Ann. Math. Stat. 18, 1 (1947), 50--60.Google ScholarCross Ref
Kent McClymont and Edward C. Keedwell. 2011. Markov chain hyper-heuristic (MCHH): An online selective hyper-heuristic for multi-objective continuous problems. In Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation (GECCO’11). ACM, New York, NY, 2003--2010. Google ScholarDigital Library
Gabriela Ochoa, Matthew Hyde, Tim Curtois, Jose A. Vazquez-Rodriguez, James Walker, Michel Gendreau, Graham Kendall, Barry McCollum, Andrew J. Parkes, Sanja Petrovic, and Edmund K. Burke. 2012. HyFlex: A benchmark framework for cross-domain heuristic search. In Evolutionary Computation in Combinatorial Optimization. Lecture Notes in Computer Science, Vol. 7245. Springer, 136--147. Google ScholarDigital Library
Matheus Henrique Esteves Paixão and Jerffeson Teixeira de Souza. 2013a. A recoverable robust approach for the next release problem. In Search Based Software Engineering., Lecture Notes in Computer Science, Vol. 8084. Springer, 172--187. Google ScholarDigital Library
Matheus Henrique Esteves Paixão and Jerffeson Teixeira de Souza. 2013b. A scenario-based robust model for the next release problem. In Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation (GECCO’13). ACM, New York, NY, 1469--1476. Google ScholarDigital Library
Matheus Henrique Esteves Paixão and Jerffeson Teixeira de Souza. 2015. A robust optimization approach to the next release problem in the presence of uncertainties. J. Syst. Softw. 103 (May 2015), 281--295. Google ScholarDigital Library
Karl Pearson. 1895. Notes on regression and inheritance in the case of two parents. Proc. R. Soc. Lond. 58 (June 1895), 240--242.Google Scholar
D. Pisinger and S. Ropke. 2007. A general heuristic for vehicle routing problems. Comput. Op. Res. 34 (2007), 2403--2435. Google ScholarDigital Library
A. M. Pitangueira, P. Tonella, A. Susi, R. S. P. Maciel, and M. Barros. 2017. Minimizing the stakeholder dissatisfaction risk in requirement selection for next release problem. Inform. Softw. Technol. 87 (July 2017), 104--118. Google ScholarDigital Library
Antônio Mauricio Pitangueira, Rita Suzana P. Maciel, and Márcio de Oliveira Barros. 2015. Software requirements selection and prioritization using SBSE approaches: A systematic review and mapping of the literature. J. Syst. Softw. 103 (May 2015), 267--280. Google ScholarDigital Library
Antonio Mauricio Pitangueira, Paolo Tonella, Angelo Susi, Rita Suzana Maciel, and Marcio Barros. 2016. Risk-aware multi-stakeholder next release planning using multi-objective optimization. In Proceedings of International Working Conference on Requirements Engineering: Foundation for Software Quality (REFSQ’16). 3--18.Google ScholarDigital Library
Riccardo Poli, William B. Langdon, and Nicholas Freitag McPhee. 2008. A Field Guide to Genetic Programming. Published via http://lulu.com and freely available at http://www.gp-field-guide.org.uk (with contributions by J. R. Koza). Google ScholarDigital Library
Outi Räihä. 2010. A survey on search-based software design. Comput. Sci. Rev. 4, 4 (2010), 203--249. Google ScholarDigital Library
Günther Ruhe. 2010. Product Release Planning: Methods, Tools and Applications. CRC Press, Boca Raton, FL.Google Scholar
Günther Ruhe and Des Greer. 2003. Quantitative studies in software release planning under risk and resource constraints. In Proceedings of the International Symposium on Empirical Software Engineering (ISESE’03). IEEE, Los Alamitos, CA, 262--270. Google ScholarDigital Library
Günther Ruhe and An Ngo-The. 2004. Hybrid intelligence in software release planning. Int. J. Hybrid Intell. Sys. 1, 1–2 (April 2004), 99--110. Google ScholarDigital Library
Günther Ruhe and Moshood Omolade Saliu. 2005. The art and science of software release planning. IEEE Softw. 22, 6 (November 2005), 47--53. Google ScholarDigital Library
Moshood Omolade Saliu and Günther Ruhe. 2007. Bi-objective release planning for evolving software systems. In Proceedings of the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering. ACM, New York, NY, 105--114. Google ScholarDigital Library
Omolade Saliu and Günther Ruhe. 2005. Supporting software release planning decisions for evolving systems. In Proceedings of the 29th Annual IEEE/NASA on Software Engineering Workshop (SEW’05). IEEE, Los Alamitos, CA, 14--26. Google ScholarDigital Library
Neil J. Salkind. 2007. Encyclopaedia of Measurement and Statistics. SAGE.Google Scholar
Martin J. Shepperd. 1995. Foundations of Software Measurement. Prentice Hall. Google ScholarDigital Library
Charles Edward Spearman. 1904. The proof and measurement of association between two things. Am. J. Psychol. 15, 1 (January 1904), 72--101.Google ScholarCross Ref
Mikael Svahnberg, Tony Gorschek, Robert Feldt, Richard Torkar, Saad Bin Saleem, and Muhammad Usman Shafique. 2010. A systematic review on strategic release planning models. Inform. Softw. Technol. 52, 3 (March 2010), 237--248. Google ScholarDigital Library
Dirk Thierens. 2005. An adaptive pursuit strategy for allocating operator probabilities. In Proceedings of the 2005 Conference on Genetic and Evolutionary Computation (GECCO’05). ACM, New York, NY, 1539--1546. Google ScholarDigital Library
Paolo Tonella, Angelo Susi, and Francis Palma. 2013. Interactive requirements prioritization using a genetic algorithm. Inform. Softw. Technol. 55, 1 (January 2013), 173--187. Google ScholarDigital Library
Roberto Ugolotti and Stefano Cagnoni. 2014. Analysis of evolutionary algorithms using multi-objective parameter tuning. In Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation (GECCO’14). ACM, New York, NY, 1343--1350. Google ScholarDigital Library
J. M. Van den Akker, S. Brinkkemper, G. Diepen, and J. Versendaal. 2005a. Determination of the next release of a software product: An approach using integer linear programming. In Proceedings of the CAiSE’05 FORUM. 119--124.Google Scholar
Marjan Van den Akker, Sjaak Brinkkemper, Guido Diepen, and Johan Versendaal. 2005b. Flexible release planning using integer linear programming. In Proceeding of the 11th International Workshop on Requirements Engineering: Foundation for Software Quality (REFSQ’05). 247--262.Google Scholar
Marjan Van den Akker, Sjaak Brinkkemper, Guido Diepen, and Johan Versendaal. 2008. Software product release planning through optimization and what-if analysis. Inform. Softw. Technol. 50, 1–2 (January 2008), 101--111. Google ScholarDigital Library
P. J. M. van Laarhoven and E. H. L. Aarts. 1987. Simulated Annealing: Theory and Practice. Kluwer Academic Publishers, Dordrecht, Netherlands. Google ScholarDigital Library
Nadarajen Veerapen, Gabriela Ochoa, Mark Harman, and Edmund K. Burke. 2015. An integer linear programming approach to the single and bi-objective next release problem. Inform. Softw. Technol. 65 (September 2015), 1--13. Google ScholarDigital Library
Kevin Vlaanderen, Slinger Jansen, Sjaak Brinkkemper, and Erik Jaspers. 2011. The agile requirements refinery: Applying SCRUM principles to software product management. Inform. Softw. Technol. 53, 1 (2011), 58--70. Google ScholarDigital Library
Frank Wilcoxon. 1945. Individual comparisons by ranking methods. Biom. Bull. 1, 6 (1945), 80--83.Google ScholarCross Ref
Jifeng Xuan, He Jiang, Zhilei Ren, and Zhongxuan Luo. 2012. Solving the large scale next release problem with a backbone based multilevel algorithm. IEEE Trans. Softw. Eng. 38, 5 (September/October 2012), 1195--1212. Google ScholarDigital Library
Yuanyuan Zhang, Enrique Alba, Juan J. Durillo, Sigrid Eldh, and Mark Harman. 2010. Today/future importance analysis. In Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation (GECCO’10). ACM, New York, NY, 1357--1364. Google ScholarDigital Library
Yuanyuan Zhang and Mark Harman. 2010. Search based optimization of requirements interaction management. In Proceedings of the 2nd International Symposium on Search Based Software Engineering (SSBSE’10). IEEE, Los Alamitos, CA, 47--56. Google ScholarDigital Library
Yuanyuan Zhang, Mark Harman, Anthony Finkelstein, and S. Afshin Mansouri. 2011. Comparing the performance of metaheuristics for the analysis of multi-stakeholder tradeoffs in requirements optimisation. Inform. Softw. Technol. 53, 7 (July 2011), 761--773. Google ScholarDigital Library
Yuanyuan Zhang, Mark Harman, and Soo Ling Lim. 2013. Empirical evaluation of search based requirements interaction management. Inform. Softw. Technol. 55, 1 (January 2013), 126--152. Google ScholarDigital Library
Yuanyuan Zhang, Mark Harman, and S. Afshin Mansouri. 2007. The multi-objective next release problem. In Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation (GECCO’07). ACM, New York, NY, 1129--1137. Google ScholarDigital Library
Donald W. Zimmerman. 2000. Statistical significance levels of nonparametric tests biased by heterogeneous variances of treatment groups. J. Gen. Psychol. 127, 4 (October 2000), 354--364.Google ScholarCross Ref
E. Zitzler and L. Thiele. 1999. Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach. IEEE Trans. Evolut. Comput. 3, 4 (November 1999), 257--271. Google ScholarDigital Library

Index Terms

An Empirical Study of Meta- and Hyper-Heuristic Search for Multi-Objective Release Planning
1. Computing methodologies
  1. Artificial intelligence
    1. Search methodologies
      1. Heuristic function construction

Index terms have been assigned to the content through auto-classification.

Recommendations

A Hyper-Heuristic Using GRASP with Path-Relinking: A Case Study of the Nurse Rostering Problem

The goal of hyper-heuristics is to design and choose heuristics to solve complex problems. The primary motivation behind the hyper-heuristics is to generalize the solving ability of the heuristics. In this paper, the authors propose a Hyper-heuristic ...
Read More
A cooperative hyper-heuristic search framework

In this paper, we aim to investigate the role of cooperation between low level heuristics within a hyper-heuristic framework. Since different low level heuristics have different strengths and weaknesses, we believe that cooperation can allow the ...
Read More
An empirical study into the structure of heuristic combinations in an evolutionary algorithm hyper-heuristic for the examination timetabling problem
SAICSIT '10: Proceedings of the 2010 Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists

A hyper-heuristic for the examination timetabling problem searches a space of constructive heuristic combinations instead of a space of examination timetables. The most optimal heuristic combination found by the search is used to construct the ...
Read More

Comments

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Published in
ACM Transactions on Software Engineering and Methodology Volume 27, Issue 1
January 2018
167 pages
ISSN:1049-331X
EISSN:1557-7392
DOI:10.1145/3208361
Editor:
David S. Rosenblum
National University of Singapore, Singapore
Issue’s Table of Contents
Copyright © 2018 ACM
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
- Published: 5 June 2018
- Accepted: 1 March 2018
- Revised: 1 February 2018
- Received: 1 September 2015
Published in tosem Volume 27, Issue 1

Permissions
Request permissions about this article.
Request Permissions

Check for updates
Author Tags
Strategic release planning
hyper-heuristics
meta-heuristics
Qualifiers
- research-article
- Research
- Refereed
Conference
Funding Sources
Other Metrics
View Article Metrics

Article Metrics
- 27
  Total Citations
  View Citations
- 454
  Total Downloads
- Downloads (Last 12 months)26
- Downloads (Last 6 weeks)2
Other Metrics
View Author Metrics
Cited By
View all

PDF Format

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

An Empirical Study of Meta- and Hyper-Heuristic Search for Multi-Objective Release Planning

ACM Transactions on Software Engineering and Methodology

Abstract

References

Cited By

Index Terms

Recommendations

A Hyper-Heuristic Using GRASP with Path-Relinking: A Case Study of the Nurse Rostering Problem

A cooperative hyper-heuristic search framework

An empirical study into the structure of heuristic combinations in an evolutionary algorithm hyper-heuristic for the examination timetabling problem