Skip to main content
SpringerLink
Log in

Adaptive pattern search for large-scale optimization

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The emergence of high-dimensional data requires the design of new optimization methods. Indeed, conventional optimization methods require improvements, hybridization, or parameter tuning in order to operate in spaces of high dimensions. In this paper, we present a new adaptive variant of a pattern search algorithm to solve global optimization problems exhibiting such a character. The proposed method has no parameters visible to the user and the default settings, determined by almost no a priori experimentation, are highly robust on the tested datasets. The algorithm is evaluated and compared with 11 state-of-the-art methods on 20 benchmark functions of 1000 dimensions from the CEC’2010 competition. The results show that this approach obtains good performances compared to the other methods tested.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Olariu S, Zomaya AY (2005) Handbook of bioinspired algorithms and applications. Chapman & Hall/CRC, London

    Book  MATH  Google Scholar 

  2. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of IEEE international conference on neural networks. Perth

  3. Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley Longman Publishing Co., Inc., p 372

  4. Bellman R (1957) Dynamic programming. Princeton University Press, Princeton

    MATH  Google Scholar 

  5. Lee EK (2007) Large-scale optimization-based classification models in medicine and biology. Ann Biomed Eng 35(6):1095–1 1109

    Article  Google Scholar 

  6. Nasiri JA et al (2009) High dimensional problem optimization using distributed multi-agent PSO. In: Third UKSim European symposium on computer modeling and simulation, 2009. EMS ’09

  7. Larranaga P et al (2006) Machine learning in bioinformatics. Brief Bioinform 7(1):86–112

    Article  MathSciNet  Google Scholar 

  8. Levitsky V et al (2007) Effective transcription factor binding site prediction using a combination of optimization, a genetic algorithm and discriminant analysis to capture distant interactions. BMC Bioinform 8(481):1–20

    Google Scholar 

  9. Saeys Y, Inza I, Larrañaga P (2007) A review of feature selection techniques in bioinformatics. Bioinformatics 23(19):2507–2517

    Article  Google Scholar 

  10. Ghalwash MF et al (2016) Structured feature selection using coordinate descent optimization. BMC Bioinform 17:158

    Article  Google Scholar 

  11. Blanco R, Larrañaga P (2001) Selection of highly accurate genes for cancer classification by estimation of distribution algorithms. in: Workshop of Bayesian models in medicine. AIME 2001. 1–4 July. Cascais

  12. Saeys Y et al (2004) Feature selection for splice site prediction: a new method using EDA-based feature ranking. BMC Bioinform 5(64):1–11

    Google Scholar 

  13. Armananzas R et al (2008) A review of estimation of distribution algorithms in bioinformatics. BioData Mining 1(6):1–12

    Google Scholar 

  14. Dittrich M et al (2008) Identifying functional modules in protein-protein interaction networks: an integrated exact approach. Bioinformatics 24(13):I223–I231

    Article  Google Scholar 

  15. Xiao X et al (2003) Gene clustering using self-organizing maps and particle swarm optimization. In: Parallel and distributed processing symposium, 22–26 April. IEEE Computer Society

  16. Maulik U, Bandyopadhyay S, Mukhopadhyay A (2011) Multiobjective genetic algorithms for clustering: applications in data mining and bioinformatics. Springer Science & Business Media

  17. Gardeux V et al (2013) Optimization for feature selection in DNA microarrays. In: Heuristics: theory and applications. Nova Publishers

  18. Handl J, Kell D, Knowles J (2007) Multiobjective optimization in bioinformatics and computational biology. IEEE/ACM Trans Comput Biol Bioinform 4(2):279–292

    Article  Google Scholar 

  19. Shan S, Wang GG (2010) Survey of modeling and optimization strategies to solve high-dimensional design problems with computationally-expensive black-box functions. Struct Multidiscip Optim 41(2):219–241

    Article  MathSciNet  MATH  Google Scholar 

  20. Regis R (2013) An initialization strategy for high-dimensional surrogate-based expensive black-box optimization. In: Zuluaga LF, Terlaky T (eds) Modeling and optimization: theory and applications. Springer, New York, pp 51–85

    Chapter  Google Scholar 

  21. Hvattum LM, Glover F (2009) Finding local optima of high-dimensional functions using direct search methods. Eur J Oper Res 195(1):31–45

    Article  MathSciNet  MATH  Google Scholar 

  22. LaTorre A, Muelas S, Pena JM (2011) A MOS-based dynamic memetic differential evolution algorithm for continuous optimization: a scalability test. Soft Comput 15(11):2187–2199

    Article  Google Scholar 

  23. Wang H, Wu ZJ, Rahnamayan S (2011) Enhanced opposition-based differential evolution for solving high-dimensional continuous optimization problems. Soft Comput 15(11):2127–2140

    Article  Google Scholar 

  24. Yang ZY, Tang K, Yao X (2011) Scalability of generalized adaptive differential evolution for large-scale continuous optimization. Soft Comput 15(11):2141–2155

    Article  Google Scholar 

  25. Zhao S-Z, Suganthan PN, Das S (2010) Self-adaptive differential evolution with modified multi-trajectory search for CEC’2010 large scale optimization. In: Swarm, evolutionary, and memetic computing. Springer, Berlin, pp 1–10

  26. Hedar A-R, Ali A (2012) Tabu search with multi-level neighborhood structures for high dimensional problems. Appl Intell 37(2):189–206

    Article  Google Scholar 

  27. Stanarevic N (2012) Hybridizing artificial bee colony (ABC) algorithm with differential evolution for large scale optimization problems. Int J Math Comput Simul 6(1):194–202

    Google Scholar 

  28. You X (2010) Differential evolution with a new mutation operator for solving high dimensional continuous optimization problems. J Comput Inf Syst 6(9):3033–3039

    Google Scholar 

  29. Ros R, Hansen N (2008) A simple modification in CMA-ES achieving linear time and space complexity. In: Rudolph G et al (eds) Parallel problem solving from nature—PPSN X. Springer, Berlin, pp 296–305

    Chapter  Google Scholar 

  30. Liao T, Montes de Oca MA (2011) Tuning parameters across mixed dimensional instances: a performance scalability study of Sep-G-CMA-ES. In: Proceedings of the 13th annual conference companion on genetic and evolutionary computation. ACM, Dublin, pp 703–706

  31. Montes de Oca MA, Aydın D, Stützle T (2011) An incremental particle swarm for large-scale continuous optimization problems: an example of tuning-in-the-loop (re)design of optimization algorithms. Soft Comput 15 (11):2233–2255

    Article  Google Scholar 

  32. Masegosa AD, Pelta DA, Verdegay JL (2013) A centralised cooperative strategy for continuous optimisation: the influence of cooperation in performance and behaviour. Inf Sci 219(0):73–92

    Article  MathSciNet  MATH  Google Scholar 

  33. Li X, Yao X (2012) Cooperatively coevolving particle swarms for large scale optimization. IEEE Trans Evol Comput 16(2):210–224

    Article  MathSciNet  Google Scholar 

  34. Li X et al (2015) Editorial for the special issue of Information Sciences Journal (ISJ) on “Nature-inspired algorithms for large scale global optimization”. Inf Sci 316:437–439

    Article  Google Scholar 

  35. Tsurkov V (2001) Large-scale optimization. Applied optimization. Springer US

  36. Liu L, Shao L, Li X (2015) Evolutionary compact embedding for large-scale image classification. Inf Sci 316:567–581

    Article  Google Scholar 

  37. Miranda V, Martins J, Palma V (2014) Optimizing large scale problems with metaheuristics in a reduced space mapped by autoencoders-application to the wind-hydro coordination. IEEE Trans Power Syst 29(6):3078–3085

    Article  Google Scholar 

  38. LaTorre A, Muelas S, Pena J (2015) A comprehensive comparison of large scale global optimizers. Inf Sci 316:517–549

    Article  Google Scholar 

  39. Gardeux V et al (2009) Unidimensional search for solving continuous high-dimensional optimization problems. In: Ninth international conference on intelligent systems design and applications. ISDA ’09. November 30–December 2, 2009. IEEE Computer Society, Pisa

  40. Yang X-S, Koziel S (2011) Computational optimization and applications in engineering and industry, vol 359. Springer Science & Business Media

  41. Conn AR, Scheinberg K, Vicente LN (2009) Introduction to derivative-free optimization, vol 8. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  42. Torczon V (1997) On the convergence of pattern search algorithms. SIAM J Optim 7(1):1–25

    Article  MathSciNet  MATH  Google Scholar 

  43. Kolda TG, Lewis RM, Torczon V (2003) Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev 45(3):385–482

    Article  MathSciNet  MATH  Google Scholar 

  44. Hooke R, Jeeves TA (1961) “Direct search” solution of numerical and statistical problems. J ACM 8 (2):212–229

    Article  MATH  Google Scholar 

  45. Lozano M, Molina D, Herrera F (2011) Editorial scalability of evolutionary algorithms and other metaheuristics for large-scale continuous optimization problems. Soft Comput 15(11):2085–2087

    Article  Google Scholar 

  46. Glover F et al (1998) A template for scatter search and path relinking. In: Hao J-K (ed) Artificial evolution. Springer, Berlin, pp 1–51

    Google Scholar 

  47. Glover F (1995) Tabu thresholding: improved search by nonmonotonic trajectories. INFORMS J Comput 7 (4):426–442

    Article  MATH  Google Scholar 

  48. Gardeux V et al (2011) EM323: a line search based algorithm for solving high-dimensional continuous non-linear optimization problems. Soft Comput 15(11):2275–2285

    Article  Google Scholar 

  49. Kirkpatrick S, Gelatt CD Jr, Vecchi MP (1983) Optimization by simulated annealing. Science 220 (4598):671–680

    Article  MathSciNet  MATH  Google Scholar 

  50. Omidvar MN, Li X, Yao X (2010) Cooperative co-evolution with delta grouping for large scale non-separable function optimization. In: IEEE congress on evolutionary computation (CEC 2010). 18–23 July. IEEE Computer Society, Barcelona

  51. Tang K et al (2010) Benchmark functions for the CEC’2010 special session and competition on large scale global optimization. In: Nature inspired computation and applications laboratory, USTC, China: http://nical.ustc.edu.cn/cec10ss.php

  52. Yang Z, Tang K, Yao X (2008) Large scale evolutionary optimization using cooperative coevolution. Inf Sci 178(15):2985–2999

    Article  MathSciNet  MATH  Google Scholar 

  53. Yang Z, Tang K, Yao X (2008) Multilevel cooperative coevolution for large scale optimization. In: IEEE congress on evolutionary computation (CEC 2008). June 1–6. IEEE Computer Society, Hong Kong

  54. Korosec P, Tashkova K, Silc J (2010) The differential Ant-Stigmergy Algorithm for large-scale global optimization. In: IEEE congress on evolutionary computation (CEC 2010). 18–23 July. IEEE Computer Society, Barcelona

  55. Wang H et al (2010) Sequential DE enhanced by neighborhood search for large scale global optimization. In: IEEE congress on evolutionary computation (CEC 2010). 18–23 July. IEEE Computer Society, Barcelona

  56. Wang Y, Li B (2010) Two-stage based ensemble optimization for large-scale global optimization. In: IEEE congress on evolutionary computation (CEC 2010). 18–23 July. IEEE Computer Society, Barcelona

  57. Molina D, Lozano M, Herrera F (2010) MA-SW-Chains: memetic algorithm based on local search chains for large scale continuous global optimization. In: IEEE congress on evolutionary computation (CEC 2010). 18–23 July. IEEE Computer Society, Barcelona

    Google Scholar 

  58. Brest J et al (2010) Large scale global optimization using self-adaptive differential evolution algorithm. In: IEEE congress on evolutionary computation (CEC 2010). 18–23 July. IEEE Computer Society, Barcelona

  59. Zhao S-Z, Suganthan PN, Das S (2010) Dynamic multi-swarm particle swarm optimizer with sub-regional harmony search. In: IEEE congress on evolutionary computation (CEC 2010). 18–23 July. IEEE Computer Society, Barcelona

  60. Brest J et al (2008) High-dimensional real-parameter optimization using self-adaptive differential evolution algorithm with population size reduction. In: IEEE congress on evolutionary computation (CEC 2008). June 1–6. IEEE Computer Society, Hong Kong

  61. Potter MA, Jong KAD (1994) A cooperative coevolutionary approach to function optimization. In: Proceedings of the international conference on evolutionary computation. The third conference on parallel problem solving from nature: parallel problem solving from nature. Springer, pp 249–257

  62. Dorigo M, Birattari M (2010) Ant colony optimization. In: Encyclopedia of machine learning. Springer, pp 36–39

  63. Kennedy J (2010) Particle swarm optimization. In: Encyclopedia of machine learning. Springer, pp 760–766

  64. Vaz AIF, Vicente LN (2007) A particle swarm pattern search method for bound constrained global optimization. J Glob Optim 39(2):197–219

    Article  MathSciNet  MATH  Google Scholar 

  65. Dolan ED, Moré JJ (2002) Benchmarking optimization software with performance profiles. Math Program 91(2):201–213

    Article  MathSciNet  MATH  Google Scholar 

  66. Ren Y, Wu Y (2013) An efficient algorithm for high-dimensional function optimization. Soft Comput 17 (6):995–1004

    Article  Google Scholar 

  67. García S et al (2009) A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 special session on real parameter optimization. J Heuristics 15(6):617–644

    Article  MATH  Google Scholar 

  68. Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate—a practical and powerful approach to multiple testing. J R Stat Soc Ser B Methodol 57(1):289–300

    MathSciNet  MATH  Google Scholar 

  69. Dass P et al (2015) Hybridisation of classical unidimensional search with ABC to improve exploitation capability. Int J Artif Intell Soft Comput 5(2):151–164

    Article  Google Scholar 

  70. Jadon S, Bansal J, Tiwari R (2016) Escalated convergent artificial bee colony. J Exp Theor Artif Intell 28(1–2):181–200

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, EISTI Engineering School, Cergy, France

    Vincent Gardeux & Rachid Chelouah

  2. Department of Computer Science, Gulf University for Science & Technology, Kuwait City, Kuwait

    Mahamed G. H. Omran

  3. LiSSi Laboratory, University of Paris-Est Creteil, Creteil, France

    Patrick Siarry

  4. Leeds School of Business, University of Colorado, Boulder, CO, USA

    Fred Glover

Authors
  1. Vincent Gardeux
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mahamed G. H. Omran
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rachid Chelouah
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Patrick Siarry
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fred Glover
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vincent Gardeux.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(XLS 621 KB)

(XLSX 42.6 KB)

(PDF 179 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gardeux, V., H. Omran, M.G., Chelouah, R. et al. Adaptive pattern search for large-scale optimization. Appl Intell 47, 319–330 (2017). https://doi.org/10.1007/s10489-017-0901-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-017-0901-8

Keywords

Search

Navigation