Skip to main content
Log in

A framework for memetic optimization using variable global and local surrogate models

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

We propose a framework of memetic optimization using variable global and local surrogate-models for optimization of expensive functions. The framework employs the trust-region approach but replaces the quadratic models with the more general RBF ones. It makes an extensive use of accuracy assessment to select the models used and to improve them if necessary. It also employs several efficient and stable numerical methods to improve its performance. Rigorous performance analysis shows the proposed framework significantly outperforms several existing surrogate-assisted evolutionary algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alexandrov NM, Dennis JE, Lewis RM, Torczon V (1998) A trust region framework for managing use of approximation models in optimization. Struct Optim 15(1): 16–23

    Article  Google Scholar 

  • Barthelemy JFM, Haftka RT (1993) Approximation concepts for optimum structural design—a review. Struct Optim 5: 129–144

    Article  Google Scholar 

  • Bellman RE (1961) Adaptive control processes:a guided tour. Princeton University Press, Princeton

    MATH  Google Scholar 

  • Booker AJ, Dennis JE, Frank PD, Serafini DB, Torczon V, Trosset MW (1999) A rigorous framework for optimization of expensive functions by surrogates. Struct Optim 17(1): 1–13

    Article  Google Scholar 

  • Box GEP, Draper NR (1987) Empirical model building and response surface. Wiley, New York

    Google Scholar 

  • Chen VP, Burton RR, Meckesheimer M, Tsui K-T (2006) A review on design modeling and applications of computer experiments. IEE Trans 38(4): 273–291

    Article  Google Scholar 

  • Chiba K, Obayashi S, Nakahashi K (2005) High-fidelity multidisciplinary design optimization of aerostructural wing shape for regional jet. In: Proceedings of the 23rd AIAA applied aerodynamics conference, number AIAA-2005-5080, American Institute for Aeronautics and Astronautics (AIAA), Reston

  • Chipperfield A, Fleming P, Pohlheim H, Fonseca C (1997) Genetic algorithm TOOLBOX for use with MATLAB, version 1.2. Department of Automatic Control and Systems Engineering, University of Sheffield

  • Conn AR, Scheinberg K, Toint PL (1997) Recent progress in unconstrained nonlinear optimization without derivatives. Math Program 79: 397–414

    MathSciNet  Google Scholar 

  • Conover WJ (1980) Practical nonparametric statistics, 2nd edn. Wiley, New York

    Google Scholar 

  • deSouza PS, Talukdar SN (1991) Genetic algorithms in asynchronous teams. In: Belew RK, Booker LB (eds) Proceedings of the fourth international conference on genetic algorithms, San Mateo, pp 392–397

  • Demmel JW (1987) The geometry of ill-conditioning. Comput J 3(2): 201–229

    MATH  MathSciNet  Google Scholar 

  • Dennis JE Jr, Torczon V (1997) Managing approximation models in optimization. In: Alexandrov NM, Hussaini MY (eds) Multidisciplinary design optimization: state of the art. SIAM, Philadelphia, pp 330–347

    Google Scholar 

  • Dixon LCW, Szegö GP (1978) The global optimization problem: An introduction. In: Dixon LCW, Szegö GP (eds) Towards global optimisation 2. North-Holland Publishing Company, Amsterdam, pp 1–15

    Google Scholar 

  • Drela M, Youngren H (2001) XFOIL 6.9 User Primer. Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge

    Google Scholar 

  • Eby D, Averill RC, Punch III, WF, Goodman ED (1998) Evaluation of injection island GA performance on flywheel design optimization. In: Proceedings of the third conference on adaptive computing in design and manufacturing, Springer, Plymouth, pp 121–136

  • El-Betalgy MA, Keane AJ (2001) Evolutionary optimization for computationally expensive problems using Gaussian processes. In: Hamid Arbania (ed) Proceedings of the international conference on artificial intelligence–IC-AI’2001, CSREA Press, pp 708–714

  • Filippone A (2006) Flight performance of fixed and rotary wing aircraft, 1st edn. Elsevier, Amsterdam

    Google Scholar 

  • Frank PD (1995) Global modeling for optimization. SIAM SIAG/OPT Views-and-News 7: 9–12

    Google Scholar 

  • Franke R (1982) Scattered data interpolation: Tests of some methods. Math Comput 38(157): 181–200

    Article  MATH  MathSciNet  Google Scholar 

  • Gaspar-Cunha A, Vieira A (2005) A multi-objective evolutionary algorithm using neural networks to approximate fitness evaluations. Int J Comput Syst Signals 6(1): 18–36

    Google Scholar 

  • Giannakoglou KC (2002) Design of optimal aerodynamic shapes using stochastic optimization methods and computational intelligence. Int Rev J Prog Aerospace Sci 38(1): 43–76

    Article  Google Scholar 

  • Gutmann HM (2001) A radial basis function method for global optimization. J Glob Optim 19: 201–227

    Article  MATH  MathSciNet  Google Scholar 

  • Hardy RL (1971) Multiquadric equations of topography and other irregular surfaces. J Geophys Res 76: 1905–1915

    Article  Google Scholar 

  • Hart WE, Belew RK (1995) Optimization with genetic algorithm hybrids that use local search. In: Belew RK, Mitchell M (eds) Adaptive Individuals in Evolving Populations: Models and Algorithms, Santa Fe Institute Studies in the Sciences of Complexity, chapter 27. Addison-Wesley, Reading, pp 483–496

    Google Scholar 

  • Holst TL, Pulliam TH (2001) Aerodynamic shape optimization using a real-number-encoded genetic algorithm. Technical Report 2001-2473, AIAA

  • Jin Y (2005) A comprehensive survey of fitness approximation in evolutionary computation. J Soft Comput 9(1): 3–12

    Article  Google Scholar 

  • Jin R, Chen W, Simpson TW (2001) Comparative studies of metamodeling techniques under multiple modeling criteria. Struct Optim 23(1): 1–13

    Article  Google Scholar 

  • Jin Y, Olhofer M, Sendhoff B (2002) A framework for evolutionary optimization with approximate fitness functions. IEEE Trans Evol Comput 6(5): 481–494

    Article  Google Scholar 

  • Johnson ME, Moore LM, Ylvisaker D (1990) Minimax and maximin distance designs. J Stat Plan Inference 26(2): 131–148

    Article  MathSciNet  Google Scholar 

  • Kansa EJ, Hon Y-C (2000) Circumventing the ill-conditioning problem with multiquadric radial basis functions: applications to elliptic partial differential equations. Comput Math Appl 39(7): 123–137

    Article  MATH  MathSciNet  Google Scholar 

  • Karakasis MK, Giannakoglou KC (2004) On the use of surrogate evaluation models in multi-objective evolutionary algorithms. In: Proceedings of the European conference on computational methods in applied sciences and engineering–ECCOMAS 2004

  • Kim H-S, Cho S-B (2001) An efficient genetic algorithm with less fitness evaluation by clustering. In: Proceedings of 2001 IEEE conference on evolutionary computation, IEEE, pp 887–894

  • Kleijnen JPC, Sargent RG (2000) A methodology for fitting and validating metamodels in simulation. Eur J Oper Res 120(1): 14–29

    Article  MATH  Google Scholar 

  • Kruskal WH, Wallis WA (1952) Use of ranks on one-criterion variance analysis. J Am Stat Assoc 47: 583–621

    Article  MATH  Google Scholar 

  • Liang K-H, Yao X, Newton C (2000) Evolutionary search of approximated N-dimensional landscapes. Int J Knowl Based Intell Eng Syst 4(3): 172–183

    Google Scholar 

  • Linhart H, Zucchini W (1986) Model selection. Wiley series in probability and mathematical statistics. Wiley-Interscience Publication, New York

    Google Scholar 

  • Mann HB, Whitney DR (1947) On a test whether one of two variables is stochastically larger than the other. Ann Math Stat 18: 50–60

    Article  MATH  MathSciNet  Google Scholar 

  • De Marchi S (2003) On optimal center locations for radial basis function interpolation: computational aspects. Rendiconti del Semin Matematico. 61(3): 343–358

    MATH  MathSciNet  Google Scholar 

  • Martin JD, Simpson TW (2005) Use of kriging models to approximate deterministic computer models. AIAA J 43(4): 853–863

    Article  Google Scholar 

  • McKay MD, Beckman RJ, Conover WJ (1979) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2): 239–245

    Article  MATH  MathSciNet  Google Scholar 

  • Meckesheimer M, Booker AJ, Burton RR, Simpson TW (1979) Computationally inexpensive metamodel assessment strategies. AIAA J 40(10): 2053–2060

    Article  Google Scholar 

  • Micceri T (1989) The unicorn, the normal curve and other improbable creatures. Psychol Bull 105(1): 156–166

    Article  Google Scholar 

  • Micchelli CA (1986) Interpolation of scattered data: distance matrices and conditionally positive definite functions:. Constructive Approximation 2: 11–22

    Article  MATH  MathSciNet  Google Scholar 

  • Mitchell TJ, Morris MD (1992) Bayesian design and analysis of computer experiments: two examples. Stat Sin 2: 359–379

    MATH  Google Scholar 

  • Mühlenbein H, Schlierkamp-Voosen D (1993) Predictive models for the breeder genetic algorithm I: continuous parameter optimization. Evol Comput 1(1): 25–49

    Article  Google Scholar 

  • Myers RH, Montgomery DC (1995) Response surface methodology: process and product optimization using designed experiments. Wiley, New York

    MATH  Google Scholar 

  • Obayashi S (2000) Airfoil shape optimization for evolutionary computation. In: Genetic algorithms for optimization in aeronautics and turbomachinery, VKI Lecture Series 2000-07. Rhode Saint Genese, Belgium and Von Karman Institute for Fluid Dynamics

  • Ong Y-S, Keane AJ (2004) Meta-Lamarckian learning in memetic algorithm. IEEE Trans Evol Comput 8(2): 99–110

    Article  Google Scholar 

  • Ong Y-S, Nair PB, Keane AJ (2003) Evolutionary optimization of computationally expensive problems via surrogate modeling. Am Inst Aeron Astron J 41(4): 687–696

    Google Scholar 

  • Ong Y-S, Nair PB, Lum KY (2006a) Max-min surrogate-assisted evolutionary algorithm for robust aerodynamic design. IEEE Trans Evol Comput 10(4): 392–404

    Article  Google Scholar 

  • Ong Y-S, Zong Z, Lim D (2006b) Curse and blessing of uncertainty in evolutionary algorithm using approximation. In: Proceedings of the IEEE congress on evolutionary computation–WCCI 2006, IEEE, pp 2928–2935

  • Oyama A, Obayashi S, Nakahashi K (2000) Real-coded adaptive range genetic algorithm and its application to aerodynamic design. Int J Jpn Soc Mech Eng 43(2): 124–129

    Google Scholar 

  • Oyama A, Obayashi S, Nakahashi T (2002) Real-coded adaptive range genetic algorithm applied to transonic wing optimization. In: Marc Schoenauer (ed) The 6th international conference on parallel problem solving from nature–PPSN VI, Springer, Berlin, pp 712–721

  • Poggio T, Girosi F(1989) A theory of networks for approximation and learning. A.I.Memo 1140, Massachusetts Institute of Technology, Artificial Intelligence Laboratory and Center for Biological Information Processing

  • Poloni C, Giurgevich A, Onseti L, Pediroda V (2000) Hybridization of a multi-objective genetic algorithm, a neural network and a classical optimizer for a complex design problem in fluid dynamics. Comput Methods Appl Mech Eng 186: 403–420

    Article  MATH  Google Scholar 

  • Quagliarella D, Vicini A (1997) Coupling genetic algorithms and gradient based optimization techniques. In: Quagliarella D, Periaux J, Poloni C, Winter G (eds) Genetic algorithms in engineering and computer science, Chap 14. Wiley, New York, pp 288–309

    Google Scholar 

  • Ratle A (1998) Accelerating the convergence of evolutionary algorithms by fitness landscape approximations. In: Eiben AE, Bäck T, Schoenauer M, Schwefel HP (eds) Proceedings of the 5th international conference on parallel problem solving from nature–PPSN V. Lecture notes in computer science, vol 1498, Springer, Berlin, pp 87–96

  • Ratle A (1999) Optimal sampling strategies for learning a fitness model. In: The 1999 IEEE congress on evolutionary computation–CEC 1999, IEEE, New York, pp 2078–2085

  • Renderes J-M, Bersini H (1994) Hybridizing genetic algorithms with hill-climbing methods for global optimization: two possible ways. In: Sebald AV, Fogel LJ (eds) Proceedings of the third annual conference on evolutionary programming, World Scientific, pp 312–317

  • Renderes J-M, Flasse SP (1996) Hybrid methods using genetic algorithms for global optimization. IEEE Trans Syst Man Cybern B 26(2): 243–258

    Article  Google Scholar 

  • Rippa S (1999) An algorithm for selecting a good value for the parameter c in radial basis function interpolation. Adv Comput Math 11(2–3): 193–210

    Article  MATH  MathSciNet  Google Scholar 

  • Rodríguez JF, Renaud JE, Watson LT (1998) Convergence of trust region augmented Lagrangian methods using variable fidelity approximation data. Struct Optim 15(3/4): 141–156

    Article  Google Scholar 

  • Schaback R (1993) Comparison of radial basis function interpolants. In: Jetter K, Utreras FI (eds) Multivariate approximations: from CAGD to wavelets approximations and decompositions. World Scientific, , pp 293–305

  • Schaback R (1996) Multivariate interpolation and approximation by translates of a basis function. In: Chui CK, Schumaker LL (eds) Approximation theory VIII. Vanderbilt University Press, , pp 491–514

  • Sefrioui M, Périaux J (2000) A hierarchical genetic algorithm using multiple models for optimization. In: Marc Schoenauer, Kalyanmoy Deb, Günter Rudolph, Xin Yao, Evelyne Lutton, Juan J. Merelo Guervós, and Hans-Paul Schwefel (eds) Proceedings of the 6th international conference on parallel problem solving from nature–PPSN VI, number 1917 in Lecture notes in computer science, Springer, pp 879–888

  • Shapiro SS, Wilk MB (1965) An analysis of variance test for normality (complete samples). Biometrika 52: 591–611

    MATH  MathSciNet  Google Scholar 

  • Shapiro SS, Wilk MB (1968) Approximation for the null distribution of the W statistic. Technometrics 10: 861–866

    Article  MathSciNet  Google Scholar 

  • Sheskin DJ (2007) Handbook of parametric and nonparametric statistical procedures, 4th edn. Chapman and Hall, Boca Raton

    MATH  Google Scholar 

  • Simpson TW, Lin DKJ, Chen W (2001a) Sampling strategies for computer experiments: design and analysis. Int J Reliab Appl 2(3): 209–240

    Google Scholar 

  • Simpson TW, Poplinski JD, Koch PN, Allen JK (2001b) Metamodels for computer-based engineering design: survey and recommendations. Eng Comput 17: 129–150

    Article  MATH  Google Scholar 

  • Simpson TW, Booker AJ, Ghosh D, Giunta AA, Koch PN, Yang R-J (2004) Approximation methods in multidisciplinary analysis and optimization: a panel discussion. Struct Multidiscip Optim 27: 302–313

    Article  Google Scholar 

  • Smith RE, Dike BA, Stegmann SA (1995) Fitness inheritance in genetic algorithms. In: George KM (ed), Proceedings of the 1995 ACM symposium on applied computing–SAC 95, ACM Press, New York, pp 345–350

  • Sobieckzy H (1999) Parametric airfoils and wings. In: Kozo Fujii, Dulikravich GS, Susumu Takanashi (eds) Recent development of aerodynamic design methodologies : inverse design and optimization. Notes on numerical fluid mechanics, vol 68, Vieweg, Braunschweig; Wiesbaden, pp 71–88

  • Sobieszczansk-Sobieski J, Haftka RT (1997) Multidisciplinary aerospace design optimization: Survey of recent developments. Struct Optim 14(1): 1–23

    Article  Google Scholar 

  • Stone CJ (1982) Optimal global rates of convergence for nonparametric regression. Ann Stat 10(4): 1040–1053

    Article  MATH  Google Scholar 

  • Stone M (1974) Cross-validatory choice and assessment of statistical predictors. J R Stat Soc Ser B 36(2): 111–147

    MATH  Google Scholar 

  • Tenne Y, Armfield SW (2007) A memetic algorithm using a trust-region derivative-free optimization with quadratic modelling for optimization of expensive and noisy black-box functions. In: Yang S, Ong Y-S, Jin Y (eds) Evolutionary computation in dynamic and uncertain environments, vol 51 of studies in computational intelligence. Springer, Berlin, pp 389–415

    Chapter  Google Scholar 

  • Tenne Y, Armfield SW (2008a) A versatile surrogate-assisted memetic algorithm for optimization of computationally expensive functions and its engineering applications. In: Yang A, Shan Y, Bui LT (eds) Success in evolutionary computation, vol 92/2008. Springer, Berlin, pp 43–72

    Chapter  Google Scholar 

  • Tenne Y, Armfield SW (2008b) A memetic algorithm assisted by an adaptive topology artificial neural network and variable local models for expensive optimization problems. In: Advances in evolutionary algorithms (accepted)

  • Tenne Y, Obayashi S, Armfield SW (2007) Airfoil shape optimization using an algorithm for minimization of expensive and discontinuous black-box functions. In: Proceedings of the AIAA InfoTec 2007, vol 2, American Institute for Aeronautics and Astronautics (AIAA), AIAA-2007-2874, pp 1691–1710

  • Toint PL (1978) Some numerical results using a sparse matrix updating formula in unconstrained optimization. Math Comput 32(143): 839–851

    Article  MATH  MathSciNet  Google Scholar 

  • Torczon V, Trosset MW (1998) Using approximations to accelerate engineering design optimization. In: Proceedings of the 7th AIAA/USAF/NASA/ISSMO symposium on multidisciplinary analysis and optimization, American Institute for Aeronautics and Astronautics (AIAA), Reston, pp 738–748

  • Törn A (1973) Global optimization as a combination of global and local search. In: Proceedings of computer simulation versus analytical solutions for business and economic models, number 17 in BAS Business Administration Studies, School of Business Administration, Gothenburg, pp 191–206

  • Törn A, Žilinskas A (1989) Global optimization. Number 350 in lecture notes in computer science. Springer, Berlin

  • Törn A, Ali MM, Viitanen S (1999) Stochastic global optimization: problems classes and solution techniques. J Glob Optim 14: 437–447

    Article  MATH  Google Scholar 

  • Trosset MW, Alexandrov NM, Watson LT (2003) New methods for robust design using computer simulation. In: Proceedings of the section on physical and engineering sciences, American Statistical Association, pp 191–206

  • Yao X, Lin G, Liu Y (1997) An analysis of evolutionary algorithms based on neighbourhood and step sizes. In: Angeline PJ, Reynolds RG, McDonnel JR, Eberhart R (eds) Evolutionary programming VI: Proceedings of the sixth annual conference on evolutionary programming, Lecture notes on computer science, vol 1213, Springer, Berlin, pp 297–307

  • Yen J, Liao JC, Lee B, Randolph D (1998) A hybrid approach to modeling metabolic systems using a genetic algorithm and simplex method. IEEE Trans Systems Man Cybern B 28(2): 173–191

    Article  Google Scholar 

  • Yun Y, Gen M, Seo S (2003) Various hybrid methods based on genetic algorithm with fuzzy logic controller. J Intell Manuf 14: 401–419

    Article  Google Scholar 

  • Zhou Z, Ong Y-S, Lim M, Lee B (2007a) Memetic algorithms using multi-surrogates for computationally expensive optimization problems. J Soft Comput 11(10): 957–971

    Article  Google Scholar 

  • Zhou Z, Ong Y-S, Nair PB, Keane AJ, Lum KY (2007b) Combining global and local surrogate models to accelerate evolutionary optimization. IEEE Trans Syst Man Cybern C 37(1): 66–76

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yoel Tenne.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Tenne, Y., Armfield, S.W. A framework for memetic optimization using variable global and local surrogate models. Soft Comput 13, 781–793 (2009). https://doi.org/10.1007/s00500-008-0348-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-008-0348-2

Keywords

Navigation