Skip to main content
SpringerLink
Log in

A rigorous framework for optimization of expensive functions by surrogates

  • Research Papers
  • Published:
Structural optimization Aims and scope Submit manuscript

Abstract

The goal of the research reported here is to develop rigorous optimization algorithms to apply to some engineering design problems for which direct application of traditional optimization approaches is not practical. This paper presents and analyzes a framework for generating a sequence of approximations to the objective function and managing the use of these approximations as surrogates for optimization. The result is to obtain convergence to a minimizer of an expensive objective function subject to simple constraints. The approach is widely applicable because it does not require, or even explicitly approximate, derivatives of the objective. Numerical results are presented for a 31-variable helicopter rotor blade design example and for a standard optimization test example.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Barthelemy, J-F.M.; Haftka, R.T. 1993: Approximation concepts for optimum structural design — a review.Struct. Optim. 5, 129–144

    Google Scholar 

  • Booker, A.J. 1994: DOE for computer output.Technical Report BCSTECH-94-052 Boeing Computer Services, Research and Technology, M/S 7L-68, Seattle, Washington 98124

    Google Scholar 

  • Booker, A.J. 1996: Case studies in design and analysis of computer experiments. InProc. Section on Physical and Engineering Sciences, American Statistical Association

  • Booker, A.J.; Conn, A.R., Dennis, J.E. Jr.; Frank, P.D.; Trosset, M.W.; Torczon, V. 1995. Global modeling for optimization Boeing/IBM/Rice collaborative project 1995 final report.Technical Report ISSTECH-95-032, Boeing Information & Support Services, Research and Technology, M/S 7L-68, Seattle, Washington 98124

    Google Scholar 

  • Booker, A.J.; Dennis, J.E., Jr.; Frank, P.D.; Serafini, D.B.; Torczon, V. 1997. Optimization using surrogate objectives on a helicopter test example.Technical Report SSGTECH-97-027, Roeing Shared Services Group, Applied Research & Technology, M/S 7L-68, Seattle, Washington 98124. Also available as Technical Report 97-31, Department of Computational and Applied Mathematics, Rice University, Houston, Texas 77005-1892. To appear in: Borggaard, J.; Burns, J.; Cliff, E.; Schreck, S. (eds.)Optimal design and control. Cambridge, MA: Birkhauser

    Google Scholar 

  • Burgee, S.L.; Giunta, A.A.; Balabanov, V.; Grossman, B.; Mason, W.H.; Narducci, R.; Haftka, R.T.; Watson, L.T. 1996: A coarse-grained parallel variable-complexity multidisciplinary optimization paradigm.Int. J. Supercomputing Appl. & High Performance Computing 10, 269–299

    Google Scholar 

  • Conn, A.R.; Scheinberg, K.; Toint, Ph.L. 1997: On the convergence of derivative-free methods for unconstrained optimization. In: Iserles, A.; Buhmann, M. (eds.)Approximation theory and optimization: tributes to M.J.D. Powell, pp. 83–108. Cambridge: Cambridge University Press

    Google Scholar 

  • Conn, A.R.; Toint, Ph.L. 1996: An algorithm using quadratic interpolation for unconstrained derivative free optimization. In: Di Pillo, G.; Giannessi, F. (eds.)Nonlinear optimization and applications, pp. 27–47. New York: Plenum Publishing

    Google Scholar 

  • Cox, D.D.; John, S. 1997: SDO: a statistical method for global optimization. In: Alexandrov, N.; Hussaini, M.Y. (eds.)Multidisciplinary design optimization: state of the art, pp. 315–329. Philadelphia: SIAM

    Google Scholar 

  • Currin, C.; Mitchell, T.; Morris, M.; Ylvisaker, D. 1988: A Bayesian approach to the design and analysis of computer experiments.Technical Report ORNL-6498, Oak Ridge National Laboratory

  • Davis, C. 1954: Theory of positive linear dependence.Amer. J. Math. 76, 733–746

    Google Scholar 

  • De Boor, C.; Ron, A. 1992: Computational aspects of polynomial interpolation in several variables.Mathematics of Computation 58, 705–727

    Google Scholar 

  • Dennis, J.E., Jr.; Torczon, V. 1991: Direct search methods on parallel machines.SIAM J. Optimiz. 1, 448–474

    Google Scholar 

  • Dennis, J.E., Jr.; Torczon, V. 1997: Managing approximation models in optimization. In: Alexandrov, N.; Hussaini, M.Y. (eds.)Multidisciplinary design optimization: state-of-the-art, pp. 330–347. Philadelphia: SIAM. Also available as Technical Report 95-19, Department of Computational and Applied Mathematics, Rice University, Houston, Texas 77005-1892

    Google Scholar 

  • Dennis, J.E., Jr.; Walker, H.F. 1984: Inaccuracy in quasi-Newton methods: local improvement theorems.Math. Prog. Study 22, 70–85

    Google Scholar 

  • Dixon, L.C.W.; Szegö G.P. (eds.) 1978:Towards global optimization 2. Amsterdam: North-Holland Pub. Co.

    Google Scholar 

  • Efron, B.; Stein, C. 1981: The jackknife estimate of variance.Annals of Statistics 9, 586–596

    Google Scholar 

  • Frank, P.D. 1995: Global modeling for optimization.SIAG/OPT Views-and-News 7, 9–12

    Google Scholar 

  • Geisser, S.; Eddy, W.F. 1979: A predictive approach to model selection.J. Amer. Statistical Assoc. 74, 153–160

    Google Scholar 

  • Giunta, A.A. 1997:Aircraft multidisciplinary optimization using design of experiments theory and response surface modeling methods. Ph.D. Thesis, Virginia Tech. Available as MAD 97-05-01, May 1997, Department of Aerospace and Ocean Engineering, Virginia Tech, 215 Randolph Hall, Blacksburg, Virginia 24061

    Google Scholar 

  • Jones, D.R.; Schonlau, M.; Welch, W.J. 1997: A data analytic approach to Bayesian global optimization.Proc. ASA

  • Koehler, J.R.; Owen, A.B. 1996: Computer experiments. In: Ghosh, S.; Rao, C.R. (eds.)Handbook of statistics, volume 13, pp. 261–308. New York: Elsevier Science

    Google Scholar 

  • Levine, D. 1996: Users guide to the PGAPack parallel genetic algorithm library.Technical Report ANL-95/18, Argonne National Laboratory, 9700 South Cass Avenue, Argonne, Illinois 60439 Available from URL ftp: //info.mcs.anl.gov/pub/pgapack/pgapack.tar.Z

    Google Scholar 

  • Lewis, R.M.; Torczon, V. 1996: Pattern search algorithms for bound constrained minimization.Technical Report 96-20 ICASE, NASA Langley Research Center, Hampton, Virginia 23681-2199. To appear inSIAM J. Optimiz.

    Google Scholar 

  • Lewis, R.M.; Torczon, V. 1996: Rank ordering and positive bases in pattern search algorithms.Technical Report 96-71, ICASE, NASA Langley Research Center, Hampton, Virginia 23681-2199. In revision forMath. Prog.

    Google Scholar 

  • Lewis, R.M.; Torczon, V. 1997: Pattern search methods for linearly constrained minimization.Technical Report 98-03, ICASE, NASA Langley Research Center, Hampton, Virginia 23681-2199.SIAM J. Optimiz. (to appear)

    Google Scholar 

  • McKay, M.D.; Conover, W.J.; Beckman, R.J. 1979 A comparison of three methods for selecting values of input variables in the analysis of output from a computer code.Technometrics 21, 239–245

    Google Scholar 

  • Owen, A.B. 1992: Orthogonal arrays for computer experiments, integration and visualization.Statistica Sinica 2, 439–452

    Google Scholar 

  • Rinnooy Kan, A.H.G.; Timmer, G.T. 1984: A stochastic approach to global optimization. In: Boggs, P.T.; Byrd, R.H.; Schnabel, R.B. (eds.)Numerical optimization 1984 (Proc. SIAM Conf. on Numerical Optimization), pp. 245–262. Philadelphia: SIAM

    Google Scholar 

  • Ripley, B.D. 1988:Statistical inference for spatial processes. Cambridge: Cambridge University Press

    Google Scholar 

  • Sacks, J.; Welch, W.J.; Mitchell, T.J.; Wynn, H.P. 1989: Design and analysis of computer experiments.Statistical Sci.4, 409–435

    Google Scholar 

  • Serafini, D.B. 1998:A framework for managing models in nonlinear optimization of computationally expensive functions. Ph.D. Thesis, Rice University

  • Shultz, L.A.; Panda, B.; Tarzanin, F.J.; Derham, R.C.; Oh, B.K.; Dadone, L. 1994: Interdisciplinary analysis for advanced rotors-approach, capabilities and status. AHS Aeromechanics Specialists Conf., PS 4-1-4-15

  • Snir, M.; Otto S.W.; Huss-Lederman, S.; Walker, D.W.; Dongarra, J. 1996.MPI: The complete reference. Cambridge, MA: The MIT Press

    Google Scholar 

  • Stein, M. 1987: Large sample properties of simulations using latin hypercube sampling.Technometrics 29, 143–151

    Google Scholar 

  • Tang, B. 1993: Orthogonal array-based latin hypercubes.J. Amer. Statistical Assoc. 88, 1392–1397

    Google Scholar 

  • Torczon, V. 1992: PDS: direct search methods for unconstrained optimization on either sequential or parallel machines.Technical Report 92-9, Department of Computational and Applied Mathematics, Rice University, Houston, Texas 77005-1892. In revision forACM Trans. Math. Software

    Google Scholar 

  • Torczon, V. 1995: Pattern search methods for nonlinear optimization.SIAG/OPT Views & News 6, 7–11

    Google Scholar 

  • Torczon, V. 1997: On the convergence of pattern search algorithms.SIAM J. Optimiz. 7, 1–25

    Google Scholar 

  • Torczon, V.; Trosset, M.W. 1998: From evolutionary operation to parallel direct search: pattern search algorithms for numerical optimization.Computing Sci. & Statistics 29, 396–401

    Google Scholar 

  • Trosset, M.W.; Torczon, V. 1997: Numerical optimization using computer experiments.Technical Report 97-38, ICASE, NASA Langley Research Center, Hampton, Virginia 23681-2199

    Google Scholar 

  • Trosset, M.W. 1997: I know it when I see it: toward a definition of direct search methods.SIAG/OPT Views & News 9, 7–10

    Google Scholar 

  • Watson, G.S. 1984: Smoothing and interpolation by kriging and with splines.Math. Geology 16, 601–615

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Boeing Shared Services Group, Applied Research and Technology, Mathematics & Engineering Analysis, Box 3707, M/S 7L-22, 98124, Seattle, WA, USA

    A. J. Booker

  2. Department of Computational and Applied Mathematics & Center for Research on Parallel Computation, Rice University, P.O. Box 1892, 77005, Houston, TX, USA

    J. E. Dennis Jr.

  3. Mathematics & Engineering Analysis, Boeing Shared Services Group, Applied Research and Technology, Box 3707, M/S 7L-21, 98124, Seattle, WA, USA

    P. D. Frank

  4. E.O. Lawrence Berkeley National Laboratory, National Energy Research Scientific Computing Center, 1 Cyclotron Road, MS 50B-2239, 94720, Berkeley, CA, USA

    D. B. Serafini

  5. Department of Computer Science, College of William & Mary, P.O. Box 8795, 23187, Williamsburg, VA, USA

    V. Torczon

  6. Department of Mathematics, College of William & Mary, P.O. Box 8795, 23187, Williamsburg, VA, USA

    M. W. Trosset

Authors
  1. A. J. Booker
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. E. Dennis Jr.
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. P. D. Frank
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. D. B. Serafini
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. V. Torczon
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. M. W. Trosset
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Communicated by J. Sobieski

Rights and permissions

Reprints and permissions

About this article

Cite this article

Booker, A.J., Dennis, J.E., Frank, P.D. et al. A rigorous framework for optimization of expensive functions by surrogates. Structural Optimization 17, 1–13 (1999). https://doi.org/10.1007/BF01197708

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF01197708

Keywords

Search

Navigation