Abstract
In this paper we develop, analyze, and test a new algorithm for the global minimization of a function subject to simple bounds without the use of derivatives. The underlying algorithm is a pattern search method, more specifically a coordinate search method, which guarantees convergence to stationary points from arbitrary starting points. In the optional search phase of pattern search we apply a particle swarm scheme to globally explore the possible nonconvexity of the objective function. Our extensive numerical experiments showed that the resulting algorithm is highly competitive with other global optimization methods also based on function values.
Similar content being viewed by others
References
Alberto P., Nogueira F., Rocha H., Vicente L.N. (2004) Pattern search methods for user-provided points: application to molecular geometry problems. SIAM J. Optim. 14, 1216–1236
Ali M.M., Khompatraporn C., Zabinsky Z.B. (2005) A numerical evaluation of several stochastic algorithms on selected continuous global optimization test problems. J. Global Optim. 31, 635–672
Audet C., Dennis J.E. (2003) Analysis of generalized pattern searches. SIAM J. Optim. 13, 889–903
Audet C., Dennis J.E. (2006) Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17, 188–217
Audet C., Orban D. (2006) Finding optimal algorithmic parameters using derivative-free optimization. SIAM J. Optim. 17, 642–664
van den Bergh, F.: An analysis of particle swarm optimizers. Ph.D thesis, Faculty of Natural and Agricultural Science, University of Pretoria (2001)
Van Den Berghand F., Engelbrecht A.P. (2006) A study of particle swarm optimization particle trajectories. Inf. Sci. 176, 937–971
Binder A.K., Stauffer A.D. (1985) A simple introduction to Monte Carlo simulations and some specialized topics. In: Binder E.K. (ed) Applications of the Monte Carlo Method in Statistical Physics. Springer, Berlin Heidelberg New York, pp. 1–36
Custódio, A.L., Vicente, L.N.: Using sampling and simplex derivatives in pattern search methods. SIAM J. Optim. (2007, in press)
Davis C. (1954) Theory of positive linear dependence. Am. J. Math. 76, 733–746
Dolan E.D., Moré J.J. (2002) Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213
Eberhart, R., Kennedy, J.: New optimizers using particle swarm theory. In: Proceedings of the 1995 6th International Symposium on Micro Machine and Human Science, pp. 39–43, Nagoya, Japan. IEEE Service Center, Piscata way, NJ (1995)
Finkel, D.E.: DIRECT Optimization Algorithm User Guide. North Carolina State University (2003) http://www4.ncsu.edu/~definkel/research/index.html
Fourer R., Gay D.M., Kernighan B.W. (1990) A modeling language for mathematical programming. Manage. Sci. 36, 519–554
Gay, D.M.: Hooking your solver to AMPL. Numerical Analysis Manuscript 93-10, AT&T Bell Laboratories (1993) http://www.ampl.com
Hart W.E. (2003) Locally-adaptive and memetic evolutionary pattern search algorithms. Evol. Comput. 11, 29–52
Hedar A.-R., Fukushima M. (2004) Heuristic pattern search and its hybridization with simulated annealing for nonlinear global optimization. Optim. Methods Softw. 19, 291–308
Hough P., Kolda T.G., Torczon V. (2001) Asynchronous parallel pattern search for nonlinear optimization. SIAM J. Sci. Comput. 23, 134–156
Huyer, W., Neumaier, A.: Global optimization by multilevel coordinate search. J. Global Optim. 14, 331–355 (1999) http://solon.cma.univie.ac.at/~neum/software/mcs
Ingber, L.: Adaptative simulated annealing (ASA): lessons learned. Control Cybern. 25, 33–54 (1996) http://www.ingber.com
Ingber L., Rosen B. (1992) Genetic algorithms and very fast simulated reannealing: a comparison. Math. Comput. Model. 16, 87–100
James F. (1990) A review of pseudorandom number generators. Comput. Phys. Commun. 60, 329–344
Jones D.R., Perttunen C.D., Stuckman B.E. (1993) Lipschitzian optimization without the Lipschitz constant. J. Optim. Theory Appl. 79, 157–181
Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of the 1995 IEEE International Conference on Neural Networks, pp. 1942–1948, Perth, Australia. IEEE Service Center, Piscataway, NJ (1995)
Kiseleva E., Stepanchuk T. (2003) On the efficiency of a global non-differentiable optimization algorithm based on the method of optimal set partitioning. J. Global Optim. 25, 209–235
Kolda T.G., Lewis R.M., Torczon V. (2003) Optimization by direct search: new prespectives on some classical and modern methods. SIAM Rev. 45, 385–482
Levine, D.: Users guide to the PGAPack parallel genetic algorithm library. Technical Report ANL-95/18, Argonne National Laboratory (1996) http://www.mcs.anl.gov/pgapack.html
Locatelli M. (2003) A note on the Griewank test function. J. Global Optim. 25, 169–174
Locatelli M., Schoen F. (2002) Fast global optimization of difficult Lennard-Jones clusters. Comput. Optim. Appl. 21, 55–70
Marsden, A.L.: Aerodynamic noise control by optimal shape design. Ph.D thesis, Stanford University (2004)
Meza J.C., Martinez M.L. (1994) On the use of direct search methods for the molecular conformation problem. J. Comput. Chem. 15, 627–632
Mongeau M., Karsenty H., Rouzé V., Hiriart-Urruty J.-B. (2000) Comparison of public-domain software for black box global optimization. Optim. Methods Softw. 13, 203–226
Park S.K., Miller K.W. (1988) Random number generators: good ones are hard to find. Commun. ACM 31, 1192–1201
Parsopoulos, K.E., Plagianakos, V.P., Magoulas, G.D., Vrahatis, M.N.: Stretching technique for obtaining global minimizers through particle swarm optimization. In: Proceedings of the Particle Swarm Optimization Workshop, pp. 22–29, Indianapolis, USA (2001)
Schutte J.F., Groenwold A.A. (2003) A study of global optimization using particle swarms. J. Global Optim. 31(1): 93–108
Author information
Authors and Affiliations
Corresponding author
Additional information
Support for A. Ismael F. Vaz was provided by Algoritmi Research Center, and by FCT under grants POCI/MAT/59442/2004 and POCI/MAT/58957/2004.
Support for Luís N. Vicente was provided by Centro de Matemática da Universidade de Coimbra and by FCT under grant POCI/MAT/59442/2004.
Rights and permissions
About this article
Cite this article
Vaz, A.I.F., Vicente, L.N. A particle swarm pattern search method for bound constrained global optimization. J Glob Optim 39, 197–219 (2007). https://doi.org/10.1007/s10898-007-9133-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10898-007-9133-5
Keywords
- Direct search
- Pattern search
- Particle swarm
- Derivative free optimization
- Global optimization
- Bound constrained nonlinear optimization