Emmanuel Kieffer
ABSTRACT
Real-life problems including transportation, planning and management often involve several decision makers whose actions depend on the interaction between each other. When involving two decision makers, such problems are classified as bi-level optimization problems. In terms of mathematical programming, a bi-level program can be described as two nested problems where the second decision problem is part of the first problem's constraints. Bi-level problems are NP-hard even if the two levels are linear. Since each solution implies the resolution of the second level to optimality, efficient algorithms at the first level are mandatory. In this work we propose BOBP, a Bayesian Optimization algorithm to solve Bi-level Problems, in order to limit the number of evaluations at the first level by extracting knowledge from the solutions which have been solved at the second level. Bayesian optimization for hyper parameter tuning has been intensively used in supervised learning (e.g., neural networks). Indeed, hyper parameter tuning problems can be considered as bi-level optimization problems where two levels of optimization are involved as well. The advantage of the bayesian approach to tackle multi-level problems over the BLEAQ algorithm, which is a reference in evolutionary bi-level optimization, is empirically demonstrated on a set of bi-level benchmarks.
- The GPyOpt authors. 2016. GPyOpt: A Bayesian Optimization framework in Python. http://github.com/SheffieldML/GPyOpt. (2016).Google Scholar
- J.F. Bard. 1983. Coordination of a multidivisional organization through two levels of management. 11 (1983), 457--468.Google Scholar
- J.F. Bard. 1991. Some properties of the bilevel programming problem. 68 (1991), 371--378. Technical Note.Google Scholar
- J.F. Bard and J. Falk. 1982. An explicit solution to the multi-level programming problem. 9 (1982), 77--100.Google Scholar
- J.F. Bard and J. Moore. 1990. A branch and bound algorithm for the bilevel programming problem. 11 (1990), 281--292.Google Scholar
- Z. Bi, P. Calamai, and A. Conn. 1989. An exact penalty function approach for the linear bilevel programming problem. Technical Report #167-O-310789. ftp://dial.uwaterloo.ca/pub/tech_reportsGoogle Scholar
- W. Bialas and M. Karwan. 1982. On two-level optimization. 27 (1982), 211--214.Google Scholar
- L. Bianco, M. Caramia, and S. Giordani. 2009. A bilevel flow model for hazmat transportation network design. Transportation Research Part C: Emerging Technologies 17, 2 (2009), 175 -- 196. Selected papers from the Sixth Triennial Symposium on Transportation Analysis (TRISTAN VI).Google ScholarCross Ref
- J. Bracken and J. McGill. 1973. Mathematical programs with optimization problems in the constraints. 21 (1973), 37--44. Google ScholarDigital Library
- J. Bracken and J. McGill. 1974. Defense applications of mathematical programs with optimization problems in the constraints. 22 (1974), 1086--1096. Google ScholarDigital Library
- L. Brotcorne, M. Labbé, Patrice Marcotte, and G. Savard. 2001. A Bilevel Model for Toll Optimization on a Multicommodity Transportation Network. Transportation Science 35, 4 (2001), 345--358. Google ScholarDigital Library
- A. Chernyshev. 2016. Bayesian Optimization of Spiking Neural Network Parameters to Solving the Time Series Classification Task. Springer International Publishing, Cham, 39--45.Google Scholar
- J. Fliege and L.N. Vicente. 2006. Multicriteria Approach to Bilevel Optimization. Journal of Optimization Theory and Applications 131, 2 (2006), 209--225.Google ScholarDigital Library
- P. Fontaine and S. Minner. 2014. Benders Decomposition for Discretefi?!-Continuous Linear Bilevel Problems with application to traffic network design. Transportation Research Part B: Methodological 70, 0 (2014), 163 -- 172.Google ScholarCross Ref
- M.R. Garey and D.S. Johnson. 1979. Computers and Intractability: A Guide to the Theory of NP-Completeness (Series of Books in the Mathematical Sciences) (first edition ed.). W. H. Freeman. http://www.amazon.com/Computers-Intractability-NP-Completeness-Mathematical-Sciences/dp/0716710455 Google ScholarDigital Library
- J. Snoek, H. Larochelle, and P.R. Adams. 2012. Practical bayesian optimization of machine learning algorithms. In Advances in neural information processing systems. 2951--2959. Google ScholarDigital Library
- A. Koh. 2007. Solving transportation bi-level programs with Differential Evolution. In 2007 IEEE Congress on Evolutionary Computation. 2243--2250.Google ScholarCross Ref
- D. Kraft. 1988. A software package for sequential quadratic programming. Forschungsbericht- Deutsche Forschungs- und Versuchsanstalt fur Luft- und Raumfahrt (1988).Google Scholar
- F. Legillon, A. Liefooghe, and E. Talbi. 2012. CoBRA: A cooperative coevolutionary algorithm for bi-level optimization. In Evolutionary Computation (CEC), 2012 IEEE Congress on. 1--8.Google Scholar
- X Li, P. Tian, and X. Min. .Google Scholar
- Bo Liu, Qingfu Zhang, and Georges G. E. Gielen. 2014. A Gaussian Process Surrogate Model Assisted Evolutionary Algorithm for Medium Scale Expensive Optimization Problems. Trans. EvoL Comp 18, 2 (April 2014), 180--192. Google ScholarDigital Library
- A.G. Mersha and S. Dempe. 2006. Linear bilevel programming with upper level constraints depending on the lower level solution. Appl. Math. Comput. 180, 1 (2006), 247 -- 254.Google ScholarCross Ref
- A. Migdalas. 1995. Bilevel programming in traffic planning: Models, methods and challenge. Journal of Global Optimization 7, 4 (1995), 381--405.Google ScholarCross Ref
- M. Köppe, M. Queyranne, and C. T. Ryan. 2009. A parametric integer programming algorithm for bilevel mixed integer programs. ArXiv e-prints (July 2009). arXiv:math.OC/0907.1298Google Scholar
- J. Mockus. 2012. Bayesian approach to global optimization: theory and applications. Vol. 37. Springer Science & Business Media.Google Scholar
- José Luis Morales and Jorge Nocedal. 2011. Remark on "Algorithm 778: L-BFGS-B: Fortran Subroutines for Large-scale Bound Constrained Optimization". ACM Trans. Math. Softw. 38, 1, Article 7 (Dec. 2011), 4 pages. Google ScholarDigital Library
- C.E. Rasmussen and C.K.I. Williams. 2005. Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). The MIT Press. Google ScholarDigital Library
- C.E. Rasmussen and C. K. I. Williams. 2005. Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). The MIT Press. Google ScholarDigital Library
- K.H. Sahin and A.R. Ciric. 1998. A dual temperature simulated annealing approach for solving bilevel programming problems. Computers and Chemical Engineering 23, 1 (1998), 11 -- 25.Google ScholarCross Ref
- G. Savard and J. Gauvin. 1994. The steepest descent direction for the nonlinear bilevel programming problem. 15 (1994), 275--282. Google ScholarDigital Library
- A. Sinha, P. Malo, and K. Deb. 2014. An improved bilevel evolutionary algorithm based on Quadratic Approximations. In 2014 IEEE Congress on Evolutionary Computation (CEC). 1870--1877.Google Scholar
- H. Stackelberg. 1952. The theory of the market economy. 328 pages.Google Scholar
- E.G. Talbi. 2013. A Taxonomy of Metaheuristics for Bi-level Optimization. Springer Berlin Heidelberg, Berlin, Heidelberg, 1--39.Google Scholar
- S. Varrette, P. Bouvry, H. Cartiaux, and F. Georgatos. 2014. Management of an Academic HPC Cluster: The UL Experience. In Proc. of the 2014 Intl. Conf. on High Performance Computing & Simulation (HPCS 2014). IEEE, Bologna, Italy, 959--967.Google ScholarCross Ref
- W.Candler, J.Fortuny-Amat, and B.McCarl. 1981. The Potential Role of Multilevel Programming in Agricultural Economics. American Journal of Agricultural Economics 63, 3 (1981), 521--531. arXiv:http://ajae.oxfordjournals.org/content/63/3/521.full.pdf+htmlGoogle ScholarCross Ref
- Frank Wilcoxon. 1945. Individual comparisons by ranking methods. Biometrics bulletin (1945), 80--83.Google Scholar
Index Terms
- Bayesian optimization approach of general bi-level problems
Recommendations
A solution to bi/tri-level programming problems using particle swarm optimization
Multilevel (including bi-level and tri-level) programming aims to solve decentralized decision-making problems that feature interactive decision entities distributed throughout a hierarchical organization. Since the multilevel programming problem is ...
Interactive approach to bi-level integer multi-objective fractional programming problem
This paper proposes an interactive approach for solving bi-level integer multi-objective fractional programming problem. At the first phase of the solution approach, we begin by finding the convex hull of its original set of constraints using the ...
Quantum binary particle swarm optimization-based algorithm for solving a class of bi-level competitive facility location problems
This paper deals with a special class of competitive facility location problems, in which two non-cooperative firms compete to capture the most of a given market, in order to maximize their profit. This paper intends to present a simple and effective ...
Comments