Quentin Renau
ABSTRACT
The configuration of radar networks is a complex problem that is often performed manually by experts with the help of a simulator. Different numbers and types of radars as well as different locations that the radars shall cover give rise to different instances of the radar configuration problem. The exact modeling of these instances is complex, as the quality of the configurations depends on a large number of parameters, on internal radar processing, and on the terrains on which the radars need to be placed. Classic optimization algorithms can therefore not be applied to this problem, and we rely on "trial-and-error" black-box approaches.
In this paper, we study the performances of 13 black-box optimization algorithms on 153 radar network configuration problem instances. The algorithms perform considerably better than human experts. Their ranking, however, depends on the budget of configurations that can be evaluated and on the elevation profile of the location. We therefore also investigate automated algorithm selection approaches. Our results demonstrate that a pipeline that extracts instance features from the elevation of the terrain performs on par with the classical, far more expensive approach that extracts features from the objective function.
Supplemental Material
Available for Download
pdf
p1263-renau-suppl.pdf (488.4 KB)
Supplemental material.
- A. Auger, D. Brockhoff, and N. Hansen. 2011. Mirrored sampling in evolution strategies with weighted recombination. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '11. ACM, 861--868. Google Scholar
Digital Library
- N. Belkhir. 2017. Per Instance Algorithm Configuration for Continuous Black-Box Optimization. Ph.D. thesis. Université Paris-Saclay. https://hal.inria.fr/tel-01669527/documentGoogle Scholar
- M. Böther, L. Schiller, P. Fischbeck, L. Molitor, M.S. Krejca, and T. Friedrich. 2021. Evolutionary minimization of traffic congestion. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '21. ACM, 937--945. Google ScholarDigital Library
- D. Brockhoff, A. Auger, N. Hansen, D. V. Arnold, and T. Hohm. 2010. Mirrored Sampling and Sequential Selection for Evolution Strategies. In Parallel Problem Solving from Nature - PPSN XI, 11th International Conference, September 11--15, 2010, Proceedings, Part I (Lecture Notes in Computer Science, Vol. 6238). Springer, 11--21. Google ScholarCross Ref
- Carola Doerr, Hao Wang, Furong Ye, Sander van Rijn, and Thomas Bäck. 2018. IOHprofiler: A Benchmarking and Profiling Tool for Iterative Optimization Heuristics. CoRR abs/1810.05281 (2018). http://arxiv.org/abs/1810.05281 Available at http://arxiv.org/abs/1810.05281. A more up-to-date documentation of IOHprofiler is available at https://iohprofiler.github.io/.Google Scholar
- R. Fletcher. 1987. Practical Methods of Optimization; (2nd Ed.). Wiley-Interscience, USA.Google ScholarCross Ref
- N. Hansen. 2009. Benchmarking a BI-population CMA-ES on the BBOB-2009 function testbed. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '09 (Companion). ACM, 2389--2396. Google ScholarDigital Library
- N. Hansen and A. Ostermeier. 2001. Completely Derandomized Self-Adaptation in Evolution Strategies. Evolutionary Computation 9, 2 (2001), 159--195. Google ScholarDigital Library
- T. Hennig, J. Kretsch, C. Pessagno, P. Salamonowicz, and W. Stein. 2001. The Shuttle Radar Topography Mission. In Proceedings of the First International Symposium on Digital Earth Moving (DEM '01). Springer, 65--77.Google Scholar
- A. Jankovic and C. Doerr. 2020. Landscape-Aware Fixed-Budget Performance Regression for Modular CMA-ES Variants. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '20. To appear. Google ScholarDigital Library
- A. Jankovic, G. Popovski, T. Eftimov, and C. Doerr. 2021. The impact of hyper-parameter tuning for landscape-aware performance regression and algorithm selection. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '21. ACM, 687--696. Google ScholarDigital Library
- A. Jarvis, H.I. Reuter, A. Nelson, and E. Guevara. 2008. Hole-filled seamless SRTM data V4. International Centre for Tropical Agriculture (CIAT) (2008). https://srtm.csi.cgiar.orgGoogle Scholar
- G.A. Jastrebski and D.V. Arnold. 2006. Improving Evolution Strategies through Active Covariance Matrix Adaptation. In IEEE International Conference on Evolutionary Computation, CEC 2006, part of WCCI 2006. IEEE, 2814--2821. Google ScholarCross Ref
- J. Kennedy and R. Eberhart. 1995. Particle swarm optimization. In Proceedings of ICNN'95 - International Conference on Neural Networks, Vol. 4. 1942--1948 vol.4. Google ScholarCross Ref
- P. Kerschke, M. Preuss, S. Wessing, and H. Trautmann. 2015. Detecting Funnel Structures by Means of Exploratory Landscape Analysis. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '15. ACM, 265--272. Google ScholarDigital Library
- P. Kerschke, M. Preuss, S. Wessing, and H. Trautmann. 2016. Low-Budget Exploratory Landscape Analysis on Multiple Peaks Models. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '16. ACM, 229--236. Google ScholarDigital Library
- P. Kerschke and H. Trautmann. 2019. Comprehensive Feature-Based Landscape Analysis of Continuous and Constrained Optimization Problems Using the R-Package Flacco. In Applications in Statistical Computing: From Music Data Analysis to Industrial Quality Improvement. Springer, 93--123.Google Scholar
- M. López-Ibáñez, J. Dubois-Lacoste, L. Pérez Cáceres, M. Birattari, and T. Stützl. 2016. The irace package: Iterated racing for automatic algorithm configuration. Operations Research Perspectives 3 (2016), 43 -- 58.Google ScholarCross Ref
- N. S. C. Merleau and M. Smerlak. 2021. A simple evolutionary algorithm guided by local mutations for an eficient RNA design. In Genetic and Evolutionary Computation Conference, GECCO '21. ACM, 1027--1034. Google ScholarDigital Library
- O. Mersmann, B. Bischl, H. Trautmann, M. Preuss, C. Weihs, and G. Rudolph. 2011. Exploratory Landscape Analysis. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '11. ACM, 829--836. Google ScholarDigital Library
- L.J.V. Miranda. 2018. PySwarms: a research toolkit for Particle Swarm Optimization in Python. Journal of Open Source Software 3, 21 (2018), 433. Google ScholarCross Ref
- J. A. Nelder and R. Mead. 1965. A Simplex Method for Function Minimization. Comput. J. 7, 4 (01 1965), 308--313. arXiv:https://academic.oup.com/comjnl/article-pdf/7/4/308/1013182/7-4-308.pdf Google ScholarCross Ref
- F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 12 (2011), 2825--2830.Google ScholarDigital Library
- M. J. D. Powell. 1964. An eficient method for finding the minimum of a function of several variables without calculating derivatives. Comput. J. 7, 2 (01 1964), 155--162. Google ScholarCross Ref
- Q. Renau, C. Doerr, J. Dreo, and B. Doerr. 2020. Exploratory Landscape Analysis is Strongly Sensitive to the Sampling Strategy. In Parallel Problem Solving from Nature - PPSN XVI - 16th International Conference, PPSN 2020, Leiden, The Netherlands, September 5--9, 2020, Proceedings, Part II (Lecture Notes in Computer Science, Vol. 12270). Springer, 139--153. Google ScholarDigital Library
- Q. Renau, J. Dreo, C. Doerr, and B. Doerr. 2019. Expressiveness and robustness of landscape features. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO '19 (Companion). ACM, 2048--2051. Google ScholarDigital Library
- Q. Renau, J. Dreo, A. Peres, Y. Semet, C. Doerr, and B. Doerr. 2022. GECCO2022 Automated Algorithm Selection for Radar Network Configuration (Version V0) [Data set]. Google ScholarCross Ref
- I.M. Sobol'. 1967. On the distribution of points in a cube and the approximate evaluation of integrals. U. S. S. R. Comput. Math. and Math. Phys. 7, 4 (Jan. 1967), 86--112. Google ScholarCross Ref
- R. Storn and K.V. Price. 1997. Differential Evolution - A Simple and Efficient Heuristic for global Optimization over Continuous Spaces. Journal of Global Optimization 11, 4 (1997), 341--359. Google ScholarDigital Library
- S. van Rijn, H. Wang, M. van Leeuwen, and T. Bäck. 2016. Evolving the structure of Evolution Strategies. In 2016 IEEE Symposium Series on Computational Intelligence, SSCI 2016. IEEE, 1--8. Google ScholarCross Ref
- P. Virtanen, R. Gommers, T.E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S.J. van der Walt, M. Brett, J. Wilson, K.J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C J Carey, İ. Polat, Y. Feng, E. W. Moore, J. VanderPlas, D. Laxalde, J. Perktold, R. Cimrman, I. Henriksen, E. A. Quintero, C. R. Harris, A. M. Archibald, A. H. Ribeiro, F. Pedregosa, P. van Mulbregt, and SciPy 1.0 Contributors. 2020. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods 17 (2020), 261--272. Google ScholarCross Ref
- Hao Wang, Diederick Vermetten, Furong Ye, Carola Doerr, and Thomas Bäck. 2022. IOHanalyzer: Detailed Performance Analysis for Iterative Optimization Heuristic. ACM Transactions on Evolutionary Learning and Optimization (2022). To appear. Free version available at https://arxiv.org/abs/2007.03953.Google Scholar
Index Terms
- Automated algorithm selection for radar network configuration
Recommendations
Improving Nevergrad’s Algorithm Selection Wizard NGOpt Through Automated Algorithm Configuration
Parallel Problem Solving from Nature – PPSN XVIIAbstractAlgorithm selection wizards are effective and versatile tools that automatically select an optimization algorithm given high-level information about the problem and available computational resources, such as number and type of decision variables, ...
Per-run Algorithm Selection with Warm-Starting Using Trajectory-Based Features
Parallel Problem Solving from Nature – PPSN XVIIAbstractPer-instance algorithm selection seeks to recommend, for a given problem instance and a given performance criterion, one or several suitable algorithms that are expected to perform well for the particular setting. The selection is classically done ...
Limitations of benchmark sets and landscape features for algorithm selection and performance prediction
GECCO '19: Proceedings of the Genetic and Evolutionary Computation Conference CompanionBenchmark sets and landscape features are used to test algorithms and to train models to perform algorithm selection or configuration. These approaches are based on the assumption that algorithms have similar performances on problems with similar ...
Comments