Thomas Pierrot
ABSTRACT
In this work, we consider the problem of Quality-Diversity (QD) optimization with multiple objectives. QD algorithms have been proposed to search for a large collection of both diverse and high-performing solutions instead of a single set of local optima. Searching for diversity was shown to be useful in many industrial and robotics applications. On the other hand, most real-life problems exhibit several potentially conflicting objectives to be optimized. Hence being able to optimize for multiple objectives with an appropriate technique while searching for diversity is important to many fields. Here, we propose an extension of the map-elites algorithm in the multi-objective setting: Multi-Objective map-elites (mome). Namely, it combines the diversity inherited from the map-elites grid algorithm with the strength of multi-objective optimizations by filling each cell with a Pareto Front. As such, it allows to extract diverse solutions in the descriptor space while exploring different compromises between objectives. We evaluate our method on several tasks, from standard optimization problems to robotics simulations. Our experimental evaluation shows the ability of mome to provide diverse solutions while providing global performances similar to standard multi-objective algorithms.
- James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. 2018. JAX: composable transformations of Python+NumPy programs. http://github.com/google/jaxGoogle Scholar
- Dimo Brockhoff. 2010. Optimal μ-distributions for the hypervolume indicator for problems with linear bi-objective fronts: Exact and exhaustive results. In Asia-Pacific Conference on Simulated Evolution and Learning. Springer, 24--34.Google ScholarCross Ref
- Yongtao Cao, Byran J. Smucker, and Timothy J. Robinson. 2015. On using the hypervolume indicator to compare Pareto fronts: Applications to multi-criteria optimal experimental design. Journal of Statistical Planning and Inference 160 (2015), 60--74. Google ScholarCross Ref
- Konstantinos Chatzilygeroudis, Antoine Cully, Vassilis Vassiliades, and Jean-Baptiste Mouret. 2021. Quality-Diversity Optimization: a novel branch of stochastic optimization. In Black Box Optimization, Machine Learning, and No-Free Lunch Theorems. Springer, 109--135.Google Scholar
- CA Coello Coello and Maximino Salazar Lechuga. 2002. MOPSO: A proposal for multiple objective particle swarm optimization. In Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No. 02TH8600), Vol. 2. IEEE, 1051--1056.Google ScholarCross Ref
- Cédric Colas, Vashisht Madhavan, Joost Huizinga, and Jeff Clune. 2020. Scaling map-elites to deep neuroevolution. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference. 67--75.Google ScholarDigital Library
- Antoine Cully. 2020. Multi-Emitter MAP-Elites: Improving quality, diversity and convergence speed with heterogeneous sets of emitters. CoRR abs/2007.05352 (2020). arXiv:2007.05352 https://arxiv.org/abs/2007.05352Google Scholar
- Antoine Cully, Jeff Clune, Danesh Tarapore, and Jean-Baptiste Mouret. 2015. Robots that can adapt like animals. Nature 521, 7553 (2015), 503--507.Google Scholar
- Antoine Cully and Yiannis Demiris. 2017. Quality and diversity optimization: A unifying modular framework. IEEE Transactions on Evolutionary Computation 22, 2 (2017), 245--259.Google ScholarCross Ref
- Kalyanmoy Deb. 2014. Multi-objective optimization. In Search methodologies. Springer, 403--449.Google Scholar
- Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and TAMT Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE transactions on evolutionary computation 6, 2 (2002), 182--197.Google Scholar
- Kalyanmoy Deb, Karthik Sindhya, and Tatsuya Okabe. 2007. Self-adaptive simulated binary crossover for real-parameter optimization. In Proceedings of the 9th annual conference on genetic and evolutionary computation. 1187--1194.Google ScholarDigital Library
- Manon Flageat and Antoine Cully. 2020. Fast and stable MAP-Elites in noisy domains using deep grids. In Proceedings of the 2020 Conference on Artificial Life.Google ScholarCross Ref
- Matthew C Fontaine and Stefanos Nikolaidis. 2021. Differentiable Quality Diversity. arXiv e-prints (2021), arXiv-2106.Google Scholar
- Matthew C. Fontaine, Julian Togelius, Stefanos Nikolaidis, and Amy K. Hoover. 2019. Covariance Matrix Adaptation for the Rapid Illumination of Behavior Space. CoRR abs/1912.02400 (2019). arXiv:1912.02400 http://arxiv.org/abs/1912.02400Google Scholar
- C Daniel Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, and Olivier Bachern. 2021. Brax-A Differentiable Physics Engine for Large Scale Rigid Body Simulation. (2021).Google Scholar
- Ahmed Khalifa, Scott Lee, Andy Nealen, and Julian Togelius. 2018. Talakat: Bullet hell generation through constrained map-elites. In Proceedings of The Genetic and Evolutionary Computation Conference. 1047--1054.Google ScholarDigital Library
- Joshua D Knowles. 2002. Local-search and hybrid evolutionary algorithms for Pareto optimization. Ph. D. Dissertation. University of Reading Reading.Google Scholar
- Joel Lehman and Kenneth O Stanley. 2011. Abandoning objectives: Evolution through the search for novelty alone. Evolutionary computation 19, 2 (2011), 189--223.Google Scholar
- Joel Lehman and Kenneth O Stanley. 2011. Evolving a diversity of virtual creatures through novelty search and local competition. In Proceedings of the 13th annual conference on Genetic and evolutionary computation. 211--218.Google ScholarDigital Library
- Bryan Lim, Maxime Allard, Luca Grillotti, and Cully Antoine. 2022. Accelerated Quality-Diversity for Robotics through Massive Parallelism. CoRR abs/2202.01258 (2022). arXiv:2202.01258 http://arxiv.org/abs/2202.01258Google Scholar
- Liane Makatura, Minghao Guo, Adriana Schulz, Justin Solomon, and Wojciech Matusik. 2021. Pareto gamuts: exploring optimal designs across varying contexts. ACM Transactions on Graphics 40 (08 2021), 1--17. Google ScholarCross Ref
- Jean-Baptiste Mouret. 2011. Novelty-based multiobjectivization. In New horizons in evolutionary robotics. Springer, 139--154.Google Scholar
- Jean-Baptiste Mouret and Jeff Clune. 2015. Illuminating search spaces by mapping elites. arXiv preprint arXiv:1504.04909.Google Scholar
- Jean-Baptiste Mouret and Glenn Maguire. 2020. Quality diversity for multi-task optimization. In Proceedings of the 2020 Genetic and Evolutionary Computation Conference. 121--129.Google ScholarDigital Library
- Olle Nilsson and Antoine Cully. 2021. Policy gradient assisted MAP-Elites. Proceedings of the Genetic and Evolutionary Computation Conference.Google ScholarDigital Library
- Thomas Pierrot, Valentin Macé, Geoffrey Cideron, Karim Beguir, Antoine Cully, Olivier Sigaud, and Nicolas Perrin. 2021. Diversity Policy Gradient for Sample Efficient Quality-Diversity Optimization. (2021).Google Scholar
- Justin K Pugh, Lisa B Soros, Paul A Szerlip, and Kenneth O Stanley. 2015. Confronting the challenge of quality diversity. In Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation. 967--974.Google ScholarDigital Library
- Leonard Andreevič Rastrigin. 1974. Systems of extremal control. Nauka (1974).Google Scholar
- Danilo Vasconcellos Vargas, Junichi Murata, Hirotaka Takano, and Alexandre Cláudio Botazzo Delbem. 2015. General subpopulation framework and taming the conflict inside populations. Evolutionary computation 23, 1 (2015), 1--36.Google Scholar
- Vassilis Vassiliades, Konstantinos Chatzilygeroudis, and Jean-Baptiste Mouret. 2016. Scaling up map-elites using centroidal voronoi tessellations. arXiv preprint arXiv:1610.05729 (2016).Google Scholar
- Vassiiis Vassiliades and Jean-Baptiste Mouret. 2018. Discovering the elite hyper-volume by leveraging interspecies correlation. In Proceedings of the Genetic and Evolutionary Computation Conference. 149--156.Google ScholarDigital Library
- Frank Wilcoxon. 1992. Individual comparisons by ranking methods. In Breakthroughs in statistics. Springer, 196--202.Google Scholar
- Qingfu Zhang and Hui Li. 2007. MOEA/D: A multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on evolutionary computation 11, 6 (2007), 712--731.Google ScholarDigital Library
- Tianping Zhang, Yuanqi Li, Yifei Jin, and Jian Li. 2020. AutoAlpha: an Efficient Hierarchical Evolutionary Algorithm for Mining Alpha Factors in Quantitative Investment. arXiv preprint arXiv:2002.08245 (2020).Google Scholar
- Eckart Zitzler. 1999. Evolutionary algorithms for multiobjective optimization: Methods and applications. Vol. 63. Citeseer.Google Scholar
- Eckart Zitzler, Marco Laumanns, and Lothar Thiele. 2001. SPEA2: Improving the strength Pareto evolutionary algorithm. TIK-report 103 (2001).Google Scholar
- Eckart Zitzler and Lothar Thiele. 1999. Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach. IEEE transactions on Evolutionary Computation 3, 4 (1999), 257--271.Google Scholar
Index Terms
- Multi-objective quality diversity optimization
Recommendations
Evolutionary multi-objective optimisation by diversity control
CSR'06: Proceedings of the First international computer science conference on Theory and ApplicationsThis paper presents an improved multi-objective diversity control oriented genetic algorithm (MODCGA-II). The performance comparison between the MODCGA-II, a non-dominated sorting genetic algorithm II (NSGA-II) and an improved strength Pareto ...
Many-objective Optimization via Voting for Elites
GECCO '23 Companion: Proceedings of the Companion Conference on Genetic and Evolutionary ComputationReal-world problems are often comprised of many objectives and require solutions that carefully trade-off between them. Current approaches to many-objective optimization often require challenging assumptions, like knowledge of the importance/...
Improving the Data Efficiency of Multi-Objective Quality-Diversity through Gradient Assistance and Crowding Exploration
GECCO '23: Proceedings of the Genetic and Evolutionary Computation ConferenceQuality-Diversity (QD) algorithms have recently gained traction as optimisation methods due to their effectiveness at escaping local optima and capability of generating wide-ranging and high-performing solutions. Recently, Multi-Objective MAP-Elites (...
Comments