Abstract
This paper introduces a reinforcement learning technique with an internal reward for a multi-agent cooperation task. The proposed method is an extension of Q-learning which changes the ordinary (external) reward to the internal reward for agent-cooperation under the condition of no communication. To increase the certainty of the proposed methods, we theoretically investigate what values should be set to select the goal for the cooperation among agents. In order to show the effectiveness of the proposed method, we conduct the intensive simulation on the maze problem for the agent-cooperation task, and confirm the following implications: (1) the proposed method successfully enable agents to acquire cooperative behaviors while a conventional method fails to always acquire such behaviors; (2) the cooperation among agents according to their internal rewards is achieved no communication; and (3) the condition for the cooperation among any number of agent is indicated.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Yong-Jae Kim Kui-Hong Park and Jong-Hwan Kim. Modular q-learning based multi-agent cooperation for robot soccer. Robotics and Autonomous System, pages 3026–3033, 2015.
Michael Camara, Oliver Bonham-Carter, and Janyl Jumadinova. A multi-agent system with reinforcement learning agents for biomedical text mining. In Proceedings of the 6th ACM Conference on Bioinformatics, Computational Biology and Health Informatics, BCB ’15, pages 634–643, New York, NY, USA, 2015. ACM.
H. Iima and Y. Kuroe. Swarm reinforcement learning methods improving certainty of learning for a multi-robot formation problem. CEC, pages 3026–3033, May 2015.
Y. Ichikawa and K. Takadama. Designing internal reward of reinforcement learning agents in multi-step dilemma problem. Journal of Computational Intelligence and Intelligent Informatics, JACIII, 17(6):926–931, 2013.
M. Gini M. Elidrisi, N. Johnson and J. Crandall. Fast adaptive learning in repeated stochastic games by game abstraction. AAMAS, pages 1141–1148, May 2014.
Prabuchandran K. J., Hemanth Kumar A. N, and S. Bhatnagar. Multi-agent reinforcement learning for traffic signal control. In Intelligent Transportation Systems (ITSC), 2014 IEEE 17th International Conference on, pages 2529–2534, Oct 2014.
Katja Verbeeck Karl Tuyls and Tom Lenaerts. A selection-mutation model for q-learning in multi-agent systems. Robotics and Autonomous System, pages 3026–3033, May 2015.
Alessandro Lazaric Enrique Munoz de Cote and Marcello Restelli. Learning to cooperate in multi-agent social dilemmas. AAMAS, pages 783–785, May 2006.
Ming Tan. Multi-agent reinforcement learning: Independent vs. cooperative agents. In In Proceedings of the Tenth International Conference on Machine Learning, pages 330–337. Morgan Kaufmann, 1993.
N. Ono and K. Fukumoto, editors. Multi-agent reinforcement learning: A modular approach, 1996.
R.S. Sutton and A.G. Barto. Reinforcement Learning. Bradford Books/MIT Press, Cambridge, MA, 1998.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Uwano, F., Takadama, K. (2017). Communication-Less Cooperative Q-Learning Agents in Maze Problem. In: Leu, G., Singh, H., Elsayed, S. (eds) Intelligent and Evolutionary Systems. Proceedings in Adaptation, Learning and Optimization, vol 8. Springer, Cham. https://doi.org/10.1007/978-3-319-49049-6_33
Download citation
DOI: https://doi.org/10.1007/978-3-319-49049-6_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-49048-9
Online ISBN: 978-3-319-49049-6
eBook Packages: EngineeringEngineering (R0)