Skip to main content
SpringerLink
Log in

Cooperative Multi-Agent Learning: The State of the Art

  • Published:
Autonomous Agents and Multi-Agent Systems Aims and scope Submit manuscript

Abstract

Cooperative multi-agent systems (MAS) are ones in which several agents attempt, through their interaction, to jointly solve tasks or to maximize utility. Due to the interactions among the agents, multi-agent problem complexity can rise rapidly with the number of agents or their behavioral sophistication. The challenge this presents to the task of programming solutions to MAS problems has spawned increasing interest in machine learning techniques to automate the search and optimization process. We provide a broad survey of the cooperative multi-agent learning literature. Previous surveys of this area have largely focused on issues common to specific subareas (for example, reinforcement learning, RL or robotics). In this survey we attempt to draw from multi-agent learning work in a spectrum of areas, including RL, evolutionary computation, game theory, complex systems, agent modeling, and robotics. We find that this broad view leads to a division of the work into two categories, each with its own special issues: applying a single learner to discover joint solutions to multi-agent problems (team learning), or using multiple simultaneous learners, often one per agent (concurrent learning). Additionally, we discuss direct and indirect communication in connection with learning, plus open issues in task decomposition, scalability, and adaptive dynamics. We conclude with a presentation of multi-agent learning problem domains, and a list of multi-agent learning resources.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. D. H. Ackley and M. Littman, “Altruism in the evolution of communication,” in Artificial Life IV: Proceedings of the International Workshop on the Synthesis and Simulation of Living Systems, (3rd edn.), MIT Press, 1994.

  2. D. Andre, F. Bennett III, and J. Koza, “Discovery by genetic programming of a cellular automata rule that is better than any known rule for the majority classification problem,” in Genetic Programming 1996: Proceedings of the First Annual Conference, MIT Press, 1996.

  3. D. Andre and A. Teller, “Evolving team Darwin United,” in M. Asada and H. Kitano, (eds.), RoboCup-98: Robot Soccer World Cup II, Springer Verlag, 1999.

  4. P. Angeline and J. Pollack, “Competitive environments evolve better solutions for complex tasks,” in S. Forrest, (ed.), Proceedings of the Fifth International Conference on Genetic Algorithms (ICGA), Morgan Kaufmann: San Mateo, CA, pp. 264–270, 1993.

  5. W. Arthur (1994) ArticleTitle“Inductive reasoning and bounded rationality” Complex. Econ. Theory 84 IssueID2 406–411

    Google Scholar 

  6. T. Bäck, Evolutionary Algorithms in Theory and Practice: Evolutionary Straegies, Evolutionary Programming, and Genetic Algorithms, Oxford Press, 1996.

  7. T. Balch, Learning roles: Behavioral diversity in robot teams, Technical Report GIT-CC-97-12, Georgia Institute of Technology, 1997.

  8. T. Balch, Behavioral Diversity in Learning Robot Teams, PhD thesis, College of Computing, Georgia Institute of Technology, 1998.

  9. T. Balch, “Reward and diversity in multirobot foraging,” in IJCAI-99 Workshop on Agents Learning About, From and With other Agents, 1999.

  10. B. Banerjee, R. Mukherjee, and S. Sen. “Learning mutual trust,” in Working Notes of AGENTS-00 Workshop on Deception, Fraud and Trust in Agent Societies, pp. 9–14, 2000.

  11. A. Barto, R. Sutton, and C. Watkins, “Learning and sequential decision making,” in M. Gabriel and J. Moore, (eds.), Learning and Computational Neuroscience: Foundations of Adaptive Networks, MIT Press: Cambridge, MA, 1990.

  12. J. Bassett and K. De Jong, “Evolving behaviors for cooperating agents,” in Z. Ras, (ed.), Proceedings from the Twelfth International Symposium on Methodologies for Intelligent Systems, Springer-Verlag: Charlotte, NC, pp. 157–165, 2000.

  13. J. K. Bassett (2002) A study of generalization techniques in evolutionary rule learning George Mason University Fairfax VA, USA

    Google Scholar 

  14. R. Beckers, O. E. Holland, and J.-L. Deneubourg. “From local actions to global tasks: Stigmergy and collective robotics,” in Artificial Life IV: Proceedings of the International Workshop on the Synthesis and Simulation of Living Systems, (3rd edn.), MIT Press, 1994.

  15. M. Benda, V. Jagannathan, and R. Dodhiawala, On optimal cooperation of knowledge sources - an empirical investigation, Technical Report BCS-G2010-28, Boeing Advanced Technology Center, Boeing Computer Services, 1986.

  16. H. Berenji and D. Vengerov, “Advantages of cooperation between reinforcement learning agents in difficult stochastic problems,” in Proceedings of 9th IEEE International Conference on Fuzzy Systems, 2000.

  17. H. Berenji and D. Vengerov, Learning, cooperation, and coordination in multi-agent systems, Technical Report IIS-00-10, Intelligent Inference Systems Corp., 333 W. Maude Avennue, Suite 107, Sunnyvale, CA 94085-4367, 2000.

  18. D. Bernstein, S. Zilberstein, and N. Immerman, “The complexity of decentralized control of MDPs,” in Proceedings of UAI-2000: The Sixteenth Conference on Uncertainty in Artificial Intelligence, 2000.

  19. H. J. Blumenthal and G. Parker, “Co-evolving team capture strategies for dissimilar robots,” in’Proceedings of Artificial Multiagent Learning. Papers from the 2004 AAAI Fall Symposium. Technical Report FS-04-02, 2004.

  20. E. Bonabeau, M. Dorigo, and G. Theraulaz, Swarm Intelligence: From Natural to Artificial Systems, SFI Studies in the Sciences of Complexity, Oxford University Press, 1999.

  21. J. C. Bongard, “The legion system: A novel approach to evolving heterogeneity for collective problem solving” in R. Poli, W. Banzhaf, W. B. Langdon, J. F. Miller, P. Nordin, and T. C. Fogarty, (eds.), Genetic Programming: Proceedings of EuroGP-2000. Vol. 1802, Edinburgh, 15–16 2000. Springer-Verlag. ISBN 3-540-67339-3, pp. 16–28.

  22. C. Boutilier, “Learning conventions in multiagent stochastic domains using likelihood estimates,” in’Uncertainty in Artificial Intelligence, pp. 106–114, 1996.

  23. C. Boutilier, “Planning, learning and coordination in multiagent decision processes,” in Proceedings of the Sixth Conference on Theoretical Aspects of Rationality and Knowledge (TARK96), pp. 195–210, 1996.

  24. M. Bowling, “Convergence problems of general-sum multiagent reinforcement learning,” in’Proceedings of the Seventeenth International Conference on Machine Learning, Morgan Kaufmann: San Francisco, CA, pp. 89–94, 2000.

  25. M. Bowling, Multiagent Learning in the Presence of Agents with Limitations, PhD thesis, Computer Science Department, Carnegie Mellon University, 2003.

  26. M. Bowling and M. Veloso, An analysis of stochastic game theory for multiagent reinforcement learning, Technical Report CMU-CS-00–165, Computer Science Department, Carnegie Mellon University, 2000.

  27. M. Bowling and M. Veloso, “Rational and convergent learning in stochastic games,” in Proceedings of Seventeenth International Joint Conference on Artificial Intelligence (IJCAI-01), pp. 1021–1026, 2001.

  28. M. Bowling and M. Veloso, Existence of multiagent equilibria with limited agents, Technical Report CMU-CS-02-104, Computer Science Department, Carnegie Mellon University, 2002.

  29. M. Bowling M. Veloso (2002) ArticleTitle“Multiagent learning using a variable learning rate” Artif. Intell 136 IssueID2 215–250 Occurrence Handle10.1016/S0004-3702(02)00121-2

    Article  Google Scholar 

  30. J. A. Boyan and M. Littman, “Packet routing in dynamically changing networks: A reinforcement learning approach,” in J. D. Cowan, G. Tesauro, and J. Alspector, (eds.), Advances in Neural Information Processing Systems, Vol. 6, Morgan Kaufmann, pp. 671–678, 1994.

  31. R. Brafman and M. Tennenholtz, “Efficient learning equilibrium,” in Advances in Neural Information Processing Systems (NIPS-2002), 2002.

  32. W. Brauer and G. Weiß, “Multi-machine scheduling - a multi-agent learning approach,” in Proceedings of the Third International Conference on Multi-Agent Systems, pp. 42–48, 1998.

  33. P. Brazdil, M. Gams, S. Sian, L. Torgo, and W. van de Velde, “Learning in distributed systems and multi-agent environments,” in Y. Kodratoff, (ed.), Lecture Notes in Artificial Intelligence, Vol. 482, Springer-Verlag, pp. 412–423, 1991.

  34. O. Buffet A. Dutech F. Charpillet (2001) “Incremental reinforcement learning for designing multi-agent systems” J.P. Müller E. Andre S. Sen C. Frasson (Eds) Proceedings of the Fifth International Conference on Autonomous Agents ACM Press Montreal, Canada 31–32

    Google Scholar 

  35. O. Buffet, A. Dutech, and F. Charpillet, “Learning to weigh basic behaviors in scalable agents,” in Proceedings of the 1st International Joint Conference on Autonomous Agents and MultiAgent Systems (AAMAS’02), 2002.

  36. H. Bui, S. Venkatesh, and D. Kieronska, “A framework for coordination and learning among team of agents,” in W. Wobcke, M. Pagnucco, and C. Zhang, (eds.), Agents and Multi-Agent Systems: Formalisms, Methodologies and Applications, Lecture Notes in Artificial Intelligence. Vol. 1441, Springer-Verlag, pp. 164–178, 1998.

  37. H. Bui S. Venkatesh D. Kieronska (1999) ArticleTitle“Learning other agents’ preferences in multi-agent negotiation using the Bayesian classifier” Int. J. Coop. Inform. Syst. 8 IssueID4 275–294 Occurrence Handle10.1142/S0218843099000149

    Article  Google Scholar 

  38. L. Bull, “Evolutionary computing in multi-agent environments: Partners,” in T. Back, (ed.), Proceedings of the Seventh International Conference on Genetic Algorithms, Morgan Kaufmann, pp. 370–377, 1997.

  39. L. Bull, “Evolutionary computing in multi-agent environments: Operators,” in D. W. V. W. Porto, N. Saravanan, and A. E. Eiben, (eds.), Proceedings of the Seventh Annual Conference on Evolutionary Programming, Springer Verlag, pp. 43–52, 1998.

  40. L. Bull and T. C. Fogarty, “Evolving cooperative communicating classifier systems”, in A. V. Sebald and L. J. Fogel, (eds.), Proceedings of the Fourth Annual Conference on Evolutionary Programming (EP94), pp. 308–315, 1994.

  41. L. Bull and O. Holland, “Evolutionary computing in multiagent environments: Eusociality”, in Proceedings of Seventh Annual Conference on Genetic Algorithms, 1997.

  42. A. Cangelosi (2001) ArticleTitle“Evolution of communication and language using signals, symbols, and words” IEEE Trans. Evol. Comput. 5 IssueID2 93–101 Occurrence Handle10.1109/4235.918429

    Article  Google Scholar 

  43. Y.U. Cao A.S. Fukunaga A.B. Kahng (1997) ArticleTitle“Cooperative mobile robotics: Antecedents and directions” Auton. Robots 4 IssueID1 7–23 Occurrence Handle10.1023/A:1008855018923

    Article  Google Scholar 

  44. D. Carmel, Model-based Learning of Interaction Strategies in Multi-agent systems, PhD thesis, Technion - Israel Institute of Technology, 1997.

  45. D. Carmel and S. Markovitch, The M* algorithm: Incorporating opponent models into adversary search. Technical Report 9402, Technion - Israel Institute of Technology, March 1994

  46. L.-E. Cederman, Emergent Actors in World Politics: How States and Nations Develop and Dissolve, Princeton University Press, 1997

  47. G. Chalkiadakis and C. Boutilier, “Coordination in multiagent reinforcement learning: A Bayesian approach,” in Proceedings of The Second International Joint Conference on Autonomous Agents & Multiagent Systems (AAMAS 2003). ACM, 2003. ISBN 1-58113-683-8

  48. H. Chalupsky, Y. Gil, C. A. Knoblock, K. Lerman, J. Oh, D. Pynadath, T. Russ, and M. Tambe, “Electric elves: Agent technology for supporting human organizations,” in AI Magazine - Summer 2002, AAAI Press, 2002

  49. Y.-H. Chang, T. Ho, and L. Kaelbling, “All learning is local: Multi-agent learning in global reward games,” in Proceedings of Neural Information Processing Systems (NIPS-03), 2003

  50. Y.-H. Chang, T. Ho, and L. Kaelbling, “Multi-agent learning in mobilized ad-hoc networks,” in Proceedings of Artificial Multiagent Learning, Papers from the 2004 AAAI Fall Symposium, Technical Report FS-04-02, 2004

  51. C. Claus and C. Boutilier, “The dynamics of reinforcement learning in cooperative multiagent systems,” in Proceedings of National Conference on Artificial Intelligence AAAI/IAAI, pp. 746–752, 1998

  52. D. Cliff and G. F. Miller, “Tracking the red queen: Measurements of adaptive progress in co-evolutionary simulations”, in Proceedings of the Third European Conference on Artificial Life, Springer-Verlag, pp. 200–218, 1995

  53. R. Collins D. Jefferson (1991) “An artificial neural network representation for artificial organisms” H.-P. Schwefel R. M´′anner (Eds) Parallel Problem Solving from Nature: 1st Workshop (PPSN I) Springer-Verlag Berlin 259–263

    Google Scholar 

  54. R. Collins D. Jefferson (1992) “AntFarm: Towards simulated evolution” C. Langton C. Taylor J.D. Farmer S. Rasmussen (Eds) Artificial Life II Addison-Wesley Redwood City, CA 579–601

    Google Scholar 

  55. E. Crawford and M. Veloso, “Opportunities for learning in multi-agent meeting scheduling”, in Proceedings of Artificial Multiagent Learning, Papers from the 2004 AAAI Fall Symposium. Technical Report FS-04-02, 2004

  56. V. Crespi, G. Cybenko, M. Santini, and D. Rus. Decentralized control for coordinated flow of multi-agent systems. Technical Report TR2002-414, Dartmouth College, Computer Science, Hanover, NH, January 2002

  57. R. H. Crites, Large-Scale Dynamic Optimization Using Teams of Reinforcement Learning Agents, PhD thesis, University of Massachusetts Amherst, 1996

  58. M.R. Cutkosky R.S. Englemore R.E. Fikes M.R. Genesereth T.R. Gruber W.S. Mark J.M. Tenenbaum J.C. Weber (1997) “PACT: An experiment in integrating concurrent engineering systems” M.N. Huhns M.P. Singh (Eds) Readings in Agents Morgan Kaufmann San Francisco, CA, USA 46–55

    Google Scholar 

  59. T. Dahl, M. Mataric, and G. Sukhatme, “Adaptive spatio-temporal organization in groups of robots,” in Proceedings of the 2002 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS-02), 2002

  60. R. Das, M. Mitchell, and J. Crutchfield, “A genetic algorithm discovers particle-based computation in cellular automata”, in Parallel Problem Solving from Nature III, LNCS 866, Springer-Verlag, pp.’344–353, 1994

  61. J. Davis and G. Kendall, “An investigation, using co-evolution, to evolve an awari player,” in Proceedings of 2002 Congress on Evolutionary Computation (CEC2002), 2002

  62. B. de Boer, “Generating vowel systems in a population of agents,” in Proceedings of the Fourth European Conference Artificial Life, MIT Press, 1997

  63. K. Jong ParticleDe (1975) An Analysis of the Behavior of a Class of Genetic Adaptive Systems University of Michigan Ann Arbor, MI

    Google Scholar 

  64. K. De Jong, Evolutionary Computation: A Unified Approach, MIT Press, 2005

  65. K. Decker, E. Durfee, and V. Lesser, “Evaluating research in cooperative distributed problem solving,” in L. Gasser and M. Huhns, (eds.), Distributed Artificial Intelligence Volume II, Pitman Publishing and Morgan Kaufmann, pp. 487–519, 1989

  66. K. Decker M. Fisher M. Luck M. Tennenholtz InstitutionalAuthorNameUKMAS’98 Contributors (1999) ArticleTitle“Continuing research in multi-agent systems” Knowl. Eng. Rev. 14 IssueID3 279–283 Occurrence Handle10.1017/S026988899900301X

    Article  Google Scholar 

  67. J. L. Deneubourg, S. Goss, N. Franks, A. Sendova-Franks, C. Detrain, and L. Chretien, “The dynamics of collective sorting: Robot-like ants and ant-like robots,” in From Animals to Animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior, MIT Press, pp.’356–363, 1991

  68. J. Denzinger and M. Fuchs, “Experiments in learning prototypical situations for variants of the pursuit game,” in Proceedings on the International Conference on Multi-Agent Systems (ICMAS-1996), pp. 48–55, 1996

  69. M. Dowell. Learning in Multiagent Systems, PhD thesis, University of South Carolina, 1995

  70. K. Dresner and P. Stone, “Multiagent traffic management: A reservation-based intersection control mechanism,” in AAMAS-2004 - Proceedings of the Third International Joint Conference on Autonomous Agents and Multi Agent Systems, 2004

  71. G. Dudek, M. Jenkin, R. Milios, and D. Wilkes, “A taxonomy for swarm robots,” in Proceedings of IEEE/RSJ Conference on Intelligent Robots and Systems, 1993

  72. E. Durfee, “What your computer really needs to know, you learned in kindergarten,” in National Conference on Artificial Intelligence, pp. 858–864, 1992

  73. E. Durfee V. Lesser D. Corkill (1987) ArticleTitle“Coherent cooperation among communicating problem solvers” IEEE Trans. Comput C-36 IssueID11 1275–1291

    Google Scholar 

  74. E. Durfee V. Lesser D. Corkill (March 1989) ArticleTitleTrends in cooperative distributed problem solving IEEE Trans. Knowl. Data Eng. KDE-1 IssueID1 63–83 Occurrence Handle10.1109/69.43404

    Article  Google Scholar 

  75. A. Dutech, O. Buffet, and F. Charpillet, “Multi-agent systems by incremental gradient reinforcement learning,” in Proceedings of Seventeenth International Joint Conference on Artificial Intelligence (IJCAI-01), pp. 833–838, 2001

  76. F. Fernandez L. Parker (2001) ArticleTitle“Learning in large cooperative multi-robot domains,” Int. J. Robot. Autom. 16 IssueID4 217–226

    Google Scholar 

  77. S. Ficici J. Pollack (1998) “Challenges in coevolutionary learning: Arms-race dynamics, open-endedness, and mediocre stable states” C. Adami (Eds) et al. Proceedings of the Sixth International Conference on Artificial Life MIT Press Cambridge, MA 238–247

    Google Scholar 

  78. S. Ficici and J. Pollack, “A game-theoretic approach to the simple coevolutionary algorithm”, in Proceedings of the Sixth International Conference on Parallel Problem Solving from Nature (PPSN VI). Springer Verlag, 2000

  79. K. Fischer, N. Kuhn, H. J. Muller, J. P. Muller, and M. Pischel, “Sophisticated and distributed: The transportation domain,” in Proceedings of the Fifth European Workshop on Modelling Autonomous Agents in a Multi-Agent World (MAAMAW’93), 1993

  80. D. Fogel, Blondie24: Playing at the Edge of Artificial Intelligence, Morgan Kaufmann, 2001. ISBN 1-55860-783-8

  81. L. Fogel, Intelligence Through Simulated Evolution: Forty Years of Evolutionary Programming, Wiley Series on Intelligent Systems, 1999

  82. D. Fudenberg and D. Levine, The Theory of Learning in Games, MIT Press, 1998

  83. A. Garland R. Alterman (2004) ArticleTitle“Autonomous agents that learn to better coordinate” Auton. Agents Multi-Agent Syst. 8 267–301 Occurrence Handle10.1023/B:AGNT.0000018808.95119.9e

    Article  Google Scholar 

  84. M. Ghavamzadeh and S. Mahadevan, “Learning to communicate and act using hierarchical reinforcement learning,” in AAMAS-2004 - Proceedings of the Third International Joint Conference on Autonomous Agents and Multi Agent Systems, 2004

  85. N. Glance B. Huberman (1994) ArticleTitle“The dynamics of social dilemmas” Sci. Am. 270 IssueID3 76–81

    Google Scholar 

  86. P. Gmytrasiewicz, A Decision-Theoretic Model of Coordination and Communication in Autonomous Systems (Reasoning Systems), PhD thesis, University of Michigan, 1992

  87. D. E. Goldberg (1989) Genetic Algorithms in Search, Optimization, and Machine Learning Addison Wesley Reading, MA

    Google Scholar 

  88. C. Goldman J. Rosenschein (1996) “Mutually supervised learning in multiagent systems” G. Weiß S. Sen (Eds) Adaptation and Learning in Multi-Agent Systems Springer-Verlag Heidelberg, Germany. Berlin 85–96

    Google Scholar 

  89. B. M. Good, Evolving multi-agent systems: Comparing existing approaches and suggesting new directions, Master’s thesis, University of Sussex, 2000

  90. M. Gordin, S. Sen, and N. Puppala, “Evolving cooperative groups: Preliminary results”, in Working Papers of the AAAI-97 Workshop on Multiagent Learning, pp. 31–35, 1997

  91. S. Grand D. Cliff (1998) ArticleTitle“Creatures: Entertainment software agents with artificial life” Auton. Agents Multi-Agent Syst. 1 IssueID1 39–57 Occurrence Handle10.1023/A:1010042522104

    Article  Google Scholar 

  92. S. Grand, D. Cliff, and A. Malhotra, “Creatures : Artificial life autonomous software agents for home entertainment”, in Proceedings of the First International Conference on Autonomous Agents (Agents-97), pp. 22–29, 1997

  93. D. L. Grecu, Using Learning to Improve Multi-Agent Systems for Design. PhD thesis, Worcester Polytechnic Institute, 1997

  94. A. Greenwald, J. Farago, and K. Hall, “Fair and efficient solutions to the Santa Fe bar problem,” in’Proceedings of the Grace Hopper Celebration of Women in Computing 2002, 2002

  95. A. Greenwald and K. Hall, “Correlated Q-learning,” in Proceedings of the Twentieth International Conference on Machine Learning, 2003

  96. J. Grefenstette (1991) “Lamarckian learning in multi-agent environments” R. Belew L. Booker (Eds) Proceedings of the Fourth International Conference on Genetic Algorithms Morgan Kaufman San Mateo, CA 303–310

    Google Scholar 

  97. J. Grefenstette C.L. Ramsey A. Schultz (1990) ArticleTitle“Learning sequential decision rules using simulation models and competition” Machine Learn. 5 355–381

    Google Scholar 

  98. C. Guestrin, M. Lagoudakis, and R. Parr, “Coordinated reinforcement learning,” in Proceedings of the 2002 AAAI Symposium Series: Collaborative Learning Agents, 2002

  99. S.M. Gustafson (2000) Layered learning in genetic programming for a co-operative robot soccer problem Kansas State University Manhattan, KS, USA

    Google Scholar 

  100. S.M. Gustafson W. H. Hsu (2001) “Layered learning in genetic programming for a co-operative robot soccer problem,” J. F. Miller M. Tomassini P. L. Lanzi C. Ryan A. G. B. Tettamanzi W. B. Langdon (Eds) Genetic Programming: Proceedings of EuroGP-2001 NumberInSeries Vol. 2038 Springer-Verlag Lake Como, Italy 291–301

    Google Scholar 

  101. A. Hara and T. Nagao, “Emergence of cooperative behavior using ADG; Automatically Defined Groups,” in Proceedings of the 1999 Genetic and Evolutionary Computation Conference (GECCO-99), pp. 1038–1046, 1999

  102. I. Harvey, P. Husbands, D. Cliff, A. Thompson, and N. Jakobi, “Evolutionary robotics: The Sussex approach,” Robot. Auton. Syst., 1996

  103. T. Haynes, K. Lau, and S. Sen, “Learning cases to compliment rules for conflict resolution in multiagent systems,” in S. Sen, (ed.), AAAI Spring Symposium on Adaptation, Coevolution, and Learning in Multiagent Systems, pp. 51–56, 1996

  104. Haynes T. S. Sen (1995) “Evolving behavioral strategies in predators and prey” G. Weiß S. Sen (Eds) Adaptation and Learning in Multiagent Systems, Lecture Notes in Artificial Intelligence Springer Verlag Berlin, Germany

    Google Scholar 

  105. T. Haynes and S. Sen, “Adaptation using cases in cooperative groups,” in I. Imam (ed.), Working Notes of the AAAI-96 Workshop on Intelligent Adaptive Agents, Portland, OR, 1996

  106. T. Haynes and S. Sen, Cooperation of the fittest, Technical Report UTULSA-MCS-96-09, The University of Tulsa, Apr. 12, 1996

  107. T. Haynes and S. Sen, “Learning cases to resolve conflicts and improve group behavior,” in M. Tambe and P. Gmytrasiewicz, (eds.), Working Notes of the AAAI-96 Workshop on Agent Modeling, Portland, OR, pp. 46–52, 1996

  108. T. Haynes and S. Sen, “Crossover operators for evolving a team,” in J. R. Koza, K. Deb, M. Dorigo, D. B. Fogel, M. Garzon, H. Iba, and R. L. Riolo, (eds.), Genetic Programming 1997: Proceedings of the Second Annual Conference, Morgan Kaufmann: Stanford University, CA, USA, pp. 162–167, 13–16 July 1997

  109. T. Haynes, S. Sen, D. Schoenefeld, and R. Wainwright, “Evolving a team,” in E. V. Siegel and J. R. Koza, (eds.), Working Notes for the AAAI Symposium on Genetic Programming, AAAI: MIT, Cambridge, MA, USA, pp. 23–30, 10–12 Nov. 1995

  110. T. Haynes, S. Sen, D. Schoenefeld, and R. Wainwright, Evolving multiagent coordination strategies with genetic programming, Technical Report UTULSA-MCS-95–04, The University of Tulsa, May 31, 1995

  111. T. Haynes, R. Wainwright, S. Sen, and D. Schoenefeld, “Strongly typed genetic programming in evolving cooperation strategies,” in L. Eshelman, (ed.), Genetic Algorithms: Proceedings of the Sixth International Conference (ICGA95), Morgan Kaufmann: Pittsburgh, PA, USA, pp. 271–278, ISBN 1-55860-370-0, 15-19 July 1995

  112. T. D. Haynes and S. Sen, “Co-adaptation in a team,” Int. J. Comput. Intell. Org. (IJCIO), 1997

  113. D. Hillis (1991) ArticleTitle“Co-evolving parasites improve simulated evolution as an optimization procedure,” Artif. Life II, SFI Stud. Sci. Complex. 10 313–324

    Google Scholar 

  114. J. Holland (1975) Adaptation in Natural and Artificial Systems The MIT Press Cambridge, MA

    Google Scholar 

  115. J. Holland, “Properties of the bucket brigade,” in Proceedings of an International Conference on Genetic Algorithms, 1985

  116. B. Hölldobler and E. O. Wilson, The Ants, Harvard University Press, 1990

  117. W. H. Hsu and S. M. Gustafson, “Genetic programming and multi-agent layered learning by reinforcements,” in W. B. Langdon, E. Cantú-Paz, K. Mathias, R. Roy, D. Davis, R. Poli, K.’Balakrishnan, V. Honavar, G. Rudolph, J. Wegener, L. Bull, M. Potter, A. C. Schultz, J. F. Miller, E. Burke, and N. Jonoska, (eds.), GECCO 2002: Proceedings of the Genetic and Evolutionary Computation Conference, Morgan Kaufmann Publishers: New York, 9-13 July 2002, ISBN 1-55860-878-8, pp. 764-771

  118. J. Hu and M. Wellman, “Self-fulfilling bias in multiagent learning,” in Proceedings of the Second International Conference on Multi-Agent Systems, 1996

  119. J. Hu and M. Wellman, “Multiagent reinforcement learning: Theoretical framework and an algorithm,” in Proceedings of the Fifteenth International Conference on Machine Learning, Morgan Kaufmann: San Francisco, CA, pp. 242–250, 1998

  120. J. Hu and M. Wellman, “Online learning about other agents in a dynamic multiagent system,” in K.’P. Sycara and M. Wooldridge, (eds.), Proceedings of the Second International Conference on Autonomous Agents (Agents’98), ACM Press: New York, 1998, pp. 239–246, ISBN 0-89791-983-1

  121. J. Hu M. Wellman (2003) ArticleTitle“Nash Q-learning for general-sum stochastic games” J. Machine Learn. Res. 4 1039–1069 Occurrence Handle10.1162/1532443041827880

    Article  Google Scholar 

  122. J. Huang N. R. Jennings J. Fox (1995) “An agent architecture for distributed medical care,” M. Wooldridge N. R. Jennings (Eds) Intelligent Agents: Theories, Architectures, and Languages (LNAI Volume 890) Springer-Verlag Heidelberg, Germany 219–232

    Google Scholar 

  123. M. Huhns and M. Singh, “Agents and multiagent systems: Themes, approaches and challenges,” in’M. Huhns and M. Singh, (eds.), Readings in Agents, Morgan Kaufmann, pp. 1–23, 1998

  124. M. Huhns and G. Weiß, “Special issue on multiagent learning,” Machine Learn. J., vol. 33, nos. 2–3, 1998

  125. H. Iba, “Emergent cooperation for multiple agents using genetic programming,” in H.-M. Voigt, W. Ebeling, I. Rechenberg, and H.-P. Schwefel, (eds.), Parallel Problem Solving from Nature IV: Proceedings of the International Conference on Evolutionary Computation, Vol. 1141 of LNCS, Springer Verlag: Berlin, Germany, 1996, pp. 32–41, ISBN 3-540-61723-X

  126. H. Iba, “Evolutionary learning of communicating agents,” Inform. Sci., vol. 108, 1998

  127. H. Iba (1999) “Evolving multiple agents by genetic programming,” L. Spector W. Langdon U.-M. O’Reilly P. Angeline (Eds) Advances in Genetic Programming 3 The MIT Press Cambridge, MA 447–466

    Google Scholar 

  128. I. Imam, (ed.), Intelligent Adaptive Agents. Papers from the 1996 AAAI Workshop. Technical Report WS-96-04, AAAI Press, 1996

  129. A. Ito, “How do selfish agents learn to cooperate?,” in Artificial Life V: Proceedings of the Fifth International Workshop on the Synthesis and Simulation of Living Systems, MIT Press, pp. 185–192, 1997

  130. T. Jansen and R. P. Wiegand, “Exploring the explorative advantage of the cooperative coevolutionary (1+1) EA,” in E. Cantu-Paz et al., (ed.), Prooceedings of the Genetic and Evolutionary Computation Conference (GECCO), Springer-Verlag, 2003

  131. N. Jennings K. Sycara M. Wooldridge (1998) ArticleTitle“A roadmap of agents research and development” Auton Agents Multi-Agent Syst. 1 7–38 Occurrence Handle10.1023/A:1010090405266

    Article  Google Scholar 

  132. N. Jennings L. Varga R. Aarnts J. Fuchs P. Skarek (1993) ArticleTitle“Transforming standalone expert systems into a community of cooperating agents” Int. J. Eng. Appl. Artif. Intell. 6 IssueID4 317–331 Occurrence Handle10.1016/0952-1976(93)90016-Q

    Article  Google Scholar 

  133. K.-C. Jim C.L. Giles (2000) ArticleTitle“Talking helps: Evolving communicating agents for the predator-prey pursuit problem” Artif. Life 6 IssueID3 237–254 Occurrence Handle10.1162/106454600568861 Occurrence Handle11224918

    Article  PubMed  Google Scholar 

  134. H. Juille and J. Pollack, “Coevolving the “ideal” trainer: Application to the discovery of cellular automata rules”, in Proceedings of the Third Annual Genetic Programming Conference (GP-98), 1998

  135. L. Kaelbling M. Littman A. Moore (1996) ArticleTitle“Reinforcement learning: A survey” J. Artif. Intell. Res. 4 237–285

    Google Scholar 

  136. S. Kapetanakis and D. Kudenko, Improving on the reinforcement learning of coordination in cooperative multi-agent systems, in Proceedings of the Second Symposium on Adaptive Agents and Multi-agent Systems (AISB02), 2002

  137. S. Kapetanakis and D. Kudenko, “Reinforcement learning of coordination in cooperative multi-agent systems”, in Proceedings of the Nineteenth National Conference on Artificial Intelligence (AAAI02), 2002

  138. G. Kendall and G. Whitwell, “An evolutionary approach for the tuning of a chess evaluation function using population dynamics,” in Proceedings of the 2001 Congress on Evolutionary Computation (CEC-2001), IEEE Press, pp. 995–1002, 27–30, 2001

  139. G. Kendall and M. Willdig, “An investigation of an adaptive poker player”, in Proceedings of the 14th Australian Joint Conference on Artificial Intelligence (AI’01), 2001

  140. H. Kitano, M. Asada, Y. Kuniŷoshi, I. Noda, and E. Osawa, “RoboCup: The robot world cup initiative,” in W. L. Johnson and B. Hayes-Roth, (eds.), Proceedings of the First International Conference on Autonomous Agents (Agents’97), ACM Press: New York, 5-8, ISBN 0-89791-877-0, pp.’340-347, 1997

  141. J. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection, MIT Press, 1992

  142. M. Lauer and M. Riedmiller, “An algorithm for distributed reinforcement learning in cooperative multi-agent systems,” in Proceedings of the Seventeenth International Conference on Machine Learning, Morgan Kaufmann: San Francisco, CA, pp. 535–542, 2000

  143. L. R. Leerink, S. R. Schultz, and M. A. Jabri, “A reinforcement learning exploration strategy based on ant foraging mechanisms,” in Proceedings of the Sixth Australian Conference on Neural Networks, Sydney, Australia, 1995

  144. V. Lesser (1999) ArticleTitle“Cooperative multiagent systems : A personal view of the state of the art” IEEE Trans. Knowl. Data Eng. 11 IssueID1 133–142 Occurrence Handle10.1109/69.755622

    Article  Google Scholar 

  145. V. Lesser, D. Corkill, and E. Durfee, An update on the distributed vehicle monitoring testbed, Technical Report UM-CS-1987-111, University of Massachessets Amherst, 1987

  146. M. I. Lichbach, The Cooperator’s Dilemma, University of Michigan Press, 1996. ISBN 0472105728

  147. M. Littman, “Markov games as a framework for multi-agent reinforcement learning”, in Proceedings of the 11th International Conference on Machine Learning (ML-94), Morgan Kaufmann: New Brunswick, NJ, pp. 157–163, 1994

  148. M. Littman, “Friend-or-foe Q-learning in general-sum games,” in Proceedings of the Eighteenth International Conference on Machine Learning, Morgan Kaufmann, pp. 322–328, 2001

  149. A. Lubberts and R. Miikkulainen, “Co-evolving a go-playing neural network,” in Coevolution: Turning Adaptive Algorithms upon Themselves, (Birds-on-a-Feather Workshop, Genetic and Evolutionary Computation Conference), 2001

  150. M. Luck M. d’Inverno M. Fisher InstitutionalAuthorNameFoMAS’97 Contributors (1998) ArticleTitle“Foundations of multi-agent systems: Techniques, tools and theory” Knowl. Eng. Rev. 13 IssueID3 297–302 Occurrence Handle10.1017/S0269888998003014

    Article  Google Scholar 

  151. S. Luke, “Genetic programming produced competitive soccer softbot teams for RoboCup97,” in J. R. Koza et al, (ed.), Genetic Programming 1998: Proceedings of the Third Annual Conference, Morgan Kaufmann, pp. 214–222, 1998

  152. S. Luke, C. Hohn, J. Farris, G. Jackson, and J. Hendler, “Co-evolving soccer softbot team coordination with genetic programming,” in Proceedings of the First International Workshop on RoboCup, at’the International Joint Conference on Artificial Intelligence, Nagoya, Japan, 1997

  153. S. Luke and L. Spector, “Evolving teamwork and coordination with genetic programming,” in J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo, (eds.), Genetic Programming 1996: Proceedings of the First Annual Conference, MIT Press: Stanford University, CA, USA, pp. 150–156, 28-31 1996

  154. S. Luke, K. Sullivan, G. C. Balan, and L. Panait, Tunably decentralized algorithms for cooperative target observation, Technical Report GMU-CS-TR-2004-1, Department of Computer Science, George Mason University, 2004

  155. S. Luke and R. P. Wiegand, “Guaranteeing coevolutionary objective measures”, in Poli et al. [201], pp. 237–251

  156. S. Mahadevan and J. Connell, “Automatic programming of behavior-based robots using reinforcement learning,” in National Conference on Artificial Intelligence, pp. 768–773, 1991

  157. R. Makar S. Mahadevan M. Ghavamzadeh (2001) “Hierarchical multi-agent reinforcement learning” J.P. Müller E. Andre S. Sen C. Frasson (Eds) Proceedings of the Fifth International Conference on Autonomous Agents ACM Press Montreal, Canada 246–253

    Google Scholar 

  158. M. Mataric (1994) Interaction and Intelligent Behavior Massachusetts Institute of Technology Cambridge, MA

    Google Scholar 

  159. M. Mataric, “Learning to behave socially,” in Third International Conference on Simulation of Adaptive Behavior, 1994

  160. M. Mataric, “Reward functions for accelerated learning,” in International Conference on Machine Learning, pp. 181–189, 1994

  161. M. Mataric, “Reinforcement learning in the multi-robot domain,” Auton. Robots, vol. 4, no. 1, pp.’73–83, 1997

  162. M. Mataric, “Using communication to reduce locality in distributed multi-agent learning,” Joint Special Issue on Learn Auton. Robots, Machine Learn, vol. 31, nos. 1-3, pp. 141–167, and Auton. Robots, vol. 5, nos. 3-4, pp. 335–354, Jul/Aug 1998

  163. M. Mataric, M. Nilsson, and K. Simsarian, “Cooperative multi-robot box-pushing,” in Proceedings of IEEE/RSJ Conference on Intelligent Robots and Systems, pp. 556–561, 1995

  164. Z. Michalewicz, Genetic Algorithms + Data Structures = Evolution Programs (3rd edn.), Springer-Verlag: Berlin, 1996

  165. T. Miconi, “A collective genetic algorithm”, in E. Cantu-Paz et al., (ed.), Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), pp. 876–883, 2001

  166. T. Miconi, “When evolving populations is better than coevolving individuals: The blind mice problem,” in Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI-03), 2003

  167. M. Mitchell, J. Crutchfield, and R. Das, “Evolving cellular automata with genetic algorithms: A’review of recent work,” in Proceedings of the First International Conference on Evolutionary Computation and its Applications (EvCA’96), 1996

  168. N. Monekosso, P. Remagnino, and A. Szarowicz, “An improved Q-learning algorithm using synthetic pheromones,” in E. N. B. Dunin-Keplicz, (ed.), From Theory to Practice in Multi-Agent Systems, Second International Workshop of Central and Eastern Europe on Multi-Agent Systems, CEEMAS 2001 Cracow, Poland, September 26–29, 2001. Revised Papers, Lecture Notes in Artificial Intelligence LNAI-2296, Springer-Verlag, 2002

  169. N. D. Monekosso and P. Remagnino, “Phe-Q: A pheromone based Q-learning,” in Australian Joint Conference on Artificial Intelligence, pp. 345–355, 2001

  170. N. D. Monekosso and P. Remagnino, “An analysis of the pheromone Q-learning algorithm,” in’Proceedings of the VIII Iberoamerican Conference on Artificial Intelligence IBERAMIA-02, pp. 224–232, 2002

  171. N. D. Monekosso, P. Remagnino, and A. Szarowicz, “An improved Q-learning algorithm using synthetic pheromones,” in Proceedings of the Second Workshop of Central and Eastern Europe on Multi-Agent Systems CEEMAS-01, pp. 197–206, 2001

  172. J. Moody, Y. Liu, M. Saffell, and K. Youn, “Stochastic direct reinforcement: Application to simple games with recurrence,” in Proceedings of Artificial Multiagent Learning, Papers from the 2004 AAAI Fall Symposium. Technical Report FS-04-02, 2004

  173. R. Mukherjee and S. Sen, “Towards a pareto-optimal solution in general-sum games,” in Agents-2001 Workshop on Learning Agents, 2001

  174. U. Mukhopadjyay, L. Stephens, and M. Huhns, “An intelligent system for document retrieval in distributed office environment,” J. Am. Soc. Inform Sci., vol. 37, 1986

  175. J. Muller M. Pischel (1994) ArticleTitle“An architecture for dynamically interacting agents” J. Intell. Coop. Inform. Syst. 3 IssueID1 25–45 Occurrence Handle10.1142/S021821579400003X

    Article  Google Scholar 

  176. M. Mundhe and S. Sen, “Evaluating concurrent reinforcement learners,” in Proceedings of the International Conference on Multiagent System, 2000

  177. M. Mundhe and S. Sen, “Evolving agent societies that avoid social dilemmas,” in D. Whitley, D.’Goldberg, E. Cantu-Paz, L. Spector, I. Parmee, and H.-G. Beyer, (eds.), Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2000), Morgan Kaufmann: Las Vegas, Nevada, USA, 10-12 2000, pp. 809–816, ISBN 1-55860-708-0

  178. Y. Nagayuki, S. Ishii, and K. Doya, “Multi-agent reinforcement learning: An approach based on the other agent’s internal model,” in Proceedings of the International Conference on Multi-Agent Systems (ICMAS-00), 2000

  179. M. V. Nagendra-Prasad, Learning Situation-Specific Control in Multi-Agent Systems, PhD thesis, University of Massachusetts Amherst, 1997

  180. R. Nair, D. Pynadath, M. Yokoo, M. Tambe, and S. Marsella, “Taming decentralized POMDPs: Towards efficient policy computation for multiagent settings,” in Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI-03), 2003

  181. M. Nowak K. Sigmund (1998) ArticleTitle“Evolution of indirect reciprocity by image scoring/the dynamics of indirect reciprocity” Nature 393 573–577 Occurrence Handle10.1038/31225 Occurrence Handle9634232

    Article  PubMed  Google Scholar 

  182. A. Nowe, K. Verbeeck, and T. Lenaerts, Learning agents in a homo egualis society, Technical report, Computational Modeling Lab - VUB, March 2001

  183. L. Nunes and E. Oliveira, “Learning from multiple sources,” in AAMAS-2004- Proceedings of the Third International Joint Conference on Autonomous Agents and Multi Agent Systems, 2004

  184. T. Ohko, K. Hiraki, and Y. Arzai, “Addressee learning and message interception for communication load reduction in multiple robots environments,” in G. Weiß, (ed.), Distributed Artificial Intelligence Meets Machine Learning: Learning in Multi-Agent Environments, Lecture Notes in Artificial Intelligence 1221, Springer-Verlag, 1997

  185. E. Ostergaard, G. Sukhatme, and M. Mataric, “Emergent bucket brigading - a simple mechanism for improving performance in multi-robot constrainedspace foraging tasks,” in Proceedings of the Fifth International Conference on Autonomous Agents, 2001

  186. L. Pagie and M. Mitchell, “A comparison of evolutionary and coevolutionary search,” in R. K. Belew and H. Juillè, (eds.), Coevolution: Turning Adaptive Algorithms upon Themselves, San Francisco, California, USA, pp. 20–25, 7 2001

  187. L. Panait and S. Luke, “Ant foraging revisited,” in Proceedings of the Ninth International Conference on the Simulation and Synthesis of Living Systems (ALIFE9), 2004

  188. L. Panait and S. Luke, “Learning ant foraging behaviors,” in Proceedings of the Ninth International Conference on the Simulation and Synthesis of Living Systems (ALIFE9), 2004

  189. L. Panait and S. Luke, “A pheromone-based utility model for collaborative foraging,” in AAMAS-2004 - Proceedings of the Third International Joint Conference on Autonomous Agents and Multi Agent Systems, 2004

  190. L. Panait, R. P. Wiegand, and S. Luke, “A sensitivity analysis of a cooperative coevolutionary algorithm biased for optimization,” in Genetic and Evolutionary Computation Conference - GECCO-2004, Springer, 2004

  191. L. Panait, R. P. Wiegand, and S. Luke, “A visual demonstration of convergence properties of cooperative coevolution,” in Parallel Problem Solving from Nature - PPSN-2004, Springer, 2004

  192. L. A. Panait, R. P. Wiegand, and S. Luke, “Improving coevolutionary search for optimal multiagent behaviors”, in Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI-03), 2003

  193. C. Papadimitriou J. Tsitsiklis (1987) ArticleTitle“Complexity of markov decision processes” Math. Operat. Res. 12 IssueID3 441–450

    Google Scholar 

  194. L. Parker, “Current state of the art in distributed autonomous mobile robotics,” in L. Parker, G.’Bekey, and J. Barhen, (eds.), Distributed Autonomous Robotic Systems 4, Springer-Verlag, pp. 3–12, 2000

  195. L. Parker, “Multi-robot learning in a cooperative observation task,” in Proceedings of Fifth International Symposium on Distributed Autonomous Robotic Systems (DARS 2000), 2000

  196. L. Parker, “Distributed algorithms for multi-robot observation of multiple moving targets,” Auton Robots, vol. 12 no. 3, 2002

  197. L. Parker, C. Touzet, and F. Fernandez, “Techniques for learning in multi-robot teams,” in T. Balch and L. Parker, (eds.), Robot Teams: From Diversity to Polymorphism, AK Peters, 2001

  198. M. Peceny, G. Weiß, and W. Brauer, Verteiltes maschinelles lernen in fertigungsumgebungen, Technical Report FKI-218-96, Institut fur Informatik, Technische Universitat Munchen, 1996

  199. M. Peeters, K. Verbeeck, and A. Nowe, “Multi-agent learning in conflicting multi-level games with incomplete information,” in Proceedings of Artificial Multiagent Learning, Papers from the 2004 AAAI Fall Symposium. Technical Report FS-04-02, 2004

  200. L. Peshkin, K.-E. Kim, N. Meuleau, and L. Kaelbling, “Learning to cooperate via policy search,” in Sixteenth Conference on Uncertainty in Artificial Intelligence, Morgan Kaufmann, 2000, pp. 307–314

  201. R. Poli, J. Rowe, and K. D. Jong, (eds.), Foundations of Genetic Algorithms (FOGA) VII, 2002, Morgan Kaufmann

  202. J. Pollack A. Blair (1998) ArticleTitle“Coevolution in the successful learning of backgammon strategy” Machine Learn. 32 IssueID3 225–240 Occurrence Handle10.1023/A:1007417214905

    Article  Google Scholar 

  203. J. Pollack A. Blair M. Land (1997) “Coevolution of a backgammon player” C.G. Langton K. Shimohara (Eds) Artificial Life V: Proc. of the Fifth Int. Workshop on the Synthesis and Simulation of Living Systems The MIT Press Cambridge, MA 92–98

    Google Scholar 

  204. E. Popovici and K. DeJong, “Understanding competitive co-evolutionary dynamics via fitness landscapes,” in Artificial Multiagent Symposium, Part of the 2004 AAAI Fall Symposium on Artificial Intelligence, 2004

  205. M. Potter (1997) The Design and Analysis of a Computational Model of Cooperative Coevolution, PhD thesis George Mason University Fairfax, Virginia

    Google Scholar 

  206. M. Potter and K. De Jong, “A cooperative coevolutionary approach to function optimization,” in Y. Davidor and H.-P. Schwefel, (eds.), Proceedings of the Third International Conference on Parallel Problem Solving from Nature (PPSN III), Springer-Verlag, pp. 249–257, 1994

  207. M. Potter K. Jong ParticleDe (2000) ArticleTitle“Cooperative coevolution: An architecture for evolving coadapted subcomponents” Evol. Comput. 8 IssueID1 1–29 Occurrence Handle10.1162/106365600568086 Occurrence Handle10753229

    Article  PubMed  Google Scholar 

  208. M. Potter, K. De Jong, and J. J. Grefenstette, “A coevolutionary approach to learning sequential decision rules,” in Proceedings from the Sixth International Conference on Genetic Algorithms, Morgan Kaufmann, pp. 366–372, 1995

  209. M. Potter, L. Meeden, and A. Schultz, “Heterogeneity in the coevolved behaviors of mobile robots: The emergence of specialists,” in Proceedings of The Seventeenth International Conference on Artificial Intelligence (IJCAI-2001), 2001

  210. N. Puppala, S. Sen, and M. Gordin, “Shared memory based cooperative coevolution,” in Proceedings of the 1998 IEEE World Congress on Computational Intelligence, IEEE Press: Anchorage, Alaska, USA, pp. 570–574, 1998

  211. M. Quinn, “A comparison of approaches to the evolution of homogeneous multi-robot teams,” in Proceedings of the 2001 Congress on Evolutionary Computation (CEC2001), IEEE Press: COEX, World Trade Center, 159 Samseong-dong, Gangnam-gu, Seoul, Korea, 27–30 2001, pp. 128–135. ISBN 0-7803-6658-1

  212. M. Quinn, “Evolving communication without dedicated communication channels,” in Advances in Artificial Life: Sixth European Conference on Artificial Life (ECAL01), 2001

  213. M. Quinn, L. Smith, G. Mayley, and P. Husbands, Evolving formation movement for a homogeneous multi-robot system: Teamwork and role-allocation with real robots, Cognitive Science Research Paper 515. School of Cognitive and Computing Sciences, University of Sussex, Brighton, BN1 9QG. ISSN 1350–3162, 2002

  214. C. Reynolds, “An evolved, vision-based behavioral model of coordinated group motion,” in From Animals to Animats 2: Proceedings of the Second International Conference on Simulation of Adaptive Behavior (SAB92), pp. 384–392, 1993

  215. C. Reynolds, “Competition, coevolution and the game of tag,” in R. A. Brooks and P. Maes, (eds.), Artificial Life IV, Proceedings of the Fourth International Workshop on the Synthesis and Simulation of Living Systems., MIT Press, pp. 59–69, 1994

  216. C.W. Reynolds (1987) ArticleTitle“Flocks, herds, and schools: a distributed behavioral model” Comput. Graph. 21 IssueID4 25–34

    Google Scholar 

  217. P. Riley and M. Veloso, “On behavior classification in adversarial environments,” in L. Parker, G.’Bekey, and J. Barhen (eds.), Distributed Autonomous Robotic Systems 4, Springer-Verlag, pp. 371–380, 2000

  218. A. Robinson and L. Spector, “Using genetic programming with multiple data types and automatic modularization to evolve decentralized and coordinated navigation in multi-agent systems,” in In’Late-Breaking Papers of the Genetic and Evolutionary Computation Conference (GECCO-2002), The International Society for Genetic and Evolutionary Computation, 2002

  219. C. Rosin R. Belew (1997) ArticleTitle“New methods for competitive coevolution” Evol. Comput. 5 IssueID1 1–29 Occurrence Handle10021751

    PubMed  Google Scholar 

  220. R. Salustowicz, M. Wiering, and J. Schmidhuber, Learning team strategies with multiple policy-sharing agents: A soccer case study, Technical report, ISDIA, Corso Elvezia 36, 6900 Lugano, Switzerland, 1997

  221. R. Salustowicz M. Wiering J. Schmidhuber (1998) ArticleTitle“Learning team strategies: Soccer case studies” Machine Learn. 33 IssueID2-3 263–282 Occurrence Handle10.1023/A:1007570708568

    Article  Google Scholar 

  222. A. Samuel (1994) ArticleTitle“Some studies in machine learning using the game of checkers” IBM J. Res. Develop. 3 IssueID3 210–229

    Google Scholar 

  223. T. Sandholm and R. H. Crites, “On multiagent Q-learning in a semi-competitive domain,” in’Adaption and Learning in Multi-Agent Systems, pp. 191–205, 1995

  224. H. Santana, G. Ramalho, V. Corruble, and B. Ratitch, “Multi-agent patrolling with reinforcement learning,” in AAMAS-2004 - Proceedings of the Third International Joint Conference on Autonomous Agents and Multi Agent Systems, 2004

  225. G. Saunders and J. Pollack, “The evolution of communication schemes over continuous channels,” in’From Animals to Animats 4 - Proceedings of the Fourth International Conference on Adaptive Behaviour, 1996

  226. J. Sauter, R. S. Matthews, H. Van Dyke Parunak, and S. Brueckner, “Evolving adaptive pheromone path planning mechanisms,” in Proceedings of First International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS-02), pp. 434–440, 2002

  227. J. Sauter, H. Van Dyke Parunak, S. Brueckner, and R. Matthews, “Tuning synthetic pheromones with evolutionary computing,” in R. E. Smith, C. Bonacina, C. Hoile, and P. Marrow, (eds.), Evolutionary Computation and Multi-Agent Systems (ECOMAS), San Francisco, California, USA, 7 pp. 321–324, 2001

  228. J. Schmidhuber, “Realistic multi-agent reinforcement learning,” in Learning in Distributed Artificial Intelligence Systems, Working Notes of the 1996 ECAI Workshop, 1996

  229. J. Schmidhuber and J. Zhao, “Multi-agent learning with the success-story algorithm,” in ECAI Workshop LDAIS/ICMAS Workshop LIOME, pp. 82–93, 1996

  230. J. Schneider, W.-K. Wong, A. Moore, and M. Riedmiller, “Distributed value functions,” in Proceedings of the Sixteenth International Conference on Machine Learning, pp. 371–378, 1999

  231. A. Schultz, J. Grefenstette, and W. Adams, “Robo-shepherd: Learning complex robotic behaviors,” in Robotics and Manufacturing: Recent Trends in Research and Applications. Vol. 6, ASME Press, pp. 763–768, 1996

  232. U. M. Schwuttke and A. G. Quan, “Enhancing performance of cooperating agents in realtime diagnostic systems”, in Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence (IJCAI-93), 1993

  233. M. Sekaran and S. Sen, “To help or not to help”, in Proceedings of the Seventeenth Annual Conference of the Cognitive Science Society, Pittsburgh, PA, pp. 736–741, 1995

  234. S. Sen, “Multiagent systems: Milestones and new horizons”, Trends Cognitive Sci., vol. 1, no. 9, pp.’334–339, 1997

  235. S. Sen, “Special issue on evolution and learning in multiagent systems,” Int. J. Human-Comput. Stud., vol. 48, no. 1, 1998

  236. S. Sen and M. Sekaran, “Using reciprocity to adapt to others”, in G. Weiß and S. Sen (eds.), International Joint Conference on Artificial Intelligence Workshop on Adaptation and Learning in Multiagent Sytems, Lecture Notes in Artificial Intelligence, Springer-Verlag, pp. 206–217, 1995

  237. S. Sen and M. Sekaran, “Multiagent coordination with learning classifier systems”, in G. Weiß and S.’Sen, (eds.), Proceedings of the IJCAI Workshop on Adaption and Learning in Multi-Agent Systems, Volume 1042, Springer Verlag, pp. 218–233, 1996. ISBN 3-540-60923-7

  238. S. Sen M. Sekaran (1998) ArticleTitle“Individual learning of coordination knowledge” J. Exp. Theo. Artif. Intel. 10 IssueID3 333–356 Occurrence Handle10.1080/095281398146798

    Article  Google Scholar 

  239. S. Sen, M. Sekaran, and J. Hale, “Learning to coordinate without sharing information”, in Proceedings of the Twelfth National Conference on Artificial Intelligence, pp. 426–431, 1994

  240. Y. Shoham, R. Powers, and T. Grenager, “On the agenda(s) of research on multi-agent learning,” in’Proceedings of Artificial Multiagent Learning, Papers from the 2004 AAAI Fall Symposium. Technical Report FS-04–02, 2004

  241. R. Smith and B. Gray, Co-adaptive genetic algorithms: An example in othello strategy, Technical Report TCGA 94002, University of Alabama, Department of Engineering Science and Mechanics, 1993

  242. L. Spector and J. Klein, “Evolutionary dynamics discovered via visualization in the breve simulation environment,” in Workshop Proceedings of the 8th International Conference on the Simulation and Synthesis of Living Systems, pp. 163–170, 2002

  243. L. Spector, J. Klein, C. Perry, and M. Feinstein, “Emergence of collective behavior in evolving populations of flying agents,” in E. Cantu-Paz et al., (ed.), Prooceedings of the Genetic and Evolutionary Computation Conference (GECCO). Springer-Verlag, 2003

  244. R. Steeb, S. Cammarata, F. Hayes-Roth, P. Thorndyke, and R. Wesson, “Distributed intelligence for air fleet control,” in A. Bond and L. Gasser (eds.), Readings in Distributed Artificial Intelligence, Morgan Kaufmann Publishers, pp. 90–101, 1988

  245. L. Steels (1995) ArticleTitle“A self-organizing spatial vocabulary” Artif. Life 2 IssueID3 319–332 Occurrence Handle8925502

    PubMed  Google Scholar 

  246. L. Steels, “Emergent adaptive lexicons,” in P. Maes, (ed.), Proceedings of the Simulation of Adaptive Behavior Conference. MIT Press, 1996

  247. L. Steels, “Self-organising vocabularies,” in Proceedings of Artificial Life V, 1996

  248. L. Steels (1996) “The spontaneous self-organization of an adaptive language” S. Muggleton (Eds) Machine Intelligence 15 Oxford University Press Oxford, UK

    Google Scholar 

  249. L. Steels, “Synthesising the origins of language and meaning using co-evolution, self-organisation and level formation,” in J. Hurford, C. Knight, and M. Studdert-Kennedy (eds.), Approaches to the Evolution of Language: Social and Cognitive Bases, Edinburgh University Press, 1997

  250. L. Steels (2000) ArticleTitle“The puzzle of language evolution” Kognitionswissenschaft 8 IssueID4 143–150 Occurrence Handle10.1007/s001970050001

    Article  Google Scholar 

  251. L. Steels and F. Kaplan, “Collective learning and semiotic dynamics,” in Proceedings of the European Conference on Artificial Life, pp. 679–688, 1999

  252. P. Stone, “Layered learning in multiagent systems,” in Proceedings of National Conference on Artificial Intelligence AAAI/IAAI, 1997

  253. P. Stone, “Layered Learning in Multi-Agent Systems,” PhD thesis, Carnegie Mellon University, 1998

  254. P. Stone and R. Sutton, “Keepaway soccer: A machine learning testbed,” in A. Birk, S. Coradeschi, and S. Tadokoro, (eds.), RoboCup 2001: Robot Soccer World Cup V, volume 2377 of Lecture Notes in Computer Science, Springer, pp. 214–223, 2002. ISBN 3-540-43912-9

  255. P. Stone M.M. Veloso (2000) ArticleTitle“Multiagent systems: A survey from a machine learning perspective” Auton. Robots 8 IssueID3 345–383 Occurrence Handle10.1023/A:1008942012299

    Article  Google Scholar 

  256. N. Sturtevant and R. Korf, “On pruning techniques for multi-player games,” in Proceedings of National Conference on Artificial Intelligence (AAAI), pp. 201–207, 2000

  257. D. Subramanian, P. Druschel, and J. Chen, “Ants and reinforcement learning: A case study in routing in dynamic networks,” in Proceedings of Fifteenth International Joint Conference on Artificial Intelligence (IJCAI-97), pp. 832–839, 1997

  258. N. Suematsu and A. Hayashi, “A multiagent reinforcement learning algorithm using extended optimal response,” in Proceedings of First International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS-02), pp. 370–377, 2002

  259. D. Suryadi and P. J. Gmytrasiewicz, “Learning models of other agents using influence diagrams,” in’Preceedings of the 1999 International Conference on User Modeling, pp. 223–232, 1999

  260. R. Sutton (1998) ArticleTitle“Learning to predict by the methods of temporal differences” Machine Learn. 3 9–44

    Google Scholar 

  261. R. Sutton and A. Barto, Reinforcement Learning: An Introduction, MIT Press, 1998

  262. J. Svennebring and S. Koenig, “Trail-laying robots for robust terrain coverage,” in Proceedings of the International Conference on Robotics and Automation (ICRA-03), 2003

  263. P. ’t Hoen and K. Tuyls, “Analyzing multi-agent reinforcement learning using evolutionary dynamics,” in Proceedings of the 15th European Conference on Machine Learning (ECML), 2004

  264. M. Tambe, “Recursive agent and agent-group tracking in a real-time dynamic environment,” in V.’Lesser and L. Gasser (eds.), Proceedings of the First International Conference on Multiagent Systems (ICMAS-95). AAAI Press, 1995

  265. M. Tan (1993) “Multi-agent reinforcement learning: Independent vs. cooperative learning” M.N. Huhns M.P. Singh (Eds) Readings in Agents Morgan Kaufmann San Francisco, CA, USA 487–494

    Google Scholar 

  266. P. Tangamchit, J. Dolan, and P. Khosla, “The necessity of average rewards in cooperative multirobot learning,” in Proceedings of IEEE Conference on Robotics and Automation, 2002

  267. G. Tesauro (1995) ArticleTitle“Temporal difference learning and TD-gammon” Commun. ACM 38 IssueID3 58–68 Occurrence Handle10.1145/203330.203343

    Article  Google Scholar 

  268. G. Tesauro J.O. Kephart (2002) ArticleTitle“Pricing in agent economies using multi-agent Q-learning” Auton. Agents Multi-Agent Syst. 8 289–304 Occurrence Handle10.1023/A:1015504423309

    Article  Google Scholar 

  269. S. Thrun, “Learning to play the game of chess,” in G. Tesauro, D. Touretzky, and T. Leen, (eds.), Advances in Neural Information Processing Systems 7, The MIT Press: Cambridge, MA, pp. 1069–1076, 1995

  270. K. Tumer, A. K. Agogino, and D. H. Wolpert, “Learning sequences of actions in collectives of autonomous agents,” in Proceedings of First International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS-02), pp. 378–385, 2002

  271. K. Tuyls, K. Verbeeck, and T. Lenaerts, “A selection-mutation model for Q-learning in multiagent systems,” in AAMAS-2003 — Proceedings of the Second International Joint Conference on Autonomous Agents and Multi Agent Systems, 2003

  272. W. Uther and M. Veloso, “Adversarial reinforcement learning. Technical Report CMU-CS-03-107, School of Computer Science, Carnegie Mellon University, 2003

  273. H. Van Dyke Parunak, “Applications of distributed artificial intelligence in industry,” in G. M. P. O’Hare and N. R. Jennings, (eds.), Foundations of Distributed AI. John Wiley & Sons, 1996

  274. L.Z. Varga N.R. Jennings D. Cockburn (1994) ArticleTitle“Integrating intelligent systems into a cooperating community for electricity distribution management” Int. J. Expert Syst. Appl. 7 IssueID4 563–579 Occurrence Handle10.1016/0957-4174(94)90080-9

    Article  Google Scholar 

  275. J. Vidal and E. Durfee, “Agents learning about agents: A framework and analysis,” in Working Notes of AAAI-97 Workshop on Multiagent Learning, 1997

  276. J. Vidal and E. Durfee, “The moving target function problem in multiagent learning,” in Proceedings of the Third Annual Conference on Multi-Agent Systems, 1998

  277. J. Vidal and E. Durfee, “Predicting the expected behavior of agents that learn about agents: The CLRI framework,” Autonomous Agents and Multi-Agent Systems, January 2003

  278. K. Wagner, “Cooperative strategies and the evolution of communication,” Artif. Life, vol. 6, no. 2, pp. 149–179, Spring 2000

  279. X. Wang and T. Sandholm, “Reinforcement learning to play an optimal Nash equilibrium in team Markov games,” in Advances in Neural Information Processing Systems (NIPS-2002), 2002

  280. R. Watson and J. Pollack, “Coevolutionary dynamics in a minimal substrate,” in E. Cantu-Paz et al, (eds.), Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), 2001

  281. R. Weihmayer and H. Velthuijsen, “Application of distributed AI and cooperative problem solving to telecommunications,” in J. Liebowitz and D. Prereau, (eds.), AI Approaches to Telecommunications and Network Management, IOS Press, 1994

  282. M. Weinberg and J. Rosenschein, “Best-response multiagent learning in non-stationary environments,” in AAMAS-2004 — Proceedings of the Third International Joint Conference on Autonomous Agents and Multi Agent Systems, 2004

  283. G. Weiß, Some studies in distributed machine learning and organizational design. Technical Report FKI-189-94, Institut f´′ur Informatik, TU München, 1994

  284. G. Weiß (1995) Distributed Machine Learning Infix Verlag Sankt Augustin

    Google Scholar 

  285. G. Weiß, ed., Distributed Artificial Intelligence Meets Machine Learning: Learning in Multi-Agent Environments, Number 1221 in Lecture Notes in Artificial Intelligence, Springer-Verlag, 1997

  286. G. Weiß (1998). “Special issue on learning in distributed artificial intelligence systems”. J. Exp. Theo. Artif. Intell. 10(3).

  287. G. Weiß, ed., Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence. MIT Press, 1999

  288. G. Weiß and P. Dillenbourg, “What is ‘multi’ in multi-agent learning?” in P. Dillenbourg, (ed.), Collaborative Learning, Cognitive and Computational Approaches, Pergamon Press, pp. 64–80, 1999

  289. G. Weiß and S. Sen (eds.), Adaptation and Learning in Multiagent Systems,” Lecture Notes in Artificial Intelligence Vol. 1042, Springer-Verlag, 1996

  290. M. Wellman J. Hu (1998) ArticleTitle“Conjectural equilibrium in multiagent learning” Machine Learn. 33 IssueID2-3 179–200 Occurrence Handle10.1023/A:1007514623589

    Article  Google Scholar 

  291. J. Werfel M. Mitchell J.P. Crutchfield (November 2000) ArticleTitle“Resource sharing and coevolution in evolving cellular automata” IEEE Trans. Evol. Comput. 4 IssueID4 388

    Google Scholar 

  292. B. B. Werger and M. Mataric, “Exploiting embodiment in multi-robot teams, Technical Report IRIS-99-378, University of Southern California, Institute for Robotics and Intelligent Systems, 1999

  293. G. M. Werner and M. G. Dyer, “Evolution of herding behavior in artificial animals,” in From Animals to Animats 2: Proceedings of the Second International Conference on Simulation of Adaptive Behavior (SAB92), 1993

  294. T. White B. Pagurek F. Oppacher (1998) “ASGA: Improving the ant system by integration with genetic algorithms” J.R. Koza W. Banzhaf K. Chellapilla K. Deb M. Dorigo D.B. Fogel M.H. Garzon D.E. Goldberg H. Iba R. Riolo (Eds) Genetic Programming 1998: Proceedings of the Third Annual Conference Morgan Kaufmann: University of Wisconsin Madison, Wisconsin, USA 22–25

    Google Scholar 

  295. S. Whiteson and P. Stone, “Concurrent layered learning,” in AAMAS-2003 - Proceedings of the Second International Joint Conference on Autonomous Agents and Multi Agent Systems, 2003

  296. R. P. Wiegand, Analysis of Cooperative Coevolutionary Algorithms, PhD thesis, Department of Computer Science, George Mason University, 2003

  297. R. P. Wiegand, W. Liles, and K. De Jong, “An empirical analysis of collaboration methods in cooperative coevolutionary algorithms,” in E. Cantu-Paz et al., (ed.), Proceedings of the Genetic and Evolutionary Computation Conference (GECCO), pp. 1235–1242, 2001

  298. R. P. Wiegand, W. Liles, and K. De Jong, “Analyzing cooperative coevolution with evolutionary game theory,” in D. Fogel, (ed.), Proceedings of Congress on Evolutionary Computation (CEC-02), IEEE Press, pp. 1600–1605, 2002

  299. R. P. Wiegand, W. Liles, and K. De Jong, “Modeling variation in cooperative coevolution using evolutionary game theory,” in Poli et al. [201], pp. 231–248

  300. R. P. Wiegand and J. Sarma, “Spatial embedding and loss of gradient in cooperative coevolutionary algorithms,” in Parallel Problem Solving from Nature - PPSN-2004, Springer, 2004

  301. M. Wiering R. Salustowicz J. Schmidhuber (1999) ArticleTitle“Reinforcement learning soccer teams with incomplete world models” J. Auton. Robots 7 IssueID1 77–88 Occurrence Handle10.1023/A:1008921914343

    Article  Google Scholar 

  302. A. Williams (2004) ArticleTitle“Learning to share meaning in a multi-agent system” Auton. Agents Multi-Agent Syst. 8 165–193 Occurrence Handle10.1023/B:AGNT.0000011160.45980.4b

    Article  Google Scholar 

  303. E. Wilson, Sociobiology: The New Synthesis, Belknap Press, 1975

  304. D.H. Wolpert K. Tumer (2001) ArticleTitle“Optimal payoff functions for members of collectives” Adv. Complex Syst. 4 IssueID2-3 265–279 Occurrence Handle10.1142/S0219525901000188

    Article  Google Scholar 

  305. D. H. Wolpert, K. Tumer, and J. Frank, “Using collective intelligence to route internet traffic,” in’Advances in Neural Information Processing Systems-11, Denver, pp. 952–958, 1998

  306. D. H. Wolpert, K. R. Wheller, and K. Tumer, “General principles of learning-based multi-agent systems,” in O. Etzioni, J. P. Müller, and J. M. Bradshaw, (eds.), Proceedings of the Third International Conference on Autonomous Agents (Agents’99), ACM Press: Seattle, WA, USA, pp. 77–83, 1999

  307. M. Wooldridge, S. Bussmann, and M. Klosterberg, “Production sequencing as negotiation,” in’Proceedings of the First International Conference on the Practical Application of Intelligent Agents and Multi-Agent Technology (PAAM-96), 1996

  308. A. Wu, A. Schultz, and A. Agah, “Evolving control for distributed micro air vehicles,” in IEEE Computational Intelligence in Robotics and Automation Engineers Conference, 1999

  309. H. Yanco and L. Stein, “An adaptive communication protocol for cooperating mobile robots,” in’From Animals to Animats: International Conference on Simulation of Adaptive Behavior, pp. 478–485, 1993

  310. N. Zaera, D. Cliff, and J. Bruten, (Not) Evolving collective behaviours in synthetic fish, Technical Report HPL-96-04, Hewlett-Packard Laboratories, 1996

  311. B. Zhang and D. Cho, “Coevolutionary fitness switching: Learning complex collective behaviors using genetic programming,” in Advances in Genetic Programming III, MIT Press, 1998, pp. 425–445

  312. J. Zhao J. Schmidhuber (1996) “Incremental self-improvement for life-time multi-agent reinforcement learning” P. Maes M. Mataric J.-A. Meyer J. Pollack S.W. Wilson (Eds) Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior: From Animals to Animats 4 MIT Press Cape Code, USA 9–13

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, George Mason University, Fairfax, VA, 22030, USA

    Liviu Panait & Sean Luke

Authors
  1. Liviu Panait
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sean Luke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liviu Panait.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Panait, L., Luke, S. Cooperative Multi-Agent Learning: The State of the Art. Auton Agent Multi-Agent Syst 11, 387–434 (2005). https://doi.org/10.1007/s10458-005-2631-2

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10458-005-2631-2

Key words

Search

Navigation