Abstract
Learning to communicate is an emerging challenge in AI research. It is known that agents interacting in decentralized, stochastic environments can benefit from exchanging information. Multi-agent planning generally assumes that agents share a common means of communication; however, in building robust distributed systems it is important to address potential miscoordination resulting from misinterpretation of messages exchanged. This paper lays foundations for studying this problem, examining its properties analytically and empirically in a decision-theoretic context. We establish a formal framework for the problem, and identify a collection of necessary and sufficient properties for decision problems that allow agents to employ probabilistic updating schemes in order to learn how to interpret what others are communicating. Solving the problem optimally is often intractable, but our approach enables agents using different languages to converge upon coordination over time. Our experimental work establishes how these methods perform when applied to problems of varying complexity.
Similar content being viewed by others
References
Allen, M., Goldman, C. V., & Zilberstein, S. (2005). Learning to communicate in decentralized systems. In Proceedings of the workshop on multiagent learning, twentieth national conference on artificial intelligence (pp. 1–8). (AAAI Tech. Report WS-05-09). PA: Pittsburgh.
Balch T. and Arkin R.C. (1994). Communication in reactive multiagent robotic systems. Autonomous Robots 1: 1–25
Becker R., Zilberstein S., Lesser V. and Goldman C.V. (2004). Solving transition independent decentralized MDPs. Journal of Artificial Intelligence Research 22: 423–455
Bernstein D.S., Givan R., Immerman N. and Zilberstein S. (2002). The complexity of decentralized control of Markov decision processes. Mathematics of Operations Research 27(4): 819–840
Bhattacharjee Y. (2004). From heofonum to heavens. Science 303: 1326–1328
Blume A. and Sobel J. (1995). Communication-proof equilibria in cheap-talk games. Journal of Economic Theory 65: 359–382
Bonarini A. and Sassaroli P. (1997). Opportunistic multimodel diagnosis with imperfect models. Information Sciences 103(1–4): 161–185
Boutilier, C. (1999). Sequential optimality and coordination in multiagent systems. In Proceedings of the sixteenth international joint conference on artificial intelligence (pp. 478–485). Sweden: Stockholm.
Coradeschi S. and Saffiotti A. (2003). An introduction to the anchoring problem. Robotics and Autonomous Systems 43(2–3): 85–96
Davidson D. (1984). Inquiries into Truth and Interpretation. Oxford University Press, Oxford, England
Decker K.S. and Lesser V.R. (1992). Generalizing the partial global planning algorithm. International Journal of Intelligent Cooperative Information Systems 1(2): 319–346
Doucet, A. (1998). On sequential simulation-based methods for bayesian filtering. Technical Report CUED/F-INFENG/TR.310 Department of Engineering, University of Cambridge.
Durfee E.H. (1988). Coordination of Distributed Problem Solvers. Kluwer Academic, Boston, MA
Durfee E.H. and Lesser V.R. (1988). Using partial global plans to coordinate distributed problem solvers. In: Bond, A.H. and Gasser, L. (eds) Readings in Distributed Artificial Intelligence, pp 285–293. Morgan Kaufmann Publishers Inc., San Mateo, CA
Finin T., Labrou Y. and Mayfield J. (1997). KQML as an agent communication language. In: Bradshaw, J. (eds) Software Agents, pp. MIT Press, Cambridge, MA
Firoiu, L., Oates, T., & Cohen, P. (1998). Learning regular languages from positive evidence. In Proceedings of the twentieth annual meeting of the cognitive science society (pp. 350–355). WI: Madison.
Gmytrasiewicz P. and Doshi P. (2005). A framework for sequential planning in multiagent settings. Journal of AI Research 24: 1–31
Gmytrasiewicz, P. J., Summers, M., & Gopal, D. (2002). Toward automated evolution of agent communication languages. In Proceedings of the thirtyfifth hawaii international conference on system sciences, Hawaii, p. 79.
Goldman, C. V., Allen, M., & Zilberstein, S. (2004). Decentralized language learning through acting. In Proceedings of the third international joint conference on autonomous agents and multiagent systems (pp. 1006–1013). New York, NY, 2004.
Goldman, C. V., Allen, M., & Zilberstein, S. (2006). Learning to communicate in a decentralized environment. Technical Report UM-CS-2006-16, 2006. Department of Computer Science, University of Massachusetts.
Goldman, C. V., & Rosenschein, J. S. (1994). Emergent coordination through the use of cooperative state-changing rules. In Proceedings of the twelfth national conference on artificial intelligence (pp. 408–413). WA: Seattle.
Goldman, C. V., & Zilberstein, S. (2003). Optimizing information exchange in cooperative multi-agent systems. In Proceedings of the second international joint conference on autonomous agents and multi-agent systems (pp. 137—144). Melbourne, Australia.
Goldman C.V. and Zilberstein S. (2004). Decentralized control of cooperative systems: Categorization and complexity analysis. Journal of Artificial Intelligence Research 22: 143–174
Goldman, C. V., & Zilberstein, S. (2004). Goal-oriented Dec-MDPs with direct communication. Technical Report UM-CS-2004-44, Department of Computer Science, University of Massachusetts.
Grosz B.J. and Kraus S. (1996). Collaborative plans for complex group action. Artificial Intelligence 86(2): 269–357
Kaelbling L.P. (1993). Learning in Embedded Systems. MIT Press, Cambridge, MA
Komarova N. and Niyogi P. (2004). Optimizing the mutual intelligibility of linguistic agents in a shared world. Artificial Intelligence 154: 1–42
MacLennan, B. (1990). Evolution of communication in a population of simple machines. Knoxville, Technical Report CS90-99, Department of Computer Science, University of Tennessee.
NASA (1999). Mars climate orbiter failure board report. Available at: ftp://ftp.hq.nasa.gov/pub/ pao/reports/1999/MCO_report.pdf
Puterman M.L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York, NY
Putnam, H. (1975). The meaning of ‘meaning’. In Mind, Language and Reality: Philosophical Papers (pp. 215–271). Vol. 2 Cambridge, England: Cambridge University Press.
Quine W.V.O. (1960). Word and Object. MIT Press, Cambridge, MA
Serafini L. and Bouquet P. (2004). Comparing formal theories of context in AI. Artificial Intelligence 155: 41–67
Sharygina N. and Peled D. (2001). A combined testing and verification approach for software reliability. Formal Methods for Increasing Software Productivity, LNCS 2021: 611–628
Smith R.G. (1988). The contract net protocol: High level communication and control in a distributed problem solver. In: Bond, A.H. and Gasser, L. (eds) Readings in Distributed Artificial Intelligence., pp 357–366. Morgan Kaufmann Publishers Inc., San Mateo, California
Steels, L., & Vogt, P. (1997). Grounding adaptive language games in robotic agents. In Proceedings of the fourth european conference on artificial life (pp. 474–482). UK: Brighton.
Sutton R.S. and Barto A.G. (2000). Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA
Thrun S., Fox D., Burgard W. and Dellaert F. (2001). Robust Monte Carlo localization for mobile robots. Artificial Intelligence 128: 99–141
Vogt P., Coumans H. (2003). Investigating social interaction strategies for bootstrapping lexicon development, Journal of Artificial Societies and Social Simulation, 6(1). http://jasss.soc.surrey.ac.uk/6/1/4.html
Wang, J., & Gasser, L. (2002). Mutual online concept learning for multiple agents. In Proceedings of the first international joint conference on autonomous agents and multi-agent systems (pp. 362–369). Bologna, Italy.
Wärneryd K. (1993). Cheap talk, coordination and evolutionary stability. Games and Economic Behavior 5: 532–546
Yanco, H. A. (1994). Robot communication: Issues and implementation. Master’s thesis, Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology.
Yanco H.A. and Stein L.A. (1993). An adaptive communication protocol for cooperating mobile robots. In: Meyer, J.A., Roitblat, H.L. and Wilson, S.W. (eds) From Animals to Animats: Proceedings of the second international conference on the simulation of adaptive behavior, pp 478–485. MIT Press, Cambridge, MA
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Goldman, C.V., Allen, M. & Zilberstein, S. Learning to communicate in a decentralized environment. Auton Agent Multi-Agent Syst 15, 47–90 (2007). https://doi.org/10.1007/s10458-006-0008-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10458-006-0008-9