Skip to main content
Log in

Learning to communicate in a decentralized environment

  • Published:
Autonomous Agents and Multi-Agent Systems Aims and scope Submit manuscript

Abstract

Learning to communicate is an emerging challenge in AI research. It is known that agents interacting in decentralized, stochastic environments can benefit from exchanging information. Multi-agent planning generally assumes that agents share a common means of communication; however, in building robust distributed systems it is important to address potential miscoordination resulting from misinterpretation of messages exchanged. This paper lays foundations for studying this problem, examining its properties analytically and empirically in a decision-theoretic context. We establish a formal framework for the problem, and identify a collection of necessary and sufficient properties for decision problems that allow agents to employ probabilistic updating schemes in order to learn how to interpret what others are communicating. Solving the problem optimally is often intractable, but our approach enables agents using different languages to converge upon coordination over time. Our experimental work establishes how these methods perform when applied to problems of varying complexity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Allen, M., Goldman, C. V., & Zilberstein, S. (2005). Learning to communicate in decentralized systems. In Proceedings of the workshop on multiagent learning, twentieth national conference on artificial intelligence (pp. 1–8). (AAAI Tech. Report WS-05-09). PA: Pittsburgh.

  2. Balch T. and Arkin R.C. (1994). Communication in reactive multiagent robotic systems. Autonomous Robots 1: 1–25

    Article  Google Scholar 

  3. Becker R., Zilberstein S., Lesser V. and Goldman C.V. (2004). Solving transition independent decentralized MDPs. Journal of Artificial Intelligence Research 22: 423–455

    MATH  MathSciNet  Google Scholar 

  4. Bernstein D.S., Givan R., Immerman N. and Zilberstein S. (2002). The complexity of decentralized control of Markov decision processes. Mathematics of Operations Research 27(4): 819–840

    Article  MATH  MathSciNet  Google Scholar 

  5. Bhattacharjee Y. (2004). From heofonum to heavens. Science 303: 1326–1328

    Article  Google Scholar 

  6. Blume A. and Sobel J. (1995). Communication-proof equilibria in cheap-talk games. Journal of Economic Theory 65: 359–382

    Article  MATH  MathSciNet  Google Scholar 

  7. Bonarini A. and Sassaroli P. (1997). Opportunistic multimodel diagnosis with imperfect models. Information Sciences 103(1–4): 161–185

    Article  Google Scholar 

  8. Boutilier, C. (1999). Sequential optimality and coordination in multiagent systems. In Proceedings of the sixteenth international joint conference on artificial intelligence (pp. 478–485). Sweden: Stockholm.

  9. Coradeschi S. and Saffiotti A. (2003). An introduction to the anchoring problem. Robotics and Autonomous Systems 43(2–3): 85–96

    Article  Google Scholar 

  10. Davidson D. (1984). Inquiries into Truth and Interpretation. Oxford University Press, Oxford, England

    Google Scholar 

  11. Decker K.S. and Lesser V.R. (1992). Generalizing the partial global planning algorithm. International Journal of Intelligent Cooperative Information Systems 1(2): 319–346

    Article  Google Scholar 

  12. Doucet, A. (1998). On sequential simulation-based methods for bayesian filtering. Technical Report CUED/F-INFENG/TR.310 Department of Engineering, University of Cambridge.

  13. Durfee E.H. (1988). Coordination of Distributed Problem Solvers. Kluwer Academic, Boston, MA

    MATH  Google Scholar 

  14. Durfee E.H. and Lesser V.R. (1988). Using partial global plans to coordinate distributed problem solvers. In: Bond, A.H. and Gasser, L. (eds) Readings in Distributed Artificial Intelligence, pp 285–293. Morgan Kaufmann Publishers Inc., San Mateo, CA

    Google Scholar 

  15. Finin T., Labrou Y. and Mayfield J. (1997). KQML as an agent communication language. In: Bradshaw, J. (eds) Software Agents, pp. MIT Press, Cambridge, MA

    Google Scholar 

  16. Firoiu, L., Oates, T., & Cohen, P. (1998). Learning regular languages from positive evidence. In Proceedings of the twentieth annual meeting of the cognitive science society (pp. 350–355). WI: Madison.

  17. Gmytrasiewicz P. and Doshi P. (2005). A framework for sequential planning in multiagent settings. Journal of AI Research 24: 1–31

    Google Scholar 

  18. Gmytrasiewicz, P. J., Summers, M., & Gopal, D. (2002). Toward automated evolution of agent communication languages. In Proceedings of the thirtyfifth hawaii international conference on system sciences, Hawaii, p. 79.

  19. Goldman, C. V., Allen, M., & Zilberstein, S. (2004). Decentralized language learning through acting. In Proceedings of the third international joint conference on autonomous agents and multiagent systems (pp. 1006–1013). New York, NY, 2004.

  20. Goldman, C. V., Allen, M., & Zilberstein, S. (2006). Learning to communicate in a decentralized environment. Technical Report UM-CS-2006-16, 2006. Department of Computer Science, University of Massachusetts.

  21. Goldman, C. V., & Rosenschein, J. S. (1994). Emergent coordination through the use of cooperative state-changing rules. In Proceedings of the twelfth national conference on artificial intelligence (pp. 408–413). WA: Seattle.

  22. Goldman, C. V., & Zilberstein, S. (2003). Optimizing information exchange in cooperative multi-agent systems. In Proceedings of the second international joint conference on autonomous agents and multi-agent systems (pp. 137—144). Melbourne, Australia.

  23. Goldman C.V. and Zilberstein S. (2004). Decentralized control of cooperative systems: Categorization and complexity analysis. Journal of Artificial Intelligence Research 22: 143–174

    MATH  MathSciNet  Google Scholar 

  24. Goldman, C. V., & Zilberstein, S. (2004). Goal-oriented Dec-MDPs with direct communication. Technical Report UM-CS-2004-44, Department of Computer Science, University of Massachusetts.

  25. Grosz B.J. and Kraus S. (1996). Collaborative plans for complex group action. Artificial Intelligence 86(2): 269–357

    Article  MathSciNet  Google Scholar 

  26. Kaelbling L.P. (1993). Learning in Embedded Systems. MIT Press, Cambridge, MA

    Google Scholar 

  27. Komarova N. and Niyogi P. (2004). Optimizing the mutual intelligibility of linguistic agents in a shared world. Artificial Intelligence 154: 1–42

    Article  MATH  MathSciNet  Google Scholar 

  28. MacLennan, B. (1990). Evolution of communication in a population of simple machines. Knoxville, Technical Report CS90-99, Department of Computer Science, University of Tennessee.

  29. NASA (1999). Mars climate orbiter failure board report. Available at: ftp://ftp.hq.nasa.gov/pub/ pao/reports/1999/MCO_report.pdf

  30. Puterman M.L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York, NY

    MATH  Google Scholar 

  31. Putnam, H. (1975). The meaning of ‘meaning’. In Mind, Language and Reality: Philosophical Papers (pp. 215–271). Vol. 2 Cambridge, England: Cambridge University Press.

  32. Quine W.V.O. (1960). Word and Object. MIT Press, Cambridge, MA

    MATH  Google Scholar 

  33. Serafini L. and Bouquet P. (2004). Comparing formal theories of context in AI. Artificial Intelligence 155: 41–67

    Article  MATH  MathSciNet  Google Scholar 

  34. Sharygina N. and Peled D. (2001). A combined testing and verification approach for software reliability. Formal Methods for Increasing Software Productivity, LNCS 2021: 611–628

    Google Scholar 

  35. Smith R.G. (1988). The contract net protocol: High level communication and control in a distributed problem solver. In: Bond, A.H. and Gasser, L. (eds) Readings in Distributed Artificial Intelligence., pp 357–366. Morgan Kaufmann Publishers Inc., San Mateo, California

    Google Scholar 

  36. Steels, L., & Vogt, P. (1997). Grounding adaptive language games in robotic agents. In Proceedings of the fourth european conference on artificial life (pp. 474–482). UK: Brighton.

  37. Sutton R.S. and Barto A.G. (2000). Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA

    Google Scholar 

  38. Thrun S., Fox D., Burgard W. and Dellaert F. (2001). Robust Monte Carlo localization for mobile robots. Artificial Intelligence 128: 99–141

    Article  MATH  Google Scholar 

  39. Vogt P., Coumans H. (2003). Investigating social interaction strategies for bootstrapping lexicon development, Journal of Artificial Societies and Social Simulation, 6(1). http://jasss.soc.surrey.ac.uk/6/1/4.html

  40. Wang, J., & Gasser, L. (2002). Mutual online concept learning for multiple agents. In Proceedings of the first international joint conference on autonomous agents and multi-agent systems (pp. 362–369). Bologna, Italy.

  41. Wärneryd K. (1993). Cheap talk, coordination and evolutionary stability. Games and Economic Behavior 5: 532–546

    Article  MATH  MathSciNet  Google Scholar 

  42. Yanco, H. A. (1994). Robot communication: Issues and implementation. Master’s thesis, Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology.

  43. Yanco H.A. and Stein L.A. (1993). An adaptive communication protocol for cooperating mobile robots. In: Meyer, J.A., Roitblat, H.L. and Wilson, S.W. (eds) From Animals to Animats: Proceedings of the second international conference on the simulation of adaptive behavior, pp 478–485. MIT Press, Cambridge, MA

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Martin Allen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Goldman, C.V., Allen, M. & Zilberstein, S. Learning to communicate in a decentralized environment. Auton Agent Multi-Agent Syst 15, 47–90 (2007). https://doi.org/10.1007/s10458-006-0008-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10458-006-0008-9

Keywords

Navigation