Abstract
As an alternative to partially observable Markov decision processes (POMDPs), Predictive State Representations (PSRs) is a recently developed method to model controlled dynamical systems. While POMDPs and PSRs provide general frameworks for solving the problem of planning under uncertainty, they rely crucially on having a known and accurate model of the environment. However, in real-world applications it can be extremely difficult to build an accurate model. In this paper, we use learned PSR model for planning under uncertainty, where the PSR model is learned from samples and may be inaccurate. We demonstrate the effectiveness of our algorithm on a standard set of POMDP test problems. Empirical results show that the algorithm we proposed is effective.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Littman, M., Sutton, R., Singh, S.: Predictive Representations of State. In: Neural Information Processing Systems (NIPS). MIT Press, Vancouver (2002)
Izadi, M.T., Precup, D.: A Planning Algorithm for Predictive State Representations. In: Eighteenth International Joint Conference on Artificial Intelligence, IJCAI (2003)
James, M., Singh, S., Littman, M.: Planning with Predictive State Representations. In: Proceedings of the International Conference on Machine Learning and Applications (ICMLA). IEEE Press, Louisville (2004)
Manly, B.F.J.: Multivariate Statistical methods: A PRIMER. Chapman and Hall, New York (1986)
Singh, S., James, M., Rudary, M.: Predictive State Representations: A New Theory for Modeling Dynamical Systems. In: Proceedings of the Twentieth Conference on Uncertainty in Artificial Intelligence (UAI). AUAI Press, Banff (2004)
Yun-Long, L., Ren-Hou, L.: Discovery and Learning of Models with Predictive State Representations for Dynamical Systems without Reset. Knowledge-Based Systems 22(8), 557–561 (2009)
James, M., Singh, S.: Learning and Discovery of Predictive State Representations in Dynamical Systems with Reset. In: Proceedings of the Twenty-First International Conference on Machine Learning (2004)
Cassandra, A.: Tony’s pomdp file repository page (1999), http://www.cs.brown.edu/research/ai/pomdp/examples/index.html (cited 2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, Y., Ji, G., Yang, Z. (2010). Using Learned PSR Model for Planning under Uncertainty. In: Farzindar, A., Kešelj, V. (eds) Advances in Artificial Intelligence. Canadian AI 2010. Lecture Notes in Computer Science(), vol 6085. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13059-5_35
Download citation
DOI: https://doi.org/10.1007/978-3-642-13059-5_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13058-8
Online ISBN: 978-3-642-13059-5
eBook Packages: Computer ScienceComputer Science (R0)