Abstract
How to improve the generalization and approximation ability in reinforcement learning (RL) is still an open issue in recent years. Aiming at this problem, this paper presents a novel kernel-based representation policy iteration (KRPI) method for reinforcement learning in optimal path tracking of mobile robots. In the proposed method, the kernel trick is employed to map the original state space into a high-dimensional feature space and the Laplacian operator in the feature space is obtained by minimizing an objective function of optimal embedding. In the experiments, the KRPI-based PD controller was applied to the optimal path tracking problem of a wheeled mobile robot. It is demonstrated that the proposed method can obtain better near-optimal control policies than previous approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Sutton, R.S., Barto, A.G.: Reinforcement learning: an introduction. The MIT Press (1998)
Gullapalli, V., Franklin, J., Benbrahim, H.: Acquiring robot skills via reinforcement learning. IEEE Control Systems 14(1), 13–24 (1994)
Xu, X., Liu, C., Yang, S., Hu, D.: Hierarchical approximate policy iteration with binary-tree state space decomposition. IEEE Transactions on Neural Networks 22(12), 1863–1877 (2011)
Wang, F.Y., Zhang, H., Liu, D.: Adaptive dynamic programming: An introduction. IEEE Computational Intelligence Magazine 4(2), 39–47 (2009)
Sutton, R.S.: Generalization in reinforcement learning: successful examples using sparse coarse coding. Advances in Neural Information Processing Systems 8, 1038–1044 (1996)
Lagoudakis, M.G., Parr, R.: Least-squares policy iteration. Journal of Machine Learning Research 4, 1107–1149 (2003)
Liu, D., Javaherian, H., Kovalenko, O., Huang, T.: Adaptive critic learning techniques for engine torque and air-fuel ratio control. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 38(4), 988–993 (2008)
Mahadevan, S.: Representation policy iteration. In: Proceedings of the 21th Annual Conference on Uncertainty in Artificial Intelligence (AAAI), pp. 372–377 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Huang, Z., Xu, X., Ye, L., Zuo, L. (2013). Kernel-Based Representation Policy Iteration with Applications to Optimal Path Tracking of Wheeled Mobile Robots. In: Sun, C., Fang, F., Zhou, ZH., Yang, W., Liu, ZY. (eds) Intelligence Science and Big Data Engineering. IScIDE 2013. Lecture Notes in Computer Science, vol 8261. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-42057-3_91
Download citation
DOI: https://doi.org/10.1007/978-3-642-42057-3_91
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-42056-6
Online ISBN: 978-3-642-42057-3
eBook Packages: Computer ScienceComputer Science (R0)