Abstract
Climbing ramps is an important ability for humanoid robots: ramps exist everywhere in the world, such as in accessibility ramps and building entrances. This works proposes the use of Reinforcement Learning to learn the action policy that will make a robot walk in an upright position, in a lightly sloped terrain. The proposed architecture of our system is a two-layer combination of the traditional gait generation control loop with a reinforcement learning component. This allows the use of an accelerometer to generate a correction for the gait, when the slope of the floor where the robot is walking changes. Experiments performed on a real robot showed that the proposed architecture is a good solution for the stability problem.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Westervelt, E.R., Grizzle, J.W., Chevallereau, C., Choi, J.H., Morris, B.: Feedback Control of Dynamic Bipedal Robot Locomotion, vol. 28. CRC Press, Boca Raton (2007)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, vol. 1. MIT Press, Cambridge (1998)
Ha, I., Tamura, Y., Asama, H.: Gait pattern generation and stabilization for humanoid robot based on coupled oscillators. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3207–3212 (2011)
Ha, I., Tamura, Y., Asama, H., Han, J., Hong, D.W.: Development of open humanoid platform DARwIn-OP. In: SICE Annual Conference (SICE), pp. 2178–2181 (2011)
Perico, D.H., Silva, I.J., Vilão Jr., C.O., Homem, T.P., Destro, R.C., Tonidandel, F., Bianchi, R.A.: Newton: a high level control humanoid robot for the RoboCup Soccer KidSize League. In: Osório, F.S., Wolf, D.F., Branco, K.C., Grassi, V., Becker, M., Romero, R.A.F. (eds.) Robotics. CCIS, vol. 507, pp. 53–73. Springer, Heidelberg (2015). doi:10.1007/978-3-662-48134-9_4
Eva Robot (2016). http://fei.edu.br/brahur2016/artigos/Artigo%202%20-%20UFU.pdf
WF Wolves and Taura Bots - Team Description Paper (2016). http://www.robocup2016.org/media/symposium/Team-Description-Papers/Humanoid/RoboCup_2016_Humanoid_TeenSize_TDP_WF_Wolves_Taura_Bots.pdf
Watkins, C.: Learning from delayed rewards. Doctoral dissertation, University of Cambridge (1989)
Marder, E., Bucher, D.: Central pattern generators and the control of rhythmic movements. Curr. Biol. 11, R986–R996 (2001)
Mcgeer, T.: Passive dynamic walking. Int. J. Robot. Res. 9, 62–82 (1990)
Vukobratović, M., Borovac, B.: Zero-moment point thirty five years of its life. Int. J. Humanoid Rob. 1, 157–173 (2004). World Scientific
Kajita, S., Kanehiro, F., Kaneko, K., Yokoi, K., Hirukawa, H.: The 3D linear inverted pendulum mode: a simple modeling for a biped walking pattern generation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 1, pp. 239–246 (2001)
Zhao, M., Dong, H., Zhang, N.: The instantaneous leg extension model of virtual slope walking. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS, pp. 3220–3225 (2009)
Vukobratović, M., Juricic, D.: Contribution to the synthesis of biped gait. IEEE Trans. Biomed. Eng. 16(2), 1–6 (1969)
Zheng, Y.F., Shen, J.: Gait synthesis for the SD-2 biped robot to climb sloping surface. IEEE Trans. Robot. Autom. 6, 86–96 (1990)
Chew, C.M., Pratt, J., Pratt, G.: Blind walking of a planar bipedal robot on sloped terrain. In: IEEE International Conference on Robotics and Automation, vol. 1, pp. 381–386 (1999)
Zhou, C., Yue, P.K., Ni, J., Chan, S.B.: Dynamically stable gait planning for a humanoid robot to climb sloping surface. In: IEEE Conference on Robotics, Automation and Mechatronics, vol. 1, pp. 341–346 (2004)
Hong, Y.D., Lee, B.J., Kim, J.H.: Command state-based modifiable walking pattern generation on an inclined plane in pitch and roll directions for humanoid robots. IEEE/ASME Trans. Mechatron. 16, 783–789 (2011)
Huang, W., Chew, C.M., Zheng, Y., Hong, G.S.: Pattern generation for bipedal walking on slopes and stairs. In: 8th IEEE-RAS International Conference on Humanoid Robots, pp. 205–210 (2008)
Iverach-Brereton, C., Baltes, J., Postnikoff, B., Carrier, D., Anderson, J.: Fuzzy logic control of a humanoid robot on unstable terrain. In: Almeida, L., Ji, J., Steinbauer, G., Luke, S. (eds.) RoboCup 2015. LNCS, vol. 9513, pp. 202–213. Springer, Heidelberg (2015). doi:10.1007/978-3-319-29339-4_17
Baltes, J., Iverach-Brereton, C., Anderson, J.: Human inspired control of a small humanoid robot in highly dynamic environments or Jimmy Darwin rocks the Bongo Board. In: Bianchi, R.A.C., Akin, H.L., Ramamoorthy, S., Sugiura, K. (eds.) RoboCup 2014. LNCS, vol. 8992, pp. 466–477. Springer, Heidelberg (2015). doi:10.1007/978-3-319-18615-3_38
Perico, D.H., Silva, I.J., Vilao, C.O., Homem, T.P., Destro, R.C., Tonidandel, F., Bianchi, R.: Hardware and software aspects of the design and assembly of a new humanoid robot for RoboCup Soccer. In: Robotics: SBR-LARS Robotics Symposium and Robocontrol (SBR LARS Robocontrol), pp. 73–78 (2014)
Farchy, A., Barrett, S., MacAlpine, P., Stone, P.: Humanoid robots learning to walk faster: from the real world to simulation and back. In: Proceedings of the 2013 International Conference on Autonomous Agents and Multi-agent Systems, pp. 39–46 (2013)
Hester, T., Quinlan, M., Stone, P.: Generalized model learning for reinforcement learning on a humanoid robot. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 2369–2374 (2010)
Acknowledgment
The authors would like to acknowledge the Centro Universitário FEI and the Robotics and Artificial Intelligence Laboratory for supporting this project. The authors would also like to thank the scholarships provided by CAPES and CNPq.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Silva, I.J., Perico, D.H., Homem, T.P.D., Vilão, C.O., Tonidandel, F., Bianchi, R.A.C. (2016). Humanoid Robot Gait on Sloping Floors Using Reinforcement Learning. In: Santos Osório, F., Sales Gonçalves, R. (eds) Robotics. SBR LARS 2016 2016. Communications in Computer and Information Science, vol 619. Springer, Cham. https://doi.org/10.1007/978-3-319-47247-8_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-47247-8_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-47246-1
Online ISBN: 978-3-319-47247-8
eBook Packages: Computer ScienceComputer Science (R0)