Abstract
This paper presents a multi-modal interface for interaction between people with physical disabilities and an assistive robot. This interaction is performed through a dialogue mechanism and augmented 3D vision glasses to provide visual assistance to an end user commanding an assistive robot to perform Daily Life Activities (DLAs). The augmented 3D vision glasses may provide augmented reality vision of menus and information dialogues over the view of the real world, or in a simulator environment for laboratory tests and user evaluation. The actual dialogue is implemented as a finite state machine, and includes possibilities of Automatic Speech Recognition (ASR), and a Text-to-Speech (TTS) converter. The final study focuses on studying the effectiveness of these visual and auditory aids for enabling the end user to command the assistive robot ASIBOT to perform a given task.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Huete, A.J., Victores, J.G., Martinez, S., Giménez, A., Balaguer, C.: Personal autonomy rehabilitation in home environments by a portable assistive robot. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 42(4), 561–570 (2012)
Azuma, R.T.: A survey of augmented reality. Presence 6(4), 355–385 (1997)
Ronald, T.: Azuma, Yohan Baillot, Reinhold Behringer, Steven Feiner, Simon Julier, and Blair MacIntyre. Recent advances in augmented reality. IEEE Computer Graphics and Applications 21(6), 34–47 (2001)
Van Krevelen, D.W.F., Poelman, R.: A survey of augmented reality technologies, applications and limitations. International Journal of Virtual Reality 9(2), 1 (2010)
Green, S.A., Billinghurst, M., Chen, X., Chase, G.: Human-robot collaboration: A literature review and augmented reality approach in design. International Journal of Advanced Robotic Systems 5(1), 1–18 (2008)
Green, S.A., Chase, G., Chen, X., Billinghurst, M.: Evaluating the augmented reality human-robot collaboration system. International Journal of Intelligent Systems Technologies and Applications 8(1), 130–143 (2010)
Carmigniani, J., Furht, B., Anisetti, M., Ceravolo, P., Damiani, E., Ivkovic, M.: Augmented reality technologies, systems and applications. Multimedia Tools and Applications 51(1), 341–377 (2011)
Milgram, P., Yin, S., Grodski, J.J.: An augmented reality based teleoperation interface for unstructured environments. In: ANS 7th Meeting on Robotics and Remote Systems, Augusta, pp. 101–123 (1997)
Fang, H.C., Ong, S.K., Nee, A.Y.C.: Interactive robot trajectory planning and simulation using augmented reality. Robotics and Computer-Integrated Manufacturing 28(2), 227–237 (2012)
Tang, S.-L., Kwoh, C.-K., Teo, M.-Y., Sing, N.W., Ling, K.-V.: Augmented reality systems for medical applications. IEEE Engineering in Medicine and Biology Magazine 17(3), 49–58 (1998)
Su, L.-M., Vagvolgyi, B.P., Agarwal, R., Reiley, C.E., Taylor, R.H., Hager, G.D.: Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3d-ct to stereoscopic video registration. Urology 73(4), 896–900 (2009)
Luo, X., Kline, T., Fischer, H.C., Stubblefield, K.A., Kenyon, R.V., Kamper, D.G.: Integration of augmented reality and assistive devices for post-stroke hand opening rehabilitation. In: 27th Annual International Conference of the Engineering in Medicine and Biology Society, IEEE-EMBS 2005, pp. 6855–6858. IEEE (2005)
Braga, R.A., Petry, M., Moreira, A.P., Reis, L.P.: A development platform for intelligent wheelchairs for disabled people. In: International Conference on Informatics in Control, Automation and Robotics, pp. 115–121 (2008)
Alonso-Martin, F., Salichs, M.A.: Integration of a voice recognition system in a social robot. Cybernetics and Systems: An International Journal 42(4), 215–245 (2011)
Foster, M.E., By, T., Rickert, M., Knoll, A.: Human-robot dialogue for joint construction tasks. In: Proceedings of the 8th International Conference on Multimodal Interfaces, pp. 68–71. ACM (2006)
Seabra Lopes, L., Teixeira, A.: Human-robot interaction through spoken language dialogue. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2000, pp. 528–534. IEEE (2000)
Becker, E., Le, Z., Park, K., Lin, Y., Makedon, F.: Event-based experiments in an assistive environment using wireless sensor networks and voice recognition. In: Proceedings of the 2nd International Conference on PErvasive Technologies Related to Assistive Environments, p. 17. ACM (2009)
Fitzpatrick, P., Metta, G., Natale, L.: Towards long-lived robot genes. Robotics and Autonomous Systems 56(1), 29–45 (2008)
Morante, S.: Interfaz y librera para visión artificial, navegación y seguimiento en robótica. Undergraduate’s honors thesis, Universidad Carlos III de Madrid, Dpto. Ing. Sistemas y Automática (June 2012)
Victores, J.G., Morante, S., Jardón, A., Balaguer, C.: Give me the red can: Assistive robot task creation through multi-modal interaction. In: V Congreso Internacional de Diseño, Redes de Investigación y Tecnología para todos, DRT4ALL (September 2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Victores, J.G., Cañadillas, F.R., Morante, S., Jardón, A., Balaguer, C. (2014). Assistive Robot Multi-modal Interaction with Augmented 3D Vision and Dialogue. In: Armada, M., Sanfeliu, A., Ferre, M. (eds) ROBOT2013: First Iberian Robotics Conference. Advances in Intelligent Systems and Computing, vol 252. Springer, Cham. https://doi.org/10.1007/978-3-319-03413-3_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-03413-3_15
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-03412-6
Online ISBN: 978-3-319-03413-3
eBook Packages: EngineeringEngineering (R0)