Skip to main content
SpringerLink
Log in

Using Human Eye Gaze Patterns as Indicators of Need for Assistance from a Socially Assistive Robot

  • Conference paper
  • First Online:
Social Robotics (ICSR 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11876))

Included in the following conference series:

Abstract

With current growth in social robotics comes a need for well developed and fine tuned agents which respond to the user in a seamless and intuitive manner. Socially assistive robots in particular have become popular for their uses in care for older adults for medication adherence and socializing. Since eye gaze cues are important mediators in human-human interactions, we hypothesize that gaze patterns can be applied to human-robot interactions to identify when the user may need assistance. We reviewed videos (N = 16) of robot supported collaborative work to explore how recognition of gaze patterns for an assistive robot in the context of a medication management task can help predict when a user needs assistance. We found that mutual gaze is a better predictor than confirmatory request, gaze away, and goal reference. While eye gaze serves as an important indicator for need for assistance, it should be combined with other indicators, such as verbal cues or facial expressions to sufficiently represent assistance needed in the interaction and provide timely assistance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://chronoviz.com.

References

  1. Admoni, H., Scassellati, B.: Social eye gaze in human-robot interaction: a review. J. Hum.-Robot Interact. 6(1), 25–63 (2017)

    Article  Google Scholar 

  2. Andrist, S., Gleicher, M., Mutlu, B.: Looking coordinated: bidirectional gaze mechanisms for collaborative interaction with virtual characters. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 2571–2582. ACM (2017)

    Google Scholar 

  3. Andrist, S., Tan, X.Z., Gleicher, M., Mutlu, B.: Conversational gaze aversion for humanlike robots. In: Proceedings of the 2014 ACM/IEEE International Conference on Human-Robot Interaction, pp. 25–32. ACM (2014)

    Google Scholar 

  4. Bruce, A., Nourbakhsh, I., Simmons, R.: The role of expressiveness and attention in human-robot interaction. In: Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No. 02CH37292), vol. 4, pp. 4138–4142. IEEE (2002)

    Google Scholar 

  5. Casas, J., et al.: Architecture for a social assistive robot in cardiac rehabilitation. In: 2018 IEEE 2nd Colombian Conference on Robotics and Automation (CCRA), pp. 1–6. IEEE (2018)

    Google Scholar 

  6. Fang, R., Doering, M., Chai, J.Y.: Embodied collaborative referring expression generation in situated human-robot interaction. In: 2015 10th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 271–278. IEEE (2015)

    Google Scholar 

  7. Feil-Seifer, D., Matarić, M.J.: Defining socially assistive robotics. In: 9th International Conference on Rehabilitation Robotics, ICORR 2005, pp. 465–468 (2005)

    Google Scholar 

  8. Grigore, E.C., Eder, K., Pipe, A.G., Melhuish, C., Leonards, U.: Joint action understanding improves robot-to-human object handover. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4622–4629. IEEE (2013)

    Google Scholar 

  9. Huang, C.M., Andrist, S., Sauppé, A., Mutlu, B.: Using gaze patterns to predict task intent in collaboration. Front. Psychol. 6, 1049 (2015)

    Google Scholar 

  10. Jeste, D.V., Depp, C.A.: Positive mental aging. Am. J. Geriatr. Psychiatry 18(1), 1–3 (2010)

    Article  Google Scholar 

  11. Jokinen, K.: Gaze and gesture activity in communication. In: Stephanidis, C. (ed.) UAHCI 2009. LNCS, vol. 5615, pp. 537–546. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-02710-9_60

    Chapter  Google Scholar 

  12. Kendon, A., Cook, M.: The consistency of gaze patterns in social interaction. Br. J. Psychol. 60(4), 481–494 (1969)

    Article  Google Scholar 

  13. Kleinke, C.L.: Gaze and eye contact: a research review. Psychol. Bull. 100(1), 78 (1986)

    Article  Google Scholar 

  14. Matarić, M.J., Eriksson, J., Feil-Seifer, D.J., Winstein, C.J.: Socially assistive robotics for post-stroke rehabilitation. J. NeuroEng. Rehabil. 4(5), 1–9 (2007)

    Google Scholar 

  15. Mehlmann, G., Häring, M., Janowski, K., Baur, T., Gebhard, P., André, E.: Exploring a model of gaze for grounding in multimodal HRI. In: Proceedings of the 16th International Conference on Multimodal Interaction, pp. 247–254. ACM (2014)

    Google Scholar 

  16. Pennisi, P., et al.: Autism and social robotics: a systematic review. Autism Res. 9(2), 165–183 (2016)

    Article  Google Scholar 

  17. Satake, S., Kanda, T., Glas, D.F., Imai, M., Ishiguro, H., Hagita, N.: How to approach humans?: strategies for social robots to initiate interaction. In: Proceedings of the 4th ACM/IEEE International Conference on Human Robot Interaction, HRI 2009, pp. 109–116. ACM, New York (2009)

    Google Scholar 

  18. Schneider, B., Pea, R.: Real-time mutual gaze perception enhances collaborative learning and collaboration quality. Int. J. Comput.-Supported Collab. Learn. 8(4), 375–397 (2013)

    Article  Google Scholar 

  19. Tapus, A., Maja, M., Scassellatti, B.: The grand challenges in socially assistive robotics. IEEE Robot. Autom. Mag. 14(1), 35–42 (2007)

    Article  Google Scholar 

  20. Wada, K., Shibata, T.: Robot therapy in a care house - change of relationship among the residents and seal robot during a 2-month long study. IEEE Trans. Robot. 23(5), 972–980 (2007)

    Article  Google Scholar 

  21. Wilson, J.R., Lee, N.Y., Saechao, A., Tickle-Degnen, L., Scheutz, M.: Supporting human autonomy in a robot-assisted medication sorting task. Int. J. Soc. Robot. 10(5), 621–641 (2018)

    Article  Google Scholar 

  22. Wilson, J.R., Wransky, M., Tierno, J.: General approach to automatically generating need-based assistance. In: Proceeding of Advances in Cognitive Systems (2018))

    Google Scholar 

  23. Yamazaki, K., et al.: Prior-to-request and request behaviors within elderly day care: implications for developing service robots for use in multiparty settings. In: Bannon, L.J., Wagner, I., Gutwin, C., Harper, R.H.R., Schmidt, K. (eds.) ECSCW 2007, pp. 61–78. Springer, London (2007). https://doi.org/10.1007/978-1-84800-031-5_4

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Northwestern University, Evanston, IL, 60208, USA

    Ulyana Kurylo & Jason R. Wilson

Authors
  1. Ulyana Kurylo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jason R. Wilson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ulyana Kurylo .

Editor information

Editors and Affiliations

  1. Robotics Lab, Universidad Carlos III de Madrid, Leganés, Madrid, Spain

    Miguel A. Salichs

  2. The National University of Singapore, Singapore, Singapore

    Shuzhi Sam Ge

  3. Faculty of Industrial Design, Eindhoven University of Technology, Eindhoven, Noord-Brabant, The Netherlands

    Emilia Ivanova Barakova

  4. Mechanical & Industrial, Qatar University, Doha, Qatar

    John-John Cabibihan

  5. Department of Aerospace Engineering, The Pennsylvania State University, University Park, PA, USA

    Alan R. Wagner

  6. Robotics Lab - Department of Systems Engineering and Automation, Universidad Carlos III de Madrid, Leganés, Madrid, Spain

    Álvaro Castro-González

  7. Wichita State University, Wichita, KS, USA

    Hongsheng He

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kurylo, U., Wilson, J.R. (2019). Using Human Eye Gaze Patterns as Indicators of Need for Assistance from a Socially Assistive Robot. In: Salichs, M., et al. Social Robotics. ICSR 2019. Lecture Notes in Computer Science(), vol 11876. Springer, Cham. https://doi.org/10.1007/978-3-030-35888-4_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-35888-4_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-35887-7

  • Online ISBN: 978-3-030-35888-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation