Skip to main content

Advertisement

Springer Nature Link
Log in

Personalized Robot Interventions for Autistic Children: An Automated Methodology for Attention Assessment

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

We propose a robot-mediated therapy and assessment system for children with autism spectrum disorder (ASD) of mild to moderate severity and minimal verbal capabilities. The objectives of the robot interaction sessions is to improve the academic capabilities of ASD patients by increasing the length and the quality of their attention. The system uses a NAO robot and an added mobile display to present emotional cues and solicit appropriate emotional responses. The interaction is semi-autonomous with minimal human intervention. Interaction occurs within an adaptive dynamic scenario composed of 13 sections. The scenario allows adaptive customization based on the attention score history of each patient. The attention score is autonomously generated by the system and depends on face attention and joint attention cues and sound responses. The scoring system allows us to prove that the customized interaction system increases the engagement and attention capabilities of ASD patients. After performing a pilot study, involving 6 ASD children, out of a total of 11 considered in the clinical setup, we conducted a long-term study. This study empirically proves that the proposed assessment system represents the attention state of the patient with 82.4% accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Rotheram-Fuller E, Kasari C, Chamberlain B, Locke J (2010) Social involvement of children with autism spectrum disorders in elementary school classrooms. J Child Psychol Psychiatry 51:1227–1234

    Article  Google Scholar 

  2. Moriuchi JM, Klin A, Jones W (2016) Mechanisms of diminished attention to eyes in autism. Am J Psychiatry 174:26–35

    Article  Google Scholar 

  3. Freeth M, Foulsham T, Kingstone A (2019) What affects social attention? Social presence, eye contact and autistic traits. PLoS One 8(1):e53296. https://doi.org/10.1371/journal.pone.0053286

    Article  Google Scholar 

  4. Roddy A, O’Neill C (2019) The economic costs and its predictors for childhood autism spectrum disorders in Ireland: how is the burden distributed? Autism 23(5):1106–1118

    Article  Google Scholar 

  5. Huerta M, Bishop SL, Duncan A, Hus V, Lord C (2012) Application of DSM-5 criteria for autism spectrum disorder to three samples of children with DSM-IV diagnoses of pervasive developmental disorders. Am J Psychiatry 169:1056–1064

    Article  Google Scholar 

  6. Volkmar F, Siegel M, Woodbury-Smith M, King B, McCracken J, State M (2014) Practice parameter for the assessment and treatment of children and adolescents with autism spectrum disorder. J Am Acad Child Adolesc Psychiatry 53:237–257. https://doi.org/10.1016/j.jaac.2013.10.013

    Article  Google Scholar 

  7. Llaneza DC, DeLuke SV, Batista M, Crawley JN, Christodulu KV, Frye CA (2010) Communication, interventions, and scientific advances in autism: a commentary. Physiol Behav 100:268–276

    Article  Google Scholar 

  8. Paul R (2008) Interventions to improve communication in autism. Child Adolesc Psychiatr Clin N Am 17:835–856

    Article  Google Scholar 

  9. Costa S (2014) Robots as tools to help children with ASD to identify emotions. Autism 4(1):1–2

    Google Scholar 

  10. De Bel-Air F (2015) Demography, migration, and the labour market in the UAE. Gulf Labour Mark Migr 7:3–22

    Google Scholar 

  11. Aresti-Bartolome N, Garcia-Zapirain B (2014) Technologies as support tools for persons with autistic spectrum disorder: a systematic review. Int J Environ Res Public Health 11:7767–7802

    Article  Google Scholar 

  12. Pennisi P, Tonacci A, Tartarisco G, Billeci L, Ruta L, Gangemi S, Pioggia G (2016) Autism and social robotics: a systematic review. Autism Res J 9:165–183

    Article  Google Scholar 

  13. Alahbabi M, Almazroei F, Almarzoqi M, Almeheri A, Alkabi M, Al Nuaimi A, Cappuccio ML, Alnajjar F (2017) Avatar based interaction therapy: a potential therapeutic approach for children with Autism. In: IEEE international conference on mechatronics and automation, pp 480–484

  14. Zheng Z, Zhang L, Bekele E, Swanson A, Crittendon JA, Warren Z, Sarkar N (2013) Impact of robot-mediated interaction system on joint attention skills for children with autism. In: IEEE international conference on rehabilitation robotics, pp 1–8

  15. Robins B, Dautenhahn K, Dubowski J (2006) Does appearance matter in the interaction of children with autism with a humanoid robot? Interact Stud 7:479–512

    Article  Google Scholar 

  16. Zhang K, Liu X, Chen J, Liu L, Xu R, Li D (2017) Assessment of children with autism based on computer games compared with PEP scale. In: 2017 international conference of educational innovation through technology (EITT), pp 106–110

  17. Rudovic O, Lee J, Dai M, Schuller B, Picard RW (2018) Personalized machine learning for robot perception of affect and engagement in autism therapy. Sci Robot 3:eaao6760

    Article  Google Scholar 

  18. Ferrer EC, Rudovic O, Hardjono T, Pentland A (2018) Robochain: a secure data-sharing framework for human–robot interaction. arXiv preprint arXiv:180204480

  19. Anzalone SM, Tilmont E, Boucenna S, Xavier J (2014) How children with autism spectrum disorder behave and explore the 4-dimensional (spatial 3D+ time) environment during a joint attention induction task with a robot. Res Autism Spectr Disord 8:814–826

    Article  Google Scholar 

  20. Kim ES, Paul R, Shic F, Scassellati B (2012) Bridging the research gap: making HRI useful to individuals with autism. J Hum Robot Interact 1:26–54

    Article  Google Scholar 

  21. Bensalem S, Gallien M, Ingrand F, Kahloul I, Thanh-Hung N (2009) Designing autonomous robots. IEEE Robot Autom Mag 16:67–77

    Article  Google Scholar 

  22. Billard A, Robins B, Nadel J, Dautenhahn K (2006) Building Robota, a mini-humanoid robot for the rehabilitation of children with autism. Assist Technol 19:37–49

    Article  Google Scholar 

  23. Kozima H, Nakagawa C, Yasuda Y (2005) Interactive robots for communication-care: a case-study in autism therapy. In: IEEE international workshop on robot and human interactive communication, pp 341–346

  24. Schopler E, Van Bourgondien M, Wellman J, Love S (2010) Childhood autism rating scale—second edition (CARS2): manual. Western Psychological Services, Los Angeles

    Google Scholar 

  25. Alnajjar FS, Renawi AM, Cappuccio M, Mubain O (2019) A low-cost autonomous attention assessment system for robot intervention with autistic children. In: 2019 IEEE global engineering education conference (EDUCON)

  26. Pokress SC, Veiga JJD (2013) MIT app inventor: enabling personal mobile computing. arXiv preprint arXiv:13102830

  27. Wilson PI, Fernandez J (2006) Facial feature detection using Haar classifiers J Comput Sci Coll 21:127–133

    Google Scholar 

  28. Ahmad MI, Mubin O, Orlando J (2017) Adaptive social robot for sustaining social engagement during long-term children–robot interaction. Int J Hum Comput Interact 33:943–962

    Article  Google Scholar 

  29. Marcos-Pablos S, González-Pablos E, Martín-Lorenzo C, Flores LA, Gómez-García-Bermejo J, Zalama E (2016) Virtual avatar for emotion recognition in patients with schizophrenia: a pilot study. Front Hum Neurosci 10:421

    Google Scholar 

  30. Powell S (1996) The use of computers in teaching people with autism. In: Autism on the agenda: papers from a National Autistic Society conference, London, pp 128–132

Download references

Funding

This study was funded by a 31R188-Research AUA- ZCHS -1–2018, Zayed Health Center.

Author information

Authors and Affiliations

  1. College of Information Technology, United Arab Emirates University, 15551, Al Ain, United Arab Emirates

    Fady Alnajjar, Massimiliano Cappuccio & Abdulrahman Renawi

  2. Intelligent Behavior Control Unit, RIKEN Center for Brain Science (CBS), Nagoya, Japan

    Fady Alnajjar

  3. Values in Defense and Security Technology, School of Engineering and Information Technology, University of New South Wales at Australian Defense Force Academy, Canberra, ACT, Australia

    Massimiliano Cappuccio

  4. School of Computer, Data and Mathematical Sciences, Western Sydney University, Sydney, Australia

    Omar Mubin

  5. Department of Artificial Intelligence, Faculty of Computer Science and Information Technology, University of Malaya, Kuala Lumpur, Malaysia

    Chu Kiong Loo

Authors
  1. Fady Alnajjar
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Massimiliano Cappuccio
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Abdulrahman Renawi
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Omar Mubin
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Chu Kiong Loo
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Fady Alnajjar.

Ethics declarations

Conflict of interest

The second and fourth authors are coeditors of the special issue to which this paper is submitted, however do not anticipate to be involved in the review process for this paper. The other authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alnajjar, F., Cappuccio, M., Renawi, A. et al. Personalized Robot Interventions for Autistic Children: An Automated Methodology for Attention Assessment. Int J of Soc Robotics 13, 67–82 (2021). https://doi.org/10.1007/s12369-020-00639-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-020-00639-8

Keywords