Skip to main content
SpringerLink
Log in

A Feasibility Study Evaluating the Emotionally Expressive Robot SAM

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

This two-part feasibility study evaluated the functionality and acceptability of Socially Animated Machine (SAM), a humanoid robotic monkey developed to elicit social interaction in children with Autism Spectrum Disorder (ASD). Socially Animated Machine was designed with an approachable, animal-like appearance, while preserving the essential features of a human face. The intent was to design a robot that would be interesting and engaging to children with ASD, yet maintain the capability to model facial expressions that convey emotional subtlety. Study 1 evaluated the accuracy of SAM’s emotional facial expressions. Typically developing children (\(N = 35\)) labeled and matched SAM’s expressions to photos of human expressions with moderate-to-substantial levels of agreement. Study 2 compared children’s level of social engagement across an interaction with SAM and an interaction with an adult experimenter. Children with ASD (\(N = 13\)) spent significantly more time attending to the partner’s face while interacting with SAM. When asked to rate their interaction with SAM, children with ASD reported high levels of happiness and comfort and requested additional interactions. These results suggest that SAM may serve as a useful tool in interventions to improve social skills, including emotion recognition, in children with ASD.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Centers for Disease Control and Prevention (2016) Prevalence and characteristics of autism spectrum disorder among children aged 8 years—autism and developmental disabilities monitoring network, 11 sites, United States, 2012. MMWR Surveill Summary 65(3):1–23

    Article  Google Scholar 

  2. American Psychiatric Association (2013) Diagnostic and statistical manual of mental disorders, 5th edn. American Psychiatric Publishing, Arlington

    Book  Google Scholar 

  3. Hobson RP (1986) The autistic child’s appraisal of expressions of emotion. J Child Psychol Psychiatry 27:321–342

    Article  Google Scholar 

  4. Paul R (2008) Interventions to improve communication in autism. Child Adolesc Psychiatr Clin N Am 17(4):835–856

    Article  Google Scholar 

  5. Shaked M, Yirmiya N (2003) Understanding social difficulties. In: Prior MR (ed) Learning and behavior problems in asperger syndrome. Guilford Press, New York, pp 104–125

    Google Scholar 

  6. Tager-Flusberg H, Joseph RM (2003) Identifying neurocognitive phenotypes in autism. Philos Trans R Soc Lond B Biol Sci 358(1430):303–314

    Article  Google Scholar 

  7. Tager-Flusberg H, Paul R, Lord C (2005) Language and communication in autism. In: Volkmar FR, Paul R, Klin A, Cohen D (eds) Handbook of autism and pervasive developmental disorders. Wiley, New York, pp 335–364

    Google Scholar 

  8. Buitelaar JK, van Engeland H, de Kogel KH, de Vries H, van Hooff JA (1991) Differences in the structure of social behaviour of autistic children and non-autistic retarded controls. J Child Psychol Psychiatry 32(6):995–1015

    Article  Google Scholar 

  9. Volkmar FR, Paul R, Klin A, Cohen DJ (2005) Handbook of autism and pervasive developmental disorders, diagnosis, development, neurobiology, and behavior. Wiley, New York

    Book  Google Scholar 

  10. Tantam D (2003) The challenge of adolescents and adults with asperger syndrome. Child Adolesc Psychiatr Clin N Am 12:143–163

    Article  Google Scholar 

  11. Chamberlain BO (2001) Isolation or involvement? The social networks of children with autism included in regular classes. Dissertation, University of California, Los Angeles

  12. Howlin P, Goode S (1998) Outcome in adult life for people with autism, asperger syndrome. In: Volkmar FR (ed) Autism and pervasive developmental disorders. Cambridge University Press, New York

    Google Scholar 

  13. White SW, Keonig K, Scahill L (2007) Social skills development in children with autism spectrum disorders: a review of the intervention research. J Autism Dev Disord 37(10):1858–1868

    Article  Google Scholar 

  14. Koegel LK, Koegel RL, Harrower JK, Carter CM (1999) Pivotal response intervention I: overview of approach. Res Pract Persons Severe Disabil 24(3):174–185

    Article  Google Scholar 

  15. Dautenhahn K, Werry I (2004) Towards interactive robots in autism therapy: background, motivation and challenges. Pragmat Cogn 12:1–35

    Article  Google Scholar 

  16. Ricks DJ, Colton MB (2010) Trends and considerations in robot-assisted autism therapy. In: Proceedings of the IEEE international conference on robotics and automation, pp 4354–4359

  17. Robins B, Dickerson P, Stribling P, Dautenhahn K (2004) Robot-mediated joint attention in children with autism: a case study in robot–human interaction. Interact Stud 5(2):161–198

    Article  Google Scholar 

  18. Scassellati B, Admoni H, Mataric M (2012) Robots for use in autism research. Annu Rev Biomed Eng 14:275–294

    Article  Google Scholar 

  19. Diehl JJ, Schmitt LM, Villano M, Crowell CR (2012) The clinical use of robots for individuals with autism spectrum disorders: a critical review. Res Autism Spect Dis 6(1):249–262

    Article  Google Scholar 

  20. Scassellati B (2007) How social robots will help us to diagnose, treat, and understand autism. Int J Robot Res 28:552–563

    Article  Google Scholar 

  21. Kozima H, Nakagawa C (2006) Interactive robots as facilitators of children’s social development. In: Lazinica A (ed) Mobile robots towards new applications. i-Tech Education and Publishing, Munich, p 784

    Google Scholar 

  22. Pioggia G, Igliozzi R, Ferro M, Ahluwalia A, Muratori F, De Rossi D (2005) An android for enhancing social skills and emotion recognition in people with autism. IEEE Trans Neural Syst Rehabil Eng 13(4):507–515

    Article  Google Scholar 

  23. Pioggia G, Igliozzi R, Sica ML, Ferro M, Muratori F, Ahluwalia A, De Rossi D (2008) Exploring emotional and imitational android-based interactions in autistic spectrum disorders. J Cyber Ther Rehabil 1(1):49–61

    Google Scholar 

  24. Robins B, Dautenhahn K, Te Boekhorst R, Billard A (2005) Robotic assistants in therapy and education of children with autism: can a small humanoid robot help encourage social interaction skills? Univers Access Inf 4(2):105–120

    Article  Google Scholar 

  25. Kozima H, Michalowski MP, Nakagawa C (2009) Keepon. Int J Soc Robot 1(1):3–18

    Article  Google Scholar 

  26. Kim ES, Berkovits LD, Bernier EP, Leyzberg D, Shic F, Paul R, Scassellati B (2012) Social robots as embedded reinforcers of social behavior in children with autism. J Autism Dev Disord 43(5):1038–1049

    Article  Google Scholar 

  27. Goris K, Saldien J, Vanderborght B, Lefeber D (2010) Probo, an intelligent huggable robot for HRI studies with children. In: Chugo D (ed) Human–robot interaction. INTECH, Rijeka, p 288

    Google Scholar 

  28. Simut R, Van de Perre C, Vanderborght B, Saldien J, Rusu AS, Pintea S, Vanderfaeillie J, Lefeber D, David DO (2011) The huggable social robot Probo for Social Story telling for robot assisted therapy with ASD children. In: Proceedings of the 3rd international conference on social robotics (ICSR-11), pp 97–100

  29. Boccanfuso L, O’Kane JM (2011) CHARLIE: an adaptive robot with hand and face tracking for use in autism therapy. Int J Soc Robot 3:337–347

    Article  Google Scholar 

  30. Iacono I, Lehmann H, Marti P, Robins B, Dautenhahn K (2011) Robots as social mediators for children with autism—a preliminary analysis comparing two different robotic platforms. In: IEEE international conference on development and learning (ICDL-11), vol 2, pp 1–6

  31. Feil-Seifer D, Matarić MJ (2009) Toward socially assistive robotics for augmenting interventions for children with autism spectrum disorders. In: Khatib O, Kumar V, Pappas GJ (eds) Experimental robotics. Springer, Berlin, pp 201–210

  32. Dautenhahn K, Nehaniv CL, Walters M, Robins B, Kose-Bagci H, Mirza NA, Blow M (2009) Kaspar—a minimally expressive humanoid robot for human–robot interaction research. Appl Bionics Biomech 6:369–397

    Article  Google Scholar 

  33. Wainer J, Dautenhahn K, Robins B, Amirabdollahian F (2014) A pilot study with a novel setup for collaborative play of the humanoid robot KASPAR with children with autism. Int J Soc Robot 6:45–65

    Article  Google Scholar 

  34. Aldebaran Robotics (2006) NAO [robot]. www.ald.softbankrobotics.com

  35. Chevalier P, Martin J, Isableu B, Bazile C, Tapus A (2017) Impact of sensory preferences of individuals with autism on the recognition of emotions expressed by two robots, an avatar, and a human. Auton Robot 41(3):613–635

    Article  Google Scholar 

  36. Huskens B, Verschuur R, Gillesen J, Didden R, Barakova E (2013) Promoting question-asking in school-aged children with autism spectrum disorders: effectiveness of a robot intervention compared to a human–trainer intervention. Dev Neurorehabil 16(5):345–356

    Article  Google Scholar 

  37. Miskam MA, Masnin NF, Jamhuri MH, Shamsuddin S, Omar AR, Yussof H (2014) Encouraging children with autism to improve social and communication skills through the game-based approach. Procedia Comput Sci 42:93–98

    Article  Google Scholar 

  38. Tapus A, Peca A, Aly A, Pop C, Jisa L, Pintea S, Rusu A, David D (2012) Children with autism social engagement in interaction with NAO, an imitative robot—a series of single case experiments. Interact Stud 13(3):315–347

    Article  Google Scholar 

  39. Billard A, Robins B, Dautenhahn K, Nadel J (2006) Building robota, a mini-humanoid robot for the rehabilitation of children with autism. RESNA Assist Technol J 19(1):37–49

    Article  Google Scholar 

  40. Dautenhahn K, Billard A (2002) Games children with autism can play with Robota, a humanoid robotic doll. In: Keates S, Clarkson PJ, Langdon PM, Robinson P (eds) Cambridge workshop on universal access and assistive technology. Springer, London, pp 179–190

    Chapter  Google Scholar 

  41. Baron-Cohen S, Golan O, Ashwin E (2009) Can emotion recognition be taught to children with autism spectrum conditions? Philos Trans R Soc B 364:3567–3574

    Article  Google Scholar 

  42. Ashwin C, Chapman E, Colle L, Baron-Cohen S (2006) Impaired recognition of negative basic emotions in autism: a test of the amygdala theory. Soc Neurosci 1:349–363

    Article  Google Scholar 

  43. Baron-Cohen S (2000) Theory of mind and autism: a fifteen year review. In: Baron-Cohen A, Tager-Flusberg H, Cohen D (eds) Understanding other minds: perspectives from developmental cognitive neuroscience, 2nd edn. Oxford University Press, Oxford, pp 3–20

    Google Scholar 

  44. Mehrabian A (1968) Communication without words. Psychol Today 2(9):52–55

    Google Scholar 

  45. Mori M (1970) The uncanny valley. Energy 7:33–35

    Google Scholar 

  46. Saygin AP, Chaminade T, Ishiguro H, Driver J, Frith C (2011) The thing that should not be: predictive coding and the uncanny valley in perceiving human and humanoid robot actions. Soc Cogn Affect Neurosci 7(4):413–422

    Article  Google Scholar 

  47. Microsoft (2013) Surface Pro 2 [tablet]. www.microsoft.com

  48. Adobe Systems Incorporated (2017) Photoshop [computer program]. www.photoshop.com

  49. Eckman P, Friesen WV (1975) Unmasking the face: a guide to recognizing emotions from facial clues. Prentice-Hall, Englewood Cliffs

    Google Scholar 

  50. Neurobehavioral Systems (2004) Presentation, version 17.2 [computer program]. www.neurobs.com

  51. Arduino LLC (2005) Arduino [computer program]. www.arduino.cc

  52. Fleiss JL (1971) Measuring nominal scale agreement among many raters. Psychol Bull 76:378–382

    Article  Google Scholar 

  53. Landis J, Koch G (1977) The measurement of observer agreement for categorical data. Biometrics 33:159

    Article  MATH  Google Scholar 

  54. Lord C, McGee JP (2001) Educating children with autism. National Academy Press, Washington

    Google Scholar 

  55. Lord C, Rutter M, Goode S, Heemsbergen J, Jordan H, Mawhood L, Schopler E (1989) Autism diagnostic observation schedule: a standardized observation of communicative and social behavior. J Autism Dev Disord 19:185–212

    Article  Google Scholar 

  56. Kaufman AS, Kaufman NL (1990) Kaufman brief intelligence test. American Guidance Service, Circle Pines

    Google Scholar 

  57. Seeing Machines (2009) FaceLAB, version 5 [computer program]. www.seeingmachines.com

  58. MacDonald PM, Kirkpatrick SW, Sullivan LA (1996) Schematic drawings of facial expressions for emotion recognition and interpretation by preschool-aged children. Genet Soc Gen Psychol Monogr 122:373–388

    Google Scholar 

  59. Castellano G, Pereira A, Leite I, Paiva A, McOwan PW (2009) Detecting user engagement with a robot companion using task and social interaction-based features. In: Proceedings of the international conference on multimodal interfaces (ICMI-09), pp 119–126

  60. Michalowski MP, Sabanovic S, Simmons R (2006) A spatial model of engagement for a social robot. In: IEEE international workshop on advanced motion control (AMC-06), pp 762–767

  61. Peters C, Asteriadis S, Karpouzis K, de Sevin E (2008) Towards a real-time gaze-based shared attention for a virtual agent. In: Workshop on affective interaction in natural environments (AFFINE), ACM international conference on multimodal interfaces (ICMI-08)

  62. Lahiri U, Warren Z, Sarkar N (2011) Design of a gaze-sensitive virtual social interactive system for children with autism. IEEE Trans Neural Syst Rehabil Eng 19:443–452

    Article  Google Scholar 

  63. Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. Erlbaum, Hillsdale

    MATH  Google Scholar 

  64. Saldien J, Goris K, Vanderborght B, Vanderfaeillie J, Lefeber D (2010) Expressing emotions with the social robot probo. Int J Soc Robot 2(4):377–389

    Article  Google Scholar 

  65. Breazeal C (2002) Designing sociable robots. MIT Press, Cambridge

    MATH  Google Scholar 

  66. Sosnowski S, Bittermann A, Kuhnlenz K, Buss M (2006) Design and evaluation of emotion-display EDDIE. In: IEEE/RSJ international conference on intelligent robots and systems (IROS-06), pp 3113–3118

  67. Canamero LD, Fredslund J (2000) How does it feel? Emotional interaction with a humanoid lego robot. In: Proceedings of the AAAI fall symposium, pp 23–28

Download references

Acknowledgements

The authors thank the Social Technology for Autism Research (STAR) Lab at the University of Alabama at Birmingham (UAB) and the children and families who participated in the study and made this research possible. This study was funded in part by a grant from Civitan International Research Center. This paper is adapted from the author’s master’s thesis.

Funding This study was funded in part by a grant from Civitan International Research Center.

Author information

Authors and Affiliations

  1. Department of Psychology, University of Alabama at Birmingham, Campbell Hall, Suite 415, 1720 2nd Avenue South, Birmingham, AL, 35294, USA

    Sarah A. Koch, Carl E. Stevens, Christian D. Clesi, Jenna B. Lebersfeld, Alyssa G. Sellers, Myriah E. McNew, Fred J. Biasini, Franklin R. Amthor & Maria I. Hopkins

  2. Department of Psychology, Florida International University, Miami, FL, USA

    Myriah E. McNew

Authors
  1. Sarah A. Koch
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carl E. Stevens
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christian D. Clesi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jenna B. Lebersfeld
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alyssa G. Sellers
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Myriah E. McNew
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Fred J. Biasini
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Franklin R. Amthor
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Maria I. Hopkins
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sarah A. Koch.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Koch, S.A., Stevens, C.E., Clesi, C.D. et al. A Feasibility Study Evaluating the Emotionally Expressive Robot SAM. Int J of Soc Robotics 9, 601–613 (2017). https://doi.org/10.1007/s12369-017-0419-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-017-0419-6

Keywords

Search

Navigation