Skip to main content

Advertisement

SpringerLink
Log in

Automated Video Tracking of Autistic Children’s Movement During Caregiver-Child Interaction: An Exploratory Study

  • Original Paper
  • Published:
Journal of Autism and Developmental Disorders Aims and scope Submit manuscript

Abstract

Objective, quantitative measures of caregiver-child interaction during play are needed to complement caregiver or examiner ratings for clinical assessment and tracking intervention responses. In this exploratory study, we examined the feasibility of using automated video tracking, Noldus EthoVision XT, to measure 159 2-to-7-year-old autistic children’s patterns of movement during play-based, caregiver-child interactions and examined their associations with standard clinical measures and human observational coding of caregiver-child joint engagement. Results revealed that autistic children who exhibited higher durations and velocity of movement were, on average, younger, had lower cognitive abilities, greater autism-related features, spent less time attending to the caregiver, and showed lower levels of joint engagement. After adjusting for age and nonverbal cognitive abilities, we found that children who remained in close proximity to their caregiver were more likely to engage in joint engagement that required support from the caregiver. These findings suggest that video tracking offers promise as a scalable, quantitative, and relevant measure of autism-related behaviors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Adamson, L. B., Bakeman, R., Deckner, D. F., & Nelson, P. B. (2012). Rating parent-child interactions: Joint engagement, communication dynamics, and shared topics in autism, Down syndrome, and typical development. Journal Of Autism And Developmental Disorders, 42(12), 2622–2635. https://doi.org/10.1007/s10803-012-1520-1.

    Article  PubMed  PubMed Central  Google Scholar 

  • Adamson, L. B., Bakeman, R., Suma, K., & Robins, D. L. (2019). An expanded view of joint attention: Skill, Engagement, and Language in typical development and autism. Child Development, 90(1), e1–e18. https://doi.org/10.1111/cdev.12973.

    Article  PubMed  Google Scholar 

  • Aman, M., & Singh, N. (1986). Aberrant behavior checklist manual. Slosson Educational Publications.

  • Anagnostou, E., Jones, N., Huerta, M., Halladay, A. K., Wang, P., Scahill, L., Horrigan, J. P., Kasari, C., Lord, C., Choi, D., Sullivan, K., & Dawson, G. (2015). Measuring social communication behaviors as a treatment endpoint in individuals with autism spectrum disorder. Autism, 19(5), 622–636. https://doi.org/10.1177/1362361314542955.

    Article  PubMed  Google Scholar 

  • Bakeman, R., & Quera, V. (2011). Sequential analysis and observational methods for the behavioral sciences. Cambridge University Press.

  • Bey, A. L., Wang, X., Yan, H., Kim, N., Passman, R. L., Yang, Y., Cao, X., Towers, A. J., Hulbert, S. W., Duffney, L. J., Gaidis, E., Rodriguiz, R. M., Wetsel, W. C., Yin, H. H., & Jiang, Y. H. (2018). Brain region-specific disruption of Shank3 in mice reveals a dissociation for cortical and striatal circuits in autism-related behaviors. Transl Psychiatry, 8(1), 94. https://doi.org/10.1038/s41398-018-0142-6.

    Article  PubMed  PubMed Central  Google Scholar 

  • Bishop, S., Farmer, C., Kaat, A., Georgiades, S., Kanne, S., & Thurm, A. (2019). The need for a developmentally based measure of Social Communication Skills. Journal Of The American Academy Of Child And Adolescent Psychiatry, 58(6), 555–560. https://doi.org/10.1016/j.jaac.2018.12.010.

    Article  PubMed  PubMed Central  Google Scholar 

  • Blank, M. S., Turner, J. B., Fisher, P. W., Guthrie, E. B., & Whitaker, A. H. (2020). The need for a clinically useful Schema of Social Communication. Journal Of The American Academy Of Child And Adolescent Psychiatry, 59(11), 1198–1200. https://doi.org/10.1016/j.jaac.2020.03.011.

    Article  PubMed  Google Scholar 

  • Carpenter, K. L. H., Hahemi, J., Campbell, K., Lippmann, S. J., Baker, J. P., Egger, H. L., Espinosa, S., Vermeer, S., Sapiro, G., & Dawson, G. (2021). Digital behavioral phenotyping detects atypical pattern of facial expression in toddlers with autism. Autism Research, 14(3), 488–499. https://doi.org/10.1002/aur.2391.

    Article  PubMed  Google Scholar 

  • Chang, Y. C., Cole, T. B., & Costa, L. G. (2017). Behavioral phenotyping for Autism Spectrum Disorders in mice. Current Protocols In Toxicology / Editorial Board, Mahin D. Maines (Editor-In-Chief). [Et Al.], 72, 112211–112221. https://doi.org/10.1002/cptx.19.

    Article  Google Scholar 

  • Cohen, I., & Sudhalter, V. (2005). Pervasive developmental disorder behavior inventory. Psychological Assessment Resources, Inc.

  • Cohen, I. L., Gardner, J. M., Karmel, B. Z., & Kim, S. Y. (2014). Rating scale measures are associated with Noldus EthoVision-XT video tracking of behaviors of children on the autism spectrum. Mol Autism, 5(1), 15. https://doi.org/10.1186/2040-2392-5-15.

    Article  PubMed  PubMed Central  Google Scholar 

  • Dawson, G., Rogers, S., Munson, J., Smith, M., Winter, J., Greenson, J., Donaldson, A., & Varley, J. (2010). Randomized, controlled trial of an intervention for toddlers with autism: The early start Denver Model. Pediatrics, 125(1), e17–23. https://doi.org/10.1542/peds.2009-0958.

    Article  PubMed  Google Scholar 

  • Dawson, G., Sun, J. M., Baker, J., Carpenter, K., Compton, S., Deaver, M., Franz, L., Heilbron, N., Herold, B., Horrigan, J., Howard, J., Kosinski, A., Major, S., Murias, M., Page, K., Prasad, V. K., Sabatos-DeVito, M., Sanfilippo, F., Sikich, L., Simmons, R., Song, A., Vermeer, S., Waters-Pick, B., Troy, J., & Kurtzberg, J. (2020). A phase II randomized clinical trial of the safety and efficacy of intravenous umbilical cord blood infusion for treatment of children with Autism Spectrum Disorder. Journal Of Pediatrics, 222, 164–173e165. https://doi.org/10.1016/j.jpeds.2020.03.011.

    Article  PubMed  Google Scholar 

  • de Belen, R. A. J., Bednarz, T., Sowmya, A., & Del Favero, D. (2020). Computer vision in autism spectrum disorder research: A systematic review of published studies from 2009 to 2019. Transl Psychiatry, 10(1), 333. https://doi.org/10.1038/s41398-020-01015-w.

    Article  PubMed  PubMed Central  Google Scholar 

  • Elliott, C. (2007). Differential ability Scales-II. Harcourt Assessment.

  • Gibson, J. L., Pritchard, E., & de Lemos, C. (2021). Play-based interventions to support social and communication development in autistic children aged 2–8 years: A scoping review. Autism & Developmental Language Impairments, 6, 23969415211015840. https://doi.org/10.1177/23969415211015840.

    Article  Google Scholar 

  • Grabowski, K., Rynkiewicz, A., Lassalle, A., Baron-Cohen, S., Schuller, B., Cummins, N., Baird, A., Podgorska-Bednarz, J., Pieniazek, A., & Lucka, I. (2019). Emotional expression in psychiatric conditions: New technology for clinicians. Psychiatry Clin Neurosci, 73(2), 50–62. https://doi.org/10.1111/pcn.12799.

    Article  PubMed  Google Scholar 

  • Guy, W. (1976). ECDEU Assessment Manual for Psychopharmacology. US Department of Heath, Education, and Welfare Public Health Service Alcohol, Drug Abuse, and Mental Health Administration.

  • Holmes, E., & Willoughby, T. (2005). Play behaviour of children with autism spectrum disorders. Journal of Intellectual & Developmental Disability, 30(3), 156–164. https://doi.org/10.1080/13668250500204034.

    Article  Google Scholar 

  • Jung, S., & Sainato, D. M. (2013). Teaching play skills to young children with autism. Journal Of Intellectual Developmental Disability, 38(1), 74–90. https://doi.org/10.3109/13668250.2012.732220.

    Article  PubMed  Google Scholar 

  • Kaale, A., Smith, L., Nordahl-Hansen, A., Fagerland, M. W., & Kasari, C. (2018). Early interaction in autism spectrum disorder: Mothers’ and children’s behaviours during joint engagement. Child: Care, Health And Development, 44(2), 312–318. https://doi.org/10.1111/cch.12532.

    Article  PubMed  Google Scholar 

  • Kasari, C., Gulsrud, A. C., Wong, C., Kwon, S., & Locke, J. (2010). Randomized controlled caregiver mediated joint engagement intervention for toddlers with autism. Journal Of Autism And Developmental Disorders, 40(9), 1045–1056. https://doi.org/10.1007/s10803-010-0955-5.

    Article  PubMed  PubMed Central  Google Scholar 

  • Kellerman, A. M., Schwichtenberg, A. J., Abu-Zhaya, R., Miller, M., Young, G. S., & Ozonoff, S. (2020). Dyadic synchrony and responsiveness in the First Year: Associations with Autism Risk. Autism Research, 13(12), 2190–2201. https://doi.org/10.1002/aur.2373.

    Article  PubMed  PubMed Central  Google Scholar 

  • Kim, K. C., Kim, P., Go, H. S., Choi, C. S., Park, J. H., Kim, H. J., Jeon, S. J., Pena, D., Han, I. C., Cheong, S. H., Ryu, J. H., J. H., & Shin, C. Y. (2013). Male-specific alteration in excitatory post-synaptic development and social interaction in pre-natal valproic acid exposure model of autism spectrum disorder. Journal Of Neurochemistry, 124(6), 832–843. https://doi.org/10.1111/jnc.12147.

    Article  PubMed  Google Scholar 

  • Kojovic, N., Natraj, S., Mohanty, S. P., Maillart, T., & Schaer, M. (2021). Using 2D video-based pose estimation for automated prediction of autism spectrum disorders in young children. Scientific Reports, 11(1), 15069. https://doi.org/10.1038/s41598-021-94378-z.

    Article  PubMed  PubMed Central  Google Scholar 

  • Lang, R., O’Reilly, M., Rispoli, M., Shogren, K., Machalicek, W., Sigafoos, J., & Regester, A. (2009). Review of interventions to increase functional and symbolic play in children with autism. Education and Training in Developmental Disabilities, 44, 481–492.

    Google Scholar 

  • Lee, K., & Schertz, H. H. (2022). Association of turn-taking functions with joint attention in toddlers with autism. Autism, 26(5), 1070–1081. https://doi.org/10.1177/13623613211039945.

    Article  PubMed  Google Scholar 

  • Lord, C., Rutter, M., & Le Couteur, A. (1994). Autism Diagnostic Interview-Revised: A revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. Journal Of Autism And Developmental Disorders, 24(5), 659–685. https://doi.org/10.1007/BF02172145.

    Article  PubMed  Google Scholar 

  • Lord, C., Risi, S., Lambrecht, L., Cook, E. H. Jr., Leventhal, B. L., DiLavore, P. C., Pickles, A., & Rutter, M. (2000). The autism diagnostic observation schedule-generic: A standard measure of social and communication deficits associated with the spectrum of autism. Journal Of Autism And Developmental Disorders, 30(3), 205–223. https://www.ncbi.nlm.nih.gov/pubmed/11055457.

    Article  PubMed  Google Scholar 

  • Lord, C., Elsabbagh, M., Bairdm, G., & Veenstra-Vanderweele, J. (2018). Autism spectrum disorder. Lancet, 392(10146), 508–520. https://doi.org/10.1016/S0140-6736(18)31129-2.

    Article  PubMed  PubMed Central  Google Scholar 

  • Malik-Soni, N., Shaker, A., Luck, H., Mullin, A. E., Wiley, R. E., Lewis, M. E. S., Fuentes, J., & Frazier, T. W. (2022). Tackling healthcare access barriers for individuals with autism from diagnosis to adulthood. Pediatric Research, 91(5), 1028–1035. https://doi.org/10.1038/s41390-021-01465-y.

    Article  PubMed  Google Scholar 

  • Mohammadzaheri, F., Koegel, L. K., Rezaee, M., & Rafiee, S. M. (2014). A randomized clinical trial comparison between pivotal response treatment (PRT) and structured applied behavior analysis (ABA) intervention for children with autism. Journal Of Autism And Developmental Disorders, 44(11), 2769–2777. https://doi.org/10.1007/s10803-014-2137-3.

    Article  PubMed  PubMed Central  Google Scholar 

  • Moy, S. S., Nadler, J. J., Young, N. B., Perez, A., Holloway, L. P., Barbaro, R. P., Barbaro, J. R., Wilson, L. M., Threadgill, D. W., Lauder, J. M., Magnuson, T. R., & Crawley, J. N. (2007). Mouse behavioral tasks relevant to autism: Phenotypes of 10 inbred strains. Behavioural Brain Research, 176(1), 4–20. https://doi.org/10.1016/j.bbr.2006.07.030.

    Article  PubMed  Google Scholar 

  • Mullen, E. (1995). Mullen Scales of early learning. American Guidance Service Inc.

  • Pappas, A. L., Bey, A. L., Wang, X., Rossi, M., Kim, Y. H., Yan, H., Porkka, F., Duffney, L. J., Phillips, S. M., Cao, X., Ding, J. D., Rodriguiz, R. M., Yin, H. H., Weinberg, R. J., Ji, R. R., Wetsel, W. C., & Jiang, Y. H. (2017). Deficiency of Shank2 causes mania-like behavior that responds to mood stabilizers. JCI Insight, 2(20), e92052. https://doi.org/10.1172/jci.insight.92052.

    Article  PubMed  PubMed Central  Google Scholar 

  • Pedersen, J., Livoir-Petersen, M. F., & Schelde, J. T. (1989). An ethological approach to autism: An analysis of visual behaviour and interpersonal contact in a child versus adult interaction. Acta Psychiatrica Scand, 80(4), 346–355. https://doi.org/10.1111/j.1600-0447.1989.tb02991.x.

    Article  Google Scholar 

  • Peleh, T., Ike, K. G. O., Wams, E. J., Lebois, E. P., & Hengerer, B. (2019). The reverse translation of a quantitative neuropsychiatric framework into preclinical studies: Focus on social interaction and behavior. Neuroscience And Biobehavioral Reviews, 97, 96–111. https://doi.org/10.1016/j.neubiorev.2018.07.018.

    Article  PubMed  Google Scholar 

  • Rieth, S. R., Stahmer, A. C., Suhrheinrich, J., Schreibman, L., Kennedy, J., & Ross, B. (2014). Identifying critical elements of treatment:Examining the use of turn taking in autism intervention. Focus on Autism and Other Developmental Disabilities, 29(3), 168–179. https://doi.org/10.1177/1088357613513792.

    Article  Google Scholar 

  • Rodriguiz, R. M., Colvin, J. S., & Wetsel, W. C. (2011). Neurophenotyping genetically modified mice for social behavior. Methods In Molecular Biology, 768, 343–363. https://doi.org/10.1007/978-1-61779-204-5_19.

    Article  PubMed  Google Scholar 

  • Rogers, S. J., Estes, A., Lord, C., Munson, J., Rocha, M., Winter, J., Greenson, J., Colombi, C., Dawson, G., Vismara, L. A., Sugar, C. A., Hellemann, G., Whelan, F., & Talbott, M. (2019). A Multisite Randomized Controlled two-phase trial of the early start Denver Model compared to treatment as Usual. Journal Of The American Academy Of Child And Adolescent Psychiatry, 58(9), 853–865. https://doi.org/10.1016/j.jaac.2019.01.004.

    Article  PubMed  Google Scholar 

  • Sabatos-DeVito, M., Murias, M., Dawson, G., Howell, T., Yuan, A., Marsan, S., Bernier, R. A., Brandt, C. A., Chawarska, K., Dzuira, J. D., Faja, S., Jeste, S. S., Naples, A., Nelson, C. A., Shic, F., Sugar, C. A., Webb, S. J., & McPartland, J. C. (2019). Methodological considerations in the use of Noldus EthoVision XT video tracking of children with autism in multi-site studies. Biological Psychology, 146, 107712. https://doi.org/10.1016/j.biopsycho.2019.05.012. & Autism Biomarkers Consortium for Clinical

    Article  PubMed  PubMed Central  Google Scholar 

  • Sparrow, S., Cicchetti, D., & Saulnier, C. (2016). Vineland Adaptive Behavior Scales, Third. Edition Pearson Assessments.

  • Swiezy, N., Smith, T., Johnson, C. R., Bearss, K., Lecavalier, L., Drill, R., Warner, D., Deng, Y., Xu, Y., Dziura, J., Handen, B., & Scahill, L. (2021). Direct observation in a large-scale randomized trial of parent training in children with autism spectrum disorder and disruptive behavior. Res Autism Spectr Disord, 89, https://doi.org/10.1016/j.rasd.2021.101879.

  • Wang, X., Bey, A. L., Katz, B. M., Badea, A., Kim, N., David, L. K., Duffney, L. J., Kumar, S., Mague, S. D., Hulbert, S. W., Dutta, N., Hayrapetyan, V., Yu, C., Gaidis, E., Zhao, S., Ding, J. D., Xu, Q., Chung, L., Rodriguiz, R. M., Wang, F., Weinberg, R. J., Wetsel, W. C., Dzirasa, K., Yin, H., & Jiang, Y. H. (2016). Altered mGluR5-Homer scaffolds and corticostriatal connectivity in a Shank3 complete knockout model of autism. Nature Communications, 7, 11459. https://doi.org/10.1038/ncomms11459.

    Article  PubMed  PubMed Central  Google Scholar 

  • Weitlauf, A. S., McPheeters, M. L., Peters, B., Sathe, N., Travis, R., Aiello, R., Williamson, E., Veenstra-VanderWeele, J., Krishnaswami, S., Jerome, R., & Warren, Z. (2014). Therapies for Children With Autism Spectrum Disorder: Behavioral Interventions Update. In: Therapies for Children With Autism Spectrum Disorder: Behavioral Interventions Update. Agency for Healthcare Research and Quality (US), Rockville (MD); Available from https://www.ncbi.nlm.nih.gov/books/NBK241444.

  • Yang, Y. C., Lu, L., Jeng, S. F., Tsao, P. N., Cheong, P. L., Li, Y. J., Wang, S. Y., Huang, H. C., & Wu, Y. T. (2019). Multidimensional Developments and Free-Play Movement Tracking in 30- to 36-Month-Old Toddlers with Autism Spectrum Disorder who were full term. Physical Therapy, 99(11), 1535–1550. https://doi.org/10.1093/ptj/pzz114.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank the Marcus Foundation for their financial support of this project, the children and families who participated, and the following members of the Duke Center for Autism and Brain Development: Mallory Harris, Abby Scheer, Gordon Keeler, and all the research support staff, especially Toni Howell, Bailey Heit, and Alexandra Kandah.

Funding

The Marcus Foundation.

Author information

Authors and Affiliations

  1. Duke Center for Autism and Brain Development, Duke University, Durham, NC, USA

    Alexandra L. Bey, Maura Sabatos-DeVito, Kimberly L.H. Carpenter, Lauren Franz, Jill Howard, Saritha Vermeer & Geraldine Dawson

  2. Department of Psychiatry and Behavioral Sciences, Duke University School of Medicine, Durham, NC, USA

    Alexandra L. Bey, Maura Sabatos-DeVito, Kimberly L.H. Carpenter, Lauren Franz, Jill Howard, Saritha Vermeer & Geraldine Dawson

  3. Duke Institute for Global Health, Duke University, Durham, NC, USA

    Lauren Franz

  4. Department of Biostatistics & Bioinformatics, Duke University School of Medicine, Duke University, Durham, NC, USA

    Ryan Simmons & Jesse D. Troy

  5. Marcus Center for Cellular Cures, Duke University School of Medicine, Duke University, Durham, NC, USA

    Geraldine Dawson

Authors
  1. Alexandra L. Bey
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maura Sabatos-DeVito
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kimberly L.H. Carpenter
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lauren Franz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jill Howard
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Saritha Vermeer
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ryan Simmons
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jesse D. Troy
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Geraldine Dawson
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Alexandra Bey: Conceptualization, Methodology, Investigation, Writing- Original Draft Preparation; Maura Sabatos-DeVito: Conceptualization, Methodology, Investigation, Writing- Review & Editing; Kimberly Carpenter: Investigation; Lauren Franz: Investigation; Jill Howard: Investigation; Saritha Vermeer: Investigation; Ryan Simmons: Formal Analysis; Jesse Troy: Methodology, Formal Analysis, Writing- Review & Editing; Geraldine Dawson: Supervision, Funding Acquisition, Writing- Review & Editing. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Geraldine Dawson.

Ethics declarations

Conflict of Interest

Dr. Dawson is on the Scientific Advisory Boards of Akili Interactive, Inc, Zynerba, Nonverbal Learning Disability Project, and Tris Pharma, is a consultant to Apple, Gerson Lehrman Group, and Guidepoint Global, Inc., and receives book royalties from Guilford Press and Springer Nature. Dr. Dawson has stock interests in Neuvana, Inc. Dr. Dawson and Dr. Troy have developed technology, data, and/or products that have been licensed to Apple, Inc. and Cryocell, Inc. and Dawson and Duke University have benefited financially. Dr. Troy also has stock interest in NeurOp, Inc. and receives consulting fees from AegisCN, LLC. Dr. Troy also serves on Data and Safety Monitoring Boards for Synthetic Biologics, Inc. and The National Institutes of Health (NIH) and he receives honoraria from The Emmes Corporation and Navitas Clinical Research. Dr. Sabatos-DeVito consults with and received personal fees from New Frontiers. For the remaining authors none were declared.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bey, A.L., Sabatos-DeVito, M., Carpenter, K.L. et al. Automated Video Tracking of Autistic Children’s Movement During Caregiver-Child Interaction: An Exploratory Study. J Autism Dev Disord (2023). https://doi.org/10.1007/s10803-023-06107-2

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10803-023-06107-2

Keywords

Search

Navigation