Skip to main content
Springer Nature Link
Log in

A Game-Based Repeated Assessment for Cognitive Monitoring: Initial Usability and Adherence Study in a Summer Camp Setting

  • OriginalPaper
  • Published:
Journal of Autism and Developmental Disorders Aims and scope Submit manuscript

Abstract

The current feasibility study examined the adherence, reliability, and assessment potential of an evidence-based game-like mobile Monitoring Tool (Akili Interactive Labs), to monitor 100 participants’ cognition for eight sessions at a summer camp for children with special needs. A validated measure of attention was administered at baseline. In the last session, participants completed an exit questionnaire. The Monitoring Tool was found to be enjoyable, and showed a high rate of adherence. No Monitor-related adverse events were reported. Monitor metrics showed good reliability across repeated measurements, indicating it is stable over long-term cognitive monitoring. There was evidence that the Monitoring Tool was able to detect differences in cognition between the children diagnosed with attention deficit hyperactivity disorder and autism spectrum disorders.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Aggarwal, A., & Lillystone, D. (2000). A follow up pilot study of objective measures in children with attention deficit hyperactivity disorder. Journal of Pediatrics and Child Health, 36(2), 134–138. https://doi.org/10.1046/j.1440-1754.2000.00464.x.

    Article  Google Scholar 

  • American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th edn.). Arlington: American Psychiatric Publishing.

    Book  Google Scholar 

  • Anderson-Hanley, C., Tureck, K., & Schneiderman, R. (2011). Autism and exergaming: Effects on repetitive behaviors and cognition. Psychology Research and Behavior Management, 4, 129–137. https://doi.org/10.2147/PRBM.S24016.

    Article  PubMed  PubMed Central  Google Scholar 

  • Anguera, J. A., Boccanfuso, J., Rintoul, J. L., Al-Hashimi, O., Faraji, F., Janowich, J., Kong, E., Larraburo, Y., Johnston, E., & Gazzaley, A. (2013). Video game training enhances cognitive control in older adults. Nature, 501(7465), 97–101. https://doi.org/10.1038/nature12486.

    Article  PubMed  PubMed Central  Google Scholar 

  • Anguera, J. A., Brandes-Aitken, A. N., Rolle, C. E., Skinner, S. N., Desai, S. S., Bower, J. D., & Marco, E. J. (2016). Characterizing cognitive control abilities in children with 16p11. 2 deletion using adaptive ‘video game’ technology: a pilot study. Translational Psychiatry, 6(9), e893.

    Article  PubMed  PubMed Central  Google Scholar 

  • Anguera, J. A., Jordan, J. T., Castaneda, D., Gazzaley, A., & Areán, P. A. (2016). Conducting a fully mobile and randomized clinical trial for depression: Access, engagement and expense. BMJ Innovations, 2(1), 14–21. https://doi.org/10.1136/bmjinnov-2015-000098.

    Article  PubMed  PubMed Central  Google Scholar 

  • Atlis, M. M., Hahn, J., & Butcher, J. N. (2006). Computer-based assessment with the MMPI-2. In J. N. Butcher (Ed.), MMPI-2: The practitioner’s handbook (pp. 445–476). Washington, DC: American Psychological Association.

    Google Scholar 

  • Bates, D., Maechler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01.

    Article  Google Scholar 

  • Bellotti, F., Kapralos, B., Lee, K., Moreno-Ger, P., & Berta, R. (2013). Assessment in and of serious games: An overview. Advances in Human-Computer Interaction. 2013, https://doi.org/10.1155/2013/136864.

    Article  Google Scholar 

  • Bioulac, S., Lallemand, S., Rizzo, A., Philip, P., Fabrigoule, C., & Bouvard, M. P. (2012). Impact of time on task on ADHD patient’s performances in a virtual classroom. European Journal of Pediatric Neurology, 16(5), 514–521. https://doi.org/10.1016/j.ejpn.2012.01.006.

    Article  Google Scholar 

  • Butcher, J. N. (2006). Assessment in clinical psychology: A perspective on the past, present challenges, and future prospects. Clinical Psychology: Science and Practice, 13(3), 205–209. https://doi.org/10.1111/j.1468-2850.2006.00025.x.

    Article  Google Scholar 

  • Centers for Disease Control and Prevention. (2010). Increasing prevalence of parent-reported attention-deficit/hyperactivity disorder among children - United States, 2003 and 2007. Morbidity and Mortality Weekly Report, 59(44), 1439–1443.

    Google Scholar 

  • Centers for Disease Control and Prevention. (2014). Prevalence of autism spectrum disorder among children aged 8 years - Autism and developmental disabilities monitoring network, United States, 2010. Morbidity and Mortality Weekly Report, 63(2), 1–21.

    Google Scholar 

  • Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment., 6(4), 284–290. https://doi.org/10.1037/1040-3590.6.4.284.

    Article  Google Scholar 

  • Corbett, B. A., Constantine, L. J., Hendren, R., Rocke, D., & Ozonoff, S. (2009). Examining executive functioning in children with autism spectrum disorder, attention deficit hyperactivity disorder and typical development. Psychiatry Research, 166(2), 210–222. https://doi.org/10.1016/j.psychres.2008.02.005.

    Article  PubMed  PubMed Central  Google Scholar 

  • Craig, F., Margari, F., Legrottaglie, A. R., Palumbi, R., De Giambattista, C., & Margari, L. (2016). A review of executive function deficits in autism spectrum disorder and attention-deficit/hyperactivity disorder. Neuropsychiatric Disease and Treatment, 12, 1191.

    PubMed  PubMed Central  Google Scholar 

  • De Vries, M., Prins, P. J. M., Schmand, B. A., & Geurts, H. M. (2015). Working memory and cognitive flexibility-training for children with an autism spectrum disorder: A randomized controlled trial. Journal of Child Psychology and Psychiatry, 56, 566–576. https://doi.org/10.1111/jcpp.12324.

    Article  PubMed  Google Scholar 

  • Deveau, J., Jaeggi, S. M., Zordan, V., Phung, C., & Seitz, A. R. (2015). How to build better memory training games. Frontiers in Systems Neuroscience, 8, 1–6. https://doi.org/10.3389/fnsys.2014.00243.

    Article  Google Scholar 

  • Diamond, A., & Liang, D. S. (2016). Conclusions about interventions, programs, and approaches for improving executive functions that appear justified and those that, despite much hype, do not. Developmental Cognitive Neuroscience, 18, 34–48. https://doi.org/10.1016/j.dcn.2015.11.005.

    Article  PubMed  Google Scholar 

  • Dolan, R., Hall, T. E., Banerjee, M., Chun, E., & Strangman, N. (2005). Applying principles of universal design to test delivery: The effect of computer-based read-aloud on test performance of high school students with learning disabilities. The Journal of Technology, Learning and Assessment, 3(7), 3–32.

    Google Scholar 

  • Dovis, S., Van der Oord, S., Wiers, R. W., & Prins, P. J. M. (2015). Improving executive functioning in children with ADHD: Training multiple executive functions within the context of a computer game. A randomized double-blind placebo controlled trial. PLoS ONE, 10(4), e0121651. https://doi.org/10.1371/journal.pone.0121651.

    Article  PubMed  PubMed Central  Google Scholar 

  • Frazier, T. W., Youngstrom, E. A., Glutting, J. J., & Watkins, M. W. (2007). ADHD and achievement: Meta-analysis of the child, adolescent, and adult literatures and a concomitant study with college students. Journal of Learning Disabilities, 40(1), 49–65. https://doi.org/10.1177/00222194070400010401.

    Article  PubMed  Google Scholar 

  • Gamberini, L., Barresi, G., Maier, A., & Scarpetta, F. (2008). A game a day keeps the doctor away: A short review of computer games in mental healthcare. Journal of CyberTherapy and Rehabilitation, 1(2), 127–145.

    Google Scholar 

  • Hanson, E., Bernier, R., Porche, K., Jackson, F. I., Goin-Kochel, R. P., Snyder, L. G., & Chen, Q. (2015). The cognitive and behavioral phenotype of the 16p11. 2 deletion in a clinically ascertained population. Biological Psychiatry, 77(9), 785–793.

    Article  PubMed  Google Scholar 

  • Happé, F., Booth, R., Charlton, R., & Hughes, C. (2006). Executive function deficits in autism spectrum disorders and attention-deficit/hyperactivity disorder: Examining profiles across domains and ages. Brain and Cognition, 61(1), 25–39.

    Article  PubMed  Google Scholar 

  • Kazmi, S., Ugail, H., Lesk, V., & Palmer, I. (2014). Interactive digital serious games for the assessment, rehabilitation, and prediction of dementia. International Journal of Computer Games Technology, 2014, 12.

  • Klingberg, T., Fernell, E., Olesen, P. J., Johnson, M., Gustafsson, P., Dahlstrçm, K., Gillberg, C. G., Forssberg, H., & Westerberg, H. (2005). Computerized training of working memory in children with ADHD – a randomized, controlled trial. Journal of the American Academy of Child and Adolescent Psychiatry, 44(2), 177–186.

    Article  PubMed  Google Scholar 

  • Lange, B., Koenig, S., McConnell, E., Chang, C. Y., Juang, R., Suma, E., Bolas, M., & Rizzo, A. (2012). Interactive game-based rehabilitation using the Microsoft Kinect. In Virtual Reality Short Papers and Posters (VRW), 2012 IEEE (pp. 171–172). IEEE.

  • Lawson, R. A., Papadakis, A. A., Higginson, C. I., Barnett, J. E., Wills, M. C., Strang, J. F., Wallace, G. L., & Kenworthy, L. (2015). Everyday executive function impairments predict comorbid psychopathology in autism spectrum and attention deficit hyperactivity disorders. Neuropsychology, 29(3), 445. https://doi.org/10.1037/neu0000145.

    Article  PubMed  Google Scholar 

  • Leek, M. R. (2001). Adaptive procedures in psychophysical research. Perception & psychophysics, 63(8), 1279–1292.

    Article  Google Scholar 

  • Lumsden, J., Edwards, E. A., Lawrence, N. S., Coyle, D., & Munafò, M. R. (2016a). Gamification of cognitive assessment and cognitive training: A systematic review of applications and efficacy. JMIR Serious Games. https://doi.org/10.2196/games.5888.

    Article  PubMed  PubMed Central  Google Scholar 

  • Matthews, T., King, M. L., Kupzyk, K., & Lake, C. (2014). Findings and implications of developmental screening for highrisk children referred to a tertiary developmental disability center. Journal of Pediatric Health Care, 28(6), 507–515. https://doi.org/10.1016/j.pedhc.2014.03.002.

    Article  Google Scholar 

  • Miyake, A., Friedman, N. P., Emerson, M. J., Witzki, A. H., Howerter, A., & Wager, T. D. (2000). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41(1), 49–100. https://doi.org/10.1006/cogp.1999.0734.

    Article  PubMed  Google Scholar 

  • Piotrowski, C. (1999). Assessment practices in the era of managed care: Current status and future directions. Journal of Clinical Psychology, 55(7), 787–796. https://doi.org/10.1002/(SICI)1097-4679(199907)55:7%3C787::AID-JCLP2%3E3.0.CO;2-U.

    Article  PubMed  Google Scholar 

  • Rapport, M. D., Orban, S. A., Kofler, M. J., & Friedman, L. M. (2013). Do programs designed to train working memory, other executive functions, and attention benefit children with ADHD? A meta-analytic review of cognitive, academic, and behavioral outcomes. Clinical Psychology Review, 33, 1237–1252.

    Article  PubMed  Google Scholar 

  • Rideout, V. (2016). Measuring time spent with media: The common sense census of media use by US 8- to 18-year-olds. Journal of Children and Media, 10(1), 138–144. https://doi.org/10.1080/17482798.2016.1129808.

    Article  Google Scholar 

  • Robinson, K. E., Kaizar, E., Catroppa, C., Godfrey, C., & Yeates, K. O. (2014). Systematic review and meta-analysis of cognitive interventions for children with central nervous system disorders and neurodevelopmental disorders. Journal of Pediatric Psychology, 39(8), 846–865.

    Article  PubMed  Google Scholar 

  • Sansosti, F. J., & Powell-Smith, K. A. (2008). Using computer-presented social stories and video models to increase the social communication skills of children with high-functioning autism spectrum disorders. Journal of Positive Behavior Interventions, 10(3), 162–178. https://doi.org/10.1177/1098300708316259.

    Article  Google Scholar 

  • Tong, T., Chignell, M., Tierney, M. C., & Lee, J. (2016). A serious game for clinical assessment of cognitive status: Validation study. JMIR Serious Games, 4(1), e7. https://doi.org/10.2196/games.5006.

    Article  PubMed  PubMed Central  Google Scholar 

  • Tseng, H. M., Tiplady, B., Macleod, H. A., & Wright, P. (1998). Computer anxiety: A comparison of pen-based personal digital assistants, conventional computer and paper assessment of mood and performance. British Journal of Psychology, 89(4), 599–610. https://doi.org/10.1177/1098300708316259.

    Article  PubMed  Google Scholar 

  • Watters, C., Oore, S., Shepherd, M., Abouzied, A., Cox, A., Kellar, M., & Otley, A. (2006). Extending the use of games in health care. In System Sciences, 2006. HICSS’06. Proceedings of the 39th Annual Hawaii International Conference on (Vol. 5, pp. 88b). IEEE.

  • Yerys, B. E., Wallace, G. L., Sokoloff, J. L., Shook, D. A., James, J. D., & Kenworthy, L. (2009). Attention deficit/hyperactivity disorder symptoms moderate cognition and behavior in children with autism spectrum disorders. Autism Research, 2(6), 322–333. https://doi.org/10.1002/aur.103.

    Article  PubMed  PubMed Central  Google Scholar 

  • Zelazo, P. D., Qu, L., Müller, U., Schneider, W., Schumann-Hengsteler, R., & Sodian, B. (2005). Hot and cool aspects of executive function: Relations in early development. In Young children’s cognitive development: Interrelationships among executive functioning, working memory, verbal ability, and theory of mind (pp. 71–93). New York: Psychology Press.

    Google Scholar 

Download references

Acknowledgments

We would like to recognize the children, staff and leadership at Ramapo for Children for their participation. We would especially like to recognize the research assistants who conducted the study Laurel Kiyabu Morgan, Juliana Demicco, Gabi Verdoni, Adriana Gonzalez, Beatrice Baker, Angela Law, Dottie Bradshaw, Reeti Pal, Brianna Morales, and Gabbi Gaddy. In addition, we would like to thank Dr. Mary McKay for her mentorship on this project. This work was presented as a poster in October 2017 at the annual meeting for the American Academy of Child & Adolescent Psychiatry, Washington, D.C.

Author information

Author notes

  1. Rachel M. Flynn

    Present address: Feinberg School of Medicine, Northwestern University, 633 N St. Clair St, 19th Floor, Chicago, IL, 60611, USA

Authors and Affiliations

  1. Silver School of Social Work, McSilver Institute for Poverty Policy and Research, New York University, 41 East 11th Street, 7th Floor, New York, NY, 10003, USA

    Rachel M. Flynn

  2. Ramapo for Children, 49 West 38th Street, 5th Floor, New York, NY, 10018, USA

    Nirmaliz Colón-Acosta

  3. Akili Interactive, 125 Broad Street, 4th Floor, Boston, MA, 02110, USA

    Jimmy Zhou & Jeffrey Bower

Authors
  1. Rachel M. Flynn
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Nirmaliz Colón-Acosta
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Jimmy Zhou
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Jeffrey Bower
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

RF conceived of the study, planned its design, coordinated data collection, participated in data interpretation, and drafted the manuscript; NC participated in the study design, coordinated data collection and helped draft the manuscript; JB planned the study design, performed the statistical analysis, and participated in data interpretation; JZ performed the statistical analysis. All authors read, edited and approved the final manuscript.

Corresponding author

Correspondence to Rachel M. Flynn.

Ethics declarations

Conflict of interest

Rachel Flynn and Nirmaliz Colón-Acosta both were employees of Ramapo for Children, the data collection site, at various points of time. Jeffrey Bower is an employee of Akili Interactive whose product was used. In addition, Jimmy Zhou was an employee of Akili Interactive.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Flynn, R.M., Colón-Acosta, N., Zhou, J. et al. A Game-Based Repeated Assessment for Cognitive Monitoring: Initial Usability and Adherence Study in a Summer Camp Setting. J Autism Dev Disord 49, 2003–2014 (2019). https://doi.org/10.1007/s10803-019-03881-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10803-019-03881-w

Keywords