Fast track articleDetecting stereotypical motor movements in the classroom using accelerometry and pattern recognition algorithms
Introduction
Health researchers in many disciplines lack effective tools for unobtrusively acquiring information about peoples’ behavior in natural settings. Ubiquitous computing systems that detect certain behaviors might create new opportunities to improve scientific understanding of the interaction between context, behavior, and health. The goal of the current work is to use ubiquitous monitoring tools for the automated detection of stereotypical motor movements observed in persons with Autism Spectrum Disorders. Autism Spectrum Disorders (ASD) affect as many as 1 in 110 children [1] and are characterized by deficits in socialization and communication, including stereotypical behavior [2]. Stereotyped behaviors are generally defined as repetitive interests and/or motor or vocal sequences that appear to the observer to be invariant in form and without any obvious eliciting stimulus or adaptive function [3]. The current work focuses on stereotypical motor movements. Several stereotypical motor movements have been identified [4], the most prevalent among them being body-rocking, mouthing, and complex hand and finger movements [5]. The majority of research in ASD focuses on social and communication deficits, rather than on restricted and repetitive behavior [4]. A lack of research in stereotypical movements is a potential problem given the high prevalence of stereotypical motor movements reported in individuals with ASD (e.g., [6]).
One reason why stereotypical motor movements may not be as thoroughly studied is because appropriate tools for measuring the behavior are not available to the research community. In this work, we present a case study on the automatic identification of stereotypical body rocking and hand flapping activity in children with ASD gathered from wireless accelerometers. Stereotypical body rocking and hand flapping are examples of movements that occur frequently in people with mental retardation and developmental disabilities [4], and less frequently in typically developing children and adults.
When severe, stereotypical motor movements can present several problems for individuals with ASD and their caregivers. First, some persons with ASD often engage in stereotypical motor movements for the majority of their waking hours. Second, if unregulated, stereotypical motor movements can become the dominant behavior in an individual with ASD’s repertoire and interfere with the acquisition of new skills and performance of established skills (e.g., [7]). Third, engagement in these movements is socially inappropriate and stigmatizing and can complicate social integration in school settings and the community [8]. Finally, stereotypical motor movements can lead to self-injurious behavior under certain environmental conditions [9].
There are currently no tools for clinicians or caregivers to easily, accurately, and reliably monitor stereotypical motor movements. Traditional measures of stereotypical motor movements rely primarily on paper-and-pencil rating scales, direct observation, and video-based methods [10], all of which have limitations.
Paper-and-pencil rating scales typically involve a global impression of the frequency and/or severity of stereotypical motor movements based on general, non-specific observations. Several paper-and-pencil rating scales have been developed that ask an informant to give a global impression of an individual’s stereotypical motor movements (e.g., [4]). From a measurement standpoint, informant rating scales are subjective, can have questionable accuracy, and fail to capture inter-individual variations in the form, amount, and duration of stereotypical motor movements [11].
Direct observation also involves a rating but the focus is on the direct observation of specific behaviors. The observer watches and records a sequence of stereotypical motor movements. According to Sprague and Newell [10], the following factors, among others, can make direct observational measures unreliable: (a) Reduced accuracy in observing and documenting high-speed motor sequences; (b) Difficulty determining when a sequence has started and ended; (c) Limitations in the ability to observe concomitantly occurring stereotypical motor movements; and (d) Limitations in the ability to note environmental antecedents and record stereotypical motor movements at the same time.
Video-based methods involve video capture of behavior and off-line coding of stereotypical motor movements by an expert. The ability to view videos repeatedly and to slow playback speeds makes video-based methods more reliable than paper-and-pencil and direct observation methods. Video-based methods, however, are tedious and time consuming. The necessity to code videos off-line also precludes real-time monitoring. However, combining video recording with other tagging technologies to permit practical, semi-automatic logging is an area of active research [12].
The primary aim of the current work is to explore whether wireless accelerometer sensor technology and pattern recognition algorithms can provide an automatic measure of stereotypical motor movements that may be more objective, detailed, and precise than rating scales and direct observation, and more time-efficient than video-based methods. An algorithm that achieves good recognition performance could also operate for much longer periods of time than a human observer. Another goal of our work is to move towards a real-time annotation and recognition system that could be used by teachers and caregivers to annotate and train algorithms while giving real-time feedback on a convenient mobile device such as a phone. To assess the viability of having teachers use our system, we report initial pilot experiments to analyze the quality of non-expert annotations and to assess their impact on the automated recognition of stereotypical motor movements.
In the remainder of this paper we describe analyses we have performed to determine whether pattern recognition techniques using mobile wireless accelerometers that have shown promise in other domains of recognition of posture, mobility, exercise, and everyday activities can be adapted to create a tool for stereotypical motor movement monitoring in children with ASD.
Section snippets
Using pattern recognition to detect physical activities
A growing body of work shows that wearable accelerometers can be used to detect activities, such as postures, ambulation, exercise, and even household activities (e.g., [13], [14], [15]). A variety of methods and models have been used for feature generation and classification. Our focus in this work is not on any particular activity recognition algorithm, per se, but instead on the issues one encounters when trying to apply pattern recognition to the problem of monitoring stereotypical motor
Data collection
The current investigation consisted of a series of six single case studies, with direct replication across participants. For each participant, the study included repeated observations of body rocking, hand flapping, and/or simultaneous body rocking and hand flapping while children wore sensors in a classroom setting (Study 1). To analyze the quality of non-expert annotations, we ran four additional data sessions where an expert and a non-expert annotator encoded the movements of one of the
Recognition evaluation and experiences
In this section we describe in detail our experience applying physical activity pattern recognition to the stereotypical motor movement recognition domain.
Discussion
The problem of accurately recognizing stereotypical motor movements in children with ASD and creating a real-time monitoring tool is more challenging than it may appear at first due to the complexity of the domain. First, there was considerable variability in the topography, duration, frequency, and consistency with which participants performed stereotypical motor movements. Each child had very specific stereotypical motor movements that required participant-dependent data to train the
Acknowledgments
The authors thank Autism Speaks and the Nancy Lurie Marks Family Foundation for funding this work. We also thank the children and parents who graciously agreed to participate in this research. The sensors used in Study 1 of this work were made possible by funding from the NSF (grant #0313065). The sensors used in Study 2 of this work were made possible by funding from the NIH/NHLBI (grant U01 HL091737). The contents of this work are solely the responsibility of the authors and do not represent
Fahd Albinali, Ph.D., is the Chief Technology Officer of EveryFit, Inc., a start-up exercise fitness and measurement company he co-founded in 2010. He was previously a Research Scientist at the House_n Consortium in the MIT Department of Architecture. His research focuses on building and studying interactive technologies that measure behavior. Currently, he is working on building wearable systems that measure physical activity to address high value societal challenges such as preventive health
References (27)
- et al.
Response latencies to auditory stimuli in autistic children engaged in self-stimulatory behavior
Behav. Res. Ther.
(1971) - et al.
The stereotypy analysis: an instrument for examining environmental variables associated with differential rates of stereotypic behavior
Res. Dev. Disabil.
(1997) - US CDC, Prevalence of autism spectrum disorders—autism and developmental disabilities monitoring network, United...
- (2000)
- A. Baumeister, R. Forehand, Stereotyped acts, in: Int’l. Rev. of Res. in Mental Retardation: VI., ed., New York, 1973,...
- et al.
Repetitive behavior disorders in autism
Ment. Retard. Dev. Disabil. Res. Rev.
(1998) - et al.
Stereotypic responding: a review of intervention research
Am. J. Ment. Def.
(1984) - et al.
Stereotyped movements of mental defectives. I. Initial survey
Am. J. Ment. Def.
(1962) - et al.
The social effects of stereotyped behaviour
J. Ment. Defic. Res.
(1990) Evolution of stereotypy into self-injury
Stereotyped Movements: Brain and Behavior Relationships
Pervasive computing and autism: assisting caregivers of children with special needs
IEEE Pervasive Comput.
Cited by (0)
Fahd Albinali, Ph.D., is the Chief Technology Officer of EveryFit, Inc., a start-up exercise fitness and measurement company he co-founded in 2010. He was previously a Research Scientist at the House_n Consortium in the MIT Department of Architecture. His research focuses on building and studying interactive technologies that measure behavior. Currently, he is working on building wearable systems that measure physical activity to address high value societal challenges such as preventive health care and support for aging and disabled populations. His work has been published in academic venues including UbiComp, AAAI, CHI and PerCom and has received one best paper award. Dr. Albinali received his Ph.D. from the University of Arizona in 2008 working on activity recognition in domestic environments, an M.Sc. from the University of Arizona in 2002, and a B.Sc. degree in Computer Science from the American University in Cairo in 1999.
Matthew S. Goodwin, Ph.D., received his B.A. in Psychology from Wheaton College in 1998 and his M.A. in 2008 and Ph.D. in 2010, both in Experimental Psychology from the University of Rhode Island. He is currently the Director of Clinical Research at the Massachusetts Institute of Technology, Media Laboratory and Associate Director of Research at the Groden Center—an Institute for Autism Spectrum Disorders in Providence, RI. He is Co-Chair of the Autism Speaks—Innovative Technology for Autism Initiative, has an Adjunct Associate Research Scientist appointment in the Department of Psychiatry and Human Behavior at Brown University, and is an Adjunct Assistant Professor in the Department of Psychology at the University of Rhode Island. He has over 15 years of research and clinical experience working with the full spectrum of children and adults with ASD and has extensive experience developing and evaluating innovative technologies for behavioral assessment, including telemetric physiological monitors, accelerometry sensors, and digital video/facial recognition systems.
Stephen Intille, Ph.D., is an Associate Professor in the College of Computer and Information Science & Dept. of Health Sciences, Bouvé College of Health Sciences at Northeastern University. His research is focused on the development of novel healthcare technologies that incorporate ideas from ubiquitous computing, user-interface design, pattern recognition, behavioral science, and preventive medicine. One area of special interest to him is the development and pilot testing of systems that support healthy aging and well-being in the home setting. Another area of interest is the creation of tools for mobile phones that permit longitudinal measurement of health behaviors for research, especially the type, duration, intensity, and location of physical activity, and sensor-enabled, mobile tools that motivate and assist people in making healthy behavior changes. Dr. Intille received his Ph.D. from MIT in 1999 working on computational vision at the MIT Media Laboratory, an S.M. from MIT in 1994, and a B.S.E. degree in Computer Science and Engineering from the University of Pennsylvania in 1992. He has published research on computational stereo depth recovery, real-time and multi-agent tracking, activity recognition, perceptually-based interactive environments, and technology for healthcare. Dr. Intille has been principal investigator on sensor-enabled health technology grants from the NSF, the NIH, foundations, and industry. In September, 2010 he joined Northeastern University to help establish a new transdiciplinary Ph.D. program in health informatics/technologies.