Altered auditory and multisensory temporal processing in autism spectrum disorders

Introduction

Autism spectrum disorders (ASD) are characterized by deficits in social reciprocity, communication, and behavioral flexibility that emerge in the first few years of life (American Psychiatric Association, 2000). Sensory disturbances were reported in Kanner’s original description of autism (Kanner, 1943), and have been reported consistently in the clinical literature (Cesaroni and Garber, 1991; O’Neill and Jones, 1997; Sigman and Capps, 1997; Talay-Ongan and Wood, 2000). Though not currently part of the diagnostic criteria for ASD, the presence of unusual sensory behaviors has been proposed for inclusion in updated diagnostic criteria for the DSM-V, highlighting emerging consensus that sensory abnormalities are central features of ASD. Reports of abnormal sensory function that span the visual, auditory, gustatory, and tactile domains reinforce the “multisensory” nature of sensory processing alterations in ASD (for review, see Rogers and Ozonoff, 2005), and emerging evidence suggests that abnormalities also extend to the selective integration of information across the different sensory modalities (i.e., multisensory integration – see Iarocci and McDonald, 2006).

Although evidence for deficits in sensory (and multisensory) processing is abundant in the ASD literature, there are also a number of reports detailing enhanced perceptual capabilities in response to specific sensory stimuli. For example, discrimination of discrete details within complex visual spatial displays widely has been found to be a relative strength in ASD (Shah and Frith, 1983; Jolliffe and Baron-Cohen, 1997; Plaisted et al., 1998; O’Riordan et al., 2001) and at least one study has reported enhanced visual acuity in autism (Ashwin et al., 2009). Similarly, in the auditory domain, Bonnel et al. (2003) have shown that individuals with autism have superior pitch discrimination and categorization abilities in comparison to controls. In an effort to reconcile these findings showing both impaired and enhanced sensory function in ASD, it has been suggested that perceptual abilities may depend on the nature and complexity of the sensory stimuli, with impairments characterizing responses to more complex stimuli and enhancements seen more often with simple stimuli (Samson et al., 2006; Mongillo et al., 2008). For instance, in the same study, adolescents and adults with autism were found to show superior orientation discrimination of luminance-defined gratings but inferior discrimination of texture-defined gratings that are believed to be processed further along the visual pathway (Bertone et al., 2005). Given that multisensory function depends on stimulus integration, it may be an inherently complex process even when the component stimuli are exceedingly low level. Such a view is supported by the tendency for multisensory processing to occur both within and beyond primary sensory cortices (Ghazanfar and Schroeder, 2006). Hence, a better understanding of multisensory processing in ASD, as well as the processing of component unisensory stimuli used to test multisensory function, may provide important clues into the neural bases of sensory differences in ASD.

In addition to these broad sensory findings, there has been some indication that the temporal aspects of sensory information processing may be impacted in ASD. Szelag et al. (2004) found that children with autism had difficulty reproducing the lengths of both auditory and visual unisensory stimuli of standardized durations. Other studies have found atypical neural responses to changes in the pitch of repeated, sequential auditory stimuli in children and adults with ASD (Tecchio et al., 2003; Gomot et al., 2006). Extending to multisensory function, children with ASD showed impairments in the detection of violations of temporal synchrony of audiovisual linguistic stimuli in comparison to TD children and to those with non-autistic developmental delays (Bebko et al., 2006). Together, these studies suggest alterations in the processing of basic timing information contained within both simple and complex (i.e., linguistic) sensory stimuli, both within and across sensory systems. However, although these studies establish the presence of temporal processing abnormalities in ASD, the extent of these deficits is unknown. Further characterization of these differences using low-level stimuli devoid of social or linguistic context is necessary to clarify the nature and scope of alterations in temporal processing specific to basic sensory functioning.

Our results from a previous study revealed an expanded temporal window for multisensory integration in children with ASD (Foss-Feig et al., 2010). This result not only replicated a previous finding that individuals with ASD are capable of integrating basic auditory and visual information (Van der Smagt et al., 2007), but also extended this work to show that significant changes in basic multisensory function appear to lie in the temporal realm. However, given the nature of the task employed in our previous study (a common multisensory illusion in which there is no direct measure of unisensory temporal acuity), it was not possible to determine the potential contribution of changes in unisensory temporal function to the demonstrated change in multisensory performance.

The goal of the current study was to expand upon this previous finding of an extended temporal binding window for simple audiovisual input in ASD. To this end, we examined both unisensory and multisensory temporal processing abilities in a single sample of children and adolescents with ASD. First, temporal acuity in the auditory and visual systems was examined using temporal order judgment (TOJ) tasks to establish baseline auditory and visual temporal resolution abilities. Then, task-irrelevant auditory signals were added to the visual TOJ task in order to assess multisensory binding processes and their temporal constraints. Previous work has shown that the addition of task-irrelevant auditory stimuli can improve performance on the visual TOJ task, but only if presented within a particular window of time that reflects the duration of the multisensory temporal binding process (Morein-Zamir et al., 2003; Hairston et al., 2005, 2006). Due to the inherently multisensory nature of language and social stimuli, an enlargement in the temporal window of multisensory binding is likely to have far-reaching consequences for children with ASD. Clarification of the nature and extent of temporal processing differences in ASD represents an important step in understanding the level at which sensory processing is altered in this disorder, which could in turn be important for developing targeted interventions.

Materials and Methods

Participants

Thirty-five children with ASD and 27 with typical development (TD) comprise the study sample, which overlaps in part with the sample reported in Foss-Feig et al. (2010). Eligibility criteria for children in both groups were as follows: (a) age 8–17 years; (b) normal or corrected-to-normal hearing and vision; (c) Full Scale IQ (FSIQ) score above 70; and (d) no evidence or past diagnosis of a specific reading disorder. Adequate cognitive functioning for inclusion in the study (i.e., FSIQ score above 70) was confirmed using the Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, 1999) unless a child had completed cognitive testing in the past year and the parents could provide the scores. Reading abilities were screened using the Letter–Word Identification and Word Attack subtests of the Woodcock–Johnson tests of achievement – third edition (WJA-III: Woodcock et al., 2001), since differences in multisensory processing have been demonstrated in individuals with reading disorders (Hairston et al., 2005). All children in both groups were required to have reading standard scores above 70 on both WJA-III subtests. Additional eligibility criteria for the ASD group required that children: (a) have a confirmed diagnosis of autistic disorder, Asperger’s disorder or Pervasive Developmental Disorder-Not Otherwise Specified; and (b) have no history of seizure disorders or identified genetic disorders (e.g., Fragile X, tuberous sclerosis). Children with ASD were not excluded based on use of psychotropic medication.

Children’s prior ASD diagnoses were confirmed in the present study using gold-standard procedures: the Autism Diagnostic Observation Schedule (ADOS; Lord et al., 2000) was administered by a research-reliable examiner, parent(s) completed the Autism Diagnostic Interview – Revised (ADI-R; Lord et al., 1994) with a research-reliable interviewer, and DSM-IV-based clinical diagnoses were made by a licensed clinical psychologist on the basis of this information. All children included in the ASD group met criteria for autism or autism spectrum on both the ADOS and ADI-R at a session scheduled prior to psychophysical testing. Additional eligibility criteria for children with TD were as follows: (a) no history of or current psychiatric, neurological, or learning disorders (e.g., ADHD, depression, epilepsy, dyslexia) or symptoms of ASD; and (b) no first-degree relatives with ASD. Parent report of ASD symptoms was obtained using the Lifetime version of the Social Communication Questionnaire (SCQ; Rutter et al., 2003); all children with TD had SCQ scores below the at-risk cutoff for ASD. No differences in age, gender, or FSIQ score were found between groups (Table 1). As expected, a significant group difference was found for parent report of ASD symptoms on the SCQ, t(56) = 11.75, p < 0.001.

Parents of all participants gave informed consent and all children in both groups gave assent prior to participation in any component of this study. All children received compensation for their participation at each visit. All procedures were approved by the Vanderbilt University Institutional Review Board.

Table 1. Participant demographics.

General Procedure

Participants sat in a light- and sound-attenuated room and wore headphones through which auditory stimuli were presented. Visual stimuli were presented as open white circles against a black background on a high-refresh rate PC monitor (NEC Multisync FE992, 22′′ screen; 150 Hz refresh rate; 640 × 480 pixel resolution). Auditory stimuli were presented via noise-canceling supra-aural headphones (Philips SBC HN110) to both ears (90 dB peak SPL). Stimulus presentation was controlled using E-Prime (Psychology Software Tools Inc., Pittsburgh, PA, USA). Responses (i.e., accuracy and response time) were recorded via a Serial Response box (Psychology Software Tools Inc., Pittsburgh, PA, USA).

Participants were monitored continuously via closed-circuit video cameras to ensure that they were engaged in the tasks. On the rare occasions that a participant was not on-task, a variety of strategies were implemented to increase engagement (e.g., reminders to stay on-task, additional breaks, parent in the testing room, etc.). Participants were allowed to take breaks as necessary to increase compliance and maintain effort, motivation, and on-task behavior. All participants completed each of the tasks described below within a single session; while some participants had difficulty completing one or more tasks or produced data that could not always be interpreted reliably (see below for further details, broken down by task), data from the maximum number of participants possible were included for analyses of individual tasks.

Tasks

Visual TOJ task

The visual TOJ task was used to test temporal acuity of the visual system and was the first task completed. In this task, participants were asked to determine which of two circles (above and below a fixation cross) presented in close temporal proximity [i.e., stimulus onset asynchronies (SOAs) ranging between 7 and 252 ms] appeared on the computer screen first. Following instructions, a white fixation cross appeared on a black screen. After a delay of 1000 ms, the first of two circles appeared, either 7 cm above or below the fixation cross, and remained on the screen. Following a variable SOA, a second circle appeared at the location opposite the first circle (e.g., above the fixation if the first circle appeared below; Figure 1). Participants indicated via button presses on the response box their judgment as to which of the two circles appeared first (i.e., “top first” or “bottom first”). Following a response, both circles disappeared simultaneously and a new trial began. Participants completed 10 practice trials (visual SOA 91–119 ms in 7 ms increments, presented randomly), which included feedback regarding response accuracy, before completing the full task. The practice session was repeated until participants could correctly identify on a majority of trials which circle occurred first.

Figure 1. Task design. (A) Visual TOJ task. (B) Auditory TOJ task. (C) Multisensory TOJ task. In multisensory conditions, two circles are presented sequentially above and below the central fixation point. One beep is always presented simultaneously with the first circle, whereas the second beep is presented with a varied delay (0–500 ms) from the onset of the second circle. See text for additional detail.

After practicing the task, a staircase procedure was used to determine the threshold SOA necessary for each participant to perform the visual TOJ task with between 70 and 75% accuracy. An adaptive staircase procedure, in which three independent staircases were run concurrently, was used. One staircase started at an SOA of 84 ms, the second started at an SOA of 7 ms, and the third started at an SOA of 56 ms. The initial step size (i.e., amount by which the SOA was adjusted) was 28 ms, which was decreased to 14 ms after five reversals in response accuracy and decreased again to 7 ms after an additional four reversals. The SOA increased one step (i.e., became longer) after each incorrect response, and decreased one step (i.e., became shorter) after two consecutive correct responses. Each staircase terminated after 16 reversals in response accuracy and an average was calculated from the last five reversals to produce the threshold SOA. The mean threshold value was calculated from the three staircase outputs and then rounded to the nearest value compatible with the vertical scan rate of the monitor (i.e., multiple of 7 ms). Following the staircase procedure, participants performed a shorter confirmation procedure with SOA values set relative to their individual threshold. In this process three SOAs were used relative to the calculated threshold: 0 ms (i.e., threshold), 7 ms above, 7 ms below. Each of these SOAs was repeated 20 times in random order; at each SOA, the first visual stimulus appeared above fixation on half of the trials. If results of the confirmation procedure did not indicate that 70–75% accuracy rates had been attained for any of the three SOAs (i.e., performance was not near threshold), the confirmation procedure was repeated with higher or lower SOA values, depending on whether accuracy rates were too low or too high in the initial confirmation procedure.

Multisensory TOJ task

This task always followed the visual TOJ task. Here, task-irrelevant auditory stimuli were added to the visual TOJ task. Previous work has shown that such auditory stimuli are capable of improving performance on the visual TOJ task (i.e., enabling individuals to discriminate between the two visual stimuli when they are presented at shorter intervals), but only if the auditory cues are presented within a defined temporal structure relative to the visual stimuli (Morein-Zamir et al., 2003; Hairston et al., 2005, 2006). Although the mechanisms responsible for these multisensory-mediated performance enhancements remain unknown, some have theorized that they are due to a temporal shift in the perception of the visual stimulus toward the auditory stimulus (i.e., temporal ventriloquism; Spence and Squire, 2003). In contrast, others have theorized that the auditory stimulus speeds the processing of the visual stimulus, thus allowing the participant to discriminate smaller time intervals between the visual stimuli (Hairston et al., 2006; Keetals and Vroomen, 2010).

For this task, visual stimuli were presented as described above for the visual TOJ task except that, on each trial, the SOA between the two visual stimuli (visual SOA) was fixed according to each individual’s threshold value. Two identical sounds were also presented on 89% of trials through supra-aural headphones, with the first sound always occurring synchronously with the first visual stimulus onset. The second sound was delayed by 0–500 ms relative to the onset of the second visual stimulus (multisensory delay increments were as follows: 0, 50, 100, 150, 200, 300, 400, 500 ms; Figure 1). A randomly interleaved no-sound (i.e., visual only) condition provided baseline performance and represented the remaining 11% of trials. This baseline was also used to ensure reliability of the behavioral threshold by identifying participants whose baseline accuracies were significantly different from the established threshold and excluding their data from further analysis.

Trials from each condition were presented 16 times in random order; at each multisensory delay as well as in the visual-only condition, the first visual stimulus appeared above fixation on half of the trials. Participants were told from the outset that while they often would be hearing sounds through the headphones, the task was the same as in the visual TOJ (i.e., determine whether the top or bottom circle appears first) and that they should ignore the sounds. Given that sounds were presented binaurally through headphones with no interaural timing or amplitude level differences, they did not provide any task-relevant spatial information that would have provided clues as to whether the “top” or “bottom” circle occurred first. However, though not relevant for making the spatial discriminations required in this task, the auditory cues did provide temporal information.

Auditory TOJ task

The auditory TOJ task was designed to test auditory temporal acuity and was completed last. In this task, participants heard two identical clicks, one presented to each ear, in close temporal proximity, and were asked to make a judgment as to which ear the first click was presented. Following instructions, a white fixation cross appeared on a black screen for 1000 ms. Immediately following the 1000 ms fixation, the first of two auditory stimuli was presented through headphones to either the right or left ear. Following a variable SOA, a second identical auditory stimulus was presented through the headphones to the opposite ear. The fixation cross then turned red, signaling participants to respond (Figure 1). Participants indicated in which ear they had heard the first auditory stimulus by pressing a button on the response box (i.e., “left first” or “right first”). Following a response, a new trial began. As in the visual TOJ task, prior to completing the full task, participants completed a 10-trial practice including feedback regarding response accuracy.

After the practice session, participants completed an adaptive staircase procedure similar to that used in the visual TOJ task with three independent, interleaved staircases (each starting at an SOA of 100 ms) to determine the SOA necessary to discriminate the order of clicks at approximately 75% accuracy. The SOA between auditory stimuli changed from a step size of 10 ms for the first five reversals to a step size of 5 ms for the next four reversals, then to 1 ms steps until 16 reversals were reached. An average threshold SOA was calculated from the final 10 reversals of each staircase, then entered into a confirmation procedure as in the visual TOJ task. In the confirmation procedure, three SOAs were used relative to this threshold: 0 ms (i.e., threshold), 10 ms above, 10 ms below. If results of the confirmation procedure did not indicate that 70–75% accuracy rates had been attained for any of the three SOAs, the confirmation procedure was repeated with higher or lower SOA values.

Data Analysis

Response accuracy and timing data were recorded for each trial within each task. For unisensory tasks, participant data were included if: (a) the participant comprehended instructions and was on-task; and, (b) a threshold value at which the participant performed the task at 75% accuracy could reliably be determined. For unisensory TOJ tasks, threshold values for each participant were obtained from the staircase confirmation procedures described above.

For the multisensory task, participant data were included if: (a) the participant comprehended instructions and was on-task; (b) the participant performed the visual-only control trials at between 60 and 89% accuracy; and (c) the participant showed at least some multisensory gains with the addition of task-irrelevant auditory stimuli. For the multisensory TOJ task, accuracy gains at each multisensory delay were defined by subtracting the accuracy rate for the visual-only baseline trials from the accuracy rate for each of the multisensory delay conditions. Improvements in response time were determined for each multisensory delay by subtracting the average response time for each of the delay conditions from the average response time for the visual-only trials.

Group differences

Data from each experiment were first analyzed using independent-samples t-tests and ANOVAs to examine any between-group differences on the dependent variables of interest. Specifically, for the auditory and visual TOJ tasks, t-tests were used to explore potential group differences in threshold SOA values. For the multisensory task, in order to explore potential groups differences in improvements in multisensory temporal processing, separate repeated measures ANOVAs for accuracy and response time gains were conducted with SOA as the within-subjects variable and group as the between-subjects variable in each. Independent-sample t-tests were also conducted with the accuracy and response time gain values at each delay to determine whether the magnitude of multisensory integration-related performance gains differed between groups at any of the delay conditions.

Differences in the temporal window of multisensory integration

The temporal binding window for integration was defined as the span of consecutive multisensory delay conditions within which there were significant gains in accuracy or (analyzed separately) significant improvements in response time at all included delay conditions. To determine the delay conditions at which significant accuracy and response time improvements were observed within each group, one-sample t-tests were conducted for each multisensory delay condition, comparing percent accuracy increase or response time decrease (improvement) to an alternative value of 0, representing no gain in accuracy or response time relative to the visual-only baseline condition. These analyses were run separately for the ASD and TD groups in order to examine group-specific temporal binding windows. p-values were not corrected for multiple comparisons because the t-tests were planned a priori. Family wise error was limited by requiring continuous significance across the entire temporal window of integration.

Results

Performance on the Unisensory TOJ Tasks

To determine whether visual and auditory temporal processing differ in children with ASD as compared to children with TD, we determined the threshold SOA values at which participants could report which of two stimuli occurred first at approximately 75% accuracy. On the visual TOJ task, data from one child with ASD were excluded because it was impossible to determine a threshold value from the confirmation procedure; remaining groups (ASD: n = 34; TD: n = 27) did not differ on gender, age, or IQ score (ps > 0.19). Performance on the visual TOJ task did not differ significantly between groups. On average, children with ASD required 52.7 ms between visual stimuli to determine which circle onset first, whereas children with TD required 60.7 ms, a difference that did not reach statistical significance, t(58) = −1.01, p = 0.32 (Figure 2A). For the auditory TOJ task, data from 15 children with ASD were excluded from analyses due to non-compliance, inattention, or inability to comprehend the task (n = 4), or inability to verify the auditory threshold from the confirmation procedure (i.e., accuracy lower than 75%; n = 11). Children with ASD who were excluded from analyses for this task did not differ from those who were included on age, gender, or FSIQ score (ps > 0.12). Five children with TD completed an alternate version of the auditory TOJ task, thus their data are not included here, though they are included in visual and multisensory TOJ analyses. Remaining groups (ASD: n = 20; TD: n = 22) did not differ on gender, age, or IQ score (ps > 0.18). In contrast to the results for the visual TOJ task, performance on the auditory TOJ task did differ significantly between groups. Specifically, children with ASD required 48% more time between auditory stimuli to reliably determine which click occurred first. Thus, whereas children with ASD required 107.8 ms between auditory stimuli to perform at threshold, children with TD required only 73.0 ms, t(40) = 3.98, p = 0.0002 (Figure 2B).

Figure 2. Threshold values for visual and auditory temporal order judgment tasks. Children with autism spectrum disorders (ASD) show similar thresholds to typically developing children for the visual TOJ task (A). However, children with ASD show significantly larger thresholds for the auditory TOJ task (B). (*p < 0.05). Note that each of these threshold values was confirmed via a validation procedure (see text for additional detail). Error bars represent standard error of the mean (SEM).

Performance on the Multisensory TOJ Task

In addition to assessing visual temporal acuity, the visual TOJ staircase procedure allowed us to specify the visual SOA within the multisensory TOJ task for each participant such that baseline visual-only condition performance was expected to be approximately equivalent (75%) across all individuals. In this way we could then determine the effect of task-irrelevant auditory stimuli on performance during this visual task, as well as assess the temporal aspects of the performance enhancements that represent the hallmark of multisensory integration in this task. Twenty-five participants (ASD: n = 14; TD: n = 11) who completed experimental procedures were excluded from the multisensory analyses because their accuracies on visual-only trials within the multisensory task were well outside of the individually established threshold criterion (i.e., <60% (23 participants) or >89% (2 participants)). The latter two participants were excluded because their high accuracy presented a ceiling issue; there was little room for improvement with the addition of auditory stimuli in these participants. The remaining 23 participants likely did not meet their initial thresholds because of fatigue from working at threshold for approximately 30–45 minutes. Because this study was primarily interested in measuring differences in temporal multisensory processing, seven additional participants (ASD: n = 5; TD: n = 2) were excluded because they did not show improvements in accuracy with the addition of auditory stimuli. Interestingly, these seven participants showed strong performance decrements with the addition of the auditory stimuli. However, there were too few participants who showed these performance decrements to analyze whether they comprise a cohesive subsample of children. Importantly, there were no significant differences in exclusions from the multisensory task based on group, χ² (3, N = 62) = 3.53, p = 0.32. Further, remaining groups (ASD: n = 16; TD: n = 14) did not differ on gender, age, or IQ score (ps > 0.59).

Due to the large number of participants excluded from analyses for the multisensory task, we first used between groups, independent-sample t-tests to compare the unisensory thresholds for the remaining ASD and TD subsets in order to confirm that the results regarding unisensory temporal acuity observed for the larger samples analyzed in the unisensory tasks (i.e., equivalent visual TOJ thresholds, but higher auditory TOJ thresholds in ASD relative to TD) held for the subsample who successfully completed the multisensory task. As for the larger samples, we found that auditory thresholds remained significantly higher in children with ASD [t(28) = 2.11, p = 0.02] whereas visual thresholds did not differ between groups [t(28) = 1.24, p = 0.23]. This finding suggests that, despite the large number of participants excluded across both groups, the remaining subset of participants described in the analyses below is representative of the full sample in displaying similar patterns of unisensory temporal functioning.

To test for multisensory integration-related performance gains related to the addition of task-irrelevant auditory stimuli to the visual TOJ task, we conducted between-group comparisons for both the accuracy and the response times for trials in which there was no delay between the onsets of the second visual and auditory stimuli (i.e., multisensory delay = 0) and trials in which only visual stimuli were presented (i.e., visual-only trials; Figure 3). Accuracy and response times for visual-only trials did not differ significantly between children with ASD and children with TD, [accuracy: t(28) = 0.60, p = 0.55; response time: t(28) = 0.22, p = 0.83], confirming that both groups performed equivalently on the baseline visual task in the context of the multisensory TOJ task. However, one-sample t-tests for the 0 ms multisensory delay condition revealed that children with ASD significantly improved with the addition of the simultaneous auditory stimuli in both performance accuracy, t(15) = 2.47, p = 0.02, and response time, t(15) = 5.48, p < 0.001, whereas children with TD improved significantly in response time, t(13) = 3.07, p = 0.005, but not in performance accuracy, t(13) = −0.31, p = 0.38.

Figure 3. Performance accuracy (A) and response time (B) for simultaneous multisensory trials (i. e., multisensory delay = 0) compared to visual-only trials. Error bars represent standard error of the mean (SEM).

Temporal Dependence of the Multisensory TOJ Task

In accordance with previous studies (Morein-Zamir et al., 2003; Hairston et al., 2005, 2006), we found that delaying the second auditory stimulus relative to the second visual stimulus led to significant improvements in both accuracy and response time over a specific range of multisensory delays for both the ASD and TD participants.

Separate ANOVAs with SOA as the within-subjects factor and group as the between-subjects factor were conducted for improvements in both accuracy and response time. The main effect of SOA was significant for both the accuracy, F(7,210) = 3.38, p = 0.002, and response time, F(7,210) = 20.9, p = 0.001, data, confirming the relation between temporal proximity and probability of integration. The main effect of group was significant for the accuracy, F(1,30) = 4.45, p = 0.04, but not response time, F(1,30) = 1.14, p = 0.29, data, indicating that children with ASD showed greater performance gains with the addition of auditory stimuli than did children with TD. The interaction between SOA and group was not significant for either accuracy, F(7,210) = 1.53, p = 0.16, or response time, F(7,210) = 0.50, p = 0.83, indicating that the global relation between temporal proximity of auditory and visual stimuli and improvement in performance did not differ between groups.

The temporal window was defined for each group as the contiguous span of multisensory delays within which significant improvements over the visual-only baseline were observed. Windows were defined separately for each group using both the accuracy and response time data. Interestingly, the range of delays (i.e., the multisensory temporal window) that led to these performance improvements differed substantially between groups (accuracy: Figure 4; response time: Figure 5).

Figure 4. Percent improvement in accuracy relative to visual-only performance as a function of multisensory delay. Whereas typically developing children show improvements for short delays (i.e., 50–150 ms), children with ASD show improvements for both moderate and short delays (i.e., 0–300 ms). The solid line indicates continuous significant (p < 0.05) differences from zero (i.e., the multisensory temporal window). Error bars represent standard error of the mean (SEM).

Figure 5. Improvement in response times relative to visual-only performance as a function of multisensory delay. Whereas typically developing children show improvements for short delays (i.e., 0–200 ms), children with ASD show improvements for both moderate and short delays (i.e., 0–300 ms). The solid line indicates continuous significant (p < 0.05) differences from zero. Error bars represent standard error of the mean (SEM).

Children with ASD showed significant improvements in accuracy from the 0 ms delay condition to the 300 ms delay condition. In contrast, children with TD showed significant improvements in accuracy from the 50 ms delay condition to the 150 ms delay condition (Table 2), a finding consistent with prior work in typical adults (Hairston et al., 2006). Lending additional support to the differences in temporal binding window size, children with ASD showed significantly more improvement in accuracy when compared to their TD peers for the 200 ms, t(28) = 2.64, p = 0.013, and 300 ms, t(28) = 2.18, p = 0.038, multisensory delay conditions, as well as for the zero multisensory delay condition reported above (i.e., at multisensory delay conditions that were inside the temporal binding window for children with ASD, but outside the window for children with TD). Thus, the extent of the multisensory temporal binding window for improvements in accuracy in children with ASD was approximately doubled compared to that in children with TD.

Table 2. One-sample t-tests for improvements in performance accuracy at each SOA for both children with ASD and TD.

With regard to response times, children with ASD showed faster responses from the 0 ms delay condition to the 300 ms delay condition. In contrast, children with TD showed improvements in response time from the 0 ms delay condition to the 200 ms delay condition (Table 3). Although children with TD showed significant improvements in response time over a greater range of delays relative to those for which they showed improvements in accuracy, children with ASD still showed improvements in response time for approximately 100 ms longer than children with TD.

Table 3. One-sample t-tests for improvements in response time at each SOA for both children with ASD and TD.

Discussion

General Findings

The results of the current study validate and extend our previous finding of a prolonged temporal window of multisensory integration in ASD (Foss-Feig et al., 2010). Previously, we demonstrated this expanded window of temporal integration within the context of a multisensory illusion [i.e., the sound-induced double flash (flash-beep) illusion]. In the current study, we expand on this finding by establishing that children with ASD show gains in performance resulting from multisensory stimuli over a longer temporal window than TD children on a TOJ task. These performance gains manifest both as improvements in accuracy and as faster responses relative to the unisensory (i.e., visual-ony) baseline condition across an increased range of multisensory delays. Together, these two studies provide converging evidence that multisensory temporal processing, and more specifically the multisensory temporal binding window, is significantly altered in ASD.

By measuring temporal processing both within and across sensory systems, the current study provides a perspective on the relative contributions of unisensory processing changes to alterations in multisensory function. Thus, whereas visual temporal acuity was comparable across groups as measured using a standard visual TOJ task, both auditory and multisensory function were significantly impacted in ASD. Our findings of intact visual temporal processing are among the first that indicate that basic visual temporal processing may be spared in children with ASD. In fact, there is very limited literature on basic visual temporal processing in ASD, relative to a fairly extensive literature on visual spatial processing in which intact and enhanced discrimination abilities are consistently found, particularly for simple stimuli (for review see Simmons et al., 2009). Along with spatial visual processing, temporal visual processing has been investigated in ASD using visual motion detection paradigms (for review see Dakin and Frith, 2005). Interestingly, it has been shown that detection of first-order motion appears to be intact in ASD, while second-order motion detection is impaired (Bertone et al., 2005). In addition, deficits are seen when motion is derived from higher-order (e.g., biological) cues likely to reflect greater integrative processes (Blake et al., 2003). The current study extends our knowledge of visual function in autism by providing the first evidence that spared perception for basic visual features may also extend to the temporal domain.

Alterations in Auditory Temporal Processing in ASD

The observed difference in auditory TOJ performance is consistent with prior psychophysical and electrophysiological studies of temporal processing conducted in individuals with ASD. Previous behavioral studies have shown that individuals with ASD have difficulties reproducing auditory stimuli of standardized duration (Szelag et al., 2004), as well as difficulties detecting duration changes among auditory stimuli (Lepisto et al., 2006). In two separate studies, Lepisto et al. (2005, 2006) demonstrated reduced mismatch negativity in response to duration changes in non-speech sounds, providing electrophysiological evidence indicating atypical responses to the temporal structure of discrete auditory stimuli. Results of the present study extend these findings by suggesting that the ability to discriminate timing information between sequential stimuli is also impaired in ASD.

These differences in auditory temporal function could reflect a decreased ability of neurons in the primary auditory cortex of children with ASD to resolve differences in the onset of neural signals produced by the individual auditory stimuli. A delay in primary auditory cortical response would be consistent with findings from (Roberts et al., 2010), who reported delayed latency of evoked potentials in superior temporal gyrus in response to tones of various pitch in individuals with ASD. Similarly, (Oram-Cardy et al., 2005) demonstrated delayed mismatch negativity to both speech and non-speech sounds, suggesting that not only is the neural response to timing information in auditory stimuli atypical, but the timing of the brain’s response itself is delayed in response to auditory input. At a mechanistic level, these delays may be the result of decreases in the signal-to-noise ratio of neural signaling processes for auditory cues in autism, resulting in poorer time-locking of neural responses to discrete sensory events (Rubenstein and Merzenich, 2003). Results from (Russo et al., 2009) support this hypothesis by showing that the ERP response to auditory speech stimuli in the absence of background noise for children with ASD was similar to that with background noise for children with TD, suggesting degraded response to auditory stimuli at baseline in ASD. Electrophysiological studies examining the neural response to timing differences between auditory stimuli could help clarify the potential contributions of a reduced signal-to-noise ratio to decreased auditory temporal acuity in ASD.

An alternate hypothesis related to our auditory TOJ findings is that poorer performance in children with ASD relative to children with TD on the auditory TOJ task may be the result of deficits in inter-hemispheric communication, rather than reflecting deficits in the basic encoding of auditory stimuli. Numerous studies have shown lateralization changes in autism (Escalante-Mead et al., 2003; Herbert et al., 2005; Stroganova et al., 2007). Since participants in the current study were asked to distinguish between auditory stimuli presented to the left and right ears, disruptions in lateralization could play a substantive role in the differences observed between groups. Further studies are needed to clarify whether individuals with ASD show decreased temporal acuity for auditory tasks that do not rely on spatial information.

Comparisons with Prior Studies of Multisensory Processing in ASD

The current study further confirms previous findings of intact multisensory integration for low-level stimuli (Van der Smagt et al., 2007; Foss-Feig et al., 2010). The results of these three studies stand in contrast to other published reports of disrupted audiovisual integration in the context of ASD (Williams et al., 2004; Smith and Bennetto, 2007). In reviewing these studies, the most straightforward source of these differences seems to lie in the more complex and social nature of the auditory information (i.e., language) used in the latter set of studies. Deficits seen with the use of such stimuli could reflect processing problems at levels well upstream of the sensory and integrative processes being indexed by more simplistic audiovisual stimuli.

Mechanistic Implications: Toward the Brain Bases for Altered Multisensory Temporal Binding

As discussed previously (Foss-Feig et al., 2010), there are several possible neural mechanisms for the extended temporal window of multisensory integration in children with ASD. Rubenstein and Merzenich’s decreased signal-to-noise ratio hypothesis described above within the auditory system could extend to the multisensory domain, where a protracted time window within which cross-modal stimuli can interact may emerge as a result of – and possibly even as a compensatory mechanism for – the imprecise time-locking of neural responses to stimuli within individual sensory domains. Alternatively, the current data are also consistent with the temporal binding deficit hypothesis proposed by Brock et al. (2002), which suggests that activity within networks of interconnected sensory areas are not as strongly correlated in ASD, resulting in disruptions in the binding of perceptual information. It may be the case that these neural signals are not so drastically uncorrelated as to cause decoupling across regions (as initially hypothesized by Brock and colleagues), but instead occur in such a way as to necessitate an extended temporal binding window within which two stimuli can continue to be bound as part of one event. Further study is needed to provide additional support for one or the other of these theories, or to identify additional mechanistic possibilities. Such studies, if capable of employing neuroimaging tools such as fMRI and EEG/ERP, will be vital for improving our understanding of how the neural networks subserving multisensory integration may be disrupted in ASD and how these disruptions could result in an enlargement in the temporal window of multisensory integration.

Disrupted Temporal Multisensory Processing: a Common Disruption in Developmental Disorders?

An enlargement in the temporal window of multisensory integration has been found in other developmental disorders including developmental dyslexia (Hairston et al., 2005). Due to the multisensory nature of the relevant stimuli in the human environment, disruptions in the temporal fidelity with which auditory and visual stimuli are paired together and integrated could lead both to symptoms of dyslexia (e.g., difficulty mapping written words to their phonemic representation in speech) and of ASD (e.g., difficulty combining speech signals with visual cues from facial expressions and gestures to interpret others’ communicative intent). However, it is important to note that, while both groups demonstrate an enlarged temporal binding window for audiovisual stimuli, some differences in the nature and extent of temporal processing abnormalities between the two disorders exist. For example, with regard to unisensory temporal processing, individuals with dyslexia show both auditory and visual temporal processing impairments (e.g., Laasonen et al., 2001; Hairston et al., 2005), while our results suggest that individuals with ASD have impairment in auditory but not visual temporal processing. These differences may reflect divergent disorder-specific neural dysfunction related to temporal processing that ultimately results in similar disruptions of multisensory temporal processing. Along these lines, it has been proposed that autism results from local hyperconnectivity with long-range hypoconnectivity (Rippon et al., 2007), whereas dyslexia involves proximal hypoconnectivity but distal hyperconnectivity (Williams and Casanova, 2010). These opposite patterns might both result in atypical temporal multisensory function, but would likely result in divergent patterns of strengths and weaknesses in ASD and dyslexia. Alternatively, morphological or functional abnormality in one or more discrete brain regions that form connected networks for multisensory integration could contribute to the shared profile of an enlargement in the multisensory temporal binding window, while simultaneously giving rise to ASD- or dyslexia-specific dysfunction when the particular site of abnormality differed between disorders. Future research exploring the neural underpinnings of the enlarged temporal window in both disorders should be directed toward distinguishing convergent and divergent neural abnormalities that could impact temporal multisensory processing, and should include examination of subcortical structures such as the cerebellum that have been implicated in both disorders and may contribute to optimizing the temporal integration of sensory inputs (Nicolson and Fawcett, 2005). Further investigation may elucidate different mechanisms resulting in similar patterns of multisensory temporal function, as well as clarify the specificity of altered temporal multisensory function in producing the divergent symptomology in developmental disorders.

Conclusions and Implications

We are constantly bombarded by information from all senses, and our brains must combine individual unisensory events that are temporally proximal and likely to have occurred together into a unitary multisensory percept. A disruption in the temporal precision with which a multisensory perception is created from its component unisensory parts is likely to be compounded at subsequent processing stages and to lead to more pronounced disruptions in the understanding of complex stimuli such as speech, as have been reported in other studies. The extended temporal multisensory binding window described in the current study is likely to have far-reaching consequences for children with ASD and could account for core deficits in social and communication abilities, though the present study does not allow this to be tested empirically on an individual level since the multisensory TOJ task does not give an accurate estimate of the temporal window for individual participants. Nonetheless, enlargement of the temporal window during language acquisition could impair a child’s ability to correctly associate the visual and auditory components of speech, thus delaying or (if severe enough) preventing the acquisition of language. With regard to core social deficits, numerous auditory and visual stimuli involved in a social interaction (e.g., subtle changes in tone of voice, facial expression, and body language) must all be integrated seamlessly for the interaction to be successful. Altered experiences with multisensory processing from early ages may have detrimental effects on subsequent development of complex social abilities such as empathy and reciprocity. Research into when and how enlargements of the temporal window for multisensory integration first emerge and how they might result in social communication deficits central to ASD should be an important focus of future studies, as altered multisensory temporal processing may serve as an early marker of later deficits or suggest potential targets for intervention.

In conclusion, this study demonstrates that the temporal processing of auditory and multisensory stimuli is disrupted in ASD, providing empirical evidence in support of the clinical and anecdotal literature that consistently reports sensory functioning impairments in ASD. A limitation of the study is the inherent bias toward high functioning ASD necessitated by the cognitive and attentional demands of the psychophysical tasks used. In fact, even in the high functioning population used in this study, many of the participants were not able to complete one or more of the tasks. Further research is needed to fully characterize the nature and extent of these unisensory and multisensory temporal processing differences in ASD, to elucidate their neural substrates, to clarify their profile in lower-functioning individuals, to assess their emergence early in development, and to relate these findings to the core deficits in ASD. It is anticipated that this line of investigation will ultimately contribute to a broader understanding of ASD and lead to more sensitive diagnostic instruments and more specific remediation strategies.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This work was supported by a Marino Autism Research Institute Discovery Grant (PI: Mark T. Wallace); National Institute of Child Health and Human Development T32 HD07226 and Dennis Weatherstone Pre-doctoral Fellowship for Jennifer H. Foss-Feig; Meharry-Vanderbilt Alliance Training Grant for Leslie D. Kwakye; and the Vanderbilt Kennedy Center for Research on Human Development.

References