The effect of measurement error on the test–retest reliability of repeated mismatch negativity measurements

Abstract

Objective

The aim was to study how the measurement error affects the repeatability of mismatch negativity (MMN) measurements.

Methods

Event-related potentials (ERPs) to changes in sound frequency, location, intensity, duration, and composition were recorded five times during 1–3 weeks from 13 healthy adults using a multi-feature MMN paradigm. The accumulation of MMN was modeled empirically with respect to measurement error, and repeatability was estimated at 0.6–3.5-μV error levels. The analysis was made for the results in the single deviant conditions and their pattern (auditory discrimination profile).

Results

At the single-subject level, the measurement error significantly affected the repeatability until it went below 9–17% of MMN peak amplitude. At the group level, the threshold was higher. Peak amplitude was generally the most repeatable parameter. Latency was superior when the error was moderate or small (<2–3 μV).

Conclusions

The measurement error affects the repeatability of MMN. In single-subject studies, it should not be neglected if it exceeds 10% of the MMN amplitude. The application of the auditory discrimination profile is recommended for future applications.

Significance

The study provided quantitative results to support the discussion on improving the repeatability of the MMN measurements. They are expected to apply conditionally to other ERP measurements, too.

Introduction

During the last few decades, an increasing number of studies have demonstrated the application of auditory event-related potentials (AERPs) in the investigation of neurological and psychiatric disorders. One of the most popular measures appearing in these studies is mismatch negativity (MMN, Näätänen et al., 1978). MMN indicates the discrimination of a change in the sensory input (Näätänen, 1990, Näätänen, 1992, Näätänen and Winkler, 1999, Näätänen et al., 2001, Garrido et al., 2009) and it can be measured by analyzing the difference in the AERPs elicited by slightly different stimuli (Schröger, 1998, Duncan et al., 2009). Applications of MMN include the investigation of, e.g., dyslexia, schizophrenia, memory disorders, and coma outcome (for reviews see Kujala et al., 2007, Näätänen et al., 2007, Duncan et al., 2009). So far, the focus has been on general research, but MMN is also a tempting option for clinical use because of its versatile nature.

However, like many other AERP components (which are modulated by cognitive functions of the brain), MMN is not yet suited to clinical applications, because it cannot be measured reliably enough (Kujala et al., 2007). According to the reported studies, the test–retest reliability of MMN varies between 0.37 and 0.87, depending on the experimental conditions (Chertoff et al., 1988, Pekkonen et al., 1995, Lang et al., 1995, Escera and Grau, 1996, Frodl-Bauch et al., 1997, Joutsiniemi et al., 1998, Kathmann et al., 1999, Tervaniemi et al., 1999, Escera et al., 2000, Dalebout and Fox, 2001, Kujala et al., 2001, Light and Braff, 2005, Hall et al., 2006, Lew et al., 2007). The repeatability of this size is tolerated in group studies, but the results from single-subject measurements show too much unwanted variation (Escera and Grau, 1996, Tervaniemi et al., 1999). In order to allow the clinical use of MMN, the single-subject measurements need to be repeatable, too (Kujala et al., 2007, Näätänen et al., 2007).

The first systematic studies that aimed at improving the test–retest reliability of MMN were reported by Pekkonen et al. (1995) and Lang et al. (1995). It was found that the repeatability depends on the presentation of the stimuli, the number of trials recorded (Pekkonen et al., 1995), the parameterization of MMN, and the characteristics (e.g., age, discrimination ability, and alertness) of the test subjects (Lang et al., 1995). This provided a good starting point for the research and still forms the basis of the methods that are used today. Currently, the best way to secure reliable recordings is to take care of the signal quality (Pivik et al., 1993, Sinkkonen and Tervaniemi, 2000), to use appropriate stimuli (Duncan et al., 2009), and to apply efficient recording procedures, such as the multi-feature paradigm (Näätänen et al., 2004, Pakarinen et al., 2007, Pakarinen et al., 2009). Valid stimulus design and a high signal-to-noise ratio (SNR) reduce the variation in the results (Lang et al., 1995, Sinkkonen and Tervaniemi, 2000). An efficient recording procedure, on the other hand, permits efficient denoising through averaging and robust rejection of contaminated data while not causing unnecessary mental fatigue and stress for the test subject (e.g., Pakarinen et al., 2009). In addition, distracting the test subjects from attending to the stimuli also improves the repeatability as a result of the reduced modulation of the recorded responses (Näätänen, 1995).

Considering the further development, many authors have suggested that a major part of the uncertainty would be contributed by the measurement error (e.g., Lang et al., 1995, Sinkkonen and Tervaniemi, 2000, Hall et al., 2006, Paukkunen et al., 2010a). As a smaller error yields higher repeatability, it is probable that even a small error could have a major effect on the repeatability of a weak response like MMN. Quantitative studies, however, have not been published on the subject and the extent to which the effect is relevant has not been properly evaluated. In this study, a series of repeated MMN recordings is made with multiple test subjects to analyze the effect of the measurement error in vivo. The main objective is to determine how the test–retest reliability of the results changes as a function of the measurement error. In addition, it is studied how this affects the parameterization of MMN.