A cognitive and neurophysiological test of change from an individual’s baseline

Abstract

Objective

An automated cognitive neurophysiological test is presented that characterizes how an individual was affected by a drug or treatment. The test calculates sub-scores for working memory task performance, cortical activation, and alertness, and combines the sub-scores into an overall score.

Methods

The test was applied in a double-blind, placebo-controlled study of alcohol, caffeine, diphenhydramine, and sleep deprivation in 16 healthy adults.

Results

The between- and within-day variability of the sub-scores and overall scores for placebo were all near zero, suggesting that the scores are stable. All treatments affected the overall score, while differential effects on sub-scores highlighted the added value of EEG measures.

Conclusions

The test is sensitive to relatively mild alterations in cognitive function. Its automation makes it suitable for use in large-scale clinical trials.

Significance

By combining task performance with EEG brain function measures, the test may prove to have better sensitivity and specificity in detecting changes due to drugs or other treatments than comparable neuropsychological test batteries that do not directly measure brain function signals.

Introduction

Cognitive brain function is affected by many diseases, by the intended and unintended effects of treatment medications, and by a variety of stressors such as disturbed sleep. There are many batteries for assessing cognition based on performance on tests of cognitive function and rating scales. However, behavior is the end product of many neural systems, some of which may be recruited or adapted in some way to compensate for deficits. For instance, a motivated impaired person may make a greater effort and not show signs of impairment, and a person who is simply drowsy may have a poor cognitive test score but not have a disease affecting cognitive brain function per se. Tests that do not directly measure brain function signals have difficulty accounting for factors such as motivation and alertness, and therefore have limited sensitivity and specificity.

The lack of a clinical standard for testing cognitive brain function has been cited as a major confounding factor in the discrepancies between the results of different clinical trials (Vermeulen and Aldenkamp, 1995). Standardized neurologically based tests of an individual’s cognitive brain function have the potential to make such evaluations more sensitive and efficient, and could be helpful to researchers, clinicians, and patients. Towards this goal, we present an automated cognitive neurophysiological test, the Sustained Working Memory Test (SWMT) that combines cognitive test performance measures with EEG measures. The multivariate analysis combines task response accuracy and speed measures with task-related and resting EEG measures to arrive at an assessment of how an experimental drug or stressor, or a disease and its treatment, has changed an individual’s neurocognitive functional status. This paper focuses on the initial version of the SWMT that assesses treatment-related changes relative to an individual’s placebo or pre-treatment baseline test.

Working memory (WM) is the fundamental cognitive function of controlling attention and actively sustaining its focus on a particular mental representation (Baddeley, 1992, Engle et al., 1999). It is essential for reasoning, planning, learning, and other higher cognitive functions, and is highly correlated with performance on psychometric tests of cognitive ability such as IQ tests (Carpenter et al., 1990, Gevins and Smith, 2000, Kyllonen and Christal, 1990).

Research on the neurophysiological signals of WM often employs “n-back” tasks in which participants respond to simple stimuli presented at different locations on a computer monitor once every few seconds (Gevins and Cutillo, 1993, Gevins et al., 1990). The load imposed on WM is varied across easy and more difficult versions, while perceptual and motor demands are kept constant (Gevins et al., 1979b, Gevins et al., 1980). A spatial n-back WM task is used in the SWMT to minimize language-dependent cultural bias in the testing. In the easier 1-back task, participants have to decide whether the location of the current stimulus (a dot) is the same as on the previous trial (1-back); in the more difficult 2-back task, the current location of the dot has to be compared with the remembered position of the dot two trials ago in a continuous block of 50 trials. This requires constant updating of the information to be remembered on each trial, as well as focused attention to new stimuli and maintenance of representations of recently presented stimuli. To be successful when WM demands are high, as in the 2-back task, participants typically must make a significant and continuous mental effort. In this regard, the easier version of the task serves as a control condition.

Functional neuroimaging studies reliably demonstrate that n-back WM tasks activate circuitry in the frontal lobes critical to the control of attention and the maintenance of representations in WM (Cohen et al., 1994, Jansma et al., 2000, Jonides et al., 1993, McCarthy et al., 1994), and that the magnitude and extent of this activation is directly related to increasing load in n-back tasks (Braver et al., 1997). The discriminant validity of the n-back task as a measure of concentration is illustrated by the task impairment exhibited by groups with deficits suggesting impaired frontal-lobe function, including patients with schizophrenia and children with head injury or ADHD (McCallister et al., 2001, Perlstein et al., 2003, Shallice et al., 2002). Abnormalities in frontal-lobe activation during n-back task performance in such groups have been noted even in cases where performance measures were insensitive (Callicott et al., 2003, McCallister et al., 2001). Such findings provide a strong rationale to use such tasks to gauge executive dysfunction and its neural correlates.

The spectral characteristics of the EEG display regular patterns of difficulty-related modulation during n-back task performance (Gevins et al., 1997). Fig. 1 illustrates regional differences in EEG in response to manipulations of WM load on spectral power (left) and high-resolution topographic maps of spectral peaks (right). At the frontal midline site, power in a 5–7 Hz (theta) band is increased during the high-load task. This “frontal-midline theta” signal is known to increase in difficult, attention demanding tasks requiring a sustained focus of concentration (Miyata et al., 1990). Topographic analyses and source modeling (Ishii et al., 1999) point to the anterior cingulate cortex as the likely origin of this signal. This region plays an important role in attention control (Posner and Rothbart, 1992), and activation in this region is known to increase with task difficulty (Paus et al., 1998). The attenuation of signals in the 8–13 Hz (alpha) band in the high relative to the low-load n-back WM task has been observed in numerous studies (e.g. Gundel and Wilson, 1992), suggesting that the magnitude of alpha activity is inversely proportional to the quantity of cortical neurons recruited into a transient functional network for purposes of task performance (Mulholland, 1995, Pfurtscheller and Klimesch, 1992). Convergent evidence is also provided by observations of a negative correlation between alpha power and regional brain activation as measured with PET (Larson et al., 1998, Sadato et al., 1998) or fMRI (Goldman et al., 2002).

The EEG and, to a somewhat lesser extent, the performance measures during n-back tasks are highly reliable (Salinsky et al., 1991). In one study (McEvoy et al., 2000), average test–retest reliabilities were greater than .9 for EEG spectral features between two n-back task sessions one week apart (p < .001), and .86 for response speed (p < .001) and .47 for response accuracy (p < .05; the relatively low reliability observed for accuracy was due to a ceiling effect). Multivariate combinations of such EEG variables can identify specific cognitive states in individual participants accurately and reliably (Gevins et al., 1979a, Gevins et al., 1979c). For instance, multivariate EEG-based functions trained on one set of WM data and then cross-validated on new data correctly identified high vs. low-load conditions with over 95% accuracy (p < .001, Gevins et al., 1998). Such results illustrate that EEG measures can reliably recognize different levels of task-related attention engagement.

A number of studies have reported how EEG signals during the n-back WM task are affected by fatigue and sleep loss (Smith et al., 2002), by medications that affect cognition and alertness (Gevins et al., 2002, McEvoy et al., 2001), and by recreational drugs including alcohol and marijuana (Ilan and Gevins, 2001, Ilan et al., 2004). Using a variety of analysis methods, detection of the effect of a drug or sleep loss was consistently most accurate when EEG measures were combined with task performance measures. For instance, sensitivity was 96% and specificity 100% in distinguishing the relatively strong neurocognitive effects of a widely prescribed anti-epileptic drug (carbamazepine) from those of a newer drug (levetiracetam) with milder side effects using EEG and task performance measures, but sensitivity and specificity were only 75% and 75%, respectively, using measures from conventional neuropsychological tests and subjective questionnaires (Meador et al., 2007).

The SWMT therefore combines EEG and n-back task performance measures to quantify how a treatment has affected cognitive brain function. The reliability of the SWMT scores is illustrated here by computing between-day and within-day variability in a large sample of healthy adults who performed the test multiple times without an active drug or other treatment. As an example of the application of the SWMT, these no-treatment variability values are then used to assess the significance of the effects of caffeine, alcohol, the antihistamine diphenhydramine, and sleep deprivation. Based on the well-known effects of these drugs and sleep loss on cognition and brain function, the SWMT scores would be expected to show negative effects of alcohol, diphenhydramine, and sleep deprivation, and positive effects of caffeine.