A cognitive and neurophysiological test of change from an individual’s baseline
Introduction
Cognitive brain function is affected by many diseases, by the intended and unintended effects of treatment medications, and by a variety of stressors such as disturbed sleep. There are many batteries for assessing cognition based on performance on tests of cognitive function and rating scales. However, behavior is the end product of many neural systems, some of which may be recruited or adapted in some way to compensate for deficits. For instance, a motivated impaired person may make a greater effort and not show signs of impairment, and a person who is simply drowsy may have a poor cognitive test score but not have a disease affecting cognitive brain function per se. Tests that do not directly measure brain function signals have difficulty accounting for factors such as motivation and alertness, and therefore have limited sensitivity and specificity.
The lack of a clinical standard for testing cognitive brain function has been cited as a major confounding factor in the discrepancies between the results of different clinical trials (Vermeulen and Aldenkamp, 1995). Standardized neurologically based tests of an individual’s cognitive brain function have the potential to make such evaluations more sensitive and efficient, and could be helpful to researchers, clinicians, and patients. Towards this goal, we present an automated cognitive neurophysiological test, the Sustained Working Memory Test (SWMT) that combines cognitive test performance measures with EEG measures. The multivariate analysis combines task response accuracy and speed measures with task-related and resting EEG measures to arrive at an assessment of how an experimental drug or stressor, or a disease and its treatment, has changed an individual’s neurocognitive functional status. This paper focuses on the initial version of the SWMT that assesses treatment-related changes relative to an individual’s placebo or pre-treatment baseline test.
Working memory (WM) is the fundamental cognitive function of controlling attention and actively sustaining its focus on a particular mental representation (Baddeley, 1992, Engle et al., 1999). It is essential for reasoning, planning, learning, and other higher cognitive functions, and is highly correlated with performance on psychometric tests of cognitive ability such as IQ tests (Carpenter et al., 1990, Gevins and Smith, 2000, Kyllonen and Christal, 1990).
Research on the neurophysiological signals of WM often employs “n-back” tasks in which participants respond to simple stimuli presented at different locations on a computer monitor once every few seconds (Gevins and Cutillo, 1993, Gevins et al., 1990). The load imposed on WM is varied across easy and more difficult versions, while perceptual and motor demands are kept constant (Gevins et al., 1979b, Gevins et al., 1980). A spatial n-back WM task is used in the SWMT to minimize language-dependent cultural bias in the testing. In the easier 1-back task, participants have to decide whether the location of the current stimulus (a dot) is the same as on the previous trial (1-back); in the more difficult 2-back task, the current location of the dot has to be compared with the remembered position of the dot two trials ago in a continuous block of 50 trials. This requires constant updating of the information to be remembered on each trial, as well as focused attention to new stimuli and maintenance of representations of recently presented stimuli. To be successful when WM demands are high, as in the 2-back task, participants typically must make a significant and continuous mental effort. In this regard, the easier version of the task serves as a control condition.
Functional neuroimaging studies reliably demonstrate that n-back WM tasks activate circuitry in the frontal lobes critical to the control of attention and the maintenance of representations in WM (Cohen et al., 1994, Jansma et al., 2000, Jonides et al., 1993, McCarthy et al., 1994), and that the magnitude and extent of this activation is directly related to increasing load in n-back tasks (Braver et al., 1997). The discriminant validity of the n-back task as a measure of concentration is illustrated by the task impairment exhibited by groups with deficits suggesting impaired frontal-lobe function, including patients with schizophrenia and children with head injury or ADHD (McCallister et al., 2001, Perlstein et al., 2003, Shallice et al., 2002). Abnormalities in frontal-lobe activation during n-back task performance in such groups have been noted even in cases where performance measures were insensitive (Callicott et al., 2003, McCallister et al., 2001). Such findings provide a strong rationale to use such tasks to gauge executive dysfunction and its neural correlates.
The spectral characteristics of the EEG display regular patterns of difficulty-related modulation during n-back task performance (Gevins et al., 1997). Fig. 1 illustrates regional differences in EEG in response to manipulations of WM load on spectral power (left) and high-resolution topographic maps of spectral peaks (right). At the frontal midline site, power in a 5–7 Hz (theta) band is increased during the high-load task. This “frontal-midline theta” signal is known to increase in difficult, attention demanding tasks requiring a sustained focus of concentration (Miyata et al., 1990). Topographic analyses and source modeling (Ishii et al., 1999) point to the anterior cingulate cortex as the likely origin of this signal. This region plays an important role in attention control (Posner and Rothbart, 1992), and activation in this region is known to increase with task difficulty (Paus et al., 1998). The attenuation of signals in the 8–13 Hz (alpha) band in the high relative to the low-load n-back WM task has been observed in numerous studies (e.g. Gundel and Wilson, 1992), suggesting that the magnitude of alpha activity is inversely proportional to the quantity of cortical neurons recruited into a transient functional network for purposes of task performance (Mulholland, 1995, Pfurtscheller and Klimesch, 1992). Convergent evidence is also provided by observations of a negative correlation between alpha power and regional brain activation as measured with PET (Larson et al., 1998, Sadato et al., 1998) or fMRI (Goldman et al., 2002).
The EEG and, to a somewhat lesser extent, the performance measures during n-back tasks are highly reliable (Salinsky et al., 1991). In one study (McEvoy et al., 2000), average test–retest reliabilities were greater than .9 for EEG spectral features between two n-back task sessions one week apart (p < .001), and .86 for response speed (p < .001) and .47 for response accuracy (p < .05; the relatively low reliability observed for accuracy was due to a ceiling effect). Multivariate combinations of such EEG variables can identify specific cognitive states in individual participants accurately and reliably (Gevins et al., 1979a, Gevins et al., 1979c). For instance, multivariate EEG-based functions trained on one set of WM data and then cross-validated on new data correctly identified high vs. low-load conditions with over 95% accuracy (p < .001, Gevins et al., 1998). Such results illustrate that EEG measures can reliably recognize different levels of task-related attention engagement.
A number of studies have reported how EEG signals during the n-back WM task are affected by fatigue and sleep loss (Smith et al., 2002), by medications that affect cognition and alertness (Gevins et al., 2002, McEvoy et al., 2001), and by recreational drugs including alcohol and marijuana (Ilan and Gevins, 2001, Ilan et al., 2004). Using a variety of analysis methods, detection of the effect of a drug or sleep loss was consistently most accurate when EEG measures were combined with task performance measures. For instance, sensitivity was 96% and specificity 100% in distinguishing the relatively strong neurocognitive effects of a widely prescribed anti-epileptic drug (carbamazepine) from those of a newer drug (levetiracetam) with milder side effects using EEG and task performance measures, but sensitivity and specificity were only 75% and 75%, respectively, using measures from conventional neuropsychological tests and subjective questionnaires (Meador et al., 2007).
The SWMT therefore combines EEG and n-back task performance measures to quantify how a treatment has affected cognitive brain function. The reliability of the SWMT scores is illustrated here by computing between-day and within-day variability in a large sample of healthy adults who performed the test multiple times without an active drug or other treatment. As an example of the application of the SWMT, these no-treatment variability values are then used to assess the significance of the effects of caffeine, alcohol, the antihistamine diphenhydramine, and sleep deprivation. Based on the well-known effects of these drugs and sleep loss on cognition and brain function, the SWMT scores would be expected to show negative effects of alcohol, diphenhydramine, and sleep deprivation, and positive effects of caffeine.
Section snippets
EEG recording
EEG signals were recorded during task and resting conditions with a stretchable nylon cap with electrodes over bilateral and midline dorsolateral prefrontal locations (F9, F10, Fp1, Fp2, FpZ, F3, F4, Fz), midline sensorimotor cortex (Cz), lateral superior parietal cortex (P3, P4) and midline parieto-occipital cortex (POz), referenced to digitally linked mastoids. These locations were selected for their sensitivity to variations in working memory load on the basis of cognitive EEG studies with
Normal variability of test scores in the absence of a treatment
The variability of the overall score and the three sub-scores in the absence of a drug or other experimental treatment was computed for 127 healthy adults (mean age 34 years, range 18–70) who were tested in seven studies with drug and sleep deprivation interventions (Gevins et al., Gevins et al., 2002, Ilan et al., 2005, McEvoy et al., 2001, Meador et al., 2007, Smith et al., 2002, Smith et al., 2006). The experiments were conducted according to protocols approved by an NIH-registered
Discussion
An efficient neurological test of mental acuity that directly measures brain signals of fundamental cognitive functions such as attention and memory could be helpful to researchers, clinicians, and patients alike. Without such an objective measure, assessing treatment of disorders that affect thinking is necessarily imprecise, as a physician can only roughly gauge whether a patient’s cognitive brain function is deteriorating or improving with treatment. Here we present a test that is a first
Acknowledgements
This research was supported by grants from U.S. government agencies including the National Institute of Neurological Diseases and Strokes, The National Institute of Mental Health, The National Heart Lung and Blood Institute, The Air Force Research Laboratory and The Office of Naval Research.
We gratefully acknowledge the essential contributions of the following scientists, engineers and associates in the development reported here, including Enoch Callaway, Behram daCosta, Ritu Chellaramani,
References (53)
- et al.
A parametric study of prefrontal cortex involvement in human working memory
NeuroImage
(1997) - et al.
Spatiotemporal dynamics of component processes in human working memory
Electroenceph Clin Neurophysiol
(1993) - et al.
Tracking the cognitive pharmacodynamics of psychoactive substances with combinations of behavioral and neurophysiological measures
Neuropsychopharmacology
(2002) - et al.
EEG patterns during “cognitive” tasks II. Analysis of controlled tasks
Electroenceph Clin Neurophysiol
(1979) - et al.
EEG patterns during “cognitive” tasks I. Methodology and analysis of complex behaviors
Electroenceph Clin Neurophysiol
(1979) - et al.
Effects of prolonged mental work on functional brain topography
Electroenceph Clin Neurophysiol
(1990) - et al.
High resolution evoked potential imaging of the cortical dynamics of human working memory
Electroenceph Clin Neurophysiol
(1996) - et al.
Prolonged neurophysiological effects of cumulative wine drinking
Alcohol
(2001) - et al.
Specific versus nonspecific brain activity in a parametric n-back task
NeuroImage
(2000) - et al.
Reasoning ability is little more than working memory capacity?!
Intelligence
(1990)
A method for assessing alertness fluctuations from EEG spectra
Electroenceph Clin Neurophysiol
Test-retest reliability of cognitive EEG
Clin Neurophysiol
Human EEG, behavioral stillness and biofeedback
Int J Psychol
Prefrontal cortex dysfunction mediates deficits in working memory and prepotent responding in schizophrenia
Biol Psychiatry
Attentional mechanisms and conscious experience
Cognitive side-effects of chronic antiepileptic drug treatment: a review of 25 years of research
Epilepsy Res
Working Memory
Science
Abnormal fMRI response of dorsolateral prefrontal cortex in cognitively intact siblings of patients with schizophrenia
Am J Psychiatry
What one intelligence test measures: a theoretical account of the processing in the Raven Progressive Matrices Test
Psychol Rev
Activation of prefrontal cortex in a non-spatial working memory task with functional MRI
Hum Brain Mapp
Human brain potentials during the onset of sleep
J Neurophysiol
Individual differences in working memory capacity and what they tell us about controlled attention, general fluid intelligence and functions of the prefrontal cortex
High-resolution EEG mapping of cortical activation related to working memory: effects of task difficulty, type of processing, and practice
Cereb Cortex
Monitoring working memory load during computer-based tasks with EEG pattern recognition methods
Hum Fact
Neurophysiological measures of working memory and individual differences in cognitive ability and cognitive style
Cereb Cortex
Cited by (14)
Long-term stability of resting state EEG-based linear and nonlinear measures
2021, International Journal of PsychophysiologyTest-retest reliability of a single-channel, wireless EEG system
2016, International Journal of PsychophysiologyCitation Excerpt :Lower test-retest reliabilities for resting state qEEG data have been attributed to difficulty controlling for transient variations in vigilance, anxiety, and thought processes during this unstructured condition (Fernández et al., 1993). As such, mental states such as mood and alertness should also be taken into account, as intra-individual differences over time in either of these state factors can significantly influence the EEG (Fernández et al., 1993; Andreassi, 2007; Gevins et al., 2011; Vuga et al., 2006). Furthermore, research in healthy individuals has demonstrated that the synchronized neural activity of the EEG varies according to developmental stage (Koenig et al., 2002; Tong and Thakor, 2009).
Computerized neuropsychological testing in epilepsy: Overview of available tools
2013, SeizureCitation Excerpt :The practical consequence of this additional knowledge is increased sensitivity and specificity to the effects of a drug.33,34 The CNT is an automated hardware and software system with a fast application EEG hat that allows test administration by personnel with minimal training.35,36 The primary cognitive functions measured are working memory (n-back task) and sustained attention,37 with options for verbal episodic memory (word recognition task) and divided attention.
Long-term and within-day variability of working memory performance and EEG in individuals
2012, Clinical NeurophysiologyCitation Excerpt :WM task-related and resting EEG measures are sensitive to a variety of conditions that affect cognitive function, including mild cognitive impairment, sleep deprivation, and drug or alcohol use (Ilan and Gevins, 2001; Gevins et al., 2002; Smith et al., 2002, 2006; Ilan et al., 2004, 2005; Kushida et al., 2006; McEvoy et al., 2006; Missonnier et al., 2006; Meador et al., 2007; Deiber et al., 2009; Hart et al., 2010; Gevins et al., 2011a,b; Meador et al., 2011). We recently described an automated method for assessing cognitive function that combines performance measures on n-back WM tasks with simultaneously-recorded EEG measures (Gevins et al., 2011c). The data analysis algorithm integrates response accuracy and speed measures with task-related and resting EEG measures to arrive at an assessment of how an experimental drug or stressor, or a disease and its treatment, has changed an individual’s neurocognitive functional status relative to a baseline state.
Effect of Cognitive Reserve on Physiological Measures of Cognitive Workload in Older Adults with Cognitive Impairments
2023, Journal of Alzheimer's Disease
- 1
Present address: CorTechs Labs, San Diego, CA, USA.
- 2
Present address: Department of Radiology, University of California, San Diego, CA, USA.