SupplementAutomatic classification of patients with Alzheimer's disease from structural MRI: A comparison of ten methods using the ADNI database
Research Highlights
► Alzheimer vs controls : high accuracies with whole-brain methods (up to 81% sensitivity - 95% specificity). ► For the detection of prodromal Alzheimer, the sensitivity was substantially lower. ► For the prediction of conversion, the accuracy was only slightly higher than chance.
Introduction
Alzheimer's disease (AD) is the most frequent neurodegenerative dementia and a growing health problem. Definite diagnosis can only be made postmortem, and requires histopathological confirmation of amyloid plaques and neurofibrillary tangles. Early and accurate diagnosis of Alzheimer's Disease (AD) is not only challenging, but is crucial in the perspective of future treatments. Clinical diagnostic criteria are currently based on the clinical examination and neuropsychological assessment, with the identification of dementia and then of the Alzheimer's phenotype (Blennow et al., 2006). Patients suffering from AD at a prodromal stage are, mostly, clinically classified as amnestic mild cognitive impairment (MCI) (Petersen et al., 1999, Dubois and Albert, 2004), but not all patients with amnestic MCI will develop AD. Recently, more precise research criteria were proposed for the early diagnostic of AD at the prodromal stage of the disease (Dubois et al., 2007). These criteria are based on a clinical core of early episodic memory impairment and the presence of at least one additional supportive feature including abnormal MRI and PET neuroimaging or abnormal cerebrospinal fluid amyloid and tau biomarkers (Dubois et al., 2007). Neuroimaging therefore adds a positive predictive value to the diagnosis and includes measurements using structural MRI to assess medial temporal lobe atrophy and positron emission tomography using fluorodeoxyglucose (FDG) or amyloid markers (Fox and Schott, 2004, Jagust, 2006).
Many group studies based on volumetric measurements of regions of interest (ROI) (Convit et al., 1997, Convit et al., 2000, Jack et al., 1997, Jack et al., 1998, Juottonen et al., 1998, Laakso et al., 1998, Laakso et al., 2000, Busatto et al., 2003, Xu et al., 2000, Good et al., 2002, Chételat and Baron, 2003, Rusinek et al., 2004, Tapiola et al., 2008), voxel-based morphometry (Good et al., 2002, Busatto et al., 2003, Karas et al., 2003, Karas et al., 2004, Chételat et al., 2005, Whitwell et al., 2007, Whitwell et al., 2008) or group comparison of cortical thickness (Thompson et al., 2001, Thompson et al., 2003, Thompson et al., 2004, Lerch et al., 2005, Lerch et al., 2008, Bakkour et al., 2009, Dickerson et al., 2009, Hua et al., 2009, McDonald et al., 2009) have shown that brain atrophy in AD and prodromal AD is spatially distributed over many brain regions including the entorhinal cortex, the hippocampus, lateral and inferior temporal structures, anterior and posterior cingulate. However these analyses measure group differences and thus are of limited value for individual diagnosis.
Advances in statistical learning with the development of new machine learning algorithms capable of dealing with high dimensional data, such as the support vector machine (SVM) (Vapnik, 1995, Shawe-Taylor and Cristianini, 2000, Schölkopf and Smola, 2001), enable the development of new diagnostic tools based on T1-weighted MRI. Recently, several approaches have been proposed to automatically classify patients with AD and/or MCI from anatomical MRI (Fan et al., 2005, Fan et al., 2007, Fan et al., 2008b, Fan et al., 2008a, Colliot et al., 2008, Davatzikos et al., 2008a, Davatzikos et al., 2008b, Klöppel et al., 2008, Vemuri et al., 2008, Chupin et al., 2009a, Chupin et al., 2009b, Desikan et al., 2009, Gerardin et al., 2009, Hinrichs et al., 2009, Magnin et al., 2009, Misra et al., 2009, Querbes et al., 2009). These approaches could have the potential to assist in the early diagnosis of AD. These approaches can roughly be grouped into three different categories, depending on the type of features extracted from the MRI (voxel-based, vertex-based or ROI-based). In the first category, the features are defined at the level of the MRI voxel. Specifically, the features are the probability of the different tissue classes (grey matter, white matter and cerebrospinal fluid) in a given voxel (Lao et al., 2004, Fan et al., 2007, Fan et al., 2008b, Fan et al., 2008a, Davatzikos et al., 2008a, Davatzikos et al., 2008b, Klöppel et al., 2008, Vemuri et al., 2008, Hinrichs et al., 2009, Magnin et al., 2009, Misra et al., 2009). Klöppel et al. (2008) directly classified these features with an SVM. All other methods first reduce the dimensionality of the feature space relying on different types of features extraction, agglomeration and/or selection methods. Vemuri et al. (2008) used smoothing, voxel-downsampling, and then a feature selection step. Another solution is to group voxels into anatomical regions through the registration of a labeled atlas (Lao et al., 2004, Ye et al., 2008, Magnin et al., 2009). However, this anatomical parcellation may not be adapted to the pathology. In order to overcome this limitation, Fan et al. (2007) have proposed an adaptive parcellation approach in which the image space is divided into the most discriminative regions. This method has been used in several studies (Davatzikos et al., 2008a, Davatzikos et al., 2008b, Fan et al., 2008b, Fan et al., 2008a, Misra et al., 2009). In the second category, the features are defined at the vertex-level on the cortical surface (Desikan et al., 2009, Querbes et al., 2009). The methods of the third category include only the hippocampus. Their approach is based on the analysis of the volume and/or shape of the hippocampus (Colliot et al., 2008, Chupin et al., 2009a, Chupin et al., 2009b, Gerardin et al., 2009).
These approaches achieve high accuracy (over 84%). However, they were evaluated on different study populations, making it difficult to compare their respective discriminative power. Indeed, many factors such as degree of impairment, age, gender, genotype, educational level and MR image quality perceptibly affect the evaluation of the prediction accuracy. This variability between evaluations is increased for statistical reasons when the number of subjects is small. Therefore a meta-analysis would be of limited value to compare the prediction accuracies of different methods.
The goal of this paper was to compare different methods for the classification of patients with AD based on anatomical MRI, using the same study population. To that purpose, we used the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. Ten methods were evaluated. We tested five voxel-based approaches: a direct approach (Klöppel et al., 2008), an approach based on a volume of interest (Klöppel et al., 2008), an atlas-based approach (Magnin et al, 2009) and the approaches proposed by Vemuri et al., 2008, Fan et al., 2008b, Fan et al., 2008a respectively. In order to assess the influence of the registration step and the features used on the classification accuracies, these latter methods were tested with two different registration steps: SPM5 (Ashburner and Friston, 2005) and DARTEL (Ashburner, 2007) and also with either only the grey matter (GM) probability maps or all the tissues probability maps including also white matter (WM) and cerebrospinal fluid (CSF). Three cortical approaches were evaluated as well: a direct one similar to (Klöppel et al., 2008), an atlas based one and an approach using only the regions found in (Desikan et al., 2009). Two methods respectively based on the volume (Colliot et al., 2008, Chupin et al., 2009a, Chupin et al., 2009b) and the shape (Gerardin et al., 2009) of the hippocampus were also tested.
Section snippets
Data
Data used in the preparation of this article were obtained from the Alzheimer's disease Neuroimaging Initiative (ADNI) database (http://www.loni.ucla.edu/ADNI). The ADNI was launched in 2003 by the National Institute on Aging (NIA), the National Institute of Biomedical Imaging and Bioengineering (NIBIB), the Food and Drug Administration (FDA), private pharmaceutical companies and non-profit organizations, as a $60 million, 5-year public–private partnership. The primary goal of ADNI has been to
Classification experiments
Three classification experiments were performed to compare the different approaches. The first one is the classification between CN subjects and patients with probable AD and is referred to as “CN vs AD” in the following. The second one is the classification between CN subjects and MCI converters and is referred to as “CN vs MCIc”. It corresponds to the detection of patients with prodromal AD as defined by Dubois and Albert (2004). Indeed, MCI patients who will convert to AD are, at baseline,
Classification results
The results of the classification experiments are summarized in Table 4, Table 5, Table 6 respectively for CN vs AD, CN vs MCIc and CN vs MCInc. The classification results of CN vs AD and CN vs MCIc are also represented in Fig. 1. In each table, the different methods are referred to either by their abbreviation or by their number defined in Table 2.
Discussion
In this paper, we compared different methods for the classification of patients with AD and MCI based on anatomical T1-weighted MRI. To evaluate and compare the performances of each method, three classification experiments were performed: CN vs AD, CN vs MCIc and CN vs MCInc. The set of participants was randomly split up into two groups of the same size: a training set and a testing set. For each approach, the optimal parameter values had been determined using a grid-search and LOOCV on the
Acknowledgments
Data collection and sharing for this project was funded by the Alzheimer's Disease Neuroimaging Initiative (ADNI; Principal Investigator: Michael Weiner; NIH grant U01 AG024904). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering (NIBIB), and through generous contributions from the following: Pfizer Inc., Wyeth Research, Bristol-Myers Squibb, Eli Lilly and Company, GlaxoSmithKline, Merck & Co. Inc., AstraZeneca AB, Novartis
References (91)
- et al.
Unified segmentation
Neuroimage
(2005) A fast diffeomorphic image registration algorithm
Neuroimage
(2007)- et al.
Can voxel based morphometry, manual segmentation and automated segmentation equally detect hippocampal volume differences in acute depression?
Neuroimage
(2009) - et al.
Alzheimer's disease
Lancet
(2006) - et al.
A voxel-based morphometry study of temporal lobe gray matter reductions in Alzheimer's disease
Neurobiol. Aging
(2003) - et al.
Early diagnosis of Alzheimer's disease: contribution of structural neuroimaging
Neuroimage
(2003) - et al.
Using voxel-based morphometry to map the structural changes associated with rapid conversion in MCI: a longitudinal MRI study
Neuroimage
(2005) - et al.
Automated segmentation of the hippocampus and the amygdala driven by competition and anatomical priors: Method and validation on healthy subjects and patients with Alzheimer's disease
Neuroimage
(2007) - et al.
Automatic segmentation of the hippocampus and the amygdala driven by hybrid constraints: method and validation
Neuroimage
(2009) - et al.
Specific hippocampal volume reductions in individuals at risk for Alzheimer's disease
Neurobiol. Aging
(1997)
Atrophy of the medial occipitotemporal, inferior, and middle temporal gyri in non-demented elderly predict decline to Alzheimer's disease
Neurobiol. Aging
Cortical surface-based analysis. I. Segmentation and surface reconstruction
Neuroimage
Detection of prodromal Alzheimer's disease via pattern classification of magnetic resonance imaging
Neurobiol. Aging
Individual patient diagnosis of AD and FTD via high-dimensional pattern classification of MRI
Neuroimage
An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest
Neuroimage
Amnestic MCI or prodromal Alzheimer's disease?
Lancet Neurol.
Research criteria for the diagnosis of Alzheimer's disease: revising the NINCDS-ADRDA criteria
Lancet Neurol.
Classification of structural images via high-dimensional image warping, robust feature extraction, and SVM
Spatial patterns of brain atrophy in MCI patients, identified via high-dimensional pattern classification, predict subsequent cognitive decline
Neuroimage
Structural and functional biomarkers of prodromal Alzheimer's disease: a high-dimensional pattern classification study
Neuroimage
Cortical surface-based analysis. II: Inflation, flattening, and a surface based coordinate system
Neuroimage
Imaging cerebral atrophy: normal ageing to Alzheimer's disease
Lancet
Multidimensional classification of hippocampal shape features discriminates Alzheimer's disease and mild cognitive impairment from normal aging
Neuroimage
A voxel-based morphometric study of ageing in 465 normal adult human brains
Neuroimage
Automatic differentiation of anatomical patterns in the human brain: validation with studies of degenerative dementias
Neuroimage
Spatially augmented LP boosting for AD classification with evaluations on the ADNI dataset
Neuroimage
Optimizing power to track brain degeneration in Alzheimer's disease and mild cognitive impairment with tensor-based morphometry: an ADNI study of 515 subjects
Neuroimage
Positron emission tomography and magnetic resonance imaging in the diagnosis and prediction of dementia
Alzheimers Dement.
Reliability in multi-site structural MRI studies: effects of gradient non-linearity correction on phantom and human data
Neuroimage
Volumes of the entorhinal and perirhinal cortices in Alzheimer's disease
Neurobiol. Aging
A comprehensive study of gray matter loss in patients with Alzheimer's disease using optimized voxel-based morphometry
Neuroimage
Global and local gray matter loss in mild cognitive impairment and Alzheimer's disease
Neuroimage
Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration
Neuroimage
MRI of the hippocampus in Alzheimer's disease: sensitivity, specificity, and analysis of the incorrectly classified subjects
Neurobiol. Aging
Hippocampus and entorhinal cortex in frontotemporal dementia and Alzheimer's disease: a morphometric MRI study
Biol. Psychiatry
Morphological classification of brains via high-dimensional shape transformations and machine learning methods
Neuroimage
Automated cortical thickness measurements from MRI can accurately separate Alzheimer's patients from normal elderly controls
Neurobiol. Aging
Baseline and longitudinal patterns of brain atrophy in MCI patients, and their use in prediction of short-term conversion to AD: results from ADNI
Neuroimage
Classifying brain states and determining the discriminating activation patterns: support vector machine on functional MRI data
Neuroimage
Compensation for surface coil sensitivity variation in magnetic resonance imaging
Magn. Reson. Imaging
Age-associated alterations in cortical gray and white matter signal intensity and gray to white matter contrast
Neuroimage
M. Resonance Image Tissue Classification Using a Partial Volume Model
Neuroimage
MRI of hippocampus and entorhinal cortex in mild cognitive impairment: a follow-up study
Neurobiol. Aging
Mapping cortical change in Alzheimer's disease, brain development, and schizophrenia
Neuroimage
Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain
Neuroimage
Cited by (0)
- 1
Data used in the preparation of this article were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (http://www.loni.ucla.edu/ADNI). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. ADNI investigators include (complete listing available at http://www.loni.ucla.edu/ADNI/Collaboration/ADNI Author ship list.pdf).