Elsevier

NeuroImage

Volume 56, Issue 2, 15 May 2011, Pages 766-781
NeuroImage

Supplement
Automatic classification of patients with Alzheimer's disease from structural MRI: A comparison of ten methods using the ADNI database

https://doi.org/10.1016/j.neuroimage.2010.06.013Get rights and content

Abstract

Recently, several high dimensional classification methods have been proposed to automatically discriminate between patients with Alzheimer's disease (AD) or mild cognitive impairment (MCI) and elderly controls (CN) based on T1-weighted MRI. However, these methods were assessed on different populations, making it difficult to compare their performance. In this paper, we evaluated the performance of ten approaches (five voxel-based methods, three methods based on cortical thickness and two methods based on the hippocampus) using 509 subjects from the ADNI database. Three classification experiments were performed: CN vs AD, CN vs MCIc (MCI who had converted to AD within 18 months, MCI converters — MCIc) and MCIc vs MCInc (MCI who had not converted to AD within 18 months, MCI non-converters — MCInc). Data from 81 CN, 67 MCInc, 39 MCIc and 69 AD were used for training and hyperparameters optimization. The remaining independent samples of 81 CN, 67 MCInc, 37 MCIc and 68 AD were used to obtain an unbiased estimate of the performance of the methods. For AD vs CN, whole-brain methods (voxel-based or cortical thickness-based) achieved high accuracies (up to 81% sensitivity and 95% specificity). For the detection of prodromal AD (CN vs MCIc), the sensitivity was substantially lower. For the prediction of conversion, no classifier obtained significantly better results than chance. We also compared the results obtained using the DARTEL registration to that using SPM5 unified segmentation. DARTEL significantly improved six out of 20 classification experiments and led to lower results in only two cases. Overall, the use of feature selection did not improve the performance but substantially increased the computation times.

Research Highlights

► Alzheimer vs controls : high accuracies with whole-brain methods (up to 81% sensitivity - 95% specificity). ► For the detection of prodromal Alzheimer, the sensitivity was substantially lower. ► For the prediction of conversion, the accuracy was only slightly higher than chance.

Introduction

Alzheimer's disease (AD) is the most frequent neurodegenerative dementia and a growing health problem. Definite diagnosis can only be made postmortem, and requires histopathological confirmation of amyloid plaques and neurofibrillary tangles. Early and accurate diagnosis of Alzheimer's Disease (AD) is not only challenging, but is crucial in the perspective of future treatments. Clinical diagnostic criteria are currently based on the clinical examination and neuropsychological assessment, with the identification of dementia and then of the Alzheimer's phenotype (Blennow et al., 2006). Patients suffering from AD at a prodromal stage are, mostly, clinically classified as amnestic mild cognitive impairment (MCI) (Petersen et al., 1999, Dubois and Albert, 2004), but not all patients with amnestic MCI will develop AD. Recently, more precise research criteria were proposed for the early diagnostic of AD at the prodromal stage of the disease (Dubois et al., 2007). These criteria are based on a clinical core of early episodic memory impairment and the presence of at least one additional supportive feature including abnormal MRI and PET neuroimaging or abnormal cerebrospinal fluid amyloid and tau biomarkers (Dubois et al., 2007). Neuroimaging therefore adds a positive predictive value to the diagnosis and includes measurements using structural MRI to assess medial temporal lobe atrophy and positron emission tomography using fluorodeoxyglucose (FDG) or amyloid markers (Fox and Schott, 2004, Jagust, 2006).

Many group studies based on volumetric measurements of regions of interest (ROI) (Convit et al., 1997, Convit et al., 2000, Jack et al., 1997, Jack et al., 1998, Juottonen et al., 1998, Laakso et al., 1998, Laakso et al., 2000, Busatto et al., 2003, Xu et al., 2000, Good et al., 2002, Chételat and Baron, 2003, Rusinek et al., 2004, Tapiola et al., 2008), voxel-based morphometry (Good et al., 2002, Busatto et al., 2003, Karas et al., 2003, Karas et al., 2004, Chételat et al., 2005, Whitwell et al., 2007, Whitwell et al., 2008) or group comparison of cortical thickness (Thompson et al., 2001, Thompson et al., 2003, Thompson et al., 2004, Lerch et al., 2005, Lerch et al., 2008, Bakkour et al., 2009, Dickerson et al., 2009, Hua et al., 2009, McDonald et al., 2009) have shown that brain atrophy in AD and prodromal AD is spatially distributed over many brain regions including the entorhinal cortex, the hippocampus, lateral and inferior temporal structures, anterior and posterior cingulate. However these analyses measure group differences and thus are of limited value for individual diagnosis.

Advances in statistical learning with the development of new machine learning algorithms capable of dealing with high dimensional data, such as the support vector machine (SVM) (Vapnik, 1995, Shawe-Taylor and Cristianini, 2000, Schölkopf and Smola, 2001), enable the development of new diagnostic tools based on T1-weighted MRI. Recently, several approaches have been proposed to automatically classify patients with AD and/or MCI from anatomical MRI (Fan et al., 2005, Fan et al., 2007, Fan et al., 2008b, Fan et al., 2008a, Colliot et al., 2008, Davatzikos et al., 2008a, Davatzikos et al., 2008b, Klöppel et al., 2008, Vemuri et al., 2008, Chupin et al., 2009a, Chupin et al., 2009b, Desikan et al., 2009, Gerardin et al., 2009, Hinrichs et al., 2009, Magnin et al., 2009, Misra et al., 2009, Querbes et al., 2009). These approaches could have the potential to assist in the early diagnosis of AD. These approaches can roughly be grouped into three different categories, depending on the type of features extracted from the MRI (voxel-based, vertex-based or ROI-based). In the first category, the features are defined at the level of the MRI voxel. Specifically, the features are the probability of the different tissue classes (grey matter, white matter and cerebrospinal fluid) in a given voxel (Lao et al., 2004, Fan et al., 2007, Fan et al., 2008b, Fan et al., 2008a, Davatzikos et al., 2008a, Davatzikos et al., 2008b, Klöppel et al., 2008, Vemuri et al., 2008, Hinrichs et al., 2009, Magnin et al., 2009, Misra et al., 2009). Klöppel et al. (2008) directly classified these features with an SVM. All other methods first reduce the dimensionality of the feature space relying on different types of features extraction, agglomeration and/or selection methods. Vemuri et al. (2008) used smoothing, voxel-downsampling, and then a feature selection step. Another solution is to group voxels into anatomical regions through the registration of a labeled atlas (Lao et al., 2004, Ye et al., 2008, Magnin et al., 2009). However, this anatomical parcellation may not be adapted to the pathology. In order to overcome this limitation, Fan et al. (2007) have proposed an adaptive parcellation approach in which the image space is divided into the most discriminative regions. This method has been used in several studies (Davatzikos et al., 2008a, Davatzikos et al., 2008b, Fan et al., 2008b, Fan et al., 2008a, Misra et al., 2009). In the second category, the features are defined at the vertex-level on the cortical surface (Desikan et al., 2009, Querbes et al., 2009). The methods of the third category include only the hippocampus. Their approach is based on the analysis of the volume and/or shape of the hippocampus (Colliot et al., 2008, Chupin et al., 2009a, Chupin et al., 2009b, Gerardin et al., 2009).

These approaches achieve high accuracy (over 84%). However, they were evaluated on different study populations, making it difficult to compare their respective discriminative power. Indeed, many factors such as degree of impairment, age, gender, genotype, educational level and MR image quality perceptibly affect the evaluation of the prediction accuracy. This variability between evaluations is increased for statistical reasons when the number of subjects is small. Therefore a meta-analysis would be of limited value to compare the prediction accuracies of different methods.

The goal of this paper was to compare different methods for the classification of patients with AD based on anatomical MRI, using the same study population. To that purpose, we used the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. Ten methods were evaluated. We tested five voxel-based approaches: a direct approach (Klöppel et al., 2008), an approach based on a volume of interest (Klöppel et al., 2008), an atlas-based approach (Magnin et al, 2009) and the approaches proposed by Vemuri et al., 2008, Fan et al., 2008b, Fan et al., 2008a respectively. In order to assess the influence of the registration step and the features used on the classification accuracies, these latter methods were tested with two different registration steps: SPM5 (Ashburner and Friston, 2005) and DARTEL (Ashburner, 2007) and also with either only the grey matter (GM) probability maps or all the tissues probability maps including also white matter (WM) and cerebrospinal fluid (CSF). Three cortical approaches were evaluated as well: a direct one similar to (Klöppel et al., 2008), an atlas based one and an approach using only the regions found in (Desikan et al., 2009). Two methods respectively based on the volume (Colliot et al., 2008, Chupin et al., 2009a, Chupin et al., 2009b) and the shape (Gerardin et al., 2009) of the hippocampus were also tested.

Section snippets

Data

Data used in the preparation of this article were obtained from the Alzheimer's disease Neuroimaging Initiative (ADNI) database (http://www.loni.ucla.edu/ADNI). The ADNI was launched in 2003 by the National Institute on Aging (NIA), the National Institute of Biomedical Imaging and Bioengineering (NIBIB), the Food and Drug Administration (FDA), private pharmaceutical companies and non-profit organizations, as a $60 million, 5-year public–private partnership. The primary goal of ADNI has been to

Classification experiments

Three classification experiments were performed to compare the different approaches. The first one is the classification between CN subjects and patients with probable AD and is referred to as “CN vs AD” in the following. The second one is the classification between CN subjects and MCI converters and is referred to as “CN vs MCIc”. It corresponds to the detection of patients with prodromal AD as defined by Dubois and Albert (2004). Indeed, MCI patients who will convert to AD are, at baseline,

Classification results

The results of the classification experiments are summarized in Table 4, Table 5, Table 6 respectively for CN vs AD, CN vs MCIc and CN vs MCInc. The classification results of CN vs AD and CN vs MCIc are also represented in Fig. 1. In each table, the different methods are referred to either by their abbreviation or by their number defined in Table 2.

Discussion

In this paper, we compared different methods for the classification of patients with AD and MCI based on anatomical T1-weighted MRI. To evaluate and compare the performances of each method, three classification experiments were performed: CN vs AD, CN vs MCIc and CN vs MCInc. The set of participants was randomly split up into two groups of the same size: a training set and a testing set. For each approach, the optimal parameter values had been determined using a grid-search and LOOCV on the

Acknowledgments

Data collection and sharing for this project was funded by the Alzheimer's Disease Neuroimaging Initiative (ADNI; Principal Investigator: Michael Weiner; NIH grant U01 AG024904). ADNI is funded by the National Institute on Aging, the National Institute of Biomedical Imaging and Bioengineering (NIBIB), and through generous contributions from the following: Pfizer Inc., Wyeth Research, Bristol-Myers Squibb, Eli Lilly and Company, GlaxoSmithKline, Merck & Co. Inc., AstraZeneca AB, Novartis

References (91)

  • A. Convit et al.

    Atrophy of the medial occipitotemporal, inferior, and middle temporal gyri in non-demented elderly predict decline to Alzheimer's disease

    Neurobiol. Aging

    (2000)
  • A.M. Dale et al.

    Cortical surface-based analysis. I. Segmentation and surface reconstruction

    Neuroimage

    (1999)
  • C. Davatzikos et al.

    Detection of prodromal Alzheimer's disease via pattern classification of magnetic resonance imaging

    Neurobiol. Aging

    (2008)
  • C. Davatzikos et al.

    Individual patient diagnosis of AD and FTD via high-dimensional pattern classification of MRI

    Neuroimage

    (2008)
  • R.S. Desikan et al.

    An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest

    Neuroimage

    (2006)
  • B. Dubois et al.

    Amnestic MCI or prodromal Alzheimer's disease?

    Lancet Neurol.

    (2004)
  • B. Dubois et al.

    Research criteria for the diagnosis of Alzheimer's disease: revising the NINCDS-ADRDA criteria

    Lancet Neurol.

    (2007)
  • Y. Fan et al.

    Classification of structural images via high-dimensional image warping, robust feature extraction, and SVM

  • Y. Fan et al.

    Spatial patterns of brain atrophy in MCI patients, identified via high-dimensional pattern classification, predict subsequent cognitive decline

    Neuroimage

    (2008)
  • Y. Fan et al.

    Structural and functional biomarkers of prodromal Alzheimer's disease: a high-dimensional pattern classification study

    Neuroimage

    (2008)
  • B. Fischl et al.

    Cortical surface-based analysis. II: Inflation, flattening, and a surface based coordinate system

    Neuroimage

    (1999)
  • N.C. Fox et al.

    Imaging cerebral atrophy: normal ageing to Alzheimer's disease

    Lancet

    (2004)
  • E. Gerardin et al.

    Multidimensional classification of hippocampal shape features discriminates Alzheimer's disease and mild cognitive impairment from normal aging

    Neuroimage

    (2009)
  • C.D. Good et al.

    A voxel-based morphometric study of ageing in 465 normal adult human brains

    Neuroimage

    (2001)
  • C.D. Good et al.

    Automatic differentiation of anatomical patterns in the human brain: validation with studies of degenerative dementias

    Neuroimage

    (2002)
  • C. Hinrichs et al.

    Spatially augmented LP boosting for AD classification with evaluations on the ADNI dataset

    Neuroimage

    (2009)
  • X. Hua et al.

    Optimizing power to track brain degeneration in Alzheimer's disease and mild cognitive impairment with tensor-based morphometry: an ADNI study of 515 subjects

    Neuroimage

    (2009)
  • W. Jagust

    Positron emission tomography and magnetic resonance imaging in the diagnosis and prediction of dementia

    Alzheimers Dement.

    (2006)
  • J. Jovicich et al.

    Reliability in multi-site structural MRI studies: effects of gradient non-linearity correction on phantom and human data

    Neuroimage

    (2006)
  • K. Juottonen et al.

    Volumes of the entorhinal and perirhinal cortices in Alzheimer's disease

    Neurobiol. Aging

    (1998)
  • G.B. Karas et al.

    A comprehensive study of gray matter loss in patients with Alzheimer's disease using optimized voxel-based morphometry

    Neuroimage

    (2003)
  • G.B. Karas et al.

    Global and local gray matter loss in mild cognitive impairment and Alzheimer's disease

    Neuroimage

    (2004)
  • A. Klein et al.

    Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration

    Neuroimage

    (2009)
  • M.P. Laakso et al.

    MRI of the hippocampus in Alzheimer's disease: sensitivity, specificity, and analysis of the incorrectly classified subjects

    Neurobiol. Aging

    (1998)
  • M.P. Laakso et al.

    Hippocampus and entorhinal cortex in frontotemporal dementia and Alzheimer's disease: a morphometric MRI study

    Biol. Psychiatry

    (2000)
  • Z. Lao et al.

    Morphological classification of brains via high-dimensional shape transformations and machine learning methods

    Neuroimage

    (2004)
  • J.P. Lerch et al.

    Automated cortical thickness measurements from MRI can accurately separate Alzheimer's patients from normal elderly controls

    Neurobiol. Aging

    (2008)
  • C. Misra et al.

    Baseline and longitudinal patterns of brain atrophy in MCI patients, and their use in prediction of short-term conversion to AD: results from ADNI

    Neuroimage

    (2009)
  • J. Mourao-Miranda et al.

    Classifying brain states and determining the discriminating activation patterns: support vector machine on functional MRI data

    Neuroimage

    (2005)
  • P.A. Narayana et al.

    Compensation for surface coil sensitivity variation in magnetic resonance imaging

    Magn. Reson. Imaging

    (1988)
  • D.H. Salat et al.

    Age-associated alterations in cortical gray and white matter signal intensity and gray to white matter contrast

    Neuroimage

    (2009)
  • D.W. Shattuck et al.

    M. Resonance Image Tissue Classification Using a Partial Volume Model

    Neuroimage

    (2001)
  • T. Tapiola et al.

    MRI of hippocampus and entorhinal cortex in mild cognitive impairment: a follow-up study

    Neurobiol. Aging

    (2008)
  • P.M. Thompson et al.

    Mapping cortical change in Alzheimer's disease, brain development, and schizophrenia

    Neuroimage

    (2004)
  • N. Tzourio-Mazoyer et al.

    Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain

    Neuroimage

    (2002)
  • Cited by (0)

    1

    Data used in the preparation of this article were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (http://www.loni.ucla.edu/ADNI). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in analysis or writing of this report. ADNI investigators include (complete listing available at http://www.loni.ucla.edu/ADNI/Collaboration/ADNI Author ship list.pdf).

    View full text