Reducing inter-scanner variability of activation in a multicenter fMRI study: Role of smoothness equalization
Introduction
Multicenter brain imaging studies have a number of advantages and are becoming increasingly common. The most obvious advantage of a multicenter study is the ability to accumulate large samples of subjects from potentially diverse demographic distributions. This is useful when studying rare diseases, but even when this is not the case, a large group of subjects can be especially useful to increase the statistical power needed for more sophisticated multivariate statistical analyses (e.g., structural equation modeling) or data mining. A large sample of subjects can also be used to elucidate and confirm small, but important, effects. Another way in which multicenter studies can be used is to study scanner differences that are of interest and importance. Such scanner differences can suggest explanations for disparate published results and can thus enlighten researchers as to the technical factors that affect their study. Finally, multicenter studies tend to engender the interaction and collaboration of scientists with different views. Discussions in the planning and execution phases of multicenter studies can serve to bridge differences and to promote the development of consensus views. For these reasons, there have been a number of recent multicenter brain imaging studies in several domains: structural brain imaging (de Certaines et al., 1993, Van Haren et al., 2003, Schnack et al., 2004), magnetic resonance spectroscopy (Paley et al., 1996), magnetization transfer (Silver et al., 1999, Berry et al., 1999) and fMRI (Casey et al., 1998).
The Biomedical Informatics Research Network (BIRN) is a National Center for Research Resources initiative that fosters distributed collaborations in biomedical science by utilizing information technology innovations (http://www.nbirn.net). The BIRN currently consists of three “test bed” projects that are conducting structural and functional studies of brain disease: The FIRST BIRN (Functional Imaging Research Schizophrenia Testbed BIRN), or fBIRN testbed, from which the present report emanates, uses multicenter fMRI to study regional brain dysfunction related to the progression and treatment of schizophrenia. The Morphometry BIRN testbed uses sophisticated structural MR techniques and tools to examine unipolar depression, mild Alzheimer's disease and mild cognitive impairment. The Mouse BIRN project studies animal models of multiple sclerosis, schizophrenia, Parkinson's disease, ADHD, Tourette's disorder, and brain cancer.
The key challenge for multicenter imaging studies stems from the fact that centers differ in numerous ways, and some of these differences can have powerful effects on the imaging results. There are two global strategies to deal with variance due to scanner: treat it as random variance, or try to understand and reduce it. In this investigation, the later approach is emphasized. The initial goal of the present effort was to evaluate scanner differences in the sensitivity to the BOLD effect in fMRI. The measure of sensitivity we used was the activation effect size, explained below. As it became clear that there were important differences in activation effect size that were not simply a reflection of field strength, we began to look for other factors that might explain such differences.
In our analyses of these multiscanner data, in addition to analysis of activation effect size, we emphasized the analysis of spatial smoothness. Increased spatial smoothness increases temporal SNR in fMRI and can greatly increase the ability to detect BOLD signal changes (Parrish et al., 2000), but this increase in SNR comes at the price of decreased spatial resolution. The effect is quite strong, even when the image smoothness results from varying degrees of k-space filtering (Lowe and Sorenson, 1997). Furthermore, Strother and colleagues have repeatedly shown that smoothing fMRI data can markedly enhance the reproducibility of fMRI results collected from a single scanner (LaConte et al., 2003, Strother et al., 2004). Since the usefulness of the multicenter strategy depends on the reproducibility of fMRI results across scanners, it may be important to pay close attention to variations in spatial smoothness produced by scanners with different field strengths and from different vendors. It is paramount to keep in mind throughout our report that when we say “increased smoothness”, this can be rephrased as “decreased spatial resolution” and vice versa.
A previous report on this data has been published by Zou et al. (2005). Zou et al. (2005) assessed the effect of several factors on the reproducibility of fMRI activations, using a novel method to develop a gold standard activation map. These authors reported that 3 T scanners had higher CNR than 1.5 T scanners and also that results from 3 T scanners were more reproducible than results from 1.5 T scanners. The present study is focused more on understanding the role that smoothness plays in explaining scanner differences activation effect size.
In the present study, we compare the 10 FIRST-BIRN scanners (Table 1) on activation effect size and spatial smoothness. Both high (3 T and 4 T) and low-field (1.5 T) scanners were represented, as well as scanners from Siemens (Siemens Medical Solutions of Siemens AG, Malvern, Pennsylvania, USA), GE (GE Healthcare Technologies, Waukesha, Wisconsin, USA) and Picker (Royal Philips Electronics, Amsterdam, The Netherlands). The fBIRN Phase I “Human Phantom” study, in which 5 subjects were scanned on all 10 scanners, provided a unique opportunity to compare scanners on activation effect size and image smoothness. The results illustrate marked scanner differences and suggest methods to reduce these differences and thus enhance multicenter reproducibility.
Section snippets
Subjects
Five healthy, English-speaking males (mean age: 25.2, range = 20.2 to 29) participated in this study. All were right-handed, had no history of psychiatric or neurological illnesses and had normal hearing in both ears. Each subject traveled to 9 sites (10 scanners) (Table 1), where they were scanned twice over a period of 2 days for a total of 20 scans per participant. There were no missing data, i.e., all 100 scans (5 subjects × 2 Visits × 10 scanners) were available for analysis. All subjects were
Scanner effects
The average FWHM for each scanner is shown in Fig. 2. The scanner effect was highly significant (F(scanner) = 111.0, df = 9,36, p < 0.0001). The smoothest scanner (D40T) was 1.44 times more smooth than the least smooth scanner (UCSD). This comparison contrasted a GE scanner running a spiral-out acquisition at 4.0 T to a Siemens 1.5 T running a conventional EPI sequence. Smoothness differences were somewhat greater among low-field scanners than among high-field scanners. The coefficient of variation
Discussion
This large multicenter study, in which the same five healthy normal subjects traveled to 9 sites and were evaluated with fMRI on 10 scanners, has provided an unprecedented opportunity to evaluate the influence of scanner differences on fMRI results. We report marked, highly statistically significant scanner differences in smoothness and activation effect size. We also show that the smoothness of the images produced by different scanners does have an important relationship to activation effect
Acknowledgments
This research was supported by a grant [#5 MOI RR 000827] to the FIRST Biomedical Informatics Research Network (BIRN, http://www.nbirn.net), that is funded by the National Center for Research Resources (NCRR) at the National Institutes of Health (NIH).
The members of the FIRST BIRN project all deserve acknowledgement for their significant efforts, but unfortunately, they are too numerous to mention. Please visit http://www.nbirn.net for more information regarding key personnel. We would identify
References (30)
- et al.
Reproducibility of fMRI results across four institutions using a spatial memory task
NeuroImage
(1998) Parametric analysis of fMRI data using linear systems methods
NeuroImage
(1997)AFNI: software for analysis and visualization of functional magnetic resonance neuroimages
Comput. Biomed. Res.
(1996)- et al.
In vivo measurements of proton relaxation times in human brain, liver, and skeletal muscle: a multicenter MRI study
Magn. Reson. Imaging
(1993) - et al.
Event-related FMRI: characterizing differential responses
Neuro-Image
(1998) - et al.
Robust estimation in statistical parametric maps using standardized residuals from the general linear model
NeuroImage
(1999) - et al.
Comparison of fMRI activation at 3 and 1.5 T during perceptual cognitive, and affective processing
NeuroImage
(2003) - et al.
The evaluation of preprocessing choices in single-subject BOLD fMRI using NPAIRS performance metrics
NeuroImage
(2003) - et al.
Estimating the delay of the fMRI response
NeuroImage
(2002) - et al.
Optimizing the fMRI data-processing pipeline using prediction and reproducibility performance metrics: I. A preliminary group analysis
NeuroImage
(2004)
Brain volumes as predictor of outcome in recent-onset schizophrenia: a multi-center MRI study
Schizophr. Res.
A general statistical analysis for fMRI data
NeuroImage
Comparison of 3D BOLD functional MRI with spiral acquisition at 1.5 T and 4.0 T
NeuroImage
MRI of human brain activation at 0.5 T, 1.5 T, and 3.0 T: comparisons of R2* and functional contrast to noise ratio [oral]
A multi-center measurement of magnetization transfer ratio in normal white matter
J. Magn. Reson. Imaging
Cited by (135)
Sometimes I feel the fear of uncertainty: How intolerance of uncertainty and trait anxiety impact fear acquisition, extinction and the return of fear
2022, International Journal of PsychophysiologyThe Open-Access European Prevention of Alzheimer's Dementia (EPAD) MRI dataset and processing workflow
2022, NeuroImage: Clinical
- 1
Functional Imaging Research Schizophrenia Testbed-Biomedical Informatics Research Network, http://www.nbirn.net, NIH-NCRR, Bethesda, MD 87131, USA