Elsevier

NeuroImage

Volume 32, Issue 4, 1 October 2006, Pages 1656-1668
NeuroImage

Reducing inter-scanner variability of activation in a multicenter fMRI study: Role of smoothness equalization

https://doi.org/10.1016/j.neuroimage.2006.03.062Get rights and content

Abstract

Scanner-to-scanner variability of activation in multicenter fMRI studies is often considered undesirable. The purpose of this investigation was to evaluate the effect of a new procedure, “smoothness equalization”, on reducing scanner differences in activation effect size as part of a multicenter fMRI project (FIRST BIRN). Five subjects were sent to 9 centers (10 scanners) and scanned on 2 consecutive days using a sensorimotor fMRI protocol. High-field (4 T and 3 T) and low-field (1.5 T) scanners from three vendors (GE, Siemens, and Picker) were included. The activation effect size of the scanners for the detection of neural activation during a sensorimotor task was evaluated as the percent of temporal variance accounted for by our model (percent of variance accounted for or PVAF). Marked scanner effects were noted for both PVAF as well as the degree of smoothness of the raw and processed images. After smoothness equalization, there was a dramatic (low field) or consistent (high-field) reduction in scanner-to-scanner variation of activation. It was shown that the likely basis of the scanner differences in smoothness was differences in k-space filtering algorithms. This work highlights the need to account for differences in smoothness when comparing scanners on activation effect size in multicenter fMRI studies.

Introduction

Multicenter brain imaging studies have a number of advantages and are becoming increasingly common. The most obvious advantage of a multicenter study is the ability to accumulate large samples of subjects from potentially diverse demographic distributions. This is useful when studying rare diseases, but even when this is not the case, a large group of subjects can be especially useful to increase the statistical power needed for more sophisticated multivariate statistical analyses (e.g., structural equation modeling) or data mining. A large sample of subjects can also be used to elucidate and confirm small, but important, effects. Another way in which multicenter studies can be used is to study scanner differences that are of interest and importance. Such scanner differences can suggest explanations for disparate published results and can thus enlighten researchers as to the technical factors that affect their study. Finally, multicenter studies tend to engender the interaction and collaboration of scientists with different views. Discussions in the planning and execution phases of multicenter studies can serve to bridge differences and to promote the development of consensus views. For these reasons, there have been a number of recent multicenter brain imaging studies in several domains: structural brain imaging (de Certaines et al., 1993, Van Haren et al., 2003, Schnack et al., 2004), magnetic resonance spectroscopy (Paley et al., 1996), magnetization transfer (Silver et al., 1999, Berry et al., 1999) and fMRI (Casey et al., 1998).

The Biomedical Informatics Research Network (BIRN) is a National Center for Research Resources initiative that fosters distributed collaborations in biomedical science by utilizing information technology innovations (http://www.nbirn.net). The BIRN currently consists of three “test bed” projects that are conducting structural and functional studies of brain disease: The FIRST BIRN (Functional Imaging Research Schizophrenia Testbed BIRN), or fBIRN testbed, from which the present report emanates, uses multicenter fMRI to study regional brain dysfunction related to the progression and treatment of schizophrenia. The Morphometry BIRN testbed uses sophisticated structural MR techniques and tools to examine unipolar depression, mild Alzheimer's disease and mild cognitive impairment. The Mouse BIRN project studies animal models of multiple sclerosis, schizophrenia, Parkinson's disease, ADHD, Tourette's disorder, and brain cancer.

The key challenge for multicenter imaging studies stems from the fact that centers differ in numerous ways, and some of these differences can have powerful effects on the imaging results. There are two global strategies to deal with variance due to scanner: treat it as random variance, or try to understand and reduce it. In this investigation, the later approach is emphasized. The initial goal of the present effort was to evaluate scanner differences in the sensitivity to the BOLD effect in fMRI. The measure of sensitivity we used was the activation effect size, explained below. As it became clear that there were important differences in activation effect size that were not simply a reflection of field strength, we began to look for other factors that might explain such differences.

In our analyses of these multiscanner data, in addition to analysis of activation effect size, we emphasized the analysis of spatial smoothness. Increased spatial smoothness increases temporal SNR in fMRI and can greatly increase the ability to detect BOLD signal changes (Parrish et al., 2000), but this increase in SNR comes at the price of decreased spatial resolution. The effect is quite strong, even when the image smoothness results from varying degrees of k-space filtering (Lowe and Sorenson, 1997). Furthermore, Strother and colleagues have repeatedly shown that smoothing fMRI data can markedly enhance the reproducibility of fMRI results collected from a single scanner (LaConte et al., 2003, Strother et al., 2004). Since the usefulness of the multicenter strategy depends on the reproducibility of fMRI results across scanners, it may be important to pay close attention to variations in spatial smoothness produced by scanners with different field strengths and from different vendors. It is paramount to keep in mind throughout our report that when we say “increased smoothness”, this can be rephrased as “decreased spatial resolution” and vice versa.

A previous report on this data has been published by Zou et al. (2005). Zou et al. (2005) assessed the effect of several factors on the reproducibility of fMRI activations, using a novel method to develop a gold standard activation map. These authors reported that 3 T scanners had higher CNR than 1.5 T scanners and also that results from 3 T scanners were more reproducible than results from 1.5 T scanners. The present study is focused more on understanding the role that smoothness plays in explaining scanner differences activation effect size.

In the present study, we compare the 10 FIRST-BIRN scanners (Table 1) on activation effect size and spatial smoothness. Both high (3 T and 4 T) and low-field (1.5 T) scanners were represented, as well as scanners from Siemens (Siemens Medical Solutions of Siemens AG, Malvern, Pennsylvania, USA), GE (GE Healthcare Technologies, Waukesha, Wisconsin, USA) and Picker (Royal Philips Electronics, Amsterdam, The Netherlands). The fBIRN Phase I “Human Phantom” study, in which 5 subjects were scanned on all 10 scanners, provided a unique opportunity to compare scanners on activation effect size and image smoothness. The results illustrate marked scanner differences and suggest methods to reduce these differences and thus enhance multicenter reproducibility.

Section snippets

Subjects

Five healthy, English-speaking males (mean age: 25.2, range = 20.2 to 29) participated in this study. All were right-handed, had no history of psychiatric or neurological illnesses and had normal hearing in both ears. Each subject traveled to 9 sites (10 scanners) (Table 1), where they were scanned twice over a period of 2 days for a total of 20 scans per participant. There were no missing data, i.e., all 100 scans (5 subjects × 2 Visits × 10 scanners) were available for analysis. All subjects were

Scanner effects

The average FWHM for each scanner is shown in Fig. 2. The scanner effect was highly significant (F(scanner) = 111.0, df = 9,36, p < 0.0001). The smoothest scanner (D40T) was 1.44 times more smooth than the least smooth scanner (UCSD). This comparison contrasted a GE scanner running a spiral-out acquisition at 4.0 T to a Siemens 1.5 T running a conventional EPI sequence. Smoothness differences were somewhat greater among low-field scanners than among high-field scanners. The coefficient of variation

Discussion

This large multicenter study, in which the same five healthy normal subjects traveled to 9 sites and were evaluated with fMRI on 10 scanners, has provided an unprecedented opportunity to evaluate the influence of scanner differences on fMRI results. We report marked, highly statistically significant scanner differences in smoothness and activation effect size. We also show that the smoothness of the images produced by different scanners does have an important relationship to activation effect

Acknowledgments

This research was supported by a grant [#5 MOI RR 000827] to the FIRST Biomedical Informatics Research Network (BIRN, http://www.nbirn.net), that is funded by the National Center for Research Resources (NCRR) at the National Institutes of Health (NIH).

The members of the FIRST BIRN project all deserve acknowledgement for their significant efforts, but unfortunately, they are too numerous to mention. Please visit http://www.nbirn.net for more information regarding key personnel. We would identify

References (30)

  • N.E.M. Van Haren et al.

    Brain volumes as predictor of outcome in recent-onset schizophrenia: a multi-center MRI study

    Schizophr. Res.

    (2003)
  • K.J. Worsley et al.

    A general statistical analysis for fMRI data

    NeuroImage

    (2002)
  • Y. Yang et al.

    Comparison of 3D BOLD functional MRI with spiral acquisition at 1.5 T and 4.0 T

    NeuroImage

    (1999)
  • P. Bandettini et al.

    MRI of human brain activation at 0.5 T, 1.5 T, and 3.0 T: comparisons of R2* and functional contrast to noise ratio [oral]

  • I. Berry et al.

    A multi-center measurement of magnetization transfer ratio in normal white matter

    J. Magn. Reson. Imaging

    (1999)
  • Cited by (135)

    View all citing articles on Scopus
    1

    Functional Imaging Research Schizophrenia Testbed-Biomedical Informatics Research Network, http://www.nbirn.net, NIH-NCRR, Bethesda, MD 87131, USA

    View full text