Elsevier

Medical Image Analysis

Volume 20, Issue 1, February 2015, Pages 135-151
Medical Image Analysis

Evaluation of automatic neonatal brain segmentation algorithms: The NeoBrainS12 challenge

https://doi.org/10.1016/j.media.2014.11.001Get rights and content

Highlights

  • Automatic algorithms for neonatal brain segmentation in MR images were compared.

  • Preterm infants underwent MRI at 30 and 40 weeks corrected gestational age.

  • Images were acquired axially and coronally with a 3T MRI scanner.

  • Automatic segmentation of brain tissues in neonatal brain MRI is feasible.

  • Automatic segmentation of myelinated white matter in these images is not reliable.

Abstract

A number of algorithms for brain segmentation in preterm born infants have been published, but a reliable comparison of their performance is lacking. The NeoBrainS12 study (http://neobrains12.isi.uu.nl), providing three different image sets of preterm born infants, was set up to provide such a comparison. These sets are (i) axial scans acquired at 40 weeks corrected age, (ii) coronal scans acquired at 30 weeks corrected age and (iii) coronal scans acquired at 40 weeks corrected age. Each of these three sets consists of three T1- and T2-weighted MR images of the brain acquired with a 3T MRI scanner. The task was to segment cortical grey matter, non-myelinated and myelinated white matter, brainstem, basal ganglia and thalami, cerebellum, and cerebrospinal fluid in the ventricles and in the extracerebral space separately. Any team could upload the results and all segmentations were evaluated in the same way. This paper presents the results of eight participating teams. The results demonstrate that the participating methods were able to segment all tissue classes well, except myelinated white matter.

Introduction

Brain development in neonates may be affected by preterm birth. It is known that intra-uterine growth restriction, inflammation and hypoxia–ischemia will impede brain development (Tolsa et al., 2004, Volpe, 2009). These factors may affect brain growth and thus may result in altered brain tissue volumes in preterm infants. Analyses of these volumes have identified several risk factors and have provided important information regarding impaired brain development (Thompson et al., 2006). Nowadays brain tissue volumes at term equivalent age (TEA) are considered predictive markers of long-term neurodevelopmental performance (Peterson et al., 2003, Inder et al., 2005).

Accurate brain tissue segmentation is a prerequisite for obtaining volumetric measurements. Thus far, a number of different algorithms have been presented but comparing their performance reliably has not proven feasible.

The algorithms have been evaluated with images made using a variety of acquisition protocols involving differences regarding magnetic field strength, scan orientation and spatial resolution. For example, several methods were applied to images acquired at 1.5T scanners (Cardoso et al., 2013, Anbeek et al., 2008, Weisenfeld and Warfield, 2009), while other used 3T scanners (Gui et al., 2012b, Prastawa et al., 2005, Xue et al., 2007), and segmentations were designed using axial (Anbeek et al., 2008) or coronal (Gui et al., 2012b, Cardoso et al., 2013) image acquisitions.

In addition, the algorithms were evaluated using different criteria. While the quality of the segmentations was generally evaluated using overlap measures, notably the Dice coefficient (DC), the definition of the evaluated tissue classes varied. Since obtaining a manually provided reference standard for brain tissue segmentations has proven extremely time-consuming and cumbersome, very few methods have been evaluated on complete scans; frequently the performance of methods was assessed on a limited subset of scan sections. In addition, the published algorithms have not consistently segmented the same tissue (sub-)types. For example, several algorithms distinguished between cortical and central grey matter (e.g. Gui et al., 2012b, Weisenfeld and Warfield, 2009, Anbeek et al., 2008), whereas others combined these structures into one segment (Prastawa et al., 2005). Similarly, several methods identified myelinated white matter (e.g. Weisenfeld and Warfield, 2009), while others did not (e.g. Anbeek et al., 2008).

The NeoBrainS12 study, providing three different image sets of preterm born infants, was set up to enable a reliable comparison of different brain tissue segmentation methods. These sets are (i) axial scans acquired at 40 weeks corrected age, (ii) coronal scans acquired at 30 weeks corrected age and (iii) coronal scans acquired at 40 weeks corrected age. Each of these three sets consists of three T1- and T2-weighted MR images of the brain acquired with a 3T MRI scanner. The task was to segment cortical grey matter (CoGM), non-myelinated white matter (UWM) and myelinated white matter (MWM), brainstem (BS), basal ganglia and thalami (BGT), cerebellum (CB), and cerebrospinal fluid in the ventricles (CSFVent) and in the extracerebral space (CSFEcS) separately. Any team could upload the results and all segmentations were evaluated in the same way. This paper presents the results of eight participating teams.

Section snippets

Data

The NeoBrainS12 data set consists of three different image sets of premature infants: (i) axial scans acquired at 40 weeks corrected gestational age, (ii) coronal scans acquired at 30 weeks corrected gestational age and (iii) coronal scans acquired at 40 weeks corrected gestational age. All scans were made as part of standard clinical practice at the neonatal intensive care unit of the University Medical Center Utrecht, The Netherlands. In these images, no brain pathology was visible. Furthermore,

Manual annotations

All eight tissue classes were manually delineated (CoGM, UWM, BS, CB, CSFVent, CSFEcS, BGT, and MWM). Each tissue class was segmented separately. Manual reference segmentations have been defined on T2-weighted images. They were generated using in-house developed software. Each voxel was assigned to only one of the eight tissue classes by mouse painting. The labeling was indicated by color overlay, where one color uniquely represented each tissue (Fig. 1). During manual segmentation, observers

Evaluation

Automatically obtained segmentations of each tissue were compared with the manually obtained reference standard. The automatic segmentations were evaluated using the Dice coefficient (DC), Hausdorff distance (HD) and mean surface distance (MSD), i.e. the mean distance between the automatic and reference segmentations along their boundaries. The DC, HD and MSD values of a tissue were computed as the average values obtained over the provided number of scans (patients) within a set.

In addition, an

Methods

Below, a brief description of the algorithms participating in the NeoBrainS12 challenge is given. Links to extensive descriptions of the methods are available in the Results section of the NeoBrainS12 webpage.1 Table 2 provides a list of the data segmented by each method, images used in segmentation, application of inhomogeneity correction and total number of segmented tissues.

Results and discussion

Teams were free to choose which set of images they would like to segment. Four methods, i.e. methods A, B, C and D, have been applied to all sets, while the remaining four methods (E, F, G and H) have been evaluated with only one of the sets. Similarly, five methods (methods B, C, D, F and G) have segmented all defined tissue classes, whereas method A segmented all tissues except MWM, method E segmented CSFEcS and CSFVent as one tissue class, and method H only segmented UWM, CoGM and CSFEcS.

Conclusions

Evaluation of eight methods for neonatal brain segmentation in cortical grey matter, non-myelinated and myelinated white matter, brainstem, basal ganglia and thalami, cerebellum, cerebrospinal fluid in the ventricles and cerebrospinal fluid in the extracerebral space on publicly available neonatal MRI scans has been presented. The methods were evaluated on axial scans acquired at 40 weeks corrected gestational age, and coronal scans acquired at 30 weeks and 40 weeks corrected gestational age. The

Acknowledgements

This study includes images of infants participating in the Neobrain study (LSHM-CT-2006-036534).

The authors from King’s College London would like to acknowledge the funding/support provided by the National Institute for Health Research (NIHR) Biomedical Research Centre based at Guy’s and St. Thomas’ NHS Foundation Trust and King’s College London. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.

The authors from University of

References (39)

  • P. Anbeek et al.

    Probabilistic brain tissue segmentation in neonatal magnetic resonance imaging

    Pediatr. Res.

    (2008)
  • J. Ashburner et al.

    Unified segmentation

    NeuroImage

    (2005)
  • B.B. Avants et al.

    Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain

    Med. Image Anal.

    (2008)
  • S. Beucher et al.

    The morphological approach to segmentation: the watershed transformation

  • M.J. Cardoso et al.

    Adaptive neonate brain segmentation

    Int. Conf. Med. Image Comput. Comput.-Assisted Inter.

    (2011)
  • Cardoso, M.J., Melbourne, A., Kendall, G.S., Robertson, N., Marlow, N.O.S., 2012. Outlier rejection for adaptive...
  • M.J. Cardoso et al.

    AdaPT: an adaptive preterm segmentation algorithm for neonatal brain MRI

    NeuroImage

    (2013)
  • T.F. Chan et al.

    Active contours without edges for vector-valued images

    J. Vis. Commun. Image Represent.

    (2000)
  • Chiţaˇ, S., Benders, M.J., Moeskops, P., Kersbergen, K.J., Viergever, M.A., Išgum, I., 2013. Automatic segmentation of...
  • G. Gerig et al.

    Nonlinear anisotropic filtering of MRI data

    IEEE Trans. Med. Imaging

    (1992)
  • Gui, L., Lisowski, R., Faundez, T., Hüppi, P., Lazeyras, F., Kocher, M., 2012a. Morphology-based segmentation of...
  • L. Gui et al.

    Morphology-driven automatic segmentation of MR images of the neonatal brain

    Med. Image Anal.

    (2012)
  • T.E. Inder et al.

    Abnormal cerebral structure is present at term in premature infants

    Pediatrics

    (2005)
  • S. Klein et al.

    elastix: a toolbox for intensity-based medical image registration

    IEEE Trans. Med. Imaging

    (2010)
  • M. Kuklisova-Murgasova et al.

    A dynamic 4D probabilistic atlas of the developing brain

    NeuroImage

    (2011)
  • B. Likar et al.

    Retrospective correction of MR intensity inhomogeneity by entropy minimization

    IEEE Trans. Med. Imaging

    (2001)
  • Lotufo, R., Falcao, A., 2002. The ordered queue and the optimality of the watershed approaches. In: Mathematical...
  • F. Maes et al.

    Multimodality image registration by maximization of mutual information

    IEEE Trans. Med. Imaging

    (1997)
  • Makropoulos, A., Ledig, C., Aljabar, P., Serag, A., Hajnal, J.V., Edwards, A.D., Counsell, S.J., Rueckert, D., 2012....
  • Cited by (86)

    View all citing articles on Scopus
    View full text