Elsevier

NeuroImage

Volume 70, 15 April 2013, Pages 402-409
NeuroImage

Estimating false positives and negatives in brain networks

https://doi.org/10.1016/j.neuroimage.2012.12.066Get rights and content

Abstract

The human brain is a complex network of anatomically segregated regions interconnected by white matter pathways, known as the human connectome. Diffusion tensor imaging can be used to reconstruct this structural brain network in vivo and noninvasively. However, due to a wide variety of influences, both false positive and false negative connections may occur. By choosing a ‘group threshold’, brain networks of multiple subjects can be combined into a single reconstruction, affecting the occurrence of these false positives and negatives. In this case, only connections that are detected in a large enough percentage of the subjects, specified by the group threshold, are considered to be present. Although this group threshold has a substantial impact on the resulting reconstruction and subsequent analyses, it is often chosen intuitively. Here, we introduce a model to estimate how the choice of group threshold influences the presence of false positives and negatives. Based on our findings, group thresholds should preferably be chosen between 30% and 90%. Our results further suggest that a group threshold of circa 60% is a suitable setting, providing a good balance between the elimination of false positives and false negatives.

Highlights

► The impact of the group threshold on false positives and negatives is modeled. ► The group threshold has a substantial impact on network metrics and errors. ► Group thresholds should preferably be chosen between 30% and 90%. ► A group threshold of circa 60% appears to be appropriate for most applications.

Introduction

The human brain is a complex system. On a macroscopic scale, anatomically segregated brain regions and bundles of axonal fibers constitute a network (Bullmore and Sporns, 2009, van den Heuvel et al., 2010), known as the structural human connectome (Sporns, 2011, Sporns et al., 2005). By tracing white matter pathways (Conturo et al., 1999, Mori et al., 1999), diffusion tensor imaging (DTI) tractography allows in vivo and noninvasive reconstruction of this structural brain network (Gong et al., 2009, Hagmann et al., 2007). However, due to a variety of influences and technical limitations, not all fiber pathways can be traced with equal accuracy (Dauguet et al., 2007, Jbabdi and Johansen-Berg, 2011). From a network perspective, there are two types of errors that may occur in the reconstruction. Brain regions which are erroneously connected give rise to false positives, while existing connections between brain regions that are absent in the reconstruction result in false negatives.

When studying the network characteristics of the healthy human brain (Bassett et al., 2011a, Gong et al., 2009, Hagmann et al., 2008, van den Heuvel et al., 2012) or differences between groups (Brown et al., 2011, Lynall et al., 2010, Stam et al., 2007, van den Heuvel et al., 2010, Zalesky et al., 2011, Zalesky et al., 2012), it is often desirable to make a combined reconstruction across a group of subjects, influencing the occurrence of false positives and negatives. One commonly applied method to create such a combined reconstruction is to set a so-called ‘group threshold’, reflecting the number of subjects (usually expressed as a percentage of the total group) that have to share a connection before it is included. The combined reconstruction then consists of those connections whose prevalence (i.e., detection count) equals or exceeds this group threshold (Hagmann et al., 2008, Honey et al., 2009, van den Heuvel and Sporns, 2011).

The choice of group threshold, which will be shown to have a substantial impact on the network properties of the reconstruction, is subject to a conflict between eliminating false positives and preventing false negatives. High group thresholds require a connection to be present in many subjects, improving the accuracy of reported connections and thus eliminating false positives, while low group thresholds allow the inclusion of existing connections that are difficult to reconstruct and therefore prevent false negatives. This choice of setting a particular group threshold is typically based on the researcher's intuition, which may introduce a bias and reduce the comparability of group based results across studies. Here, we aim to provide systematic insight into the influence of the group threshold on connectome reconstruction. First, we examine the effect of the group threshold on overall network organization. Second, we introduce a computational model to quantify the influence of the group threshold on the occurrence of false positives and negatives in the combined reconstruction.

Section snippets

Data acquisition

Analyses were performed on a collection of reconstructed structural brain networks of 50 healthy subjects. The reconstructions reflect the connectivity of 68 cortical brain areas and were obtained at the University Medical Center Utrecht using diffusion tensor imaging and deterministic streamline tractography as described previously (van den Heuvel and Sporns, 2011, van den Heuvel et al., 2010).

For each subject, two high-angular DTI sets with opposite k-space readouts (Andersson et al., 2003)

Impact on network metrics

All tested network metrics displayed significantly different values between the selected group thresholds of 20%, 40%, 60% and 80% (p < 0.001, Bonferroni corrected, Fig. 2b), underscoring the importance of good insight in the choice of the group threshold t. As can be seen from Fig. 2a, the combined reconstruction did not exhibit a small-world topology, characterized by a high normalized clustering coefficient and a low normalized path length (Watts and Strogatz, 1998), if all measured

Discussion

Based on the quantifications provided by our model (Fig. 3, Fig. 4), the most informative combined connectome reconstructions, balancing elimination of false positives and prevention of false negatives, are formed using a group threshold within the range of 30% to 90%. The results of our study further show that the group threshold – reflecting the number of subjects that have to share a connection before it is included in the reconstruction – has a substantial impact on commonly used network

Acknowledgements

The authors gratefully thank Guusje Collin for fruitful discussions. MPvdH was supported by a VENI (#451-12-001) grant of the Netherlands Organization for Scientific Research (NWO). MPvdH and MAdR were supported by a Rudolf Magnus Fellowship.

References (43)

  • A. Zalesky et al.

    Disrupted axonal fiber connectivity in schizophrenia

    Biol. Psychiatry

    (2011)
  • A. Zalesky et al.

    Connectivity differences in brain networks

    Neuroimage

    (2012)
  • D.S. Bassett et al.

    Dynamic reconfiguration of human brain networks during learning

    Proc. Natl. Acad. Sci.

    (2011)
  • J.A. Brown et al.

    Brain network local interconnectivity loss in aging APOE-4 allele carriers

    Proc. Natl. Acad. Sci.

    (2011)
  • E. Bullmore et al.

    Complex brain networks: graph theoretical analysis of structural and functional systems

    Nat. Rev. Neurosci.

    (2009)
  • L.C. Chang et al.

    RESTORE: Robust estimation of tensors by outlier rejection

    Magn. Reson. Med.

    (2005)
  • T.E. Conturo et al.

    Tracking neuronal fiber pathways in the living human brain

    Proc. Natl. Acad. Sci.

    (1999)
  • D.J. Felleman et al.

    Distributed hierarchical processing in the primate cerebral cortex

    Cereb. Cortex

    (1991)
  • B. Fischl et al.

    Automatically parcellating the human cerebral cortex

    Cereb. Cortex

    (2004)
  • G. Gong et al.

    Mapping anatomical connectivity patterns of human cerebral cortex using in vivo diffusion tensor imaging tractography

    Cereb. Cortex

    (2009)
  • P. Hagmann et al.

    Mapping human whole-brain structural networks with diffusion MRI

    PLoS One

    (2007)
  • Cited by (205)

    View all citing articles on Scopus
    View full text