Estimating false positives and negatives in brain networks
Highlights
► The impact of the group threshold on false positives and negatives is modeled. ► The group threshold has a substantial impact on network metrics and errors. ► Group thresholds should preferably be chosen between 30% and 90%. ► A group threshold of circa 60% appears to be appropriate for most applications.
Introduction
The human brain is a complex system. On a macroscopic scale, anatomically segregated brain regions and bundles of axonal fibers constitute a network (Bullmore and Sporns, 2009, van den Heuvel et al., 2010), known as the structural human connectome (Sporns, 2011, Sporns et al., 2005). By tracing white matter pathways (Conturo et al., 1999, Mori et al., 1999), diffusion tensor imaging (DTI) tractography allows in vivo and noninvasive reconstruction of this structural brain network (Gong et al., 2009, Hagmann et al., 2007). However, due to a variety of influences and technical limitations, not all fiber pathways can be traced with equal accuracy (Dauguet et al., 2007, Jbabdi and Johansen-Berg, 2011). From a network perspective, there are two types of errors that may occur in the reconstruction. Brain regions which are erroneously connected give rise to false positives, while existing connections between brain regions that are absent in the reconstruction result in false negatives.
When studying the network characteristics of the healthy human brain (Bassett et al., 2011a, Gong et al., 2009, Hagmann et al., 2008, van den Heuvel et al., 2012) or differences between groups (Brown et al., 2011, Lynall et al., 2010, Stam et al., 2007, van den Heuvel et al., 2010, Zalesky et al., 2011, Zalesky et al., 2012), it is often desirable to make a combined reconstruction across a group of subjects, influencing the occurrence of false positives and negatives. One commonly applied method to create such a combined reconstruction is to set a so-called ‘group threshold’, reflecting the number of subjects (usually expressed as a percentage of the total group) that have to share a connection before it is included. The combined reconstruction then consists of those connections whose prevalence (i.e., detection count) equals or exceeds this group threshold (Hagmann et al., 2008, Honey et al., 2009, van den Heuvel and Sporns, 2011).
The choice of group threshold, which will be shown to have a substantial impact on the network properties of the reconstruction, is subject to a conflict between eliminating false positives and preventing false negatives. High group thresholds require a connection to be present in many subjects, improving the accuracy of reported connections and thus eliminating false positives, while low group thresholds allow the inclusion of existing connections that are difficult to reconstruct and therefore prevent false negatives. This choice of setting a particular group threshold is typically based on the researcher's intuition, which may introduce a bias and reduce the comparability of group based results across studies. Here, we aim to provide systematic insight into the influence of the group threshold on connectome reconstruction. First, we examine the effect of the group threshold on overall network organization. Second, we introduce a computational model to quantify the influence of the group threshold on the occurrence of false positives and negatives in the combined reconstruction.
Section snippets
Data acquisition
Analyses were performed on a collection of reconstructed structural brain networks of 50 healthy subjects. The reconstructions reflect the connectivity of 68 cortical brain areas and were obtained at the University Medical Center Utrecht using diffusion tensor imaging and deterministic streamline tractography as described previously (van den Heuvel and Sporns, 2011, van den Heuvel et al., 2010).
For each subject, two high-angular DTI sets with opposite k-space readouts (Andersson et al., 2003)
Impact on network metrics
All tested network metrics displayed significantly different values between the selected group thresholds of 20%, 40%, 60% and 80% (p < 0.001, Bonferroni corrected, Fig. 2b), underscoring the importance of good insight in the choice of the group threshold t. As can be seen from Fig. 2a, the combined reconstruction did not exhibit a small-world topology, characterized by a high normalized clustering coefficient and a low normalized path length (Watts and Strogatz, 1998), if all measured
Discussion
Based on the quantifications provided by our model (Fig. 3, Fig. 4), the most informative combined connectome reconstructions, balancing elimination of false positives and prevention of false negatives, are formed using a group threshold within the range of 30% to 90%. The results of our study further show that the group threshold – reflecting the number of subjects that have to share a connection before it is included in the reconstruction – has a substantial impact on commonly used network
Acknowledgements
The authors gratefully thank Guusje Collin for fruitful discussions. MPvdH was supported by a VENI (#451-12-001) grant of the Netherlands Organization for Scientific Research (NWO). MPvdH and MAdR were supported by a Rudolf Magnus Fellowship.
References (43)
- et al.
How to correct susceptibility distortions in spin-echo echo-planar images: application to diffusion tensor imaging
Neuroimage
(2003) - et al.
Conserved and variable architecture of human white matter connectivity
Neuroimage
(2011) - et al.
Mapping the human connectome at multiple scales with diffusion spectrum MRI
J. Neurosci. Methods
(2012) - et al.
Comparison of fiber tracts derived from in-vivo DTI tractography with 3D histological neural tract tracer reconstruction on a macaque brain
Neuroimage
(2007) - et al.
Using graph theoretical analysis of multi channel EEG to evaluate the neural efficiency hypothesis
Neurosci. Lett.
(2006) - et al.
Complex network measures of brain connectivity: uses and interpretations
Neuroimage
(2010) - et al.
The connectional organization of neural systems in the cat cerebral cortex
Curr. Biol.
(1993) - et al.
An exponential random graph modeling approach to creating group-based representative whole-brain connectivity networks
Neuroimage
(2012) Functional connectivity patterns of human magnetoencephalographic recordings: a ‘small-world’ network?
Neurosci. Lett.
(2004)- et al.
Small-world and scale-free organization of voxel-based resting-state functional connectivity in the human brain
Neuroimage
(2008)
Disrupted axonal fiber connectivity in schizophrenia
Biol. Psychiatry
Connectivity differences in brain networks
Neuroimage
Dynamic reconfiguration of human brain networks during learning
Proc. Natl. Acad. Sci.
Brain network local interconnectivity loss in aging APOE-4 allele carriers
Proc. Natl. Acad. Sci.
Complex brain networks: graph theoretical analysis of structural and functional systems
Nat. Rev. Neurosci.
RESTORE: Robust estimation of tensors by outlier rejection
Magn. Reson. Med.
Tracking neuronal fiber pathways in the living human brain
Proc. Natl. Acad. Sci.
Distributed hierarchical processing in the primate cerebral cortex
Cereb. Cortex
Automatically parcellating the human cerebral cortex
Cereb. Cortex
Mapping anatomical connectivity patterns of human cerebral cortex using in vivo diffusion tensor imaging tractography
Cereb. Cortex
Mapping human whole-brain structural networks with diffusion MRI
PLoS One
Cited by (205)
Associated Genetics and Connectomic Circuitry in Schizophrenia and Bipolar Disorder
2023, Biological PsychiatryShared and Specific Patterns of Structural Brain Connectivity Across Affective and Psychotic Disorders
2023, Biological Psychiatry