Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Correspondence
  • Published:

A community-developed open-source computational ecosystem for big neuro data

Nature Methods volume 15, pages 846–847 (2018)Cite this article

Subjects

Access through your institution
Buy or subscribe

To the Editor — Recent technological developments, such as high-throughput imaging and sequencing, enable experimentalists to collect increasingly large, complex, and heterogeneous ‘big’ data. These studies result in terabytes of data per day, yielding petabytes across experiments and laboratories. These experimental capabilities exceed the scale or feature set of existing software. For example, such data cannot be stored, processed, and visualized on a laptop or workstation. Instead, big data must be stored on data centers and processed on high-performance compute clusters.

In 2011, we launched Open Connectome Project1, an open-access data repository powered by open-source web-services software applications that store, analyze, and visualize large imaging datasets. However, as technology changed, features were added, and scale increased, our academic development team and resources became overwhelmed. We overhauled our custom stack into a community-built and -maintained software ecosystem deployed in the commercial cloud, integrating multiple open-source projects and extending them for our needs (https://neurodata.io). The ecosystem makes it possible to analyze disparate datasets by reusing components originally designed for other applications.

This is a preview of subscription content, access via your institution

Access options

Access through your institution

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: NeuroData’s open-source software ecosystem and example visualizations.

References

  1. Burns, R. et al. The open connectome project data cluster: scalable analysis and vision for high-throughput neuroscience. In Proc. 25th International Conference on Scientific and Statistical Database Management (eds Szalay, A. et al.) Article 27 (ACM, New York, 2013).

  2. Collman, F. et al. J. Neurosci. 35, 5792–5807 (2015).

    Article  CAS  Google Scholar 

  3. Simhal, A. K. et al. PLoS Comput. Biol. 13, e1005493 (2017).

    Article  Google Scholar 

  4. Shen, C. et al. arXiv Preprint at https://arxiv.org/abs/1609.05148 (2016).

  5. Chung, K. et al. Nature 497, 332–337 (2013).

    Article  CAS  Google Scholar 

  6. Kutten, K. S. et al. A large deformation diffeomorphic approach to registration of CLARITY images via mutual information. In Medical Image Computing and Computer Assisted Intervention—MICCAI 2017 Vol. 10433 (eds Descoteaux, M. et al.) 275–282 (Springer, Cham, 2017).

    Chapter  Google Scholar 

Download references

Acknowledgements

R.B., E.P., B.F., A.B., V.C., K.L., M.K., E.B., J.L.P., D.J.T., M.I.M., and J.T.V. were supported by the Defense Advanced Research Projects Agency (DARPA) SIMPLEX program, SPAWAR contract N66001-15-C-4041, NIH-NINDS TRA 1R01NS092474, “Synaptomes of Mouse and Man,” and NSF 16-569 NeuroNex contract 1707298. W.G.R., B.W., R.H., D.P., T.G., P.M., and J.M. were supported by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via IARPA contract no. 2017-17032700004-005 under the MICrONS program and APL Internal Research and Development Funds. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the ODNI, IARPA, or the US Government. The US Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation.

Author information

Authors and Affiliations

  1. Johns Hopkins University, Baltimore, MD, USA

    Joshua T. Vogelstein, Eric Perlman, Benjamin Falk, Alex Baden, Vikram Chandrashekhar, Jesse L. Patsolic, Kunal Lillaney, Michael Kazhdan, Eric Bridgeford, Daniel J. Tward, Michael I. Miller & Randal Burns

  2. Applied Physics Laboratory, Johns Hopkins University, Laurel, MD, USA

    William Gray Roncal, Robert Hider Jr., Derek Pryor, Jordan Matelsky, Timothy Gion, Priya Manavalan & Brock Wester

  3. Allen Institute for Brain Sciences, Seattle, WA, USA

    Forrest Collman, Sharmishtaa Seshamani & Stephen J. Smith

  4. Facebook, Menlo Park, CA, USA

    Mark Chevillet

  5. Janelia Research Campus, Ashburn, VA, USA

    Eric T. Trautman & Khaled Khairy

  6. Gigantum, Washington, DC, USA

    Dean M. Kleissas & R. Jacob Vogelstein

  7. Stanford University, Stanford, CA, USA

    Ailey K. Crow, Brian Hsueh, Matthew A. Wright & Karl Deisseroth

Authors
  1. Joshua T. Vogelstein
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Eric Perlman
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Benjamin Falk
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Alex Baden
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. William Gray Roncal
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Vikram Chandrashekhar
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Forrest Collman
    View author publications

    You can also search for this author inPubMed Google Scholar

  8. Sharmishtaa Seshamani
    View author publications

    You can also search for this author inPubMed Google Scholar

  9. Jesse L. Patsolic
    View author publications

    You can also search for this author inPubMed Google Scholar

  10. Kunal Lillaney
    View author publications

    You can also search for this author inPubMed Google Scholar

  11. Michael Kazhdan
    View author publications

    You can also search for this author inPubMed Google Scholar

  12. Robert Hider Jr.
    View author publications

    You can also search for this author inPubMed Google Scholar

  13. Derek Pryor
    View author publications

    You can also search for this author inPubMed Google Scholar

  14. Jordan Matelsky
    View author publications

    You can also search for this author inPubMed Google Scholar

  15. Timothy Gion
    View author publications

    You can also search for this author inPubMed Google Scholar

  16. Priya Manavalan
    View author publications

    You can also search for this author inPubMed Google Scholar

  17. Brock Wester
    View author publications

    You can also search for this author inPubMed Google Scholar

  18. Mark Chevillet
    View author publications

    You can also search for this author inPubMed Google Scholar

  19. Eric T. Trautman
    View author publications

    You can also search for this author inPubMed Google Scholar

  20. Khaled Khairy
    View author publications

    You can also search for this author inPubMed Google Scholar

  21. Eric Bridgeford
    View author publications

    You can also search for this author inPubMed Google Scholar

  22. Dean M. Kleissas
    View author publications

    You can also search for this author inPubMed Google Scholar

  23. Daniel J. Tward
    View author publications

    You can also search for this author inPubMed Google Scholar

  24. Ailey K. Crow
    View author publications

    You can also search for this author inPubMed Google Scholar

  25. Brian Hsueh
    View author publications

    You can also search for this author inPubMed Google Scholar

  26. Matthew A. Wright
    View author publications

    You can also search for this author inPubMed Google Scholar

  27. Michael I. Miller
    View author publications

    You can also search for this author inPubMed Google Scholar

  28. Stephen J. Smith
    View author publications

    You can also search for this author inPubMed Google Scholar

  29. R. Jacob Vogelstein
    View author publications

    You can also search for this author inPubMed Google Scholar

  30. Karl Deisseroth
    View author publications

    You can also search for this author inPubMed Google Scholar

  31. Randal Burns
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Joshua T. Vogelstein.

Ethics declarations

Competing interests

D.M.K. and R.J.V. are employed by Gigantum; this company provided support in the form of salaries for these authors, but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vogelstein, J.T., Perlman, E., Falk, B. et al. A community-developed open-source computational ecosystem for big neuro data. Nat Methods 15, 846–847 (2018). https://doi.org/10.1038/s41592-018-0181-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41592-018-0181-1

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing