Skip to main content
SpringerLink
Log in

Detection and Labeling of Vertebrae in MR Images Using Deep Learning with Clinical Annotations as Training Data

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

The purpose of this study was to investigate the potential of using clinically provided spine label annotations stored in a single institution image archive as training data for deep learning-based vertebral detection and labeling pipelines. Lumbar and cervical magnetic resonance imaging cases with annotated spine labels were identified and exported from an image archive. Two separate pipelines were configured and trained for lumbar and cervical cases respectively, using the same setup with convolutional neural networks for detection and parts-based graphical models to label the vertebrae. The detection sensitivity, precision and accuracy rates ranged between 99.1–99.8, 99.6–100, and 98.8–99.8% respectively, the average localization error ranges were 1.18–1.24 and 2.38–2.60 mm for cervical and lumbar cases respectively, and with a labeling accuracy of 96.0–97.0%. Failed labeling results typically involved failed S1 detections or missed vertebrae that were not fully visible on the image. These results show that clinically annotated image data from one image archive is sufficient to train a deep learning-based pipeline for accurate detection and labeling of MR images depicting the spine. Further, these results support using deep learning to assist radiologists in their work by providing highly accurate labels that only require rapid confirmation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Bengio Y.: Learning deep architectures for AI. Foundations and trends in Machine Learning 2(1):1–127,2009.

    Article  Google Scholar 

  2. Bengio Y, Courville A, Vincent P: Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence 35(8):1798–1828,2013.

    Article  PubMed  Google Scholar 

  3. LeCun Y, Bengio Y, Hinton GE: Deep learning. Nature 521(7553):436–444,2015.

    Article  CAS  PubMed  Google Scholar 

  4. Krizhevsky A, Sutskever I, Hinton GE: Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ Eds. Advances in Neural Information Processing Systems,2012 25, 1097–1105

  5. Szegedy C, Toshev A, Erhan D: Deep neural networks for object detection. In: Burges CJC, Bottou L, Welling M, Ghahramani Z, Weinberger KQ Eds. Advances in Neural Information Processing Systems,2013, 2553–2561

    Google Scholar 

  6. Shin HC, Lu L, Kim L, Seff A, Yao J, Summers RM: Interleaved text/image deep mining on a very large-scale radiology database. In: 2015 I.E. Conference on Computer Vision and Pattern Recognition (CVPR), 2015, 1090–1099

  7. Pereira S, Pinto A, Alves V, Silva CA: Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Transactions on Medical Imaging 35(5):1240–1251,2016.

    Article  PubMed  Google Scholar 

  8. Setio AAA, Ciompi F, Litjens G, et al.: Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Transactions on Medical Imaging 35(5):1160–1169,2016

    Article  PubMed  Google Scholar 

  9. Yan Z, Zhan Y, Peng Z, et al.: Multi-instance deep learning: Discover discriminative local anatomies for bodypart recognition. IEEE Transactions on Medical Imaging 35(5):1332–1343,2016

    Article  Google Scholar 

  10. Greenspan H, van Ginneken B, Summers RM: Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique. IEEE Transactions on Medical Imaging 35(5):1153–1159,2016

    Article  Google Scholar 

  11. Klinder T, Ostermann J, Ehm M, Franz A, Kneser R, Lorenz C: Automated model-based vertebra detection, identification, and segmentation in CT images. Medical Image Analysis 13(3):471–482,2009

    Article  PubMed  Google Scholar 

  12. Oktay AB and Akgul YS: Simultaneous localization of lumbar vertebrae and intervertebral discs with svm-based mrf. Biomedical Engineering, IEEE Transactions on 60(9):2375–2383,2013

    Article  Google Scholar 

  13. Major D, Hladůvka J, Schulze F, Bühler K: Automated landmarking and labeling of fully and partially scanned spinal columns in CT images. Medical Image Analysis 17(8):1151–1163,2013

    Article  PubMed  Google Scholar 

  14. Zhan Y, Jian B, Maneesh D, Zhou XS: Cross-Modality Vertebrae Localization and Labeling Using Learning- Based Approaches. In: Li S, Yao J Eds. Spinal Imaging and Image Analysis, 2015, 301–322

  15. Huang SH, Chu YH, Lai SH, Novak CL: Learning-based vertebra detection and iterative normalized-cut segmentation for spinal MRI. Medical Imaging, IEEE Transactions on 2009 28(10):1595–1605, 2009.

    Article  Google Scholar 

  16. Lootus M, Kadir T, Zisserman A: Vertebrae Detection and Labelling in Lumbar MR Images. In: Yao J, Klinder T, Li S Eds. Computational Methods and Clinical Applications for Spine Imaging, 2014, 219–230.

  17. Cai Y, Osman S, Sharma M, Landis M, Li S: Multi-modality vertebra recognition in arbitrary views using 3d deformable hierarchical model. IEEE Transactions on Medical Imaging 34(8):1676–1693, 2015.

    Article  PubMed  Google Scholar 

  18. Glocker B, Feulner J, Criminisi A, Haynor DR, Konukoglu E. Automatic Localization and Identification of Vertebrae in Arbitrary Field-of-View CT Scans. In: Ayache N, Delingette H, Golland P, Mori K Eds. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2012, 2012, 590–598

  19. Glocker B, Zikic D, Konukoglu E, Haynor DR, Criminisi A: Vertebrae localization in pathological spine CT via dense classification from sparse annotations. In: Mori K, Sakuma I, Sato Y, Barillot C, Navab N Eds. Medical image computing and computer-assisted intervention – MICCAI 2013. Heidelberg: Springer Berlin, 2013, 262–270.

    Chapter  Google Scholar 

  20. Chen H, Shen C, Qin J, Ni D, Shi L, Cheng JCY, Heng PA: Automatic Localization and Identification of Vertebrae in Spine CT via a Joint Learning Model with Deep Neural Networks. In: Navab N, Hornegger J, Wells WM, Frangi AF Eds. Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2015, 2015, 515–522

  21. Suzani A, Seitel A, Liu Y, Fels S, Rohling RN, Abolmaesumi P: Fast Automatic Vertebrae Detection and Localization in Pathological CT Scans - A Deep Learning Approach. In: Navab N, Hornegger J, Wells WM, Frangi AF Eds. Medical Image Computing and Computer-Assisted Intervention -- MICCAI 2015, 2015, 678–686

  22. Cai Y, Landis M, Laidley DT, Kornecki A, Lum A, Li S: Multi-modal vertebrae recognition using transformed deep convolution network. Computerized Medical Imaging and Graphics 51:11–19,2016.

    Article  PubMed  Google Scholar 

  23. Dijkstra EW: A note on two problems in connexion with graphs. Numerische Mathematik 1(1):269–271,1959.

    Article  Google Scholar 

  24. Al-Rfou R, Alain, G, Almahairi A: Theano: A Python framework for fast computation of mathematical expressions. arXiv e-prints 2016;abs/1605.02688. http://arxiv.org/abs/1605.02688. Published May 9, 2016. Accessed June 6, 1026,2016

  25. Volpi D, Sarhan MH, Ghotbi R, Navab N, Mateus D, Demirci S: Online tracking of interventional devices for endovascular aortic repair. International Journal of Computer Assisted Radiology and Surgery 10(6):773–781,2015.

    Article  PubMed  Google Scholar 

  26. Shin HC, Roth HR, Gao M, et al.: Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. Medical Imaging, IEEE Transactions on 35(5):1285–1298,2016.

    Article  Google Scholar 

  27. Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang J: Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Transactions on Medical Imaging 35(5):1299–1312,2016.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The Titan X GPU used for this research was donated by the NVIDIA Corporation.

Author information

Authors and Affiliations

  1. Sectra, Teknikringen 20, 583 30, Linköping, SE, Sweden

    Daniel Forsberg & Erik Sjöblom

  2. Department of Radiology, Case Western Reserve University and University Hospitals Cleveland Medical Center, 11100 Euclid Avenue, Cleveland, OH, 44106, USA

    Daniel Forsberg & Jeffrey L. Sunshine

Authors
  1. Daniel Forsberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Erik Sjöblom
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jeffrey L. Sunshine
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel Forsberg.

Ethics declarations

The study was ruled exempt by the local institutional review board and informed consent was waived.

Funding

D. Forsberg is supported by a grant (2014–01422) from the Swedish Innovation Agency (VINNOVA).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Forsberg, D., Sjöblom, E. & Sunshine, J.L. Detection and Labeling of Vertebrae in MR Images Using Deep Learning with Clinical Annotations as Training Data. J Digit Imaging 30, 406–412 (2017). https://doi.org/10.1007/s10278-017-9945-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-017-9945-x

Keywords

Search

Navigation