Skip to main content
SpringerLink
Log in

Automated Anatomic Labeling Architecture for Content Discovery in Medical Imaging Repositories

  • Systems-Level Quality Improvement
  • Published:
Journal of Medical Systems Aims and scope Submit manuscript

Abstract

The combination of textual data with visual features is known to enhance medical image search capabilities. However, the most advanced imaging archives today only index the studies’ available meta-data, often containing limited amounts of clinically useful information. This work proposes an anatomic labeling architecture, integrated with an open source archive software, for improved multimodal content discovery in real-world medical imaging repositories. The proposed solution includes a technical specification for classifiers in an extensible medical imaging archive, a classification database for querying over the extracted information, and a set of proof-of-concept convolutional neural network classifiers for identifying the presence of organs in computed tomography scans. The system automatically extracts the anatomic region features, which are saved in the proposed database for later consumption by multimodal querying mechanisms. The classifiers were evaluated with cross-validation, yielding a best F1-score of 96% and an average accuracy of 97%. We expect these capabilities to become common-place in production environments in the future, as automated detection solutions improve in terms of accuracy, computational performance, and interoperability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. www.dicoogle.com

  2. Available on GitHub: https://github.com/Enet4/dicoogle-classification-api

  3. https://lucene.apache.org

  4. http://www.imagenglab.com/newsite/pddca/

  5. Available on GitHub: https://github.com/Enet4/dicoogle-class-db

References

  1. National Electrical Manufacturers Association (NEMA), Digital imaging and communications in medicine (DICOM) standard. Rosslyn: National Electrical Association, 2018.

    Google Scholar 

  2. Guld, M. O. et al. (2002) Quality of DICOM header information for image categorization, in SPIE, vol. 4685, pp. 280–287.

  3. Valente, F., Costa, C., Silva, A. (2013) Content based retrieval Systems in a Clinical Context, Medical Imaging in Clinical Practice, pp. 3–22.

  4. Akgül, C. B., Rubin, D. L., Napel, S., Beaulieu, C. F., Greenspan, H., and Acar, B., Content-based image retrieval in radiology: Current status and future directions. Journal of Digital Imaging 24(2):208–222, 2011.

    Article  PubMed  Google Scholar 

  5. Faruque, J., Beaulieu, C. F., Rosenberg, J., Rubin, D. L., Yao, D., and Napel, S., Content-based image retrieval in radiology: Analysis of variability in human perception of similarity. Journal of Medical Imaging 2(2):25501, 2015.

    Article  Google Scholar 

  6. Kumar, A., Kim, J., Cai, W., Fulham, M., and Feng, D., Content-based medical image retrieval: A survey of applications to multidimensional and multimodality data. Journal of Digital Imaging 26(6):1025–1039, 2013.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Valente, F., Silva, L. A. B., Godinho, T. M., and Costa, C., Anatomy of an extensible open source PACS. Journal of Digital Imaging 29(3):284–296, Jun. 2016.

    Article  PubMed  Google Scholar 

  8. Doi, K., Computer-aided diagnosis in medical imaging: Historical review, current status and future potential. Computerized Medical Imaging and Graphics 31(4):198–211, 2007.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Le, A. H. T., Liu, B., and Huang, H. K., Integration of computer-aided diagnosis/detection (CAD) results in a PACS environment using CAD–PACS toolkit and DICOM SR. International Journal of Computer Assisted Radiology and Surgery 4(4):317–329, 2009.

    Article  PubMed  Google Scholar 

  10. Dicken, V., Lindow, B., Bornemann, L., Drexl, J., Nikoubashman, A., and Peitgen, H.-O., Rapid image recognition of body parts scanned in computed tomography datasets. International Journal of Computer Assisted Radiology and Surgery 5(5):527–535, 2010.

    Article  PubMed  Google Scholar 

  11. Zhou, X et al. (2004) Automated recognition of human strucure from torso CT images, in International conference on visualization, imaging, and image processing, pp. 584–589.

  12. Desikan, R. S. et al., An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage 31(3):968–980, 2006.

    Article  PubMed  Google Scholar 

  13. Litjens, G. et al., A survey on deep learning in medical image analysis. Medical Image Analysis 42:60–88, 2017.

    Article  PubMed  Google Scholar 

  14. H. R. Roth et al., (2015) Anatomy-specific classification of medical images using deep convolutional nets, in IEEE 12th international symposium on biomedical imaging, pp. 101–104.

  15. Kumar, A., Kim, J., Lyndon, D., Fulham, M., and Feng, D., An Ensemble of Fine-Tuned Convolutional Neural Networks for medical image classification. IEEE Journal of Biomedical and Health Informatics PP(99):1, 2016.

    Google Scholar 

  16. Rajkomar, A., Lingam, S., Taylor, A. G., Blum, M., and Mongan, J., High-throughput classification of radiographs using deep convolutional neural networks. Journal of Digital Imaging 30(1):95–101, 2017.

    Article  PubMed  Google Scholar 

  17. Ronneberger, O., Fischer, P., Brox, T. (2015) U-net: Convolutional networks for biomedical image segmentation, in International conference on medical image computing and computer-assisted intervention, pp. 234–241.

  18. Wang, Y., Qiu, Y., Thai, T., Moore, K., Liu, H., and Zheng, B., A two-step convolutional neural network based computer-aided detection scheme for automatically segmenting adipose tissue volume depicting on CT images. Computer Methods and Programs in Biomedicine 144:97–104, 2017.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Lehmann, T. M., Wein, B. E., Keysers, D., Kohnen, M., Schubert, H. (2002) A monohierarchical multiaxial classification code for medical images in content-based retrieval, in IEEE international symposium on biomedical imaging, pp. 313–316.

  20. Markonis, D. et al. (2014) A Visual Information Retrieval System for Radiology Reports and the Medical Literature, in Multimedia modeling conference.

  21. Riegler, M. et al., From annotation to computer-aided diagnosis. ACM Transactions on Multimedia Computing, Communications, and Applications 13(3):1–26, 2017.

    Article  Google Scholar 

  22. Kundu, M. K., Chowdhury, M., and Das, S., Interactive radiographic image retrieval system. Computer Methods and Programs in Biomedicine 139:209–220, 2017.

    Article  PubMed  Google Scholar 

  23. Valente, F., Viana-Ferreira, C., Costa, C., and Oliveira, J. L., A RESTful image gateway for multiple medical image repositories. IEEE Transactions on Information Technology in Biomedicine 16(3):356–364, 2012.

    Article  PubMed  Google Scholar 

  24. Costa, C., Freitas, F., Pereira, M., Silva, A., and Oliveira, J. L., Indexing and retrieving DICOM data in disperse and unstructured archives. International Journal of Computer Assisted Radiology and Surgery 4(1):71–77, 2009.

    Article  PubMed  Google Scholar 

  25. Valente, F., Costa, C., and Silva, A., Dicoogle, a PACS featuring profiled content based image retrieval. Public Library of Science One 8(5):e61888, 2013.

    PubMed  CAS  Google Scholar 

  26. Pinho, E., Godinho, T., Valente, F., Costa, C., A multimodal search engine for medical imaging studies, Journal of Digital Imaging 30(1):39–48, 2017.

  27. Pinho, E, Costa, C. (2016) Extensible Architecture for Multimodal Information Retrieval in Medical Imaging Archives, in Signal image technology & internet based systems, pp. 316–322.

  28. Krizhevsky, A., Sutskever, I., Hinton, G. E. (2012) ImageNet classification with deep convolutional neural networks, in Advances in neural information processing systems, pp. 1097–1105.

  29. Ioffe, S., Szegedy, C. (2015) Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift, arXiv preprint arXiv:1502.03167, pp. 1–11.

  30. He, K., Zhang, X., Ren, S., Sun, J. (2015) Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification, in IEEE international conference on computer vision, pp. 1026–1034.

  31. Kingma, D. P., Ba, J. L. (2015) Adam: A Method for Stochastic Optimization, in International conference on learning representations.

  32. National Electrical Manufacturers Association (NEMA) (2018) Digital imaging and communications in medicine (DICOM) standard, PS3.3 – information object definitions. National Electrical Association, Rosslyn, VA, USA.

  33. Silva, L. A. B., Costa, C., Oliveira, J. L. (2014) Semantic search over DICOM repositories, in IEEE international conference on healthcare informatics, pp. 238–246.

  34. Kurtz, C., Depeursinge, A., Napel, S., Beaulieu, C. F., and Rubin, D. L., On combining image-based and ontological semantic dissimilarities for medical image retrieval applications. Medical Image Analysis 18(7):1082–1100, 2014.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Funding

This work is financed by the ERDF – European Regional Development Fund through the Operational Programme for Competitiveness and Internationalisation - COMPETE 2020 Programme, and by National Funds through the FCT – Fundação para a Ciência e a Tecnologia (Portuguese Foundation for Science and Technology) within project PTDC/EEI-ESS/6815/2014; POCI-01-0145-FEDER-016694. Eduardo Pinho was funded by FCT under grant agreement PD/BD/105806/2014.

Author information

Authors and Affiliations

  1. Instituto de Engenharia Electrónica e Informática de Aveiro, DETI / IEETA - University of Aveiro, Campus Universitário de Santiago, 3810-193, Aveiro, Portugal

    Eduardo Pinho & Carlos Costa

Authors
  1. Eduardo Pinho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos Costa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eduardo Pinho.

Ethics declarations

Conflict of interest

The authors Eduardo Pinho and Carlos Costa declare that they have no conflicts of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors. All medical imaging data used in this work were anonymized before provision.

Additional information

This article is part of the Topical Collection on Systems-Level Quality Improvement

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pinho, E., Costa, C. Automated Anatomic Labeling Architecture for Content Discovery in Medical Imaging Repositories. J Med Syst 42, 145 (2018). https://doi.org/10.1007/s10916-018-1004-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-018-1004-8

Keywords

Search

Navigation