Skip to main content
SpringerLink
Log in

Content-Based Medical Image Retrieval: A Survey of Applications to Multidimensional and Multimodality Data

  • Published:
Journal of Digital Imaging Aims and scope Submit manuscript

Abstract

Medical imaging is fundamental to modern healthcare, and its widespread use has resulted in the creation of image databases, as well as picture archiving and communication systems. These repositories now contain images from a diverse range of modalities, multidimensional (three-dimensional or time-varying) images, as well as co-aligned multimodality images. These image collections offer the opportunity for evidence-based diagnosis, teaching, and research; for these applications, there is a requirement for appropriate methods to search the collections for images that have characteristics similar to the case(s) of interest. Content-based image retrieval (CBIR) is an image search technique that complements the conventional text-based retrieval of images by using visual features, such as color, texture, and shape, as search criteria. Medical CBIR is an established field of study that is beginning to realize promise when applied to multidimensional and multimodality medical data. In this paper, we present a review of state-of-the-art medical CBIR approaches in five main categories: two-dimensional image retrieval, retrieval of images with three or more dimensions, the use of nonimage data to enhance the retrieval, multimodality image retrieval, and retrieval from diverse datasets. We use these categories as a framework for discussing the state of the art, focusing on the characteristics and modalities of the information used during medical image retrieval.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. QBIC: http://wwwqbic.almaden.ibm.com/.

  2. Click the camera icon in the search bar on http://images.google.com/.

  3. IRMA Homepage (English): http://www.irma-project.org/index_en.php.

  4. ImageCLEF Homepage: http://www.imageclef.org/.

  5. ImageCLEF medical image task: http://www.imageclef.org/2013/medical.

  6. PEIR Digital Library: http://peir.path.uab.edu/.

  7. NHANES: http://www.cdc.gov/nchs/nhanes.htm.

  8. TCIA: http://cancerimagingarchive.net/.

References

  1. Doi K: Computer-aided diagnosis in medical imaging: Historical review, current status and future potential. Comput Med Imaging Graph 31(4–5):198–211, 2007

    Article  PubMed  Google Scholar 

  2. Zaidi H, Vees H, Wissmeyer M: Molecular PET/CT imaging-guided radiation therapy treatment planning. Acad Radiol 16(9):1108–33, 2009

    Article  PubMed  Google Scholar 

  3. Marcus C, Ladam-Marcus V, Cucu C, Bouché O, Lucas L, Hoeffel C: Imaging techniques to evaluate the response to treatment in oncology: Current standards and perspectives. Crit Rev Oncol/Hematol 72(3):217–38, 2009

    Article  CAS  Google Scholar 

  4. Holt A, Bichindaritz I, Schmidt R, Perner P: Medical applications in case-based reasoning. Knowl Eng Rev 20(03):289–92, 2005

    Article  Google Scholar 

  5. Sedghi S, Sanderson M, Clough P: How do health care professionals select medical images they need? ASLIB Proc 64(4):437–56, 2012

    Article  Google Scholar 

  6. Haux R: Health information systems—Past, present, future. Int J Med Inform 75(3–4):268–81, 2006

    Article  PubMed  Google Scholar 

  7. Huang HK. PACS and Imaging Informatics: Basic Principles and Applications. New York: Wiley, 2004

  8. Müller H, Michoux N, Bandon D, Geissbuhler A: A review of content-based image retrieval systems in medical applications—Clinical benefits and future directions. Int J Med Inform 73(1):1–23, 2004

    Article  PubMed  Google Scholar 

  9. Müller H, Zhou X, Depeursinge A, Pitkanen M, Iavindrasana J, Geissbuhler A: Medical visual information retrieval: State of the art and challenges ahead. In: Proceedings of the IEEE International Conference on Multimedia and Expo, Beijing, 2007, pp 683–686

  10. Müller H, Kalpathy-Cramer J, Caputo B, Syeda-Mahmood T, Wang F: Overview of the first workshop on medical content-based retrieval for clinical decision support at MICCAI 2009. In: Caputo B, Müller H, Syeda-Mahmood T, Duncan J, Wang F, Kalpathy-Cramer J Eds. Medical Content-Based Retrieval for Clinical Decision Support, Vol. 5853 of Lecture Notes in Computer Science. Berlin: Springer, 2010, pp 1–17

  11. Huang HK: Utilization of medical imaging informatics and biometrics technologies in healthcare delivery. Int J Comput Assist Radiol Surg 3:27–39, 2008

    Article  Google Scholar 

  12. Tagare HD, Jaffe CC, Duncan J: Medical image databases: A content-based retrieval approach. J Am Med Inform Assoc 4(3):184–98, 1997

    Article  PubMed  CAS  Google Scholar 

  13. Lehmann TM, Guld MO, Thies C, Fischer B, Keysers D, Kohnen M, et al: Content-based image retrieval in medical applications for picture archiving and communication systems. In: Huang HK, Ratib OM Eds. Proceedings of SPIE 5033, 2003, pp 109–117

  14. Brown KR, Silver I, Musgrave J, Roberts A: The use of μCT technology to identify skull fracture in a case involving blunt force trauma. Forensic Sci Int 206(1–3):8–11, 2011

    Article  Google Scholar 

  15. Blodgett TM, Meltzer CC, Townsend DW: PET/CT: Form and function. Radiology 242(2):360–85, 2007

    Article  PubMed  Google Scholar 

  16. Smeulders A, Worring M, Santini S, Gupta A, Jain R: Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell 22(12):1349–80, 2000

    Article  Google Scholar 

  17. Cai TW, Kim J, Feng DD: Content-based medical image retrieval. In: Feng DD Ed. Biomedical Information Technology. Burlington: Academic Press, 2008, pp 83–113

  18. Long LR, Antani S, Deserno TM, Thoma GR: Content-based image retrieval in medicine: Retrospective assessment, state of the art, and future directions. Int J Healthcare Inf Syst Inform 4(1):1–16, 2009

    Article  Google Scholar 

  19. Akgül C, Rubin D, Napel S, Beaulieu C, Greenspan H, Acar B: Content-based image retrieval in radiology: Current status and future directions. J Digit Imaging 24:208–22, 2011

    Article  PubMed  Google Scholar 

  20. Lew MS, Sebe N, Djeraba C, Jain R: Content-based multimedia information retrieval: State of the art and challenges. ACM Trans Multimed Comput Commun Appl 2(1):1–19, 2006

    Article  Google Scholar 

  21. Rui Y, Huang TS, Chang SF: Image retrieval: Current techniques, promising directions, and open issues. J Vis Commun Image Represent 10(1):39–62, 1999

    Article  Google Scholar 

  22. Datta R, Joshi D, Li J, Wang JZ: Image retrieval: Ideas, influences, and trends of the new age. ACM Comput Surv 40(2):5:1–5:60, 2008

    Google Scholar 

  23. Flickner M, Sawhney H, Niblack W, Ashley J, Huang Q, Dom B, et al: Query by image and video content: The QBIC system. Computer 28(9):23–32, 1995

    Article  Google Scholar 

  24. Bach JR, Fuller C, Gupta A, Hampapur A, Horowitz B, Humphrey R, et al: Virage image search engine: an open framework for image management. In: Sethi IK, Jain RC Eds. Proceedings of SPIE 2670, 1, 1996, pp 76–87

  25. Pentland A, Picard RW, Sclaroff S: Photobook: Content-based manipulation of image databases. Int J Comput Vis 18:233–54, 1996

    Article  Google Scholar 

  26. Chechik G, Sharma V, Shalit U, Bengio S: Large scale online learning of image similarity through ranking. J Mach Learn Res 11:1109–35, 2010

    Google Scholar 

  27. Duncan JS, Ayache N: Medical image analysis: Progress over two decades and the challenges ahead. IEEE Trans Pattern Anal Mach Intell 22(1):85–106, 2000

    Article  Google Scholar 

  28. Townsend DW, Beyer T: A combined PET/CT scanner: The path to true image fusion. Br J Radiol 75(Supplement 9):S24–30, 2002

    PubMed  Google Scholar 

  29. Townsend DW, Beyer T, Blodgett TM: PET/CT scanners: A hardware approach to image fusion. Semin Nucl Med 33(3):193–204, 2003

    Article  PubMed  Google Scholar 

  30. Judenhofer MS, Catana C, Swann BK, Siegel SB, Jung WI, Nutt RE, et al: PET/MR images acquired with a compact MR-compatible PET detector in a 7-T magnet. Radiology 244(3):807–14, 2007

    Article  PubMed  Google Scholar 

  31. Shyu CR, Brodley CE, Kak AC, Kosaka A, Aisen AM, Broderick LS: ASSERT: A physician-in-the-loop content-based retrieval system for HRCT image databases. Comp Vision Image Underst 75(1–2):111–32, 1999

    Article  Google Scholar 

  32. Aisen AM, Broderick LS, Winer-Muram H, Brodley CE, Kak AC, Pavlopoulou C, et al: Automated storage and retrieval of thin-section CT images to assist diagnosis: System description and preliminary assessment. Radiology 228(1):265–70, 2003

    Article  PubMed  Google Scholar 

  33. Napel SA, Beaulieu CF, Rodriguez C, Cui J, Xu J, Gupta A, et al: Automated retrieval of CT images of liver lesions on the basis of image similarity: Method and preliminary results. Radiology 256(1):243–52, 2010

    Article  PubMed  Google Scholar 

  34. Müller H, Rosset A, Garcia A, Vallée JP, Geissbuhler A: Benefits of content-based visual data access in radiology. Radiographics 25(3):849–58, 2005

    Article  PubMed  Google Scholar 

  35. Keysers D, Dahmen J, Ney H, Wein BB, Lehmann TM: Statistical framework for model-based image retrieval in medical applications. J Electron Imaging 12(1):59–68, 2003

    Article  Google Scholar 

  36. Güld MO, Thies C, Fischer B, Lehmann TM: A generic concept for the implementation of medical image retrieval systems. Int J Med Inform 76(2–3):252–9, 2007

    Article  PubMed  Google Scholar 

  37. Iakovidis D, Pelekis N, Kotsifakos E, Kopanakis I, Karanikas H, Theodoridis Y: A pattern similarity scheme for medical image retrieval. IEEE Trans Inf Technol Biomed 13(4):442–50, 2009

    Article  PubMed  Google Scholar 

  38. Antani S, Lee D, Long LR, Thoma GR: Evaluation of shape similarity measurement methods for spine X-ray images. J Vis Commun Image Represent 15(3):285–302, 2004

    Article  Google Scholar 

  39. Antani S, Long LR, Thoma GR, Lee DJ: Evaluation of shape indexing methods for content-based retrieval of X-ray images. In: Yeung MM, Lienhart RW, Li CS Eds. Proceedings of SPIE 5021, 2003, pp 405–416

  40. Lee DJ, Antani S, Long LR: Similarity measurement using polygon curve representation and Fourier descriptors for shape-based vertebral image retrieval. In: Sonka M, Fitzpatrick JM Eds. Proceedings of SPIE 5032, 2003, pp 1283–1291

  41. Xu X, Lee DJ, Antani S, Long L: A spine X-ray image retrieval system using partial shape matching. IEEE Trans Inf Technol Biomed 12(1):100–8, 2008

    Article  PubMed  CAS  Google Scholar 

  42. Hsu W, Antani S, Long LR, Neve L, Thoma GR: SPIRS: A web-based image retrieval system for large biomedical databases. Int J Med Inform 78(Supplement 1):S13–24, 2009

    Article  PubMed  Google Scholar 

  43. Lee DJ, Antani S, Chang Y, Gledhill K, Long LR, Christensen P: CBIR of spine X-ray images on inter-vertebral disc space and shape profiles using feature ranking and voting consensus. Data Knowl Eng 68(12):1359–69, 2009

    Article  Google Scholar 

  44. Qian X, Tagare HD, Fulbright RK, Long R, Antani S: Optimal embedding for shape indexing in medical image databases. Med Image Anal 14(3):243–54, 2010

    Article  PubMed  Google Scholar 

  45. Xue Z, Antani S, Long LR, Jeronimo J, Thoma GR: Investigating CBIR techniques for cervicographic images. In: Proceedings of the Annual Symposium of American Medical Information Association, 2007, pp 826–830

  46. Xue Z, Antani S, Long L, Thoma G: A system for searching uterine cervix images by visual attributes. In: IEEE International Symposium on Computer-Based Medical Systems, 2009, pp 1–5

  47. Korn P, Sidiropoulos N, Faloutsos C, Siegel E, Protopapas Z: Fast and effective retrieval of medical tumor shapes. IEEE Trans Knowl Data Eng 10(6):889–904, 1998

    Article  Google Scholar 

  48. Yang L, Jin R, Mummert L, Sukthankar R, Goode A, Zheng B, et al: A boosting framework for visuality-preserving distance metric learning and its application to medical image retrieval. IEEE Trans Pattern Anal Mach Intell 32(1):30–44, 2010

    Article  PubMed  Google Scholar 

  49. Quellec G, Lamard M, Cazuguel G, Cochener B, Roux C: Wavelet optimization for content-based image retrieval in medical databases. Med Image Anal 14(2):227–41, 2010

    Article  PubMed  CAS  Google Scholar 

  50. Quellec G, Lamard M, Bekri L, Cazuguel G, Roux C, Cochener B: Medical case retrieval from a committee of decision trees. IEEE Trans Inf Technol Biomed 14(5):1227–35, 2010

    Article  PubMed  Google Scholar 

  51. Quellec G, Lamard M, Cazuguel G, Roux C, Cochener B: Case retrieval in medical databases by fusing heterogeneous information. IEEE Trans Med Imaging 30(1):108–18, 2011

    Article  PubMed  Google Scholar 

  52. Dy JG, Brodley CE, Kak A, Broderick LS, Aisen AM: Unsupervised feature selection applied to content-based retrieval of lung images. IEEE Trans Pattern Anal Mach Intell 25(3):373–8, 2003

    Article  Google Scholar 

  53. Unay D, Ekin A, Jasinschi R: Local structure-based region-of-interest retrieval in brain MR images. IEEE Trans Inf Technol Biomed 14(4):897–903, 2010

    Article  PubMed  Google Scholar 

  54. Petrakis EG: Design and evaluation of spatial similarity approaches for image retrieval. Image Vis Comput 20(1):59–76, 2002

    Article  Google Scholar 

  55. Alajlan N, Kamel M, Freeman G: Geometry-based image retrieval in binary image databases. IEEE Trans Pattern Anal Mach Intell 30(6):1003–13, 2008

    Article  PubMed  Google Scholar 

  56. Cai W, Feng D, Fulton R: Content-based retrieval of dynamic pet functional images. IEEE Trans Inf Technol Biomed 4(2):152–8, 2000

    Article  PubMed  CAS  Google Scholar 

  57. Kim J, Cai W, Feng D, Wu H: A new way for multidimensional medical data management: Volume of interest (VOI)-based retrieval of medical images with visual and functional features. IEEE Trans Inf Technol Biomed 10(3):598–607, 2006

    Article  PubMed  Google Scholar 

  58. Kim J, Constantinescu L, Cai W, Feng DD: Content-based dual-modality biomedical data retrieval using co-aligned functional and anatomical features. In: Proceedings of the MICCAI Workshop on Content-Based Image Retrieval for Biomedical Image Archives: Achievements, Problems and Prospects, 2007, pp 45–52

  59. Song Y, Cai W, Eberl S, Fulham M, Feng D: A content-based image retrieval framework for multi-modality lung images. In: IEEE International Symposium on Computer-Based Medical Systems, 2010, pp 285–290

  60. Song Y, Cai W, Eberl S, Fulham M, Feng D: Structure-adaptive feature extraction and representation for multi-modality lung images retrieval. In: International Conference on Digital Image Computing: Techniques and Applications, 2010, pp 152–157

  61. Song Y, Cai W, Eberl S, Fulham M, Feng D: Thoracic image case retrieval with spatial and contextual information. In: 2011 I.E. International Symposium on Biomedical Imaging: From Nano to Macro, 2011, pp 1885–1888

  62. Song Y, Cai W, Eberl S, Fulham M, Feng D: Thoracic image matching with appearance and spatial distribution. In: International Conference of the IEEE Engineering in Medicine and Biology Society, 2011, pp 4469–4472

  63. Song Y, Cai W, Feng D: Hierarchical spatial matching for medical image retrieval. In: Proceedings of the International ACM Multimedia Workshop on Medical Multimedia Analysis and Retrieval, 2011, pp 1–6

  64. Cai W, Song Y, Feng DD: Regression and classification based distance metric learning for medical image retrieval. In: IEEE International Symposium on Biomedical Imaging, 2012, pp 1775–1778

  65. Kumar A, Kim J, Cai W, Eberl S, Feng D: A graph-based approach to the retrieval of dual-modality biomedical images using spatial relationships. In: International Conference of the IEEE Engineering in Medicine and Biology Society, 2008, pp 390–393

  66. Kumar A, Kim J, Wen L, Feng D: A graph-based approach to the retrieval of volumetric PET-CT lung images. In: Proceedings of the 34th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2012, pp 5408–5411

  67. Kumar A, Kim J, Fulham M, Feng D: Graph-based retrieval of multi-modality medical images: A comparison of representations using simulated images. In: IEEE International Symposium on Computer-Based Medical Systems, 2012, pp 1–6

  68. Kumar A, Haraguchi D, Kim J, Wen L, Eberl S, Fulham M, et al: A query and visualisation interface for a PET-CT image retrieval system. Int J Comput Assist Radiol Surg 6(Supplement 1):69, 2011

    Google Scholar 

  69. Kumar A, Kim J, Bi L, Feng D: An image retrieval interface for volumetric multi-modal medical data: Application to PET-CT content-based image retrieval. Int J Comput Assist Radiol Surg 7(Supplement 1):475–7, 2012

    Google Scholar 

  70. Radhouani S, Lim J, Chevallet JP, Falquet G: Combining textual and visual ontologies to solve medical multimodal queries. In: IEEE International Conference on Multimedia and Expo, 2006, pp 1853–1856

  71. Lacoste C, Lim JH, Chevallet JP, Le D: Medical-image retrieval based on knowledge-assisted text and image indexing. IEEE Trans Circ Syst Video Technol 17(7):889–900, 2007

    Article  Google Scholar 

  72. Gobeill J, Müller H, Ruch P: Translation by text categorisation: Medical image retrieval in ImageCLEFmed 2006. In: Peters C, Clough P, Gey F, Karlgren J, Magnini B, Oard D, et al. Eds. Evaluation of Multilingual and Multi-modal Information Retrieval, Vol. 4730 of Lecture Notes in Computer Science, 2007, pp 706–710

  73. Villena-Román J, Lana-Serrano S, González-Cristóbal J: MIRACLE at ImageCLEFmed 2007: Merging textual and visual strategies to improve medical image retrieval. In: Peters C, Jijkoun V, Mandl T, Müller H, Oard D, Peñas A, et al. Eds. Advances in Multilingual and Multimodal Information Retrieval, Vol. 5152 of Lecture Notes in Computer Science, 2008, pp 593–596

  74. Caicedo JC, Moreno JG, Niño EA, González FA: Combining visual features and text data for medical image retrieval using latent semantic kernels. In: Proceedings of the International Conference on Multimedia Information Retrieval, ACM, 2010, pp 359–366

  75. Rahman M, Antani S, Long R, Demner-Fushman D, Thoma G: Multi-modal query expansion based on local analysis for medical image retrieval. In: Caputo B, Müller H, Syeda-Mahmood T, Duncan J, Wang F, Kalpathy-Cramer J Eds. Medical Content-Based Retrieval for Clinical Decision Support, Vol. 5853 of Lecture Notes in Computer Science, 2010, pp 110–119

  76. Müller H, Kalpathy-Cramer J, Charles E. Kahn J, Hersh W: Comparing the quality of accessing medical literature using content-based visual and textual information retrieval. In: Siddiqui KM, Liu BJ Eds. Proceedings of SPIE 7264, 2009, pp 726405:1–726405:11

  77. Chu WW, Ieong IT, Taira RK: A semantic modeling approach for image retrieval by content. VLDB J—Int J Very Large Data Bases 3(4):445–77, 1994

    Article  Google Scholar 

  78. Chu W, Hsu CC, Cardenas A, Taira R: Knowledge-based image retrieval with spatial and temporal constructs. IEEE Trans Knowl Data Eng 10(6):872–88, 1998

    Article  Google Scholar 

  79. Névéol A, Deserno TM, Darmoni SJ, Güld MO, Aronson AR: Natural language processing versus content-based image analysis for medical document retrieval. J Am Soc Inf Sci Technol 60(1):123–34, 2009

    Article  Google Scholar 

  80. Langlotz CP: RadLex: A new method for indexing online educational materials. Radiographics 26(6):1595–7, 2006

    Article  PubMed  Google Scholar 

  81. Müller H, Deselaers T, Deserno T, Kalpathy-Cramer J, Kim E, Hersh W: Overview of the ImageCLEFmed 2007 medical retrieval and medical annotation tasks. In: Peters C, Jijkoun V, Mandl T, Müller H, Oard D, Peñas A, et al. Eds. Advances in Multilingual and Multimodal Information Retrieval, Vol. 5152 of Lecture Notes in Computer Science, 2008, pp 472–491

  82. Müller H, Kalpathy-Cramer J, Kahn C, Hatt W, Bedrick S, Hersh W: Overview of the ImageCLEFmed 2008 medical image retrieval task. In: Peters C, Deselaers T, Ferro N, Gonzalo J, Jones G, Kurimo M, et al. Eds. Evaluating Systems for Multilingual and Multimodal Information Access, Vol. 5706 of Lecture Notes in Computer Science, 2009, pp 512–522

  83. Müller H, Kalpathy-Cramer J, Eggel I, Bedrick S, Radhouani S, Bakke B, et al: Overview of the CLEF 2009 medical image retrieval track. In: Peters C, Caputo B, Gonzalo J, Jones G, Kalpathy-Cramer J, Müller H, et al. Eds. Multilingual Information Access Evaluation II. Multimedia Experiments, Vol. 6242 of Lecture Notes in Computer Science, 2010, pp 72–84

  84. Liu J, Hu Y, Li M, Ma S, ying Ma W: Medical image annotation and retrieval using visual features. In: Evaluation of Multilingual and Multi-modal Information Retrieval, Vol. 4730 of Lecture Notes in Computer Science, 2007, pp 678–685

  85. Rahman MM, Desai BC, Bhattacharya P: Medical image retrieval with probabilistic multi-class support vector machine classifiers and adaptive similarity fusion. Comput Med Imaging Graph 32(2):95–108, 2008

    Article  PubMed  Google Scholar 

  86. Akakin H, Gurcan M: Content-based microscopic image retrieval system for multi-image queries. IEEE Trans Inf Technol Biomed 16(4):758–69, 2012

    Article  PubMed  Google Scholar 

  87. Allampalli-Nagaraj G, Bichindaritz I: Automatic semantic indexing of medical images using a web ontology language for case-based image retrieval. Eng Appl Artif Intell 22(1):18–25, 2009

    Article  Google Scholar 

  88. Zhou X, Stern R, Müller H: Case-based fracture image retrieval. Int J Comput Assist Radiol Surg 7:401–11, 2012

    Article  PubMed  Google Scholar 

  89. Huang SC, Phelps ME, Hoffman EJ, Sideris K, Selin CJ, Kuhl DE: Noninvasive determination of local cerebral metabolic rate of glucose in man. Am J Physiol—Endocrinol Metab 238(1):E69–82, 1980

    CAS  Google Scholar 

  90. Chang E, Goh K, Sychay G, Wu G: CBSA: Content-based soft annotation for multimodal image retrieval using Bayes point machines. IEEE Trans Circ Syst Video Technol 13(1):26–38, 2003

    Article  Google Scholar 

  91. Hersh W, Müller H, Kalpathy-Cramer J: The ImageCLEFmed medical image retrieval task test collection. J Digit Imaging 22:648–55, 2009

    Article  PubMed  Google Scholar 

  92. Tian G, Fu H, Feng D: Automatic medical image categorization and annotation using LBP and MPEG-7 edge histograms. In: International Conference on Information Technology and Applications in Biomedicine, 2008, pp 51–53

  93. Spitzer V, Ackerman MJ, Scherzinger AL, Whitlock D: The visible human male: A technical report. J Am Med Inform Assoc 3(2):118–30, 1996

    Article  PubMed  CAS  Google Scholar 

  94. Lowe DG: Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110, 2004

    Article  Google Scholar 

  95. Czernin J, Dahlbom M, Ratib O, Schiepers C: Atlas of PET/CT Imaging in Oncology. Springer, Berlin, 2004

    Book  Google Scholar 

  96. Goerres GW, von Schulthess GK, Steinert HC: Why most PET of lung and head-and-neck cancer will be PET/CT. J Nucl Med 45(Supplement 1):66S–71S, 2004

    PubMed  Google Scholar 

  97. Fu KS: A step towards unification of syntactic and statistical pattern recognition. IEEE Trans Pattern Anal Mach Intell 8(3):398–404, 1986

    Article  PubMed  CAS  Google Scholar 

  98. Jing Y, Rowley H, Rosenberg C, Wang J, Zhao M, Covell M: Google image swirl, a large-scale content-based image browsing system. In: IEEE International Conference on Multimedia and Expo, 2010, p 267

  99. Tory M, Moller T: Human factors in visualization research. IEEE Trans Vis Comput Graph 10(1):72–84, 2004

    Article  PubMed  Google Scholar 

  100. Wilson ML: Search user interface design. Synth Lect Inf Concepts Retr Serv 3(3):1–143, 2011

    CAS  Google Scholar 

  101. Etzold J, Brousseau A, Grimm P, Steiner T: Context-aware querying for multimodal search engines. In: Schoeffmann K, Merialdo B, Hauptmann A, Ngo CW, Andreopoulos Y, Breiteneder C Eds. Advances in Multimedia Modeling, Vol. 7131 of Lecture Notes in Computer Science. Berlin: Springer, 2012, pp 728–739

  102. Ekin A, Jasinschi R, van der Grond J, Van Buchem M: Improving information quality of MR brain images by fully automatic and robust image analysis methods. J Soc Inf Disp 15(6):367–76, 2007

    Article  Google Scholar 

  103. van Rikxoort EM, Isgum I, Arzhaeva Y, Staring M, Klein S, Viergever MA, et al: Adaptive local multi-atlas segmentation: Application to the heart and the caudate nucleus. Medical Image Analysis 14(1):39–49, 2010

    Article  PubMed  Google Scholar 

  104. Jones KN, Woode DE, Panizzi K, Anderson PG: PEIR digital library: Online resources and authoring system. In: Proceedings of the American Medical Informatics Association Symposium, 2001, p 1075

  105. Long LR, Antani SK, Thoma GR: Image informatics at a national research center. Comput Med Imaging Graph 29(2–3):171–93, 2005

    Article  PubMed  Google Scholar 

  106. The Cancer Imaging Archive. 2011. http://cancerimagingarchive.net/

  107. Armato III, SG, McLennan G, McNitt-Gray MF, Meyer CR, Yankelevitz D, Aberle DR, et al: Lung image database consortium: Developing a resource for the medical imaging research community. Radiology 232(3):739–48, 2004

    Article  PubMed  Google Scholar 

  108. Langs G, Müller H, Menze BH, Hanbury A: VISCERAL: Towards large data in medical imaging—Challenges and directions. In: MICCAI Workshop on Medical Content-Based Retrieval for Clinical Decision Support 2012, Vol. 7723 of Springer LNCS, 2013, pp 92–98

  109. Detterbeck FC, Boffa DJ, Tanoue LT: The new lung cancer staging system. Chest 136(1):260–71, 2009

    Article  PubMed  Google Scholar 

  110. Edge SB, Byrd DR, Compton CC, Frtiz AG, Greene FL, Trotti A Eds. AJCC Cancer Staging Manual. New York: Springer, 2010

  111. Bodenreider O: The unified medical language system (UMLS): Integrating biomedical terminology. Nucleic Acids Res 32(Supplement 1):D267–70, 2004

    Article  PubMed  CAS  Google Scholar 

  112. Depeursinge A, Vargas A, Gaillard F, Platon A, Geissbuhler A, Poletti PA, et al: Case-based lung image categorization and retrieval for interstitial lung diseases: Clinical workflows. Int J Comput Assist Radiol Surg 7(1):97–110, 2012

    Article  PubMed  Google Scholar 

  113. Antani S, Xue Z, Long LR, Bennett D, Ward S, Thoma GR: Is there a need for biomedical CBIR systems in clinical practice? Outcomes from a usability study. In: Proceedings of SPIE 7967, 2011, pp 796708-1–796708-7

Download references

Acknowledgments

We are grateful to our collaborators at the Royal Prince Alfred Hospital, Sydney, Australia for their direct and indirect contributions to this work. This work was supported in part by ARC grants.

Author information

Authors and Affiliations

  1. Biomedical and Multimedia Information Technology (BMIT) Research Group, School of Information Technologies, University of Sydney, Building J12, Sydney, NSW, 2006, Australia

    Ashnil Kumar, Jinman Kim, Weidong Cai, Michael Fulham & Dagan Feng

  2. Department of Molecular Imaging, Royal Prince Alfred Hospital, Sydney, Australia

    Michael Fulham

  3. Sydney Medical School, University of Sydney, Sydney, Australia

    Michael Fulham

  4. Med-X Research Institute, Shanghai Jiao Tong University, Shanghai, China

    Dagan Feng

Authors
  1. Ashnil Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jinman Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weidong Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Fulham
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dagan Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ashnil Kumar.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kumar, A., Kim, J., Cai, W. et al. Content-Based Medical Image Retrieval: A Survey of Applications to Multidimensional and Multimodality Data. J Digit Imaging 26, 1025–1039 (2013). https://doi.org/10.1007/s10278-013-9619-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10278-013-9619-2

Keywords

Search

Navigation