Abstract
Recent works in deep learning using Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) models have yielded state of the art results on a variety of image processing tasks. Multimodal representation, especially Image captioning is gaining popularity due to their primordial role in constricting heterogeneity gap among different modalities which are very helpful in cross-modality analysis tasks. The uncountable amounts of medical images, as well as medical documents, need to be processed to discover hidden knowledge. The purpose of this research is to present biomedical information retrieval system in order to know more about their strengths and weakness. Then we will propose our approach that tries to resolve some gaps and gives some solution to the existing systems and engine retrieval by giving an insight into the images captioning benefit in cross-modality retrieval.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Somai, S.B., Karaa, W.B.A., Ghezala, H.H.B.: Deep learning and semantic medical image processing and retrieval: datasets, software, applications and perspectives. In: The 31st IBIMA Conference, Milan, April (2018)
Hochreiter, S., Schmidhuber, J.: Long short term memory. Neural Comput. 9(8), 1735–1780 (1997)
Shin, H.C., Roberts, K., Lu, L., Demner-Fushman, D., Yao, J., Summers, R.M.: Learning to read chest x-rays: recurrent neural cascade model for automated image annotation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2497–2506 (2016)
Wang, X., et al.: Unsupervised category discovery via looped deep pseudo-task optimization using a large scale radiology image database (2016). arXiv preprint arXiv:1603.07965
Xue, Y., et al.: Multimodal recurrent model with attention for automated radiology report generation. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 457–466. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_52
Jing, B., Xie, P., Xing, E.: On the automatic generation of medical imaging reports (2017). arXiv preprint arXiv:1711.08195
Pavlopoulos, J., Kougia, V., Androutsopoulos, I.: A survey on biomedical image captioning. In: Proceedings of the Second Workshop on Shortcomings in Vision and Language, Minneapolis, USA, 6 June 2019, pp. 26–36 (2019)
Azizpour, H., Sharif Razavian, A., Sullivan, J., Maki, A., Carlsson, S.: From generic to specific deep representations for visual recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 36–45 (2015)
Zulueta, M.A., Moreno, L., Bordons, M., Vázquez, M.: Biblimed: a new Spanish database in clinical medicine. In: Bakker, S. (ed.) Health Information Management: What Strategies?, pp. 173–176. Springer, Dordrecht (1997). https://doi.org/10.1007/978-94-015-8786-0_56
Névéol, A., Soualmia, L.F., Douyère, M., Rogozan, A., Thirion, B., Darmoni, S.J.: Using CISMeF MeSH “Encapsulated” terminology and a categorization algorithm for health resources. Int. J. Med. Inf. 73(1), 57–64 (2004)
Yu, H., Cao, Y.G.: Automatically extracting information needs from ad hoc clinical questions. In: AMIA Annual Symposium Proceedings, vol. 2008, p. 96. American Medical Informatics Association (2008)
Hearst, M.A., et al.: BioText search engine: beyond abstract search. Bioinformatics 23(16), 2196–2197 (2007)
Jeannot, J.G.: Dr Google: moteur-de-recherche-medical.org, un moteur de recherche Google pour la medicine. Rev Med Suisse 4, 1280–1284 (2008)
Müller, H., Geissbühler, A., Marty, J., Lovis, C., Ruch, P.: The use of MedGIFT and EasyIR for ImageCLEF 2005. In: Peters, C. (ed.) CLEF 2005. LNCS, vol. 4022, pp. 724–732. Springer, Heidelberg (2006). https://doi.org/10.1007/11878773_79
Lim, J., Chevallet, J.: VisMed: a visual vocabulary approach for medical image indexing and retrieval. In: Proceedings of Asia Information Retrieval Symposium, pp. 84–96 (2005)
Lam, M.O., Disney, T., Raicu, D.S., Furst, J., Channin, D.S.: BRISC—an open source pulmonary nodule image retrieval framework. J. Digit. Imag. 20(1), 63–71 (2007)
Hussain, A., Gul, S., Shah, T.A., Shueb, S.: Retrieval effectiveness of image search engines. Electron. Libr. 37(1), 173–184 (2019)
Beam, A.L., et al.: Clinical Concept Embeddings Learned from Massive Sources of Multimodal Medical Data (2018). arXiv preprint arXiv:1804.01486
Chen, Q., Peng, Y., Lu, Z.: BioSentVec: creating sentence embeddings for biomedical texts (2018). arXiv preprint arXiv:1810.09302
Agarwal, K., Eftimov, T., Addanki, R., Choudhury, S., Tamang, S., Rallo, R.: Snomed2Vec: random walk and poincaré embeddings of a clinical knowledge base for healthcare analytics. In: 2019 KDD Workshop on Applied Data Science for Healthcare, DSHealth ‘19 (2019)
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR_2016, pp. 2818–2826 (2016)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, June 2009, pp. 248–255. IEEE (2009)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Deng, J., Berg, A.C., Li, K., Fei-Fei, L.: What does classifying more than 10,000 image categories tell us? In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6315, pp. 71–84. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15555-0_6
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization (2014). arXiv preprint arXiv:1412.6980
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Benzarti, S., Ben Abdessalem Karaa, W., Hajjami Ben Ghezala, H. (2021). Cross-Model Retrieval Via Automatic Medical Image Diagnosis Generation. In: Abraham, A., Siarry, P., Ma, K., Kaklauskas, A. (eds) Intelligent Systems Design and Applications. ISDA 2019. Advances in Intelligent Systems and Computing, vol 1181. Springer, Cham. https://doi.org/10.1007/978-3-030-49342-4_54
Download citation
DOI: https://doi.org/10.1007/978-3-030-49342-4_54
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-49341-7
Online ISBN: 978-3-030-49342-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)