Abstract
Segmentation of Prostate Cancer (PCa) tissues from Gleason graded histopathology images is vital for accurate diagnosis. Although deep learning (DL) based segmentation methods achieve state-of-the-art accuracy, they rely on large datasets with manual annotations. We propose a method to synthesize PCa histopathology images by learning the geometrical relationship between different disease labels using self-supervised learning. Manual segmentation maps from the training set are used to train a Shape Restoration Network (ShaRe-Net) that predicts missing mask segments in a self-supervised manner. Using DenseUNet as the backbone generator architecture we incorporate latent variable sampling to inject diversity in the image generation process and thus improve robustness. Experimental results demonstrate the superiority of our method over competing image synthesis methods for segmentation tasks. Ablation studies show the benefits of integrating geometry and diversity in generating high-quality images. Our self-supervised approach with limited class-labeled data achieves better performance than fully supervised learning.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Antoniou, A., Storkey, A., Edwards, H.: Data augmentation generative adversarial networks. arXiv preprint arXiv:1711.04340 (2017)
Baumgartner, C.F., et al.: PHiSeg: capturing uncertainty in medical image segmentation. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 119–127. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_14
Bozorgtabar, B., Mahapatra, D., Vray, G., Thiran, J.-P.: SALAD: self-supervised aggregation learning for anomaly detection on X-rays. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12261, pp. 468–478. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_46
Bozorgtabar, B., et al.: Informative sample generation using class aware generative adversarial networks for classification of chest Xrays. Comput. Vis. Image Underst. 184, 57–65 (2019)
Campanella, G., Silva, V., Fuchs, T.: Terabyte-scale deep multiple instance learning for classification and localization in pathology. arXiv preprint arXiv:1805.06983 (2018)
Chen, L., Bentley, P., Mori, K., Misawa, K., Fujiwara, M., Rueckert, D.: Self-supervised learning for medical image analysis using image context restoration. Medical Imag. Anal. 58, 1–12 (2019)
Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)
Gupta, L., Klinkhammer, B.M., Boor, P., Merhof, D., Gadermayr, M.: GAN-based image enrichment in digital pathology boosts segmentation accuracy. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 631–639. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_70
Han, C., et al.: GAN-based synthetic brain MR image generation. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), pp. 734–738. IEEE (2018)
Ing, N., et al.: Semantic segmentation for prostate cancer grading by convolutional neural networks. In: Medical Imaging 2018: Digital Pathology, pp. 343–355 (2018)
Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K.: Spatial transformer networks. In: NIPS (2015)
Jamaludin, A., Kadir, T., Zisserman, A.: Self-supervised learning for spinal MRIs. In: Cardoso, M.J., et al. (eds.) DLMIA/ML-CDS 2017. LNCS, vol. 10553, pp. 294–302. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67558-9_34
Karimi, D., Nir, G., Fazli, L., Black, P., Goldenberg, L., Salcudean, S.: Deep learning-based Gleason grading of prostate cancer from histopathology images-role of multiscale decision aggregation and data augmentation. IEEE J. Biomed. Health Inform. 24(5), 1413–1426 (2020)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Koohbanani, N.A., Unnikrishnan, B., Khurram, S.A., Krishnaswamy, P., Rajpoot, N.: Self-path: self-supervision for classification of pathology images with limited annotations. arXiv:2008.05571 (2020)
Leo, P., et al.: Stable and discriminating features are predictive of cancer presence and Gleason grade in radical prostatectomy specimens: a multi-site study. Sci. Rep. 8, 1–13 (2018)
Li, X., Chen, H., Qi, X., Dou, Q., Fu, C.W., Heng, P.A.: H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Trans. Med. Imag. 37(12), 2663–2674 (2018)
Lu, M.Y., Chen, R.J., Wang, J., Dillon, D., Mahmood, F.: Semi-supervised histology classification using deep multiple instance learning and contrastive predictive coding. arXiv:1910.10825 (2019)
Mahapatra, D., Antony, B., Sedai, S., Garnavi, R.: Deformable medical image registration using generative adversarial networks. In: Proceedings of the IEEE ISBI, pp. 1449–1453 (2018)
Mahapatra, D., Bozorgtabar, B., Shao, L.: Pathological retinal region segmentation from oct images using geometric relation based augmentation. In: Proceedings of the IEEE CVPR, pp. 9611–9620 (2020)
Mahapatra, D., Bozorgtabar, B., Thiran, J.-P., Shao, L.: Structure preserving stain normalization of histopathology images using self supervised semantic guidance. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12265, pp. 309–319. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59722-1_30
Mahapatra, D., Ge, Z.: Training data independent image registration with GANs using transfer learning and segmentation information. In: Proceedings of the IEEE ISBI, pp. 709–713 (2019)
Mahapatra, D., Bozorgtabar, B., Thiran, J.-P., Reyes, M.: Efficient active learning for image classification and segmentation using a sample selection and conditional generative adversarial network. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11071, pp. 580–588. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00934-2_65
Mahapatra, D., Ge, Z.: Training data independent image registration using generative adversarial networks and domain adaptation. Pattern Recogn. 100, 1–14 (2020, in press)
Mahapatra, D., Poellinger, A., Shao, L., Reyes, M.: Interpretability-driven sample selection using self supervised learning for disease classification and segmentation. IEEE TMI, pp. 1–15 (2021)
Milletari, F., Navab, N., Ahmadi, S.A.: V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: Proceedings of the International Conference on 3D vision, pp. 565–571 (2016)
Muhammad, H., et al.: Towards unsupervised cancer subtyping: predicting prognosis using a histologic visual dictionary. arXiv:1903.05257 (2019)
Nielsen, C., Okoniewski, M.: GAN data augmentation through active learning inspired sample acquisition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 109–112 (2019)
Persson, J., et al.: Interobserver variability in the pathological assessment of radical prostatectomy specimens: findings of the laparoscopic prostatectomy robot open (LAPPRO) study. Scand. J. Urol. 48(2), 160–167 (2014)
Tong, J., Mahapatra, D., Bonnington, P., Drummond, T., Ge, Z.: Registration of histopathology images using self supervised fine grained feature maps. In: Albarqouni, S., et al. (eds.) DART/DCL 2020. LNCS, vol. 12444, pp. 41–51. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60548-3_5
Xue, Y., et al.: Synthetic augmentation and feature-based filtering for improved cervical histopathology image classification. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11764, pp. 387–396. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32239-7_43
Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: UNet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans. Med. Imag. 39, 1–10 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Mahapatra, D., Kuanar, S., Bozorgtabar, B., Ge, Z. (2021). Self-supervised Learning of Inter-label Geometric Relationships for Gleason Grade Segmentation. In: Albarqouni, S., et al. Domain Adaptation and Representation Transfer, and Affordable Healthcare and AI for Resource Diverse Global Health. DART FAIR 2021 2021. Lecture Notes in Computer Science(), vol 12968. Springer, Cham. https://doi.org/10.1007/978-3-030-87722-4_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-87722-4_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87721-7
Online ISBN: 978-3-030-87722-4
eBook Packages: Computer ScienceComputer Science (R0)
