Abstract
2D cardiac MR cine images provide data with a high signal-to-noise ratio for the segmentation and reconstruction of the heart. These images are frequently used in clinical practice and research. However, the segments have low resolution in the through-plane direction, and standard interpolation methods are unable to improve resolution and precision. We proposed an end-to-end pipeline for producing high-resolution segments from 2D MR images. This pipeline utilised a bilateral optical flow warping method to recover images in the through-plane direction, while a SegResNet automatically generated segments of the left and right ventricles. A multi-modal latent-space self-alignment network was implemented to guarantee that the segments maintain an anatomical prior derived from unpaired 3D high-resolution CT scans. On 3D MR angiograms, the trained pipeline produced high-resolution segments that preserve an anatomical prior derived from patients with various cardiovascular diseases.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alfakih, K., Reid, S., Jones, T., Sivananthan, M.: Assessment of ventricular function and mass by cardiac magnetic resonance imaging. Eur. Radiol. 14(10), 1813–1822 (2004)
Barkhof, F., Pouwels, P.J., Wattjes, M.P.: The holy grail in diagnostic neuroradiology: 3T OR 3D? (2011)
Bernard, O., et al.: Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE Trans. Med. Imaging 37(11), 2514–2525 (2018)
Biffi, C., et al.: 3D high-resolution cardiac segmentation reconstruction from 2D views using conditional variational autoencoders. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), pp. 1643–1646. IEEE (2019)
Delannoy, Q., et al.: SegSRGAN: super-resolution and segmentation using generative adversarial networks-application to neonatal brain MRI. Comput. Biol. Med. 120, 103755 (2020)
Fedorov, A., et al.: 3D slicer as an image computing platform for the quantitative imaging network. Magn. Reson. Imaging 30(9), 1323–1341 (2012)
Greenspan, H., Oz, G., Kiryati, N., Peled, S.: MRI inter-slice reconstruction using super-resolution. Magn. Reson. Imaging 20(5), 437–446 (2002)
Kingma, D.P., Ba, J.: Adam, a method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations (ICLR), vol. 1412 (2015)
Mauger, C.A., et al.: Right-left ventricular shape variations in tetralogy of Fallot: associations with pulmonary regurgitation. J. Cardiovasc. Magn. Reson. 23(1), 1–14 (2021)
Myronenko, A.: 3D MRI brain tumor segmentation using autoencoder regularization. In: Crimi, A., Bakas, S., Kuijf, H., Keyvan, F., Reyes, M., van Walsum, T. (eds.) BrainLes 2018. LNCS, vol. 11384, pp. 311–320. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11726-9_28
Oktay, O., et al.: Anatomically constrained neural networks (ACNNs): application to cardiac image enhancement and segmentation. IEEE Trans. Med. Imaging 37(2), 384–395 (2017)
Park, J., Ko, K., Lee, C., Kim, C.-S.: BMBC: bilateral motion estimation with bilateral cost volume for video interpolation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12359, pp. 109–125. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58568-6_7
Park, J., Lee, C., Kim, C.S.: Asymmetric bilateral motion estimation for video frame interpolation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14539–14548 (2021)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Rousseau, F., Habas, P.A., Studholme, C.: A supervised patch-based approach for human brain labeling. IEEE Trans. Med. Imaging 30(10), 1852–1862 (2011)
Sohn, K.: Improved deep metric learning with multi-class N-pair loss objective. Adv. Neural Inf. Process. Syst. 29 (2016)
Wang, S., et al.: Joint motion correction and super resolution for cardiac segmentation via latent optimisation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12903, pp. 14–24. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87199-4_2
Wolberg, G.: Digital Image Warping, vol. 10662. IEEE Computer Society Press, Los Alamitos (1990)
Xue, T., Chen, B., Wu, J., Wei, D., Freeman, W.T.: Video enhancement with task-oriented flow. Int. J. Comput. Vision 127(8), 1106–1125 (2019)
Zhuang, X.: Multivariate mixture model for myocardial segmentation combining multi-source images. IEEE Trans. Pattern Anal. Mach. Intell. 41(12), 2933–2946 (2018)
Zhuang, X., Shen, J.: Multi-scale patch and multi-modality atlases for whole heart segmentation of MRI. Med. Image Anal. 31, 77–87 (2016)
Acknowledgement
This work was supported by the Kings-China Scholarship Council PhD Scholarship Program and the National Institutes of Health Grant (R01HL121754).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Deng, Y. et al. (2022). Multi-modal Latent-Space Self-alignment for Super-Resolution Cardiac MR Segmentation. In: Camara, O., et al. Statistical Atlases and Computational Models of the Heart. Regular and CMRxMotion Challenge Papers. STACOM 2022. Lecture Notes in Computer Science, vol 13593. Springer, Cham. https://doi.org/10.1007/978-3-031-23443-9_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-23443-9_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-23442-2
Online ISBN: 978-3-031-23443-9
eBook Packages: Computer ScienceComputer Science (R0)
