Abstract
Purpose
Multimodal articulated image registration (MAIR) is a challenging problem because the resulting transformation needs to maintain rigidity for bony structures while allowing elastic deformation for surrounding soft tissues. Existing deep learning-based methods ignore the articulated structures and consider it as a pure deformable registration problem, leading to suboptimal results.
Methods
We propose a novel weakly supervised anatomy-aware multimodal articulated image registration network, referred as MAIRNet, to solve the challenging problem. The architecture of MAIRNet comprises of two branches: a non-learnable polyrigid registration branch to estimate an initial velocity field, and a learnable deformable registration branch to learn an increment. These two branches work together to produce a velocity field that can be integrated to generate the final displacement field.
Results
We designed and conducted comprehensive experiments on three datasets to evaluate the performance of the proposed method. Specifically, on the hip dataset, our method achieved, respectively, an average dice of 90.8%, 92.4% and 91.3% for the pelvis, the right femur, and the left femur. On the lumbar spinal dataset, our method obtained, respectively, an average dice of 86.1% and 85.9% for the L4 and the L5 vertebrae. On the thoracic spinal dataset, our method achieved, respectively, an average dice of 76.7%, 79.5%, 82.9%, 85.5% and 85.7% for the five thoracic vertebrae ranging from T6 to T10.
Conclusion
In summary, we developed a novel approach for multimodal articulated image registration. Comprehensive experiments conducted on three typical yet challenging datasets demonstrated the efficacy of the present approach. Our method achieved better results than the state-of-the-art approaches.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Gill S, Abolmaesumi P, Fichtinger G, Boisvert J, Pichora D, Borshneck D, Mousavi P (2012) Biomechanically constrained groupwise ultrasound to CT registration of the lumbar spine. Med Image Anal 16(3):662–674
Arsigny V, Commowick O, Pennec X, Ayache N ( 2006) A log-euclidean polyaffine framework for locally rigid or affine registration. In: WBIR 2006, Utrecht, The Netherlands, July 9–11, 2006. Proceedings 3. Springer, pp 120–127
Loeckx D, Maes F, Vandermeulen D, Suetens P ( 2004) Nonrigid image registration using free-form deformations with a local rigidity constraint. In: MICCAI 2004, Saint-Malo, France, September 26–29, 2004. Proceedings, Part I 7. Springer, pp 639–646
Staring M, Klein S, Pluim JP (2007) A rigidity penalty term for nonrigid registration. Med Phys 34(11):4098–4108
Reaungamornrat S, Wang A, Uneri A, Otake Y, Khanna A, Siewerdsen J (2014) Deformable image registration with local rigidity constraints for cone-beam CT-guided spine surgery. Phys Med Biol 59(14):3761
Modersitzki J (2008) Flirt with rigidity-image registration with a local non-rigidity penalty. Int J Comput Vis 76:153–163
Rohlfing T, Maurer CR, Bluemke DA, Jacobs MA (2003) Volume-preserving nonrigid registration of MR breast images using free-form deformation with an incompressibility constraint. IEEE Trans Med Imaging 22(6):730–741
Balakrishnan G, Zhao A, Sabuncu MR, Guttag J, Dalca AV (2019) VoxelMorph: a learning framework for deformable medical image registration. IEEE Trans Med Imaging 38(8):1788–1800
Dalca AV, Balakrishnan G, Guttag J, Sabuncu MR (2019) Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces. Med Image Anal 57:226–236
Jian B, Azampour MF, De Benetti F, Oberreuter J, Bukas C, Gersing AS, Foreman SC, Dietrich A-S, Rischewski J, Kirschke JS, Navab N (2022) Weakly-supervised biomechanically-constrained CT/MRI Registration of the spine. In: MICCAI 2022, Singapore, September 18–22, 2022, Proceedings, Part VI. Springer, pp 227–236
Zhao L, Pang S, Chen Y, Zhu X, Jiang Z, Su Z, Lu H, Zhou Y, Feng Q (2023) SpineRegNet: Spine Registration Network for volumetric MR and CT image by the joint estimation of an affine-elastic deformation field. Med Image Anal 86:102786
Chu C, Bai J, Wu X, Zheng G (2015) MASCG: multi-atlas segmentation constrained graph method for accurate segmentation of hip CT images. Med Image Anal 26(1):173–184
Tao R, Liu W, Zheng G (2022) Spine-transformers: vertebra labeling and segmentation in arbitrary field-of-view spine CTs via 3D transformers. Med Image Anal 75:102258
Commowick O, Arsigny V, Isambert A, Costa J, Dhermain F, Bidault F, Bondiau P-Y, Ayache N, Malandain G (2008) An efficient locally affine framework for the smooth registration of anatomical structures. Med Image Anal 12(4):427–441
Yang H, Sun J, Carass A, Zhao C, Lee J, Xu Z, Prince J (2018) Unpaired brain MR-to-CT synthesis using a structure-constrained CycleGan. In: In: DLMIA 2018 and ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, September 20, 2018, Proceedings 4. Springer, pp 174–182
Heinrich MP, Jenkinson M, Bhushan M, Matin T, Gleeson FV, Brady M, Schnabel JA (2012) Mind: modality independent neighbourhood descriptor for multi-modal deformable registration. Med Image Anal 16(7):1423–1435
Cai Y, Osman S, Sharma M, Landis M, Li S (2015) Multi-modality vertebra recognition in arbitrary views using 3D deformable hierarchical model. IEEE Trans Med Imaging 34(8):1676–1693
Funding
This research is supported by the National Key R &D Program of China (Grant No. 2023YFB4706302), by Beijing Natural Science Foundation (Grant No. L202010), and by the National Natural Science Foundation of China (Grant No. U20A20199).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethics approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Informed consent
Informed context was obtained from all individuals included in the study.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Gao, X., Zhong, W., Wang, R. et al. MAIRNet: weakly supervised anatomy-aware multimodal articulated image registration network. Int J CARS 19, 507–517 (2024). https://doi.org/10.1007/s11548-023-03056-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-023-03056-0