Physics Contribution
Comparison of Deep Learning-Based and Patch-Based Methods for Pseudo-CT Generation in MRI-Based Prostate Dose Planning

https://doi.org/10.1016/j.ijrobp.2019.08.049Get rights and content

Purpose

Deep learning methods (DLMs) have recently been proposed to generate pseudo-CT (pCT) for magnetic resonance imaging (MRI) based dose planning. This study aims to evaluate and compare DLMs (U-Net and generative adversarial network [GAN]) using various loss functions (L2, single-scale perceptual loss [PL], multiscale PL, weighted multiscale PL) and a patch-based method (PBM).

Methods and Materials

Thirty-nine patients received a volumetric modulated arc therapy for prostate cancer (78 Gy). T2-weighted MRIs were acquired in addition to planning CTs. The pCTs were generated from the MRIs using 7 configurations: 4 GANs (L2, single-scale PL, multiscale PL, weighted multiscale PL), 2 U-Net (L2 and single-scale PL), and the PBM. The imaging endpoints were mean absolute error and mean error, in Hounsfield units, between the reference CT (CTref) and the pCT. Dose uncertainties were quantified as mean absolute differences between the dose volume histograms (DVHs) calculated from the CTref and pCT obtained by each method. Three-dimensional gamma indexes were analyzed.

Results

Considering the image uncertainties in the whole pelvis, GAN L2 and U-Net L2 showed the lowest mean absolute error (≤34.4 Hounsfield units). The mean errors were not different than 0 (P ≤ .05). The PBM provided the highest uncertainties. Very few DVH points differed when comparing GAN L2 or U-Net L2 DVHs and CTref DVHs (P ≤ .05). Their dose uncertainties were ≤0.6% for the prostate planning target Volume V95%, ≤0.5% for the rectum V70Gy, and ≤0.1% for the bladder V50Gy. The PBM, U-Net PL, and GAN PL presented the highest systematic dose uncertainties. The gamma pass rates were >99% for all DLMs. The mean calculation time to generate 1 pCT was 15 s for the DLMs and 62 min for the PBM.

Conclusions

Generating pCT for MRI dose planning with DLMs and PBM provided low-dose uncertainties. In particular, the GAN L2 and U-Net L2 provided the lowest dose uncertainties together with a low computation time.

Introduction

Magnetic resonance imaging (MRI) is clearly superior to computed tomography (CT) for organ delineation and could therefore improve tumor targeting in dose planning.1 However, MRI does not provide electron density information that is necessary for dose calculation. To overcome this issue, several methods have been developed to generate pseudo-CTs (pCTs) for MRI-based dose planning.2,3 These methods can be divided into 4 categories: bulk density methods (BDM)4, 5, 6, 7, 8; probabilistic methods9; atlas-based methods (ABM)10, 11, 12, 13, 14, 15, 16, 17; and more recently, machine learning methods such as patch-based methods (PBM), including random forest modeling18, 19, 20, 21, 22 and deep learning methods (DLMs).23, 24, 25, 26, 27, 28, 29 The BDMs assign homogeneous densities to the volumes of interest (VOIs) that are manually delineated from the patient's MRI. Probabilistic methods use a probability density function to determine the corresponding Hounsfield Unit (HU) of each voxel of the patient's MRI. The ABMs involve complex nonrigid registrations of CT-MRI atlases with the patient's MRI, followed by a CT fusion step to obtain the pCT. The PBMs select the k closest CT patches from a training cohort for a given MRI patch from the patient. The selected CT patches are then fused to generate the corresponding pCT patch. This process is reiterated for each patient's MRI patch to obtain the whole pCT.

DLMs enable the computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction.30 Deep learning has recently been introduced in radiation therapy for multiple applications, such as image segmentation, image processing and reconstruction, image registration, treatment planning, and radiomics.31, 32, 33, 34, 35, 36, 37 DLMs have been more recently proposed for pCT generation from MRI.38, 39, 40, 41, 42, 43 They are particularly appealing because of their fast computation time. These methods model relations between the HU values of the CTs and the intensities of the MRIs by training neural networks. Once the optimal network parameters are estimated, the model can be finally applied to a test patient MRI to generate its corresponding pCT. One of the first DLMs for pCT generation from MRI was based on the U-Net architecture (U-Net DLM).23 More recently, DLMs that use a generative adversarial network (GAN DLM) architecture have also been proposed (Fig. 1),24,25,27,29,44 with the theoretical advantage of GAN compared with U-Net to provide more realistic pCTs by obtaining an adversarial feedback from a discriminator network. Although GAN and U-Net DLMs provide promising preliminary results, they most often use a standard loss function (L2 and L1 norms), which may also produce blurring and loss of details.29 Perceptual loss could overcome this issue by mimicking human visual perception using similar features (such as multiscale features), but it has never been investigated in this pCT generation application.45, 46, 47 Network hyperparameters such as layer level, the number and weight associated with each level (for perceptual loss), and the discriminator weight compared with the generator weight can also affect the image accuracy. Overall, all these DLM configurations lack a thorough dose evaluation for pCT generation from MRI.

We previously showed that PBM provided lower imaging and dose uncertainties in the pelvis compared with ABM and BDM.20 PBM was found to be faster than ABM. In another study, the U-Net DLM with L2 loss function has been shown to provide better imaging results than the ABM, similar dosimetric results as the ABM, and fewer uncertainties than BDM.48 However, even though the PBMs and DLMs can be considered the most suitable methods for MRI-based dose planning, they have never been compared. Finally, U-Net and GAN DLMs have never been dosimetrically compared in the literature.

This study aims to evaluate and compare the U-Net and GAN DLMs using various hyperparameters and loss functions (L2, single-scale PL, multiscale PL, weighted multiscale PL), in addition to PBM, for prostate cancer MRI-only dose planning.

Section snippets

Methods and Materials

Thirty-nine patients received a volumetric modulated arc therapy for localized prostate cancer. The ethics approval for the study protocol was provided by the local area health ethics committee, and informed consent was obtained from all patients.10 The study follows the same workflow described in our previous study.20

Imaging endpoints and calculation time

Examples of MRI, CTref, and pCTs generated by each method are illustrated in Figure E1 (available online at https://doi.org/10.1016/j.ijrobp.2019.08.049).

Table 1 lists the imaging endpoints corresponding to each pCT generation method for the VOIs. The GAN L2 and U-Net L2 showed the lowest MAE and ME (in absolute value) for soft tissue and bone. The GAN PL showed significantly lower MAE for the whole pelvis and the soft tissue than the U-Net PL. The PBM provided the highest corresponding values.

Discussion

A total of 6 DLMs for pelvis pCT generation from MRI were investigated and compared with a PBM. Several hyperparameters of the DLMs were optimized according to imaging endpoints (Appendix 4, available online at https://doi.org/10.1016/j.ijrobp.2019.08.049). Compared with the CTref, the pCTs generated by DLMs and PBM provided overall low dose uncertainties, thereby making them clinically acceptable for MRI-based prostate dose planning (Fig. 2). Regarding dose accuracy and calculation time, in

Conclusions

To generate pCT for MRI-based prostate dose planning, DLMs appear to be particularly promising for clinical practice owing to the low dose uncertainty and fast calculation time. The U-Net and GAN DLMs with L2 loss function provide the lowest dose uncertainties. These MRI approaches in prostate cancer radiation therapy, which do not require any CT, could thereby improve the accuracy of VOI delineation and can also be used for (re)planning in the MRI-LINAC workflow.72

Acknowledgments

The authors thank Eugenia Mylona for her contribution to statistical analyses, especially her strong expertise in permutation tests.

References (72)

  • P. Meyer et al.

    Survey on deep learning for radiotherapy

    Comput Biol Med

    (2018)
  • D. Rivest-Hénault et al.

    Robust inverse-consistent affine CT–MR registration in MRI-assisted and MRI-alone prostate radiation therapy

    Med Image Anal

    (2015)
  • V. Beckendorf et al.

    70 Gy Versus 80 Gy in localized prostate cancer: 5-year results of GETUG 06 randomized trial

    Int J Radiat Oncol Biol Phys

    (2011)
  • L.B. Marks et al.

    Use of normal tissue complication probability models in the clinic

    Int J Radiat Oncol Biol Phys

    (2010)
  • L. Xiang et al.

    Deep embedding convolutional neural network for synthesizing CT image from T1-weighted MR image

    Med Image Anal

    (2018)
  • C. Studholme et al.

    An overlap invariant entropy measure of 3D medical image alignment

    Pattern Recognit

    (1999)
  • A.U. Pathmanathan et al.

    Comparison of prostate delineation on multimodality imaging for MR-guided radiotherapy

    Br J Radiol

    (2019)
  • J.M. Edmund et al.

    A review of substitute CT generation for MRI-only radiation therapy

    Radia. Oncol

    (2017)
  • S.J. Hoogcarspel et al.

    The feasibility of utilizing pseudo CT-data for online MRI based treatment plan adaptation for a stereotactic radiotherapy treatment of spinal bone metastases

    Phys Med Biol

    (2014)
  • A.L. Chin et al.

    Feasibility and limitations of bulk density assignment in MRI for head and neck IMRT treatment planning

    J Appl Clin Med Phys

    (2014)
  • M.S.R. Gudur et al.

    A unifying probabilistic Bayesian approach to derive electron density from MRI for radiation therapy treatment planning

    Phys Med Biol

    (2014)
  • N. Burgos et al.

    Robust CT synthesis for radiotherapy planning: Application to the head and neck region

  • J. Sjölund et al.

    Generating patient specific pseudo-CT of the head from MR using atlas-based regression

    Phys Med Biol

    (2015)
  • J. Uh et al.

    MRI-based treatment planning with pseudo CT generated through atlas registration

    Med Phys

    (2014)
  • H. Arabi et al.

    Atlas-guided generation of pseudo-CT images for MRI-only and hybrid PET–MRI-guided radiotherapy treatment planning

    Phys Med Biol

    (2016)
  • D. Andreasen et al.

    A patch-based pseudo-CT approach for MRI-only radiotherapy in the pelvis

    Med Phys

    (2016)
  • A. Largent et al.

    Pseudo-CT generation for MRI-only radiotherapy treatment planning: Comparison between patch-based, atlas-based, and bulk density methods

    Int J Radia. Oncol

    (2018)
  • D. Andreasen et al.

    Patch-based generation of a pseudo CT from conventional MRI sequences for MRI-only radiotherapy of the brain

    Med Phys

    (2015)
  • G. Shafai-Erfani et al.

    Dose evaluation of MRI-based synthetic CT generated using a machine learning method for prostate cancer radiotherapy

    Med Dosim

    (2019)
  • X. Han

    MR-based synthetic CT generation using a deep convolutional neural network method

    Med Phys

    (2017)
  • D. Nie et al.

    Medical image synthesis with context-aware generative adversarial networks

  • J.M. Wolterink et al.

    Deep MR to CT synthesis using unpaired data

  • M. Maspero et al.

    Dose evaluation of fast synthetic-CT generation using a generative adversarial network for general pelvis MR-only radiotherapy

    Phys Med Biol

    (2018)
  • J. Fu et al.

    Male pelvic synthetic CT generation from T1-weighted MRI using 2D and 3D convolutional neural networks

    ArXiv180300131,

    (2018)
  • D. Nie et al.

    Medical image synthesis with deep convolutional adversarial networks

    IEEE Trans Biomed Eng

    (2018)
  • Y. LeCun et al.

    Deep learning

    Nature

    (2015)
  • Cited by (64)

    • The Future of MR-Guided Radiation Therapy

      2024, Seminars in Radiation Oncology
    • A high-performance method of deep learning for prostate MR-only radiotherapy planning using an optimized Pix2Pix architecture

      2022, Physica Medica
      Citation Excerpt :

      The generated sCTs from Pix2Pix were compared with sCTs generated from two others: A U-Net (with L2 loss function) and a GAN (with U-Net generator and L2 loss function). The Adam algorithm was used (Learning rate = 1 × 10-4, β1 = 0.9, epochs = 300, batch size = 5, λ1 = 5 and λ2 = 1) [18]. The organ delineations made by the radiation oncologist were rigidly propagated from the reference CT to the sCT images.

    View all citing articles on Scopus

    This work was supported by Cancer Council New South Wales research grant rg11-05, the Prostate Cancer Foundation of Australia (Movember Young Investigator grant yi2011), and Cure Cancer Australia.

    Disclosures: none

    View full text