Skip to main content
SpringerLink
Log in

An artificial intelligence algorithm for pulmonary embolism detection on polychromatic computed tomography: performance on virtual monochromatic images

  • Imaging Informatics and Artificial Intelligence
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

Virtual monochromatic images (VMI) are increasingly used in clinical practice as they improve contrast-to-noise ratio. However, due to their different appearances, the performance of artificial intelligence (AI) trained on conventional CT images may worsen. The goal of this study was to assess the performance of an established AI algorithm trained on conventional polychromatic computed tomography (CT) images (CPI) to detect pulmonary embolism (PE) on VMI.

Methods

Paired 60 kiloelectron volt (keV) VMI and CPI of 114 consecutive patients suspected of PE, obtained with a detector-based spectral CT scanner, were retrospectively analyzed by an established AI algorithm. The CT pulmonary angiography (CTPA) were classified as positive or negative for PE on a per-patient level. The reference standard was established using a comprehensive method that combined the evaluation of the attending radiologist and three experienced cardiothoracic radiologists aided by two different detection tools. Sensitivity, specificity, positive and negative predictive values and likelihood ratios of the algorithm on VMI and CPI were compared.

Results

The prevalence of PE according to the reference standard was 35.1% (40 patients). None of the diagnostic accuracy measures of the algorithm showed a significant difference between CPI and VMI. Sensitivity was 77.5% (95% confidence interval (CI) 64.6–90.4%) and 85.0% (73.9–96.1%) (p = 0.08) on CPI and VMI respectively and specificity 96.0% (91.4–100.0%) and 94.6% (89.4–99.7%) (p = 0.32).

Conclusions

Diagnostic performance of the AI algorithm that was trained on CPI did not drop on VMI, which is reassuring for its use in clinical practice.

Clinical relevance statement

A commercially available AI algorithm, trained on conventional polychromatic CTPA, could be safely used on virtual monochromatic images. This supports the sustainability of AI-aided detection of PE on CT despite ongoing technological advances in medical imaging, although monitoring in daily practice will remain important.

Key Points

• Diagnostic accuracy of an AI algorithm trained on conventional polychromatic images to detect PE did not drop on virtual monochromatic images.

• Our results are reassuring as innovations in hardware and reconstruction in CT are continuing, whilst commercial AI algorithms that are trained on older generation data enter healthcare.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Abbreviations

AI:

Artificial intelligence

CAD:

Computer-aided detection

CI:

Confidence interval

CPI:

Conventional polychromatic images

CTPA:

Computed tomography pulmonary angiography

keV:

Kiloelectron volt

LR:

Likelihood ratio

PE:

Pulmonary embolism

SDCT:

Detector-based spectral CT

VMI:

Virtual monochromatic images

References

  1. Sb Smith, Jb Geske, Kathuria P et al (2016) Analysis of national trends in admissions for pulmonary embolism. Chest 150:35–45 (S0012-3692(16)01257-5)

    Article  Google Scholar 

  2. Stein Pd, Se Fowler, Al Goodman Lr Et (2006) Multidetector computed tomography for acute pulmonary embolism. N Engl J Med 354:2317–2327 (354/22/2317)

    Article  CAS  PubMed  Google Scholar 

  3. Zhang Lj Lu, Gm MF, Mcquiston Ad, Jg R, Uj S (2015) Computed tomography of acute pulmonary embolism: state-of-the-art. Eur Radiol 25:2547–2557. https://doi.org/10.1007/S00330-015-3679-2

    Article  PubMed  Google Scholar 

  4. Bruls Rjm, Kwee Rm (2020) Workload for radiologists during on-call hours: dramatic increase in the past 15 years. Insights Imaging 11:121-Z. https://doi.org/10.1186/S13244-020-00925-Z

  5. Soffer S, Klang E, Shimon O et al (2021) Deep learning for pulmonary embolism detection on computed tomography pulmonary angiogram: a systematic review and meta-analysis. Sci Rep 11:15814–3. https://doi.org/10.1038/S41598-021-95249-3

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Ab C, Gorincour G, Et NH, Al, (2022) How artificial intelligence improves radiological interpretation in suspected pulmonary embolism. Eur Radiol 32:5831–5842. https://doi.org/10.1007/S00330-022-08645-2

    Article  Google Scholar 

  7. Do Td, Rheinheimer S, Kauczor Hu, Stiller W, Weber T, Skornitzke S (2020) Image quality evaluation of dual-layer spectral CT in comparison to single-layer CT in a reduced-dose setting. Eur Radiol 30:5709–5719. https://doi.org/10.1007/S00330-020-06894-7

    Article  PubMed  PubMed Central  Google Scholar 

  8. Van Ommen F, Hwam De Jong, Jw Dankbaar, Bennink E, Leiner T, Amr Schilham (2019) Dose of CT protocols acquired in clinical routine using a dual-layer detector CT scanner: a preliminary report. Eur J Radiol 112:65–71 (S0720-048x(19)30010-5)

    Article  PubMed  Google Scholar 

  9. Meier A, Wurnig M, Desbiolles L, Leschka S, Frauenfelder T, Alkadhi H (2015) Advanced virtual monoenergetic images: improving the contrast of dual-energy CT pulmonary angiography. Clin Radiol 70:1244–1251 (S0009-9260(15)00313-X)

    Article  CAS  PubMed  Google Scholar 

  10. Bae K, Kn J, Al CSE (2018) Improved opacification of a suboptimally enhanced pulmonary artery in chest CT: experience using a dual-layer detector spectral CT. AJR Am J Roentgenol 210:734–741. https://doi.org/10.2214/Ajr.17.18537

  11. Leithner D, Jl W, Al VTE (2017) Virtual monoenergetic imaging and iodine perfusion maps improve diagnostic accuracy of dual-energy computed tomography pulmonary angiography with suboptimal contrast attenuation. Invest Radiol 52(11):659–665. https://doi.org/10.1097/Rli.0000000000000387

    Article  PubMed  Google Scholar 

  12. Langius-Wiffen E, Im N, De Boer E, Al Et (2021) Computer-aided pulmonary embolism detection on virtual monochromatic images compared to conventional CT angiography. Radiology 301:420–422. https://doi.org/10.1148/Radiol.2021204620

    Article  PubMed  Google Scholar 

  13. Kröger Jr, Hickethier T, Pahn G, Gerhardt F, Maintz D, Bunck Ac (2017) Influence of spectral detector CT based monoenergetic images on the computer-aided detection of pulmonary artery embolism. Eur J Radiol 95:242-248. S0720-048x(17)30354-6

  14. Yu Ac, Mohajer B, Eng J (2022) External validation of deep learning algorithms for radiologic diagnosis: a systematic review. Radiol Artif Intell 4:E210064. https://doi.org/10.1148/Ryai.210064

  15. So A, Nicolaou S (2021) Spectral computed tomography: fundamental principles and recent developments. Korean J Radiol 22:86–96. https://doi.org/10.3348/Kjr.2020.0144

    Article  PubMed  Google Scholar 

  16. Cn D, Bs B, Bj B, Aj G-W (2020) Artificial intelligence in radiology: a call for thoughtful application. Clin Transl Sci 13:216–218. https://doi.org/10.1111/Cts.12704

    Article  Google Scholar 

  17. Se Jones, Wittram C (2005) The indeterminate CT pulmonary angiogram: imaging characteristics and patient clinical outcome. Radiology 237:329–337 (237/1/329)

    Article  Google Scholar 

  18. Sauter Ap, Shapira N, Kopp Fk Et Al (2020) CTPA with a conventional ct at 100 kvp vs. a spectral-detector CT at 120 kvp: comparison of radiation exposure, diagnostic performance and image quality. Eur J Radiol Open 7:100234. https://doi.org/10.1016/J.Ejro.2020.100234

  19. Cheng J, Yin Y, Wu H et al (2013) Optimal monochromatic energy levels in spectral CT pulmonary angiography for the evaluation of pulmonary embolism. PLoS One 8:E63140. https://doi.org/10.1371/Journal.Pone.0063140

  20. Petry M, Lansky C, Chodakiewitz Y, Maya M, Pressman B (2022) Decreased hospital length of stay for ICH And PE after adoption of an artificial intelligence-augmented radiological worklist triage system. Radiol Res Pract 2022:2141839. https://doi.org/10.1155/2022/2141839

    Article  PubMed  PubMed Central  Google Scholar 

  21. Ht SC (2013) Dtcompair: comparison of binary diagnostic tests in a paired study design. R Package Version 1:00

    Google Scholar 

  22. Der Pol V, Lm BI, Al VMTE (2018) Lower prevalence of subsegmental pulmonary embolism after application of the years diagnostic algorithm. Br J Haematol 183:629–635. https://doi.org/10.1111/Bjh.15556

    Article  PubMed  PubMed Central  Google Scholar 

  23. Van Der Hulle T, Wy Cheung, Kooij S et al (2017) Simplified diagnostic management of suspected pulmonary embolism (the Years Study): a prospective, multicentre, cohort study. Lancet 390:289–297 (S0140-6736(17)30885-1)

    Article  PubMed  Google Scholar 

  24. Rl W, Na L (2016) The ASA statement on p-values: context, process, and purpose. Am Stat 70:129–133. https://doi.org/10.1080/00031305.2016.1154108

    Article  Google Scholar 

  25. Weikert T, Dj Winkel, Bremerich J et al (2020) Automated detection of pulmonary embolism in CT pulmonary angiograms using an AI-powered algorithm. Eur Radiol 30:6545–6553. https://doi.org/10.1007/S00330-020-06998-0

    Article  PubMed  Google Scholar 

  26. Buls N, Watté N, Nieboer K, Ilsen B, De Mey J (2021) Performance of an artificial intelligence tool with real-time clinical workflow integration - detection of intracranial hemorrhage and pulmonary embolism. Phys Med 83:154–160 (S1120-1797(21)00131-9)

    Article  PubMed  Google Scholar 

  27. Liu Z, Yuan H, Wang H (2022) CAM-WNET: an effective solution for accurate pulmonary embolism segmentation. Med Phys 49:5294–5303. https://doi.org/10.1002/Mp.15719

    Article  PubMed  Google Scholar 

  28. Huang S, Kothari T, Banerjee I et al (2020) PENET—a scalable deep-learning model for automated diagnosis of pulmonary embolism using volumetric CT imaging. NPJ Digital Medicine 3:61. https://doi.org/10.1038/S41746-020-0266-Y

Download references

Acknowledgements

The authors would like to thank the University Medical Centre Utrecht for providing the anonymized spectral CT scan data and Aidoc Medical, in particular Amitai Mandelbaum, for facilitating analysis of the data.

Funding

The authors state that this work has not received any funding.

Author information

Authors and Affiliations

  1. Department of Radiology, Isala Hospital, Dr. Van Heesweg 2, 8025 AB, Zwolle, The Netherlands

    Eline Langius-Wiffen, Ingrid M. Nijholt, Rogier A. van Dijk, Erwin de Boer, Jacqueline Nijboer-Oosterveld & Martijn F. Boomsma

  2. Department of Radiology, University Medical Centre Utrecht, Utrecht, the Netherlands

    Wouter B. Veldhuis & Pim A. de Jong

  3. Division of Imaging and Oncology, University Medical Centre Utrecht, Utrecht, The Netherlands

    Martijn F. Boomsma

Authors
  1. Eline Langius-Wiffen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ingrid M. Nijholt
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rogier A. van Dijk
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Erwin de Boer
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jacqueline Nijboer-Oosterveld
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wouter B. Veldhuis
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Pim A. de Jong
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Martijn F. Boomsma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Eline Langius-Wiffen.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Dr. Martijn F. Boomsma.

Conflict of interest

The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.

Statistics and biometry

One of the authors has significant statistical expertise: I.M. Nijholt, Ph.D. No complex statistical methods were necessary for this paper.

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethical approval

Institutional Review Board approval was obtained.

Study subjects or cohorts overlap

Study subjects have been previously reported in:

Langius-Wiffen E, Nijholt IM, de Boer E et al (2021) Computer-aided pulmonary embolism detection on virtual monochromatic images compared to conventional CT angiography. Radiology 301:420–422. 10.1148/radiol.2021204620 [doi].

Methodology

  • Retrospective

  • Diagnostic study

  • Performed at one institution

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Langius-Wiffen, E., Nijholt, I.M., van Dijk, R.A. et al. An artificial intelligence algorithm for pulmonary embolism detection on polychromatic computed tomography: performance on virtual monochromatic images. Eur Radiol 34, 384–390 (2024). https://doi.org/10.1007/s00330-023-10048-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-023-10048-w

Keywords

Search

Navigation