Skip to main content
SpringerLink
Log in

Ensemble deep learning model for predicting anterior cruciate ligament tear from lateral knee radiograph

  • Scientific Article
  • Published:
Skeletal Radiology Aims and scope Submit manuscript

Abstract

Objective

To develop an ensemble deep learning model (DLM) predicting anterior cruciate ligament (ACL) tears from lateral knee radiographs and to evaluate its diagnostic performance.

Materials and methods

In this study, 1433 lateral knee radiographs (661 with ACL tear confirmed on MRI, 772 normal) from two medical centers were split into training (n = 1146) and test sets (n = 287). Three single DLMs respectively classifying radiographs with ACL tears, abnormal lateral femoral notches, and joint effusion were developed. An ensemble DLM predicting ACL tears was developed by combining the three DLMs via stacking method. The sensitivities, specificities, and area under the receiver operating characteristic curves (AUCs) of the DLMs and three radiologists were compared using McNemar test and Delong test. Subgroup analysis was performed to identify the radiologic features associated with the sensitivity.

Results

The sensitivity, specificity, and AUC of the ensemble DLM were 86.8% (95% confidence interval [CI], 79.9–92.0%), 89.4% (95% CI, 83.4–93.8%), and 0.927 (95% CI, 0.891–0.954), achieving diagnostic performance comparable with that of a musculoskeletal radiologist (P = 0.193, McNemar test; P = 0.131, Delong test). The AUC of the ensemble DLM was significantly higher than those of non-musculoskeletal radiologists (P = 0.043, P < 0.001). The sensitivity of the DLM was higher than that of the radiologists in the absence of an abnormal lateral femoral notch or joint effusion.

Conclusion

The diagnostic performance of the ensemble DLM in predicting lateral knee radiographs with ACL tears was comparable to that of a musculoskeletal radiologist.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Abbreviations

ACL:

Anterior cruciate ligament

DLM:

Deep learning model

AUC:

Area under the receiver operating characteristic curve

CI:

Confidence interval

References

  1. Murrell GA, Maddali S, Horovitz L, Oakley SP, Warren RF. The effects of time course after anterior cruciate ligament injury in correlation with meniscal and cartilage loss. Am J Sports Med. 2001;29(1):9–14.

    Article  CAS  Google Scholar 

  2. Granan L-P, Bahr R, Lie SA, Engebretsen L. Timing of anterior cruciate ligament reconstructive surgery and risk of cartilage lesions and meniscal tears: a cohort study based on the Norwegian National Knee Ligament Registry. Am J Sports Med. 2009;37(5):955–61.

    Article  Google Scholar 

  3. Lubowitz JH, Bernardini BJ, Reid JB III. Current concepts review: comprehensive physical examination for instability of the knee. Am J Sports Med. 2008;36(3):577–94.

    Article  Google Scholar 

  4. Pao DG. The lateral femoral notch sign. Radiology. 2001;219(3):800–1.

    Article  CAS  Google Scholar 

  5. Grimberg A, Shirazian H, Torshizy H, Smitaman E, Chang EY, Resnick D. Deep lateral notch sign and double notch sign in complete tears of the anterior cruciate ligament: MR imaging evaluation. Skeletal Radiol. 2015;44(3):385–91.

    Article  Google Scholar 

  6. Nakauchi M, Kurosawa H, Kawakami AJ. Abnormal lateral notch in knees with anterior cruciate ligament injury. J Orthop Sci. 2000;5(2):92–5.

    Article  CAS  Google Scholar 

  7. Frobell R, Le Graverand M-P, Buck R, et al. The acutely ACL injured knee assessed by MRI: changes in joint fluid, bone marrow lesions, and cartilage during the first year. Osteoarthr Cartil. 2009;17(2):161–7.

    Article  CAS  Google Scholar 

  8. Yoon AP, Lee Y-L, Kane RL, Kuo C-F, Lin C, Chung KC. Development and Validation of a Deep Learning Model Using Convolutional Neural Networks to Identify Scaphoid Fractures in Radiographs. JAMA Netw Open. 2021;4(5):e216096–e216096.

    Article  Google Scholar 

  9. Park HS, Jeon K, Cho YJ, et al. Diagnostic Performance of a New Convolutional Neural Network Algorithm for Detecting Developmental Dysplasia of the Hip on Anteroposterior Radiographs. Korean J Radiol. 2021;22(4):612.

    Article  Google Scholar 

  10. Nam JG, Park S, Hwang EJ, et al. Development and validation of deep learning–based automatic detection algorithm for malignant pulmonary nodules on chest radiographs. Radiology. 2019;290(1):218–28.

    Article  Google Scholar 

  11. Ganaie MA, Hu M, Tanveer M, Suganthan PN. Ensemble deep learning: A review. arXiv. 2021; arXiv:2104.02395.  https://arxiv.org/abs/2104.02395. Accessed 4 Oct 2021.

  12. Sanders TL, Maradit Kremers H, Bryan AJ, et al. Incidence of anterior cruciate ligament tears and reconstruction: a 21-year population-based study. Am J Sports Med. 2016;44(6):1502–7.

    Article  Google Scholar 

  13. Yoon JP, Chang CB, Yoo JH, et al. Correlation of magnetic resonance imaging findings with the chronicity of an anterior cruciate ligament tear. J Bone Joint Surg Am. 2010;92(2):353–60.

    Article  Google Scholar 

  14. Yoon JP, Yoo JH, Chang CB, et al. Prediction of chronicity of anterior cruciate ligament tear using MRI findings. Clin Orthop Surg. 2013;5(1):19–25.

    Article  Google Scholar 

  15. Hall FM. Radiographic diagnosis and accuracy in knee joint effusions. Radiology. 1975;115(1):49–54.

    Article  CAS  Google Scholar 

  16. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-cam: Visual explanations from deep networks via gradient-based localization. In: 2017 IEEE Int Conf Comput Vis. IEEE; 2017: 618–26. https://doi.org/10.1109/ICCV.2017.74

  17. Crawshaw M. Multi-task learning with deep neural networks: A survey. arXiv. 2020; arXiv:2009.09796. https://arxiv.org/abs/2009.09796.  Accessed 11 May 2022.

  18. Zhou Y, Chen H, Li Y, et al. Multi-task learning for segmentation and classification of tumors in 3D automated breast ultrasound images. Med Image Anal. 2021;70:101918.

    Article  Google Scholar 

  19. Wu B, Zhou Z, Wang J, Wang Y. Joint learning for pulmonary nodule segmentation, attributes and malignancy prediction. IEEE 15th Int Symp Biomed Imaging. 2018; 1109–13, https://doi.org/10.1109/ISBI.2018.8363765.

  20. Hussein S, Cao K, Song Q, Bagci U. Risk stratification of lung nodules using 3D CNN-based multi-task learning. In: 2017 Int Conf Inf Process Med Imaging. Springer Verlag; 2017: 249–60. https://doi.org/10.1007/978-3-319-59050-9_20

  21. Xing J, Chen C, Lu Q, et al. Using BI-RADS stratifications as auxiliary information for breast masses classification in ultrasound images. IEEE J Biomed Health Inform. 2020;25(6):2058–70.

    Article  Google Scholar 

  22. Herbst E, Hoser C, Tecklenburg K, et al. The lateral femoral notch sign following ACL injury: frequency, morphology and relation to meniscal injury and sports activity. Knee Surg Sports Traumatol Arthrosc. 2015;23(8):2250–8.

    Article  Google Scholar 

  23. Abreu MR, Chung CB, Trudell D, Resnick D. Hoffa’s fat pad injuries and their relationship with anterior cruciate ligament tears: new observations based on MR imaging in patients and MR imaging and anatomic correlation in cadavers. Skeletal Radiol. 2008;37(4):301–6.

    Article  Google Scholar 

  24. Guenoun D, Le Corroller T, Amous Z, et al. The contribution of MRI to the diagnosis of traumatic tears of the anterior cruciate ligament. Diagn Interv Imaging. 2012;93(5):331–41.

    Article  CAS  Google Scholar 

  25. Calmbach WL, Hutchens M. Evaluation of patients presenting with knee pain: Part I. History, physical examination, radiographs, and laboratory tests. Am Fam Physician. 2003;68(5):907–12.

    PubMed  Google Scholar 

  26. Sternbach GL. Evaluation of the knee. J Emerg Med. 1986;4(2):133–43.

    Article  CAS  Google Scholar 

Download references

Acknowledgements

The authors thank Kangwhi Lee and Jung Oh Lee for data analysis.

Author information

Author notes

  1. Dong Hyun Kim and Jee Won Chai contributed equally to this work.

Authors and Affiliations

  1. Department of Radiology, SMG-SNU Boramae Medical Center, Seoul National University College of Medicine, Seoul, Republic of Korea

    Dong Hyun Kim, Jee Won Chai, Ji Hyun Lee, Hyo Jin Kim & Jiwoon Seo

  2. Department of Radiology, Konkuk University Medical Center, 120-1 Neungdong-ro, Gwangjin-gu, Seoul, 05030, Republic of Korea

    Ji Hee Kang

  3. Armed Forces Yangju Hospital, Yangju, Republic of Korea

    Jae Won Choi

  4. Department of Radiology, Seoul National University Hospital, Seoul, Republic of Korea

    Jae Won Choi

Authors
  1. Dong Hyun Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jee Won Chai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ji Hee Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ji Hyun Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hyo Jin Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jiwoon Seo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jae Won Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ji Hee Kang.

Ethics declarations

Ethics approval

Approval from the Institutional Review Board was obtained, and in keeping with the policies for a retrospective review, informed consent was not required.

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1

(DOCX 14.5 KB)

Supplementary file2

(DOCX 14.2 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, D.H., Chai, J.W., Kang, J.H. et al. Ensemble deep learning model for predicting anterior cruciate ligament tear from lateral knee radiograph. Skeletal Radiol 51, 2269–2279 (2022). https://doi.org/10.1007/s00256-022-04081-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00256-022-04081-x

Keywords

Search

Navigation