Abstract
Objectives
To establish a breast lesion risk stratification system using ultrasound images to predict breast malignancy and assess Breast Imaging Reporting and Data System (BI-RADS) categories simultaneously.
Methods
This multicenter study prospectively collected a dataset of ultrasound images for 5012 patients at thirty-two hospitals from December 2018 to December 2020. A deep learning (DL) model was developed to conduct binary categorization (benign and malignant) and BI-RADS categories (2, 3, 4a, 4b, 4c, and 5) simultaneously. The training set of 4212 patients and the internal test set of 416 patients were from thirty hospitals. The remaining two hospitals with 384 patients were used as an external test set. Three experienced radiologists performed a reader study on 324 patients randomly selected from the test sets. We compared the performance of the DL model with that of three radiologists and the consensus of the three radiologists.
Results
In the external test set, the DL model achieved areas under the receiver operating characteristic curve (AUCs) of 0.980 and 0.945 for the binary categorization and six-way categorizations, respectively. In the reader study set, the DL BI-RADS categories achieved a similar AUC (0.901 vs. 0.933, p = 0.0632), sensitivity (90.98% vs. 95.90%, p = 0.1094), and accuracy (83.33% vs. 79.01%, p = 0.0541), but higher specificity (78.71% vs. 68.81%, p = 0.0012) than those of the consensus of the three radiologists.
Conclusions
The DL model performed well in distinguishing benign from malignant breast lesions and yielded outcomes similar to experienced radiologists. This indicates the potential applicability of the DL model in clinical diagnosis.
Key Points
• The DL model can achieve binary categorization for benign and malignant breast lesions and six-way BI-RADS categorizations for categories 2, 3, 4a, 4b, 4c, and 5, simultaneously.
• The DL model showed acceptable agreement with radiologists for the classification of breast lesions.
• The DL model performed well in distinguishing benign from malignant breast lesions and had promise in helping reduce unnecessary biopsies of BI-RADS 4a lesions.
Similar content being viewed by others
Abbreviations
- ACR:
-
American College of Radiology
- AI:
-
Artificial intelligence
- AUC:
-
Area under the receiver operating characteristic curve
- BI-RADS:
-
Breast Imaging Reporting and Data System
- CAD:
-
Computer-aided diagnosis
- CNN:
-
Convolutional neural network
- DL:
-
Deep learning
- ETC:
-
External test cohort
- ITC:
-
Internal test cohort
- ML:
-
Machine learning
- NPV:
-
Negative predictive value
- PPV:
-
Positive predictive value
- TC:
-
Training cohort
- US:
-
Ultrasound
References
Harbeck N, Gnant M (2017) Breast cancer. Lancet 389:1134–1150
Lei S, Zheng R, Zhang S et al (2021) Breast cancer incidence and mortality in women in China: temporal trends and projections to 2030. Cancer Biol Med 18:900–909
Zheng C, Yu ZG, Chinese Society of Breast S (2021) Clinical practice guidelines for pre-operative evaluation of breast cancer: Chinese Society of Breast Surgery (CSBrS) practice guidelines 2021. Chin Med J 134:2147–2149
Hooley RJ, Scoutt LM, Philpotts LE (2013) Breast ultrasonography: state of the art. Radiology 268:642–659
Chang JM, Leung JWT, Moy L, Ha SM, Moon WK (2020) Axillary nodal evaluation in breast cancer: state of the art. Radiology 295:500–515
D’Orsi C, Sickles E, Mendelson E, Morris E et al (2013) ACR BI-RADS® Atlas, Breast Imaging Reporting and Data System. 5th ed. American College of Radiology, Reston, VA
Raza S, Chikarmane SA, Neilsen SS, Zorn LM, Birdwell RL (2008) BI-RADS 3, 4, and 5 lesions: value of US in management--follow-up and outcome. Radiology 248:773–781
Raza S, Goldkamp AL, Chikarmane SA, Birdwell RL (2010) US of breast masses categorized as BI-RADS 3, 4, and 5: pictorial review of factors influencing clinical management. Radiographics 30:1199–1213
Berg WA, Cosgrove DO, Dore CJ et al (2012) Shear-wave elastography improves the specificity of breast US: the BE1 multinational study of 939 masses. Radiology 262:435–449
Lazarus E, Mainiero MB, Schepps B, Koelliker SL, Livingston LS (2006) BI-RADS lexicon for US and mammography: interobserver variability and positive predictive value. Radiology 239:385–391
Abdullah N, Mesurolle B, El-Khoury M, Kao E (2009) Breast imaging reporting and data system lexicon for US: interobserver agreement for assessment of breast masses. Radiology 252:665–672
Menezes GLG, Pijnappel RM, Meeuwis C et al (2018) Downgrading of breast masses suspicious for cancer by using optoacoustic breast imaging. Radiology 288:355–365
Berg WA, Blume JD, Cormack JB et al (2008) Combined screening with ultrasound and mammography vs mammography alone in women at elevated risk of breast cancer. JAMA 299:2151–2163
Nothacker M, Duda V, Hahn M et al (2009) Early detection of breast cancer: benefits and risks of supplemental breast ultrasound in asymptomatic women with mammographically dense breast tissue. A systematic review. BMC Cancer 9:335
Berg WA (2020) Reducing unnecessary biopsy and follow-up of benign cystic breast lesions. Radiology 295:52–53
Han S, Kang HK, Jeong JY et al (2017) A deep learning framework for supporting the classification of breast lesions in ultrasound images. Phys Med Biol 62:7714–7728
Zhuang Z, Yang Z, Zhuang S, Joseph Raj AN, Yuan Y, Nersisson R (2021) Multi-features-based automated breast tumor diagnosis using ultrasound image and support vector machine. Comput Intell Neurosci 2021:9980326
Shia WC, Lin LS, Chen DR (2021) Classification of malignant tumours in breast ultrasound using unsupervised machine learning approaches. Sci Rep 11:1418
Shia WC, Chen DR (2021) Classification of malignant tumors in breast ultrasound using a pretrained deep residual network model and support vector machine. Comput Med Imaging Graph 87:101829
Romeo V, Cuocolo R, Apolito R et al (2021) Clinical value of radiomics and machine learning in breast ultrasound: a multicenter study for differential diagnosis of benign and malignant lesions. Eur Radiol 31:9511–9519
Huo L, Tan Y, Wang S et al (2021) Machine learning models to improve the differentiation between benign and malignant breast lesions on ultrasound: a multicenter external validation study. Cancer Manag Res 13:3367–3379
Kalafi EY, Jodeiri A, Setarehdan SK et al (2021) Classification of breast cancer lesions in ultrasound images by using attention layer and loss ensemble in deep convolutional neural networks. Diagn (Basel) 11:1859
Qian X, Pei J, Zheng H et al (2021) Prospective assessment of breast cancer risk from multimodal multiview ultrasound images via clinically applicable deep learning. Nat Biomed Eng 5:522–532
Shen Y, Shamout FE, Oliver JR et al (2021) Artificial intelligence system reduces false-positive findings in the interpretation of breast ultrasound exams. Nat Commun 12:5645
Xing J, Chen C, Lu Q et al (2021) Using BI-RADS stratifications as auxiliary information for breast masses classification in ultrasound images. IEEE J Biomed Health Inform 25:2058–2070
Shen WC, Chang RF, Moon WK (2007) Computer aided classification system for breast ultrasound based on Breast Imaging Reporting and Data System (BI-RADS). Ultrasound Med Biol 33:1688–1698
Huang Y, Han L, Dou H et al (2019) Two-stage CNNs for computerized BI-RADS categorization in breast ultrasound images. Biomed Eng Online 18:8
Ciritsis A, Rossi C, Eberhard M, Marcon M, Becker AS, Boss A (2019) Automatic classification of ultrasound breast lesions using a deep convolutional neural network mimicking human decision-making. Eur Radiol 29:5458–5468
Qian X, Zhang B, Liu S et al (2020) A combined ultrasonic B-mode and color Doppler system for the classification of breast masses using neural network. Eur Radiol 30:3023–3033
Zhang H, Han L, Chen K, Peng Y, Lin J (2020) Diagnostic efficiency of the breast ultrasound computer-aided prediction model based on convolutional neural network in breast cancer. J Digit Imaging 33:1218–1223
Qi X, Zhang L, Chen Y et al (2019) Automated diagnosis of breast ultrasonography images using deep neural networks. Med Image Anal 52:185–198
Liu J, Li W, Zhao N et al (2018) Integrate domain knowledge in training CNN for ultrasonography breast cancer diagnosis. Springer International Publishing, Cham, pp 868–875
Al-Dhabyani W, Gomaa M, Khaled H, Fahmy A (2020) Dataset of breast ultrasound images. Data Brief 28:104863
Ding X, Zhang X, Ma N, Han J, Ding G, Sun J (2021) RepVGG: Making VGG-style ConvNets Great Again. https://arxiv.org/abs/2101.03697. Accessed 19 Apr 2021
Spinelli VMA, Teixeira DCJ, Rauber A, Varella IS, Fleck JF, Moreira LF (2018) Role of BI-RADS ultrasound subcategories 4A to 4C in predicting breast cancer. Clin Breast Cancer 18:e507–e511
Stavros AT, Freitas AG, deMello GGN et al (2017) Ultrasound positive predictive values by BI-RADS categories 3-5 for solid masses: an independent reader study. Eur Radiol 27:4307–4315
Fu CY, Hsu HH, Yu JC et al (2011) Influence of age on PPV of sonographic BI-RADS categories 3, 4, and 5. Ultraschall Med 32(Suppl 1):S8–S13
Yoon JH, Kim MJ, Moon HJ, Kwak JY, Kim EK (2011) Subcategorization of ultrasonographic BI-RADS category 4: positive predictive value and clinical factors affecting it. Ultrasound Med Biol 37:693–699
Lee HJ, Kim EK, Kim MJ et al (2008) Observer variability of Breast Imaging Reporting and Data System (BI-RADS) for breast ultrasound. Eur J Radiol 65:293–298
Jales RM, Sarian LO, Torresan R, Marussi EF, Alvares BR, Derchain S (2013) Simple rules for ultrasonographic subcategorization of BI-RADS(R)-US 4 breast masses. Eur J Radiol 82:1231–1235
He P, Cui LG, Chen W, Yang RL (2019) Subcategorization of ultrasonographic BI-RADS category 4: assessment of diagnostic accuracy in diagnosing breast lesions and influence of clinical factors on positive predictive value. Ultrasound Med Biol 45:1253–1258
Hu Y, Yang Y, Gu R et al (2018) Does patient age affect the PPV3 of ACR BI-RADS Ultrasound categories 4 and 5 in the diagnostic setting? Eur Radiol 28:2492–2498
Lee YJ, Choi SY, Kim KS, Yang PS (2016) Variability in observer performance between faculty members and residents using Breast Imaging Reporting and Data System (BI-RADS)-Ultrasound, Fifth Edition (2013). Iran J Radiol 13:e28281
Park CS, Kim SH, Jung NY, Choi JJ, Kang BJ, Jung HS (2015) Interobserver variability of ultrasound elastography and the ultrasound BI-RADS lexicon of breast lesions. Breast Cancer 22:153–160
Funding
This work is supported by the Beijing Natural Science Foundation (7202156), and the Foundation of International Health Exchange and Cooperation Center NHC PRC (ihecc2018C0032-2).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Guarantor
The scientific guarantors of this publication are Hongyan Wang and Yuxin Jiang.
Conflict of interest
Four of the authors are engineers in Shenzhen Mindray Bio-Medical Electronics Co., Ltd, which provides the ultrasound system and technical support to our research.
Statistics and biometry
No complex statistical methods were necessary for this paper.
Informed consent
Written informed consent was obtained from all patients before they underwent US.
Ethical approval
Institutional Review Board approval was obtained.
Methodology
• prospective
• diagnostic study
• multicenter study
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
ESM 1
(DOCX 736 kb)
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Gu, Y., Xu, W., Liu, T. et al. Ultrasound-based deep learning in the establishment of a breast lesion risk stratification system: a multicenter study. Eur Radiol 33, 2954–2964 (2023). https://doi.org/10.1007/s00330-022-09263-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00330-022-09263-8