Diagnostic Performance and Accuracy of Strain Elastography for BI-RADS Category 4 Lesions among Asian Females

By Meng Dong¹, Boyuan Xing¹, Bingyi Zhang¹, Xingdong Xu², Quan Zhou³

Affiliations

1. Department of Ultrasound, The People’s Hospital of China Three Gorges University, The First People's Hospital of Yichang, Hubei, China
2. Department of General Surgery, The People’s Hospital of China Three Gorges University, The First People's Hospital of Yichang, Hubei, China
3. Department of Gynaecology and Obstetrics, The People’s Hospital of China Three Gorges University, The First People's Hospital of Yichang, Hubei, China

doi: 10.29271/jcpsp.2023.10.1181

ABSTRACT
This work aimed to assess the diagnostic performance of strain elastography (elasticity score) and its accuracy for breast BI-RADS category 4 lesions. Online databases including Cochrane Library, Embase, PubMed, and Web of Science were searched for eligible articles published prior to March 10, 2022. The pooled effect indicators including sensitivity, specificity, positive and negative likelihood ratios (PLR, NLR), the area under the summary receiver operating characteristic (SROC) curve, and diagnostic odds ratio (DOR) were utilised to assess the strain elastography’s performance in diagnosing BI-RADS category 4 lesions. Subgroup analyses and meta-regressions were used to explore the potential sources of heterogeneity, and the Deeks' funnel plot asymmetry test was used to detect any publication bias. The literature search yielded 11 studies involving 5028 BI-RADS category 4 lesions (including 1809 malignant lesions). The recruited lesions were all from Asian females. The pooled sensitivity, specificity, PLR, NLR, and DOR with the 95% confidence intervals were 0.68 (0.66-0.71), 0.83 (0.82-0.85), 3.36 (2.45–4.60), 0.32 (0.21-0.49), and 12.11 (7.46-19.65), respectively. The area under the SROC curve was 0.85. No significant publication bias was detected. Taken together, strain elastography had suboptimal sensitivity but desirable specificity for the accurate diagnosis of BI-RADS category 4 lesions among Asian females, which can help avoid unnecessary biopsies and reduce patient anxiety.

Key Words: BI-RADS category 4 lesions, Strain elastography, Meta-analysis.

INTRODUCTION

Breast cancer ranked as the most commonly diagnosed malignancy worldwide in 2020, which could further bring somatopsychic disorders among its female sufferers.¹ Early detection and diagnosis, and appropriate, timely treatment are paramount and significantly helpful in improving their quality of life and survival rate. According to the BI-RADS guideline for breast lesions, BI-RADS category 4 lesions are subdivided into low (4A, likelihood: >2% to ≤10%), moderate (4B, likelihood: >10% to ≤50%), and high (4C, likelihood: >50% to <95%) suspicious ones in 2013.²
 

Since the wide range of likelihood of malignancy for such category lesions and morphological overlap between malignant and benign lesions, the conventional B-mode US presents a relatively low specificity in the diagnosis of breast malignancies. US elastography, developed in 1991 by Ophir et al.³ and rapidly applied as a new clinically diagnostic modality, has been confirmed in the American College of Radiology’s consensus on the 5^th BI-RADS guideline for its better lesion characterisation and risk stratification of the breast. This technology is sensitive to tissue stiffness and can provide a non-invasive evaluation of the stiffness of breast lesions based on two-dimensional ultrasonography, namely, providing effective, complementary information support for precise diagnosis.

Previous meta-analyses have assessed the diagnostic accuracy of strain elastography or shear wave for all BI-RADS categories of breast lesions or non-palpable ones.^4-8 These studies conclude that strain elastography could serve as an effective technique to complement the B-mode US in predicting malignant breast lesions and thus, reduce biopsies for benign breast lesions. This work is currently the first meta-analysis for the assessment of the most controversial BI-RADS category 4 lesions with strain elastography. This study aimed to assess the diagnostic performance of strain elastography (elasticity score) and its accuracy for BI-RADS category 4 lesions.

METHODOLOGY

This meta-analysis was conducted as per the PRISMA reporting guidelines updated in 2020.⁹ A systematic literature search in online databases (Embase, Cochrane Library, PubMed, Web of Science) was performed by two authors independently. There was no limitation on the start date for targeting publications, and the end date was March 10, 2022. The search strategy included the Medical Subject Heading terms: elasticity imaging techniques, elasticity, elastogram, tissue elasticity imaging, sonoelastography, elastography, strain elastography, real time elastography breast lesions, and BI-RADS 4. The two authors independently judged whether a potentially eligible article could be used as per the pre-established inclusion and exclusion criteria. Any discrepancies in data collection were resolved via discussion with a third author.

Publications that had assessment test(s) for the diagnostic accuracy of strain elastography (elasticity score) for BI-RADS category 4 lesions, and pathological investigation of the specimen(s) via biopsy or surgery (serving as the reference standard for diagnostic confirmation), and provided direct or indirect data about false positive (FP), true positive (TP), true negative (TN), false negative (FN) rates or their number of cases, were included. Literature was excluded if presented as a review, a correspondence or conference article, or duplicate ones, or not written in English, or had less than 50 samples (breast lesions).

The extracted data from each eligible publication included the first author’s surname, publication year, study design type, country, the patients’ mean age, the number of lesions, mean lesion size, excitation method for strain elastography, the reference standard for confirming a diagnosis, the cut-off value for diagnosing a malignancy, and numbers of TP, FP, TN, and FN breast lesions. The methodological quality of a publication was evaluated by the same authors independently using a revised QUADAS tool for assessing the quality of the diagnostic accuracy of a study.¹⁰ The assessment of the tool contained two domains, risk of bias and concern regarding applicability.

The Cochrane Collaboration’s RevMan 5.3 (Oxford, UK) was utilised to evaluate the quality of included literature and obtain a risk-of-bias assessment chart. Meta-DiSc 1.4 (Ramony Cajal Hospital, Madrid, Spain) was applied to perform the data analysis. The sensitivity, specificity, positive and negative likelihood ratios (PLR and NLR), diagnostic odds ratio (DOR) were pooled and calculated, and the corresponding forest plots were acquired, and the area under the SROC (summary receiver operating characteristic) curve was developed. No threshold effect existed in the absence of typical "shoulder-arm" distribution in the curve or when there was a negative Spearman correlation coefficient between the logarithm of sensitivity and that of 1-specificity. The heterogeneity of the study was quantitatively judged by I² and p-values. Mantel-Haenszel’s fixed-effects method was applied when meeting heterogeneity (p >0.1 or I² <50%); otherwise, a random-effects model (with the DerSimonian-Laird approach) was used. The source of heterogeneity (if existed) was explored using subgroup analysis and meta-regression.¹¹ Deeks' funnel plot asymmetry test was used through Stata 14.0 (StataCorp LLC, TX, USA) to gauge potential publication bias.¹² The plot was constructed by the inverse of the square root of the effective sample size (1/ESS^1/2) at the vertical axis, and the diagnostic odds ratio (DOR) at the horizontal axis. The p-value <0.05 implied the significant publication bias.

RESULTS

As shown in Figure 1, the initial literature search in the four databases yielded 1138 articles/studies. After removing duplicate studies, the remaining 875 articles received serious screening and checked for relevance concerning their titles/abstracts; 731 articles were excluded owing to their failure to meet the inclusion criteria. The other 144 full-text ones were assessed for eligibility. Eventually, 11 articles were selected for the meta-analysis.^13-23

As shown in Table I, 5028 BI-RADS category 4 lesions (including 1809 malignant lesions) were recruited in the meta-analysis. All the recruited lesions were from Asian females. The studies were conducted across China (seven articles), Korea (three articles), and Japan (one article). There were seven prospective and four retrospective studies.

Figure 1: PRISMA flow diagram of study selection.

Table I: Characteristics of the included studies.

First Author	Publication Year	Country	Study Design Type	Lesions (n)	Mean Age, (year)	Mean Lesion Size (cm)	Excitation Method	Cut-off	Reference Standard	TP (n)	FP (n)	FN (n)	TN (n)
Cho13	2008	Korea	PRO	69	ND	ND	MC	≥4	PD	8	1	5	55
Gu14	2022	China	PRO	2049	42.91	1.76	MC	≥4	PD	645	264	280	860
Han15	2019	China	PRO	278	44.68	1.46	Cardiovascular/ respiratory pulsation	≥4	PD	114	55	13	96
Hao16	2016	China	RETRO	433	ND	ND	MC	≥4	PD	183	74	14	162
Liang17	2020	China	PRO	104	ND	ND	MC	≥4	PD	42	15	6	41
Liu18	2015	China	PRO	116	ND	ND	MC	≥4	PD	21	3	16	76
Liu19	2014	China	RETRO	92	ND	ND	MC	≥4	PD	64	7	3	18
Li20	2020	China	RETRO	228	48.90	ND	MC	≥4	PD	70	11	56	91
Satake21	2011	Japan	RETRO	52	ND	ND	MC	≥4	PD	18	8	8	18
Yi22	2012	Korea	PRO	1481	ND	ND	MC	≥4	PD	46	42	161	1232
Yoon23	2016	Korea	PRO	126	ND	ND	MC	≥4	PD	27	60	9	30
PRO, Prospective study; RETRO, Retrospective study; ND, Not described; PD, Pathological diagnosis; MC, Manual compression; TP, True positive; FP, False positive; FN, False negative; TN, True negative.

Table II: Result of subgroup analyses for investigating the potential sources of heterogeneity.

Subgroups	Studies (n)	Sensitivity (95%CI)	Specificity (95%CI)	PLR (95%CI)	NLR (95%CI)	DOR (95%CI)	Area under the SROC curve
Overall	11	0.68 (0.66-0.71)	0.83 (0.82-0.85)	3.36 (2.45-4.60)	0.32 (0.21-0.49)	12.11 (7.46-19.65)	0.85
Country	-	-	-	-	-	-	-
China	7	0.75 (0.72-0.77)	0.76 (0.74-0.78)	3.14 (2.62-3.77)	0.24 (0.16-0.36)	17.29 (9.63-31.04)	0.87
Others	4	0.35 (0.30-0.41)	0.92 (0.91-0.94)	3.98 (1.11-14.30)	0.61 (0.41-0.92)	6.46 (1.98-21.10)	0.68
Study Design Type	-	-	-	-	-	-	-
Prospective study	7	0.65 (0.62-0.67)	0.84 (0.83-0.86)	3.61 (2.25-5.79)	0.39 (0.24-0.64)	10.07 (5.74-17.68)	0.82
Retrospective study	4	0.81 (0.76-0.84)	0.74 (0.70-0.79)	3.20 (2.42-4.23)	0.21 (0.06-0.65)	16.39 (6.79-39.56)	0.86
Number of Lesions	-	-	-	-	-	-	-
>200	5	0.67 (0.64-0.69)	0.85 (0.83-0.86)	3.46 (2.69-4.46)	0.32 (0.18-0.58)	11.79 (7.33-18.96)	0.85
<200	6	0.79 (0.73-0.84)	0.72 (0.67-0.76)	3.91 (1.67-9.20)	0.33 (0.19-0.56)	14.88 (4.03-54.89)	0.88
Excitation Method	-	-	-	-	-	-	-
Cardiovascular/ respiratory pulsation	1	0.90 (0.83-0.94)	0.63 (0.55-0.71)	2.46 (1.98-3.07)	0.16 (0.10-0.27)	15.38 (—)	0.82
Manual compression	10	0.67 (0.65-0.69)	0.84 (0.83-0.85)	3.58 (2.48-5.16)	0.35 (0.23-0.53)	11.86 (6.98-20.15)	0.85
CI, Confidence interval; PLR, Positive likelihood ratio; NLR, Negative likelihood ratio; DOR, Diagnostic odds ratio; SROC, Summary receiver operating characteristic.

Table III: Result of meta-regressions for investigating the potential sources of heterogeneity.

Variate	Co-efficient	Standard Error	RDOR (95% CI)	p
Country	1.303	0.6070	3.68 (0.91-14.92)	0.0642
Study Design Type	­0.473	0.7323	0.62 (0.12-3.37)	0.5364
Number of Lesions	-0.159	0.7179	0.85 (0.16-4.47)	0.8302
Excitation Method	-0.307	1.2128	0.74 (0.04-12.06)	0.8067
RDOR, Relative diagnostic odds ratio; CI, Confidence interval.

Five brands of US machines equipped with strain elasto-graphy function were used in the 11 included studies, namely, Hitachi from Japan (five studies), General Electric from the USA (three studies), Siemens from Germany (one study), Mindray from China (one study), and Samsung from Korea (one study). Siemens’s excitation method for strain elastography in the study by Han et al.¹⁵ was cardiovascular/ respiratory pulsation, while manual compression was used for excitation in the other 10 studies. Six studies had a sample size (number of lesions) of less than 200, and the other five had more than 200 sample lesions.

As shown in Figure 2, the risk of bias in the recruited studies came from patient selection, flow and timing. A high risk of bias was detected in four studies regarding patient selection and two studies regarding flow and timing. An unclear risk of bias was present in one study regarding patient selection and three studies regarding flow and timing. Of the 11 included articles, four were retrospective studies and three did not mention the use of consecutive patient enrollment. Besides, exclusion of palpable BI-RADS category 4 lesions and assessment of non-palpable breast lesions alone in the studies by Cho et al.¹³ and Yi et al.²² could introduce a risk of bias.

Figure 2: Quality assessment according to the Diagnostic Accuracy Studies-2 guideline.

In this meta-analysis, the positive Spearman correlation coefficient was 0.718 and the p-value was 0.013, which suggested the existence of a threshold effect and consequently, the potential presence of heterogeneity. As illustrated in Figures 3A–D and Figure 4, the pooled sensitivity, specificity, PLR, NLR, and DOR with the 95% confidence intervals were 0.68 (0.66–0.71) (p=<0.001, I²=97.1%), 0.83 (0.82–0.85) (p<0.001, I²=97.9%), 3.36 (2.45–4.60) (p<0.001, I²=90.3%), 0.32 (0.21–0.49) (p<0.001, I²=96.9%), and 12.11 (7.46–19.65) (p<0.001, I²=81.0%), respectively. As shown in Figure 5, the area under the SROC curve was 0.85, indicating a relatively high diagnostic accuracy of strain elastography in differentiating benign BI-RADS category 4 lesions from malignant ones. The inconsistency (I² value) for sensitivity and specificity was 97.1% and 97.9% respectively (both p<0.001) reflecting a significant heterogeneity within the included studies.

In terms of the substantial heterogeneity, subgroup analyses were performed based on country (China or others), study design type (prospective or retrospective), the number of lesions (>200 or <200), and excitation method (cardio-vascular/respiratory pulsation or manual compression), respectively (shown in Table II). Besides, as shown in Table III, the meta-regressions confirmed that none of the four subgroup factors was the source of heterogeneity. With the symmetrical shape of the funnel plot, the absence of publication bias was suggested by the p-value of 0.19 (Figure 6).

Figure 3: Forest plots of (A) sensitivity, (B) specificity, (C) PLR, (D) NLR for qualitative analysis of stain elastography in the diagnosis of BI-RADS category 4 lesions.

Figure 4: Forest plot of DOR for qualitative analysis of stain elastography in the diagnosis of BI-RADS category 4 lesions.

Figure 5: Summary receiver operating characteristic (SROC) curve for qualitative analysis of stain elastography in the diagnosis of BI-RADS category 4 lesions.

Figure 6: The Deek's funnel plot of asymmetry test for evaluating publication bias among the included studies. No significant publication bias was found. ESS, Effective sample size.

DISCUSSION

BI-RADS category 4 lesions are usually suspicious enough to warrant a recommendation for biopsies, especially in the absence of typical imaging for malignancy. A prospective multicentre study in China reported that less than half of BI-RADS category 4 lesions, (925/2049) were pathologically confirmed to be malignant.¹⁴ In a clinical practice, over-diagnosis of breast lesions brings about overtreatment and collateral increase in patients’ medical expenses, as well as unnecessary anxiety for the patients. Hence, more accurate non-invasive diagnostic methods are required to raise the specificity of diagnosing BI-RADS category 4 lesions and thus, reduce the requirement for biopsies. Encouragingly, shear wave and strain elastography can provide a better quantitative as well as qualitative assessment of breast masses. Their addition to conventional B-mode US could increase the specificity from 0.27 to 0.76 without loss in sensitivity (p <0.001).²⁴ Several studies have recently demonstrated that the strain elastography-derived elasticity scores above 3 are highly associated with malignant breast lesions (p <0.05).^20,25 Hao et al. proposed that category 4A lesions, when assessed with an elasticity score of 1, could be downgraded to category 3 (without the intervention of biopsies); a short-term follow-up or tissue-based diagnosis was recommended for the category 4A lesions with a score of 2 or 3.¹⁶

In this meta-analysis, 11 eligible studies were finally included, and the recruited lesions were all from Asian females. The pooled sensitivity and specificity were 0.68 and 0.83, respectively, and the area under the SROC was 0.85, which manifested a relatively high diagnostic value for the category 4 lesions. Another similar meta-analysis focused on the elasticity score for all breast lesions and achieved consistent but superior sensitivity (0.77) and specificity (0.88).⁷ Most category 2 and 3 lesions are benign and the majority of category 5 lesions are considered malignant. Thus, the reliable diagnostic outcome based on elasticity scores from these lesions could explain the slight gap in sensitivity and specificity. FP and FN results are unavoidable when applying stain elastography, partly due to the overlap between the characteristics of benign and malignant category 4 lesions. Increased stiffness of lesions owing to calcification in fibroadenoma, atypical ductal hyperplasia, and sclerosing adenosis, would yield a higher FP rate. The papillary proliferation of papillomas in the mammary duct and secondary changes (haemorrhage, fibrosis, or infarction) could affect the elasticity scores and FP results.^21,25 FN results mainly come from mucinous carcinoma and ductal carcinoma in situ. The mucin-rich structure of the former could cause lower elasticity scores.²¹ The unobvious invasive and desmoplastic reactions of the latter at its early stage would lead to relatively small stiffness.²² The pooled PLR and NLR in this meta-analysis were 3.36 and 0.32. This means that breast cancer patients had 3.36 times greater odds (95% CI 2.45-4.60) of receiving an elasticity score of above 3 than healthy controls, and the error rate was 32% when confirming the TN cases through breast elasticity scoring.²⁶ DOR serves as a single indicator for the accuracy of a diagnostic test, and its higher value indicates a better discriminatory test performance.²⁷ The DOR in the present meta-analysis was 12.11 (95% CI 7.46–19.65), indicating a relatively high diagnostic value of elasticity score in category 4 lesions.

Exploration of heterogeneity was an essential issue when running a meta-analysis. The Spearman correlation coefficient was 0.718 (p <0.05) in this meta-analysis, which indicated the presence of a threshold effect (a primary cause of heterogeneity). Afterwards, through subgroup analysis, it was found that the retrospective studies showed relatively higher sensitivity and DOR but lower specificity than the prospective studies. The studies conducted in China possessed a better diagnostic performance than those performed in other countries. Besides, the groups with a smaller sample size presented higher accuracy in diagnosing BI-RADS category 4 lesions through stain elastography compared with the groups with a larger sample size. Notably, the only study, whose strain elastography was excited by cardiovascular/respiratory pulsation, had higher sensitivity but lower specificity in contrast to the pooled corresponding values of the other ten studies (through manual compression). Further large-size studies concentrating on the latter three subgroup factors are needed to confirm these findings. Meta-regressions were then performed from four aspects (country, study design type, number of lesions, and excitation method), and none of them significantly affected the heterogeneity.

Without a doubt, several limitations exist in this meta-analysis. Firstly, in the very beginning it aimed to focus on category 4 lesions among females, but the recruited lesions in the included studies were all from oriental Asian females (Chinese, Korean and Japanese) and not from South Asia. In addition, over half of lesions (3300/5028) were from Chinese females, and there were obvious differences between Asian and non-Asian women in breast masses. Therefore, the findings might not be extrapolated to other Asian or non-Asian female populations. More multicentric and prospective studies involving patients from various countries or regions are required for further confirmation. Secondly, substantial heterogeneity was detected in this meta-analysis, but subgroup analyses and meta-regressions did not detect any sources of heterogeneity. Thirdly, the data about mean breast lesion size were provided in only two included studies, and the influence of lesion size on elasticity degree was not studied in this meta-analysis. Hence, the influence of the size or mean size of category 4 lesion on elasticity score in the diagnostic accuracy test demands further investigation.

CONCLUSION

This meta-analysis indicated that the diagnostic accuracy of strain elastography has unsatisfactory sensitivity but desirable specificity for BI-RADS category 4 lesions among Asian females, which can reduce unnecessary biopsies and decrease patient anxiety and medical cost.

FUNDING:
This work was supported by the Yichang Science and Technology Bureau Project, China (Grant No. A21-2-029).

ACKNOWLEDGEMENT:
The authors would like to sincerely acknowledge the assistance of the combined faculty of the Department of Ultrasound, The People’s Hospital of China Three Gorges University and The First People's Hospital of Yichang, for providing guidance and necessary critique during the study period. In particular, we are grateful to Prof. Chaojun Yang and Jun Yang for their intensive role in this work. We are also grateful to the library management for providing us with the electronic databases that made this work possible.

COMPETING INTEREST:
The authors declared no competing interest.

AUTHORS’ CONTRIBUTION:
MD, BX: Designed the study, searched the literature, collected and analysed the data.
XX, QZ, BZ: Collected the data and provided inter-pretation of the results.
BZ: Drafted the manuscript.
All authors revised and approved the final version of the manuscript.
 

REFERENCES

1. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, et al. Global cancer statistics 2020: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 2021; 71(3): 209-49. doi: 10.3322/caac.21660. 
2. Rao AA, Feneis J, Lalonde C, Ojeda-Fournier H. A pictorial review of changes in the BI-RADS fifth edition. Radiographics 2016; 36(3):623-39. doi: 10.1148/rg.2016150178.
3. Ophir J, Céspedes I, Ponnekanti H, Yazdi Y, Li X. Elastography: A quantitative method for imaging the elasticity of biological tissues. Ultrason Imag 1991; 13(2): 111-34. doi: 10.1177/016173469101300201.
4. Park SY, Kang BJ. Combination of shear-wave elastography with ultrasonography for detection of breast cancer and reduction of unnecessary biopsies: A systematic review and meta-analysis. Ultrasonography 2021; 40(3):318-32. doi: 10.14366/usg.20058.
5. Hong S, Li W, Gao W, Liu M, Song D, Dong Y, et al. Diagnostic performance of elastography for breast non-mass lesions: A systematic review and meta-analysis. Eur J Radiol. 2021; 144:109991. doi: 10.1016/j.ejrad.2021. 109991.
    
6. Pillai A, Voruganti T, Barr R, Langdon J. Diagnostic accuracy of shear-wave elastography for breast lesion characterization in women: A systematic review and meta-analysis. J Am Coll Radiol 2022; 19(5):625-34.e0. doi: 10.1016/j.jacr.2022.02.022.
7. Barr RG, De Silvestri A, Scotti V, Manzoni F, Rebuffi C, Capittini C, et al. Diagnostic performance and accuracy of the 3 interpreting methods of breast strain elastography: A systematic review and meta-analysis. J Ultrasound Med 2019; 38(6):1397-404. doi: 10.1002/jum.14849.
8. Mutala TM, Mwango GN, Aywak A, Cioni D, Neri E. Determining the elastography strain ratio cut off value for differentiating benign from malignant breast lesions: Systematic review and meta-analysis. Cancer Imag 2022; 22(1):12. doi: 10.1186/s40644-022-00447-5. 
9. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021; 372:n71. doi: 10.1136/bmj.n71.
10. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011; 155(8):529-36.doi: 10.7326/0003-4819-155-8- 201110180-00009.
11. Zamora J, Abraira V, Muriel A, Khan K, Coomarasamy A. Meta-DiSc: A software for meta-analysis of test accuracy data. BMC Med Res Methodol 2006; 6:31. doi: 10.1186/ 1471-2288-6-31.
12. Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol 2005; 58(9):882-93. doi: 10.1016/j.jclinepi. 2005.01.016.
13. Cho N, Moon WK, Park JS, Cha JH, Jang M, Seong MH. Nonpalpable breast masses: Evaluation by US elasto-graphy. Korean J Radiol 2008; 9(2):111-8. doi: 10.3348/ kjr.2008.9.2.111.
14. Gu Y, Tian JW, Ran HT, Ren WD, Chang C, Yuan JJ, et al. The utility of the fifth edition of the BI-RADS ultrasound lexicon in category 4 breast lesions: A prospective multicenter study in China. Acad Radiol 2022; 29 Suppl 1:S26-S34. doi: 10.1016/j.acra.2020.06.027. 
15. Han J, Li F, Peng C, Huang Y, Lin Q, Liu Y, et al. Reducing unnecessary biopsy of breast lesions: Preliminary results with combination of strain and shear-wave elastography. Ultrasound Med Biol 2019; 45(9):2317-27. doi: 10. 1016/j.ultrasmedbio.2019.05.014. 
16. Hao SY, Jiang QC, Zhong WJ, Zhao XB, Yao JY, Li LJ, et al. Ultrasound elastography combined with BI-RADS-US classification system: Is it helpful for the diagnostic performance of conventional ultrasonography? Clin Breast Cancer 2016; 16(3):e33-41. doi: 10.1016/j.clbc.2015.10. 003. 
17. Liang M, Ou B, Wu J, Xiao X, Ruan J, Tian J, et al. Combined use of strain elastography and superb microvascular imaging with grayscale ultrasound according to the BI-RADS classification for differentiating benign from malignant solid breast masses. Clin Hemorheol Microcirc 2020; 74(4):391-403. doi: 10.3233/CH-190693.
18. Liu D, Huang Y, Tian D, Yin J, Deng LJ. Value of sonographic bidirectional arterial flow combined with elastography for diagnosis of breast imaging reporting and data system category 4 breast masses. J Ultrasound Med 2015; 34(5):759-66. doi: 10.7863/ultra.35.5.759.
19. Liu XJ, Zhu Y, Liu PF, Xu YL. Elastography for breast cancer diagnosis: A useful tool for small and BI-RADS 4 lesions. Asian Pac J Cancer Prev 2014; 15(24):10739-43. doi: 10.7314/apjcp.2014.15.24.10739.
20. Li XL, Lu F, Zhu AQ, Du D, Zhang YF, Guo LH, et al. Multimodal ultrasound imaging in breast imaging-reporting and data system 4 breast lesions: A prediction model for malignancy. Ultrasound Med Biol 2020; 46 (12):3188-99. doi: 10.1016/j.ultrasmedbio.2020.08.003.
21. Satake H, Nishio A, Ikeda M, Ishigaki S, Shimamoto K, Hirano M, et al. Predictive value for malignancy of suspicious breast masses of BI-RADS categories 4 and 5 using ultrasound elastography and MR diffusion-weighted imaging. AJR Am J Roentgenol 2011; 196(1):202-9. doi: 10.2214/AJR.09.4108.
22. Yi A, Cho N, Chang JM, Koo HR, La Yun B, Moon WK. Sonoelastography for 1,786 non-palpable breast masses: Diagnostic value in the decision to biopsy. Eur Radiol 2012; 22(5):1033-40. doi: 10.1007/s00330-011-2341-x.
23. Yoon JH, Song MK, Kim EK. Semi-quantitative strain ratio in the differential diagnosis of breast masses: Measure-ments using one region-of-interest. Ultrasound Med Biol 2016; 42(8):1800-6. doi: 10.1016/j.ultrasmedbio.2016. 03.030.
24. Lee SH, Chung J, Choi HY, Choi SH, Ryu EB, Ko KH, et al. Evaluation of screening us-detected breast masses by combined use of elastography and color doppler US with B-Mode US in women with dense breasts: A multicenter prospective study. Radiol 2017; 285(2):660-9. doi: 10. 1148/radiol.2017162424.
25. Kanagaraju V, Dhivya B, Devanand B, Maheswaran V. Utility of ultrasound strain elastography to differentiate benign from malignant lesions of the breast. J Med Ultrasound 2020; 29(2):89-93. doi: 10.4103/JMU.JMU_ 32_20.
26. Deeks JJ, Altman DG. Diagnostic tests 4: Likelihood ratios. BMJ 2004; 329(7458):168-9. doi: 10.1136/bmj.329. 7458.168. 
27. Glas AS, Lijmer JG, Prins MH, Bonsel GJ, Bossuyt PM. The diagnostic odds ratio: A single indicator of test performance. J Clin Epidemiol 2003; 56(11):1129-35. doi: 10.1016/s0895-4356(03)00177-x.