Machine Learning-Based Shear Wave Elastography Elastic Index (SWEEI) in Predicting Cervical Lymph Node Metastasis of Papillary Thyroid Microcarcinoma: A Comparative Analysis of Five Practical Prediction Models

Introduction

Worldwide, thyroid cancer, as the most common endocrine malignancy, accounts for about one-third of the total number of head and neck malignancies.^1–3 In all cases of thyroid cancer, more than four-fifths of new cases are papillary thyroid carcinoma (PTC).^4,5 Alarmingly, one-third of new cases of thyroid cancer are papillary thyroid microcarcinoma (PTMC), whose diameter is less than or equal to 1 cm.⁶ Because of its invasive and metastatic characteristics, as well as poor early diagnosis, the prognosis of PTMC is still fatal. So far, surgery is still the first choice for the treatment of PTMC, but the treatment of cervical lymph nodes is still controversial.^7,8 It is worth mentioning that although PTMC is considered to be an inert tumor, some cancer cells will metastasize to lymph nodes around the thyroid gland, including central lymph nodes and lateral cervical lymph nodes. Therefore, it is of great significance for improving the prognosis of patients to evaluate cervical lymph node metastasis(CLNM) reasonably and accurately before elective surgery.

For a long time, several studies have been proposed to evaluate the risk of CLNM in PTMC patients, among which imageomics is the most famous.^9,10 With the continuous improvement of diagnostic technology, especially the improvement of ultrasonic resolution, the detection rate of thyroid nodules is also increasing. Previous studies have shown that preoperative ultrasound is a valuable tool to evaluate CLNM in patients with PTMC, which can provide relatively reliable lateral neck information to assist in surgical treatment.¹¹ Nevertheless, as a non-invasive diagnostic method, the accuracy of ultrasonic diagnosis still needs to be further improved. Previous studies have shown that preoperative ultrasound can detect only about one-third of central cervical lymph nodes, and may change the surgical procedure in only one-fifth of patients.^12–14 Given this situation, there is an urgent need for a non-destructive and effective method to predict the LNM risk of PTMC patients and guide the clinical diagnosis and treatment process.

Recently, ultrasound-based shear wave elastography (SWE) and machine learning(ML) algorithms have been widely used in medical research.^15–18 It is worth mentioning that the prediction model built based on machine learning algorithm can iterate on valuable potential variables for many times, and finally realize the optimization of the prediction model to improve the accuracy of its prediction.¹⁹ In this study, we established a prediction model of machine learning algorithm based on B-mode ultrasound images of shear wave elastography elastic index(SWEEI) to predict the CLNM risk of PTMC patients. We expected that the prediction model based on machine learning can provide relatively reliable lateral neck information to assist surgical treatment in the future.

Methods

Process of the Patient Enrollment

We retrospectively analyzed 387 PTMC patients hospitalized in the Department of Medical Oncology, Enshi Tujia and Miao Autonomous Prefecture Central Hospital from January 1, 2015, to January 31, 2022. Patient inclusion criteria were as follows:(i) Patients underwent thyroid cancer surgery for the first time;(ii) Classic PTMC was confirmed by pathology after operation;(iii) Patients with complete clinical, pathological and ultrasonic imaging information. The exclusion criteria were as follows:(i) Patients had PTMC with other cell subtypes;(ii) PTMC with tumors of other systemic systems;(iii) Patients with serious heart, liver, kidney, or other serious systemic diseases;(iv) Patients with incomplete clinical data. This study followed the declaration of Helsinki and was approved by the Ethics Committee of the Central Hospital of Enshi Tujia and Miao Autonomous Prefecture(EN-20220109). Patients/participants have provided written informed consent to participate before operation and the privacy of all patients is also guaranteed in this study. The construction steps of the screening and prediction model for patients were shown in Figure 1.

Figure 1 The flow chart of patient inclusion and CLNM prediction model construction.

Acquisition of Shear Wave Elastic(SWE) Imaging

Step1: Ultrasonic examination. In this study, a color ultrasonic diagnostic instrument (French sound imaging company, model: aixplorer; probe frequency:5.6~10.0 MHz) was used to obtain the ultrasonic image data of patients. The patient was in the supine position, with the patient’s head tilted back to fully expose the neck, and the neck and thyroid were explored. The size, location, morphology, aspect ratio, boundary, the relationship between capsules, blood flow, and calcification of thyroid nodules were observed, and the suspicious lesions were located. Step2: SWE inspection. After locating the suspicious lesions, adjust the ultrasound to SWI mode, and the probe frequency is 5.6~10.0 MHz; The preset Young’s modulus value is 0~180 kPa; Move the signal frame to the target lesion, including the surrounding normal thyroid tissue and muscle tissue as far as possible; The patient held breath and stood for 3~5 seconds, then intercepted the image, repeated for more than 3 times, each time lasting for more than 10 seconds, and excluded the unqualified image. Step3: Parameter acquisition. The computer system automatically calculates SWE parameters, including average value (Emean), maximum value (Emax), minimum value (Emix), and takes the final measurement average, etc.

Construction of ML-Based Prediction Model

In this study, with the help of supervised learning, each instance is composed of an input object (ie SWEEI) and a desired output value (ie CLNM). In short, the supervised learning algorithm analyzes the training data and generates an inference function, which can be used to map out new instances. Based on supervised learning algorithms, namely random forest classifier(RFC), artificial neural network(ANN), support vector machine(SVM), decision tree(DT), and extreme gradient boosting gradient(XGboost) algorithm, the CLNM prediction model was constructed.^20–25 Additionally, the prediction efficiency of the ML-based model was evaluated via receiver operating characteristic curve(ROC) and decision curve analysis(DCA).

Statistical Analysis

For continuous and discontinuous data, interquartile range(IQR) and percentage (%) are used respectively. Chi-square test, Mann–Whitney U-test, or Wilcoxon test were used for statistical analysis between CLNM and non- CLNM groups, respectively. In addition, the caret software package was used to randomly divide all data into the training set and verification set in proportion (7:3). The data analysis and visualization involved in this study were completed with the help of R software(version 4.0.4, http://www.r-project.org/). All P values were two-tailed, and P <0.05 was considered statistically significant.

Results

Baseline Characteristics and SWE Data of Patients With or Without CLNM

In this study, following the principle of random grouping via caret package, a total of 387 inpatients were randomly divided into a training set and a testing set according to a 70% to 30% ratio (Table 1). The incidence of CLNM in the training set and the validation set were 28.52% and 29.06% respectively. Among the patients with CLNM, there were 87 (78.38%) central and 24 (21.62%) lateral CLNM, respectively (Supplementary Table 1). In the comparison of SWE between CLNM and non-CLNM patients, aspect ratio, blood flow, blood flow typing, echo intensity, pulsatility index (PI), and resistance index (RI) were statistically significant (P<0.05). In addition, relevant indicators based on SWEEI, including average value(Emean), maximum value(Emax) and minimum value(Emix), statistically significant (P < 0.01). Previous studies have shown that LNM in the lateral cervical region usually has typical ultrasonic features, including microcalcification, partial cystic appearance, increased vascularization, and hyperecho.²⁶ As shown in the Supplementary Table 1, based on the candidate variables detected by SWEEI, there was no significant statistical significance in the central and lateral CLNM (P>0.05), which suggested that there was no objective bias in the prediction of central and lateral CLNM by SWEEI.

Table 1 Clinicopathological Features and Ultrasonic Indexes of Patients

Screening of Candidate Predictive Variables for ML-Based Prediction Model

As previously reported, the Pearson correlation analysis is a concise analysis method to measure the relationship between quantitative data, which can analyze the relationship between variables and the strength of the relationship.²⁷ As shown in Figure 2A, CLNM, as a dependent variable, was significantly correlated with the SWEEI, including Emean(r=0.704), Emax(r=0.778), and Emix(r=0.467). Similarly, the iterative algorithm of RFC drew the same conclusion, in which the top seven candidate variables were PI, peak systolic blood flow velocity(PSBV), RI, Emax, Emix, and systolic acceleration time(SAT), see Figure 2B. Consistent with the conclusions of other machine learning algorithms, this shows that ultrasonic image acquisition data combined with SWEEI is expected to become a potential predictor of CLNM (Supplementary Figure 1).

Figure 2 Correlation analysis between candidate variables and lymph node metastasis (A) Spearman correlation matrix analysis (B) Weight ranking of random forest classifier.

Construction of ML-Based Prediction Model Based on Training Set

For training data, each patient has a result (CLNM or non-CLNM), and the final judgment result was output. As shown in the formula: . The RFC algorithm represents a computational method for effectively navigating the free parameter space to obtain a robust model (Figure 2A). The variable Gini index in the RFC model was depicted in Supplementary Table 2. As shown in Figure 3, data mining through the DT model was very useful, as shown by impurity analysis: . Among the candidate variables related to PSBV, Emean, Emax, and Emix also played an important role in DT as “branch weight”, which can be used as an important predictor of CLNM. At the same time, the ANN model also showed more robust prediction efficiency than other models, but it was slightly inferior to RFC (Figure 4).

Figure 3 Visualization of lymph node metastasis prediction model based on decision tree classifier. Notes: According to the weight of the candidate variables, the corresponding weight is allocated to each branch of the decision tree. According to the probability that the weight accounts for 50% respectively, the next node is finally entered, and the final input result (whether lymph node metastasis) is obtained through multiple iterations.

Figure 4 Visualization of lymph node metastasis prediction model based on neural network algorithm (A) Hierarchical model visualization of neural network model (B) Weight distribution of candidate variables.

Prediction Performance of ML-Based Prediction Model

To explore the effectiveness of five supervised learning models for postoperative pf evaluation, we used the area under curve(AUC) and decision curve analysis(DCA) for evaluation, which was consistent with the results of the included candidate variables. Even if different prediction models included the same variables, there were certain differences in their prediction effectiveness, as shown in Table 2 and Figure 5. In addition, the prediction efficiency of RFC was the best (0.889, 95% CI:0.838–0.940) compared with the other four prediction models, followed by ANN (0.854, 95% CI:0.803–0.905), DT (0.812, 95% CI:0.761–0.863), XGboost (0.792, 95% CI:0.741–0.843), and SVM (0.767, 95% CI:0.716–0.818) in the training set. In the internal validation set, we also tested five machine learning prediction models, and the results were highly consistent with the training set. Undoubtedly, the iterative algorithm analysis using supervised learning, RFC, and ANN, as well as DT (machine learning aided decision support) models were properly used to guide postoperative CLNM prediction.

Table 2 Efficiency Evaluation of Prediction Model Based on Five Machine Learning Algorithms

Figure 5 Efficiency evaluation of prediction model based on machine learning algorithm. Notes: Decision curve distribution of five prediction models in (A) training set and (B) testing set.

Discussion

PTMC, as an inert tumor, LNM will occur in the early stage, so it needs to be paid enough attention. Clinically, the decision of lymph node metastasis(central or lateral cervical region) during thyroidectomy usually depends on whether the lymph nodes suspected of malignancy can be identified before surgery.^11,28,29 Given this situation, unnecessary lymph node dissection will lead to more recurrent laryngeal nerve injury, leading to serious postoperative complications, which will affect the prognosis and quality of life of patients with PTMC.³⁰ Vigilantly, CLNM is not only an important indicator of PTMC prognosis, the scope, and mode of surgery but also an important risk factor for patients with high recurrence rates and low survival rates. Therefore, it is urgent to adopt predictive methods for accurate preoperative evaluation. The main outstanding contributions of our research are as follows: First, the elasticity index based on SWEEI predicts cervical lymph node metastasis of papillary thyroid cancer, which improves the prediction efficiency of ultrasound imaging. Second, with the help of machine learning algorithm, the iterative analysis of SWEEI can be applied to different clinical scenarios.

Previous studies have reported potential prognostic biomarkers that can be used for disease recurrence in patients with papillary thyroid cancer, such as the role of PD-L1 immunohistochemistry and Hashimoto’s thyroiditis as a potential prognostic biomarker for disease recurrence in patients with papillary thyroid cancer.^31,32 Nowadays, newly studies have shown that there may be a strong correlation between some characteristics of ultrasound examination and CLNM, but the conclusions of these studies are contradictory, and there are still doubts about the accuracy of predicting CLNM.^11,33 Similarly, some prediction models show that ultrasound features, including tumor size, thyroid infiltration, and microcalcification, can also be potential predictors of CLNM.^34,35 In this study, the CLNM prediction model based on SWEEI has ideal prediction accuracy, which highlights the advantages of machine learning iterative algorithm compared with the traditional prediction model. It also suggested that even the same ultrasonic imaging indicators can be applied to LNM prediction after adopting optimized algorithm input. We summarized our experience as follows. On the one hand, when moving to the new iteration layer, machine learning algorithm can avoid the overfitting problem in the training data set, so as to reduce the amount of data required for modeling. On the other hand, it improves the generalization ability of modeling, so as to make the model more robust. In addition, our model also was confirmed in the validation set, which prompted us to verify the clinical availability of the machine learning algorithm model in cross-machine and non-operator scenarios.

It is no wonder that machine learning has been widely used in the field of medicine, especially in clinical prediction models. In addition, with the solution of technical problems, the application of automatic algorithms in thyroid pathology will increase and be adopted after appropriate validation studies.³⁶ In view of this, the focus of our research is to determine how to change medical practice through statistical learning methods, and discuss how to overcome these obstacles, and then find a better prediction model.¹⁹ Among the five prediction models we have established, we are surprised to find that the prediction efficiency of RFC is the best, which is consistent with the results of previous studies.^37–39 For instance, a random forest can generate a large number of decision trees based on the random sub-samples of the training set, and also randomly change the characteristics used in the tree, so the iteration is uncertain.⁴⁰ Therefore, the random forest model can be regarded as a set of many decision tree models. In this study, we found that SWEEI can become the entropy of RFC. As the depth of the tree increases, the entropy should be reduced (the faster the entropy is reduced, the higher the efficiency of the decision tree). Therefore, the candidate entropy finally included includes PI, PSBV, RI, Emax, Emix, and SAT. Coincidentally, even in the ANN model, these entropy values also show strong prediction efficiency but are slightly inferior to the RFC model. Taken together, we are reminded that SWEEI combined with other ultrasonic image indicators can be used as a predictor of CLNM, but the algorithm is also an important factor that cannot be ignored in building the prediction model.

Our research also inevitably has the following limitations. First, Our current study is limited by retrospective analysis, which may be confused by selection bias. Second, the CLNM prediction of PTMC patients based on multimodal ultrasound is mainly applied to the limited and single-center cohort. In the future, it is expected to further improve the performance of the prediction model, which needs to be verified by the multi-center scenario. Third, some captured features on the ultrasound image data set show defects, which may mask the potential correlation between LNM and some variables. In the future, it still needs to be further optimized, such as symbiotic gray matrix, to improve the utilization efficiency of ultrasound image omics.

Conclusion

In conclusion, based on ultrasound imaging technology(such as SWEEI) and machine learning algorithm, we have developed a stable and powerful feature code for evaluating CLNM and the benefits of PTMC-based patients. Among them, the RFC model has the best prediction efficiency and the strongest practicability. This prediction model is a promising tool to optimize the decision-making and monitoring scheme of whether a single PTMC patient merges CLNM.