The clinical records of 526 patients who were diagnosed with UEGC were confirmed through pathological examination after radical gastrectomy without endoscopic treatment at four tertiary hospitals. These hospitals are Shaanxi Provincial People’s Hospital, Shaanxi Provincial Tumour Hospital, the First Affiliated Hospital of Xi’an Jiaotong University, and the Second Affiliated Hospital of Xi’an Jiaotong University. The clinical records were between January 2015 to December 2021 and were retrospectively reviewed. The following were the inclusion criteria: (1) Imaging examination was performed; (2) Patients have a complete set of medical data; (3) Primary lesion was resected either via open surgery or laparoscopic surgery and not via EMR or ESD; and (4) The status of infiltrating lymph nodes was assessed through routine hematoxylin-eosin staining. To minimize the confounding effect of unnecessary variables, the following were the exclusion criteria: (1) Sufficient information cannot be extracted or mismatched clinical data of patients; and (2) Patients without complete magnetic resonance imaging (MRI) plain scan or the MRI image quality being unacceptable. This study complies with the provisions of the Helsinki Declaration (revised in 2013) and was approved by the Institutional Review Committee of Shaanxi Provincial People’s Hospital (2021-Y024). Figure 1 presents in detail the patient screening steps and modeling process.

Open in New Tab   Full Size Figure   Download Figure

Figure 1 Flowchart of patient selection and data processing. UEGC: Undifferentiated early gastric cancer; EMR: Endoscopic mucosal resection; ESD: Endoscopic submucosal dissection; RFC: Random forest classifier; SVM: Support vector machine; DT: Decision tree; ANN: Artificial neural network; XGboost: Extreme gradient boosting; ROC: Receiver operating characteristic; DCA: Decision curve analysis; CIC: Clinical impact curve; LNM: Lymph node metastasis.

All texture parameter post-processing was conducted on Omni dynamics software (GE pharmaceuticals, Shanghai). Two radiologists who have vast experience in gastrointestinal diagnosis referred to the MRI images to sketch the lesions on the ADC map. First, they manually sketch the entire area with cancer on each layer of the map, avoiding the gas in the intestine, until the whole tumor volume was cut out. Second, the software automatically generates the texture features. In this study, the following are the selected texture parameters of the GLCM: Total frequency, energy value, entropy, inertia value, correlation coefficient, inverse moment, cluster shadow, and cluster prominence.

For variables with missing values (often this missing value is less than 10%), the variable’s mean value should be filled. If ≥ 10% of the given variables are missing, this value is excluded from the variable screening of the final model. Similarly, this study adopted unit feature interpolation for the missing values that meet the interpolation requirements. That is, the missing values can be interpolated using the constant values provided or using the statistical data of each column where these missing values are located (e.g., average value, median value, or the most frequently occurring value)[16,17].

Based on the machine learning (ML) algorithm, the commonly used iterative algorithm models are included: Random forest classifier (RFC), decision tree (DT), support vector machine (SVM), eXtreme gradient boosting (XGBoost), and artificial neural network (ANN). The RFC is an integrated method that forms a cumulative effect by integrating multiple relatively simple evaluators. Random forest is an integrated learning tool based on DT. The SVM is a type of a generalized linear classifier that categorizes data binary through supervised learning. The ANN is a nonlinear equation transformation output algorithm comprising input, hidden, and output layers. Finally, XGBoost is an additive model. In each iteration, only the sub-models in the current step are optimized. In this study, we refer to the guide proposed by Luo et al[18] for the best use of prediction models in biomedical research, that is, the Delphi method, which is used to generate the list of reported items.

For the screening of candidate variables, we mainly rely on the principle of “bag repeatedly put back and extract”, sort according to variables’ weight, and finally obtain the final predictor of the prediction model from the top 10 variables[19]. For the effectiveness evaluation of the prediction model, the receiver operating characteristic (ROC) curve is used to evaluate the accuracy of the model. Meanwhile, the decision curve analysis and clinical impact curve (CIC) were used to evaluate the model’s robustness and differentiation, respectively.

The measurement and counting data in this study are expressed by interquartile spacing (25%, 75%) and percentage (%), respectively. For the comparison between groups, the continuous variables adopt the t-test or Mann-Whitney U test of independent samples (provided that it does not conform to the normal distribution). The counting data adopt the chi-square goodness-of-fit test. Values of Bonferroni corrected probability are used to compare the qualitative data[20]. The prediction model visualization and other data analysis are performed using R software (version 4.0.4, http://www.r-project.org/). For the comparison between groups, P value < 0.05 is considered statistically significant and vice versa.

Table 1 summarizes the baseline characteristics of 526 hospitalized patients with UEGC. For internal validation, the patients were randomly divided into two sets using the caret package: Training set (n = 368, 70%) and validation set (n = 158, 30%). Regarding the LNM rate, the training and validation cohorts were 62 (16.85%) and 29 (18.35%), respectively. In addition to the previously reported clinical-related indicators (e.g., tumor size, infiltration depth, vascular_invasion, and vascular tumor thrombus), significant differences exist between the LNM and non-LNM groups. We found that GLCM-based texture acquisition features also have significant statistical differences between the two groups.

We conducted a correlation analysis on the variables with significant differences based on the statistical difference analysis of baseline data. As shown in Figure 2A, the correlation matrix (based on Pearson correlation analysis) indicates that the characteristic variables in the GLCM and LNM had a strong correlation degree (r > 0.6). For example, Entropy, Haralick full angle (Haralick_all), Haralick 30° (Haralick_30), Inverse gap full angle (IG_all), Inverse gap 45° (IG_45), Inverse gap 0° (IG_0), etc. were highly correlated with LNM. This suggests that these potential candidate variables can be used as LNM predictors and for the construction of subsequent models. Interestingly, in the subsequent models developed based on ML algorithms, we found that Entropy, Haralick_all, Haralick_30, IG_all, IG_45, IG_0, and Inertia value 45° (IV_45) occupied high weights as the top 7 GLCM-based factors (Figure 2B). Specifically, Entropy has the largest weight among these factors.

Open in New Tab   Full Size Figure   Download Figure

Figure 2 Variable screening and weight allocation. A: Correlation matrix analysis of candidate features; B: Weight distribution of candidate variables for each mL based model. RFC: Random forest classifier; SVM: Support vector machine; DT: Decision tree; ANN: Artificial neural network; XGboost: Extreme gradient boosting.

When constructing the RFC model [training set: Areas under the ROC curve (AUC): 0.925, 95% confidence interval (CI): 0.378-1.472; testing set: AUC: 0.912, 95%CI: 0.355-1.469], we repeatedly randomly selected N samples from the original training set N to generate the new training set DT and then generate M DTs to form a random forest according to the above steps. As shown in Figure 3A and Supplementary Table 1, the smallest Gini index after splitting was selected, including that for Entropy, Haralick_all, Haralick_30, IG_all, IG_45, IG_0, and IV_45. Similarly, Haralick_30 and IG_all served as important weight at DT branches (training set: AUC: 0.856, 95%CI: 0.309-1.403; testing set: AUC: 0.813, 95%CI: 0.256-1.370) (Figure 3B). In the ANN model (Figure 4), the accuracy of the prediction model developed using the prediction variables in the GLCM can also reach 0.887 (95%CI: 0.340-1.434) and 0.837 (95%CI: 0.280-1.394) in the training and verification sets, respectively. Although this accuracy is slightly inferior to that of the RFC model, it is better than those of other prediction models (i.e., DT, XGBoost, and SVM). Table 2, Supplementary Table 1, and Figure 5 summarize the predictive performance of ML-based models. In general, the prediction model constructed by using any ML algorithm was better than the logistic regression algorithm in predicting LNM, further confirming the superiority of ML algorithm, especially the robustness of the RFC.

Open in New Tab   Full Size Figure   Download Figure

Figure 3 Visualization model prediction based on machine learning based algorithm. A: Random forest classifier model; B: Decision tree model. Candidate factors associated with fracture risk are named through random forest classifier algorithm, and prediction nodes and weights are assigned by the decision tree algorithm.

Open in New Tab   Full Size Figure   Download Figure

Figure 4 Visualization of prediction models based on artificial neural network algorithm. A: Artificial neural network model; B: Importance of variables using connection weights. Candidate factors associated with lymph node metastasis are ordered via artificial neural network (ANN) algorithm and prediction nodes, and weights are assigned via an ANN algorithm. IV_0: Inertia value 0°; IV_45: Inertia value 45°; IG_0: Inverse gap 0°; IG_45: Inverse gap 45°; IG_all: Inverse gap full angle; Haralick_30: Haralick 30°; Haralick_all: Haralick full angle.

Open in New Tab   Full Size Figure   Download Figure

Figure 5 Predictive performance of candidate models based on machine learning based algorithm. A: Decision curve analysis (DCA) for five mL based models in training sets; B: DCA for five ml based models in test sets. RFC: Random forest classifier; SVM: Support vector machine; DT: Decision tree; ANN: Artificial neural network; XGboost: Extreme gradient boosting.

The prediction efficiency of the RFC model was the best in the process of precise stratification of LNM patients. To further evaluate the “stratification effect” of the RFC, results of CIC analysis indicate that high-risk LNM was accurately distinguished using the RFC model, and “cross-linking” did not occur in the stratification process. The results of this model for the validation and training sets were consistent (Supplementary Table 2), implying that the robustness and LNM discrimination of the RFC model were satisfactory.

Gray-level co-occurrence matrix (GLCM) based feature extraction could serve as a robust and promising tool to improve the predictive efficiency for lymph node metastasis (LNM) of individual undifferentiated early gastric cancer (UEGC) patients. Additionally, machine learning (ML) adopts more optimized algorithms and more clear feature extraction. Models built using random forest classifier (RFC) have the highest predictive accuracy in Entropy, Haralick full angle (Haralick_all), Haralick 30° (Haralick_30), Inverse gap full angle (IG_all), Inverse gap 45° (IG_45), Inverse gap 0° (IG_0), and Inertia value 45° (IV_45). Further research is needed to develop these models for clinical practice.

The evaluation results indicate that the method of selecting radiological and textural features becomes more effective in the discrimination of LNM from UEGC patients. In addition, an ML-based prediction model developed using RFC can be used to derive treatment options and identify LNM that can improve clinical outcomes.

GLCM based feature extraction significantly correlated with LNM. The top 7 GLCM based factors included Inertia value 0°, IV_45, IG_0, IG_45, IG_all, Haralick_30, Haralick_all, and Entropy. The areas under the receiver operating characteristic (ROC) curve (AUCs) of the RFC model, support vector machine (SVM), eXtreme gradient boosting (XGBoost), artificial neural network (ANN), and decision tree (DT) ranged from 0.805 [95% confidence interval (CI): 0.258-1.352] to 0.925 (95%CI: 0.378-1.472) in the training set and from 0.794 (95%CI: 0.237-1.351) to 0.912 (95%CI: 0.355-1.469) in the testing set, respectively. The RFC (training set: AUC: 0.925, 95%CI: 0.378-1.472; testing set: AUC: 0.912, 95%CI: 0.355-1.469) model incorporating Entropy, Haralick_all, Haralick_30, IG_all, IG_45, IG_0, and IV_45 had the highest predictive accuracy.

We retrospectively selected 526 cases of UEGC confirmed by pathological examination after radical gastrectomy without endoscopic treatment in four tertiary hospitals between January 2015 to December 2021. GLCM-based features were extracted from grayscale images and ML was applied to the classification of candidate predictive variables. In order to evaluate robustness and clinical utility of each model, the following were made: ROC, decision curve analysis, and clinical impact curve.

Identifying a potential biomarker that predicts LNM is proven to be very useful in determining treatment.

To develop a ML-based integral procedure to construct the LNM gray level co-occurrence matrix (GLCM) prediction model.

The risk of LNM is the most important consideration in determining treatment strategies for UEGC. Therefore, identifying a potential biomarker that predicts LNM is proven to be very useful in determining treatment.