A machine learning-based model for clinical prediction of distal metastasis in chondrosarcoma: a multicenter, retrospective study

Introduction

Chondrosarcoma (CS) is a rare malignant tumor that originates in cartilage tissue. Treatment usually includes surgery and radiation therapy (Li et al., 2022). Symptoms may include pain, swelling, and movement disorders. CS has an annual incidence of five per million, accounting for 30% of all malignant bone tumors, second only to osteosarcoma (Biermann et al., 2017; Giuffrida et al., 2009; Song et al., 2018). Studies have reported that the incidence of CS has been steadily increasing over the past half century, placing a tremendous burden on patients and their families (Anfinsen et al., 2011; Whelan et al., 2012). In the face of such an important malignant disease, the emergence of distant metastases (DM) has overshadowed the tragedy of CS. Identifying the risk of distal metastasis in patients with chondrosarcoma is very important as it affects the patient’s treatment plan and prognosis (Song et al., 2018; Thorkildsen et al., 2019; Thorkildsen et al., 2020; Tsuda et al., 2019). Previous studies have concluded that the combination of surgery and chemotherapy is a reliable treatment option to improve patient prognosis (Chen et al., 2017; Duffaud et al., 2019; Laitinen et al., 2019), and precision medicine is an approach to formulate treatments in an individualized manner, which is based on the patient’s characteristics, for example, a treatment plan that is quantitatively determined based on the risk of metastasis in chondrosarcoma. This can help improve treatment outcomes and reduce treatment side effects as well as personalization of cases. It has been shown that for chondrosarcoma, stratified identification of chondrosarcoma metastatic risk can lead to better treatment planning and improved prognosis. Therefore, precision medicine is very important for the treatment of chondrosarcoma patients and poses a great challenge to clinicians.

In previous studies, machine learning has pioneered novel and effective analytical methods for the medical field, especially those related to oncology (Ngiam & Khor, 2019; Yamazawa et al., 2022). Nevertheless, there are no reports about the application of machine learning in the evaluation of the metastases of CS. However, the current applications of machine learning in the field of chondrosarcoma are still relatively few and have yet to realize its full potential. Although studies have confirmed the feasibility and potential benefits of machine learning in diagnosis and treatment, large-scale experiments and validation for clinical applications are still lacking, and data acquisition and data quality issues also need to be addressed for better application and dissemination in clinical practice (Yue et al., 2022). There are no reports on the application of machine learning to CS metastasis assessment, and only the studies by Bongers et al. (2020), Bongers et al. (2019) and Thio et al. (2018) mainly focused on the analysis of the overall survival (OS) of CS using machine learning.

Given the low incidence of CS, we obtained sufficient cases from the Surveillance, Epidemiology, and End Results (SEER), and performed external validation based on data from multiple academic institutions.

Materials and Methods

Results

Univariate and multivariable logistic regression

The ROC curves assessed the predictive ability of the six ML algorithm models in the validation group. The training group was validated using 10-fold cross-validation, and the results showed that the XGB model had an average AUC of 0.985, which significantly outperformed the other MLs (Fig. 1). The XGB model also occupied the largest area in the ROC of the external validation group, indicating that it was still the most efficient equation (Fig. 2), and thus the XGB model was ultimately selected as the predictive model.

The results of logistic regression analysis were shown in Table 3. Univariate logistic analysis was carried out for age (p = 0.001), gender (p = 0.003), grade (p < 0.005), Laterality (p < 0.005), T (p < 0.001), N (p < 0.001), surgery (p < 0.001) and chemotherapy drugs (p < 0.001) on DM. The odds ratio (OR) revealed the factors associated with DM. In multivariate logistic analysis, we found that patients with pathological grade, T-stage, N-stage and non-left primary lesions, as well as those who did not receive surgery and chemotherapy, had a significantly higher risk of DM at 0.05 significance level (Table 2).

Table 3:

Univariate and multifactorial logistic regression analysis of risk factors for metastases in patients with chondrosarcoma.

Variables	Univariate			Multivariate
	OR	95% CI	p value	OR	95% CI	p value
Age (years)	1.023	1.009–1.036	0.001	1.005	0.983–1.027	0.097
Race
White	Ref		Ref	Ref		Ref
Black	1.106	0.466–2.625	0.820	/		/
Other	1.391	0.762–2.538	0.283	/		/
Sex
Male	Ref		Ref	Ref		Ref
Female	0.474	0.290–0.776	0.003	0.589	0.322–1.076	0.085
Primary site
Limb bones	Ref		Ref	Ref		Ref
Axis of a bone	1.501	0.927–2.428	0.098	/		/
other	1.725	0.688–4.321	0.245	/		/
Grade
I	Ref		Ref	Ref		Ref
II	3.701	1.606–8.528	0.002	2.112	0.810–5.510	0.126
III	8.171	3.260–20.479	0.000	3.224	1.075–9.667	0.037*
IV	14.212	4.705–42.932	0.000	1.987	0.502–7.869	0.328
V	11.020	4.673–25.985	0.000	3.037	1.111–8.307	0.030*
Laterality
Left	Ref		Ref	Ref		Ref
Right	2.396	1.338–4.291	0.003	2.281	1.103–4.715	0.026*
Other	2.831	1.518–5.283	0.001	2.373	1.057–5.328	0.036*
T
T1	Ref		Ref	Ref		Ref
T2	6.485	3.445–12.207	0.000	3.445	1.680–7.067	0.001**
T3	22.985	6.374–82.885	0.000	18.774	4.130–85.339	0.000***
TX	8.059	3.999–16.241	0.000	2.578	1.079–6.160	0.033*
N
N0	Ref		Ref	Ref		Ref
N1	11.263	4.291–29.560	0.000	4.289	1.164–15.805	0.029*
NX	5.068	2.948–12.490	0.000	3.348	1.271–8.820	0.014*
Surgery
No	Ref		Ref	Ref		Ref
Yes	0.087	0.054–0.139	0.000	0.208	0.111–0.389	0.000***
Lymph node dissection
No	Ref		Ref	Ref		Ref
Yes	0.966	0.381–2.453	0.942
Radiation
No	Ref		Ref	Ref		Ref
Yes	0.777	0.351–1.719	0.533	/		/
Chemotherapy
No	Ref		Ref	Ref		Ref
Yes	16.934	9.694–29.581	0.000	11.623	5.571–24.252	0.000***

DOI: 10.7717/peerj.16485/table-3

Notes:

Grade I, Well-differentiated; Grade II, Moderately differentiated; Grade III, Poorly differentiated; Grade IV, Undifferentiated anaplastic; Grade V, Unknown.

*At 0.05 significance level.

**At 0.01 significance level.

***At 0.001 significance level.

Clinical predictive model development

The relative importance of the identified risk factors was shown in Fig. 3. A general trend was seen in the evidence: although the importance of each variable varied slightly, surgery was still ranked first in three algorithms (LR, RF and GBM), chemotherapy ranked first in two algorithms (NBC and DT), and age ranked first in one algorithm (XGB). In contrast, laterality, T-stage, N-stage, and gender had little effect across multiple prediction models. The risk factors of the XGB model were listed as follows by correlation: age, surgery, chemotherapy, grade, laterality, T, gender, and N (Fig. 3). The association between risk factors was also shown in the correlation heat map, and the closer the yellows is, the greater the intensity is (Fig. 4). No significant association has been proven among these independent risk factors in this study.

Web calculator design

We provided users with a web-based calculator, a digital tool based on the best predictive performance modle (XGB model) (https://share.streamlit.io/liuwencai2/chs_m/main/chs_m.py) (Fig. 5).

Discussion

Accurate diagnosis of metastatic status is a critical component of oncology management, and a large amount of scholars are dedicated to promoting this field (Chan et al., 2015; Sheen et al., 2019; Song et al., 2018; Zhang et al., 2021). CS poses a serious threat to patients with OS, especially for patients with DM, where treatment is severely limited and prognosis deteriorates dramatically (Thorkildsen et al., 2019; Thorkildsen et al., 2020; Tsuda et al., 2019). In this regard, prominent contributions have been made by Song et al. (2018) to facilitate the risk stratification of patients with DM, pointing out that this challenge must be overcome on the way to precision medicine. In fact, machine learning algorithms have been widely accepted as a disease investigation tool from which patients can benefit, and these novel mathematical tools could demonstrate superior performance to traditional statistical methods by inputting sufficient amounts of clinical data (Gao et al., 2023; Li et al., 2021; Pereira et al., 2021). However, the extremely low incidence of CS has prevented most investigators from exploring the clinical characteristics of patients thoroughly. The SEER database provides a promising solution to this dilemma. Furthermore, compared with previous reports, the multicenter validation datasets within China in this study is more convincing.

Our study found that high histologic grades was a key factor in DM and revealed statistical differences between the two populations. Pathological grade was a recognized prognostic factor for CS. Grade I sarcomas were well differentiated, while grade II and III sarcoma were malignant (Thorkildsen et al., 2019; Thorkildsen et al., 2020). Grade II tumors were distinguished by nuclear enlargement, hyperchromatism, and changes in size and shape (Weinschenk, Wang & Lewis, 2021). Grade III chondrosarcomas typically had more prominent osteolytic changes and tend to have cortical breakthroughs in soft tissue extension. These malignant cells showed marked nuclear heterogeneity, pleomorphism, and mitosis. The local recurrence rate of CS with poor histologic grade was increased, with metastases occurring in more than 50%, and the most common site of involvement was in the lung (Weinschenk, Wang & Lewis, 2021). It was noteworthy that even in patients underwent surgery, 11% of tumors were of higher grade than originally diagnosed once recurrence or metastasis occurred (Pritchard et al., 1980).

The American Joint Committee on Cancer (AJCC) Staging System Version 8 recommends the use of 8 cm as the cut-off value for risk stratification (Amin et al., 2017). Although the association between tumor size and patient prognosis has not been well established, Roos et al. (2016) and Wang et al. (2019) found that patients with smaller tumor tended to have longer survival. Nevertheless, Kamal et al. (2015) and Bindiganavile et al. (2015) rejected the predictive role of tumor size in CS. We concluded that high-grade T-stage suggested a high risk of DM. This may be linked to the tendency of larger tumors to break through the borders and thus invade the peripheral vessels. In addition, N-stage, a classifier indicating lymphatic status of the tumor, was identified as an independent risk factor for DM. Although lymphatic metastasis is considered to be a threat to patient survival, we believe that inclusion of the independent validation group in the study design ensured convincing conclusions, given the large sample size reported from the SEER database (Basile et al., 2020; Giuffrida et al., 2009; Wan et al., 2019).

Negative margin surgery resulted in fewer DM and improved survival rates (Weinschenk, Wang & Lewis, 2021). However, currently unclear definitions have prevented studies from agreeing on the evaluation of this risk factor. In the reports of Streitbuerger et al. (2012) and Laitinen et al. (2019) patients did not receive the same incidence of margin-negative surgery (76% vs 49%), but achieved the same survival. In the view of the above, we recommend Enneking’s classification system, which requires an acceptable negative edge of the normal tissue cuff when removing the involved portion of the bone. Ensuring consistent variable measurement criteria is the first step in achieving predictive validity. Meanwhile, preoperative neoadjuvant therapy is expected to effectively improve the microenvironment of the sarcoma, which is considered to be the intention of radical surgery (Casali et al., 2018; Radaelli et al., 2014). Frustaci et al. (2001) and Gronchi et al. (2012) found that the chemotherapy regimen of epirubicin, ifosfamide and G-CSF can help control patients’ DM. Interestingly, our findings are consistent with published studies. Preoperative radiotherapy without neoadjuvant therapy does not seem to help control the local recurrence and metastasis of CS (Radaelli et al., 2014; Strander, Turesson & Cavallin-Ståhl, 2003).

The XGB model (best predictive performance) suggested that age played an extraordinary role in DM of CS. Compared with data from SEER database, the average age of patients in this study was younger, which may be due to lack of rational management, resulting in higher tumor-specific death and shorter tumor-free survival (Wang et al., 2020). In national Norwegian studies, older patients were found to have larger tumors and a higher risk of DM (Thorkildsen et al., 2019; Thorkildsen et al., 2020). This may be because longer exposure to tumors leads to an increased likelihood that malignant cells will spread hematologically. Additionally, advanced age consistently predicted shorter OS in survival analyses after local recurrence or DM (Fromm et al., 2018; Laitinen et al., 2019; Song et al., 2018).

Traditional protocols for tumor surveillance and personalized treatment decisions include clinical presentation and symptoms, molecular markers, periodic follow-up, and an experienced physician team. These methods can be used as adjuncts, but may lack precision and reproducibility compared to accurate stratification, and are costly. Machine learning algorithms can be used as an effective tool to predict the risk of chondrosarcoma metastasis. With machine learning algorithms, we can better understand the risk factors associated with distal metastasis in order to predict and treat patients as early as possible, thereby improving the effectiveness of treatment and patient survival. Second, the online risk calculator in this study provides an easy-to-use tool to help clinicians quickly measure a patient’s risk of metastasis and better plan diagnosis and treatment. In addition, this study highlights the importance of chemotherapy and surgery, which is informative for clinicians to choose the best treatment options.

In summary, the use of machine learning algorithms helped determine a formula that accurately classified CS patients into high-risk and low-risk groups according to the probability of DM. However, the study has some limitations. First, it is a retrospective study, which needs to be further verified by prospective research. Second, despite we attempted to define treatment options, the SEER database does not provide details of surgery and chemotherapy, suggesting that this risk factor requires further subgroup analysis to determine its role in DM. In addition, the predictions of the algorithm may not be accurate enough due to flaws in the algorithm or the quality of the sample data, while the external validation group only contains cohort data from China, so further validation is still needed for use in other regions. Machine-learning algorithms cannot completely replace the practical experience and expertise of clinicians, and algorithms need to be more interpretable so that physicians better understand the factors that identify and predict distal metastasis in chondrosarcoma. Finally, the included clinical data may lack characterization, and the unique biomarkers and radiological findings of CS may be potentially associated with DM (Jeong & Kim, 2018; Miwa et al., 2021; Nazarizadeh et al., 2021). To improve the accuracy and generalization of this model by integrating multi-modal and multi-dimensional data is highly recommended in future studies.

Supplemental Information