Retrospective Cohort Study Open Access
Copyright ©The Author(s) 2022. Published by Baishideng Publishing Group Inc. All rights reserved.
World J Clin Cases. Nov 16, 2022; 10(32): 11726-11742
Published online Nov 16, 2022. doi: 10.12998/wjcc.v10.i32.11726
Development and validation of novel nomograms to predict survival of patients with tongue squamous cell carcinoma
Xia-Yan Luo, Ya-Min Zhang, Run-Qiu Zhu, Hui-Yong Zhu, Department of Oral and Maxillofacial Surgery, The First Affiliated Hospital of Zhejiang University School of Medicine, Hangzhou 310003, Zhejiang Province, China
Shan-Shan Yang, Department of Stomatology, Sanmen People’s Hospital, Taizhou 317100, Zhejiang Province, China
Lu-Fang Zhou, Department of Stomatology, Jiangshan People's Hospital, Quzhou 324199, Zhejiang Province, China
ORCID number: Xia-Yan Luo (0000-0002-2968-6996); Ya-Min Zhang (0000-0001-5743-0624); Run-Qiu Zhu (0000-0001-9618-0103); Shan-Shan Yang (0000-0003-2952-306X); Lu-Fang Zhou (0000-0002-5770-7176); Hui-Yong Zhu (0000-0003-0883-5355).
Author contributions: Luo XY and Zhang YM contributed equally to this work and share the first authorship; Luo XY, Zhang YM, and Zhu HY designed the research study; Zhu RQ, Yang SS, and Zhou LF collected data; Luo XY and Zhang YM analyzed the data and wrote the manuscript; All authors have read and approve the final manuscript.
Institutional review board statement: The study was reviewed and approved by the Clinical Research Ethics Committee of the First Affiliated Hospital, Zhejiang University School of Medicine (Approval No. IIT20210346A).
Informed consent statement: The informed consent was exempted.
Conflict-of-interest statement: All the authors declare that they have no conflicts of interest.
Data sharing statement: The datasets of this study are available on request to the corresponding author.
STROBE statement: All the authors have read the STROBE Statement-checklist of items, and the manuscript was prepared and revised according to the STROBE Statement-checklist of items.
Open-Access: This article is an open-access article that was selected by an in-house editor and fully peer-reviewed by external reviewers. It is distributed in accordance with the Creative Commons Attribution NonCommercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: https://creativecommons.org/Licenses/by-nc/4.0/
Corresponding author: Hui-Yong Zhu, MD, PhD, Professor, Department of Oral and Maxillofacial Surgery, The First Affiliated Hospital of Zhejiang University School of Medicine, No. 79 Qingchun Road, Shangcheng District, Hangzhou 310003, Zhejiang Province, China. zhuhuiyong@zju.edu.cn
Received: August 8, 2022
Peer-review started: August 8, 2022
First decision: September 25, 2022
Revised: October 2, 2022
Accepted: October 17, 2022
Article in press: October 17, 2022
Published online: November 16, 2022

Abstract
BACKGROUND

There is no unified standard to predict postoperative survival in patients with tongue squamous cell carcinoma (TSCC), hence the urgency to develop a model to accurately predict the prognosis of these patients.

AIM

To develop and validate nomograms for predicting overall survival (OS) and cancer-specific survival (CSS) of patients with TSCC.

METHODS

A cohort of 3454 patients with TSCC from the Surveillance, Epidemiology, and End Results (SEER) database was used to develop nomograms; another independent cohort of 203 patients with TSCC from the Department of Oral and Maxillofacial Surgery, First Affiliated Hospital of Zhejiang University School of Medicine, was used for external validation. Univariate and multivariate analyses were performed to identify useful variables for the development of nomograms. The calibration curve, area under the receiver operating characteristic curve (AUC) analysis, concordance index (C-index), net reclassification index (NRI), and decision curve analysis (DCA) were used to assess the calibration, discrimination ability, and clinical utility of the nomograms.

RESULTS

Eight variables were selected and used to develop nomograms for patients with TSCC. The C-index (0.741 and 0.757 for OS and CSS in the training cohort and 0.800 and 0.830 in the validation cohort, respectively) and AUC indicated that the discrimination abilities of these nomograms were acceptable. The calibration curves of OS and CSS indicated that the predicted and actual values were consistent in both the training and validation cohorts. The NRI values (training cohort: 0.493 and 0.482 for 3- and 5-year OS and 0.424 and 0.402 for 3- and 5-year CSS; validation cohort: 0.635 and 0.750 for 3- and 5-year OS and 0.354 and 0.608 for 3- and 5-year CSS, respectively) and DCA results indicated that the nomograms were significantly better than the tumor-node-metastasis staging system in predicting the prognosis of patients with TSCC.

CONCLUSION

Our nomograms can accurately predict patient prognoses and assist clinicians in improving decision-making concerning patients with TSCC in clinical practice.

Key Words: Tongue squamous cell carcinoma, Overall survival, Cancer-specific survival, Nomogram, Prognosis

Core Tip: In order to predict prognosis more accurately and precisely, we used two cohorts to develop nomograms in predicting overall survival and cancer-specific survival of patients with tongue squamous cell carcinoma. We adhered to the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis statement-not only evaluated these nomograms in discrimination, calibration, but also their clinical utility. Additionally, the net reclassification index was also used to assess the accuracy of them. These nomograms provide patients and clinicians with an accurate prognosis, so as to facilitate patient-clinician communications and assist clinicians in improving decision-making.

INTRODUCTION

Tongue squamous cell carcinoma (TSCC) is the most common malignancy of the oral cavity and pharynx and has a high risk of local invasion and lymph node metastasis[1-3]. Surgical resection is the first-line treatment, followed by adjuvant radiotherapy, chemotherapy, or chemoradiation therapy. Despite substantial improvements in diagnostic techniques and multimodal treatment in recent years, the survival rate of TSCC remains low[4,5].

Treatment strategies for TSCC and its prognosis are based principally on the tumor-node-metastasis (TNM) cancer staging system established by the American Joint Committee on Cancer (AJCC)[6]. However, the prognoses can vary among patients with the same TNM stage who are receiving similar treatments[7-9]. Such variation suggests that the TNM staging system does not adequately predict prognosis because it does not consider patient characteristics (e.g., age and marital status) or treatment (e.g., type of surgery)[10,11]. Therefore, a new model that incorporates these variables is required to supplement the TNM staging system and accurately predict patient prognoses.

A nomogram is a graphical model that estimates the probability of a clinical event for an individual patient based on specific biological and clinical factors[12]. Nomograms are more accurate than the TNM staging system in predicting prognoses; they have been widely used to evaluate gastric[13-15], hepatocellular[16-19], and head and neck[20-23] carcinomas. However, there are few studies regarding the prediction of the prognosis of TSCC. Although Mair et al[24] predicted the prognosis of TSCC, the clinical utility of the prediction model (i.e., whether they facilitate decision-making and thus improve patient outcomes[12]) was not evaluated; thus, the model would be difficult to apply in clinical practice. Currently, individually predicting the prognosis of patients with TSCC remains insufficient.

Therefore, this study aimed to develop nomograms for predicting overall survival (OS) and cancer-specific survival (CSS) in patients with TSCC to externally validate the established nomograms (discrimination, calibration, and clinical utility) and to assist clinicians in improving therapeutic decision-making.

MATERIALS AND METHODS
Patient selection

Patients diagnosed with TSCC between 2010 and 2015 were selected from the Surveillance, Epidemiology, and End Results (SEER) database using SEERStat 8.3.9.2. The inclusion and exclusion criteria are shown in Figure 1. Overall, 3454 cases were selected as the training cohort for the development of new nomograms. When performing the internal validation, it was assigned by the bootstrapping method. Another independent cohort that was diagnosed between January 2010 and December 2020 was obtained from the Department of Oral and Maxillofacial Surgery, First Affiliated Hospital of Zhejiang University School of Medicine. The National Comprehensive Cancer Network diagnosis and treatment guidelines for TSCC were followed. Using the same inclusion and exclusion criteria, 203 cases were selected as a the validation cohort to externally validate the established nomograms (Figure 1).

Figure 1
Open in New Tab   Full Size Figure   Download Figure
Figure 1 Flowchart of data selection. SEER: Surveillance, Epidemiology, and End Results; TSCC: Tongue squamous cell carcinoma.

We retrospectively retrieved data regarding age, sex, marital status, ethnicity, tumor site, T stage, N stage, TNM stage, pathology grade, neck dissection status, and radiation treatment status. The tumor grading system of the 7th edition of the AJCC Cancer Staging Manual was used. The subclassifications of each variable are shown in Table 1. The study was approved by the Clinical Research Ethics Committee of the First Affiliated Hospital of Zhejiang University School of Medicine.

Table 1 Clinicopathological characteristics of patients with tongue squamous cell carcinoma.
VariablesTraining cohort (n = 3454)
Validation cohort (n = 203)
Cases (%)
Cases (%)
Age1(yr)
< 50685 (19.8)42 (20.7)
≥ 50, < 601049 (30.4)65 (32.0)
≥ 60, < 70976 (28.3)54 (26.6)
≥ 70744 (21.5)42 (20.7)
Sex
Male2218 (64.2)121(59.6)
Female1236 (35.8)82 (40.4)
Marital status
Married2098 (60.7)179 (88.2)
Unmarried21356 (39.3)24 (11.8)
Race
White2978 (86.2)0
Black175 (5.1)0
Other3301 (8.7)203 (100)
Site
Anterior 2/3 of tongue42577 (74.6)167 (82.3)
Base of tongue877 (25.4)36 (17.7)
T stage
T11720 (49.8)88 (43.3)
T21095 (31.7)98 (48.3)
T3348 (10.1)12 (5.9)
T4291 (8.4)5 (2.5)
N stage
N01920 (55.6)133 (65.5)
N1513 (14.9)24 (11.8)
N2973 (28.2)45 (22.2)
N348 (1.4)1 (0.5)
TNM stage
I1237 (35.8)65 (32.0)
II516 (14.9)59 (29.1)
III565 (16.4)31 (15.3)
IV1136 (32.9)48 (23.6)
Pathology grade
Well differentiated722 (20.9)141 (69.5)
Moderately differentiated1787 (51.7)56 (27.6)
Poorly differentiated945 (27.4)6 (3.0)
Neck dissection
No963 (27.9)9 (4.4)
Yes2491 (72.1)194 (95.6)
Radiation
No radiation1701 (49.2)107 (52.7)
Radiation prior to surgery43 (1.2)4 (2.0)
Radiation after surgery1698 (49.2)73 (36.0)
Radiation before and after surgery12 (0.3)19 (9.4)
1Age at diagnosis.
2Unmarried, including divorced, separated, sigle, and widowed.
3Other, including American Indian/AK Native and Asian/Pacific Islander.
4Anterior 2/3 of the tongue refers to all parts of the tongue except for the base of the tongue and includes the border, ventral surface, dorsal surface, and overlapping lesions of the tongue.
HR: Hazard ratio; CI: Confidence interval; OS: Overall survival; TNM: Tumor-node-metastasis.
Statistical analysis and nomogram development

First, descriptive statistics were generated for the demographic and tumor clinicopathological characteristics. Then, univariate and multivariate Cox proportional hazards models were constructed. Coefficients, hazard ratios, and 95% confidence intervals (CIs) were obtained for prognostic factors in the training cohort. Finally, nomograms that integrated significant independent risk factors were constructed based on the predicted 3- and 5-year OS and CSS in the training cohort. OS was defined as the time from surgery until death from any cause or the last follow-up. CSS was defined as the time from surgery until death from TSCC or the last follow-up.

Validation and evaluation of nomograms

Internal and external validation analyses were performed to assess the predictive accuracies of the nomograms for the training and validation cohorts. Discriminative ability was evaluated based on the concordance index (C-index) and area under the receiver operating characteristic curve (AUC). The C-index and AUC values are often used interchangeably and range from 0.5 to 1 (no discrimination ability and perfect discrimination, respectively)[12]. Meanwhile, a C-index or AUC value of > 0.7 indicates satisfactory discrimination. The concordance between predicted and actual survival was assessed using calibration curves. The reference line is a 45° diagonal line that ideally includes both predicted and actual survival rates.

The clinical benefits and utility of the nomograms were compared with those of the TNM staging system using the net reclassification index (NRI) and decision curve analysis (DCA). The NRI is used to assess the predictive accuracies and utility of nomograms[25,26]. The DCA is used to estimate the clinical and net benefits of nomograms based on threshold probabilities[27,28]. A horizontal reference line indicates that no intervention was performed (i.e., there was no clinical benefit), while an oblique line indicates that all patients underwent the intervention (i.e., the clinical benefit was maximized).

R statistical software (ver. 4.0.5; R Development Core Team, Vienna, Austria) was used to perform all analyses. P values < 0.05 were considered statistically significant.

RESULTS
Clinicopathological characteristics

The clinicopathological characteristics of the SEER cohort and our cohort are described in Table 1. Most of the patients [training cohort, n = 1049 (30.4%); validation cohort, n = 65 (32.0%)] were aged 50-59 years, and approximately 60% patients were men. Overall, the proportion of married patients was significantly greater than that of unmarried patients; the proportion of married patients was greater in the validation cohort [n = 179 (88.2%)] than in the training cohort [n = 2098 (60.7%)]. Approximately 90% of patients in the training cohort were White, whereas all patients in the validation cohort were Asian. In both cohorts, the proportion of TSCCs located on the anterior 2/3 of the tongue was greater than that located on the base of the tongue (training cohort, 74.6% vs 25.4%; validation cohort, 82.3% vs 17.7%, respectively). In both cohorts, most TSCCs were stage T1 and T2 [training cohort, n = 2815 (81.5%); validation cohort, n = 186 (91.6%)]. Meanwhile, more than half of all TSCCs were stage N0 [training cohort, n = 1920 (55.6%); validation cohort, n = 133 (65.5%)], while a few TSCCs were stage N3 [training cohort, n = 48 (1.4%); validation cohort, n = 1 (0.5%)]. The proportion of TSCCs was evenly distributed across subclassifications of TNM stages. Approximately half of the TSCCs in the training cohort was moderately differentiated, whereas 69.5% of TSCCs in the validation cohort was well-differentiated. Most of the patients in both cohorts underwent neck dissection [training cohort, n = 2491 (72.1%); validation cohort, n = 194 (95.6%)]. The proportion of patients who did and did not undergo radiation after surgery was 49.2% and 49.2% in the training cohort, and 52.7% and 36.0% in the validation cohort, respectively.

Nomogram development

Eleven candidate variables associated with OS and CSS were evaluated by univariate and multivariate Cox analyses of the SEER cohort. Univariate analysis showed that age, marital status, ethnicity, tumor site, T stage, N stage, TNM stage, pathology grade, neck dissection status, and radiation treatment status were significantly associated with OS and CSS in (P < 0.05 for all; Tables 2 and 3). Multivariate analysis showed that age, marital status, tumor site, T stage, N stage, pathology grade, neck dissection status, and radiation treatment status were independently associated with OS and CSS (P < 0.05 for all; Tables 2 and 3).

Table 2 Univariate and multivariate analyses of variables associated with overall survival.
VariablesOS
Univariate analysis
Multivariate analysis
HR (95%CI)
P value
HR (95%CI)
P value
Age1 (yr)
< 501.01.0
≥ 50, < 601.002 (0.821, 1.223)0.9840.982 (0.803, 1.201)0.858
≥ 60, < 701.181 (0.971, 1.438)0.0971.276 (1.045, 1.559)0.017
≥ 701.800 (1.482, 2.187)< 0.0012.217 (1.815, 2.710)< 0.001
Sex
Male1.01.0
Female0.983 (0.859, 1.124)0.8010.901 (0.783, 1.037)0.147
Marital status
Married1.01.0
Unmarried21.606 (1.413, 1.827)< 0.0011.388 (1.216, 1.585)< 0.001
Race
White1.01.0
Black1.649 (1.296, 2.100)< 0.0011.199 (0.935, 1.538)0.153
Other31.077 (0.857, 1.354)0.5261.102 (0.874, 1.390)0.411
Site
Anterior 2/3 of tongue41.01.0
Base of tongue0.757 (0.647, 0.886)< 0.0010.413 (0.342, 0.497)< 0.001
T stage
T11.01.0
T22.168 (1.847, 2.544)< 0.0011.969 (1.540, 2.518)< 0.001
T33.997 (3.293, 4.852)< 0.0013.142 (2.411, 4.095)< 0.001
T45.070 (4.171, 6.163)< 0.0014.682 (3.498, 6.268)< 0.001
N stage
N01.01.0
N12.066 (1.725, 2.475)< 0.0012.080 (1.497, 2.889)< 0.001
N22.489 (2.154, 2.878)< 0.0013.749 (2.554, 5.503)< 0.001
N33.040 (1.995, 4.634)< 0.0015.641 (3.223, 9.873)< 0.001
TNM stage
I1.01.0
II1.831 (1.459, 2.299)< 0.0011.011 (0.719, 1.420)0.952
III2.617 (2.129, 3.216)< 0.0011.071 (0.709, 1.619)0.745
IV3.439 (2.890, 4.092)< 0.0010.774 (0.475, 1.262)0.305
Pathology grade
Well differentiated1.01.0
Moderately differentiated2.141 (1.748, 2.622)< 0.0011.781 (1.442, 2.200)< 0.001
Poorly differentiated2.045 (1.642, 2.546)< 0.0011.733 (1.360, 2.209)< 0.001
Neck dissection
No1.01.0
Yes1.407 (1.209, 1.638)< 0.0010.766 (0.640, 0.916)0.004
Radiation
No radiation1.01.0
Radiation prior to surgery3.235 (2.156, 4.854)< 0.0011.345 (0.870, 2.080)0.182
Radiation after surgery1.629 (1.425, 1.861)< 0.0010.716 (0.5994, 0.856)< 0.001
Radiation before and after surgery3.111 (1.472, 6.575)0.0031.025 (0.475, 2.212)0.0.949
1Age at diagnosis.
2Unmarried, including divorced, separated, single, and widowed.
3Other, including American Indian/AK Native and Asian/Pacific Islander.
4Anterior 2/3 of the tongue refers to all parts of the tongue except for the base of the tongue and includes the border, ventral surface, dorsal surface, and overlapping lesions of the tongue.
HR: Hazard ratio; CI: Confidence interval; OS: Overall survival; TNM: Tumor-node-metastasis.
Table 3 Univariate and multivariate analyses of variables associated with cancer-specific survival.
VariablesCSS
Univariate analysis
Multivariate analysis
HR (95%CI)
P value
HR (95%CI)
P value
Age1 (yr)
< 501.01.0
≥ 50, < 600.940 (0.760, 1.164)0.5710.910 (0.734, 1.129)0.390
≥ 60, < 701.093 (0.885, 1.350)0.4081.180 (0.952, 1.463)0.131
≥ 701.359 (1.094, 1.689)0.0061.750 (1.399, 2.189)< 0.001
Sex
Male1.01.0
Female1.021 (0.879, 1.185)0.7860.999 (0.854, 1.167)0.986
Marital status
Married1.01.0
Unmarried21.515 (1.312, 1.749)< 0.0011.291 (1.114, 1.497)< 0.001
Race
White1.01.0
Black1.739 (1.337, 2.262)< 0.0011.213 (0.925, 1.590)0.163
Other31.079 (0.836, 1.394)0.5581.113 (0.0.859, 1.442)0.420
Site
Anterior 2/3 of tongue41.01.0
Base of tongue0.795 (0.669, 0.945)0.0090.393 (0.320, 0.482)< 0.001
T stage
T11.01.0
T22.397 (1.994, 2.880)< 0.0011.973 (1.520, 2.561)< 0.001
T34.832 (3.898, 5.991)< 0.0013.220 (2.429, 4.268)< 0.001
T45.933 (4.771, 7.377)< 0.0014.786 (3.519, 6.510)< 0.001
N stage
N01.01.0
N12.756 (2.250, 3.375)< 0.0012.376 (1.635, 3.454)< 0.001
N23.401 (2.880, 4.016)< 0.0015.216 (3.337, 8.154)< 0.001
N34.400 (2.843, 6.810)< 0.0018.289 (4.498, 15.275)< 0.001
TNM stage
I1.01.0
II1.865 (1.405, 2.475)< 0.0011.019 (0.689, 1.507)0.926
III3.597 (2.829, 4.572)< 0.0011.226 (0.766, 1.964)0.396
IV4.720 (3.830, 5.816)< 0.0010.710 (0.404, 1.247)0.233
Pathology grade
Well differentiated1.01.0
Moderately differentiated2.586 (2.024, 3.304)< 0.0012.895 (1.469, 2.444)< 0.001
Poorly differentiated2.632 (2.030, 3.414)< 0.0011.911 (1.438, 2.540)< 0.001
Neck dissection
No1.01.0
Yes1.710 (1.429, 2.047)< 0.0010.775 (0.628, 0.957)0.018
Radiation
No radiation1.01.0
Radiation prior to surgery4.294 (2.800, 6.586)< 0.0011.511 (0.952, 2.399)0.080
Radiation after surgery2.079 (1.782, 2.426)< 0.0010.769 (0.628, 0.943)0.012
Radiation before and after surgery4.478 (2.112, 9.495)< 0.0011.271 (0.585, 2.760)0.545
1Age at diagnosis.
2Unmarried, including divorced, separated, single, and widowed.
3Other, including American Indian/AK Native and Asian/Pacific Islander.
4Anterior 2/3 of the tongue refers to all parts of the tongue except for the base of the tongue and includes the border, ventral surface, dorsal surface, and overlapping lesions of the tongue.
CSS: Cancer-specific survival; CI: Confidence interval; HR: Hazard ratio; TNM: Tumor-node-metastasis.

Based on the results of the multivariate analysis, eight prognostic variables (age, marital status, tumor site, T stage, N stage, pathology grade, neck dissection status, and radiation treatment status) were used to develop the nomograms. Figure 2 shows the OS and CSS predictions from the nomograms. N and T stages had the greatest effects on OS followed by tumor site and age. N stage had the greatest effect on CSS followed by T stage and tumor site. Generally, OS and CSS were better in younger patients with lower T and N stages. The predicted 3- and 5-year OS and CSS for individual patients are shown at the bottom of the nomograms based on the sum of scores across variables.

Figure 2
Open in New Tab   Full Size Figure   Download Figure
Figure 2 Nomograms predicting the 3- and 5-year overall survival and cancer-specific survival in patients with squamous cell carcinoma of the tongue. A: Nomogram predicting the 3- and 5-year overall survival (OS) in patients with squamous cell carcinoma of the tongue (TSCC); B: Nomogram predicting the 3- and 5-year cancer-specific survival (CSS) in patients with TSCC. The points for each variable were summed, and the probabilities of 3- and 5-year OS and CSS were predicted based on the total number of points (shown at the bottom of the nomogram). For example, consider a 60-year-old unmarried patient with moderately differentiated T2 and N1 stage cancer on the anterior 2/3 of tongue who underwent neck dissection and postoperative chemotherapy. Top red lines represent the points for each variable, the sum (209) of these points is the total score, and the bottom red line indicates the probabilities of 3- (64%) and 5-year (55%) overall survival.
Nomogram validation and evaluation

The results of the internal and external validation analyses are shown in Figure 3. In the training cohort, the internal calibration curves indicated excellent consistency between the predicted and actual 3- and 5-year OS and CSS (Figures 3A, B, E, and F), which was also observed in the validation cohort (Figures 3C, D, G, and H). The C-index values were 0.741 (95%CI: 0.725, 0.756) and 0.757 (95%CI: 0.739, 0.775) for OS and CSS in the internal validation analysis; these respective values were 0.800 (95%CI: 0.747, 0.853) and 0.830 (95%CI: 0.779, 0.881) in the external validation analysis, respectively (Table 4). Overall, the nomograms exhibited satisfactory discrimination and calibration.

Figure 3
Open in New Tab   Full Size Figure   Download Figure
Figure 3 Calibration curves for 3- and 5-year overall survival and 3- and 5-year cancer-specific survival in patients with squamous cell carcinoma of the tongue. A: Calibration curves for 3-year overall survival (OS) in the training cohort; B: Calibration curves for 5-year OS in the training cohort; C: Calibration curves for 3-year OS in the validation cohort; D: Calibration curves for 5-year OS in the validation cohort; E: Calibration curves for 3-year cancer-specific survival (CSS) in the training cohort; F: Calibration curves for 5-year CSS in the training cohort; G: Calibration curves for 3-year CSS in the validation cohort; H: Calibration curves for 5-year CSS in the validation cohort. The gray line indicates perfect prediction. Blue lines indicate 95% confidence intervals (CIs). The red line indicates nomogram performance. Red and gray lines close together indicates greater nomogram accuracy.
Table 4 Net reclassification index and concordance index values of the nomograms and tumor-node-metastasis staging system, reflecting the predictive accuracy for overall and cancer-specific survival in tongue squamous cell carcinoma patients.
IndexTraining cohort
Validation cohort
Estimate
95%CI
Estimate
95%CI
NRI (vs TNM stage)
For 3-year OS0.493(0.418, 0.589)0.635(0.228, 1.096)
For 5-year OS0.482(0.413, 0.613)0.750(0.397, 1.240)
For 3-year CSS0.424(0.354, 0.523)0.354(0.145, 1.037)
For 5-year CSS0.402(0.345, 0.536)0.608(0.180, 1.186)
C-index
The nomogram OS0.741(0.725, 0.756)0.800(0.747, 0.853)
The nomogram CSS0.757(0.739, 0.775)0.830(0.779, 0.881)
TNM stage OS0.643(0.636, 0.668)0.695(0.617, 0.750)
TNM stage CSS0.678(0.660, 0.696)0.749(0.673, 0.825)
CI: Confidence interval; NRI: Net reclassification index; C-index: Concordance index; OS: Overall survival; CSS: Cancer-specific survival; TNM: Tumor-node-metastasis.
Comparison of clinical utility between the nomograms and the TNM staging system

The C-index values of the TNM staging system for OS and CSS were also estimated in both the internal and external validation analyses (Table 4). The C-index values of the nomograms were higher than those of the TNM staging system (Table 4). In terms of predictive accuracy, the AUC values for the nomograms were higher than those of the TNM staging system (3-year OS, 74.2 vs 66.0; 5-year OS, 73.9 vs 65.9; 3-year CSS, 75.4 vs 68.3; 5-year CSS, 75.7 vs 69.4) in the training cohort (Figures 4A, B, E, and F) as well as in the validation cohort (3-year OS, 83.3 vs 75.3; 5-year OS, 87.1 vs 71.3; 3-year CSS, 86.4 vs 80.4; 5-year CSS, 87.9 vs 75.0) (Figures 4C, D, G, and H).

Figure 4
Open in New Tab   Full Size Figure   Download Figure
Figure 4 Receiver operating characteristic curves of the nomograms and tumor-node-metastasis staging system for overall survival and cancer-specific survival in patients with squamous cell carcinoma of the tongue. A: Receiver operating characteristic curves (ROC) for 3-year overall survival (OS) in the training cohort; B: ROC for 5-year OS in the training cohort; C: ROC for 3-year OS in the validation cohort; D: ROC for 5-year OS in the validation cohort; E: ROC for 3-year cancer-specific survival (CSS) in the training cohort; F: ROC for 5-year CSS in the training cohort; G: ROC for 3-year CSS in the validation cohort; H: ROC for 5-year CSS in the validation cohort. TNM: Tumor-node-metastasis.

As shown in Table 4, the NRI values for the 3- and 5-year OS and CSS in the training cohort were 0.493 (95%CI: 0.418, 0.589) and 0.482 (95%CI: 0.413, 0.613), and 0.424 (95%CI: 0.354, 0.523) and 0.402 (95%CI: 0.345, 0.536), respectively, which were confirmed in the validation cohort (Table 4). Notably, the nomograms performed significantly better than the TNM staging system in both the training and validation cohorts.

The DCA was used to compare clinical benefits between the nomograms and the TNM staging system. As shown in Figure 5, the nomograms exhibited greater net benefits than the TNM staging system at all threshold probabilities in the training cohort (i.e., they were better able to predict both 3- and 5-year OS and CSS). For the 3-year OS and CSS in the validation cohort, the net benefits of the TNM staging system were generally equivalent to the nomograms, whereas the nomograms showed greater net benefits than the TNM staging system at almost all threshold probabilities for the 5-year OS and CSS.

Figure 5
Open in New Tab   Full Size Figure   Download Figure
Figure 5 Decision curve analysis of the accuracy of the nomograms and tumor-node-metastasis staging system for predicting overall survival and cancer-specific survival in patients with squamous cell carcinoma of the tongue. A: 3-year overall survival (OS) benefits in the training cohort; B: 5-year OS benefits in the training cohort; C: 3-year OS benefits in the validation cohort; D: 5-year OS benefits in the validation cohort; E: 3-year cancer-specific survival (CSS) benefits in the training cohort; F: 5-year CSS benefits in the training cohort; G: 3-year CSS benefits in the validation cohort; H: 5-year CSS benefits in the validation cohort. TNM: Tumor-node-metastasis.
DISCUSSION

We developed new nomograms to predict the 3- and 5-year OS and CSS in patients with TSCC, evaluated their discrimination and calibration abilities, and compared their clinical utilities with those of the TNM staging system. Our results showed that our nomograms accurately predicted both the OS and CSS of patients with TSCC. Additionally, the C-index and AUC values along with the calibration curves showed that the nomograms had satisfactory discrimination and calibration. Moreover, compared with the TNM staging system, the predictive accuracies of OS and CSS were higher for the nomograms, as revealed by the NRI values and DCA curves. Thus, the aforementioned results indicate that our nomograms exhibited satisfactory discrimination, calibration, and clinical utility.

In this study, age, marital status, tumor site, T stage, N stage, pathology grade, neck dissection status, and radiation treatment status were selected to develop nomograms to predict the 3- and 5-year OS and CSS of patients with TSCC. As an example, Figure 2 compares two patients with similar staging results but different treatments. The first patient was 60 years old, married, and with T2 and N1 stage cancer on the anterior 2/3 of the tongue that exhibited moderate differentiation; that patient underwent neck dissection and received postoperative chemotherapy. The second patient was 70 years old, unmarried, and with T2 and N1 stage cancer on the anterior 2/3 of the tongue that exhibited high differentiation; that patient underwent neck dissection but did not receive radiation treatment. According to the conventional TNM staging system, both patients had the same TNM stage and therefore should have similar OS. However, our nomograms predicted that the respective 3- and 5-year OS were 64% and 55% for the first patient, whereas they were 43% and 33% for the second patient. The inclusion of additional information regarding clinicopathological characteristics and demographics provides our nomograms with a more accurate prognosis prediction ability; we expect these nomograms to serve as a powerful supplement to the TNM staging system for predicting prognoses.

The N stage had the greatest prognostic power followed by T stage, tumor site, and age (Figure 2). Advanced T and N stages were associated with poor OS and CSS, consistent with findings in previous studies[4,9]. These results indicate that the prognosis of patients with TSCC is greatly affected by the T and N stages; the more advanced the T and/or N stage, the worse the OS and CSS. Meanwhile, the inclusion of age and radiation treatment status in our nomograms may be considered controversial. Previous studies revealed that age was independently associated with both OS and CSS; younger patients had better survival, whereas older patients had a significantly greater mortality risk[29-31]. Moreover, compared with younger patients, older patients with advanced tumor stages (III, IV) had a nearly two-fold greater mortality risk. Similar to radiation treatment, surgery alone is generally associated with a high risk of relapse, particularly in patients with advanced TSCC; adjuvant therapies are thus necessary[32]. Radiation treatment has been shown to improve locoregional control and survival in patients with TSCC after surgery, particularly in advanced cases[33-36]. Here we found that the ability of radiation treatment status for predicting OS and CSS was not inferior to that of pathology grade (Figure 2). Additionally, as shown in Tables 2 and 3, age and radiation treatment status were independent predictors of OS and CSS in patients with TSCC. Taken together, our results indicate that age and radiation treatment status have prognostic significance. It has been demonstrated that marital status is an independent prognostic factor in patients with TSCC[9]. Married patients had better OS and CSS than unmarried patients[37], which is consistent with our findings in this study. We found the independent and significant role of marital status as a prognostic factor of patients with TSCC. In addition to the above variables, our study identified tumor site, pathology grade, and neck dissection status as independent prognostic factors of patients with TSCC. The OS and CSS of patients with TSCC are affected by these factors, which are shown in Tables 2 and 3, and Figure 2.

Our nomograms accurately and effectively predicted the prognosis of patients with TSCC and exhibited high clinical potential. The satisfactory discrimination and calibration abilities of these nomograms were confirmed by the calibration and receiver operating characteristic curves as well as the C-index and AUC values. The C-index values in external validation were higher than that in the training cohort, which is consistent with that constructed by Lu and Zhang for predicting tongue cancer and low-grade endometrial stromal sarcoma, respectively[7,38]. These results may indicate the extensionality and applicability of the constructed model. Moreover, we also compared the clinical utilities of the established nomograms with that of the TNM staging system, with the NRI values indicating that our nomograms had significantly better predictive accuracy. Similarly, DCA revealed that the nomograms had more clinical benefits and were better able to predict survival compared with the TNM staging system.

To reduce potential bias, we used multi-institution and multi-population data from the SEER database to develop our nomograms and to validate their discrimination and calibration abilities as well as their clinical utilities in both internal and external cohorts. Additionally, we adhered to the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis statement[39]. In summary, our nomograms were used to accurately determine the clinical prognosis of patients with TSCC.

Due to its retrospective nature, this study has some limitations. First, the depth of invasion (DOI) has been recognized as an independent predictor of survival[8,40]. Among the tumor parameters that were significant for prognosis, such as the tumor width, area, volume, and depth, the DOI was considered the most important[41]. Additionally, extranodal extension (ENE) has been widely recognized as a significant poor prognostic factor for patients with HNSCC[42,43]. Hence, the DOI and ENE were incorporated into the T and N classification, respectively, in the AJCC 8th edition of the cancer staging manual[44]. However, they were not available in the SEER database, thus not being included in our constructed model. Further improvements by incorporating these factors into the constructed nomogram should be undertaken in the future. Second, the current model only incorporates clinicopathological parameters to predict patient outcomes, which is nonsufficient for screening patients appropriate for adjuvant therapies, especially preoperative/postoperative adjuvant immunotherapy. More molecular markers should be incorporated into the constructed model to improve its clinical application value, such as PD-1[45-47], CD47[48], CXCL11[49], and CXCR3[50], which have been reported to engage in tumor immunity and included in some efficient predictive models. Third, this retrospective study had an unavoidable risk of selection bias. Thus, prospective validation studies are needed before these nomograms can be used in clinical practice.

CONCLUSION

We used two databases to develop and validate new nomograms for predicting the 3- and 5-year OS and CSS in patients with TSCC. Compared with the TNM staging system, these nomograms exhibit greater accuracy, effectiveness, and clinical utility for predicting the prognosis of patients with TSCC. Thus, they are a strong complement to the TNM staging system in the prediction of patient prognosis.

ARTICLE HIGHLIGHTS
Research background

There is no unified standard to predict postoperative survival in patients with tongue squamous cell carcinoma (TSCC), hence the urgency to develop a model to accurately predict the prognosis of these patients.

Research motivation

Development of new models for predicting survival in patients with TSCC is important for facilitating patient-clinician communications and assisting clinicians in improving decision-making.

Research objectives

This study aimed to develop nomograms for predicting overall survival and cancer-specific survival in patients with TSCC based on demographic and histopathological variables, and to externally validate the established nomograms.

Research methods

Two databases of patients with TSCC were used to develop nomograms and to perform external validation, respectively.

Research results

Eight variables were selected and used to develop nomograms for patients with TSCC. The C-index and area under the curve indicated that the discrimination abilities of these nomograms were acceptable. The calibration curves indicated that predicted and actual values were consistent. The NRI values and decision curve analysis results indicated that the nomograms were significantly better than the TNM staging system in predicting the prognosis of patients with TSCC.

Research conclusions

The nomograms we developed exhibit great accuracy, effectiveness, and clinical utility for predicting the prognosis of patients with TSCC.

Research perspectives

In addition to the demographic and histopathological characteristics, some molecular markers that have an impact on survival, such as PD-1, CD47, CXCL11, may be incorporated to predict the prognosis of patients with TSCC in future.

ACKNOWLEDGEMENTS

We thank Gui-Qi Zhu in the University of Fudan for his guidance in using R statistical software. He has no responsibility for the manuscript content.

Footnotes

Provenance and peer review: Unsolicited article; Externally peer reviewed.

Peer-review model: Single blind

Specialty type: Oncology

Country/Territory of origin: China

Peer-review report’s scientific quality classification

Grade A (Excellent): 0

Grade B (Very good): B

Grade C (Good): C

Grade D (Fair): 0

Grade E (Poor): 0

P-Reviewer: Eccher A, Italy; Tang XB, China S-Editor: Liu JH L-Editor: A P-Editor: Liu JH

References
1.  Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, Bray F. Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA Cancer J Clin. 2021;71:209-249.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 50630]  [Cited by in F6Publishing: 42811]  [Article Influence: 14270.3]  [Reference Citation Analysis (47)]
2.  Siegel RL, Miller KD, Fuchs HE, Jemal A. Cancer statistics, 2022. CA Cancer J Clin. 2022;72:7-33.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 4235]  [Cited by in F6Publishing: 8042]  [Article Influence: 4021.0]  [Reference Citation Analysis (1)]
3.  Miranda-Filho A, Bray F. Global patterns and trends in cancers of the lip, tongue and mouth. Oral Oncol. 2020;102:104551.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 178]  [Cited by in F6Publishing: 146]  [Article Influence: 36.5]  [Reference Citation Analysis (0)]
4.  Li Y, Zhao Z, Liu X, Ju J, Chai J, Ni Q, Ma C, Gao T, Sun M. Nomograms to estimate long-term overall survival and tongue cancer-specific survival of patients with tongue squamous cell carcinoma. Cancer Med. 2017;6:1002-1013.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 26]  [Cited by in F6Publishing: 23]  [Article Influence: 3.3]  [Reference Citation Analysis (0)]
5.  da Silva Souto AC, Vieira Heimlich F, Lima de Oliveira L, Bergmann A, Dias FL, Spíndola Antunes H, de Melo AC, Thuler LCS, Cohen Goldemberg D. Epidemiology of tongue squamous cell carcinoma: A retrospective cohort study. Oral Dis. 2021;.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 3]  [Cited by in F6Publishing: 3]  [Article Influence: 3.0]  [Reference Citation Analysis (0)]
6.  Edge SB, Compton CC. The American Joint Committee on Cancer: the 7th edition of the AJCC cancer staging manual and the future of TNM. Ann Surg Oncol. 2010;17:1471-1474.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 5537]  [Cited by in F6Publishing: 6108]  [Article Influence: 436.3]  [Reference Citation Analysis (0)]
7.  Lu Z, Yan W, Liang J, Yu M, Liu J, Hao J, Wan Q, Luo C, Chen Y. Nomogram Based on Systemic Immune-Inflammation Index to Predict Survival of Tongue Cancer Patients Who Underwent Cervical Dissection. Front Oncol. 2020;10:341.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 18]  [Cited by in F6Publishing: 20]  [Article Influence: 5.0]  [Reference Citation Analysis (0)]
8.  Chang B, He W, Ouyang H, Peng J, Shen L, Wang A, Wu P. A Prognostic Nomogram Incorporating Depth of Tumor Invasion to Predict Long-term Overall Survival for Tongue Squamous Cell Carcinoma With R0 Resection. J Cancer. 2018;9:2107-2115.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 15]  [Cited by in F6Publishing: 19]  [Article Influence: 3.2]  [Reference Citation Analysis (0)]
9.  Sun W, Cheng M, Zhuang S, Chen H, Yang S, Qiu Z. Nomograms to predict survival of stage IV tongue squamous cell carcinoma after surgery. Medicine (Baltimore). 2019;98:e16206.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 8]  [Cited by in F6Publishing: 9]  [Article Influence: 1.8]  [Reference Citation Analysis (0)]
10.  Kantola S, Parikka M, Jokinen K, Hyrynkangs K, Soini Y, Alho OP, Salo T. Prognostic factors in tongue cancer - relative importance of demographic, clinical and histopathological factors. Br J Cancer. 2000;83:614-619.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 104]  [Cited by in F6Publishing: 114]  [Article Influence: 4.8]  [Reference Citation Analysis (0)]
11.  Aksu G, Karadeniz A, Saynak M, Fayda M, Kadehci Z, Kocaelli H. Treatment results and prognostic factors in oral tongue cancer: analysis of 80 patients. Int J Oral Maxillofac Surg. 2006;35:506-513.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 32]  [Cited by in F6Publishing: 33]  [Article Influence: 1.8]  [Reference Citation Analysis (0)]
12.  Balachandran VP, Gonen M, Smith JJ, DeMatteo RP. Nomograms in oncology: more than meets the eye. Lancet Oncol. 2015;16:e173-e180.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1119]  [Cited by in F6Publishing: 1873]  [Article Influence: 208.1]  [Reference Citation Analysis (0)]
13.  Lu J, Xu BB, Zheng CH, Li P, Xie JW, Wang JB, Lin JX, Chen QY, Truty MJ, Huang CM. Development and External Validation of a Nomogram to Predict Recurrence-Free Survival After R0 Resection for Stage II/III Gastric Cancer: An International Multicenter Study. Front Oncol. 2020;10:574611.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 2]  [Cited by in F6Publishing: 5]  [Article Influence: 1.3]  [Reference Citation Analysis (0)]
14.  Gao Z, Ni J, Ding H, Yan C, Ren C, Li G, Pan F, Jin G. A nomogram for prediction of stage III/IV gastric cancer outcome after surgery: A multicenter population-based study. Cancer Med. 2020;9:5490-5499.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 5]  [Cited by in F6Publishing: 8]  [Article Influence: 2.0]  [Reference Citation Analysis (0)]
15.  Chen D, Liu Z, Liu W, Fu M, Jiang W, Xu S, Wang G, Chen F, Lu J, Chen H, Dong X, Li G, Chen G, Zhuo S, Yan J. Predicting postoperative peritoneal metastasis in gastric cancer with serosal invasion using a collagen nomogram. Nat Commun. 2021;12:179.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 35]  [Cited by in F6Publishing: 72]  [Article Influence: 24.0]  [Reference Citation Analysis (0)]
16.  Fang Q, Chen H. Development of a Novel Autophagy-Related Prognostic Signature and Nomogram for Hepatocellular Carcinoma. Front Oncol. 2020;10:591356.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 27]  [Cited by in F6Publishing: 26]  [Article Influence: 6.5]  [Reference Citation Analysis (0)]
17.  Huang WY, Tsai CL, Que JY, Lo CH, Lin YJ, Dai YH, Yang JF, Shen PC, Lee MH, Cheng JC. Development and Validation of a Nomogram for Patients with Nonmetastatic BCLC Stage C Hepatocellular Carcinoma after Stereotactic Body Radiotherapy. Liver Cancer. 2020;9:326-337.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 7]  [Cited by in F6Publishing: 7]  [Article Influence: 1.8]  [Reference Citation Analysis (0)]
18.  Wang YY, Xiang BD, Ma L, Zhong JH, Ye JZ, Wang K, Xing BC, Li LQ. Development and Validation of a Nomogram to Preoperatively Estimate Post-hepatectomy Liver Dysfunction Risk and Long-term Survival in Patients With Hepatocellular Carcinoma. Ann Surg. 2021;274:e1209-e1217.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 23]  [Cited by in F6Publishing: 35]  [Article Influence: 8.8]  [Reference Citation Analysis (0)]
19.  Serenari M, Han KH, Ravaioli F, Kim SU, Cucchetti A, Han DH, Odaldi F, Ravaioli M, Festi D, Pinna AD, Cescon M. A nomogram based on liver stiffness predicts postoperative complications in patients with hepatocellular carcinoma. J Hepatol. 2020;73:855-862.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 42]  [Cited by in F6Publishing: 61]  [Article Influence: 15.3]  [Reference Citation Analysis (0)]
20.  Mell LK, Shen H, Nguyen-Tân PF, Rosenthal DI, Zakeri K, Vitzthum LK, Frank SJ, Schiff PB, Trotti AM 3rd, Bonner JA, Jones CU, Yom SS, Thorstad WL, Wong SJ, Shenouda G, Ridge JA, Zhang QE, Le QT. Nomogram to Predict the Benefit of Intensive Treatment for Locoregionally Advanced Head and Neck Cancer. Clin Cancer Res. 2019;25:7078-7088.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 13]  [Cited by in F6Publishing: 14]  [Article Influence: 2.8]  [Reference Citation Analysis (0)]
21.  Li X, Guo K, Feng Y, Guo Y. Analysis of chemotherapy effect on the second primary malignancy for head and neck cancer patients by a nomogram based on SEER database. Cancer Med. 2020;9:8029-8042.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 4]  [Cited by in F6Publishing: 8]  [Article Influence: 2.0]  [Reference Citation Analysis (0)]
22.  Huang Y, Liu Z, Zhong L, Wen Y, Ye Q, Cao D, Li P, Liu Y. Construction of an 11-microRNA-based signature and a prognostic nomogram to predict the overall survival of head and neck squamous cell carcinoma patients. BMC Genomics. 2020;21:691.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 8]  [Cited by in F6Publishing: 11]  [Article Influence: 2.8]  [Reference Citation Analysis (0)]
23.  Chen L, Wen Y, Zhang J, Sun W, Lui VWY, Wei Y, Chen F, Wen W. Prediction of radiotherapy response with a 5-microRNA signature-based nomogram in head and neck squamous cell carcinoma. Cancer Med. 2018;7:726-735.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 26]  [Cited by in F6Publishing: 31]  [Article Influence: 5.2]  [Reference Citation Analysis (0)]
24.  Mair M, Nair D, Nair S, Malik A, Mishra A, Kannan S, Bobdey S, Singhvi H, Chaturvedi P. Comparison of tumor volume, thickness, and T classification as predictors of outcomes in surgically treated squamous cell carcinoma of the oral tongue. Head Neck. 2018;40:1667-1675.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 2]  [Cited by in F6Publishing: 7]  [Article Influence: 1.2]  [Reference Citation Analysis (0)]
25.  van Smeden M, Moons KGM. Event rate net reclassification index and the integrated discrimination improvement for studying incremental value of risk markers. Stat Med. 2017;36:4495-4497.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 7]  [Cited by in F6Publishing: 9]  [Article Influence: 1.8]  [Reference Citation Analysis (0)]
26.  Thomas LE, O'Brien EC, Piccini JP, D'Agostino RB, Pencina MJ. Application of net reclassification index to non-nested and point-based risk prediction models: a review. Eur Heart J. 2019;40:1880-1887.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 18]  [Cited by in F6Publishing: 29]  [Article Influence: 7.3]  [Reference Citation Analysis (0)]
27.  Fitzgerald M, Saville BR, Lewis RJ. Decision curve analysis. JAMA. 2015;313:409-410.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 323]  [Cited by in F6Publishing: 414]  [Article Influence: 46.0]  [Reference Citation Analysis (0)]
28.  Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26:565-574.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 2176]  [Cited by in F6Publishing: 2908]  [Article Influence: 171.1]  [Reference Citation Analysis (0)]
29.  Mukdad L, Heineman TE, Alonso J, Badran KW, Kuan EC, St John MA. Oral tongue squamous cell carcinoma survival as stratified by age and sex: A surveillance, epidemiology, and end results analysis. Laryngoscope. 2019;129:2076-2081.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 26]  [Cited by in F6Publishing: 29]  [Article Influence: 4.8]  [Reference Citation Analysis (0)]
30.  Ansarin M, De Berardinis R, Corso F, Giugliano G, Bruschini R, De Benedetto L, Zorzi S, Maffini F, Sovardi F, Pigni C, Scaglione D, Alterio D, Cossu Rocca M, Chiocca S, Gandini S, Tagliabue M. Survival Outcomes in Oral Tongue Cancer: A Mono-Institutional Experience Focusing on Age. Front Oncol. 2021;11:616653.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 6]  [Cited by in F6Publishing: 8]  [Article Influence: 2.7]  [Reference Citation Analysis (0)]
31.  Tagliabue M, Belloni P, De Berardinis R, Gandini S, Chu F, Zorzi S, Fumagalli C, Santoro L, Chiocca S, Ansarin M. A systematic review and meta-analysis of the prognostic role of age in oral tongue cancer. Cancer Med. 2021;10:2566-2578.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 20]  [Cited by in F6Publishing: 22]  [Article Influence: 7.3]  [Reference Citation Analysis (0)]
32.  Langendijk JA, Ferlito A, Takes RP, Rodrigo JP, Suárez C, Strojan P, Haigentz M Jr, Rinaldo A. Postoperative strategies after primary surgery for squamous cell carcinoma of the head and neck. Oral Oncol. 2010;46:577-585.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 35]  [Cited by in F6Publishing: 39]  [Article Influence: 2.8]  [Reference Citation Analysis (0)]
33.  Fuwa N, Kodaira T, Furutani K, Tachibana H, Nakamura T, Nakahara R, Tomoda T, Inokuti H, Daimon T. Arterial chemoradiotherapy for locally advanced tongue cancer: analysis of retrospective study of therapeutic results in 88 patients. Int J Radiat Oncol Biol Phys. 2008;72:1090-1100.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 17]  [Cited by in F6Publishing: 13]  [Article Influence: 0.8]  [Reference Citation Analysis (0)]
34.  Yokota T, Iida Y, Ogawa H, Kamijo T, Onozawa Y, Todaka A, Hamauchi S, Onoe T, Nakagawa M, Yurikusa T, Tanuma A, Yamashita A, Nishimura T, Yasui H, Onitsuka T. Prognostic Factors and Multidisciplinary Postoperative Chemoradiotherapy for Clinical T4a Tongue Cancer. Oncology. 2016;91:78-84.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 3]  [Cited by in F6Publishing: 4]  [Article Influence: 0.5]  [Reference Citation Analysis (0)]
35.  Kim TH, Cha IH, Choi EC, Kim HR, Kim HJ, Kim SH, Keum KC, Lee CG. Postoperative Concurrent Chemoradiotherapy Versus Radiotherapy Alone for Advanced Oral Cavity Cancer in the Era of Modern Radiation Techniques. Front Oncol. 2021;11:619372.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 2]  [Cited by in F6Publishing: 3]  [Article Influence: 1.0]  [Reference Citation Analysis (0)]
36.  Silva PB, Lemos JV, Borges MM, do Rêgo TJ, Dantas TS, Leite CH, Lima MV, Cunha MP, Sousa FB. Prognostic factors on surgically and non-surgically treated oral squamous cell carcinoma: Advances in survival in fifteen years of follow up. J Clin Exp Dent. 2021;13:e240-e249.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1]  [Cited by in F6Publishing: 1]  [Article Influence: 0.3]  [Reference Citation Analysis (0)]
37.  Sun W, Qiu Z, Tan W, Liu Z, Wang Z, Huang W, Cao M. The influence of marital status on survival in patients with oral tongue squamous cell carcinoma. Oncotarget. 2017;8:82092-82102.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 8]  [Cited by in F6Publishing: 11]  [Article Influence: 1.6]  [Reference Citation Analysis (0)]
38.  Wu J, Zhang H, Li L, Hu M, Chen L, Xu B, Song Q. A nomogram for predicting overall survival in patients with low-grade endometrial stromal sarcoma: A population-based analysis. Cancer Commun (Lond). 2020;40:301-312.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 33]  [Cited by in F6Publishing: 155]  [Article Influence: 38.8]  [Reference Citation Analysis (0)]
39.  Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. 2015;350:g7594.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1604]  [Cited by in F6Publishing: 1591]  [Article Influence: 176.8]  [Reference Citation Analysis (0)]
40.  Tam S, Amit M, Zafereo M, Bell D, Weber RS. Depth of invasion as a predictor of nodal disease and survival in patients with oral tongue squamous cell carcinoma. Head Neck. 2019;41:177-184.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 26]  [Cited by in F6Publishing: 36]  [Article Influence: 6.0]  [Reference Citation Analysis (0)]
41.  Yuen AP, Lam KY, Wei WI, Ho CM, Chow TL, Yuen WF. A comparison of the prognostic significance of tumor diameter, length, width, thickness, area, volume, and clinicopathological features of oral tongue carcinoma. Am J Surg. 2000;180:139-143.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 157]  [Cited by in F6Publishing: 166]  [Article Influence: 6.9]  [Reference Citation Analysis (0)]
42.  de Juan J, García J, López M, Orús C, Esteller E, Quer M, León X. Inclusion of extracapsular spread in the pTNM classification system: a proposal for patients with head and neck carcinoma. JAMA Otolaryngol Head Neck Surg. 2013;139:483-488.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 70]  [Cited by in F6Publishing: 76]  [Article Influence: 6.9]  [Reference Citation Analysis (0)]
43.  Wreesmann VB, Katabi N, Palmer FL, Montero PH, Migliacci JC, Gönen M, Carlson D, Ganly I, Shah JP, Ghossein R, Patel SG. Influence of extracapsular nodal spread extent on prognosis of oral squamous cell carcinoma. Head Neck. 2016;38 Suppl 1:E1192-E1199.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 109]  [Cited by in F6Publishing: 111]  [Article Influence: 12.3]  [Reference Citation Analysis (0)]
44.  Lydiatt WM, Patel SG, O'Sullivan B, Brandwein MS, Ridge JA, Migliacci JC, Loomis AM, Shah JP. Head and Neck cancers-major changes in the American Joint Committee on cancer eighth edition cancer staging manual. CA Cancer J Clin. 2017;67:122-137.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 893]  [Cited by in F6Publishing: 914]  [Article Influence: 130.6]  [Reference Citation Analysis (0)]
45.  Girolami I, Pantanowitz L, Munari E, Martini M, Nocini R, Bisi N, Molteni G, Marchioni D, Ghimenton C, Brunelli M, Eccher A. Prevalence of PD-L1 expression in head and neck squamous precancerous lesions: a systematic review and meta-analysis. Head Neck. 2020;42:3018-3030.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 13]  [Cited by in F6Publishing: 20]  [Article Influence: 5.0]  [Reference Citation Analysis (0)]
46.  Paolino G, Pantanowitz L, Barresi V, Pagni F, Munari E, Moretta L, Brunelli M, Bariani E, Vigliar E, Pisapia P, Malapelle U, Troncone G, Girolami I, Eccher A. PD-L1 evaluation in head and neck squamous cell carcinoma: Insights regarding specimens, heterogeneity and therapy. Pathol Res Pract. 2021;226:153605.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 8]  [Cited by in F6Publishing: 24]  [Article Influence: 8.0]  [Reference Citation Analysis (0)]
47.  Munari E, Mariotti FR, Quatrini L, Bertoglio P, Tumino N, Vacca P, Eccher A, Ciompi F, Brunelli M, Martignoni G, Bogina G, Moretta L. PD-1/PD-L1 in Cancer: Pathophysiological, Diagnostic and Therapeutic Aspects. Int J Mol Sci. 2021;22.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 60]  [Cited by in F6Publishing: 54]  [Article Influence: 18.0]  [Reference Citation Analysis (0)]
48.  Pai S, Bamodu OA, Lin YK, Lin CS, Chu PY, Chien MH, Wang LS, Hsiao M, Yeh CT, Tsai JT. CD47-SIRPα Signaling Induces Epithelial-Mesenchymal Transition and Cancer Stemness and Links to a Poor Prognosis in Patients with Oral Squamous Cell Carcinoma. Cells. 2019;8.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 32]  [Cited by in F6Publishing: 41]  [Article Influence: 8.2]  [Reference Citation Analysis (0)]
49.  Cao Y, Jiao N, Sun T, Ma Y, Zhang X, Chen H, Hong J, Zhang Y. CXCL11 Correlates With Antitumor Immunity and an Improved Prognosis in Colon Cancer. Front Cell Dev Biol. 2021;9:646252.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 31]  [Cited by in F6Publishing: 66]  [Article Influence: 22.0]  [Reference Citation Analysis (0)]
50.  Zhang Y, Luo X, Yu J, Qian K, Zhu H. An Immune Feature-Based, Three-Gene Scoring System for Prognostic Prediction of Head-and-Neck Squamous Cell Carcinoma. Front Oncol. 2021;11:739182.  [PubMed]  [DOI]  [Cited in This Article: ]  [Cited by in Crossref: 1]  [Cited by in F6Publishing: 1]  [Article Influence: 0.5]  [Reference Citation Analysis (0)]