Evaluation of nodal status in intrahepatic cholangiocarcinoma: a population-based study

Introduction

Intrahepatic cholangiocarcinoma (ICC) accounts for approximately 15% of primary liver cancers and 3% of gastrointestinal cancers. In the past few decades, the morbidity and mortality of ICC have increased worldwide, especially in Eastern Asia (1-4). Radical surgery is still the unique curative treatment for patients suffering ICC, with an expected median overall survival time of 51.1 months (3-5). ICC could be divided into two subtypes with different biological behaviors according to the tumor location: the hilar type and the peripheral type (6). Due to the similar surgical approach and perioperative management, the hilar type of ICC and hilar cholangiocarcinoma (Klatskin tumor) are often divided into the same category, known as perihilar cholangiocarcinoma (7,8). Therefore, the main focus of this study was the peripheral type of ICC.

Lymph node metastasis (LNM) has been considered one of ICC’s most widely accepted prognostic factors. However, the therapeutic value of regional lymph node dissection (LND) is still controversial (9-14). Several studies demonstrated no benefits of routine LND because of the similar survival, prolonged hospital-stays and increased surgical risks compared with patients not receiving LND (11,13-18). In addition, the minimum requirements of the total lymph node count (TLNC) also remain a matter of debate. The current American Joint Committee on Cancer (AJCC) staging system (8^th edition) recommends having 6 or more lymph nodes to be eligible for evaluation, but only a few patients could meet this criterion in previous studies (23.8–43.3%) (10,13,17-21). In this context, the first aim of this study is to investigate the role and minimum requirement of LND in patients with ICC.

An adequate assessment of nodal status is critical for selecting patients to receive adjuvant therapies (AT), and the current AJCC staging system (8^th edition) only differentiates between LNM and non-LNM disease. Except for the number of positive lymph nodes (pLN), 2 promising schemes are proposed for nodal assessment have been identified, namely the lymph node ratio (LNR) and the log odds of positive lymph nodes (LODDS). Therefore, this study also aims to compare prognostic performances among the different nodal staging schemes and then develop a nomogram to predict prognosis in patients with ICC.

We present the following article following the STROBE reporting checklist (available at https://dx.doi.org/10.21037/atm-21-2785). This population-based study was conducted using the Surveillance, Epidemiology, and End Results (SEER) database, which is an authoritative source of information on cancer incidence and survival in the United States, and it covers approximately 34.6% of the U.S. population. Data was downloaded with SEER*Stat software (Version 8.3.9; The SEER Program, https://seer.cancer.gov).

Methods

Patients

This study is a retrospective cohort study following the Declaration of Helsinki (as revised in 2013). Patients with diagnosed intrahepatic bile duct cancer from 2004 to 2013 in the SEER database (with additional treatment fields) were enrolled. The inclusion criteria were as follows: (I) age ≥18 years old; (II) diagnosis of ICC with positive histology (TNM 7/CS v0204+ Schema=BileDuctIntraHepat, ICD-O-3 Topography code=C22.1, ICD-O-3 Histology Code=8160/3); (III) without distant metastasis or previous history of other malignancies; (IV) surgery performed; (V) had a complete 5-year follow-up and survived at least 1 month after surgery; (VI) with complete clinicopathological data. The stepwise extraction process from the SEER database is shown in Figure S1. As the SEER database is public and desensitized, an ethical review was exempted, and no consent was needed in this study.

Definitions

The primary outcome was cancer-specific survival (CSS), defined as either the time from the patient’s diagnosis until their death caused by ICC or their most recent follow-up. The T stage was re-staged according to the current AJCC staging system (8^th edition). The pLN model was defined as the number of positive lymph nodes (LN) and the LNR model was defined as the ratio of pLN to TLNC. The LODDS model is defined as the natural logarithm of the probability ratio between LN with or without tumor invasion and is calculated as ln [(positive LN+0.5)/(negative LN+0.5)].

Statistics

Survival curves were plotted using the Kaplan-Meier method and compared by the log-rank test. Clinicopathological variables possibly related to survival were evaluated by a multivariate analysis using the Cox regression model. Propensity score matching (PSM) was used to reduce selection bias between groups. A one-to-one match was performed by the nearest-neighbor method within 0.20 standard deviations between the two groups. Independent risk factors were analyzed by a multivariable binary logistic regression analysis with a threshold of P<0.10. The Youden Index was used to determine the optimal cut-off value for the TLNC to find a positive lymph node. X-tile software (Version 3.6.1; Yale University, New Haven, CT, USA) was used to determine the optimal outcome-based cut-off value of each nodal staging scheme (22).

The study cohort was randomly divided into a training set and a validation set, with a ratio of 3:1. Models were developed using the training set, and external validation was performed using the validation set. Akaike information criterion (AIC), Bayesian information criterion (BIC), Harrell’s C-index, and the area under receiver operating curves (AUROC) of CSS probability were calculated to compare the prognostic performances of different schemes. A nomogram was constructed based on multivariate survival analysis to provide a visual tool for clinical use. Calibration curves to evaluate the predictive accuracy of models were plotted via bootstrapping with 1,000 resamples. A decision curve analysis (DCA) was performed to estimate the clinical utility of the models by quantifying the net benefits at different threshold probabilities (23). A result was considered statistically significant when two-tailed P<0.05. All statistical analyses were completed using R software (Version 3.6.3; The R Foundation for Statistical Computing, http://www.r-project.org).

Results

Baseline characteristics and survival analysis

The baseline characteristics and survival analysis of study patients are shown in Table 1. A total of 664 patients were enrolled in the study, comprising 354 (55.0%) females and 290 (45.0%) males. The average age was 62.2±11.5 years (range: 25–89 years), and the mean tumor size was 6.1±3.4 cm (range 1.0–17.0 cm). There were 278 (43.2%) patients who received AT. According to the current AJCC staging system (8^th edition), 474 (73.6%) patients were classified as T1–T2 stage.

The final patient follow-up was in November 2018, with a median follow-up of 36.0 months. There were 433 (67.2%) patients who died. The 1-yr, 3-yr, and 5-yr CSS were 86.5%, 55.8% and 41.6%, respectively. The median CSS time was 45.0 [95% confidence interval (CI): 39.1–50.9] months. Multivariate survival analysis shows that age, tumor numbers, tumor size, T stage, tumor grade, and pLN stage were all independent risk factors for CSS (all P<0.05).

Impact of nodal status on survival and minimum requirement for TLNC

A total of 331 (51.4%) patients underwent LND. There were 1,370 lymph nodes retrieved in all, with the TLNC averaging 4.1±4.3 (median: 3) and ranging from 1–32. As shown in Table 1, an increased TLNC could not improve prognosis (P=0.876). After PSM (251 patients in each group), there was no survival difference between the LND and non-LND groups (P=0.095, Figure 1A & Table S1).

Figure 1 Kaplan-Meier analysis of CSS relative to nodal status. (A) LND versus non-LND in all patients (251 patients in each group after PSM); (B) LNM versus non-LNM in patients with LND (96 patients in each group after PSM); (C) AT versus non-AT in pN0 patients (54 patients in each group after PSM); (D) AT versus non-AT in pN+ patients. CSS, cancer-specific survival; LND, lymph node dissection; LNM, lymph node metastasis; AT, adjuvant therapy (chemotherapy and/or radiotherapy); PSM, propensity score matching.

LNM was identified in 103 (31.1%) patients. The 1-yr, 3-yr, 5-yr CSS of patients with and without LNM were 77.1%, 27.0%, 13.1% and 88.1%, 61.5%, 44.9%, respectively, while the median CSS times were 19.0 (95% CI: 15.5–22.5) and 54.0 (95% CI: 44.7–63.3) months, respectively. After PSM (96 patients in each group), patients with LNM were confirmed to have worse chances of survival (P<0.0001, Figure 1B & Table S2). Furthermore, 2 potential preoperative risk factors for LNM (T stage and tumor number) were examined by multivariable analysis (Table S3), and T stage was indicated as the only independent risk factor of LNM (P<0.05).

There were 165 (49.8%) patients with LND who received postoperative AT (105 patients with chemotherapy, 6 patients with radiotherapy, 54 patients with both). After PSM (54 patients in each group), AT showed no therapeutic benefit to patients without LNM (P=0.140, Figure 1C & Table S4). However, in the LNM group, AT was shown to significantly improve the prognosis of patients (P=0.018, Figure 1D & Table S5).

In this study, only 73 (22.1%) patients met the LND criterion recommended by the AJCC. The more lymph nodes that are examined, the higher the possibility of finding positive lymph nodes. The ROC curve of the TLNC for predicting LNM in patients with LND is shown in Figure 2, and the AUC is 0.702 (95% CI: 0.642–0.763, P<0.001). The ROC analysis confirmed that a TLNC ≥4 displayed the greatest discriminatory power of LNM (Youden index =0.302, sensitivity =0.583, specificity =0.719).

Figure 2 ROC analysis illustrated that the dissection of 4 or more lymph nodes had the highest discriminatory power relative to CSS. ROC, receiver operative characteristic; CSS, cancer-specific survival; TLNC, total lymph node count; AUC, area under the curve.

Prognostic performance of regional LN staging schemes

A total of 644 patients were randomly divided into a training set (n=483, 75.0%) and a validation set (n=161, 25.0%). The baseline characteristics of the 2 sets are shown in Table S6. The X-tile plots for determining the optimal cut-off values of the different schemes are displayed in Figure S2. In the training set, 248 (51.3%) patients with LND were classified into 2 subgroups based on the optimal cut-off value of each scheme: 175 (70.6%) without LNM (pLN1) and 73 (29.4%) patients with LNM (pLN2), 188 (75.8%) patients with LNR ≤0.15 (LNR1) and 60 (24.2%) patients with LNR >0.15 (LNR2), 193 (77.8%) patients with LODDS ≤−0.85 (LODDS1) and 55 (22.2%) patients with LODDS >−0.85 (LODDS2). As shown in Table 2 & Figure S3, the LNR classification had the highest Harrell’s C-index and AUROC and the lowest AIC and BIC in both the training and validation sets, supporting the theory that LNR classification may have better prognostic performance.

Development, performance, and validation of prediction models

In the training set, a multivariate survival analysis (Table S7) confirmed that age, tumor numbers, tumor size, T stage, tumor grade, and the LNR classification were independent risk factors for CSS (all P<0.05). Based on these findings, a nomogram was developed (Figure 3A). The Harrell’s C-index of the nomogram was 0.710 (95% CI: 0.670−0.750) in the training set and 0.743 (95% CI: 0.695−0.790) in the validation set. Compared with the current AJCC staging system (8^th edition), the nomogram had a better discriminatory power relative to the CSS in patients with ICC (Table 3 & Figure S4). The calibration curves demonstrated favorable calibration of the nomogram both in the training and validation sets (Figure 3B-3G). The DCA for the nomogram is presented in Figure 3H-3M. The nomogram provided a better net benefit than the ‘treat-all’ or ‘treat-none’ schemes and the current AJCC staging system.

Figure 3 Development, validation, and comparison nomogram. (A) The nomogram to predict CSS developed from the training set; (B-D) Calibration curve analysis nomogram and the current AJCC staging system (8^th edition) in the prediction of prognosis at 1-, 3-, and 5-year points for CSS in the training set; (E-G) Calibration curve analysis nomogram and the current AJCC staging system (8^th edition) in the prediction of prognosis at 1-, 3-, and 5-year points for CSS in the validation set; (H-J) DCA nomogram and the current AJCC staging system (8^th edition) in the prediction of prognosis at 1-, 3-, and 5-year points for CSS in the training set; (K-M) DCA nomogram and the current AJCC staging system (8^th edition) in the prediction of prognosis at 1-, 3-, and 5-year points for CSS in the validation set. CSS, cancer-specific survival; DCA, decision curve analysis; AJCC, American Joint Committee on Cancer.

Discussion

Radical surgery remains the only chance for patients with ICC to acquire long-time survival. Although perioperative management and surgical techniques have made great developments in recent decades, the prognosis of patients with ICC is still unsatisfactory (3-5). LNM presents more aggressive biological behavior, and it has been confirmed as an independent predictive factor for prognosis by multivariate analysis. However, unlike other biliary cancers, the role of LND is still a topic of debate in ICC (9-14).

The incidences of LND and LNM in our cohort were 51.4% and 31.1%, respectively, which is similar to previous studies (LND: 27.9–78.5%; LNM: 25.2–45.2%) (9,10,13,16,18-21,24-29). After PSM, LND showed no oncologic benefit in patients with ICC as survival was similar to the non-LND group (P=0.095). The same finding was also obtained in several previous studies (11,13-18,25,26,28), with 2 main factors potentially explaining this result: Firstly, the LND procedure meant longer operating times, more blood transfusions, and higher morbidity of postoperative complications, all of which were not beneficial to prognosis (16,17). Secondly, the indication and anatomic area of the LND were still varied in different centers, which might increase selection bias (13). This study also showed that the prognosis of the non-LND group was similar to the non-LNM group and not the LNM group (P=0.813 and P<0.001), indicating that not all patients would benefit from LND, which was supported by Lee et al. (26).

However, it is undeniable that LND could provide necessary staging information, as patients with LNM could benefit from AT (P=0.018) while patients without LNM could not (P=0.140). Multivariate analysis demonstrated that an increased TLNC could not improve prognosis (P=0.876), and the AJCC T stage was the only independent risk factor for LNM (P<0.05). Considering the incidence (31.1% in this study) and adverse effects of LNM, a routine but limited LND should be strongly considered in ICC patients, especially in patients with advanced AJCC T stages. In this context, to determine the minimally required TLNC, it is important to find potential positive lymph nodes and control surgical trauma. The current AJCC staging system (8^th edition) recommends that 6 or more lymph nodes be examined, but only 22.1% of patients in this study could meet this criterion and receive an adequate evaluation, which was similar to other studies (10,13,17-21). The increased TLNC is indicative of a decreased sensitivity and an increased specificity in finding positive lymph nodes. Based on the Youden index, a TLNC ≥4 showed the greatest discriminating power in our cohort. Therefore, 4 or more lymph nodes may be adequate for patients with ICC to acquire accurate staging.

As for location, the regular pattern of LNM in ICC patients is not yet fully understood. Several factors were reported to affect the nodal status of ICC in previous studies, including size (20), location (9,26,28), macroscopic type (30), and initial liver disease (31,32). Due to a lack of detailed patient-level clinical data in the SEER database, further analyses were limited in our study. In their study, Kang et al. (9) found that the first 2 frequently metastatic lymph nodes were the number No. 12 (36%) and No. 8 (21%) stations. Zhang et al. (19) analyzed the nodal status of 216 patients with LNM and found that at least 153 (70.8%) patients had LNM within the No. 12 station only. Shimada et al. (28) specified that the No. 8 stations should be resected in left-sided tumors. Therefore, No. 12 and No. 8 stations should be paid more attention to at the time of surgery. A meta-analysis compared the outcomes between laparoscopic (LLR) and open liver resection (OLR). Although LND was less common in the LLR group, the proportion is becoming more frequent. The unique advantages of laparoscopy may lead to a more accurate intraoperative evaluation of nodal status (29).

It has also been debated without a consensus on the best scheme for evaluating regional nodal status. Unlike extrahepatic cholangiocarcinoma and gallbladder cancer, the current AJCC staging system (8^th edition) differentiates patients with ICC between LNM and non-LN only. Visually, the use of the LNR and LODDS is more rational than pLN as they take both pLN and TLNC into account, and the 2 models have also been applied to other gastrointestinal malignancies (33-38). Some researchers preferred the LODDS classification because the LNR classification has a congenital limitation in its dependence on the denominator (or the TLNC), especially when LNR =0 or LNR =1 (39,40). However, the TLNC for ICC is usually smaller than other gastrointestinal malignancies, such as extrahepatic cholangiocarcinoma and pancreatic cancer (35-38). The median of the TLNC was only 3 [interquartile range (IQR) =1–5] in ICC patients with LND in the cohort, which restricted the advantage of LODDS in these edge cases. In this study, the LNR classification showed the best prognostic performance. Several studies reported the satisfactory predictive ability of the LNR classification, with cut-off values from 0.10–0.50 (25,39-42). Likewise, we observed that patients with an LNR >0.15 had a better chance of survival than those with LNR ≤0.15 (P<0.001). Moreover, the LNR classification was again identified as a significant prognostic factor upon multivariate analysis (P<0.001). Considering the simplicity of its calculation, the LNR seems to be a promising prognosticator for the nodal status of ICC.

A nomogram is an intuitive, comprehensible, and user-friendly statistical tool that allows multiple factors to be considered simultaneously and visually provides a probability of a specific outcome for an individual patient. On account of the multivariate analysis, we incorporated 6 easily accessible clinicopathological factors (LNR, age, tumor size, tumor number, T stage, and grade) and developed a nomogram for predicting the CSS in patients with ICC. We then conducted external validation. The nomogram showed relatively high accuracy with Harrell’s C-indexes exceeding 0.700 and well-fitted calibration curves in both the training and validation sets. Besides, the nomogram also displayed better goodness of fit according to lower its AIC and BIC values. However, high prediction accuracy is not equal to a high clinical practical value. The DCA could quantify the net benefits of the prediction models based on the threshold probability introduced to this study to examine the value of the nomogram in clinical practice (23). The DCA confirmed the validity of the nomogram for the CSS and demonstrated that the nomogram had better clinical value than the current AJCC (8^th edition) staging system.

The evaluation of nodal status in ICC has received increasing attention recently, but no prospective or real-world study has yet been published. In a high-volume cohort, utilizing the SEER database, we investigated the role and minimum requirements of LND, explored the clinical value of the LNR, and then developed a nomogram to predict the CSS of patients with ICC. There are several limitations to the research: Firstly, the major drawback of this study is the inherent bias of retrospective study. Secondly, the SEER database lacks detailed clinicopathological data, which caused unknown bias and limited further subgroup analysis; however, the sample capacity, complete 5-year follow-up, and population-based research background could make up for the short slab to a certain degree.

In conclusion, LNM is a powerful and independent prognostic factor in patients with ICC. Although showing no oncologic benefit, LND should still be considered at the time of surgery to stratify patients, and 4 or more lymph nodes retrieved may be enough for appropriate staging. The LNR appears to serve as a promising and easy-to-use tool for nodal staging in ICC, while the constructed nomogram could predict the CSS with good performance, which is meaningful to individual treatment strategies optimization in patients with ICC.

Acknowledgments

Funding: This study was supported by grants from the National Natural Science Foundation of China (Grant No. 31930020, 81870488, 81521004, 81530048) and Key Laboratory of Liver Transplantation, Chinese Academy of Medical Sciences (Grant No. 2017PT320008, 2018PT31043, 2019PT320015).

Footnote

Reporting Checklist: The authors have completed the STROBE reporting checklist. Available at https://dx.doi.org/10.21037/atm-21-2785

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure forms (available at https://dx.doi.org/10.21037/atm-21-2785). XW serves as an Editor-in-Chief of Annals of Translational Medicine from Aug 2019 to July 2024. The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted following the Declaration of Helsinki (as revised in 2013). The SEER database is public and desensitized, so the ethical review was exempted, and no consent was needed in this study.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Valle JW, Kelley RK, Nervi B, et al. Biliary tract cancer. Lancet 2021;397:428-44. [Crossref] [PubMed]
2. Florio AA, Ferlay J, Znaor A, et al. Global trends in intrahepatic and extrahepatic cholangiocarcinoma incidence from 1993 to 2012. Cancer 2020;126:2666-78. [Crossref] [PubMed]
3. Banales JM, Marin JJG, Lamarca A, et al. Cholangiocarcinoma 2020: the next horizon in mechanisms and management. Nat Rev Gastroenterol Hepatol 2020;17:557-88. [Crossref] [PubMed]
4. Cillo U, Fondevila C, Donadon M, et al. Surgery for cholangiocarcinoma. Liver Int 2019;39:143-55. [Crossref] [PubMed]
5. Xia YX, Zhang F, Li XC, et al. Surgical treatment of primary liver cancer:a report of 10 966 cases. Zhonghua Wai Ke Za Zhi 2021;59:6-17. [PubMed]
6. Orimo T, Kamiyama T, Mitsuhashi T, et al. Impact of tumor localization on the outcomes of surgery for an intrahepatic cholangiocarcinoma. J Gastroenterol 2018;53:1206-15. [Crossref] [PubMed]
7. Nakeeb A, Pitt HA, Sohn TA, et al. Cholangiocarcinoma. A spectrum of intrahepatic, perihilar, and distal tumors. Ann Surg 1996;224:463-73; discussion 473-5. [Crossref] [PubMed]
8. Ebata T, Kosuge T, Hirano S, et al. Proposal to modify the International Union Against Cancer staging system for perihilar cholangiocarcinomas. Br J Surg 2014;101:79-88. [Crossref] [PubMed]
9. Kang CM, Suh KS, Yi NJ, et al. Should Lymph Nodes Be Retrieved in Patients with Intrahepatic Cholangiocarcinoma? A Collaborative Korea-Japan Study. Cancers (Basel) 2021;13:445. [Crossref] [PubMed]
10. Ke Q, Wang L, Lin Z, et al. Prognostic Value of Lymph Node Dissection for Intrahepatic Cholangiocarcinoma Patients With Clinically Negative Lymph Node Metastasis: A Multi-Center Study From China. Front Oncol 2021;11:585808 [Crossref] [PubMed]
11. Machairas N, Lang H, Jayant K, et al. Intrahepatic cholangiocarcinoma: Limitations for resectability, current surgical concepts and future perspectives. Eur J Surg Oncol 2020;46:740-6. [Crossref] [PubMed]
12. Kim SH, Han DH, Choi GH, et al. Oncologic Impact of Lymph Node Dissection for Intrahepatic Cholangiocarcinoma: a Propensity Score-Matched Study. J Gastrointest Surg 2019;23:538-44. [Crossref] [PubMed]
13. Amini N, Ejaz A, Spolverato G, et al. Management of lymph nodes during resection of hepatocellular carcinoma and intrahepatic cholangiocarcinoma: a systematic review. J Gastrointest Surg 2014;18:2136-48. [Crossref] [PubMed]
14. Adachi T, Eguchi S. Lymph node dissection for intrahepatic cholangiocarcinoma: a critical review of the literature to date. J Hepatobiliary Pancreat Sci 2014;21:162-8. [Crossref] [PubMed]
15. Kim DH, Choi DW, Choi SH, et al. Is there a role for systematic hepatic pedicle lymphadenectomy in intrahepatic cholangiocarcinoma? A review of 17 years of experience in a tertiary institution. Surgery 2015;157:666-75. [Crossref] [PubMed]
16. Wang L, Lin ZG, Yang T, et al. A multicenter retrospective study on clinical value of lymph node dissection in the radical resection of intrahepatic cholangiocarcinoma. Chin J Dig Surg 2020;19:72-80.
17. Zhou R, Lu D, Li W, et al. Is lymph node dissection necessary for resectable intrahepatic cholangiocarcinoma? A systematic review and meta-analysis. HPB (Oxford) 2019;21:784-92. [Crossref] [PubMed]
18. Bagante F, Spolverato G, Weiss M, et al. Assessment of the Lymph Node Status in Patients Undergoing Liver Resection for Intrahepatic Cholangiocarcinoma: the New Eighth Edition AJCC Staging System. J Gastrointest Surg 2018;22:52-9.
19. Zhang XF, Xue F, Dong DH, et al. Number and Station of Lymph Node Metastasis After Curative-intent Resection of Intrahepatic Cholangiocarcinoma Impact Prognosis. Ann Surg 2020; [Epub ahead of print].
20. Ruzzenente A, Conci S, Viganò L, et al. Role of Lymph Node Dissection in Small (≤ 3 cm) Intrahepatic Cholangiocarcinoma. J Gastrointest Surg 2019;23:1122-9. [Crossref] [PubMed]
21. Wu L, Tsilimigras DI, Paredes AZ, et al. Trends in the Incidence, Treatment and Outcomes of Patients with Intrahepatic Cholangiocarcinoma in the USA: Facility Type is Associated with Margin Status, Use of Lymphadenectomy and Overall Survival. World J Surg 2019;43:1777-87. [Crossref] [PubMed]
22. Camp RL, Dolled-Filhart M, Rimm DL. X-tile: a new bio-informatics tool for biomarker assessment and outcome-based cut-point optimization. Clin Cancer Res 2004;10:7252-9. [Crossref] [PubMed]
23. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making 2006;26:565-74. [Crossref] [PubMed]
24. Meng ZW, Lin XQ, Zhu JH, et al. A nomogram to predict lymph node metastasis before resection in intrahepatic cholangiocarcinoma. J Surg Res 2018;226:56-63. [Crossref] [PubMed]
25. Tamandl D, Kaczirek K, Gruenberger B, et al. Lymph node ratio after curative surgery for intrahepatic cholangiocarcinoma. Br J Surg 2009;96:919-25. [Crossref] [PubMed]
26. Lee W, Jeong CY, Jang JY, et al. Clinical implication of tumor site in terms of node metastasis for intrahepatic cholangiocarcinoma. Eur J Surg Oncol 2020;46:832-8. [Crossref] [PubMed]
27. de Jong MC, Nathan H, Sotiropoulos GC, et al. Intrahepatic cholangiocarcinoma: an international multi-institutional analysis of prognostic factors and lymph node assessment. J Clin Oncol 2011;29:3140-5. [Crossref] [PubMed]
28. Shimada M, Yamashita Y, Aishima S, et al. Value of lymph node dissection during resection of intrahepatic cholangiocarcinoma. Br J Surg 2001;88:1463-6. [Crossref] [PubMed]
29. Machairas N, Kostakis ID, Schizas D, et al. Meta-analysis of laparoscopic versus open liver resection for intrahepatic cholangiocarcinoma. Updates Surg 2021;73:59-68. [Crossref] [PubMed]
30. Shimada K, Sano T, Nara S, et al. Therapeutic value of lymph node dissection during hepatectomy in patients with intrahepatic cholangiocellular carcinoma with negative lymph node involvement. Surgery 2009;145:411-6. [Crossref] [PubMed]
31. Bagante F, Gani F, Spolverato G, et al. Intrahepatic Cholangiocarcinoma: Prognosis of Patients Who Did Not Undergo Lymphadenectomy. J Am Coll Surg 2015;221:1031-40.e1-4.
32. Huang J, Wang X, Zhu Y, et al. Specific prognostic factors in hepatitis B virus-related and non-hepatitis B virus-related intrahepatic cholangiocarcinoma after macroscopic curative resection. J Surg Oncol 2019;119:40-6. [Crossref] [PubMed]
33. Kano K, Yamada T, Yamamoto K, et al. Evaluation of Lymph Node Staging Systems as Independent Prognosticators in Remnant Gastric Cancer Patients with an Insufficient Number of Harvested Lymph Nodes. Ann Surg Oncol 2021;28:2866-76. [Crossref] [PubMed]
34. Baqar AR, Wilkins S, Wang W, et al. Log odds of positive lymph nodes is prognostically equivalent to lymph node ratio in non-metastatic colon cancer. BMC Cancer 2020;20:762. [Crossref] [PubMed]
35. Li P, Song L. A novel prognostic nomogram for patients with surgically resected perihilar cholangiocarcinoma: a SEER-based study. Ann Transl Med 2021;9:54. [Crossref] [PubMed]
36. You Y, Shin YC, Choi DW, et al. Proposed Modification of Staging for Distal Cholangiocarcinoma Based on the Lymph Node Ratio Using Korean Multicenter Database. Cancers (Basel) 2020;12:762. [Crossref] [PubMed]
37. Conci S, Ruzzenente A, Sandri M, et al. What is the most accurate lymph node staging method for perihilar cholangiocarcinoma? Comparison of UICC/AJCC pN stage, number of metastatic lymph nodes, lymph node ratio, and log odds of metastatic lymph nodes. Eur J Surg Oncol 2017;43:743-50. [Crossref] [PubMed]
38. Pu N, Gao S, Beckman R, et al. Defining a minimum number of examined lymph nodes improves the prognostic value of lymphadenectomy in pancreas ductal adenocarcinoma. HPB (Oxford) 2021;23:575-86. [Crossref] [PubMed]
39. Kim Y, Spolverato G, Amini N, et al. Surgical Management of Intrahepatic Cholangiocarcinoma: Defining an Optimal Prognostic Lymph Node Stratification Schema. Ann Surg Oncol 2015;22:2772-8. [Crossref] [PubMed]
40. Chen C, Wu YH, Li Q, et al. Application value of different lymph node staging system in predicting prognosis of patients with intrahepatic cholangiocarcinoma. Zhonghua Wai Ke Za Zhi 2020;58:295-302. [PubMed]
41. Guglielmi A, Ruzzenente A, Campagnaro T, et al. Patterns and prognostic significance of lymph node dissection for surgical treatment of perihilar and intrahepatic cholangiocarcinoma. J Gastrointest Surg 2013;17:1917-28. [Crossref] [PubMed]
42. Huang G, Liu K, Yi L. Prognostic Value of the Positive Lymph Node Ratio among Intrahepatic Cholangiocarcinoma Patients. J Am Coll Surg 2020;231:S166-7. [Crossref]

(English Language Editors: J. Collie and J. Chapnick)