Abstract
Background
The treatment situation for hepatocellular carcinoma remains critical. The use of deep learning algorithms to assess immune infiltration is a promising new diagnostic tool.
Methods
Patient data and whole slide images (WSIs) were obtained for the Xijing Hospital (XJH) cohort and TCGA cohort. We wrote programs using Visual studio 2022 with C# language to segment the WSI into tiles. Pathologists classified the tiles and later trained deep learning models using the ResNet 101V2 network via ML.NET with the TensorFlow framework. Model performance was evaluated using AccuracyMicro versus AccuracyMacro. Model performance was examined using ROC curves versus PR curves. The percentage of immune infiltration was calculated using the R package survminer to calculate the intergroup cutoff, and the Kaplan‒Meier method was used to plot the overall survival curve of patients. Cox regression was used to determine whether the percentage of immune infiltration was an independent risk factor for prognosis. A nomogram was constructed, and its accuracy was verified using time-dependent ROC curves with calibration curves. The CIBERSORT algorithm was used to assess immune infiltration between groups. Gene Ontology was used to explore the pathways of differentially expressed genes.
Results
There were 100 WSIs and 165,293 tiles in the training set. The final deep learning models had an AccuracyMicro of 97.46% and an AccuracyMacro of 82.28%. The AUCs of the ROC curves on both the training and validation sets exceeded 0.95. The areas under the classification PR curves exceeded 0.85, except that of the TLS on the validation set, which might have had poor results (0.713) due to too few samples. There was a significant difference in OS between the TIL classification groups (p < 0.001), while there was no significant difference in OS between the TLS groups (p = 0.294). Cox regression showed that TIL percentage was an independent risk factor for prognosis in HCC patients (p = 0.015). The AUCs according to the nomogram were 0.714, 0.690, and 0.676 for the 1-year, 2-year, and 5-year AUCs in the TCGA cohort and 0.756, 0.797, and 0.883 in the XJH cohort, respectively. There were significant differences in the levels of infiltration of seven immune cell types between the two groups of samples, and gene ontology showed that the differentially expressed genes between the groups were immune related. Their expression levels of PD-1 and CTLA4 were also significantly different.
Conclusion
We constructed and tested a deep learning model that evaluates the immune infiltration of liver cancer tissue in HCC patients. Our findings demonstrate the value of the model in assessing patient prognosis, immune infiltration and immune checkpoint expression levels.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and materials
The datasets generated and/or analyzed during the current study, as well as the C#/R software source code used, can be found at https://github.com/iWiley/Assessing-prognosis-with-deep-learning or https://portal.gdc.cancer.gov/.
References
Aggarwal R, Sounderajah V, Martin G et al (2021) Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis. NPJ Digit Med 4(1):65. https://doi.org/10.1038/s41746-021-00438-z
Ahmed Z, Amizadeh S, Bilenko M, et al (2019) Machine learning at microsoft with ML.NET. https://doi.org/10.1145/3292500.3330667
Ashburner M, Ball CA, Blake JA et al (2000) Gene Ontology: tool for the unification of biology. Nat Genet 25(1):25–29. https://doi.org/10.1038/75556
Calderaro J, Seraphin TP, Luedde T, Simon TG (2022) Artificial intelligence for the prevention and clinical management of hepatocellular carcinoma. J Hepatol 76(6):1348–1361. https://doi.org/10.1016/j.jhep.2022.01.014
Cao P, Ma B, Sun D et al (2021) hsa_circ_0003410 promotes hepatocellular carcinoma progression by increasing the ratio of M2/M1 macrophages through the miR-139–3p/CCL5 axis. Cancer Sci. https://doi.org/10.1111/cas.15238
Chen B, Garmire L, Calvisi DF, Chua MS, Kelley RK, Chen X (2020) Harnessing big “omics” data and AI for drug discovery in hepatocellular carcinoma. Nat Rev Gastroenterol Hepatol. https://doi.org/10.1038/s41575-019-0240-9
Cheng N, Ren Y, Zhou J et al (2022) Deep learning-based classification of hepatocellular nodular lesions on whole-slide histopathologic images. Gastroenterology 162(7):1948-1961.e7. https://doi.org/10.1053/j.gastro.2022.02.025
Deo RC (2015) Machine learning in medicine. Circulation 132(20):1920–1930. https://doi.org/10.1161/CIRCULATIONAHA.115.001593
Dieu-Nosjean MC, Antoine M, Danel C et al (2008) Long-term survival for patients with non-small-cell lung cancer with intratumoral lymphoid structures. J Clin Oncol off J Am Soc Clin Oncol 26(27):4410–4417. https://doi.org/10.1200/JCO.2007.15.0284
Duggento A, Conti A, Mauriello A, Guerrisi M, Toschi N (2021) Deep computational pathology in breast cancer. Semin Cancer Biol 72:226–237. https://doi.org/10.1016/j.semcancer.2020.08.006
Ferlay J, Laversanne M, Ervik M, et al (2020) Global cancer observatory: cancer tomorrow. Cancer tomorrow. https://gco.iarc.fr/tomorrow. Accessed 14 Nov 2021
Foerster F, Gairing SJ, Ilyas SI, Galle PR (2022) Emerging immunotherapy for HCC: a guide for hepatologists. Hepatol Baltim Md 75(6):1604–1626. https://doi.org/10.1002/hep.32447
Gene Ontology Consortium (2021) The Gene Ontology resource: enriching a GOld mine. Nucleic Acids Res 49(D1):D325–D334. https://doi.org/10.1093/nar/gkaa1113
Goecks J, Jalili V, Heiser LM, Gray JW (2020) How machine learning will transform biomedicine. Cell 181(1):92–101. https://doi.org/10.1016/j.cell.2020.03.022
Groeneveld CS, Fontugne J, Cabel L et al (1990) Tertiary lymphoid structures marker CXCL13 is associated with better survival for patients with advanced-stage bladder cancer treated with immunotherapy. Eur J Cancer Oxf Engl 2021(148):181–189. https://doi.org/10.1016/j.ejca.2021.01.036
Helmink BA, Reddy SM, Gao J et al (2020) B cells and tertiary lymphoid structures promote immunotherapy response. Nature 577(7791):549–555. https://doi.org/10.1038/s41586-019-1922-8
Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554. https://doi.org/10.1162/neco.2006.18.7.1527
Li H, Wang J, Liu H et al (2020) Existence of intratumoral tertiary lymphoid structures is associated with immune cells infiltration and predicts better prognosis in early-stage hepatocellular carcinoma. Aging 12(4):3451–3472. https://doi.org/10.18632/aging.102821
Litjens G, Kooi T, Bejnordi BE et al (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88. https://doi.org/10.1016/j.media.2017.07.005
Lu C, Rong D, Hui B et al (2021) CircETFA upregulates CCL5 by sponging miR-612 and recruiting EIF4A3 to promote hepatocellular carcinoma. Cell Death Discov 7(1):321. https://doi.org/10.1038/s41420-021-00710-x
Moen E, Bannon D, Kudo T, Graf W, Covert M, Van Valen D (2019) Deep learning for cellular image analysis. Nat Methods 16(12):1233–1246. https://doi.org/10.1038/s41592-019-0403-1
Nensa F, Demircioglu A, Rischpler C (2019) Artificial intelligence in nuclear medicine. J Nucl Med off Publ Soc Nucl Med 60(Suppl 2):29S-37S. https://doi.org/10.2967/jnumed.118.220590
Newman AM, Liu CL, Green MR et al (2015) Robust enumeration of cell subsets from tissue expression profiles. Nat Methods 12(5):453–457. https://doi.org/10.1038/nmeth.3337
Noorbakhsh J, Farahmand S, Foroughi Pour A et al (2020) Deep learning-based cross-classifications reveal conserved spatial behaviors within tumor histological images. Nat Commun 11(1):6367. https://doi.org/10.1038/s41467-020-20030-5
Qu WF, Tian MX, Qiu JT et al (2022) Exploring pathological signatures for predicting the recurrence of early-stage hepatocellular carcinoma based on deep learning. Front Oncol 12:968202. https://doi.org/10.3389/fonc.2022.968202
Quhal F, Pradere B, Sari Motlagh R et al (2021) Prognostic value of preoperative albumin to globulin ratio in patients treated with salvage radical prostatectomy for radiation recurrent prostate cancer. Minerva Urol Nephrol 73(5):610–615. https://doi.org/10.23736/S2724-6051.20.03938-7
Ruf B, Heinrich B, Greten TF (2021) Immunobiology and immunotherapy of HCC: spotlight on innate and innate-like immune cells. Cell Mol Immunol 18(1):112–127. https://doi.org/10.1038/s41423-020-00572-w
Saillard C, Schmauch B, Laifa O et al (2020) Predicting survival after hepatocellular carcinoma resection using deep learning on histological slides. Hepatol Baltim Md 72(6):2000–2013. https://doi.org/10.1002/hep.31207
Sangro B, Sarobe P, Hervás-Stubbs S, Melero I (2021) Advances in immunotherapy for hepatocellular carcinoma. Nat Rev Gastroenterol Hepatol 18(8):525–543. https://doi.org/10.1038/s41575-021-00438-0
Sautès-Fridman C, Petitprez F, Calderaro J, Fridman WH (2019) Tertiary lymphoid structures in the era of cancer immunotherapy. Nat Rev Cancer 19(6):307–325. https://doi.org/10.1038/s41568-019-0144-6
TensorFlow Developers (2022) TensorFlow https://doi.org/10.5281/ZENODO.6574269
Tran KA, Kondrashova O, Bradley A, Williams ED, Pearson JV, Waddell N (2021) Deep learning in cancer diagnosis, prognosis and treatment selection. Genome Med 13(1):152. https://doi.org/10.1186/s13073-021-00968-x
Vogel A, Meyer T, Sapisochin G, Salem R, Saborowski A (2022) Hepatocellular carcinoma. The Lancet 400(10360):1345–1362. https://doi.org/10.1016/S0140-6736(22)01200-4
Xiao Z, Li T, Zheng X et al (2022) Nanodrug enhances post-ablation immunotherapy of hepatocellular carcinoma via promoting dendritic cell maturation and antigen presentation. Bioact Mater 21:57–68. https://doi.org/10.1016/j.bioactmat.2022.07.027
Zhang YL, Li Q, Yang XM et al (2018) SPON2 promotes M1-like macrophage recruitment and inhibits hepatocellular carcinoma metastasis by distinct integrin-rho GTPase-hippo pathways. Cancer Res 78(9):2305–2317. https://doi.org/10.1158/0008-5472.CAN-17-2867
Zheng Y, Jiang Z, Xie F et al (2021) Diagnostic regions attention network (DRA-Net) for histopathology wsi recommendation and retrieval. IEEE Trans Med Imaging 40(3):1090–1103. https://doi.org/10.1109/TMI.2020.3046636
Funding
This study was supported by the Special Support Program for High-level Talents in Shaanxi Province (W. J. Song).
Author information
Authors and Affiliations
Contributions
WJ participated in proposing ideas and designing experimental plans. CS and AF completed the data collection and organization. ZM carefully reviewed the first draft of the article. QY, ZZ, and YW identified the tiles. WShi participated in the discussion. WSong oversaw all aspects of the literature review design and manuscript writing. All the authors contributed to the manuscript and approved the submitted version.
Corresponding authors
Ethics declarations
Conflict of interest
The authors declare that they have no competing interests.
Ethics approval and consent to participate
Human liver biopsy specimens were obtained from the archives of the Hepatobiliary Surgery Biospecimen Bank at Xijing Hospital and remained anonymous. The study protocol was approved by the Ethics Committee for Drug Clinical Trials of the First Affiliated Hospital of the Fourth Military Medical University and complied with the guidelines of the Declaration of Helsinki, approval number: KY20172013-1. All patients have signed an informed consent form, while these patients did not receive any treatment prior to the procedure.
Consent for publication
Not applicable, no personally identifiable information is contained in this manuscript.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Jia, W., Shi, W., Yao, Q. et al. Identifying immune infiltration by deep learning to assess the prognosis of patients with hepatocellular carcinoma. J Cancer Res Clin Oncol 149, 12621–12635 (2023). https://doi.org/10.1007/s00432-023-05097-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00432-023-05097-z