Skip to main content

Advertisement

SpringerLink
Log in

Accurately predicting the risk of unfavorable outcomes after endovascular coil therapy in patients with aneurysmal subarachnoid hemorrhage: an interpretable machine learning model

  • Original Article
  • Published:
Neurological Sciences Aims and scope Submit manuscript

Abstract

Background

Despite endovascular coiling as a valid modality in treatment of aneurysmal subarachnoid hemorrhage (aSAH), there is a risk of poor prognosis. However, the clinical utility of previously proposed early prediction tools remains limited. We aimed to develop a clinically generalizable machine learning (ML) models for accurately predicting unfavorable outcomes in aSAH patients after endovascular coiling.

Methods

Functional outcomes at 6 months after endovascular coiling were assessed via the modified Rankin Scale (mRS) and unfavorable outcomes were defined as mRS 3-6. Five ML algorithms (logistic regression, random forest, support vector machine, deep neural network, and extreme gradient boosting) were used for model development. The area under precision-recall curve (AUPRC) and receiver operating characteristic curve (AUROC) was used as main indices of model evaluation. SHapley Additive exPlanations (SHAP) method was applied to interpret the best-performing ML model.

Results

A total of 371 patients were eventually included into this study, and 85.4% of them had favorable outcomes. Among the five models, the DNN model had a better performance with AUPRC of 0.645 (AUROC of 0.905). Postoperative GCS score, size of aneurysm, and age were the top three powerful predictors. The further analysis of five random cases presented the good interpretability of the DNN model.

Conclusion

Interpretable clinical prediction models based on different ML algorithms have been successfully constructed and validated, which would serve as reliable tools in optimizing the treatment decision-making of aSAH. Our DNN model had better performance to predict the unfavorable outcomes at 6 months in aSAH patients compared with Yan’s nomogram model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The datasets generated and analyzed during the current study are not publicly available due protection of patients’ personal information but are available from the corresponding author on reasonable request.

Abbreviations

aSAH :

aneurysmal subarachnoid hemorrhage

AUPRC :

area under precision-recall curve

AUROC :

area under receiver operating characteristic curve

DNN :

deep neural network

FGB :

fasting blood glucose

GCS :

Glasgow Coma Scale

HbA1c :

glycated hemoglobin

LASSO :

least absolute shrinkage and selection operator

LR :

logistic regression

ML :

machine learning

NPV :

negative predictive value

mRS :

modified Rankin Scale

PPV :

positive predictive value

RF :

random forest

SBP :

systolic blood pressure

SHAP :

SHapley Additive exPlanations

SVM :

support vector machine

XGBoost :

extreme gradient boost

References

  1. Etminan N, Chang H-S, Hackenberg K et al (2019) Worldwide incidence of aneurysmal subarachnoid hemorrhage according to region, time period, blood pressure, and smoking prevalence in the population: a systematic review and meta-analysis. JAMA Neurol 76:588–597. https://doi.org/10.1001/jamaneurol.2019.0006

    Article  PubMed  PubMed Central  Google Scholar 

  2. Gawlitza M, Soize S, Barbe C et al (2019) Aneurysm characteristics, study population, and endovascular techniques for the treatment of intracranial aneurysms in a large, prospective, multicenter cohort: results of the analysis of recanalization after endovascular treatment of intracranial aneurysm study. AJNR Am J Neuroradiol 40:517–523. https://doi.org/10.3174/ajnr.A5991

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Pierot L, Wakhloo AK (2013) Endovascular treatment of intracranial aneurysms: current status. Stroke 44:2046–2054. https://doi.org/10.1161/STROKEAHA.113.000733

    Article  PubMed  Google Scholar 

  4. Campos JK, Lien BV, Wang AS, Lin L-M (2020) Advances in endovascular aneurysm management: coiling and adjunctive devices. Stroke Vasc Neurol 5:14–21. https://doi.org/10.1136/svn-2019-000303

    Article  PubMed  PubMed Central  Google Scholar 

  5. Molyneux A, Kerr R, Stratton I et al (2002) International Subarachnoid Aneurysm Trial (ISAT) of neurosurgical clipping versus endovascular coiling in 2143 patients with ruptured intracranial aneurysms: a randomised trial. Lancet 360:1267–1274. https://doi.org/10.1016/s0140-6736(02)11314-6

    Article  PubMed  Google Scholar 

  6. Jaja B, Saposnik G, Lingsma HF et al (2018) Development and validation of outcome prediction models for aneurysmal subarachnoid haemorrhage: the SAHIT multinational cohort study. BMJ 360:j5745

    Article  PubMed  Google Scholar 

  7. Naval NS, Kowalski RG, Chang TR et al (2014) The SAH score: a comprehensive communication tool. J Stroke Cerebrovasc Dis 23:902–909. https://doi.org/10.1016/j.jstrokecerebrovasdis.2013.07.035

    Article  PubMed  Google Scholar 

  8. van Donkelaar CE, Bakker NA, Birks J et al (2019) Prediction of outcome after aneurysmal subarachnoid hemorrhage. Stroke 50:837–844. https://doi.org/10.1161/STROKEAHA.118.023902

    Article  PubMed  Google Scholar 

  9. Yan Y, Hu J, Fang X et al (2021) Predicting the poor recovery risk of aneurysmal subarachnoid hemorrhage: clinical evaluation and management based on a new predictive nomogram. Clin Neurol Neurosurg 200:106302. https://doi.org/10.1016/j.clineuro.2020.106302

    Article  PubMed  Google Scholar 

  10. Connolly ES, Rabinstein AA, Carhuapoma JR et al (2012) Guidelines for the management of aneurysmal subarachnoid hemorrhage: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke 43:1711–1737. https://doi.org/10.1161/STR.0b013e3182587839

    Article  PubMed  Google Scholar 

  11. Stienen MN, Germans M, Burkhardt J-K et al (2018) Predictors of in-hospital death after aneurysmal subarachnoid hemorrhage: analysis of a nationwide database (Swiss SOS [Swiss Study on Aneurysmal Subarachnoid Hemorrhage]). Stroke 49:333–340. https://doi.org/10.1161/STROKEAHA.117.019328

    Article  PubMed  Google Scholar 

  12. van Niftrik CHB, van der Wouden F, Staartjes VE et al (2019) Machine learning algorithm identifies patients at high risk for early complications after intracranial tumor surgery: registry-based cohort study. Neurosurgery 85:E756–E764. https://doi.org/10.1093/neuros/nyz145

    Article  PubMed  Google Scholar 

  13. Senders JT, Arnaout O, Karhade AV et al (2018) Natural and artificial intelligence in neurosurgery: a systematic review. Neurosurgery 83:181–192. https://doi.org/10.1093/neuros/nyx384

    Article  PubMed  Google Scholar 

  14. Park A, Chute C, Rajpurkar P et al (2019) Deep learning-assisted diagnosis of cerebral aneurysms using the HeadXNet model. JAMA Netw Open 2:e195600. https://doi.org/10.1001/jamanetworkopen.2019.5600

    Article  PubMed  PubMed Central  Google Scholar 

  15. Tanioka S, Ishida F, Nakano F et al (2019) Machine learning analysis of matricellular proteins and clinical variables for early prediction of delayed cerebral ischemia after aneurysmal subarachnoid hemorrhage. Mol Neurobiol 56:7128–7135. https://doi.org/10.1007/s12035-019-1601-7

    Article  CAS  PubMed  Google Scholar 

  16. Diringer MN, Bleck TP, Claude Hemphill J et al (2011) Critical care management of patients following aneurysmal subarachnoid hemorrhage: recommendations from the Neurocritical Care Society’s Multidisciplinary Consensus Conference. Neurocrit Care 15:211–240. https://doi.org/10.1007/s12028-011-9605-9

    Article  PubMed  Google Scholar 

  17. Horan TC, Andrus M, Dudeck MA (2008) CDC/NHSN surveillance definition of health care-associated infection and criteria for specific types of infections in the acute care setting. Am J Infect Control 36:309–332. https://doi.org/10.1016/j.ajic.2008.03.002

    Article  PubMed  Google Scholar 

  18. Teasdale G, Jennett B (1974) Assessment of coma and impaired consciousness. A practical scale. Lancet 2:81–84. https://doi.org/10.1016/s0140-6736(74)91639-0

    Article  CAS  PubMed  Google Scholar 

  19. Brennan PM, Murray GD, Teasdale GM (2020) A practical method for dealing with missing Glasgow Coma Scale verbal component scores. J Neurosurg:1–6. https://doi.org/10.3171/2020.6.JNS20992

  20. Li X, Pan X, Jiang C et al (2020) Predicting 6-month unfavorable outcome of acute ischemic stroke using machine learning. Front Neurol 11:539509. https://doi.org/10.3389/fneur.2020.539509

    Article  PubMed  PubMed Central  Google Scholar 

  21. Vasquez MM, Hu C, Roe DJ et al (2016) Least absolute shrinkage and selection operator type methods for the identification of serum biomarkers of overweight and obesity: simulation and application. BMC Med Res Methodol 16:154. https://doi.org/10.1186/s12874-016-0254-8

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Shalabi LA, Shaaban Z, Kasasbeh B (2006) Data mining: a preprocessing engine. J Comput Sci 2:735–739. https://doi.org/10.3844/jcssp.2006.735.739

    Article  Google Scholar 

  23. Okada S, Ohzeki M, Taguchi S (2019) Efficient partition of integer optimization problems with one-hot encoding. Sci Rep 9:13036. https://doi.org/10.1038/s41598-019-49539-6

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Davis J, Goadrich M (2006) The relationship between Precision-Recall and ROC curves. In: Proceedings of the 23rd international conference on Machine learning - ICML ’06. ACM Press, Pittsburgh, Pennsylvania, pp 233–240

    Google Scholar 

  25. Saito T, Rehmsmeier M (2015) The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS One 10:e0118432. https://doi.org/10.1371/journal.pone.0118432

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845

    Article  CAS  PubMed  Google Scholar 

  27. Lundberg SM, Lee S–I (2017) A unified approach to interpreting model predictions. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, CA, USA

  28. Challen R, Denny J, Pitt M et al (2019) Artificial intelligence, bias and clinical safety. BMJ Qual Saf 28:231–237. https://doi.org/10.1136/bmjqs-2018-008370

    Article  PubMed  PubMed Central  Google Scholar 

  29. Ribeiro MT, Singh S, Guestrin C (2016) “Why should i trust you?”: explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, New York, USA, pp 1135–1144

    Chapter  Google Scholar 

  30. Shrikumar A, Greenside P, Shcherbina A, Kundaje A (2016) Not just a black box: learning important features through propagating activation differences. https://doi.org/10.48550/arXiv.1605.01713

  31. Ayling OGS, Ibrahim GM, Drake B et al (2015) Operative complications and differences in outcome after clipping and coiling of ruptured intracranial aneurysms. J Neurosurg 123:621–628. https://doi.org/10.3171/2014.11.JNS141607

    Article  PubMed  Google Scholar 

  32. Wartenberg KE, Schmidt JM, Claassen J et al (2006) Impact of medical complications on outcome after subarachnoid hemorrhage. Crit Care Med 34:617–623; quiz 624. https://doi.org/10.1097/01.ccm.0000201903.46435.35

    Article  PubMed  Google Scholar 

  33. Jabbarli R, Reinhard M, Roelz R et al (2016) The predictors and clinical impact of intraventricular hemorrhage in patients with aneurysmal subarachnoid hemorrhage. Int J Stroke 11:68–76. https://doi.org/10.1177/1747493015607518

    Article  PubMed  Google Scholar 

  34. Zheng K, Zhong M, Zhao B et al (2019) Poor-grade aneurysmal subarachnoid hemorrhage: risk factors affecting clinical outcomes in intracranial aneurysm patients in a multi-center study. Front Neurol 10:123. https://doi.org/10.3389/fneur.2019.00123

    Article  PubMed  PubMed Central  Google Scholar 

  35. Ryttlefors M, Enblad P, Kerr RSC, Molyneux AJ (2008) International subarachnoid aneurysm trial of neurosurgical clipping versus endovascular coiling: subgroup analysis of 278 elderly patients. Stroke 39:2720–2726. https://doi.org/10.1161/STROKEAHA.107.506030

    Article  PubMed  Google Scholar 

  36. Bjerkne Wenneberg S, Odenstedt Hergès H, Svedin P et al (2021) Association between inflammatory response and outcome after subarachnoid haemorrhage. Acta Neurol Scand 143:195–205. https://doi.org/10.1111/ane.13353

    Article  CAS  PubMed  Google Scholar 

  37. Vergouwen MDI, Ilodigwe D, Macdonald RL (2011) Cerebral infarction after subarachnoid hemorrhage contributes to poor outcome by vasospasm-dependent and -independent effects. Stroke 42:924–929. https://doi.org/10.1161/STROKEAHA.110.597914

    Article  PubMed  Google Scholar 

  38. Galea JP, Dulhanty L, Patel HC, UK and Ireland Subarachnoid Hemorrhage Database Collaborators (2017) Predictors of outcome in aneurysmal subarachnoid hemorrhage patients: observations from a multicenter data set. Stroke 48:2958–2963. https://doi.org/10.1161/STROKEAHA.117.017777

  39. Abla AA, Wilson DA, Williamson RW et al (2014) The relationship between ruptured aneurysm location, subarachnoid hemorrhage clot thickness, and incidence of radiographic or symptomatic vasospasm in patients enrolled in a prospective randomized controlled trial. J Neurosurg 120:391–397. https://doi.org/10.3171/2013.10.JNS13419

    Article  PubMed  Google Scholar 

  40. Pierot L, Cognard C, Anxionnat R et al (2010) Ruptured intracranial aneurysms: factors affecting the rate and outcome of endovascular treatment complications in a series of 782 patients (CLARITY study). Radiology 256:916–923. https://doi.org/10.1148/radiol.10092209

    Article  PubMed  Google Scholar 

  41. Spetzler RF, McDougall CG, Zabramski JM et al (2015) The barrow ruptured aneurysm trial: 6-year results. J Neurosurg 123:609–617. https://doi.org/10.3171/2014.9.JNS141749

    Article  PubMed  Google Scholar 

  42. Fisher CM, Kistler JP, Davis JM (1980) Relation of cerebral vasospasm to subarachnoid hemorrhage visualized by computerized tomographic scanning. Neurosurgery 6:1–9. https://doi.org/10.1227/00006123-198001000-00001

    Article  CAS  PubMed  Google Scholar 

Download references

Funding

This study was supported by the National Natural Science Foundation of China (82173899), Jiangsu Pharmaceutical Association (JY202207, Q202202, A2021024, H202108), Hunan Natural Science Foundation (2021JJ70021), and Hunan innovation guidance grant of clinical medical technology (2020SK50920).

Author information

Author notes

  1. Zhou Zhou, Anran Dai, Yuqing Yan, and Yuzhan Jin have contributed equally to this work and share the first authorship.

Authors and Affiliations

  1. Department of Pharmacy, Nanjing First Hospital, Nanjing Medical University, Nanjing, China

    Zhou Zhou, Anran Dai, Yuqing Yan, Yuzhan Jin, DaiZun Zou & JianJun Zou

  2. School of Basic Medicine and Clinical Pharmacy, China Pharmaceutical University, Nanjing, China

    Anran Dai, Yuqing Yan, Yuzhan Jin & DaiZun Zou

  3. Office of Clinical Trials, Nanjing First Hospital, Nanjing Medical University, Nanjing, China

    XiaoWen Xu

  4. Department of Neurology, The First Affiliated Hospital (People’s Hospital of Hunan Province), Hunan Normal University, Changsha, China

    Lan Xiang, LeHeng Guo, Liang Xiang & ZhiHong Zhao

  5. Department of Geriatrics, Nanjing First Hospital, Nanjing Medical University, Nanjing, China

    FuPing Jiang

Authors
  1. Zhou Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anran Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuqing Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuzhan Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. DaiZun Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. XiaoWen Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Lan Xiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. LeHeng Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Liang Xiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. FuPing Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. ZhiHong Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. JianJun Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

ZZ and DAR conceived and designed the study. YYQ and JYZ provided a review of the previous literature. XWX, LX, and LHG were responsible for data collection. JYZ, YYQ, and DZZ carried out the statistical analyses and interpreted the results. ZZ and DAR drafted the manuscript. ZZ, DAR, YYQ, JYZ, JFP, ZZH, and JJZ participated in the discussion of the study. LX, DZZ, and XWX polished this manuscript, including grammatical checks. All authors contributed to manuscript revision, read, and approved the submitted version.

Corresponding authors

Correspondence to FuPing Jiang, ZhiHong Zhao or JianJun Zou.

Ethics declarations

Ethical approval and Informed consent

The studies involving human participants were reviewed and approved by the Human Research Ethics Committee of Hunan Provincial People’s Hospital (article number: [2015]-10). The Ethics Committee waived the requirement of written informed consent for participation.

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

ESM 1

(DOCX 182 kb)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, ., Dai, A., Yan, Y. et al. Accurately predicting the risk of unfavorable outcomes after endovascular coil therapy in patients with aneurysmal subarachnoid hemorrhage: an interpretable machine learning model. Neurol Sci 45, 679–691 (2024). https://doi.org/10.1007/s10072-023-07003-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10072-023-07003-4

Keywords

Search

Navigation