Skip to main content
SpringerLink
Log in

Automated ASPECTS calculation may equal the performance of experienced clinicians: a machine learning study based on a large cohort

  • Computed Tomography
  • Published:
European Radiology Aims and scope Submit manuscript

Abstract

Objectives

The Alberta Stroke Program Early CT Score (ASPECTS) is a semi-quantitative method to evaluate the severity of early ischemic change on non-contrast computed tomography (NCCT) in patients with acute ischemic stroke (AIS). In this work, we propose an automated ASPECTS method based on large cohort of data and machine learning.

Methods

For this study, we collected 3626 NCCT cases from multiple centers and annotated directly on this dataset by neurologists. Based on image analysis and machine learning methods, we constructed a two-stage machine learning model. The validity and reliability of this automated ASPECTS method were tested on an independent external validation set of 300 cases. Statistical analyses on the total ASPECTS, dichotomized ASPECTS, and region-level ASPECTS were presented.

Results

On an independent external validation set of 300 cases, for the total ASPECTS results, the intraclass correlation coefficient between automated ASPECTS and expert-rated was 0.842. The agreement between ASPECTS threshold of ≥ 6 versus < 6 using a dichotomized method was moderate (κ = 0.438, 0.391–0.477), and the detection rate (sensitivity) was 86.5% for patients with ASPECTS threshold of ≥ 6. Compared with the results of previous studies, our method achieved a slight lead in sensitivity (67.8%) and AUC (0.845), with comparable accuracy (78.9%) and specificity (81.2%).

Conclusion

The proposed automated ASPECTS method driven by a large cohort of NCCT images performed equally well compared with expert-rated ASPECTS. This work further demonstrates the validity and reliability of automated ASPECTS evaluation method.

Clinical relevance statement

The automated ASPECTS method proposed by this study may help AIS patients to receive rapid intervention, but should not be used as a stand-alone diagnostic basis.

Key Points

  • NCCT-based manual ASPECTS scores were poorly consistent.

  • Machine learning can automate the ASPECTS scoring process.

  • Machine learning model design based on large cohort data can effectively improve the consistency of ASPECTS scores.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Abbreviations

AIS:

Acute ischemic stroke

ASPECTS:

Alberta Stroke Program Early CT Score

C:

Caudate head

GLCM:

Gray-level co-occurrence matrix

GLDM:

Gray-level dependence matrix

GLRLM:

Gray-level run length matrix

GLSZM:

Gray-level size zone matrix

I:

Insula

IC:

Internal capsule

ICC:

Intraclass correlation coefficient

IQR:

Interquartile range

L:

Lentiform nucleus

LBP:

Local binary pattern

LVO:

Large vessel occlusion

M1:

Frontal operculum

M2:

Temporal lobe

M3:

Posterior temporal lobe

M4:

Anterior MCA

M5:

Lateral MCA

M6:

Posterior MCA

MCA:

Middle cerebral artery

NCCT:

Non-contrast computed tomography

NGTDM:

Neighboring gray-tone difference matrix

SOTA:

State-of-the-art

References

  1. Van der Worp HB, Van Gijn J (2007) Acute ischemic stroke. N Engl J Med 357(6):572–579

    Article  PubMed  Google Scholar 

  2. Xing L, Jing L, Tian Y et al (2020) High prevalence of stroke and uncontrolled associated risk factors are major public health challenges in rural northeast China: a population-based study. Int J Stroke 15(4):399–411

    Article  PubMed  Google Scholar 

  3. Demeestere J, Scheldeman L, Cornelissen SA et al (2018) Alberta stroke program early CT score versus computed tomographic perfusion to predict functional outcome after successful reperfusion in acute ischemic stroke. Stroke 49(10):2361–2367

    Article  PubMed  PubMed Central  Google Scholar 

  4. Barber PA, Demchuk AM, Zhang J, Buchan AM (2000) Validity and reliability of a quantitative computed tomography score in predicting outcome of hyperacute stroke before thrombolytic therapy. Lancet 355(9216):1670–1674

    Article  CAS  PubMed  Google Scholar 

  5. Kim JG, Lee D, Choi JC, Song Y, Lee DH, Suh DC (2019) DWI-pc-ASPECT score in basilar artery occlusion: is 6 points or less always indicative of a bad outcome? Interv Neuroradiol 25(4):371–379

    Article  PubMed  PubMed Central  Google Scholar 

  6. Finlayson O, John V, Yeung R et al (2013) Interobserver agreement of ASPECT score distribution for noncontrast CT, CT angiography, and CT perfusion in acute stroke. Stroke 44(1):234–236

    Article  PubMed  Google Scholar 

  7. Kuang H, Najm M, Chakraborty D et al (2019) Automated ASPECTS on noncontrast CT scans in patients with acute ischemic stroke using machine learning. AJNR Am J Neuroradiol 40(1):33–38

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Maegerlein C, Fischer J, Mönch S et al (2019) Automated calculation of the Alberta Stroke Program Early CT Score: feasibility and reliability. Radiology 291(1):141–148

    Article  PubMed  Google Scholar 

  9. Sundaram VK, Goldstein J, Wheelwright D et al (2019) Automated ASPECTS in acute ischemic stroke: a comparative analysis with CT perfusion. AJNR Am J Neuroradiol 40(12):2033–2038

    CAS  PubMed  PubMed Central  Google Scholar 

  10. Cao Z, Xu J, Song B et al (2022) Deep learning derived automated ASPECTS on non-contrast CT scans of acute ischemic stroke patients. Hum Brain Mapp 43(10):3023–3036

    Article  PubMed  PubMed Central  Google Scholar 

  11. Guberina N, Dietrich U, Radbruch A et al (2018) Detection of early infarction signs with machine learning-based diagnosis by means of the Alberta Stroke Program Early CT Score (ASPECTS) in the clinical routine. Neuroradiology 60:889–901

    Article  PubMed  Google Scholar 

  12. Nagel S, Sinha D, Day D et al (2017) e-ASPECTS software is non-inferior to neuroradiologists in applying the ASPECT score to computed tomography scans of acute ischemic stroke patients. Int J Stroke 12(6):615–622

    Article  PubMed  Google Scholar 

  13. Li L, Chen Y, Bao Y et al (2020) Comparison of the performance between Frontier ASPECTS software and different levels of radiologists on assessing CT examinations of acute ischemic stroke patients. Clin Radiol 75(5):358–365

    Article  CAS  PubMed  Google Scholar 

  14. Qiu W, Kuang H, Teleg E et al (2020) Machine learning for detecting early infarction in acute stroke with non–contrast-enhanced CT. Radiology 294(3):638–644

    Article  PubMed  Google Scholar 

  15. Herweh C, Ringleb PA, Rauch G et al (2016) Performance of e-ASPECTS software in comparison to that of stroke physicians on assessing CT scans of acute ischemic stroke patients. Int J Stroke 11(4):438–445

    Article  PubMed  Google Scholar 

  16. Brinjikji W, Abbasi M, Arnold C et al (2021) e-ASPECTS software improves interobserver agreement and accuracy of interpretation of aspects score. Interv Neuroradiol 27(6):781–787

    Article  PubMed  PubMed Central  Google Scholar 

  17. Adamou A, Beltsios ET, Bania A et al (2022) Artificial intelligence-driven ASPECTS for the detection of early stroke changes in non-contrast CT: a systematic review and meta-analysis. J Neurointerv Surg 0:1–7

  18. Hoopes A, Mora JS, Dalca AV, Fischl B, Hoffmann M (2022) SynthStrip: skull-stripping for any brain image. Neuroimage 260:119474

    Article  PubMed  Google Scholar 

  19. Dong X, Yu Z, Cao W, Shi Y, Ma Q (2020) A survey on ensemble learning. Front Comput Sci 14:241–258

    Article  Google Scholar 

  20. Chen T, Guestrin C (2016) XGBoost: a scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining 2016: 785-794

  21. Wolff L, Berkhemer OA, Van Es AC et al (2021) Validation of automated Alberta Stroke Program Early CT Score (ASPECTS) software for detection of early ischemic changes on non-contrast brain CT scans. Neuroradiology 63:491–498

    Article  PubMed  Google Scholar 

  22. Chen W, Wu J, Wei R et al (2022) Improving the diagnosis of acute ischemic stroke on non-contrast CT using deep learning: a multicenter study. Insights Imaging 13(1):1–12

    Article  Google Scholar 

  23. Hoelter P, Muehlen I, Goelitz P, Beuscher V, Schwab S, Doerfler A (2020) Automated ASPECT scoring in acute ischemic stroke: comparison of three software tools. Neuroradiology 62:1231–1238

    Article  PubMed  Google Scholar 

  24. Delio PR, Wong ML, Tsai JP et al (2021) Assistance from automated ASPECTS software improves reader performance. J Stroke Cerebrovasc Dis 30(7):105829

    Article  PubMed  Google Scholar 

  25. Albers GW, Wald MJ, Mlynash M et al (2019) Automated calculation of Alberta Stroke Program Early CT Score: validation in patients with large hemispheric infarct. Stroke 50(11):3277–3279

    Article  PubMed  Google Scholar 

  26. Kuang H, Menon BK, Sohn SI, Qiu W (2021) EIS-Net: segmenting early infarct and scoring ASPECTS simultaneously on non-contrast CT of patients with acute ischemic stroke. Med Image Anal 70:101984

    Article  PubMed  Google Scholar 

  27. Aljabar P, Heckemann RA, Hammers A, Hajnal JV, Rueckert D (2009) Multi-atlas based segmentation of brain images: atlas selection and its effect on accuracy. Neuroimage 46(3):726–738

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to acknowledge the contribution of neurologists Dr. Shu Wan and Dr. Sheng Guan for their valuable assistance with the clinical interpretation. We also would like to thank the following colleagues: Wei Lu, Yu Fu, Ming Wang, Kaizheng Liu, Sijing Chen, Wubiao Chen, Yang Wang, Jun Wu, Xiaochang Leng, Jens Fiehler, Adnan H. Siddiqui, and Jianping Xiang. Special thanks to ArteryFlow Technology Co., Ltd., for the technical support of this research.

Funding

This work was supported by the Key Research and Development Project of Zhejiang Provincial Department of Science and Technology (Grant No. 2021C03105), Medical Health Science and Technology Project of Zhejiang Provincial Health Commission (Grant No. WKJ-ZJ-2014), and Hangzhou Leading Innovation and Entrepreneurship Team Project (TD2022007).

Author information

Author notes

  1. Shu Wan and Wei Lu have contributed equally to this work and share first authorship.

Authors and Affiliations

  1. Brain Center, Affiliated Zhejiang Hospital, Zhejiang University School of Medicine, Hangzhou, China

    Shu Wan & Ming Wang

  2. ArteryFlow Technology Co., Ltd., Hangzhou, China

    Wei Lu, Kaizheng Liu, Xiaochang Leng & Jianping Xiang

  3. Department of Neurointervention Radiology, The First Affiliated Hospital of Zhengzhou University, Zhengzhou, China

    Yu Fu & Sheng Guan

  4. Department of Radiology, Affiliated Hospital of Guangdong Medical University, Zhanjiang, China

    Sijing Chen & Wubiao Chen

  5. Department of Neurosurgery, Beijing Chao-Yang Hospital, Capital Medical University, Beijing, China

    Yang Wang

  6. Department of Neurology, Qingtian County People’s Hospital, Lishui, China

    Jun Wu

  7. Department of Diagnostic and Interventional Neuroradiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany

    Jens Fiehler

  8. Departments of Neurosurgery and Radiology, University at Buffalo, Buffalo, NY, USA

    Adnan H. Siddiqui

Authors
  1. Shu Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yu Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ming Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kaizheng Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sijing Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Wubiao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jun Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Xiaochang Leng
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Jens Fiehler
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Adnan H. Siddiqui
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Sheng Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Jianping Xiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Sheng Guan or Jianping Xiang.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Jianping Xiang.

Conflict of interest

The authors of this manuscript declare relationships with the following companies: ArteryFlow Technology Co., Ltd. The authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.

Statistics and biometry

No complex statistical methods were necessary for this paper. Some simple descriptive statistics and graphs were used in the study.

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethical approval

Institutional Review Board approval was obtained.

Study subjects or cohorts overlap

All of the study subjects or cohorts have not been previously reported.

Methodology

  • retrospective

  • multicenter study

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wan, S., Lu, W., Fu, Y. et al. Automated ASPECTS calculation may equal the performance of experienced clinicians: a machine learning study based on a large cohort. Eur Radiol 34, 1624–1634 (2024). https://doi.org/10.1007/s00330-023-10053-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-023-10053-z

Keywords

Search

Navigation