Skip to main content
SpringerLink
Log in

Non-contrast CT markers of intracerebral hematoma expansion: a reliability study

  • Neuro
  • Published:
European Radiology Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

Objectives

We evaluated whether clinicians agree in the detection of non-contrast CT markers of intracerebral hemorrhage (ICH) expansion.

Methods

From our local dataset, we randomly sampled 60 patients diagnosed with spontaneous ICH. Fifteen physicians and trainees (Stroke Neurology, Interventional and Diagnostic Neuroradiology) were trained to identify six density (Barras density, black hole, blend, hypodensity, fluid level, swirl) and three shape (Barras shape, island, satellite) expansion markers, using standardized definitions. Thirteen raters performed a second assessment. Inter- and intra-rater agreement were measured using Gwet’s AC1, with a coefficient > 0.60 indicating substantial to almost perfect agreement.

Results

Almost perfect inter-rater agreement was observed for the swirl (0.85, 95% CI: 0.78–0.90) and fluid level (0.84, 95% CI: 0.76–0.90) markers, while the hypodensity (0.67, 95% CI: 0.56–0.76) and blend (0.62, 95% CI: 0.51–0.71) markers showed substantial agreement. Inter-rater agreement was otherwise moderate, and comparable between density and shape markers. Inter-rater agreement was lower for the three markers that require the rater to identify one specific axial slice (Barras density, Barras shape, island: 0.46, 95% CI: 0.40–0.52 versus others: 0.60, 95% CI: 0.56–0.63). Inter-observer agreement did not differ when stratified for raters’ experience, hematoma location, volume, or anticoagulation status. Intra-rater agreement was substantial to almost perfect for all but the black hole marker.

Conclusion

In a large sample of raters with different backgrounds and expertise levels, only four of nine non-contrast CT markers of ICH expansion showed substantial to almost perfect inter-rater agreement.

Key Points

• In a sample of 15 raters and 60 patients, only four of nine non-contrast CT markers of ICH expansion showed substantial to almost perfect inter-rater agreement (Gwet’s AC1> 0.60).

• Intra-rater agreement was substantial to almost perfect for eight of nine hematoma expansion markers.

• Only the blend, fluid level, and swirl markers achieved substantial to almost perfect agreement across all three measures of reliability (inter-rater agreement, intra-rater agreement, agreement with the results of a reference reading).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Abbreviations

CI:

Confidence interval

CTA:

CT angiography

EM:

Expansion marker

HE:

Hematoma expansion

ICH:

Intracerebral hemorrhage

IQR:

Interquartile range

NCCT:

Non-contrast computed tomography

SD:

Standard deviation

References

  1. Al-Shahi Salman R, Frantzias J, Lee RJ et al (2018) Absolute risk and predictors of the growth of acute spontaneous intracerebral haemorrhage: a systematic review and meta-analysis of individual patient data. Lancet Neurol 17:885–894

    Article  PubMed  PubMed Central  Google Scholar 

  2. Wang X, Arima H, Al-Shahi Salman R et al (2015) Clinical prediction algorithm (BRAIN) to determine risk of hematoma growth in acute intracerebral hemorrhage. Stroke 46:376–381

    Article  CAS  PubMed  Google Scholar 

  3. Qureshi AI, Palesch YY, Barsan WG et al (2016) Intensive blood-pressure lowering in patients with acute cerebral hemorrhage. N Engl J Med 375:1033–1043

    Article  PubMed  PubMed Central  Google Scholar 

  4. Brott T, Broderick J, Kothari R et al (1997) Early hemorrhage growth in patients with intracerebral hemorrhage. Stroke 28:1–5

    Article  CAS  PubMed  Google Scholar 

  5. Dowlatshahi D, Demchuk AM, Flaherty ML, Ali M, Lyden PL, Smith EE (2011) Defining hematoma expansion in intracerebral hemorrhage: relationship with patient outcomes. Neurology 76:1238–1244

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Davis SM, Broderick J, Hennerici M et al (2006) Hematoma growth is a determinant of mortality and poor outcome after intracerebral hemorrhage. Neurology 66:1175–1181

    Article  CAS  PubMed  Google Scholar 

  7. Wada R, Aviv RI, Fox AJ et al (2007) CT angiography “spot sign” predicts hematoma expansion in acute intracerebral hemorrhage. Stroke 38:1257–1262

    Article  PubMed  Google Scholar 

  8. Demchuk AM, Dowlatshahi D, Rodriguez-Luna D et al (2012) Prediction of haematoma growth and outcome in patients with intracerebral haemorrhage using the CT-angiography spot sign (PREDICT): a prospective observational study. Lancet Neurol 11:307–314

    Article  PubMed  Google Scholar 

  9. Sprigg N, Flaherty K, Appleton JP et al (2018) Tranexamic acid for hyperacute primary IntraCerebral Haemorrhage (TICH-2): an international randomised, placebo-controlled, phase 3 superiority trial. Lancet 391:2107–2115

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Barras CD, Tress BM, Christensen S et al (2009) Density and shape as CT predictors of intracerebral hemorrhage growth. Stroke 40:1325–1331

    Article  PubMed  Google Scholar 

  11. Li Q, Liu QJ, Yang WS et al (2017) Island sign: an imaging predictor for early hematoma expansion and poor outcome in patients with intracerebral hemorrhage. Stroke 48:3019–3025

    Article  PubMed  Google Scholar 

  12. Li Q, Zhang G, Huang YJ et al (2015) Blend sign on computed tomography: novel and reliable predictor for early hematoma growth in patients with intracerebral hemorrhage. Stroke 46:2119–2123

    Article  PubMed  Google Scholar 

  13. Li Q, Zhang G, Xiong X et al (2016) Black hole sign: novel imaging marker that predicts hematoma growth in patients with intracerebral hemorrhage. Stroke 47:1777–1781

    Article  PubMed  Google Scholar 

  14. Selariu E, Zia E, Brizzi M, Abul-Kasim K (2012) Swirl sign in intracerebral haemorrhage: definition, prevalence, reliability and prognostic value. BMC Neurol 12:109

    Article  PubMed  PubMed Central  Google Scholar 

  15. Yu Z, Zheng J, Ali H et al (2017) Significance of satellite sign and spot sign in predicting hematoma expansion in spontaneous intracerebral hemorrhage. Clin Neurol Neurosurg 162:67–71

    Article  PubMed  Google Scholar 

  16. Boulouis G, Morotti A, Brouwers HB et al (2016) Association between hypodensities detected by computed tomography and hematoma expansion in patients with intracerebral hemorrhage. JAMA Neurol 73:961–968

    Article  PubMed  PubMed Central  Google Scholar 

  17. Blacquiere D, Demchuk AM, Al-Hazzaa M et al (2015) Intracerebral hematoma morphologic appearance on noncontrast computed tomography predicts significant hematoma expansion. Stroke 46:3111–3116

    Article  CAS  PubMed  Google Scholar 

  18. Boulouis G, Morotti A, Charidimou A, Dowlatshahi D, Goldstein JN (2017) Noncontrast computed tomography markers of intracerebral hemorrhage expansion. Stroke 48:1120–1125

    Article  PubMed  PubMed Central  Google Scholar 

  19. Fisher CM (1971) Pathological observations in hypertensive cerebral hemorrhage. J Neuropathol Exp Neurol 30:536–550

    Article  CAS  PubMed  Google Scholar 

  20. Arba F, Rinaldi C, Boulouis G, Fainardi E, Charidimou A, Morotti A (2021) Noncontrast computed tomography markers of cerebral hemorrhage expansion: diagnostic accuracy meta-analysis. Int J Stroke. https://doi.org/10.1177/17474930211061639

  21. Morotti A, Boulouis G, Dowlatshahi D et al (2019) Standards for detecting, interpreting, and reporting noncontrast computed tomographic markers of intracerebral hemorrhage expansion. Ann Neurol 86:480–492

    Article  PubMed  Google Scholar 

  22. Nawabi J, Elsayed S, Kniep H et al (2020) Inter- and intrarater agreement of spot sign and noncontrast CT markers for early intracerebral hemorrhage expansion. J Clin Med. https://doi.org/10.3390/jcm9041020

  23. Dowlatshahi D, Morotti A, Al-Ajlan FS et al (2019) Interrater and intrarater measurement reliability of noncontrast computed tomography predictors of intracerebral hemorrhage expansion. Stroke 50:1260–1262

    Article  PubMed  Google Scholar 

  24. Kottner J, Audigé L, Brorson S et al (2011) Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed. J Clin Epidemiol 64:96–106

    Article  PubMed  Google Scholar 

  25. Fedorov A, Beichel R, Kalpathy-Cramer J et al (2012) 3D Slicer as an image computing platform for the Quantitative Imaging Network. Magn Reson Imaging 30:1323–1341

    Article  PubMed  PubMed Central  Google Scholar 

  26. Pasi M, Marini S, Morotti A et al (2018) Cerebellar hematoma location: implications for the underlying microangiopathy. Stroke 49:207–210

    Article  PubMed  Google Scholar 

  27. Donner A, Rotondi MA (2010) Sample size requirements for interval estimation of the kappa statistic for interobserver agreement studies with a binary outcome and multiple raters. Int J Biostat. https://doi.org/10.2202/1557-4679.1275

  28. Rotondi MA (2016) kappaSize: sample size estimation functions for studies of interobserver agreement, R package version 1.2. Available via: https://CRAN.R-project.org/package=kappaSize. Accessed 1 April 2021

  29. Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33:159–174

    Article  CAS  PubMed  Google Scholar 

  30. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG (2009) Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 42:377–381

    Article  PubMed  Google Scholar 

  31. Harris PA, Taylor R, Minor BL et al (2019) The REDCap consortium: building an international community of software platform partners. J Biomed Inform 95:103208

    Article  PubMed  PubMed Central  Google Scholar 

  32. R Development Core Team (2021) R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing

  33. Gamer M, Lemon J, Fellows I, Singh P (2012) irr: various coefficients of interrater reliability and agreement. R package version 0.84.1 2012. Available via: https://CRAN.R-project.org/package=irr. Accessed 1 April 2021

  34. Gwet KL (2019) irrCAC: computing chance-corrected agreement coefficients (CAC). R package version 1.0 2019. Available via: https://CRAN.R-project.org/package=irrCAC. Accessed 1 April 2021

  35. Feinstein AR, Cicchetti DV (1990) High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol 43:543–549

    Article  CAS  PubMed  Google Scholar 

  36. Cicchetti DV, Feinstein AR (1990) High agreement but low kappa: II. Resolving the paradoxes. J Clin Epidemiol 43:551–558

    Article  CAS  PubMed  Google Scholar 

  37. Gwet KL (2008) Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol 61:29–48

    Article  PubMed  Google Scholar 

  38. Gwet KL (2014) Handbook of inter-rater reliability, 4th edition. Advanced Analytics, LLC, United States of America

  39. Varmdal T, Ellekjær H, Fjærtoft H, Indredavik B, Lydersen S, Bonaa KH (2015) Inter-rater reliability of a national acute stroke register. BMC Res Notes 8:584

    Article  PubMed  PubMed Central  Google Scholar 

  40. Hemphill JC 3rd, Bonovich DC, Besmertis L, Manley GT, Johnston SC (2001) The ICH score: a simple, reliable grading scale for intracerebral hemorrhage. Stroke 32:891–897

    Article  PubMed  Google Scholar 

  41. Morotti A, Arba F, Boulouis G, Charidimou A (2020) Noncontrast CT markers of intracerebral hemorrhage expansion and poor outcome: a meta-analysis. Neurology 95:632–643

    Article  PubMed  Google Scholar 

  42. Yang H, Luo Y, Chen S et al (2020) The predictive accuracy of satellite sign for hematoma expansion in intracerebral hemorrhage: a meta-analysis. Clin Neurol Neurosurg 197:106139

    Article  PubMed  Google Scholar 

  43. Morotti A, Dowlatshahi D, Boulouis G et al (2018) Predicting intracerebral hemorrhage expansion with noncontrast computed tomography: the BAT score. Stroke 49:1163–1169

    Article  PubMed  PubMed Central  Google Scholar 

  44. Yogendrakumar V, Moores M, Sikora L et al (2020) Evaluating hematoma expansion scores in acute spontaneous intracerebral hemorrhage: a systematic scoping review. Stroke 51:1305–1308

    Article  PubMed  Google Scholar 

  45. Morotti A, Boulouis G, Charidimou A et al (2018) Integration of computed tomographic angiography spot sign and noncontrast computed tomographic hypodensities to predict hematoma expansion. Stroke 49:2067–2073

    Article  PubMed  PubMed Central  Google Scholar 

  46. Law ZK, Ali A, Krishnan K et al (2020) Noncontrast computed tomography signs as predictors of hematoma expansion, clinical outcome, and response to tranexamic acid in acute intracerebral hemorrhage. Stroke 51:121–128

    Article  PubMed  Google Scholar 

  47. Falcone GJ, Biffi A, Brouwers HB et al (2013) Predictors of hematoma volume in deep and lobar supratentorial intracerebral hemorrhage. JAMA Neurol 70:988–994

    Article  PubMed  Google Scholar 

  48. Kim YS, Chae HY, Jeong HG et al (2021) Size-related differences in computed tomography markers of hematoma expansion in acute intracerebral hemorrhage. Neurocrit Care. https://doi.org/10.1007/s12028-021-01347-5

  49. Seiffge DJ, Goeldlin MB, Tatlisumak T et al (2019) Meta-analysis of haematoma volume, haematoma expansion and mortality in intracerebral haemorrhage associated with oral anticoagulant use. J Neurol 266:3126–3135

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  50. Zimmer S, Meier J, Minnerup J et al (2020) Prognostic value of non-contrast CT markers and spot sign for outcome prediction in patients with intracerebral hemorrhage under oral anticoagulation. J Clin Med. https://doi.org/10.3390/jcm9041077

  51. Quarfoot D, Levine RA (2016) How robust are multirater interrater reliability indices to changes in frequency distribution? Am Stat 70:373–384

    Article  Google Scholar 

Download references

Funding

LLG is funded through the following grants: Radiological Society of North America Seed Grant, Fonds de Recherche en Santé du Québec/Fondation de l’association des Radiologistes du Québec – Recherches en radiologie, Programme de support professoral du Département de radiologie, radio-oncologie et médecine nucléaire de l’Université de Montréal.

Author information

Author notes

  1. Laura C. Gioia and Laurent Létourneau-Guillon contributed equally to this work.

Authors and Affiliations

  1. Neurovascular Health Program, Centre Hospitalier de l’Université de Montréal (CHUM), Montréal, Québec, Canada

    Ahmad Nehme, Célina Ducroux, Marie-Andrée Panzini, Olena Bereznyakova, Yan Deschaintre, Grégory Jacquin, Alexandre Y. Poppe, Bastien Rioux, Lahoud Touma & Laura C. Gioia

  2. Département de Radiologie, Centre Hospitalier de l’Université de Montréal (CHUM), Montréal, Québec, Canada

    Céline Bard, William Boisseau, Jose Danilo Bengzon Diestro, François Guilbert, Mohamed Taoubane Maallah, Kristoff Nelson, Igor Gomes Padilha, Daniel Roy, Alain Weill & Laurent Létourneau-Guillon

  3. Axe Neurosciences, Centre de Recherche du CHUM (CRCHUM), Montréal, Québec, Canada

    Olena Bereznyakova, Yan Deschaintre, Grégory Jacquin, Alexandre Y. Poppe & Laura C. Gioia

  4. Département de Neurosciences, Université de Montréal, Montréal, Québec, Canada

    Olena Bereznyakova, Yan Deschaintre, Grégory Jacquin, Alexandre Y. Poppe & Laura C. Gioia

  5. Imaging and Engineering Axis, Centre de Recherche du CHUM (CRCHUM), Montréal, Québec, Canada

    Laurent Létourneau-Guillon

Authors
  1. Ahmad Nehme
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Célina Ducroux
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marie-Andrée Panzini
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Céline Bard
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Olena Bereznyakova
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. William Boisseau
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yan Deschaintre
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jose Danilo Bengzon Diestro
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. François Guilbert
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Grégory Jacquin
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Mohamed Taoubane Maallah
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Kristoff Nelson
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Igor Gomes Padilha
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Alexandre Y. Poppe
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Bastien Rioux
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Daniel Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Lahoud Touma
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Alain Weill
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Laura C. Gioia
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Laurent Létourneau-Guillon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ahmad Nehme.

Ethics declarations

Guarantor

The scientific guarantor of this publication is Laurent Létourneau-Guillon.

Conflict of interest

The authors of this manuscript declare no relationships with any companies whose products or services may be related to the subject matter of the article.

Statistics and biometry

No complex statistical methods were necessary for this paper.

Informed consent

Written informed consent was waived by the Institutional Review Board.

Ethics approval

Institutional Review Board approval was obtained.

Methodology

• prospective

• observational

• performed at one institution

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

ESM 1

(DOCX 616 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nehme, A., Ducroux, C., Panzini, MA. et al. Non-contrast CT markers of intracerebral hematoma expansion: a reliability study. Eur Radiol 32, 6126–6135 (2022). https://doi.org/10.1007/s00330-022-08710-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00330-022-08710-w

Keywords

Search

Navigation