Outcome prediction in hospitalized COVID-19 patients: Comparison of the performance of five severity scores
Hsin-Pei Chung1,2† 
Yen-Hsiang Tang2,3† 
Chun-Yen Chen4 
Chao-Hsien Chen1,2 Wen-Kuei Chang1,2 Kuan-Chih Kuo1,2 Yen-Ting Chen1,2 Jou-Chun Wu1,2 Chang-Yi Lin1,2 
Chieh-Jen Wang1,2*- 1Division of Pulmonary, Department of Internal Medicine, MacKay Memorial Hospital, Taipei, Taiwan
- 2Department of Critical Care Medicine, MacKay Memorial Hospital, Taipei, Taiwan
- 3Department of Medicine, MacKay Medical College, New Taipei City, Taiwan
- 4Division of Cardiology, Department of Internal Medicine, MacKay Memorial Hospital, Taipei, Taiwan
Background: The aim of our study was to externally validate the predictive capability of five developed coronavirus disease 2019 (COVID-19)-specific prognostic tools, including the COVID-19 Spanish Society of Infectious Diseases and Clinical Microbiology (SEIMC), Shang COVID severity score, COVID-intubation risk score-neutrophil/lymphocyte ratio (IRS-NLR), inflammation-based score, and ventilation in COVID estimator (VICE) score.
Methods: The medical records of all patients hospitalized for a laboratory-confirmed COVID-19 diagnosis between May 2021 and June 2021 were retrospectively analyzed. Data were extracted within the first 24 h of admission, and five different scores were calculated. The primary and secondary outcomes were 30-day mortality and mechanical ventilation, respectively.
Results: A total of 285 patients were enrolled in our cohort. Sixty-five patients (22.8%) were intubated with ventilator support, and the 30-day mortality rate was 8.8%. The Shang COVID severity score had the highest numerical area under the receiver operator characteristic (AUC-ROC) (AUC 0.836) curve to predict 30-day mortality, followed by the SEIMC score (AUC 0.807) and VICE score (AUC 0.804). For intubation, both the VICE and COVID-IRS-NLR scores had the highest AUC (AUC 0.82) compared to the inflammation-based score (AUC 0.69). The 30-day mortality increased steadily according to higher Shang COVID severity scores and SEIMC scores. The intubation rate exceeded 50% in the patients stratified by higher VICE scores and COVID-IRS-NLR score quintiles.
Conclusion: The discriminative performances of the SEIMC score and Shang COVID severity score are good for predicting the 30-day mortality of hospitalized COVID-19 patients. The COVID-IRS-NLR and VICE showed good performance for predicting invasive mechanical ventilation (IMV).
1. Introduction
In December 2019, there was an emerging viral infection outbreak in Wuhan, China. The pathogen was later identified as a new strain of coronavirus, severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2), and the disease it caused was named coronavirus disease 2019 (COVID-19). The disease rapidly spread from Wuhan to the rest of the world (1).
The disease severity ranged widely, from asymptomatic or minor symptoms, such as rhinorrhea, productive cough, anosmia, ageusia, and fever, to more severe conditions, such as pneumonia, acute respiratory failure, and even acute respiratory distress syndrome (ARDS). It can progress rapidly (2, 3) and advanced life support with intensive care, such as oxygen therapy, non-invasive ventilation (NIV), and invasive mechanical ventilation (IMV), may be warranted. Excessive demand on healthcare services overwhelmed healthcare systems worldwide (4–7). Administration triage for optimized patient care became essential.
Clinical evaluation alone may lead to misjudgment, under- or overestimation of disease severity and result in suboptimal medical treatment and admission to an inappropriate setting (8). Disease severity scores have been proposed since the early 1980s to help physician decision-making and predict outcomes. For instance, the CURB-65 (confusion, uremia, respiratory rate, BP, age ≥ 65 years) score and qSOFA (quick sepsis-related organ failure assessment) score are clinically relevant predictive tools for community-acquired pneumonia and sepsis, but their risk prediction performance in COVID-19 is not satisfactory (9, 10).
Several predictive scores had been published (11–20), but only a handful of them had ever been validated externally (10). The first wave of the COVID-19 pandemic in Taiwan occurred with a delay of several months in comparison with the first waves in other countries; nevertheless, the waves had similar viral characteristics. During the first wave, all COVID-19 patients had to be admitted to a hospital for quarantine according to the Taiwan Centers for Disease Control (CDC) regulation (21), regardless of the severity. Therefore, our cohort may be more representative of the spectrum of COVID-19 disease and be a good cohort to validate the accuracy of the previous predictive scores. The Spanish Society of Infectious Diseases and Clinical Microbiology (SEIMC) score (11) and the Shang COVID severity score (12) were developed to predict mortality, while the COVID-intubation risk score-neutrophil/lymphocyte ratio (IRS-NLR) score (13), inflammation-based risk score (14), and ventilation in COVID estimator (VICE) (22) score were designed to predict the need for IMV. All five predictive scores are useful since only clinical parameters and commonly available laboratory results were included, but the accuracy of these scores has been uncertain. Herein, the primary aim of the present study was to validate these severity scores and predictive models to predict mortality and the need for IMV.
2. Materials and methods
2.1. Study design and patient selection
We retrospectively studied all COVID-19 adult patients admitted for COVID-19 from 1 May 2021 to 30 June 2021, the first wave of COVID-19 infection in Taiwan with low COVID-19 vaccination coverage, to MacKay Memorial Hospital, a tertiary referral center in Taipei, Taiwan. All patients were confirmed to be diagnosed by a polymerase chain reaction using a nasopharyngeal sample. Patients who were under 20 years of age or identified as “do not intubate (DNI)” were excluded. The patients’ medical records and laboratory results were reviewed. Five different kinds of predictive scores were calculated, including the Shang COVID severity score (12), SEIMC score (11), COVID-IRS-NLR score (12), inflammation-based risk scoring system (14), and VICE score (22). The Institutional Review Board of MacKay Memorial Hospital approved this study with approval number 21MMHIS330e.
2.2. Outcome measurement
Our primary outcome was 30-day mortality. The secondary outcome was intubation with IMV support. Of note, non-IMV and high-flow nasal cannula were not included in the secondary outcome. The patients were followed until they expired or were discharged, depending on which developed first.
2.3. Definitions
The severity of COVID-19 scores was calculated by laboratory tests performed on or within 24 h of hospital admission. The patients were assessed for the presence of diabetes mellitus, coronary artery disease (CAD), and home statin medication history, and these factors were extracted from the electronic medical records. The estimated glomerular filtration rate (eGFR) was defined as the modification of diet in renal disease [Modification of diet in renal disease (MDRD) equation, which was 186 × (creatinine) (−1.154) × (age) (−0.203) for males and 186 × (creatinine) (−1.154) × (age) (−0.203) × 0.742 in females].
2.4. Statistical analysis
Categorical variables are presented as numbers (percentages). The frequencies of categorical variables were compared using the chi-squared test or Fisher’s exact test. Continuous variables are reported as the mean ± standard deviation (SD). The means of two continuous variables were compared by the independent samples t-test. We used a univariable logistic regression model to determine variables that would be included in our predictive risk score algorithms for mechanical ventilation needs and in-hospital death. In addition, variables with p < 0.05 were considered statistically significant and then were entered into a multivariate logistic regression model to determine independent predictors. We built receiver operating characteristic (ROC) curves to assess the predictive performance of all scores for the primary and secondary outcomes. We calculated pooled areas under the curve (AUCs) and 95% confidence intervals (CIs). The Hosmer–Lemeshow test was used to evaluate the goodness of fit for logistic regression models. For all tests, a two-sided p-value less than 0.05 was considered significant. Data were analyzed using SPSS software (version 22; IBM Corporation, Armonk, NY, USA).
3. Results
A total of 311 patients were enrolled in this study period and were followed until they were discharged from our hospital or died. A total of 26 patients who refused intubation during respiratory failure with DNI orders were excluded, leaving 285 patients for inclusion in the analysis.
3.1. Patient characteristics
The patient characteristics are shown in Table 1. The mean age was 61.5 ± 14.8 years, and 156 patients (54.7%) were male. Additionally, patients with the comorbidity of diabetes accounted for 29.8% of the cohort, and CAD patients accounted for 7.4% of our cohort. The lowest SpO2 level recorded within 24 h of admission was 93.7 ± 6.2%, and the SpO2/FiO2 ratio was 402.4 ± 106.2. A total of 65 patients (22.8%) were intubated with ventilator support, and 25 patients died with an 30-day mortality of 8.8%. Only 4.6% (13/285) of our cohort received one dose of a COVID-19 vaccine at the time of admission. The majority of our population were not vaccinated.
Table 1. Baseline characteristics and laboratory findings among survivors and non-survivors among hospitalized COVID-19 patients.
Clinical and laboratory parameters that were associated with 30-day mortality and the need for IMV were identified (Tables 1, 2). The factors on admission were consistently predictive of both mortality and a requirement for IMV, and these factors included age, lowest SpO2, D-dimer, albumin, and C-reactive protein (CRP) level. Comorbid CAD is a predictor of mortality only. The odds ratio (OR) of age in mortality was 1.07 (95% CI: 1.03–1.11) and 1.03 (95% CI: 1.01–1.05) in IMV requirement. The CRP level is the most predictive laboratory parameter in both mortality and IMV need, with ORs of 1.13 (95% CI: 1.07–1.19) and 1.14 (95% CI: 1.09–1.19), respectively.
Table 2. Baseline characteristics and laboratory findings with and without ventilator use with COVID-19 infection.
3.2. Comparison of mortality and intubation rate by risk class
Figure 1 shows the mortality rate and ventilation rate across different scoring system risk classes, including the COVID-19 SEIMC score, COVID-IRS-NLR score, inflammatory score, Shang COVID severity score, and VICE score. There was a significant difference in the mortality rate and intubation rate among the lowest- to highest-risk classes in all five scoring systems. The 30-day mortality increased steadily according to higher COVID-IRS-NLR, Shang COVID severity score, and SEIMC score. The intubation rate exceeded 50% in the patients stratified by higher VICE score and IRS-NLR score quintiles, which revealed 76.2 and 90.9% in the 4th and 5th quintiles in the VICE score and 56.8, 100, and 100% in the 3rd–5th quintiles in the IRS-NLR score, respectively.
Figure 1. Distribution of the SEIMC, IRS-NLR, inflammatory, Shang et al., and VICE scores by risk class in our patients. (A) The mortality rate was plotted against five score stratifications. (B) The intubation rate was plotted against five score stratifications. The correlation between each of the five scoring systems and the increase in severity. *p < 0.001; SEIMC, Spanish Society of Infectious Diseases and Clinical Microbiology; IRS-NLR, intubation risk score-neutrophil/lymphocyte ratio; VICE, ventilation in COVID estimator.
3.3. Performance of risk prediction and modeling for COVID-19 mortality
The area under the ROC curves (AUC) for 30-day mortality for each prognostic score for COVID-19 is shown in Figure 2A and Table 3. The Shang COVID severity score showed the highest prediction of mortality, with an AUC of 0.836. The AUCs for the SEIMC score and VICE score were 0.807 and 0.804, respectively, suggesting good predictive performance for 30-day mortality. The performance of the DICE score was not validated because of missing values in our cohort and loss of statistical power.
Figure 2. (A) The receiver-operator characteristic (ROC) curve (with AUC) for predicting mortality among patients with coronavirus disease 2019 (COVID-19) in our cohort. (B) The ROC curve for predicting mechanical ventilation requirements among patients with COVID-19 in our cohort. HR, hazard ratio; AUC, area under the curve; SEIMC, Spanish Society of Infectious Diseases and Clinical Microbiology; IRS-NLR, intubation risk score-neutrophil/lymphocyte ratio; VICE, ventilation in COVID estimator.
Table 3. Summary of the prognostic performance of different severity scores for mortality/intubation in hospitalized patients with COVID-19.
Sex and other reliable mortality-associated variables, including age, lowest SpO2, CRP, albumin, and D-dimer, were selected. Logistic regression models were generated by combining the scoring systems with the above variables (Table 4). The risk predictive model with the VICE score, Shang COVID severity score, and IRS-NLR score showed significant prognostic accuracy for mortality [OR: 19.6; (3.06–126.0); 1.71 (1.09–2.69); 1.31 (1.06–1.62), respectively]. The Hosmer–Lemeshow test of all the models yielded a non-significant statistic, indicating that there was no departure from perfect fit.
Table 4. Models of predictors of mortality.
3.4. Performance of risk prediction for intubation and further modeling
The ROC curves for the IMV requirement for each scoring system in COVID-19 patients are shown in Figure 2B and Table 3. The IRS-NLR and VICE scores showed the strongest prediction of mortality, with AUCs of 0.82 for both.
Logistic regression models were generated by combining scoring systems with sex, age, lowest SpO2, CRP, albumin, and D-dimer (Table 5). The risk predictive models with the VICE score and IRS-NLR score showed the greatest prognostic accuracy of intubation [OR: 84.9 (14.6–492.4) and 1.62 (1.33–1.99), respectively]. The Hosmer–Lemeshow test of all the models yielded a non-significant statistic, indicating that there was no departure from perfect fit.
Table 5. Models of predictors of ventilation use.
4. Discussion
This study evaluated the performance of the COVID-19 SEIMC score, the COVID-IRS-NLR score, the inflammatory-based risk score, the Shang COVID severity score, and the VICE prediction rule in predicting 30-day mortality and IMV requirements in hospitalized COVID-19 patients. In our cohort, the SEIMC score and Shang COVID severity score were good models for predicting 30-day mortality. For intubation prediction, the IRS-NLR, and VICE score prediction rules showed the best performance.
Our results reinforce the results of several previous studies that found specific initial parameters to be significant predictors of poor outcome in patients with COVID-19. Age (23, 24), lower SpO2 (25), higher D-dimer, higher CRP, and hypoalbuminemia (26) increased the chances of mortality and ventilation requirements in our study. Previous studies found that an increase in D-dimer and fibrinogen is associated with an increase in COVID-19 severity and mortality (27–30). Increased D-dimer represents activation of coagulation cascades secondary to systemic inflammation and causes microthrombi formation inside the blood vessels that can induce disseminated intravascular coagulation (DIC) (31). CRP levels are reliable markers for prognostic factors (32). Taken together, these results indicate that COVID-19 virus-induced inflammatory and hypercoagulation responses drive the severity of disease (33).
Hundreds of prognostic scoring systems have been developed and studied during the COVID-19 pandemic to predict different outcomes, including mortality (11, 12, 15), severe illness (16–18), critical illness (34), intensive care unit (ICU) admission (19, 20, 35), or IMV use (13, 14). The definition of severe COVID-19 varies across different studies, and the hospitalization criteria may differ by disease prevention policy across countries. The complexity of the ICU admission criteria may fluctuate and be affected by demography and ICU bed scarcity (10). Therefore, the outcomes used in our study, 30-day mortality and IMV, are the most clinically relevant, while they have objective and reduced diversity.
Clinical scoring systems are designed to aid decision making and add to clinical judgment in different healthcare services. Some simplified scoring methods are designed to frequently assess dynamic requirements for escalating levels of respiratory support and to rescore after interventions, such as the Brescia-COVID respiratory severity scale (BCRSS) (36) and the Quick COVID-19 Severity Index (qCSI) (37). These kinds of scoring systems are practically applicable in emergent department settings, which allows quick triage. Most COVID-19 risk scores aimed to predict ultimate outcomes by using the initial evaluation, while some scores use parameters that have not been routinely collected in our cohort [such as CT scan (38) or red cell distribution width (39)]. Further prospective studies are needed for further validation.
To date, the largest, multicenter cohort of 14,343 patients to validate systematically selected prognostic scores for 30-day in-hospital mortality had been demonstrated that 4C mortality (15) and ABCS score had modest utility (AUC > 0.75) (10). The SEIMC scores had low prediction in that French cohort but good prediction in our cohort. The Shang et al. (12) severity score possesses the highest AUC-ROC curve in our patient population. The mortality rate observed within our cohort was much lower than expected in the Shang et al. (12) cohort. The mortality rate was 81.1% among the high-risk group (above 2 points), and the observed mortality rate of the high-risk group in our cohort was only 19.6%. The differences between the two cohorts may be because they faced the first wave of COVID-19, and there were no well-established treatment strategies. In contrast, the SEIMC score predicted the mortality rate in our cohort more precisely (11), while incremental risk stratification represented increased mortality.
The COVID-IRS-NLR and VICE score showed the strongest prediction and greatest accuracy of IMV among our patient population. The intubation rate of the high-risk and very high-risk patients based on the COVID-IRS-NLR score and those who were in quintiles 4th and 5th based on the VICE score was high (13, 22), and these factors may warrant intubation. Early intubation could prevent patient self-inflicted lung injury (40). However, IMV is also associated with complications, such as prolonged sedation and paralysis, infection, and barotrauma (41), and it was reasonable to treat the patients in the low-risk group using a high-flow nasal cannula or awake proning rather than early intubation.
Triage is important in the face of the COVID pandemic (42). If we can foresee the possible progress of the patient’s condition during the first episode, different approaches, treatment options, and patient relocation can be arranged accordingly (43, 44). Patients who are predicted to be less likely to worsen can be treated with a step-down approach and at home without consuming medical resources (45). Patients with a high risk of death or requiring mechanical ventilation support should be admitted to the hospital ward or even the ICU. Therefore, we can reduce the possibility of missed diagnosis of severe COVID-19 and mortality (43).
The underlying reasons for the performance differences among these severity scores could be multifactorial. Patient characteristics (46), vaccination status (47), healthcare system (48), and the variables included in each of the score all contribute to the performance differences. Table 3 shows the different variables used in each scoring systems. The variables could be categorized into patient characteristics, symptoms/signs, laboratory data, and clinical parameters, such as respiratory rate, SpO2/FiO2. Age was included both in the SEIMC score and Shang COVID severity score which were the better predictive scores of 30-day mortality. These finding implied the importance of age in driving severity among hospitalized COVID-19 patients (49). The good predictive scores for IMV in our study were the COVID-IRS-NLR and VICE score, and the common variables were lactic dehydrogenase (LDH) and SpO2/FiO2. Elevated LDH was associated with poor outcome in COVID-19 patients (50). SpO2/FiO2 may substitute for PaO2/FiO2 as a diagnostic and prognostic marker in COVID-19 patients (51). The performance differences do exist in different race/ethnicity and regions; external validation is needed (52). Multi-center, cross-country studies to verify the accuracy of these severity scores need to be conducted.
5. Limitations
This study had several limitations. First, it was conducted at a single center, and the results may not be generalizable worldwide. Second, the retrospective design with some missing values for several important variables, such as body mass index (BMI) and statin use, results in an inability to validate more scoring systems. Third, due to the small number of cases in our cohort, it may not be significant to identify new cut-off points in each score. Finally, our cohort was composed of COVID-19 alpha variant patients and most of them were unvaccinated. COVID-19 vaccination status is associated with the severity of COVID-19 illness. The prediction ability of the five risk scores for other variants or vaccinated populations is unknown. Further well-designed prospective studies are needed to validate these findings in the future.
6. Conclusion
The discriminative performance of the SEIMC score and Shang COVID severity score were good for 30-day mortality in COVID-19 hospitalized patients. For intubation prediction, the COVID-IRS-NLR, and VICE score prediction rules showed the best performance.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving human participants were reviewed and approved by the Institutional Review Board of MacKay Memorial Hospital. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.
Author contributions
C-JW contributed to the conception and study design. H-PC and Y-HT were in charge of execution, acquisition of data, interpretation, and drafting. C-YC took charge of data analysis. C-HC, W-KC, K-CK, and Y-TC were in charge of execution, acquisition of data, analysis, and interpretation. J-CW and C-YL took charge of drafting, revising, and critically reviewing the article. All authors gave final approval of the version to be published, had agreed on the journal to which the article has been submitted, and agreed to be accountable for all aspects of the work.
Acknowledgments
We would like to thank Prof. Rajeev Malhotra for providing equations to calculate the VICE and DICE scores.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Abbreviations
SARS-CoV-2, severe acute respiratory syndrome coronavirus-2; COVID-19, coronavirus disease 2019; ARDS, acute respiratory distress syndrome; NIV, non-invasive ventilation; IMV, invasive mechanical ventilation; DNI, do not intubate; CAD, coronary artery disease; SD, standard deviation; AUC, areas under the curve, CIs, confidence intervals; CRP, C-reactive protein; ROC, receiver-operator characteristic, DIC, disseminated intravascular coagulation.
References
1. Lu R, Zhao X, Li J, Niu P, Yang B, Wu H, et al. Genomic characterisation and epidemiology of 2019 novel coronavirus: implications for virus origins and receptor binding. Lancet. (2020) 395:565–74. doi: 10.1016/S0140-6736(20)30251-8
2. Guan W, Ni Z, Hu Y, Liang W, Ou C, He J, et al. Clinical characteristics of coronavirus disease 2019 in China. N Engl J Med. (2020) 382:1708–20. doi: 10.1056/NEJMoa2002032
3. Huang C, Wang Y, Li X, Ren L, Zhao J, Hu Y, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan. China. Lancet. (2020) 395:497–506. doi: 10.1016/S0140-6736(20)30183-5
4. Cavallo J, Forman H. The economic impact of the COVID-19 pandemic on radiology practices. Radiology. (2020) 296:E141–4. doi: 10.1148/radiol.2020201495
5. Shang Y, Li H, Zhang R. Effects of pandemic outbreak on economies: evidence from business history context. Front Public Health. (2021) 9:632043. doi: 10.3389/fpubh.2021.632043
6. Martin B, Brodke D, Wilson F, Chaiyakunapruk N, Nelson R. The impact of halting elective admissions in anticipation of a demand surge due to the coronavirus pandemic (COVID-19). Med Care. (2021) 59:213–9. doi: 10.1097/MLR.0000000000001496
7. Romani G, Dal Mas F, Massaro M, Cobianchi L, Modenese M, Barcellini A, et al. Population health strategies to support hospital and intensive care unit resiliency during the COVID-19 pandemic: the Italian experience. Popul Health Manag. (2021) 24:174–81. doi: 10.1089/pop.2020.0255
8. Capelastegui A, Espana P, Quintana J, Areitio I, Gorordo I, Egurrola M, et al. Validation of a predictive rule for the management of community-acquired pneumonia. Eur Respir J. (2006) 27:151–7. doi: 10.1183/09031936.06.00062505
9. Lim W, van der Eerden M, Laing R, Boersma W, Karalus N, Town G, et al. Defining community acquired pneumonia severity on presentation to hospital: an international derivation and validation study. Thorax. (2003) 58:377–82. doi: 10.1136/thorax.58.5.377
10. Lombardi Y, Azoyan L, Szychowiak P, Bellamine A, Lemaitre G, Bernaux M, et al. External validation of prognostic scores for COVID-19: a multicenter cohort study of patients hospitalized in greater Paris University Hospitals. Intensive Care Med. (2021) 47:1426–39.
11. Berenguer J, Borobia A, Ryan P, Rodriguez-Bano J, Bellon J, Jarrin I, et al. Development and validation of a prediction model for 30-day mortality in hospitalised patients with COVID-19: the COVID-19 SEIMC score. Thorax. (2021) 76:920–9. doi: 10.1136/thoraxjnl-2020-216001
12. Shang Y, Liu T, Wei Y, Li J, Shao L, Liu M, et al. Scoring systems for predicting mortality for severe patients with COVID-19. Eclinicalmedicine. (2020) 24:100426. doi: 10.1016/j.eclinm.2020.100426
13. Garcia-Gordillo J, Camiro-Zuniga A, Aguilar-Soto M, Cuenca D, Cadena-Fernandez A, Khouri L, et al. COVID-IRS: a novel predictive score for risk of invasive mechanical ventilation in patients with COVID-19. PLoS One. (2021) 16:e0248357. doi: 10.1371/journal.pone.0248357
14. Amezcua-Guerra L, Audelo K, Guzman J, Santiago D, Gonzalez-Flores J, Garcia-Avila C, et al. A simple and readily available inflammation-based risk scoring system on admission predicts the need for mechanical ventilation in patients with COVID-19. Inflamm Res. (2021) 70:731–42. doi: 10.1007/s00011-021-01466-x
15. Knight S, Ho A, Pius R, Buchan I, Carson G, Drake T, et al. Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO clinical characterisation protocol: development and validation of the 4C Mortality Score. BMJ. (2020) 370:m3339.
16. Gong J, Ou J, Qiu X, Jie Y, Chen Y, Yuan L, et al. A tool for early prediction of severe coronavirus disease 2019 (COVID-19): a multicenter study using the risk nomogram in Wuhan and Guangdong. China. Clin Infect Dis. (2020) 71:833–40. doi: 10.1093/cid/ciaa443
17. Chen Y, Zhou X, Yan H, Huang H, Li S, Jiang Z, et al. CANPT score: a tool to predict severe COVID-19 on admission. Front Med. (2021) 8:608107. doi: 10.3389/fmed.2021.608107
18. Kodama T, Obinata H, Mori H, Murakami W, Suyama Y, Sasaki H, et al. Prediction of an increase in oxygen requirement of SARS-CoV-2 pneumonia using three different scoring systems. J Infect Chemother. (2021) 27:336–41. doi: 10.1016/j.jiac.2020.12.009
19. Statsenko Y, Al Zahmi F, Habuza T, Gorkom K, Zaki N. Prediction of COVID-19 severity using laboratory findings on admission: informative values, thresholds, ML model performance. BMJ Open. (2021) 11:e044500. doi: 10.1136/bmjopen-2020-044500
20. Ozpolat C, Altunbas E. Diagnostic utility of the covichem score in predicting COVID-19 disease. Am J Emerg Med. (2022) 60:50–6. doi: 10.1016/j.ajem.2022.07.025
21. Lin C, Braund W, Auerbach J, Chou J, Teng J, Tu P, et al. Policy decisions and use of information technology to fight COVID-19, Taiwan. Emerg Infect Dis. (2020) 26:1506–12. doi: 10.3201/eid2607.200574
22. Nicholson C, Wooster L, Sigurslid H, Li R, Jiang W, Tian W, et al. Estimating risk of mechanical ventilation and in-hospital mortality among adult COVID-19 patients admitted to Mass General Brigham: the VICE and DICE scores. Eclinicalmedicine. (2021) 33:100765. doi: 10.1016/j.eclinm.2021.100765
23. Chung H, Wu K, Lin C, Tang Y, Chen C, Wu J, et al. Characteristics and outcomes of hospitalized geriatric patients with COVID-19 infection in Taiwan. Int J Gerontol. (2022) 16:207–12.
24. Zhou F, Yu T, Du R, Fan G, Liu Y, Liu Z, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet. (2020) 395:1054–62. doi: 10.1016/S0140-6736(20)30566-3
25. Marin B, Aghagoli G, Lavine K, Yang L, Siff E, Chiang S, et al. Predictors of COVID-19 severity: a literature review. Rev Med Virol. (2021) 31:1–10.
26. Wei W, Wu X, Jin C, Mu T, Gu G, Min M, et al. Predictive significance of the prognostic nutritional index (PNI) in patients with severe COVID-19. J Immunol Res. (2021) 2021:9917302. doi: 10.1155/2021/9917302
27. Saurabh A, Dey B, Raphael V, Deb P, Khonglah Y, Tiewsoh I. Role of coagulation profile in predicting disease severity among patients of COVID-19. Cureus. (2021) 13:e19124. doi: 10.7759/cureus.19124
28. Yao Y, Cao J, Wang Q, Shi Q, Liu K, Luo Z, et al. D-dimer as a biomarker for disease severity and mortality in COVID-19 patients: a case control study. J Intensive Care. (2020) 8:49. doi: 10.1186/s40560-020-00466-z
29. Di Micco P, Russo V, Carannante N, Imparato M, Cardillo G, Lodigiani C. Prognostic value of fibrinogen among COVID-19 patients admitted to an emergency department: an Italian cohort study. J Clin Med. (2020) 9:4134. doi: 10.3390/jcm9124134
30. Lu H, Chen M, Tang S, Yu W. Association of coagulation disturbances with severity of COVID-19: a longitudinal study. Hematology. (2021) 26:656–62. doi: 10.1080/16078454.2021.1968648
31. Luo H, You C, Lu S, Fu Y. Characteristics of coagulation alteration in patients with COVID-19. Ann Hematol. (2021) 100:45–52. doi: 10.1007/s00277-020-04305-x
32. Yue-Liang X, Jiang-Lin W, Hui-Qin Y, Ge Z, Hongyu D, Wei-Jin F, et al. The risk factors for severe patients with COVID-19 in China: a systematic review and meta-analysis. Eur J Inflamm. (2021) 19:1–13. doi: 10.1177/20587392211000890
33. An H, Zhang J, Zhou T, Li T, Li S, Huang C, et al. Inflammation/coagulopathy/fibrinolysis: dynamic indicators of COVID-19 progression in patients with moderate COVID-19 in Wenzhou, China. Clin Immunol. (2021) 232:108852. doi: 10.1016/j.clim.2021.108852
34. Liang W, Liang H, Ou L, Chen B, Chen A, Li C, et al. Development and validation of a clinical risk score to predict the occurrence of critical illness in hospitalized patients with COVID-19. JAMA Intern Med. (2020) 180:1081–9. doi: 10.1001/jamainternmed.2020.2033
35. Rodriguez-Nava G, Yanez-Bello M, Trelles-Garcia D, Chung C, Friedman H, Hines D. Performance of the quick COVID-19 severity index and the Brescia-COVID respiratory severity scale in hospitalized patients with COVID-19 in a community hospital setting. Int J Infect Dis. (2021) 102:571–6. doi: 10.1016/j.ijid.2020.11.003
36. Duca A, Piva S, Foca E, Latronico N, Rizzi M. Calculated decisions: brescia-COVID respiratory severity scale (BCRSS)/algorithm. Emerg Med Pract. (2020) 22(Suppl. 5):CD1–2.
37. Haimovich A, Ravindra N, Stoytchev S, Young H, Wilson F, van Dijk D, et al. Development and validation of the quick COVID-19 severity index: a prognostic tool for early clinical decompensation. Ann Emerg Med. (2020) 76:442–53. doi: 10.1016/j.annemergmed.2020.07.022
38. Rahimi E, Shahisavandi M, Royo A, Azizi M, El Bouhaddani S, Sigari N, et al. The risk profile of patients with COVID-19 as predictors of lung lesions severity and mortality-development and validation of a prediction model. Front Microbiol. (2022) 13:893750. doi: 10.3389/fmicb.2022.893750
39. Cubukcu H, Topcu D, Bayraktar N, Gulsen M, Sari N, Arslan A. Detection of COVID-19 by machine learning using routine laboratory tests. Am J Clin Pathol. (2022) 157:758–66. doi: 10.1093/ajcp/aqab187
40. Marini J, Gattinoni L. Management of COVID-19 respiratory distress. JAMA. (2020) 323:2329–30. doi: 10.1001/jama.2020.6825
42. Mallah S, Ghorab O, Al-Salmi S, Abdellatif O, Tharmaratnam T, Iskandar M, et al. COVID-19: breaking down a global health crisis. Ann Clin Microbiol Antimicrob. (2021) 20:35. doi: 10.1186/s12941-021-00438-7
43. Peng Z, Chen X, Hu Q, Hu J, Zhao Z, Zhang M, et al. Prediction of severe outcomes of patients with COVID-19. Zhonghua Liu Xing Bing Xue Za Zhi. (2020) 41: 1595–600.
44. Xu J, Zhao F, Han M, Ma L, Zhang T. Analysis of the clinical characteristics and early warning model construction of severe/critical coronavirus disease 2019 patients. Zhonghua Wei Zhong Bing Ji Jiu Yi Xue. (2020) 32:401–6.
45. Tsui E, Lui C, Woo P, Cheung A, Lam P, Tang V, et al. Development of a data-driven COVID-19 prognostication tool to inform triage and step-down care for hospitalised patients in Hong Kong: a population-based cohort study. BMC Med Inform Decis Mak. (2020) 20:323. doi: 10.1186/s12911-020-01338-0
46. Roth G, Emmons-Bell S, Alger H, Bradley S, Das S, de Lemos J, et al. Trends in patient characteristics and COVID-19 In-hospital mortality in the united states during the COVID-19 pandemic. JAMA Netw Open. (2021) 4:e218828. doi: 10.1001/jamanetworkopen.2021.8828
47. Rustagi V, Bajaj M, Tanvi Singh P, Aggarwal R, AlAjmi M, et al. Analyzing the effect of vaccination over COVID cases and deaths in asian countries using machine learning models. Front Cell Infect Microbiol. (2022) 11:806265. doi: 10.3389/fcimb.2021.806265
48. Neogi S, Pandey S, Preetha G, Swain S. The predictors of COVID-19 mortality among health systems parameters: an ecological study across 203 countries. Health Res Policy Syst. (2022) 20:75. doi: 10.1186/s12961-022-00878-3
49. Bonanad C, García-Blas S, Tarazona-Santabalbina F, Sanchis J, Bertomeu-González V, Fácila L, et al. The effect of age on mortality in patients with COVID-19: a meta-analysis with 611,583 subjects. J Am Med Dir Assoc. (2020) 21:915–8. doi: 10.1016/j.jamda.2020.05.045
50. Malik P, Patel U, Mehta D, Patel N, Kelkar R, Akrmah M, et al. Biomarkers and outcomes of COVID-19 hospitalisations: systematic review and meta-analysis. BMJ Evid Based Med. (2021) 26:107–8. doi: 10.1136/bmjebm-2020-111536
51. Carvalho E, Leite T, Sacramento R, Nascimento P, Samary C, Rocco P, et al. Rationale and limitations of the SpO2/FiO2 as a possible substitute for PaO2/FiO2 in different preclinical and clinical scenarios. Rev Bras Ter Intensiva. (2022) 34:185–96. doi: 10.5935/0103-507X.20220013-en
Keywords: COVID-19, SEIMC score, mortality prediction, IRS-NLR score, VICE score
Citation: Chung H-P, Tang Y-H, Chen C-Y, Chen C-H, Chang W-K, Kuo K-C, Chen Y-T, Wu J-C, Lin C-Y and Wang C-J (2023) Outcome prediction in hospitalized COVID-19 patients: Comparison of the performance of five severity scores. Front. Med. 10:1121465. doi: 10.3389/fmed.2023.1121465
Received: 11 December 2022; Accepted: 26 January 2023;
Published: 08 February 2023.
Edited by:
Reza Lashgari, Shahid Beheshti University, IranReviewed by:
Ramit Singla, The University of Tennessee Health Science Center (UTHSC), United StatesPengyue Zhao, The First Center of Chinese PLA General Hospital, China
Copyright © 2023 Chung, Tang, Chen, Chen, Chang, Kuo, Chen, Wu, Lin and Wang. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Chieh-Jen Wang, 
jagdzaku@gmail.com
†These authors have contributed equally to this work and share first authorship