Article Text
PDF
Abstract
Objectives Identifying patients with a possible SARS-CoV-2 infection in the emergency department (ED) is challenging. Symptoms differ, incidence rates vary and test capacity may be limited. As PCR-testing all ED patients is neither feasible nor effective in most centres, a rapid, objective, low-cost early warning score to triage ED patients for a possible infection is developed.
Design Case–control study.
Setting Secondary and tertiary hospitals in the Netherlands.
Participants The study included patients presenting to the ED with venous blood sampling from July 2019 to July 2020 (n=10 417, 279 SARS-CoV-2-positive). The temporal validation cohort covered the period from July 2020 to October 2021 (n=14 080, 1093 SARS-CoV-2-positive). The external validation cohort consisted of patients presenting to the ED of three hospitals in the Netherlands (n=12 061, 652 SARS-CoV-2-positive).
Primary outcome measures The primary outcome was one or more positive SARS-CoV-2 PCR test results within 1 day prior to or 1 week after ED presentation.
Results The resulting ‘CoLab-score’ consists of 10 routine laboratory measurements and age. The score showed good discriminative ability (AUC: 0.930, 95% CI 0.909 to 0.945). The lowest CoLab-score had high sensitivity for COVID-19 (0.984, 95% CI 0.970 to 0.991; specificity: 0.411, 95% CI 0.285 to 0.520). Conversely, the highest score had high specificity (0.978, 95% CI 0.973 to 0.983; sensitivity: 0.608, 95% CI 0.522 to 0.685). The results were confirmed in temporal and external validation.
Conclusions The CoLab-score is based on routine laboratory measurements and is available within 1 hour after presentation. Depending on the prevalence, COVID-19 may be safely ruled out in over one-third of ED presentations. Highly suspect cases can be identified regardless of presenting symptoms. The CoLab-score is continuous, in contrast to the binary outcome of lateral flow testing, and can guide PCR testing and triage ED patients.
- COVID-19
- statistics & research methods
- accident & emergency medicine
- Clinical chemistry
- Health informatics
Data availability statement
Data are available in a public, open access repository. Data is de-indentified.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- COVID-19
- statistics & research methods
- accident & emergency medicine
- Clinical chemistry
- Health informatics
STRENGTHS AND LIMITATIONS OF THIS STUDY
A comprehensive panel of 28 laboratory tests were measured for 10 417 emergency department (ED) presentations and combined with SARS-CoV-2 PCR test results.
Using adaptive lasso regression analysis, the panel of 28 laboratory tests was reduced to a single score consisting of a subset of 10 routine ED laboratory tests and age.
The score was temporally validated from July 2020 to October 2021, in the presence of vaccine roll-out and emergence of new SARS-CoV-2 variants.
The score was externally validated in three other centres in the Netherlands.
Missingness in the panel of laboratory tests varied between the external centres, limiting generalisability of the score to the ED population for which the complete panel of laboratory tests was available.
The score was not directly compared with lateral flow testing.
Introduction
COVID-19, caused by SARS-CoV-2, evolved into a global pandemic in 2020.1 For emergency department (ED) physicians, identifying presenting patients with a possible COVID-19 infection remains challenging since symptoms like fever, shortness of breath or coughing overlap with other illnesses.2 3 It is crucial, however, to identify a possible COVID-19 infection as early as possible. Early identification prevents further spreading and protects hospital staff by isolating a suspected patient, pending the results of a SARS-CoV-2 RNA PCR test and/or chest CT. Conversely, when PCR testing or isolation treatment capacity is limited, ruling out COVID-19 as soon as possible can save valuable resources.
In the era of electronic health records and clinical prediction models, developing an early warning score that can assist ED physicians in identifying patients presenting to the ED with COVID-19 is of great value. Moreover, if only routine ED test results are required as input, the score can be easily adopted by EDs worldwide, potentially reduce diagnostic costs and accelerate patient triage.
Many COVID-19 prediction models have already been developed; the living systematic review by Wynants et al4 provides an extensive overview and critical appraisal. Unfortunately, only few models have found their way into routine care at the ED.5 6 Early models were based on relatively small sample sizes, hampered by selection bias or were overfitted by selecting too many features.4–6 Aside from the methodological shortcomings of early models, most models are not developed as an early warning score for all ED patients. First, they require features from tests that are not routinely performed or logged for all ED patients (eg, the COVID-19 Reporting and Data System-score from a CT scan7 or non-laboratory-based clinical variables in the Pandemic Respiratory Infection Emergency System Triage Early Warning Score)8 and are therefore not straightforward to implement or scale to a large ED patient population. Second, the population on which models are commonly based are PCR-tested patients, that is, a preselection of a possible COVID-19 infection has already been done by physicians.
Only two studies were identified that focus on patients presenting to the ED, include unsuspected (and prepandemic) patients as controls and rely solely on routine (laboratory) tests.9 10
In this study we report the development and validation of an early warning score that, based on routine ED laboratory tests, estimates the risk of a possible COVID-19 infection in patients who undergo routine laboratory testing at presentation. The score can assist ED physicians in triaging patients and prevent further transmission of COVID-19 by quickly identifying possibly infected patients or ruling out a possible infection when resources are scarce.
Methods
Study design
This is a retrospective case–control study where routine laboratory test results, combined with age and gender, from all patients presenting to the ED of the Catharina Hospital Eindhoven from July 2019 to July 2020 were combined with SARS-CoV-2 PCR test results in a development data set. A model that could predict the presence of a COVID-19 infection was fit to this data set. The performance of the model was assessed by (1) internal validation, (2) temporal validation and (3) external validation by using data from the ED of three other centres.
Patient and public involvement
Patients were not involved in the design, conduct or reporting of this study.
Development data set
All ED presentations at the Catharina Hospital Eindhoven from July 2019 to July 2020 were included in the development data set, provided that routine laboratory testing had been requested by the attending ED physician. The rationale for this inclusion period is to limit the effect of seasonal variation in the ED patient population by including the summer, fall and winter seasons of 2019 (control patients) and the winter, spring and summer seasons of 2020 (case and control patients). The routine laboratory panel at the ED consists of 28 laboratory tests. In some cases not all tests in the routine panel were requested or one or more quantitative results were not available due to analytical interference (haemolysis, lipaemia or icterus). The routine ED laboratory panel is requested for (adult) patients presenting with abdominal pain, chest pain, shortness of breath, syncope, sepsis or other non-specific complaints, or for patients (including non-adult patients) presenting with specific complaints where a suspected diagnosis has to be ruled in or ruled out. Presentations with one or more missing values in any of the 28 laboratory tests in the routine ED panel were excluded. Presentations with one or more extreme laboratory results, >10 times the SD from the median, were also excluded to minimise the effect on the estimation of regression coefficients. The median was chosen as a measure of central tendency due to its resistance for outliers. After the first case of COVID-19 in the Netherlands, all patients with symptoms of COVID-19 (either fever and/or respiratory symptoms) were subjected to nasopharyngeal PCR testing for SARS-CoV-2 RNA. PCR testing was performed by commercial tests that were approved by the Dutch National Institute of Public Health (RIVM). If a patient had a positive PCR result in the past, subsequent presentations were excluded as re-presentations might be clinically different from de novo presentations.
The ED laboratory panel results were matched to SARS-CoV-2 PCR results if the underlying nasopharyngeal swab had been taken ≤1 day prior or ≤1 week after initial blood withdrawal at the ED. If multiple PCR tests were performed in this window and at least one PCR test was positive, the presentation was labelled ‘PCR-positive’. If all PCR test results in the time window were negative, the presentation was labelled as ‘PCR-negative’. If no PCR tests were performed in the time window and the presentation occurred after the first case of COVID-19 in the Netherlands, the presentation was labelled as ‘Untested’. All presentations before the first case were labelled as ‘Pre-COVID-19’.
Laboratory tests
The routine laboratory panel consisted of haemocytometric and chemical analyses. The haemocytometric tests were performed on Sysmex XN-10 instruments (Sysmex, Kobe, Japan) and consisted of haemoglobin, haematocrit, erythrocytes, mean corpuscular volume, mean cellular haemoglobin, mean cellular haemoglobin concentration, thrombocytes, leucocytes, neutrophils, eosinophils, basophils, lymphocytes and monocytes. The chemical analyses were performed on a Cobas 8000 Pro (Roche Dx, Basel, Switzerland) instrument and consisted of glucose, total bilirubin, aspartate aminotransferase (ASAT), alanine aminotransferase (ALAT), lactate dehydrogenase (LD), creatine kinase (CK), alkaline phosphatase (ALP), gamma-glutamyltransferase (gGT), blood urea nitrogen, creatinine, chronic kidney disease epidemiology collaboration (CKD-epi) estimated glomerular filtration rate (eGFR), potassium, sodium, chloride, albumin (bromocresol green) and C reactive protein (CRP). These results were combined with age and gender.
Modelling
All data were processed and analysed in R V.4.1.1.11 Laboratory results, combined with age and gender, were used as covariates in the regression model. Cases were defined as ED presentations labelled as ‘PCR-positive’; controls were all other presentations (ie, ‘PCR-negative’, ‘Untested’ or ‘Pre-COVID-19’). To achieve predictive accuracy, limit overfitting and perform feature selection, penalised logistic regression with an adaptive lasso penalty was chosen.12 13 To minimise missing data, all non-numeric results at the extremes of the measuring range were converted to numeric results by removing the ‘<’ and ‘>’ signs. For eGFR (CKD-epi) and CRP the raw precursor value was used instead of >90 mL/min/m2 and <6 mg/L, respectively. Considering that laboratory results of bilirubin, ASAT, ALAT, LD, CK, ALP and gGT can have heavy (right) tailed distributions, which in turn impact model predictions, these variables were transformed logarithmically. More details regarding model fitting can be found in online supplemental material 1. Models were fitted using the glmnet package.14
Supplemental material
CoLab-score
Since this is a retrospective case–control study, the sample prevalence may not reflect the true/current COVID-19 prevalence. To obtain well-calibrated probabilities, the intercept term in the model should be adjusted according to the current prevalence (details can be found in online supplemental material 1).15 However, adjusting the intercept term is not straightforward to implement in clinical practice; therefore, the linear predictor of the model was categorised into a score and this score is hereafter referred to as the ‘CoLab-score’. The categorisation is based on a number needed to test of 15 (ie, one is willing to PCR-test 15 patients to find one positive) and prevalence cut-points of 1%, 2%, 5%, 10% and 40% using the intercept adjustment formula by King and Zeng.15 The intervals obtained through these breaks correspond to CoLab-scores 5 to 0, respectively. A score of 0 reflects low risk for COVID-19 and a score of 5 reflects high risk. More details regarding the rationale of the CoLab-score categorisation can be found in online supplemental material 1.
Internal validation
To assess model performance while taking overfitting into account, bootstrapping was performed. From the original data, 1000 bootstrap samples were generated. On each bootstrap sample, full model fitting procedure and CoLab-score conversion were performed. Optimism-adjusted performance measures of the CoLab-score were obtained by applying the 0.632 bootstrap rule to the in-sample and out-of-bag-sample performance.16 Performance measures included the area under the ROC-curve (AUC), sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of each CoLab-score. The pROC package was used to calculate the performance measures.17 Although the full inclusion period from July 2019 to July 2020 was used for model fitting, the performance was evaluated on the period starting from the first COVID-19 infection (24 February 2020) to July 2020. This was done to obtain performance measures that would reflect real-world performance.
Temporal validation
For temporal validation, the results from our centre were prospectively analysed from July 2020 to October 2021. During this period, the Netherlands was struck by a second wave of COVID-19 infections, starting in the fall of 2020 and subsiding in the summer of 2021. In this period there was also more widespread external PCR testing by municipal health services. The results of external conducted PCR tests were not available for our study. To overcome this limitation, the outcome in the temporal validation cohort was chosen as a composite of the hospital registration of a confirmed COVID-19 infection and/or at least one positive PCR test result. This period also covers both the emergence of new SARS-CoV-2 variants as well as vaccine roll-out. However, neither vaccination status nor genomic sequencing was available to determine whether a patient was vaccinated or which variant caused the infection. Therefore, data from the Dutch National Institute of Public Health (RIVM) were used to divide the temporal validation period into three phases: (1) from July 2020 until March 2021, no vaccination and no variants of concern identified; (2) from March 2021 until June 2021, partial vaccination and B.1.1.7 (Alpha) variant identified as dominant; and (3) from June 2021 until October 2021, widespread vaccination and B.1.617.2 (Delta) variant identified as dominant. See figure 1 in online supplemental material 2 for more details. The temporal validation consisted of assessment of the AUC, sensitivity, specificity, PPV and NPV of each CoLab-score threshold for the entire period, as well as for each phase separately to determine a possible effect of vaccination and new variants on performance (results in online supplemental material 2). Model calibration was assessed graphically using the rms package.18
Supplemental material
Inclusion flow of patients (pts) in the development (A) and temporal validation (B) data set. All patient admissions with routine venous blood sampling at the emergency department (ED) were included. For the development data set, completeness of the laboratory panel was assessed for all 28 laboratory tests; for the temporal validation data set this was only necessary for 10 laboratory tests. The major causes of missingness are described in the text. In the development data set, presentations with extreme values (>10 SD) were excluded. The same limits were applied to the temporal validation data set (see table 2 for limits).
External validation
For the external validation, several centres in the Netherlands were approached and assessed if the required panel of laboratory tests and SARS-CoV-2 PCR test results were available. Seven centres responded and three centres fulfilled the inclusion criteria: Gelre Hospitals (centre 1), Atalmedial Diagnostic Centers, location Alrijne Hospital Leiderdorp (centre 2) and Zuyderland Medical Center (centre 3). The haematological parameters were measured with Sysmex XN-10/XN-20 (centre 1), CELL-DYN Sapphire (Abbott Laboratories) (centre 2) and Sysmex XN-10 instruments (centre 3). The clinical chemistry parameters were measured with Architect c14100/c160000 (Abbott Laboratories) (centre 1), Architect ci4100 (Abbott Laboratories) (centre 2) and Cobas 8000 instruments (Roche Dx) (centre 3). The external validation was similar to the temporal validation and consisted of assessment of the AUC, sensitivity, specificity, PPV and NPV of each CoLab-score threshold. Calibration was assessed graphically analogous to the temporal validation data set.
Results
Development data set
The study included 12 879 ED presentations of 10 327 patients from July 2019 to July 2020. After excluding cases with an incomplete laboratory panel, patient presentations that occurred after a positive PCR test in the past (re-presentations) and presentations with extreme values (>10 times SD) in any of the laboratory results, 10 417 presentations of 8610 patients remained (figure 1A).19
Descriptive statistics of ED presentations are shown in table 1, where the symbol ‘‡’ indicates a clinically relevant difference from the pre-COVID-19 category (based on the total allowable error20). For the PCR positives (n=279), 91% (95% CI 88% to 94%) of the cases were tested positive in their first PCR. The remaining 24 patients were positive in their second (n=18), third (n=5) or fourth (n=1) PCR.
Descriptive statistics of the development data set and laboratory concentrations
CoLab-score
The model obtained through adaptive lasso regression contained 11 variables, which are depicted with their regression coefficients (weights) in table 2.
Calculation of the CoLab linear predictor
Probability density plot of the CoLab linear predictor. The probability density plots for patients with COVID-19 (dark blue) and those without COVID-19 (light blue) are plotted against the linear predictor (see table 2). The CoLab-score cut-offs (−5.83, −4.02, −3.29, −2.34 and −1.64) are depicted with vertical dashed lines. The white-boxed numbers (between the cut-offs) represent the corresponding CoLab-score. Note that while the area under both curves is identical (since these are probability density functions), in absolute numbers the ‘negative or untested’ group is about 36 times larger than the PCR-positive group.
A larger β-coefficient does not imply that a variable is more important in predicting the odds of testing positive for SARS-CoV-2 since the variables are on different scales. The most important variables are basophils, eosinophils and LD.
As shown in figure 2, the linear predictor clearly discriminates between COVID-19 and non-COVID-19. The linear predictor is converted to CoLab-scores 0–5, with the cut-points depicted in figure 2.
Internal validation
The model was validated in the period starting from the first COVID-19 infection to July 2020, and in this period the mean prevalence was 7.2%. The AUC of the CoLab-score is 0.930 (95% CI 0.909 to 0.945).
Diagnostic performance is shown in table 3. A CoLab-score of 0 has an NPV of 0.997 (95% CI 0.993 to 0.999) and a PPV of 0.115 (95% CI 0.0934 to 0.147); one-third (38%, 95% CI 28% to 514%) of all ED presentations were assigned this score and can therefore be safely excluded. Conversely, 6% (95% CI 6% to 8%) of the ED patients had a CoLab-score of 5. Given the PPV of this score (0.683, 95% CI 0.628 to 0.746; NPV: 0.970, 95% CI 0.963 to 0.978), subsequent PCR testing is advised.
Bootstrapped diagnostic performance of the CoLab-score in the development data set
Temporal validation
As the CoLab-score was developed at our centre after the first COVID-19 wave in the Netherlands, the performance was evaluated at our centre from July 2020 until October 2021. Laboratory results from 17 489 ED presentations were collected. After applying the inclusion flow as shown in figure 1B and 14080 presentations remained, of which 1039 were associated with a COVID-19 infection.19
The mean prevalence in this period was 7.4%. The AUC of the CoLab-score in the temporal validation set is 0.916 (95% CI 0.906 to 0.927). The performance is comparable with the development cohort, although sensitivity is slightly lower and specificity slightly higher (cf, table 3 and table 4). The temporal validation data set was also split into three phases according to the dominant SARS-CoV-2 variants and vaccine roll-out (see figure 1 in online supplemental material 2). The discriminative ability was not lower in the second or third phase compared with the first phase. Diagnostic performance is preserved in terms of sensitivity and specificity, except a moderately reduced sensitivity of scores ≥3 in the third phase as compared with the first phase. PPV and NPV are incomparable due to different prevalence/pretest probabilities in each phase (see table 1 in online supplemental material 2).
Diagnostic performance of the CoLab-score in the validation data set (temporal) and three external hospitals
In terms of the predicted probabilities, model calibration shows that overall predicted probabilities are too low (see figure 1 in online supplemental material 3 for the calibration plot), which is expected since the prevalence differs and the intercept has to be adjusted to the prevalence.
Supplemental material
In this period at least 22 COVID-19-positive patients were identified by the CoLab-score, who initially did not present with COVID-19-specific symptoms. Most patients had neurological or orthopaedic presenting symptoms.
External validation
For external validation, data obtained from three other centres were used: centre 1 (n=1284, 52 COVID-19-positive), centre 2 (n=2899, 99 COVID-19-positive) and centre 3 (n=3545, 336 COVID-19-positive).19 The inclusion flow is summarised in figure 3. The COVID-19 prevalence differed among the three centres (4.0%, 3.4% and 9.5%, respectively) and was lower in centres 1 and 2 and higher in centre 3 than in the development data set. The AUCs of the CoLab-score are 0.904 (95% CI 0.866 to 0.942), 0.886 (95% CI 0.851 to 0.922) and 0.891 (95% CI 0.872 to 0.909) for centres 1, 2 and 3, respectively.
Inclusion flow of emergency department (ED) patients (pts) in three external centres. All ED presentations with routine venous blood sampling were included. Missingness of laboratory panels was assessed for the 11 variables in the CoLab-score (see table 2). Re-presentations after a positive PCR result or clinical COVID-19 registration were excluded as ‘previous COVID-19+’. Presentations with any laboratory result above the limits of the CoLab-score (see table 2) were excluded.
Diagnostic performance is shown in table 4. The sensitivity of CoLab-score 0 in all centres is ≥0.96. Therefore, the NPV of CoLab-score 0 was more than 99%. Calibration plots for external centres are shown in figure 1 in online supplemental material 3. The observed fraction of COVID-19 positives is slightly lower than expected in centres 1 and 2. For centre 3, low probabilities appear slightly underestimated and high probabilities slightly overestimated.
Discussion
Given the impact of COVID-19 on society and healthcare, there is a need for simple and fast detection of patients with a possible COVID-19 infection in the ED. The CoLab-score described in this study is a fast and accurate risk score to triage patients presenting to the ED based on 10 routine blood biomarkers and age.
The main strength of this study is that this score can be used as an early warning or triaging tool for the ED population presenting with abdominal pain, chest pain, shortness of breath, syncope, sepsis or other non-specific complaints where a routine blood panel is requested. This is in contrast to the vast majority of COVID-19 diagnostic models that have been developed on a preselected population of PCR-tested patients.9 21–27 Moreover, the CoLab-score requires only routine blood tests, instead of (features from) imaging such as CT scans or laboratory tests that are not routinely collected in the ED, for example, interleukin 6 or 3-hydroxybutyric acid.4 Compared with lateral flow tests (LFTs), which provide a dichotomous result within 30 min and are widely adopted in EDs, the CoLab-score is a continuous score. The lowest CoLab-scores (0–1) offer higher sensitivity and are therefore more suitable to rule out COVID-19 than LFT, which is only moderately sensitive (although more specific).28 29
Two other studies have been published which are similar to this study.9 10 Interestingly, the study by Soltan et al10 ranked basophils and eosinophils as the two most important features in predicting the outcome, similar to our results. Eosinophils were also seen as one of the most important features by Plante et al.9 However, both studies focus on an artificial intelligence/machine learning approach. While their approach likely results in higher predictive performance, due to the ability of machine learning models to capture non-linear and interaction effects, the goal of this study was to develop a simple, fast and robust model that can easily be implemented in current hospital information systems.
Since this is a retrospective case–control study, there are some unavoidable missing data. In our cohort 17.6% of the ED presentations could not be used due to one or more missing laboratory results. This is lower or equal to similar studies: 22%,24 17%22 and 11%.27 Important to note is that 7.7% of missingness is due to analytical errors, which can be assumed to be missing completely at random. For the remaining 9.9% of missingness, the full laboratory panel was most frequently missing for paediatric, obstetric and surgery patients. These patients are presenting with specific complaints for which specific laboratory tests are requested and hence do not match the inclusion criteria for a routine blood panel. Overall the missingness was significantly lower in the PCR-tested group versus the untested group (χ2 test p<0.001). It is assumed that all presentations in the untested group are COVID-19-negative. However, some presentations with asymptomatic COVID-19 could be present in the untested control group. The impact of these ‘false controls’ is most likely small as other studies indicate that there is a very low positivity rate among asymptomatic ED presentations (only a few in over 1000 tested asymptomatic cases).30 31 The vast majority of controls were not tested for COVID-19 because they were either prepandemic or untested patients (89% in the development data set). Clinical data always contain some unavoidable ‘noise’ in the form of misregistrations, misdiagnoses or patients who were missed. We have tried to mitigate this by including a large prepandemic control group and including all PCR tests within 1 week after discharge.
In the external centres, there is a high level of missingness as a result of an incomplete laboratory panel. In the case of centres 1 and 2, only internal medicine ED presentations were tested with a laboratory panel containing the 10 tests required for the CoLab-score. The ED laboratory panel of other disciplines (eg, urology, surgery or paediatrics) differed and did not contain the required tests. Nevertheless, the majority of patients with COVID-19 were internal medicine ED presentations, reflected by the few PCR-positive patients excluded. Due to these high levels of missingness, the results of the external centres cannot be used to show that the CoLab-score generalises to the entire ED population. Rather, the results show that for the majority of COVID-19-positive patients presenting to the ED, a routine laboratory panel is available from which the CoLab-score can be calculated and that the performance of the CoLab-score in this population is comparable with the development population. Differences in the distribution of CoLab variables between centres are shown in figure 2 in online supplemental material 3.
The performance of the CoLab-score is affected by the time between the onset of symptoms and ED presentations. The score increases with the duration of symptoms and gradually decreases after day 7 (see figure 1 in online supplemental material 4 for a plot of the duration of COVID-19-related symptoms and the CoLab linear predictor). As a consequence, some patients with COVID-19 with early or late presentation after onset of symptoms can be missed. Optimal performance of the CoLab-score is achieved when the onset of symptoms is >1 and <10 days prior to ED presentation. Chemotherapy that causes myeloid suppression will decrease neutrophilic, basophilic and eosinophilic counts and thereby ‘falsely’ increasing the CoLab-score. Conversely, patients with COVID-19 with severe anaemia could have ‘falsely’ lowered CoLab-scores. To minimise false negatives, we have therefore advised to report CoLab-scores only when the concentration of erythrocytes is ≥2.9/pL.
Supplemental material
It was chosen to exclude re-presentations after a previous presentation with COVID-19. Since the median time between initial presentation and re-presentation was 12 days, these patients were most likely not reinfected patients, but patients who deteriorated after initial presentation/treatment. Given that the CoLab-score follows the host immune response, the score is time-sensitive (see figure 1 in online supplemental material 4). Including these patients would impact the performance of the CoLab-score as patients in a later phase of the disease show different biomarker profiles. The CoLab-score is aimed towards alerting clinicians to patients presenting with a novel SARS-CoV-2 infection, rather than patients who deteriorate after treatment for COVID-19. Other re-presentations were not excluded, which results in some patients appearing multiple times in a data set. This was not adjusted for in the regression model since the assumption was made that ED presentations are independent observations. The median time between re-presentations is 38 days, most likely resulting in variations in laboratory results between presentations and hence little to no correlation between presentations. A sensitivity analysis was performed whereby only the first presentation was included for each patient (table 1 in online supplemental material 4) but no difference was found in performance in terms of sensitivity, specificity and AUC.
The CoLab-score does not serve as a replacement for PCR testing or LFT and can be used to guide PCR testing when routine blood tests are available. Important to note is that the CoLab-score is only valid for ED presentations where routine blood testing is requested, and as a consequence does not generalise to the ED population who is otherwise well and does not undergo routine blood testing. Using the CoLab-score in a symptomatic/PCR-tested cohort also results in different diagnostic performance characteristics, as compared with using the score on the full ED cohort (see table 1 in online supplemental material 4).
Finally, the CoLab-score could lead to false positives by other viral infections. However, in a historical patient cohort, the CoLab-score had only limited discriminative ability in separating influenza-PCR-negative from influenza-PCR-positive patients (see figure 2 in online supplemental material 4), implying specificity for SARS-CoV-2. Since the CoLab-score reflects the host response to the virus, it is hypothesised that the CoLab-score could also be sensitive to future SARS-CoV-2 variants. This is supported by the fact that the discriminative ability is sustained in periods with different dominant variants, although the sensitivity of scores ≥3 is somewhat lower in the third phase (see table 1 in online supplemental material 2). Although vaccination status is not registered for all presenting patients, in a small subgroup of 12 patients for whom vaccination status was registered and were COVID-19-positive, 8 of 12 patients had the highest CoLab-score (score=5) (see figure 2 in online supplemental material 2). Continuous assessment of the performance of the CoLab-score is required due to the emergence of new variants and changes in the host’s immune response.
To conclude, the CoLab-score developed and validated in this study, based on 10 routine laboratory results and age, is available within 1 hour for any patient presenting to the ED where routine blood testing is requested. The score can be used by clinicians to guide PCR testing or triage patients and helps to identify COVID-19 in patients presenting to the ED with abdominal pain, chest pain, shortness of breath, syncope, sepsis or other non-specific complaints where a routine blood panel is requested. The lowest CoLab-score can be used to effectively rule out a possible SARS-CoV-2 infection, the highest score to alert physicians to a possible infection. The CoLab-score is therefore a valuable tool to rule out COVID-19, guide PCR testing and is available to any centre with access to routine laboratory tests.
Data availability statement
Data are available in a public, open access repository. Data is de-indentified.
Ethics statements
Patient consent for publication
Ethics approval
The study was reviewed by the Medical Research Ethics Committees United (MEC-U) under study number W20.071, which confirmed that the Medical Research Involving Human Subjects Act (in Dutch: WMO) does not apply to this study. The study was thereafter reviewed and approved by the internal hospital review board.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
A-KB and RD contributed equally.
Contributors A-KB: guarantor, conceptualisation (lead), data curation (lead), funding acquisition (lead), investigation (equal), methodology (equal), supervision (equal), writing - original draft (equal), writing - review and editing (equal). RD: data curation (equal), formal analysis (equal), investigation (equal), methodology (lead), software (lead), visualisation (lead), writing - original draft (equal), writing - review and editing (equal). MMa, RHHvB, SB: conceptualisation (supporting), resources (supporting), supervision (supporting), validation (supporting), writing - review and editing (equal). HSMA: conceptualisation (supporting), resources (supporting), supervision (supporting), validation (equal), writing - review and editing (equal). WAHMT: conceptualisation (supporting), resources (supporting), supervision (supporting), validation (supporting), writing - review and editing (equal). ML, RJHM, MMB, JJK, MMe: resources (equal), validation (equal), writing - review and editing (equal). JJvS: resources (supporting), validation (supporting), writing - review and editing (equal). NAWvR: methodology (supporting), resources (supporting), supervision (equal), writing - review and editing (equal). VS: conceptualisation (equal), funding acquisition (equal), project administration (lead), resources (equal), supervision (lead), writing - review and editing (equal).
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.