To facilitate the identification of in-hospital patients who are acutely decompensating and may require critical care management, early warning scores (EWS) have been developed.1,2,3 These scores, based on changes in clinical status, including respiratory rate, heart rate, systolic blood pressure, and neurologic status, are designed to trigger the activation of critical care outreach teams (CCOT), who respond and attend to the decompensating patient.

Controversy exists around the effectiveness of the EWS and CCOT system.4 Multiple studies, including two randomized trials, have reported success in the ability of EWS to identify patients at risk of catastrophic deterioration and poor outcomes, but failed to show a difference in the need for intensive care unit (ICU) admission, cardiopulmonary arrest, or mortality.5,6,7,8,9 Moreover, a recent meta-analysis of the use of modified EWS for patients suffering from sepsis has brought into question the ability of these scores to predict mortality.10 Conversely, other studies have shown that the implementation of EWS and CCOT has been associated with a decrease in in-hospital cardiac arrests, a reduction in the number of patients receiving cardiopulmonary resuscitation prior to ICU admission, and an improvement in survival to hospital discharge.11,12,13,14,15,16

One possible explanation for these conflicting results is the way in which EWS are being acted upon.1,8 Data suggest that EWS are often used to quantify how sick a patient is rather than as a trigger for CCOT activation.17,18 As a result, delays in CCOT activation are common.19 Three recent studies have shown that a delay in CCOT activation is associated with worse patient outcomes.19,20,21 Nevertheless, none of these studies contained Canadian patients, limiting their external validity as it relates to the Canadian healthcare system. Furthermore, the impact of admitting service was not accounted for in the analyses. Surgical patients have been shown to have different outcomes after CCOT activation than medical patients, and the interaction between delay in CCOT activation and admitting service is unknown.6,22,23

The primary objective of this study was to determine if a ≥ 60 min delay from the time a patient was identified as meeting EWS criteria until the CCOT was activated impacted in-hospital mortality. The secondary objective was to assess differences in outcomes between medical and surgical patients where CCOT was activated.

Methods

Study design and patients

This was a historical cohort study of all CCOT activations for inpatients ≥ 18 yr of age at two academic tertiary care hospitals in London, Ontario, between 1 June 2007 and 31 August 2011. Although the CCOT is intended to be activated for hospital inpatients, it is occasionally activated for outpatients. As outpatients represent a different patient population, CCOT activations for outpatients were excluded from this study. All other CCOT activations were included. The study protocol was approved on 6 February 2013 by the Health Sciences Research Ethics Board at The University of Western Ontario.

The criteria and mechanism for the activation of a CCOT at both sites were identical (Fig. 1). Similar to the MERIT (Medical Emergency Response Intervention and Treatment) study, the CCOT is to be activated if any one of the EWS criteria are met.8 During the study period, admitted patients were monitored as per routine hospital policies. At both sites, inpatient vital signs are checked every four to six hours as well as at times of clinical concern. Although EWS criteria are not listed in the bedside chart, reminders are placed on all wards and it is expected that if a patient’s clinical condition meets EWS criteria, the CCOT is activated. Both of the included sites have a dedicated CCOT available 24 hr a day, seven days a week, comprised of a registered nurse, respiratory therapist, and a critical care physician. To activate the CCOT, a member of the patient’s healthcare team calls the CCOT nurse using a dedicated emergency phone number.

Fig. 1
figure 1

Early warning scores and criteria for critical care outreach team (CCOT) activation

Full size image

Data collection and definitions

Trained members of the CCOT recorded the time the EWS criteria for the CCOT activation were documented in the patient’s bedside chart as having been met, the time when the CCOT was called, primary reason for the call, outcome from the patient encounter, patient demographic characteristics, code status, and the patient’s primary care team. The type of service under which a patient was admitted was defined by the specialty of the primary care team. Trained research personal extracted the length of ICU and hospital stay, receipt of invasive mechanical ventilation, and in-hospital mortality from the patient’s health record and linked this information to the data captured by the CCOT using the patient’s unique identification number. All data related to the CCOT activation, including patient characteristics, were entered into the study database the day that the patient was assessed. Outcome data were extracted from the patient record after hospital discharge.

Delay in CCOT activation was defined as the duration of time elapsed between when a patient was identified as having met the EWS criteria and the time CCOT was activated. A clinically significant delay in CCOT activation was defined as a CCOT activation that occurred ≥ 60 min after the time a patient was documented as having met EWS criteria. This one-hour time frame was chosen based on data from patients with sepsis that showed an increase in mortality for each hour that appropriate antimicrobial therapy was delayed.24

Outcomes

The primary outcome was in-hospital mortality. Secondary patient outcomes included the need for ICU admission, need for invasive mechanical ventilation, ICU length of stay (LOS), and hospital LOS. The effect of delay on the need for ventilation was described two ways: if any ventilation was performed and if patients were ventilated for > 96 hr. Extended ventilation represents more complex management, and 96 hr was chosen as this is the current cut-off for extended mechanical ventilation as defined by the International Statistical Classification of Diseases (ICD).

Statistical analyses

Data were entered directly into a study-specific Microsoft Excel database (Microsoft Corporation, Redmond, WA, USA). Descriptive statistics were summarized using mean (standard deviation [SD]), median [interquartile range (IQR)], or proportional differences where appropriate. Normality was determined by review of kurtosis and skew (acceptable ranges -1 to +1), and distribution plots were visually assessed. Proportional differences were assessed using the Pearson Chi-squared statistic. Differences in continuous variables were assessed using an independent t test for normally distributed data, and the Kruskal-Wallis test was used for non-parametric data. The impact of delay in CCOT activation on mortality was analyzed on both a per-activation level and then a per-patient level using the first activation to account for death as a competing variable. Multivariable logistic regression accounting for repeated measures using generalized estimating equations (GEE) was used to determine the effect (odds ratio [OR] with 95% confidence interval [CI]) of delay in CCOT activation on mortality, ICU admission, and need for ventilation, adjusting for age, sex, hospital site, admitting service, code status, and EWS criteria including the absence of listed criteria. Similar linear regression models were used for hospital and ICU length of stay. All variables included in the models, aside from age, were treated as categorical variables. To evaluate the robustness of the relationship between time to CCOT activation and mortality, two sensitivity analyses were performed on the primary outcome. In the first sensitivity analysis, the multivariable logistic regression using GEE was repeated excluding all patients missing EWS criteria. In the second sensitivity analysis, the multivariable logistic regression using GEE was repeated using time to CCOT activation as a continuous variable.

A pre-specified secondary objective was to determine the impact of admission service on patient outcomes. Similar models were used to determine the effect of admitting service on mortality, adjusting for the delay in CCOT activation. All statistical analyses were done using STATA 12.0 (StataCorp LP, College Station TX, USA).

Results

Baseline characteristics

There were 3,133 CCOT activations for 1,684 (53.8%) medical patients and 1,449 (46.2%) surgical patients during the study period. Of these, 2,160 (68.9%) activations representing 2,013 unique patients occurred within one hour of the patient meeting EWS criteria, while 973 (31.1%) activations representing 948 unique patients occurred ≥ one hour after the patient had met EWS criteria (Fig. 2). Age and sex did not differ significantly between groups, whereas delays of ≥ 60 min were more common at one of the hospital sites (P < 0.001) and among medical admitting services (P = 0.04) (Table 1). Common reasons for CCOT activation were desaturation (21%), systolic blood pressure ≤ 90 mmHg (19%), and altered mental status (12%). Nevertheless, 1,555 (49.6%) activations were missing the indication for CCOT activation.

Fig. 2
figure 2

Flowchart of subjects examined in this study

Full size image
Table 1 Baseline activation characteristics at the time of CCOT activation compared between delay groups (n = 3,133)
Full size table

Impact of delay in activation on in-hospital mortality

Outcome data were available for all patients included in the study. In-hospital mortality was significantly higher in the group with a delay in CCOT activation of ≥ 60 min than in those with CCOT activation < 60 min (34.8% vs 29.4%, respectively; difference, 5.4%; 95% CI, 1.8 to 9.0) (Table 2). To account for the possibility of multiple activations per patient, we limited our analysis to the first CCOT activation. In this analysis, there were 2,853 unique activations with 1,954 (68.5%) occurring within one hour of the patient meeting EWS criteria. In-hospital mortality remained significantly higher in the group with a delay in CCOT activation (34.1% vs 28.5%; difference, 5.5%; 95% CI, 1.8 to 9.2). Furthermore, after multivariable regression adjusting for potential confounders deemed clinically relevant, the odds of mortality remained significantly higher among the group with a delay > 60 min (odds ratio [OR] 1.30; 95% CI, 1.08 to 1.56). Details of the full regression with covariates are shown in the Electronic Supplementary Material (ESM); (ESM Table 1).

Table 2 Multivariable regression analysis in CCOT activation assessing in-hospital mortality and intensive care utilization among subjects with delays < or > 1 hr (n = 3,133)
Full size table

The results were also robust after two sensitivity analyses: one in which all patients with a missing cause of CCOT activation were excluded, and another exploratory analysis where time was treated as a continuous value. In the multivariable regression model excluding all patients missing EWS criteria, the relationship between a delay in CCOT activation ≥ 60 min and in-hospital mortality was further accentuated (OR, 1.37; 95% CI, 1.09 to 1.72). For the multivariable regression modeling of the relationship between delayed CCOT activation and in-hospital mortality in which time was treated as a continuous variable, it was necessary to perform a log transformation of time to meet the assumptions of a linear relationship. This model also showed an association between an increase in the duration of the delay in CCOT activation and the odds of in-hospital mortality (OR, 1.07; 95% CI, 1.02 to 1.13). Due to the unintuitive nature of log time, we presented our final model with time as a categorical value.

Impact of delay in activation on ICU admission, invasive ventilation, and length of stay

Patients with a delay in CCOT activation ≥ 60 min were admitted to the ICU more often than those with CCOT activation < 60 min (47.5% vs 41.5%; difference, 6.0%; 95% CI, 2.2 to 9.8). This finding was confirmed by multivariable regression after adjustment for admitting service, site, age, sex, reason for CCOT activation, and code status (OR, 1.22; 95% CI, 1.07 to 1.47). Nevertheless, no difference was seen in the proportion of patients needing invasive mechanical ventilation after being admitted to the ICU (62.8% vs 67.9%; difference, − 5.1%; 95% CI, −0.3 to 10.5) or in the proportion of patients requiring prolonged ventilation (34.2% vs 32.8%; difference, 1.4%; 95% CI, −3.9 to 6.7). These findings remained constant after multivariable adjustment.

Outcome of patients based on admitting service

Regarding the effect of admitting service, multivariable logistic regression showed that after adjustment for delay, site, age, sex, reason for CCOT activation, and code status, surgical patients were half as likely to suffer in-hospital mortality compared with medical patients (OR, 0.46; 95% CI, 0.39 to 0.55). This was despite the finding that surgical patients were just as likely to be transferred to the ICU as medical patients (OR, 0.93; 95% CI, 0.80 to 1.08) and were more likely to require invasive mechanical ventilation once transferred to the ICU (OR, 1.30; 95% CI, 1.02 to 1.66) (Table 3).

Table 3 Multivariable regression analysis in CCOT activation assessing outcome differences between surgical and medical patients (n =3,133)
Full size table

Discussion

The implementation of EWS and CCOT was based on the premise that early intervention would mitigate downstream patient harm. Although the data on the impact of EWS and CCOT on patient outcomes are conflicting, multiple studies have shown that EWS and CCOT can decrease in-hospital mortality and evidence is starting to emerge regarding the impact of timely implementation.11,12,13,14,15,16,19,20,21 Our study is the first Canadian study to examine the relationship between a delay in the activation of CCOT and patient outcomes. Results from our study suggest that for adult patients in which the CCOT is activated, a delay in CCOT activation of ≥ 60 min after patients have met EWS criteria is associated with a 30% increase in the odds of morality on a per-patient level and that, regardless of delay, surgical patients for whom the CCOT is activated have a lower mortality than medical patients.

The results of this study are part of a growing pool of data strengthening the hypothesis that a delay in CCOT activation leads to worse patient outcomes.19,20,21 Nevertheless, previous studies have described a significantly higher OR for mortality than found in our study, ranging from 1.6–1.8.19,20,21 This may be due multiple differences between our study and those previously published. For example, the baseline mortality rates in the studies by Boniatti and Chen were significantly higher than the morality observed in our study.20,21 Additionally, our study was the only one to include the admission service as a variable in the adjusted model. We showed that surgical patients have lower odds of mortality regardless of their delay, and as the previous studies did not adjust for possible differences in surgical versus medical patients, we were not able to assess their possible influence on patient outcomes. Regardless of these differences, the evidence across all the studies suggests the same: as CCOT activation delays increase, so do the odds of poor patient outcomes.

The impact of a delay in CCOT activation provides a new lens with which to examine the conflicting results of studies looking at the effectiveness of EWS and CCOT. The beneficial impact of CCOT shown in some studies, but not others, may in part be related to differences in adherence to activation criteria. Given that the activation of CCOT was delayed by > 60 min in nearly a third of the patients in our cohort, it is clear that there is still room for improvement. These results closely mirror those found by Chen et al., which showed that although the implementation of criteria for activating a rapid response team was associated with a decrease in the proportion of delayed activation, delays remained common.20

The fact that a larger proportion of patients survived in the surgical group irrespective of length of delay prior to CCOT activation may be related to a number of factors. Patients selected for surgery may be healthier to begin with and, as such, have a higher chance of surviving a clinical deterioration. Additionally, patients in the surgical group may have had readily reversible pathology, and a subsequent surgery or intervention may have been able to address the underlying cause of their deterioration.

This study has multiple strengths. The relatively large sample size allowed for estimation of the effect of the delay in the activation of CCOT on the primary outcome after adjusting for a number of clinical variables. Second, the utilization of two separate hospitals increases the generalizability of the findings. Finally, the long study period helped ensure that the results are not representative of seasonal variability, such as the 2009 H1N1 pandemic, and mitigated any potential implementation bias.

One limitation of this study is related to the patient cohort. Similar to the other studies that examine delays in CCOT activation, we only have data for patients where CCOT was activated. As such, it is possible there was a significant number of patients who met the EWS criteria but were never identified or were identified but CCOT was not activated and improved without the involvement of critical care. If this is the case, the 30% increase in the odds of mortality shown in this study may overestimate the odds of mortality associated with a delay in CCOT activation.

The retrospective nature of this study also presented specific limitations. While the primary outcome (mortality) was not subjective, the two data abstractors were not blinded to the objective of this study. Likewise, we were only able to report what was documented in the patient chart. Although we were able to adjust for age, sex, site, admitting service, reason for CCOT activation, and code status, we were unable to account for the severity of the patient’s illness or comorbidities. In addition to the limited number of captured variables, the reason for CCOT activation and the patient’s code status were often missing. Nevertheless, when we excluded patients who were missing the reason for CCOT activation from our analysis, we showed an increase in the odds of in-hospital mortality associated with CCOT delay. This suggests that the inclusion of these patients in the analysis biased our results towards the null, further supporting the identified relationship between a delay in CCOT activation and in-hospital mortality.

The etiology of the delays in CCOT activation is currently unclear and likely multifactorial. Potential reasons may include a lack of confidence in the EWS/CCOT system, incomplete dissemination of information regarding CCOT, high workloads limiting the frequency of assessments, and a lack of structured prompts built into the bedside chart. Future initiatives focused on quality improvement should examine the adherence to activation criteria as well as barriers to their utilization in clinical practice. While works such as that done by Dummett et al.,25examining the impact of imbedded EWS within the patient chart, and Mohan et al.,26 assessing methods to improve physician judgement regarding triage decisions, are two such examples, further research is needed.

Conclusions

A delay in CCOT activation by > one hour was associated with an increase in the odds of in-hospital mortality and ICU admission. Nevertheless, irrespective of delay, surgical patients in which CCOT was activated had reduced in-hospital mortality compared with medical patients. Rates of CCOT delay and details of the admitting service should be included in the interpretation of studies examining the impact of EWS and CCOT activation and patient outcomes. Likewise, future endeavours to improve the function of CCOT should include quality improvement initiatives targeted at decreasing delays in activation.