Impact of computerized provider order entry (CPOE) on length of stay and mortality

INTRODUCTION

Background and significance

Over the last 15 years the Institute of Medicine,^1–3 the United States government,⁴ and others^5–11 have examined health care safety and efficiency, including electronic health records (EHRs). Despite these publications, the impact of EHRs on patient outcome measures remains uncertain. This study aims to expand on the previous studies’ understanding of the impact of computerized provider order entry (CPOE) on patient outcomes using commonly referenced measures such as length of stay (LOS) and mortality.

Anticipating that a single-system EHR would improve patient outcomes and efficiency and increase reimbursement, the University of Utah Health Sciences Center progressively implemented a system between 1999 and 2007 (Figure 1). The components of this EHR included laboratory and radiology results, dictated provider notes, and most nursing documentation, including the medication administration record. With a majority of EHR components in place and the introduction of CPOE in the EHR in May 2009, we had the opportunity and the responsibility to assess the impact of CPOE implementation at our institution (Figure 1).

Figure 1.

Open in new tabDownload slide

Electronic health record implementation timeline

Per institutional policy, computerized orders were to be entered by physicians, nurse practitioners, and physician assistants. Nurses were only allowed to enter orders in the event of an emergency. Although our preliminary analyses were promising, system-wide outcomes can be influenced by multiple factors. Therefore, we adopted an antecedent structure process methodology^12,13 to study the impact of CPOE implementation on national benchmarks such as LOS and inpatient mortality (Figure 2).

Figure 2.

Open in new tabDownload slide

Antecedent, structure, and process variables included in the study

MATERIALS AND METHODS

Design

This 5-year retrospective, pre-post design (2.5 years each) study was conducted at an academic medical center comprising a 450-bed medical and surgical hospital, a 50-bed cancer hospital, and a 90-bed psychiatric hospital. The total sample of 104 153 hospital admissions included 66 188 unique patients. The purpose of this study was to expand upon the existing literature by including additional structure and process covariates while taking advantage of the discrete CPOE implementation within a larger EHR implementation to evaluate the potential impact on mortality and LOS.

Seventeen covariates were included in the study design (Figure 3). These covariates were used in previous studies reporting CPOE outcomes on LOS or mortality^5–11 or were cited in the literature as being associated with LOS and/or mortality^14–20 and thus were potential confounders. Antecedent variables represent intrinsic patient characteristics and include demographics (age, gender, race, ethnicity, marital status) and potential confounders that can vary with each inpatient admission: smoking status, body mass index, state of home residence (by zip code), insurance type, and admission via the emergency department (ED), a surrogate measure for severity of illness. Structure variables model systematic processes in the health care system: patient care unit, private vs semiprivate room, case mix index, and palliative care status. Process variables reflect actions performed during care, including CPOE implementation (the independent variable), rapid response team activation, resuscitation team activation, and registered nurse hours.

Figure 3.

Open in new tabDownload slide

Selection criteria

Sampling

Following Institutional Review Board approval, hospital admissions for adult patients 18 years and older admitted for >24 hours with a status of inpatient were included in the study. Labor and delivery visits were excluded from the study because maternal and neonatal mortality are calculated differently.²¹

A discrete study observation included an inpatient admission/visit to 1 of 22 distinct patient care units. Visits were included in the pre-CPOE phase if the patient was admitted between October 1, 2006, and March 31, 2009 (n = 49 683). The post-CPOE phase included patient visits between June 1, 2009, and October 31, 2011 (n = 54 470). Both phases were 2.5 years in length. Visits during the CPOE implementation phase (April 1, 2009, through May 31, 2009) were excluded from the analyses (n = 2333) because of a rolling implementation schedule with ongoing changes in process during that phase.

Patients who expired were excluded from the LOS sample, resulting in a final LOS sample size of 89 818 observations and 58 242 patients (Figure 3). The final mortality sample included 104 153 observations comprising 65 113 distinct patients.

Statistical analysis strategy

Traditional statistical analyses were deemed inappropriate for the overall analyses, due to the clustering (nesting) of observations created by patients who had multiple visits. Mixed effects linear modeling procedures that accounted for clustering and other dependence were used for the overall analyses (A.L., G.D.). HPMixed procedure in SAS (v. 9.3) was used for analysis of LOS, a continuous dependent variable. Mortality was assessed with generalized linear mixed procedures in SPSS for Windows (v. 21.0). All variables were entered into the model and removed in a step-wise fashion until only statistically significant (P < .05) variables remained.

Two models were constructed for each dependent variable. The first model evaluated each of the 22 patient care units as a unique unit of analysis. An intermediate step grouped the units by the Centers for Disease Control’s National Healthcare Safety Network (NHSN) classification,²² reducing the number of units to 14. Similar unit types were further recoded as 1 of 5 major patient care unit types: medical/surgical, intensive care, oncology, psychiatry, or rehabilitation.

Due to excessive missing values, outdated documentation policies, and/or changes in coding, body mass index (60% missing) and case mix index (25% missing) were excluded from the final analyses, leaving 15 initial covariates in each beginning model.

RESULTS

Design

The study was designed with patient care unit as the unit of analysis. However, results varied by structure variables (Table 1). Statistically significant decreases occurred in both measures at the facility level. Differences between patient care units were noted when comparing units individually and when similar units were grouped. At the room level, statistically significant differences were found between private and semiprivate rooms for both LOS and mortality.

Sample characteristics

Almost two-thirds (62.5%) of the hospital visits involved unique patients who had a single visit to the hospital. The remaining 37.5% of visits were patients who had multiple visits during the study. Table 2 presents the percentages for each covariate by pre- and post-CPOE phase. Although most of the covariates showed statistical differences between the phases, the strength of association was weak. Cramer’s V was <0.2 for all variables except private room status, which had a value of 0.46, a moderate strength of association.²³ Given the very large sample size,²⁴ it was not unexpected that all covariates had a significant P-value. However, these preliminary exploratory analyses supported the decision to use generalized linear mixed models for data analysis.

Length of stay

The outcome variable, LOS, was not normally distributed and was therefore transformed to a more normally distributed variable using a Box-Cox nonlinear formula.²⁵ We verified that the transformed data were more normally distributed. All significance tests were run using the approximately normal transformed data. Because of the nonlinear transformation, additivity is not preserved and the transformed results lack a natural metric for interpretation.^25,26 To assist with interpretation, results were back-transformed (Table 3) using the inverse Box-Cox formula.

A total of 89 818 admissions were used in the analyses, pre n = 39 387 (43.9%) and post n = 50 431 (56.1%). The 28% increase in the number of admissions during the post phase was due in part to the addition of hospital rooms.

Two models were created, one based on individual patient care units (Table 1, row 4) and one based on grouped unit types (Table 1, row 2). Twelve variables remained significant predictors of LOS in both models: study phase, age, race, marital status, zip code, insurance type, private room status, smoking status, admitted via ED, palliative care status, code activated, and rapid response team activated. The 3 most significant predictors in both models were study phase (pre- or post-CPOE implementation), room status, and state of residence (Table 2).

When reviewing the results from the facility level, CPOE was found to be a significant predictor of LOS in both models, with a decrease of 0.90 days and 0.92 days (Table 3). The model that aggregated patient care units by type had a wider confidence interval (2.92, 9.31).

Also found at the facility level was a gradual decrease in LOS over the course of the study (Figure 4). Quarter 4, the quarter after nursing documentation was implemented, showed a decrease below the trend line. There was an increase in LOS in quarters 8 and 15, which coincided with the arrival of new graduate medical trainees, and quarter 17, which coincided with winter (seasonal variation).

Figure 4.

Open in new tabDownload slide

Mean LOS by quarter

Findings at the patient care unit level show that LOS varied by unit type and by individual unit. All psychiatric units had a decrease in LOS, one unit by almost 3 days (Table 4). All medical/surgical units averaged a decrease in LOS, while 4 of 5 intensive care units showed an increase in LOS.

Mortality

A total of 104 153 admissions were included, pre n = 49 683 and post n = 54 470, representing a 9.6% increase in admissions during the study phase. Covariates remaining in both mortality models as significant predictors of mortality were study phase, age, ethnicity, marital status, insurance type, private room status, and admitted via the ED. Smoking status had a weak influence and was significant only in the model that aggregated units by type. The 5 most significant predictors (Table 5) in the mortality models were study phase (pre- or post-CPOE), room status, admitted via the ED, ethnicity, and age.

Gender and state of residence were not statistically significant predictors of mortality in either model. Palliative care status, code team activation, and rapid response team activation magnified the results (tripling or quadrupling). During statistical analyses we discovered that these variables acted as intermediate outcomes (mediating variables) and were therefore removed from the analyses.

The number of patient rooms, private rooms, and admissions increased in the post-CPOE phase. Although mortality incidences also increased, from 833 to 955, mortality rate decreased, from 1.78% in the pre-implementation phase to 1.75% in the post-implementation phase. Furthermore, study phase (pre- or post-CPOE) remained a statistically significant predictor of mortality after accounting for confounders. After adjusting for confounders, the model evaluating patient care units individually (Table 3) showed a decrease in mortality of 3 deaths per 1000 observations (n = 54 470), or 162 deaths extrapolated over the study. Similarly, after adjusting for confounders, the model aggregating patient care units by type showed a decrease of 1 death per 1000 visits, or 54 fewer deaths over the 2.5-year post-implementation phase.

Quarters 1–10 represent the pre-CPOE phase and quarters 11–20 represent the post phase. April and May of 2009 are excluded from the analyses due to the extended implementation taking place during these months. Quarter 11 contains an extra month (June 2009) and represents the end of the implementation phase. Figure 5 shows that the pre-CPOE mortality rate ranged from 13.81 to 22.40, while the post-implementation rate ranged from 15.90 to 19.35, a decrease from 8.59 to 3.45. Figure 5 also shows pertinent events that occurred during the course of the study. In the pre phase, mortality spiked during the nursing documentation implementation, and again when the new residents arrived. In the post-CPOE phase, mortality fell from 21.11 to 17.20 immediately after the implementation phase and in each quarter when new residents arrived.

Figure 5.

Open in new tabDownload slide

Mortality rate per 1000 visits per fiscal quarter

DISCUSSION

Length of stay

Differences in LOS were associated with structure variables including facility, patient care unit, patient care unit type, and private or semiprivate patient room. At the facility level, LOS was shortened, on average, by almost 1 day. At the patient unit level, fluctuations were noted between patient care units. Differences were also found between private and semiprivate rooms.

The decrease in LOS at the facility level may represent the natural progression as LOS has decreased globally over the last 50 years.²⁷ Changes that could have influenced the trend include better diagnostic tools, an increase in knowledge, changes in insurance billing policies, increased use of minimally invasive procedures, and more appropriate use of home health, rehabilitation, and skilled nursing facilities. While it is likely that the above changes contributed to the decrease in LOS, CPOE was the most significant difference between pre and post phases and thus a contributor to the overall decrease.

Patients assigned a semiprivate room and those who resided in Utah were more likely have a shorter LOS. It is likely out-of-state patients were transferred to this facility because of the severity of their illness, whereas some local patients use this hospital for routine care. Patients assigned to semiprivate rooms going home sooner was an unexpected result, contradictory to common thought and other studies.¹⁶ This result may reflect an artifact, because immunocompromised and infectious patients are rarely assigned to semiprivate rooms. Excluding these covariates from the study would have left gaps in the analyses and not exposed a potentially controversial finding, and may in part explain contradictory results from previous studies.

Hypothesized benefits for discharging patients 1 day early include reduced risk of exposure to nosocomial infections, reduced risk of receiving incorrect medications, reduced financial burden on all parties, and increased hospital bed availability. Like previous studies evaluating process change,^28,29 CPOE reflects increased efficiency with placing, transmitting, and initiating patient orders, reducing LOS.

It is not possible to control for all the longitudinal changes in care; however, 15 variables were controlled for, supporting the finding that CPOE had an independent, statistically significant impact.

Mortality

Both models in this study showed a statistically significant decrease in mortality, while controlling for multiple confounders. The 5 most significant predictors of mortality were study phase (pre- or post-CPOE), admitted via the ED, room status, age older than 80, and ethnicity (Table 5). Logical associations are expected with admission via the ED and being >80 years old. However, this study discovered anomalous results regarding room status. Semiprivate room status was a predictor of fewer deaths and perhaps represents an artifact of patient assignment based on severity of illness. Ethnicity of not Hispanic/not Latino was associated with fewer deaths. Ethnicity as a predictor may have been related to a mandatory documentation change in the patient registration process during part of the post-CPOE phase, reflecting more accurate recording of this variable, and might not represent care delivered or be related to CPOE implementation.

Statistically significant reductions in mortality were noted at the facility level. Differences were found at the patient unit and room levels, again possibly explaining the results in previous studies.^5–11 Most previous studies used a single patient care unit or a limited subset of patient care units.

Most medical/surgical units had reduced mortality rate, while an opposite trend was observed in the ICUs. The results in the med/surg units may be representative of standards of care, in the form of diagnosis- or procedure-specific order sets created by medical content experts for the CPOE implementation. The increase in mortality noted in ICUs may be due to increased patient complexity, complexity that may not have been represented in the original CPOE order sets. These findings suggest the need to control for patient severity indicator in future studies and may help identify why differences occur between unit types. These findings, which make a direct comparison difficult, suggest that the choice of patient care units profoundly influences the findings and may explain the mixed results found in previous studies.^5–11

Two very interesting patterns were noted in Figure 5. The first pattern showed that mortality increased in the 3 months immediately preceding both nursing documentation and CPOE implementation (Figure 5). One might consider whether (a) mortality is sensitive to institutional changes and (b) the nursing care is negatively influenced by institution-level changes and/or EHR changes. Many disruptions occur before a large-scale implementation, including shifting floor nurses to a “super-user” role and sending nursing staff to offsite training. Mandatory overtime was also discussed, which must have had nurses anticipating working extra hours and wondering how that would impact their home lives. We know nurses are constantly disrupted³⁰ in their daily work, but we do not know if the added “changeruption” of an upcoming EHR implementation pushes them past their personal cognitive stacking load and ultimately impacts mortality rates. We encourage further exploration of this finding to determine if mortality is a sensitive outcome measure and/or institution-level “changeruptions” impact nursing staff and, ultimately, patient mortality.

The second observed pattern was a decrease in mortality in the post-CPOE implementation phase during the quarter when new physician residents arrived. In the pre-implementation period, mortality increased during these quarters (Figure 5). In the post-implementation period, there was an unexpected decrease in the quarter immediately following CPOE implementation. We speculate that the 350 go-live order sets provided enough structured information to bridge the gap between medical studies and resident experience. Another consideration is the timing of the implementation. The isolated CPOE implementation allowed the institution to focus on physician access, training, and order set updates.

These findings associated with mortality shed light on the mixed results found in previous studies and introduce facets on mortality as an outcome variable. The new results illuminate the need to continue researching EHR implementation, especially if nursing care might be compromised due to institutional changes and new medical trainee care improves.

Strengths and limitations

This study was unique in examining the impact of CPOE on patient outcome measures from the facility, patient unit, and individual patient levels. One of the key strengths is the large sample size, which included 3 hospitals, 22 patient care units, and more than 89 000 observations on over 58 000 patients, which makes it larger than any previous study. The study was also unique in the number of potential confounders used to challenge the association between CPOE and the outcome variables of LOS and mortality. An additional strength is the robust statistical modeling techniques that accounted for clustering, multiple patient visits, and non-normal distributions.

The data for this study were collected from a single academic medical center; other institutions may have different findings. Other studies relied on a variety of study designs and sites used diverse EHR implementation strategies, making it difficult to directly compare the findings. However, the results from this study may partly explain the lack of consistency in previous studies.

Determining if facility- or institution-level changes impact nursing and patient outcomes is worthy of future study. Exploring the potential impact of CPOE on new residents should lead to further order set analysis and could potentially lead to changes in medical training and resident orientation.

This retrospective study did not examine the effects of differences in a patient care unit’s culture, policies and procedures, management style, patient acuity and comorbidities, or training. This study did not account for changes in government regulations and requirements or in insurance policies, or the advancement of medical and nursing sciences. Additionally, changes due to the use of minimally invasive procedures, outpatient surgeries, increased referrals to home health services, skilled nursing facilities, and rehabilitation units were not part of the analysis. Future studies may want to explore the unit effect by including indicators of severity such as the Case Mix Index,³¹ Apache II³² scores, or the Comprehensive Severity Index.³³ Future studies should consider adding 30-day readmissions, deaths occurring within 30 days of discharge, and comparing single-visit patients with multiple-visit patients.

CONCLUSIONS

To our knowledge, this is one of the first studies to evaluate the relationship between CPOE implementation, LOS, and mortality from multiple structural levels. This study generated expected and unexpected results. Statistically significant reductions in both length of stay and mortality were expected and found in the post-CPOE phase. Patient care unit had a differential effect on both length of stay and mortality, and while this was an unexpected result, it is a logical conclusion that may explain conflicting results from previous studies. Unexpected results include an increase in mortality in the 3 months preceding the 2 major implementations, revealing potential “changeruptions” to the nursing staff, and the repeated decrease in mortality in the quarters corresponding to new resident arrival. A third unexpected result is the association noted with semiprivate rooms and decreased LOS and mortality. Although the reduced LOS and mortality could be directly attributed to CPOE, based on a retrospective observational study alone, the results from this study illustrate the need to further explore the impact of CPOE implementation at the hospital system, individual patient care unit, and room levels.

DISCLOSURE

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sector.

The authors have no competing interests to declare.

ACKNOWLEDGMENTS

The authors would like to thank Chris Brooks and Chris Martineau for their assistance with tables and figures.

AUTHOR CONTRIBUTORS

Conception and study design: A.M.L, K.A.S., V.G.D., J.T., M.A.P.

Acquisition of data: A.M.L., V.G.D.

Analysis and/or interpretation of data: A.M.L., K.A.S., M.A.P., G.W.D., V.G.D., J.T.

Drafting the manuscript: A.M.L., V.G.D., K.A.S., M.A.P., G.W.D.

Revising the manuscript critically for important intellectual content: A.M.L., K.A.S., V.G.D., M.A.P., G.W.D., J.T.

Approval of the version of the manuscript to be published: A.M.L., K.A.S., V.G.D., M.A.P., G.W.D., J.T.

REFERENCES