FormalPara Key Points

This study synthesises data from two multi-centre studies in different countries that adopted similar methods totalling 3172 observations of intravenous medication administrations.

Given differences in the adoption of advanced technologies such as smart pumps, error rates and severities are surprisingly similar, with some differences in error types.

The findings are consistent with viewing intravenous medication as a complex adaptive system, in which interventions do not have a simple deterministic effect, but locally appropriate interventions can improve patient safety.

1 Introduction

Intravenous medication administration practices involving infusion devices have been identified as a source of errors, potentially compromising patient safety [1]. The practice of using smart infusion devices, where the device is integrated with information systems and drug libraries to set safe limits on medication administration, has been advocated to block critical medication administration errors [2,3,4].

In a systematic review, Keers et al. [5] note wide variations in definitions of a medication administration error, contexts in which they have been studied, and methods used to measure error rates. Hertzel and Souza [6] also note wide variation in methods across studies of smart infusion devices, with some reporting a positive effect in reducing errors and others no effect. They highlight the importance of staff training and full implementation of smart features for reducing errors, noting that the main source of errors is staff overriding smart features. They conclude that smart infusion devices “could prevent medication errors and save lives when properly configured and used”. Ohashi et al. [7] make similar points; they highlight opportunities for improvement including standardising and maintaining drug libraries, and enforcing hard limits to minimise workarounds. Reporting on smart infusion device implementation in one hospital, Keohane et al. [8] highlight the importance of creating a culture in which staff value their own competence and patient safety, and in which technology is easy to use correctly and implementation is viewed as a process of continual upgrade. Schnock et al. [9] introduced an infusion safety intervention bundle in nine United States (US) hospitals comprising eight interventions to address areas of medication labelling and tube tagging, administration of unauthorised medications and smart infusion device use. Different components of the bundle were adopted and adapted by different hospitals, and local contextual factors influenced uptake and effectiveness of the interventions. Changes in error rates were statistically significant in some cases, but not in most.

Most studies of the role of infusion devices in ensuring safe medication administration practices have taken place in the US [10, 11]. Fewer have taken place in the UK [12, 13]. Cousins et al. [14] conducted an audit across six hospitals in three different countries in Europe (England, France and Germany) and highlighted uncontrolled risks but did not conduct an explicit comparison among countries. Wirtz and Barber [15] conducted a comparative study across three large teaching hospitals in the UK and Germany with different drug supply and pharmacy services, applying an observational method to count and classify errors; the focus was on the effects of different kinds of service on error rates and types rather than the country in which observations took place. Dean et al. [16] compared non-intravenous medication administration errors between the US and England. No previous study has compared intravenous medication administration practices and errors between the US and England. Smart infusion devices have been widely implemented in the US [17]. Take-up in England has been patchy, and even where smart devices have been introduced, drug libraries may only be partially implemented [18].

Our aim was to better understand differences in practices and errors around intravenous medication administration in the US and England, based on our experiences in conducting parallel studies in each country.

2 Method

We conducted a comparative analysis of parallel studies from the US and England [11, 13]. The methods are presented in detail elsewhere [11, 19]. In brief, each study recruited a number of hospitals, selected to provide generalisable findings. Within each hospital, data were collected on intravenous medication administration across three to five clinical areas (Table 1). Data were collected in each area over approximately a day; this typically resulted in data on 30–40 administrations per area in each hospital. Data collection involved two trained observers (typically a nurse and a pharmacist from that hospital) visiting each clinical area, checking every intravenous medication being administered or scheduled to be administered at the time of their visit, comparing details of what was being administered and its documentation with the medication order and relevant national and local policy, and entering these data into a REDCap database [20]. The observers worked together to validate each other’s work and to agree on error type(s), as listed in Table 2, and a severity rating for every error observed, based on an adapted US National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) Medication Error index (Table 3). All data were checked and cleaned where needed. The original NCC MERP index [21, 22] classifies errors according to their outcomes, so the main adaptation (across both countries) was that the consequences of errors had to be anticipated, since errors were corrected where possible at the point of observation (see [19] for details).

Table 1 Planned differences in methodology
Full size table
Table 2 Errors as percentage of infusions observed across contexts and infusion device types
Full size table
Table 3 Adapted National Coordinating Council for Medication Error Reporting and Prevention medication error severity index
Full size table

A further adaptation for the England study was that the least severe deviations (category A) were classified as discrepancies rather than errors and further subdivided into A1 and A2 to provide a more fine-grained account of deviations from protocol. This was done in response to preliminary discussions with clinicians at participating sites who regarded such deviations as normal practice: many were workarounds that contributed to, rather than undermined, patient safety (although the factors that provoked the workarounds might compromise patient safety at a larger scale). In the US, these were classed as errors, reflecting US Joint Commission definitions [23]. For comparative purposes in the analyses reported here, all are considered errors.

The primary analysis in each national study focused on the proportions of errors of different types and severities, across different clinical settings and (in England) for different modes of administration (gravity, or through a traditional or smart pump).

The US study was conducted in 2013–2014 and the England study in 2015–2016. The England study used the same methods for data collection and analysis wherever possible, with adaptations as summarised in Table 1. The US study had set out to recruit sites that used smart infusion pumps whereas the England study (where such pumps are not widely used) recruited sites that used both smart and traditional pumps (and gravity feed was also used at several participating sites). There were also differences in the clinical areas observed, because it is not common practice in England to separate medical and surgical ICUs and the funders of the England study stipulated that paediatrics and oncology day care should be studied (as areas that make extensive use of infusion pumps, and where errors might have particularly severe consequences).

Once the separate studies had been completed, members of both study teams convened to identify themes for comparison across the two studies. Initially, the well-sorted tool (https://www.well-sorted.org/) was used to enable all authors to individually identify and collectively agree key themes, which were then honed through rounds of discussion. We conducted comparative analyses of data from both studies, including the prevalence and types of errors, to better understand factors likely to be related to differences across countries, levels of technological maturity, or other factors.

3 Results

We present the results of this comparative study under three themes: commonalities and differences in findings from the two study contexts; nuanced differences in methods that became apparent through the process of comparing findings; and differences in contextual factors that were tacitly known prior to the study, but whose significance became apparent through discussions.

3.1 Commonalities and Differences in Findings

Table 2 shows a comparison between the US and England studies for the different types of error detected.

Documentation errors were not recorded as a separate category in the US study: computerised physician order entry (CPOE) had been implemented at most US study sites, which enforced compliance with most documentation requirements. Differences in the data for labels not being completed according to policy and tubing not being tagged according to policy can largely be accounted for by differences in policy. There is a more detailed expectation of what should be included on labels in the US, as specified by the Joint Commission [24], although these guidelines may be interpreted differently by different hospitals. In England, policies are written locally, so there is more substantial variation among study sites [25].

Patient identification errors (i.e. omission of a record of having identified a patient) were higher in the England study than the US study; the main contributory factor is likely to be the widespread use of barcode medication administration (BCMA) in the US study, which makes it more difficult to administer medication without formal identification.

The category of ‘infusion device setting error’ was not included in the England study as all errors were encoded by the resulting error type rather than their cause, and no errors of ‘incomplete or delayed completion’ were recorded in the US, as any such errors were classified under another heading (e.g. omission or delay).

The majority of errors in each country were classified as A–C according to the adapted NCC MERP severity index (Table 3). Very few (n = 25; 0.79% of all infusions) were classified as D (“an error occurred that would be likely to have required increased monitoring and/or intervention to preclude harm”). Across all 3172 observations, only two (one in the US, one in England) were assessed to be level E (“an error occurred that would be likely to have caused temporary harm”). None were F (“an error occurred that would be likely to have caused temporary harm and prolonged hospitalisation”) or higher.

Table 4 summarises the error types for the more serious errors (categories D and E). Error types are omitted from this table if no errors in these categories were observed. This shows that the greatest proportion of category D errors concerned English gravity feed infusions running at the wrong rate. The category E errors concerned one delay (US, smart infusion device) and one concentration error (England, gravity administration).

Table 4 Number of errors classified as severity categories D or E (shown as ‘E’) across study contexts and types of infusion devices
Full size table

Overall, the commonalities across the US and England studies are much more striking than the differences, and the use of smart infusion devices makes little consistent difference across contexts.

3.2 Nuanced Differences in Methods

As noted above, the process of comparing findings across the study contexts revealed unplanned variations in data collection methods as well as those that were planned. These unplanned variations are listed in Table 5.

Table 5 Differences in methodology that were identified through the comparison
Full size table

Some of the variations were incidental and concerned minor differences in the ways that the study protocol was interpreted. For example, some ‘wrong rate’ errors were classified as ‘infusion device setting’ errors by the US team, where the infusion device setting caused the wrong rate; conversely, the English team recorded roller clamp positioning errors (principally with gravity feed), which would have been classified by the US team as wrong rate errors.

Some variations were also a consequence of the use of different technologies. For example, some kinds of paper documentation omissions observed in England were blocked by CPOE/BCMA technology in the US, and were therefore recorded in the England study but not considered in the US study.

3.3 Differences in Contextual Factors

There were some key factors across contexts and legislatures that shaped differences in methods and outcomes.

The first was technology implementation. In the US study, all sites had implemented electronic health records (EHRs), and most BCMA and CPOE, and intravenous infusions were routinely scanned before administration. In the England study, some sites had implemented EHRs, and some CPOE. Few had implemented smart infusion devices with drug libraries across all clinical areas; more had implemented smart infusion devices in selected areas (most commonly critical care); others were using traditional infusion devices and gravity feed. The implementation of EHRs and CPOE lags behind in England, although it is now progressing rapidly [26].

A second was the use of drug libraries. In the US, vendors supply drug library packages, but there is, nevertheless, large variation among hospitals [27]. In England, sites that made use of a drug library reported having constructed it locally; this was typically a pharmacy responsibility.

At a national policy level, in the US, The Joint Commission guidelines drive policy [28], although they are often interpreted differently by different hospitals. In England, the Care Quality Commission expects every hospital to have relevant policies [29], but does not mandate their content.

Finally, in terms of local prescription policies, in the US, fluids might be given without always requiring a repeated medication order (as long as there is an original order for that fluid), and keep vein open (KVO) administrations may run without a medication order (according to a standing hospital policy); verbal orders are discouraged by The Joint Commission but are sometimes used. In England, there is variation in local policy about what can be given without prescription and KVO practices are rarely reported.

4 Discussion

This combined study draws on the observation of 3172 infusion administrations. As found in the England study alone [13], the only clear conclusion is that medication administered by gravity feed is more likely to result in patient harm than medication administered via infusion devices. A dominant theme across contexts was the variability within and between sites, particularly relating to intravenous medication administration policy [11, 13].

We found less variability in error rates or types across the studies than might have been expected, given the differences in the use of smart infusion devices. This suggests that introducing more sophisticated technology without reviewing the policies and practices that surround those technologies does not necessarily improve patient safety.

This comparison reinforces the findings reported by Furniss et al. that there are many sources of variability in practice, and while some of those are essential responses to local circumstances, others introduce unnecessary complexity [25]. In the UK, multiple organisations (e.g. the National Institute for Health and Care Excellence, the Royal College of Nursing, the Royal Pharmaceutical Society) publish guidelines and standards for intravenous medication administration, but these are typically subject to interpretation. Although in the US The Joint Commission has published more tightly defined standards, those are still interpreted in different ways across contexts. One promising area for future review is the setting and interpretation of standards and policies to ensure that they are fit for purpose and support safe practices.

Arguably, the levels of safety observed in these studies have been achieved through vigilance, system improvements, and the diligence and professionalism of staff. Others have discussed CPOE as a complex adaptive system [30, 31]; we have not identified any previous studies that have discussed intravenous infusion administration in these terms; however, this comparative study highlights the many sources of variability in intravenous infusion administration across contexts, and suggests intravenous infusion administration is also a complex adaptive system.

The established literature on complex systems in healthcare repeatedly points out the futility of trying to reduce any complex adaptive system to a set of independently addressable problems [32, 33]. Braithwaite et al. [33] argue that, despite their apparent potential for chaos, complex adaptive systems self-regulate through the actions of people who develop structures and routines that create stability and acceptable performance within the system. The challenge with such systems is to ensure that performance remains acceptable over time, and where possible improves, as external pressures change.

The introduction of smart infusion devices, standards, or the interventions introduced by Schnock et al. [9] are not instant solutions to the problem of intravenous infusion administration errors, and may be better considered as ‘probes’ to better understand what does and does not work in a particular context, and what might work well in the future [34]. Thus, the question is not whether smart infusion devices are safer than traditional infusion devices, whether particular standards are optimal, or whether a particular intervention will universally lead to safer practices, but whether particular configurations of technology, procedures, practices and people are likely to enhance or diminish safety. Future technologies and policies need to support staff in doing their work effectively. One focus for future research should be how to optimise configurations of work system elements across sites and clinical areas.

4.1 Limitations

Any international comparison unavoidably introduces confounding factors due to differences in culture and practices. We aimed to minimise the differences in data collection that could be attributed to the training of the data collectors and analysts; however, small differences remain. We have summarised all the differences identified in Tables 1 and 5.

The US study had recruited only sites that had already implemented smart pumps across all clinical areas, so we have no evidence on the contributions of different pump types in the US context.

As noted above, neither the US nor the England study identified any events where patient safety was significantly compromised; it is more likely that such incidents would feature using a different method that focuses on reports of more serious incidents involving medication administration or potentially the analysis of large numbers of infusion device logs. Such a study, including rich causal accounts of factors contributing to serious untoward incidents, combined with a systematic risk analysis of ‘low frequency but high consequence’ events, might show benefits of more advanced technology solutions.

5 Conclusions

We compared the findings from separate studies in the US and England that aimed to mirror each other and identified both similarities and differences in error rates and practices around intravenous infusion administration. Considering the different levels of technological maturity across the study contexts, the observed error rates are surprisingly similar: there are few serious errors, but several errors of some importance, and some differences in the potential underlying causes of errors. No one intervention, whether related to novel technology, adoption of standards, or particular policy changes, has a deterministic effect on behaviour or error rates, but together a set of interventions, appropriate to local circumstances, can support and guide staff to behave as safely and effectively as possible.