Interventions for reducing wrong‐site surgery and invasive clinical procedures
Abstract
Background
Specific clinical interventions are needed to reduce wrong‐site surgery, which is a rare but potentially disastrous clinical error. Risk factors contributing to wrong‐site surgery are variable and complex. The introduction of organisational and professional clinical strategies have a role in minimising wrong‐site surgery.
Objectives
To evaluate the effectiveness of organisational and professional interventions for reducing wrong‐site surgery (including wrong‐side, wrong‐procedure and wrong‐patient surgery), including non‐surgical invasive clinical procedures such as regional blocks, dermatological, obstetric and dental procedures and emergency surgical procedures not undertaken within the operating theatre.
Search methods
For this update, we searched the following electronic databases: the Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register (January 2014), the Cochrane Central Register of Controlled Trials (The Cochrane Library 2014), MEDLINE (June 2011 to January 2014), EMBASE (June 2011 to January 2014), CINAHL (June 2011 to January 2014), Dissertations and Theses (June 2011 to January 2014), African Index Medicus, Latin American and Caribbean Health Sciences database, Virtual Health Library, Pan American Health Organization Database and the World Health Organization Library Information System. Database searches were conducted in January 2014.
Selection criteria
We searched for randomised controlled trials (RCTs), non‐randomised controlled trials, controlled before‐after studies (CBAs) with at least two intervention and control sites, and interrupted‐time‐series (ITS) studies where the intervention time was clearly defined and there were at least three data points before and three after the intervention. We included two ITS studies that evaluated the effectiveness of organisational and professional interventions for reducing wrong‐site surgery, including wrong‐side and wrong‐procedure surgery. Participants included all healthcare professionals providing care to surgical patients; studies where patients were involved to avoid the incorrect procedures or studies with interventions addressed to healthcare managers, administrators, stakeholders or health insurers.
Data collection and analysis
Two review authors independently assesses the quality and abstracted data of all eligible studies using a standardised data extraction form, modified from the Cochrane EPOC checklists. We contacted study authors for additional information.
Main results
In the initial review, we included one ITS study that evaluated a targeted educational intervention aimed at reducing the incidence of wrong‐site tooth extractions. The intervention included examination of previous cases of wrong‐site tooth extractions, educational intervention including a presentation of cases of erroneous extractions, explanation of relevant clinical guidelines and feedback by an instructor. Data were reported from all patients on the surveillance system of a University Medical centre in Taiwan with a total of 24,406 tooth extractions before the intervention and 28,084 tooth extractions after the intervention. We re‐analysed the data using the Prais‐Winsten time series and the change in level for annual number of mishaps was statistically significant at ‐4.52 (95% confidence interval (CI) ‐6.83 to ‐2.217) (standard error (SE) 0.5380). The change in slope was statistically significant at ‐1.16 (95% CI ‐2.22 to ‐0.10) (SE 0.2472; P < 0.05).
This update includes an additional study reporting on the incidence of neurological WSS at a university hospital both before and after the Universal Protocol’s implementation. A total of 22,743 patients undergoing neurosurgical procedures at the University of Illionois College of Medicine at Peoria, Illinois, United States of America were reported. Of these, 7286 patients were reported before the intervention and 15,456 patients were reported after the intervention. The authors found a significant difference (P < 0.001) in the incidence of WSS between the before period, 1999 to 2004, and the after period, 2005 to 2011. Similarly, data were re‐analysed using Prais‐Winsten regression to correct for autocorrelation. As the incidences were reported by year only and the intervention occurred in July 2004, the intervention year 2004 was excluded from the analysis. The change in level at the point the intervention was introduced was not statistically significant at ‐0.078 percentage points (pp) (95% CI ‐0.176 pp to 0.02 pp; SE 0.042; P = 0.103). The change in slope was statistically significant at 0.031 (95% CI 0.004 to 0.058; SE 0.012; P < 0.05).
Authors' conclusions
The findings of this update added one additional ITS study to the previous review which contained one ITS study. The original review suggested that the use of a specific educational intervention in the context of a dental outpatient setting, which targets junior dental staff using a training session that included cases of wrong‐site surgery, presentation of clinical guidelines and feedback by an instructor, was associated with a reduction in the incidence of wrong‐site tooth extractions. The additional study in this update evaluated the annual incidence rates of wrong‐site surgery in a neurosurgical population before and after the implementation of the Universal Protocol. The data suggested a strong downward trend in the incidence of wrong‐site surgery prior to the intervention with the incidence rate approaching zero. The effect of the intervention in these studies however remains unclear, as data reflect only two small low‐quality studies in very specific population groups.
PICOs
Plain language summary
Interventions for reducing wrong‐site surgery
Wrong‐site surgery is a rare, but serious event that can have substantial consequences for patients and healthcare providers. It occurs when a surgical or invasive procedure is undertaken on the wrong body part, wrong patient, or the wrong procedure is performed. A number of interventions to reduce surgical error or prevent WSS, mainly involving pre‐operative verification, such as the development of Universal Protocol, site marking and 'time‐out' procedures have been proposed over recent years. This updated review contains two interrupted‐time‐series (ITS) studies (studies in which data are collected at multiple time points before and after an intervention), one from the original review, which evaluated a targeted educational intervention aimed at reducing the incidence of wrong‐site surgery, and which was found to reduce its incidence. An additional study evaluated the incidence of wrong‐site surgery before and after the introduction of the Universal Protocol, however the relevance of these findings regarding the impact of the intervention is unclear given that prior to its introduction, the incidence was decreasing due to other unclear factors. Overall, this review now contains two studies, of relatively low quality evidence, on very specific populations and their generalisability to a larger audience is low.
Authors' conclusions
Summary of findings
| Educational training programme to prevent wrong‐site tooth extraction | ||||
| Patient or population: Dental patients requiring tooth extraction Settings: Outpatient department of a university hospital, Taiwan Intervention: Educational training programme (Professional intervention) Comparison: Not applicable | ||||
| Outcomes | Change in level/slope | No of | Quality of the evidence | Comments |
| Annual incidence rates of wrong‐site tooth extraction | Change in level: ‐4.52%; 95% CI ‐6.83% to ‐2.21%; SE 0.53; P < 0.05
Change in slope: ‐1.16; 95% CI ‐2.22 to ‐0.10; SE 0.24; P < 0.05 | 1 ITS study | Low1 | Data re‐analysed |
| GRADE Working Group grades of evidence | ||||
| 1. Quality of the evidence: findings of the included study were from a single institution and were specific to the individual patient population. Therefore, the generalisability and applicability of the results was questionable with the need for caution to be exercised in applying the results to other settings. This research provides some indication of the likely effect. However, the likelihood that it will be substantially different is high. | ||||
| Implementation of Universal Protocol | ||||
| Patient or population: Patients having cranial or spinal neurosurgery Settings: Inpatients at the Department of Neurosurgery, University of Illinois College of Medicine, Peoria Intervention: Implementation of the Universal Protocol Comparison: Not applicable | ||||
| Outcomes | Change in level/slope | No of Participants | Quality of the evidence | Comments |
| Annual incidence rates of wrong‐site neurosurgery | Change in level in percentage points (pp) : ‐0.078 pp; (95% CI ‐0.176 pp to 0.02 pp); SE: 0.042; P value = 0.103 Change in slope: 0.031; (95% CI 0.004 to 0.058); SE: 0.012; P value < 0.05 | 1 ITS study (7286 operations before the intervention; 15,456 after the intervention) | Low1 | Data re‐analysed |
| GRADE Working Group grades of evidence | ||||
| 1. Quality of the evidence: findings of the included study were from a single institution and were specific to the individual patient population. Therefore, the generalisability and applicability of the results was questionable with the need for caution to be exercised in applying the results to other settings. This research provides some indication of the likely effect. However, the likelihood that it will be substantially different is high. | ||||
Background
Description of the condition
Wrong‐site surgery (WSS) is defined as surgery undertaken on the wrong person, the wrong organ or limb, wrong side or the wrong vertebral level, and can encompass invasive procedures such as regional blocks, dermatological, obstetric and dental procedures along with emergency surgical procedures not undertaken within the operating theatre. These critical errors are rare but often have major consequences for affected patients, practitioners and healthcare organisations. As a result, there has been much work done to determine which specific risk factors contribute to WSS, and if modifiable, whether WSS might be preventable (Ammerman 2006; Canale 2005; DeVine 2010; Giles 2006).
Risk factors that have been systematically identified in the literature as contributing to WSS include incorrect patient positioning or preparation of operative site; patient or family providing incorrect information; incorrect or lack of patient consent; failure to use site markings; surgeon fatigue; multiple surgeons; multiple procedures on the same patient; unusual time pressures; emergent operations; unusual patient anatomy; poor communication among, and between, treating staff, patients and patient families, inadequate radiological visualisation and morbid obesity (DeVine 2010; Longo 2012).
In recognition of this global problem, the Joint Commission on Accreditation of Healthcare Organisations (JHACO 2003), in 2003, established a Universal Protocol for preventing WSS that emphasises pre‐operative verification, site marking and 'time‐out' procedures (JHACO 2003). Despite universal acceptance of such protocols, they have been criticised as being considerably complex without adding clear benefit in preventing WSS (Kwaan 2006).
How the intervention might work
In recent years, bodies such as the World Health Organization (WHO) have launched new safety checklists as part of a major drive to make surgery safer around the world (WHO 2009). Aspects of these checklists, such as confirmation of patient identity, site and procedure at multiple stages, and enhancing communication between team members involved in surgical cases, appear to have been designed to counter some of the risk factors that have been previously associated with WSS. Dissemination of such checklists appears to be based on evaluation of internal pilot programmes in the absence of evidence‐based recommendations supported by updated systematic reviews of current literature (WHO 2008). Nonetheless, their implementation appears to have been effective in reducing adverse surgical outcomes in certain circumstances (Haynes 2009; Treadwell 2014).
Why it is important to do this review
Although the utility of measures designed to reduce WSS were analysed by Gibbs 2005, it was acknowledged by the authors that not enough time had elapsed since the introduction of the Universal Protocol or other preventive measures to determine their effectiveness conclusively. A further review reported that few studies have investigated preventive strategies for WSS, and that clinical recommendations were made on the basis of low levels of evidence (DeVine 2010). Although the study identified risk factors for WSS and provided an estimate of incidence, it did not assess the effectiveness of interventions that may help to prevent WSS. The search strategy was also limited in that it excluded non‐English‐based articles. Given the passage of time since the introduction and international acceptance of the Universal Protocol, as well as the subsequent endorsements of newer safety checklists by authoritative bodies, there is a need for a consistently updated review of the evidence of effectiveness of strategies to reduce WSS.
Objectives
To evaluate the effectiveness of organisational and professional interventions for reducing WSS (including wrong‐site, wrong‐side, wrong‐procedure and wrong‐patient surgery).
Methods
Criteria for considering studies for this review
Types of studies
Randomised controlled trials (RCTs), quasi‐experimental studies, controlled clinical trials (CCTs), controlled before‐after studies (CBAs) with at least two intervention sites and two control sites, and interrupted‐time‐series analyses (ITS) where the intervention time was clearly defined and there were at least three data points before and three after the intervention (EPOC 2002).
Types of participants
Participants undergoing any type of surgery; nurses or clinicians involved in delivering surgical care; operating room technicians, healthcare managers or administrators and health insurers involved in delivering surgical care. We also planned to include all studies involving healthcare professionals providing care to surgical patients; studies where patients were involved to avoid the incorrect procedures or studies with interventions addressed to healthcare managers, administrators, stakeholders or health insurers.
Types of interventions
All studies that included interventions designed to address documentation, site, procedure and patient identification, communication among healthcare team members, patients and their carers.
Types of outcome measures
Primary outcomes
Incidence of WSS, including wrong‐site, wrong‐side, wrong‐procedure or wrong‐patient surgery were the primary outcomes used as criteria for including/excluded studies.
Secondary outcomes
Secondary outcomes were not used as criteria for including/excluding studies. If studies included secondary outcomes, but no primary outcomes, they were not included. Secondary outcomes reported on included mortality, health service resource consumption, healthcare professional behaviour and resource burden on healthcare providers in terms of additional time taken to undertake the intervention. We also included process measures (i.e. completion rate of checklists) where available.
Search methods for identification of studies
A sensitive search strategy was designed to retrieve trials studies and relevant systematic reviews from electronic bibliographic databases.
We identified items from the following databases on January 24 2014:
-
MEDLINE via OVID (1948 to Jan 2014) (Appendix 1)
-
EMBASE via OVID (1948 to Jan 2014) (Appendix 1)
-
CINAHL via Ebsco (1980 to Jan 2014.) (Appendix 2)
-
The Cochrane Library via Wiley (2014, Issue 1 of 12) including CENTRAL, and Database of Reviews of Effects (DARE) (Appendix 3)
-
Grey literature, which included databases such as Dissertations & Theses, African Index Medicus, etc. (Appendix 4) until 2011.
Trial Registries
-
International Clinical Trials Registry Platform (ICTRP), Word Health Organization (WHO) http://www.who.int/ictrp/en/ (searched 24/01/2014)
-
ClinicalTrials.gov, US National Institutes of Health (NIH) http://clinicaltrials.gov/ (searched 24/01/2014)
Search strategies were developed by M. Fiander, EPOC Trials Search Co‐ordinator (TSC) in consultation with the authors. The final search strategies reflect an iterative development process whereby results of test strategies were screened by authors for relevance; strategies and terms yielding no relevant results were removed. Although Medical Subject Headings (MeSH) and other controlled vocabulary were explored extensively, none were sufficiently useful to include.
Two methodological search filters were used to limit retrieval to appropriate study design and interventions of interest: the Cochrane RCT Sensitivity/Precision Maximizing Filter (Lefebvre 2011); and the EPOC Filter. No language restriction was applied. The search strategy was devised for the OVID MEDLINE interface and then adapted for the other databases.
Electronic searches
For this first update we searched the following databases: Ovid MEDLINE (In‐Process & Other Non‐Indexed Citations 2011 to January 2014);Ovid EMBASE (2011 to January 2014); The Cochrane Central Register of Controlled Trials (CENTRAL) (The Cochrane Library 2014, Issue 1); CINAHL, EbscoHost (2011 to February 2014) and The EPOC Specialised Register.
Searching other resources
An extensive grey literature search was conducted by C. Gabriel, assistant to the EPOC TSC up until 2011. Details of these searches and the results are available in Appendix 4. Additional studies were identified as follows: screened individual journals and conference proceedings (e.g. handsearching); reviewed reference lists of relevant systematic reviews or other publications; contacted authors of relevant studies or reviews to clarify reported published information or seek unpublished results/data; contacted researchers with expertise relevant to the review topic or EPOC interventions.
Data collection and analysis
Selection of studies
In the original review, three review authors (PM, JW and LB) reviewed the titles and abstracts to identify potentially relevant trials using the selection criteria. For this first update in 2014, a similar screening process took place where records were retrieved, scanned and reviewed in a similar manner by the same authorship team. Four review authors (PM, JW, LB, CA) independently screened the titles and abstracts of all studies obtained from the search. Full‐text copies of all potentially relevant articles were retrieved for closer inspection. Further articles known to the authors were also retrieved. The review authors independently determined whether studies met the inclusion criteria (JW, LB, CA). All studies that, on examination, failed to meet the inclusion criteria were detailed in the Characteristics of excluded studies. Any disagreements that arose between the review authors were resolved through discussion, or with a fourth review author (PM).
Data extraction and management
Data from the eligible studies were extracted independently by two review authors (JW, RM) using a modified version of the Effective Practice and Organisation of Care (EPOC) Group data collection checklist. Data extracted included information on study design, the intervention evaluated (including process), participants (including number in each group), setting, methods, outcomes and results.
Assessment of risk of bias in included studies
Two review authors (JW, PM) independently assessed the risk of bias for the two eligible studies using the criteria described in the EPOC Group module (see additional information, assessment of methodological quality under group details) using RevMan 2011. Discrepancies were resolved by discussion between two review authors. We assessed specific quality criteria for ITS studies: intervention independent of other changes, shape of the intervention effect pre‐specified, intervention unlikely to affect data collection, knowledge of allocated interventions adequately prevented during study, incomplete outcome data adequately addressed, study free from selective outcome reporting and study free from other biases. We noted each criterion as 'low risk of bias', 'unclear risk of bias' or 'high risk of bias'. If we had found RCTs or CBAs, we would have assessed them using the EPOC quality criteria for RCTs, CCTs and CBAs: allocation sequence adequately generated, allocation adequately concealed, baseline outcome measurements similar, baseline characteristics similar, incomplete outcome data adequately addressed, knowledge of the allocated interventions adequately prevented during the study, study adequately protected against contamination, study free from selective outcome reporting and study free from other risks of bias.
Measures of treatment effect
We presented the findings in a tabular form for narrative synthesis. We reported two types of effect sizes for both included ITS studies: the change in level of the outcome immediately after the introduction of the intervention and yearly thereafter within the post‐intervention time period; and the change in the slope of the regression lines. The data were re‐analysed using the methods described in Ramsey 2003 (time series regression using Prais‐Winsten adjustment for autocorrelation).
In the event that RCTs were identified, we planned to analyse data as follows: for dichotomous outcomes, the risk ratio (RR) and the risk difference (RD) together with their respective 95% confidence intervals (CI) would be calculated. For studies reporting continuous outcomes, the percentage change (i.e. the per cent improvement relative to the post‐intervention average in the control group) would also be reported. For CBA studies, we planned to report on relative effects. For dichotomous outcomes we would have reported on the RR adjusted for baseline differences in the outcome measures. For continuous variables we planned to report, if possible, on the relative change, adjusted for baseline differences in the outcome measures.
Unit of analysis issues
Re‐analysis of ITS studies
For both ITS studies, we followed the recommendation of Ramsey 2003 and computed a difference in slopes and level effect. As a unit of analysis error was present, we re‐analysed the study data using data provided in the original papers.
Re‐analysis of RCTs and CBAs with potential unit of analysis errors
Comparisons that randomise or allocate clusters (professionals or healthcare organisations) but do not account for clustering during the analysis have potential unit of analysis errors resulting in artificially extreme P values and over‐narrow CIs (Ukoumunne 1999). If we had identified any RCTs or CBAs with potential unit of analysis errors, we would have attempted to re‐analyse studies if information was available on the size/number of clusters and the value of the intracluster correlation coefficient (ICC), in addition to the outcome data ignoring the cluster design. Had we re‐analysed a comparison, we would have quoted the P value and annotated it with 're‐analysed'. If this was not possible, we would have reported only the point estimate.
Assessment of heterogeneity
Substantial variation in the study findings was anticipated owing to various sources of heterogeneity, such as differences in the type of intervention, the type of setting (big versus small hospital, in‐hospital versus day‐surgery hospital), study design and methodological quality. If we had found studies similar enough to undertake a meta‐analysis, we would have assessed statistical heterogeneity using the Chi2 test for heterogeneity and the I2 statistic.
Data synthesis
If we had found two or more studies that were considered to be measuring essentially the same outcomes, using the same intervention in a similar population, we planned to pool the results of these studies using standard Cochrane methodology for meta‐analysis (Higgins 2011). For continuous outcome data expressed in different units across different studies, we would have calculated standardised mean differences (SMD).
Results
Description of studies
See: Characteristics of included studies; Characteristics of excluded studies
Results of the search
In the original search in 2012, a total of 3210 references were identified from searching the literature, of which we identified 18 potentially relevant articles and excluded 17 studies. Two further studies published after the search, known to the authors, were excluded as one was a review article (Ko 2011), and the other did not include the incidence of WSS as an outcome measure (van Klei 2012). The remaining single study by Chang 2004 met the EPOC criteria for inclusion in the review and is listed in the Characteristics of included studies.
In this 2014 update, an additional 3311 reference were identified, after duplicates were removed, of which there were 18 potentially relevant articles. Two further studies published prior to the search, known to the authors, but which were not identified in the original review were also considered but excluded. One article (Vachhani 2013) met the criteria for inclusion in this review and was added to our analysis (see PRISMA flowchart in Figure 1).
Search results for update 2014
Included studies
We included two studies in this review. One from the original search (Chang 2004) and one from the 2014 update search (Vachhani 2013)
In Chang 2004, data were collected from cases of wrong‐site tooth extraction during 1996 to 1998, which were used to develop a specific educational intervention that was implemented from 1999 to 2001 in a university hospital in Taiwan. Following case‐collection of instances of wrong‐site tooth extraction, surgeons and relevant personnel were interviewed within 72 hours of the mishap in each case to investigate the contribution of various factors in error production. Medical records and x‐rays were examined and additional information regarding the sites involved and related pathology was obtained. Following this a committee comprising attending dentists reviewed the information, identified factors contributing to the errors and developed clinical guidelines for preventing erroneous extraction. Subsequently, the annual incidence of erroneous extraction was compared between the pre‐intervention and intervention periods.The specific educational intervention involved targeting residents and interns from specialties within the relevant dental department and providing a training session including presentation of cases of erroneous extraction, explanation of clinical guidelines and feedback to each speciality by the instructor.The primary outcome of this study was the annual incidence rates of wrong‐site tooth extraction during 1996 to 2001 obtained by dividing the number of erroneous extractions by the total number of tooth extractions in each year.
The 2013 study by Vachhani 2013 presents wrong‐site cranial and spinal neurosurgical events recorded in the morbidity and mortality database of a single institution in the United States of America from 1991 to 2011. In 2003, the Joint Commission of the United States released the Universal Protocol for preventing wrong‐site, wrong‐procedure and wrong‐person surgery, which subsequently became mandatory for all hospitals (JHACO 2012). Data were analysed before and after implementation of this protocol at their institution in 2005. The Universal Protocol was developed in 2003 by the Joint Commission in collaboration with the American Medical Association, American Hospital Association, American College of Physicians, American College of Surgeons, American Dental Association and American Academy of Orthopaedic Surgeons. The three steps involved in the Universal Protocol are a pre‐operative verification process, marking of the operative site and a time out (final verification) that is performed immediately before starting the operation.The authors calculated annual incidence rates of wrong‐site surgery by determining the ratio of the number of wrong‐site surgical events per year to the total number of surgical procedures for that year.
Risk of bias in included studies
The methodological characteristics of the studies are shown in the Characteristics of included studies table. Whilst the Vachhani 2013 study was independent of other changes and had sufficient data points, it did not report on the reason for the number of data points pre‐ and post‐intervention, nor was there any explanation for the shape of the intervention effect. As a result, the point of analysis was also the point of the intervention. In addition, Vachhani 2013 did not report missing data, owing to an unclear risk as to whether incomplete outcome data were adequately addressed. More so, there was no distinction between the intervention and the records collected, therefore determining whether the intervention affected data collection was unclear. The data required re‐analysis. In the interrupted‐time‐series analyses (ITS) study by Chang 2004, there was a low risk of bias regarding whether the methods for data collection before and after the intervention were the same. Knowledge of the allocated interventions in both studies was prevented owing to the primary outcomes being objective. As noted in the original publication, the Chang 2004 study did not report on ARIMA (autoregressive integrated moving average) models or time series linear regression, therefore additional re‐analysis of the data could not be undertaken.
Effects of interventions
See: Summary of findings for the main comparison ; Summary of findings 2
The ITS study by Chang 2004 reported on the effectiveness of an educational programme on the incidence of wrong‐site tooth extraction in an outpatient department of a university hospital. The annual incidence rates of erroneous tooth extraction for 1996, 1997 and 1998 were 0.026%, 0.025% and 0.046%, respectively. During the intervention period from 1999 to 2001, wrong‐site tooth extraction did not occur in the relevant department, revealing a significant difference in the incidence of erroneous extraction between the pre‐intervention and intervention period (P < 0.01). When the data were re‐analysed using the Prais‐Winsten time‐series regression, the change in level at each 12‐month period for annual number of mishaps was statistically significant at ‐4.52 percentage point (pp) (95% CI ‐6.83 pp to ‐2.21 pp; SE 0.53; P < 0.05). The change in slope was statistically significant at ‐1.16 (95% CI ‐2.22 to ‐0.10; SE 0.24; P < 0.05. This study did not assess any of the other secondary outcomes of interest in the review. A further summary of these results can be found in Table 1.
| Author | Study design | Intervention | Results | Notes |
| ITS | Educational training programme (professional intervention) | Rates of erroneous tooth extraction for 1996, 1997 and 1998: 0.026%, 0.025% and 0.046%. From 1999 to 2001, no erroneous tooth extraction occurred Change in level: ‐4.52%; 95% CI ‐6.83% to ‐2.21%; SE 0.53; P < 0.05
Change in slope: ‐1.16; 95% CI ‐2.22 to ‐0.10; SE: 0.24; P < 0.05 | Data re‐analysed |
CI: confidence interval; ITS: interrupted time series.
The ITS study by Vachhani 2013 reported on the incidence of neurological WSS at a university hospital both before and after the Universal Protocol’s implementation. The authors found a significant difference (P < 0.001) in the incidence of WSS between the before period, 1999 to 2004, and the after period, 2005 to 2011. No other outcome measures were reported.The data were re‐analysed using Prais‐Winsten regression to correct for autocorrelation. As the incidences were reported by year only and the intervention occurred in July 2004, the intervention year 2004 was excluded from the analysis. The change in level at the point the intervention was introduced was not statistically significant at ‐0.078 pp (95% CI ‐0.176 pp to 0.02 pp; SE 0.042; P = 0.103). The change in slope was statistically significant at 0.031 pp (95% CI 0.004 pp to 0.058 pp; SE 0.012; P < 0.05). A further summary of these results can be found in Table 2.
| Author | Study design | Intervention | Results | Notes |
| ITS | Implementation of the Universal Protocol | Change in level in percentage points (pp): ‐0.078 pp; (95% CI ‐0.176 pp to 0.02 pp); SE: 0.042; P value = 0.103 Change in slope: 0.031; (95% CI 0.004 to 0.058); SE: 0.012; P value < 0.05 | Data re‐analysed |
Discussion
Summary of main results
This review aimed to assess the effectiveness of interventions to reduce wrong‐site surgery (WSS), a rare but potentially disastrous clinical error that may be preventable. The estimated range of WSS has been described as varying widely, ranging from 0.09 per 10,000 to 4.5 per 10,000 surgical procedures (DeVine 2010). We included two interrupted‐time‐series (ITS) studies.
Chang 2004 evaluated a targeted educational intervention aiming to reduce the incidence of wrong‐site tooth extractions. The intervention included examination of previous cases of wrong‐site tooth extractions, development of an education intervention including presentation of erroneous case presentations, explanation of relevant clinical guidelines and feedback by an instructor. A significant reduction in the incidence of wrong‐site tooth extraction was achieved by addressing multiple risk factors that had been previously identified in a broader surgical population.
Vachhani 2013 evaluated annual incidence rates of wrong‐site neurosurgery before and after implementation of the Universal Protocol. The data presented suggest that a strong downward trend in the incidence of WSS existed prior to the intervention (statistically significant change in slope). Any significant downward level change in the incidence rate was thus not likely as it was already close to zero, and any downward trend would also naturally flatten. This is reflected in the non‐significant change in level of incidence of WSS and significant flattening of the trend to near zero. The effect of the intervention in this study is therefore unclear.
Overall completeness and applicability of evidence
Many features of the intervention addressed systematically identified risk factors for WSS, such as poor patient positioning; providing incorrect information to patient or patient family members; and poor communication between treating staff, patients and patient families (DeVine 2010), but it must be noted that the findings of the included studies were either from single institutions and were specific to the individual patient populations. Therefore, the generalisability and applicability of the results was questionable with the need for caution to be exercised in applying the results to other settings.
Quality of the evidence
The evidence was generally low quality because both ITS studies had some methodological shortcomings. The result of these two single studies should be interpreted with caution.
Potential biases in the review process
We followed the EPOC Group guidelines for conducting the review. However, publication bias remains a possible (but unknown) source of important bias.
Agreements and disagreements with other studies or reviews
We are not aware of any other systematic reviews that quantitatively evaluate the effectiveness of either the Universal Protocol for preventing WSS (established by JHACO 2003) or the commonly used WHO Surgical Safety Checklist in reducing the incidence of WSS. A systematic review by Treadwell 2014 summarises 33 studies implementing the Universal Protocol and WHO checklist for WSS prevention. The authors conclude that while the WHO Surgical Safety Checklist is associated with decreases in patient mortality and inpatient complications, no literature exists to evaluate the efficacy of these checklists in preventing wrong‐site surgery
We are aware of a number of retrospective studies that were of interest to this review. In particular, Kwaan 2006 examined a series of WSS to determine whether the Universal Protocol, emphasising pre‐operative verification, site marking and 'time‐out' practices might have been preventive. Based on the authors' retrospective judgement, rather than a prospective clinical implementation and evaluation of the protocol's effectiveness, they determined that approximately 38% of cases identified would have been unlikely to have been prevented by implementation of the Universal Protocol. While interesting, this finding is subject to the biases inherent in the retrospective judgement of the authors, and should be interpreted with caution.
Working with JCAHO, Knight 2010 developed an innovative surrogate for physically marking the surgical site as part of the Universal Protocol in the form of an "anatomic marking form". This form features the name of the procedure and the anatomical site marked on a gender‐specific diagram of the surgical site. The patient also signed this form. The authors reported a case series of over 112,500 surgical procedures during a four and a half‐year period with one documented case of WSS. In the absence of a comparison group, this study was excluded from our review; however, this study did demonstrate an adaptation in the Universal Protocol with the aim of increasing efficiency in the perioperative period.
In 2007 to 2008, Haynes 2009 conducted an international, multicentre study implementing the 19‐item WHO Surgical Safety Checklist designed to improve consistency of care and team communication. The primary end point in this study was the complication rate (including death) during hospitalisation within the first 30 days after the operation. The mortality rate in this study was 1.5% before the checklist was introduced and declined to 0.8% afterwards (P = 0.003). Inpatient complications occurred in 11% of patients at baseline and in 7% after introduction of the checklist. A retrospective cohort study conducted by van Klei 2012 on over 25,000 patients noted a reduction in crude mortality after checklist implementation, the effect of which was strongly related to checklist compliance. Similarly, between October 2007 and March 2009, deVries 2010 examined the effects on patient outcomes of another comprehensive, multidisciplinary Surgical Safety Checklist, including items such as medication, marking of the operative side and use of postoperative instructions. The rate of complications at six hospitals was compared during three‐month periods before and after the implementation of the checklist. Similar data from a control group of five hospitals were collected. Overall, results showed a significant reduction in the total number of complications from the pre‐implementation period compared to the post‐implementation period, with no change in the control hospitals. Although the checklists emphasised aspects of team communication, site marking and verification, these two studies did not specifically investigate the incidence of WSS before and after their introduction. Thus, it is not possible to conclude that the introduction of such checklists definitively reduces the incidence of this specific complication. Nonetheless, the value of checklists such as these in the process management of high‐stakes activities in both medical and non‐medical practice has been emphasised (Merry 2010), and the low‐cost, low‐risk, adaptable nature of the intervention has been highlighted (Merry 2010). Further prospective studies evaluating checklists that target the incidence of WSS as a specific complication would be very useful.
Search results for update 2014
| Educational training programme to prevent wrong‐site tooth extraction | ||||
| Patient or population: Dental patients requiring tooth extraction Settings: Outpatient department of a university hospital, Taiwan Intervention: Educational training programme (Professional intervention) Comparison: Not applicable | ||||
| Outcomes | Change in level/slope | No of | Quality of the evidence | Comments |
| Annual incidence rates of wrong‐site tooth extraction | Change in level: ‐4.52%; 95% CI ‐6.83% to ‐2.21%; SE 0.53; P < 0.05
Change in slope: ‐1.16; 95% CI ‐2.22 to ‐0.10; SE 0.24; P < 0.05 | 1 ITS study | Low1 | Data re‐analysed |
| GRADE Working Group grades of evidence | ||||
| 1. Quality of the evidence: findings of the included study were from a single institution and were specific to the individual patient population. Therefore, the generalisability and applicability of the results was questionable with the need for caution to be exercised in applying the results to other settings. This research provides some indication of the likely effect. However, the likelihood that it will be substantially different is high. | ||||
| Implementation of Universal Protocol | ||||
| Patient or population: Patients having cranial or spinal neurosurgery Settings: Inpatients at the Department of Neurosurgery, University of Illinois College of Medicine, Peoria Intervention: Implementation of the Universal Protocol Comparison: Not applicable | ||||
| Outcomes | Change in level/slope | No of Participants | Quality of the evidence | Comments |
| Annual incidence rates of wrong‐site neurosurgery | Change in level in percentage points (pp) : ‐0.078 pp; (95% CI ‐0.176 pp to 0.02 pp); SE: 0.042; P value = 0.103 Change in slope: 0.031; (95% CI 0.004 to 0.058); SE: 0.012; P value < 0.05 | 1 ITS study (7286 operations before the intervention; 15,456 after the intervention) | Low1 | Data re‐analysed |
| GRADE Working Group grades of evidence | ||||
| 1. Quality of the evidence: findings of the included study were from a single institution and were specific to the individual patient population. Therefore, the generalisability and applicability of the results was questionable with the need for caution to be exercised in applying the results to other settings. This research provides some indication of the likely effect. However, the likelihood that it will be substantially different is high. | ||||
| Author | Study design | Intervention | Results | Notes |
| ITS | Educational training programme (professional intervention) | Rates of erroneous tooth extraction for 1996, 1997 and 1998: 0.026%, 0.025% and 0.046%. From 1999 to 2001, no erroneous tooth extraction occurred Change in level: ‐4.52%; 95% CI ‐6.83% to ‐2.21%; SE 0.53; P < 0.05
Change in slope: ‐1.16; 95% CI ‐2.22 to ‐0.10; SE: 0.24; P < 0.05 | Data re‐analysed | |
| CI: confidence interval; ITS: interrupted time series. | ||||
| Author | Study design | Intervention | Results | Notes |
| ITS | Implementation of the Universal Protocol | Change in level in percentage points (pp): ‐0.078 pp; (95% CI ‐0.176 pp to 0.02 pp); SE: 0.042; P value = 0.103 Change in slope: 0.031; (95% CI 0.004 to 0.058); SE: 0.012; P value < 0.05 | Data re‐analysed |
