Abstract
Background
The National Training Programme for laparoscopic colorectal surgery in England was implemented to ensure training was supervised, structured, safe and effective. Delegates were required to pass a competency assessment (sign-off) before undertaking independent practice. This study described the types of errors identified and associated these errors with competency to progress to independent laparoscopic colorectal practice.
Methods
All sign-off submissions from the start of the process in January 2008 until July 2013 were included. Content analysis was used to categorise errors. Bayes factor (BF) was used to measure the impact of individual error on assessment outcome. A smaller BF indicates that an error has stronger associations with unsuccessful assessments. Bayesian network was employed to graphically represent the reasoning process whereby the chance of successful assessment diminished with the identification of each error. Quality of the error feedback was measured by the area under the ROC curve which linked the predictions from the Bayesian model to the expert verdict.
Results
Among 370 assessments analysed, 240 passed and 130 failed. On average, 2.5 different types of error were identified in each assessment. Cases that were more likely to fail had three or more different types of error (χ 2 = 72, p < 0.0001) and demonstrated poorer technical skills (CAT score <2.7, χ 2 = 164, p < 0.0001). Case complexity or right- versus left-sided resection did not have a significant impact. Errors associated with dissection (BF = 0.18), anastomosis (BF = 0.23) and oncological quality (BF = 0.19) were critical determinants of surgical competence, each reducing the odds of pass by at least fourfold. The area under the ROC curve was 0.84.
Conclusions
Errors associated with dissection, anastomosis and oncological quality were critical determinants of surgical competency. The detailed error analysis reported in this study can guide the design of future surgical education and clinical training programmes.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Birkmeyer JD, Finks JF, O’Reilly A, Oerline M, Carlin AM, Nunn AR, Dimick J, Banerjee M, Birkmeyer NJ, Michigan Bariatric Surgery C (2013) Surgical skill and complication rates after bariatric surgery. N Engl J Med 369:1434–1442
Miskovic D, Wyles SM, Carter F, Coleman MG, Hanna GB (2011) Development, validation and implementation of a monitoring tool for training in laparoscopic colorectal surgery in the English National Training Program. Surg Endosc 25:1136–1142
Vassiliou MC, Feldman LS (2011) Objective assessment, selection, and certification in surgery. Surg Oncol 20:140–145
Miskovic D, Ni M, Wyles SM, Parvaiz A, Hanna GB (2012) Observational clinical human reliability analysis (OCHRA) for competency assessment in laparoscopic colorectal surgery at the specialist level. Surg Endosc 26:796–803
Coleman MG, Hanna GB, Kennedy R, National Training Programme L (2011) The National Training Programme for Laparoscopic Colorectal Surgery in England: a new training paradigm. Colorectal Dis Off J Assoc Coloproctol Great Br Irel 13:614–616
Mackenzie H, Ni M, Miskovic D, Motson RW, Gudgeon AM, Khan Z, Longman R, Coleman MG, Hanna GB (2015 expected) Does technical skills assessment of specialists predict clinical outcome in laparoscopic colorectal surgery? A study for the National Training Programme in England. Br J Surg (in print)
Holsti OR (1968) Content Analysis. In: Lindzey G, Aronson E (eds) The handbook of social psychology. Addison-Wesley, Reading
Goodman SN (1999) Toward evidence-based medical statistics. 2: the Bayes factor. Ann Intern Med 130:1005–1013
Edwards W, Lindman H, Savage L (1963) Bayesian statistical inference for psychological research. Psychol Rev 70:193–242
Miskovic D, Ni M, Wyles SM, Kennedy RH, Francis NK, Parvaiz A, Cunningham C, Rockall TA, Gudgeon AM, Coleman MG, Hanna GB, National Training Programme in Laparoscopic Colorectal Surgery in E (2013) Is competency assessment at the specialist level achievable? A study for the National Training Programme in laparoscopic colorectal surgery in England. Ann Surg 257:476–482
Tang B, Hanna GB, Bax NM, Cuschieri A (2004) Analysis of technical surgical errors during initial experience of laparoscopic pyloromyotomy by a group of Dutch pediatric surgeons. Surg Endosc 18:1716–1720
Tang B, Hanna GB, Joice P, Cuschieri A (2004) Identification and categorization of technical errors by Observational Clinical Human Reliability Assessment (OCHRA) during laparoscopic cholecystectomy. Arch Surg 139:1215–1220
Acknowledgments
We wish to thank all the assessors of the National Training Programme for Laparoscopic Colorectal Surgery in England for their contribution to the sign-off assessment process. This research is supported by a grant awarded to GH by the National Cancer Action team on behalf of the Department of Health England.
Disclosures
Melody Ni, Hugh Mackenzie1, Adam Widdison, John T. Jenkins, Steve Mansfield, Tony Dixon, Dominic Slade, Mark G. Coleman, George B. Hanna declare that they have no conflicts of interest.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Ni, M., Mackenzie, H., Widdison, A. et al. What errors make a laparoscopic cancer surgery unsafe? An ad hoc analysis of competency assessment in the National Training Programme for laparoscopic colorectal surgery in England. Surg Endosc 30, 1020–1027 (2016). https://doi.org/10.1007/s00464-015-4289-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00464-015-4289-4