Abstract
Background
We previously developed nine inanimate training exercises as part of a comprehensive, proficiency-based robotic training curriculum that addressed 23 unique skills identified via task deconstruction of robotic operations. The purpose of this study was to evaluate construct validity, workload, and expert levels for the nine exercises.
Methods
Expert robotic surgeons (n = 8, fellows and faculty) and novice trainees (n = 4, medical students) each performed three to five consecutive repetitions of nine previously reported exercises (five FLS models with or without modifications and four custom-made models). Each task was scored for time and accuracy using modified FLS metrics; task scores were normalized to a previously established (preliminary) proficiency level and a composite score equaled the sum of the nine normalized task scores. Questionnaires were administered regarding prior experience. After each exercise, participants completed a validated NASA-TLX Workload Scale to rate the mental, physical, temporal, performance, effort, and frustration levels of each task.
Results
Experts had performed 119 (range = 15–600) robotic operations; novices had observed ≤1 robotic operation. For all nine tasks and the composite score, experts achieved significantly better performance than novices (932 ± 67 vs. 618 ± 111, respectively; P < 0.001). No significant differences in workload between experts and novices were detected (32.9 ± 3.5 vs. 32.0 ± 9.1, respectively; n.s.). Importantly, frustration ratings were relatively low for both groups (4.0 ± 0.7 vs. 3.8 ± 1.6, n.s.). The mean performance of the eight experts was deemed suitable as a revised proficiency level for each task.
Conclusion
Using objective performance metrics, all nine exercises demonstrated construct validity. Workload was similar between experts and novices and frustration was low for both groups. These data suggest that the nine structured exercises are suitable for proficiency-based robotic training.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Sroka G, Feldman LS, Vassiliou MC, Kaneva PA, Favez R, Fried GM (2010) Fundamentals of laparoscopic surgery simulator training to proficiency improves laparoscopic performance in the operating room—a randomized controlled trial. Am J Surg 199:115–120
McCluney AL, Vassiliou MC, Kaneva PA, Cao J, Stanbridge DD, Feldman LS, Fried GM (2007) FLS simulator performance predicts intraoperative laparoscopic skill. Surg Endosc 21:1991–1995
Fundamentals of laproscopic surgery. http://www.flsprogram.org. Accessed 25 May 2011
Gallagher AG, Ritter EM, Satava RM (2003) Fundamental principles of validation, and reliability: rigorous science for the assessment of surgical education and training. Surg Endosc 17:1525–1529
Hart SG, Staveland LE (1988) Development of a multi-dimensional workload rating scale. In: Human mental workload. Amsterdam, Elsevier
Ritter EM, Scott DJ (2007) Design of a proficiency-based skills training curriculum for the fundamentals of laparoscopic surgery. Surg Innov 14:107–112
Scott DJ (2006) Proficiency-based training for surgical skills. Semin Colon Rectal Surg 19:72–80
Goova MT, Hollett LA, Tesfay ST, Gala RB, Puzziferri N, Kehdy FJ, Scott DJ (2008) Implementation, construct validity and benefit of a proficiency based knot-tying and suturing curriculum. J Surg Educ 65:309–315
Mashaud LB, Castellvi AO, Hollett LA, Hogg DC, Tesfay ST, Scott DJ (2010) Two-year skill retention and certification exam performance after fundamentals of laparoscopic skills training and proficiency maintenance. Surgery 2:194–201
Stefanidis D, Korndorffer JR, Black FW, Dunne JB, Sierra R, Touchard CL, Rice DA, Markert RJ, Kastl PR, Scott DJ (2006) Psychomotor testing predicts rate of skill acquisition for proficiency-based laparoscopic skills training. Surgery 140:252–262
Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D, Ghitulescu G, Andrew CG (2004) Proving the value of simulation in laparoscopic surgery. Ann Surg 240:518–525
Stefanidis D, Wang F, Korndorffer JR Jr, Dunne JB, Scott DJ (2010) Robotic assistance improves intracorporeal suturing performance and safety in the operating room while decreasing operator workload. Surg Endosc 24:377–382
Stefanidis D, Hope WW, Scott DJ (2011) Robotic suturing on the FLS model possesses construct validity, is less physically demanding, and is favored by more surgeons compared with laparoscopy. Surg Endosc 25(7):2141–2146
Korndorffer JR Jr, Clayton JL, Tesfay ST, Brunner WC, Sierra R, Dunne JB, Jones DB, Rege RV, Touchard CL, Scott DJ (2005) Multicenter construct validity for Southwestern laparoscopic videotrainer stations. J Surg Res 128:114–119
Hamilton EC, Scott DJ, Fleming JB, Rege RV, Laycock R, Bergen PC, Tesfay ST, Jones DB (2002) Comparison of video trainer and virtual reality training systems on acquisition of laparoscopic skills. Surg Endosc 16:406–411
Seixas-Mikelus SA, Stegemann AP, Kesavadas T (2011) Content validation of a novel robotic surgical simulator. BJU Int 107:1130–1135
Disclosures
G. Dulan, R. V. Rege, D. C. Hogg, K. M. Gilberg-Fisher, N. A. Arain, S. T. Tesfay, and D. J. Scott have no conflicts of interest or financial ties to disclose.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Dulan, G., Rege, R.V., Hogg, D.C. et al. Proficiency-based training for robotic surgery: construct validity, workload, and expert levels for nine inanimate exercises. Surg Endosc 26, 1516–1521 (2012). https://doi.org/10.1007/s00464-011-2102-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00464-011-2102-6