Skip to main content
SpringerLink
Log in

Objective performance indicators of cardiothoracic residents are associated with vascular injury during robotic-assisted lobectomy on porcine models

  • Original Article
  • Published:
Journal of Robotic Surgery Aims and scope Submit manuscript

Abstract

Surgical training relies on subjective feedback on resident technical performance by attending surgeons. A novel data recorder connected to a robotic-assisted surgical platform captures synchronized kinematic and video data during an operation to calculate quantitative, objective performance indicators (OPIs). The aim of this study was to determine if OPIs during initial task of a resident’s robotic-assisted lobectomy (RL) correlated with bleeding during the procedure. Forty-six residents from the 2019 Thoracic Surgery Directors Association Resident Boot Camp completed RL on an ex vivo perfused porcine model while continuous video and kinematic data were recorded. For this pilot study, RL was segmented into 12 tasks and OPIs were calculated for the initial major task. Cases were reviewed for major bleeding events and OPIs of bleeding cases were compared to those who did not. Data from 42 residents were complete and included in the analysis. 10/42 residents (23.8%) encountered bleeding: 10/40 residents who started with superior pulmonary vein exposure and 0/2 residents who started with pulmonary artery exposure. Twenty OPIs for both hands were assessed during the initial task. Six OPIs related to instrument usage or smoothness of motion were significant for bleeding. Differences were statistically significant for both hands (p < 0.05). OPIs showing bimanual asymmetry indicated lower proficiency. This study demonstrates that kinematic and video analytics can establish a correlation between objective performance metrics and bleeding events in an ex vivo perfused lobectomy. Further study could assist in the development of focused exercises and simulation on objective domains to help improve overall performance and reducing complications during RL.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Abbreviations

GEARS:

Global evaluation assessment robotic skills

OPI:

Objective performance indicator

OSATS:

Objective structured assessment of technical skill

PA:

Pulmonary artery

PGY:

Post-graduate year

RL:

Robotic-assisted lobectomy

SPV:

Superior pulmonary vein

STS:

Society of thoracic surgeons

TSDA:

Thoracic Surgery Directors Association

VR:

Virtual reality

References

  1. Hung AJ, Chen J, Jarc A, Hatcher D, Djaladat H, Gill IS (2018) Development and validation of objective performance metrics for robot-assisted radical prostatectomy: a pilot study. J Urol 199(1):296–304. https://doi.org/10.1016/j.juro.2017.07.081 (Epub 2017 Jul 29 PMID: 28765067 PMCID: PMC8266176)

    Article  PubMed  Google Scholar 

  2. Goh AC, Goldfarb DW, Sander JC, Miles BJ, Dunkin BJ (2012) Global evaluative assessment of robotic skills: validation of a clinical assessment tool to measure robotic surgical skills. J Urol 187(1):247–252. https://doi.org/10.1016/j.juro.2011.09.032 (Epub 2011 Nov 17 PMID: 22099993)

    Article  PubMed  Google Scholar 

  3. Aghazadeh MA, Jayaratna IS, Hung AJ, Pan MM, Desai MM, Gill IS, Goh AC (2015) External validation of global evaluative assessment of robotic skills (GEARS). Surg Endosc 29(11):3261–3266. https://doi.org/10.1007/s00464-015-4070-8 (Epub 2015 Jan 22 PMID: 25609318)

    Article  PubMed  Google Scholar 

  4. Brown KC, Bhattacharyya KD, Kulason S, Zia A, Jarc A (2020) How to bring surgery to the next level: interpretable skills assessment in robotic-assisted surgery. Visc Med. 36(6):463–470. https://doi.org/10.1159/000512437 (Epub 2020 Oct 28 PMID: 33447602 PMCID: PMC7768128)

    Article  PubMed  PubMed Central  Google Scholar 

  5. Lyman WB, Passeri MJ, Murphy K, Siddiqui IA, Khan AS, Iannitti DA, Martinie JB, Baker EH, Vrochides D (2021) An objective approach to evaluate novice robotic surgeons using a combination of kinematics and stepwise cumulative sum (CUSUM) analyses. Surg Endosc 35(6):2765–2772. https://doi.org/10.1007/s00464-020-07708-z (Epub 2020 Jun 16 PMID: 32556751)

    Article  PubMed  Google Scholar 

  6. Hung AJ, Oh PJ, Chen J, Ghodoussipour S, Lane C, Jarc A, Gill IS (2019) Experts vs super-experts: differences in automated performance metrics and clinical outcomes for robot-assisted radical prostatectomy. BJU Int 123(5):861–868. https://doi.org/10.1111/bju.14599 (Epub 2018 Nov 18 PMID: 30358042)

    Article  PubMed  Google Scholar 

  7. Estrada S, Duran C, Schulz D, Bismuth J, Byrne MD, O’Malley MK (2016) Smoothness of surgical tool tip motion correlates to skill in endovascular tasks. IEEE Trans Hum Mach Syst 46(5):647–659

    Article  Google Scholar 

  8. Lin HC, Shafran I, Yuh D, Hager GD (2006) Towards automatic skill evaluation: detection and segmentation of robot-assisted surgical motions. Comput Aided Surg 11(5):220–230. https://doi.org/10.3109/10929080600989189 (PMID: 17127647)

    Article  PubMed  Google Scholar 

  9. Gulde P, Hermsdörfer J (2018) Smoothness metrics in complex movement tasks. Front Neurol 9:615. https://doi.org/10.3389/fneur.2018.00615

    Article  PubMed  PubMed Central  Google Scholar 

  10. Veronesi G, Dorn P, Dunning J, Cardillo G, Schmid RA, Collins J, Baste JM, Limmer S, Shahin GMM, Egberts JH, Pardolesi A, Meacci E, Stamenkovic S, Casali G, Rueckert JC, Taurchini M, Santelmo N, Melfi F, Toker A (2018) Outcomes from the delphi process of the thoracic robotic curriculum development committee. Eur J Cardiothorac Surg 53(6):1173–1179. https://doi.org/10.1093/ejcts/ezx466 (PMID: 29377988)

    Article  PubMed  Google Scholar 

  11. Turner SR, Lai H, Nasir BS, Yasufuku K, Schieman C, Huang J, Bédard ELR (2020) Development and pilot testing of an assessment tool for performance of anatomic lung resection. Ann Thorac Surg 109(6):1922–1930. https://doi.org/10.1016/j.athoracsur.2019.09.052 (Epub 2019 Nov 9 PMID: 31706874)

    Article  PubMed  Google Scholar 

  12. “Thoracic milestones”. KindHeart, Chapel Hill, NC, USA https://www.acgme.org/Portals/0/PDFs/Milestones/ThoracicSurgeryMilestones.pdf

  13. Martin JA, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C, Brown M (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84(2):273–278. https://doi.org/10.1046/j.1365-2168.1997.02502.x (PMID: 9052454)

    Article  CAS  PubMed  Google Scholar 

  14. Hogan N, Sternad D (2009) Sensitivity of smoothness measures to movement duration, amplitude, and arrests. J Mot Behav 41(6):529–534. https://doi.org/10.3200/35-09-004-RC.PMID:19892658;PMCID:PMC3470860

    Article  PubMed  Google Scholar 

  15. Dromey BP, Ahmed S, Vasconcelos F, Mazomenos E, Kunpalin Y, Ourselin S, Deprest J, David AL, Stoyanov D, Peebles DM (2021) Dimensionless squared jerk: an objective differential to assess experienced and novice probe movement in obstetric ultrasound. Prenat Diagn 41(2):271–277. https://doi.org/10.1002/pd.5855 (Epub 2020 Nov 5 PMID: 33103808 PMCID: PMC7894282)

    Article  PubMed  Google Scholar 

Download references

Funding

A grant from Intuitive, Inc. helped fund this research including the recorders to perform this research.

Author information

Authors and Affiliations

  1. Department of Surgery, Division of Thoracic Surgery, MedStar Georgetown University Hospital, 110 Irving St, G-253, Washington, DC, 20010, USA

    John F. Lazar & Hayley Henderson

  2. Data and Analytics, Intuitive Surgical, Inc., Sunnyvale, CA, USA

    Kristen Brown, Sadia Yousaf, Anthony Jarc & Daniel S. Oh

  3. Department of General Surgery, MedStar Georgetown University Hospital, Washington, DC, USA

    Ariana Metchik

  4. Division of Thoracic Surgery, University of North Carolina, Chapel Hill, NC, USA

    Richard H. Feins

  5. Division of Thoracic Surgery, Emory University, Atlanta, GA, USA

    Manu S. Sancheti

  6. Division of Thoracic Surgery, University of Michigan, Ann Arbor, MI, USA

    Jules Lin

  7. Division of Thoracic Surgery, Johns Hopkins University, Baltimore, MD, USA

    Stephen Yang

  8. Department of Thoracic Surgery, Vanderbilt University, Nashville, TN, USA

    Jonathan Nesbitt

  9. Division of Thoracic Surgery, The Ohio State University, Columbus, OH, USA

    Desmond D’Souza

  10. Division of Thoracic Surgery, University of Southern California, Los Angeles, CA, USA

    Daniel S. Oh

Authors
  1. John F. Lazar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kristen Brown
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sadia Yousaf
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anthony Jarc
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ariana Metchik
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hayley Henderson
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Richard H. Feins
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Manu S. Sancheti
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jules Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Stephen Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Jonathan Nesbitt
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Desmond D’Souza
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Daniel S. Oh
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All the authors contributed to the study conception and design. Material preparation, data collection and analysis were performed by JFL, KB, SY, and DO. The first draft of the manuscript was written by JFL and all the authors commented on previous versions of the manuscript. All the authors read and approved the final manuscript.

Corresponding author

Correspondence to John F. Lazar.

Ethics declarations

Conflict of interest

The authors have the following financial disclosures: Manu Sancheti, Stephen Yang, Jules Lin, John Lazar, and Desmond D’Souza. Daniel Oh is an Intuitive employee as well as an academic surgeon at the University of Southern California.

IRB approval

Western IRB, Work Order Number 1-1112682-1 approved September 26, 2018.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lazar, J.F., Brown, K., Yousaf, S. et al. Objective performance indicators of cardiothoracic residents are associated with vascular injury during robotic-assisted lobectomy on porcine models. J Robotic Surg 17, 669–676 (2023). https://doi.org/10.1007/s11701-022-01476-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11701-022-01476-9

Keywords

Search

Navigation