Skip to main content

Advertisement

SpringerLink
Log in

SAGES video acquisition framework—analysis of available OR recording technologies by the SAGES AI task force

  • Original Article
  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Surgical video recording provides the opportunity to acquire intraoperative data that can subsequently be used for a variety of quality improvement, research, and educational applications. Various recording devices are available for standard operating room camera systems. Some allow for collateral data acquisition including activities of the OR staff, kinematic measurements (motion of surgical instruments), and recording of the endoscopic video streams. Additional analysis through computer vision (CV), which allows software to understand and perform predictive tasks on images, can allow for automatic phase segmentation, instrument tracking, and derivative performance-geared metrics. With this survey, we summarize available surgical video acquisition technologies and associated performance analysis platforms.

Methods

In an effort promoted by the SAGES Artificial Intelligence Task Force, we surveyed the available video recording technology companies. Of thirteen companies approached, nine were interviewed, each over an hour-long video conference. A standard set of 17 questions was administered. Questions spanned from data acquisition capacity, quality, and synchronization of video with other data, availability of analytic tools, privacy, and access.

Results

Most platforms (89%) store video in full-HD (1080p) resolution at a frame rate of 30 fps. Most (67%) of available platforms store data in a Cloud-based databank as opposed to institutional hard drives. CV powered analysis is featured in some platforms: phase segmentation in 44% platforms, out of body blurring or tool tracking in 33%, and suture time in 11%. Kinematic data are provided by 22% and perfusion imaging in one device.

Conclusion

Video acquisition platforms on the market allow for in depth performance analysis through manual and automated review. Most of these devices will be integrated in upcoming robotic surgical platforms. Platform analytic supplementation, including CV, may allow for more refined performance analysis to surgeons and trainees. Most current AI features are related to phase segmentation, instrument tracking, and video blurring.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Spaner SJ, Warnock GL (1997) A brief history of endoscopy, laparoscopy, and laparoscopic surgery. J Laparoendosc Adv Surg Tech 7:369–373

    Article  CAS  Google Scholar 

  2. LaRose D, Taylor RH, Funda J, Eldridge B, Gomory S, Talamini MA, Kavoussi L, Anderson J, Gruben K (1995) A telerobotic assistant for laparoscopic surgery. IEEE Engin in Med and Biol Mag 14:279–288. https://doi.org/10.1109/51.391776

    Article  Google Scholar 

  3. Talamini MA (2001) Surgery of the 21st century. Ann Surg 234:8–9

    Article  CAS  PubMed  Google Scholar 

  4. Filicori F, Addison P (2021) Intellectual property and data ownership in the age of video recording in the operating room. Surg Endosc. https://doi.org/10.1007/s00464-021-08692-8

    Article  PubMed  Google Scholar 

  5. Hashimoto DA, Rosman G, Witkowski ER, Stafford C, Navarette-Welton AJ, Rattner DW, Lillemoe KD, Rus DL, Meireles OR (2019) Computer vision analysis of intraoperative video: automated recognition of operative steps in laparoscopic sleeve gastrectomy. Ann Surg 270:414–421. https://doi.org/10.1097/SLA.0000000000003460

    Article  PubMed  Google Scholar 

  6. Mascagni P, Fiorillo C, Urade T, Emre T, Yu T, Wakabayashi T, Felli E, Perretta S, Swanstrom L, Mutter D, Marescaux J, Pessaux P, Costamagna G, Padoy N, Dallemagne B (2020) Formalizing video documentation of the critical view of safety in laparoscopic cholecystectomy: a step towards artificial intelligence assistance to improve surgical safety. Surg Endosc 34:2709–2714. https://doi.org/10.1007/s00464-019-07149-3

    Article  PubMed  Google Scholar 

  7. Mascagni P, Vardazaryan A, Alapatt D, Urade T, Emre T, Fiorillo C, Pessaux P, Marescaux J, Costamagna G, Dallemagne B, Padoy N (2020) Artificial intelligence for surgical safety automatic assessment of the critical view of safety in laparoscopic cholecystectomy using deep learning. Ann Surg. https://doi.org/10.1097/SLA.0000000000004351

    Article  PubMed  Google Scholar 

  8. Mascagni P, Alapatt D, Urade T, Vardazaryan A, Mutter D, Marescaux J, Costamagna G, Dallemagne B, Padoy N (2021) A computer vision platform to automatically locate critical events in surgical videos: documenting safety in laparoscopic cholecystectomy. Ann Surg 274:e93–e95. https://doi.org/10.1097/SLA.0000000000004736

    Article  PubMed  Google Scholar 

  9. Mascagni P, Alapatt D, Laracca GG, Guerriero L, Spota A, Fiorillo C, Vardazaryan A, Quero G, Alfieri S, Baldari L, Cassinotti E, Boni L, Cuccurullo D, Costamagno G, Dallemagne B, Padoy N (2022) Multicentric validation of endodigest: a computer vision platform for video documentation of the critical view of safety in laparoscopic cholecystectomy. Surg Endosc. https://doi.org/10.1007/s00464-022-09112-1

    Article  PubMed  Google Scholar 

  10. Ward TM, Mascagni P, Madani A, Padoy N, Perretta S, Hashimoto DA (2021) Surgical data science and artificial intelligence for surgical education. J Surg Oncol 124:221–230. https://doi.org/10.1002/jso.26496

    Article  PubMed  Google Scholar 

  11. al Abbas AI, Jung JP, Rice MJK, Zureikat AH, Zeh HJ, Hogg ME, (2019) Methodology for developing an educational and research video library in minimally invasive surgery. J Surg Edu 76:745–755. https://doi.org/10.1016/j.jsurg.2018.10.011

    Article  Google Scholar 

  12. Ibrahim AM, Varban OA, Dimick JB (2016) Novel uses of video to accelerate the surgical learning curve. J Laparoendosc Adv Surg Tech 26:240–242. https://doi.org/10.1089/lap.2016.0100

    Article  Google Scholar 

  13. Mazer L, Varban O, Montgomery JR, Awad MM, Schulman A (2021) Video is better: why aren’t we using it ? A mixed-methods study of the barriers to routine procedural video recording and case review. Surg Endosc. https://doi.org/10.1007/s00464-021-08375-4

    Article  PubMed  Google Scholar 

  14. Pernar LIM, Robertson FC, Tavakkoli A, Sheu EG, Brooks DC, Smink DS (2017) An appraisal of the learning curve in robotic general surgery. Surg Endosc 31:4583–4596. https://doi.org/10.1007/s00464-017-5520-2

    Article  PubMed  Google Scholar 

  15. Birkmeyer JD, Finks JF, O’Reilly A, Oerline M, Carlin AM, Nunn AR, Dimick J, Banerjee M, Birkmeyer NJO (2013) Surgical skill and complication rates after bariatric surgery. New Eng J Med 369:1434–1442. https://doi.org/10.1056/nejmsa1300625

    Article  CAS  PubMed  Google Scholar 

  16. Stulberg JJ, Stulberg JJ, Huang R, Kreutzer L, Ban K, Champagne BJ, Steele SR, Johnson JK, Holl JL, Greenberg CC, Bilimoria KY (2020) Association between surgeon technical skills and patient outcomes. JAMA Surg 155:960–968. https://doi.org/10.1001/jamasurg.2020.3007

    Article  PubMed  PubMed Central  Google Scholar 

  17. Varban OA, Thumma JR, Finks JF, Carlin AM, Ghaferi AA, Dimick JB (2021) Evaluating the effect of surgical skill on outcomes for laparoscopic sleeve gastrectomy: a video-based study. Ann Surg 273:766–771. https://doi.org/10.1097/SLA.0000000000003385

    Article  PubMed  Google Scholar 

  18. Idrees JJ, Rosinski BF, Chen Q, Bagante F, Merath K, White S, Pawlik TM (2018) Variation in medicare payments and reimbursement rates for hepatopancreatic surgery based on quality: is there a financial incentive for high-quality hospitals? J Am Coll of Surg 227:212–222. https://doi.org/10.1016/j.jamcollsurg.2018.04.005

    Article  Google Scholar 

  19. Steastedt KP, Schwab P, O’Brien Z, Wakida E, Herrera K, Marcelo PGF, Agha-Mir-Salim L, Frigola XB, Ndulue EB, Marcelo A, Celi LA (2022) Global healthcare fairness: we should be sharing more, not less, data. PLOS Digit Health 1(10):e0000102. https://doi.org/10.1371/journal.pdig.0000102

    Article  Google Scholar 

Download references

Acknowledgements

None.

Funding

None.

Author information

Authors and Affiliations

  1. Intraoperative Performance Analytics Laboratory (IPAL), Department of General Surgery, Northwell Health, Lenox Hill Hospital, New York, NY, USA

    Filippo Filicori & Daniel P. Bitner

  2. Donald and Barbara Zucker School of Medicine at Hofstra/Northwell, Hempstead, NY, USA

    Filippo Filicori, Daniel P. Bitner & Mark Talamini

  3. Department of Surgery, Division of Surgical Robotics and Artificial Intelligence, University of Cologne, Cologne, Germany

    Hans F. Fuchs

  4. Center for Surgical Invention and Innovation, Department of Surgery, McMaster University, Hamilton, ON, Canada

    Mehran Anvari

  5. Artificial Intelligence and Medical Simulation (AIMS) Lab, Department of Surgery, UT Southwestern Medical Center, Dallas, TX, USA

    Ganesh Sankaranaraynan

  6. Minimally Invasive Surgery Laboratory, Department of Surgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA

    Matthew B. Bloom

  7. Department of Surgery, University Hospitals Cleveland Medical Center, Cleveland, OH, USA

    Daniel A. Hashimoto

  8. Surgical Artificial Intelligence Research Academy, Department of Surgery, University Health Network, Toronto, ON, Canada

    Amin Madani

  9. Fondazione Policlinico Universitario A. Gemelli, Rome, Italy

    Pietro Mascagni

  10. Institute of Image-Guided Surgery, IHU-Strasbourg, Strasbourg, France

    Pietro Mascagni

  11. Canadian Surgical Technologies & Advanced Robotics (CSTAR), London Health Sciences Centre, London, ON, Canada

    Christopher M. Schlachta

  12. Surgical Artificial Intelligence and Innovation Laboratory (SAIIL), Department of General Surgery, Massachusetts General Hospital, 15 Parkman Street, WAC 339, Boston, MA, 02139, USA

    Ozanan R. Meireles

Authors
  1. Filippo Filicori
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel P. Bitner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hans F. Fuchs
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mehran Anvari
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ganesh Sankaranaraynan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Matthew B. Bloom
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Daniel A. Hashimoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Amin Madani
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Pietro Mascagni
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Christopher M. Schlachta
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Mark Talamini
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Ozanan R. Meireles
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ozanan R. Meireles.

Ethics declarations

Disclosures

FF is a consultant for Boston Scientific and Activ Surgical, DAH is a consultant for Johnson and Johnson Institute, and previously Activ Surgical and Verily Life Sciences. AM is a consultant for Activ Surgical. DPB is a consultant for Deep Surgery.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 19 kb)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Filicori, F., Bitner, D.P., Fuchs, H.F. et al. SAGES video acquisition framework—analysis of available OR recording technologies by the SAGES AI task force. Surg Endosc 37, 4321–4327 (2023). https://doi.org/10.1007/s00464-022-09825-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-022-09825-3

Keywords

Search

Navigation