Skip to main content

Advertisement

Log in

Robotic duodenopancreatectomy assisted with augmented reality and real-time fluorescence guidance

  • Video
  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

The minimally invasive surgeon cannot use ‘sense of touch’ to orientate surgical resection, identifying important structures (vessels, tumors, etc.) by manual palpation. Robotic research has provided technology to facilitate laparoscopic surgery; however, robotics has yet to solve the lack of tactile feedback inherent to keyhole surgery. Misinterpretation of the vascular supply and tumor location may increase the risk of intraoperative bleeding and worsen dissection with positive resection margins.

Methods

Augmented reality (AR) consists of the fusion of synthetic computer-generated images (three-dimensional virtual model) obtained from medical imaging preoperative work-up and real-time patient images with the aim of visualizing unapparent anatomical details.

Results

In this article, we review the most common modalities used to achieve surgical navigation through AR, along with a report of a case of robotic duodenopancreatectomy using AR guidance complemented with the use of fluorescence guidance.

Conclusions

The presentation of this complex and high-technology case of robotic duodenopancreatectomy, and the overview of current technology that has made it possible to use AR in the operating room, highlights the needs for further evolution and the windows of opportunity to create a new paradigm in surgical practice.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Tekkis PP, Senagore AJ, Delaney CP, Fazio VW (2005) Evaluation of the learning curve in laparoscopic colorectal surgery: comparison of right-sided and left-sided resections. Ann Surg 242:83–91

    Article  PubMed Central  PubMed  Google Scholar 

  2. Diana M, Pessaux P, Marescaux J (2014) New technologies for single-site robotic surgery in hepato-biliary-pancreatic surgery. J Hepatobiliary Pancreat Sci 21:34–42

    Article  PubMed  Google Scholar 

  3. Nicolau S, Soler L, Mutter D, Marescaux J (2011) Augmented reality in laparoscopic surgical oncology. Surg Oncol 20:189–201

    Article  PubMed  Google Scholar 

  4. D’Agostino J, Diana M, Soler L, Vix M, Marescaux J (2012) 3D virtual neck exploration prior to parathyroidectomy. N Engl J Med 367(11):1072–1073

    Article  PubMed  Google Scholar 

  5. Iseki H, Masutani Y, Iwahara M, Tanikawa T, Muragaki Y, Taira T et al (1997) Volumegraph (overlaid three-dimensional image-guided navigation). Clinical application of augmented reality in neurosurgery. Stereotact Funct Neurosurg 68:18–24

    Article  CAS  PubMed  Google Scholar 

  6. Wagner A, Ploder O, Enislidis G, Truppe M, Ewers R (1995) Virtual image guided navigation in tumor surgery: technical innovation. J Craniomaxillofac Surg 23:217–223

    Article  CAS  PubMed  Google Scholar 

  7. Shuhaiber JH (2004) Augmented reality in surgery. Arch Surg 139:170–174

    Article  PubMed  Google Scholar 

  8. Calhoun PS, Kuszyk BS, Heath DG, Carley JC, Fishman EK (1999) Three-dimensional volume rendering of spiral CT data: theory and method. Radiographics 19:745–764

    Article  CAS  PubMed  Google Scholar 

  9. Volonte F, Pugin F, Bucher P, Sugimoto M, Ratib O, Morel P (2011) Augmented reality and image overlay navigation with OsiriX in laparoscopic and robotic surgery: not only a matter of fashion. J Hepatobiliary Pancreat Sci 18:506–509

    Article  PubMed  Google Scholar 

  10. Pieper S, Halle M, Kikinis R (2004) 3D SLICER. In: Proceedings of the 1st IEEE International Symposium on Biomedical Imaging: from nano to macro, pp 632–635

  11. Fichtinger G, Deguet A, Fischer G, Iordachita I, Balogh E, Masamune K et al (2005) Image overlay for CT-guided needle insertions. Comput Aided Surg 10:241–255

    Article  PubMed  Google Scholar 

  12. Liao H, Inomata T, Sakuma I, Dohi T (2010) 3-D augmented reality for MRI-guided surgery using integral videography autostereoscopic image overlay. IEEE Trans Biomed Eng 57:1476–1486

    Article  PubMed  Google Scholar 

  13. Okamoto T, Onda S, Matsumoto M, Gocho T, Futagawa Y, Fujioka S et al (2013) Utility of augmented reality system in hepatobiliary surgery. J Hepatobiliary Pancreat Sci 20(2):249–253

    Article  PubMed  Google Scholar 

  14. Sugimoto M, Yasuda H, Koda K, Suzuki M, Yamazaki M, Tezuka T et al (2010) Image overlay navigation by markerless surface registration in gastrointestinal, hepatobiliary and pancreatic surgery. J Hepatobiliary Pancreat Sci 17:629–636

    Article  PubMed  Google Scholar 

  15. Marescaux J, Rubino F, Arenas M, Mutter D, Soler L (2004) Augmented-reality-assisted laparoscopic adrenalectomy. JAMA 292:2214–2215

    CAS  PubMed  Google Scholar 

  16. Mutter D, Soler L, Marescaux J (2010) Recent advances in liver imaging. Expert Rev Gastroenterol Hepatol 4:613–621

    Article  CAS  PubMed  Google Scholar 

  17. D’Agostino J, Wall J, Soler L, Vix M, Duh QY, Marescaux J (2013) Virtual neck exploration for parathyroid adenomas: a first step toward minimally invasive image-guided surgery. JAMA Surg 148:232–238 discussion 238

    Article  PubMed  Google Scholar 

  18. Marzano E, Piardi T, Soler L, Diana M, Mutter D, Marescaux J et al (2013) Augmented reality-guided artery-first pancreatico-duodenectomy. J Gastrointest Surg 17:1980–1983

    Article  PubMed  Google Scholar 

  19. Marvik R, Lango T, Tangen GA, Andersen JO, Kaspersen JH, Ystgaard B et al (2004) Laparoscopic navigation pointer for three-dimensional image-guided surgery. Surg Endosc 18:1242–1248

    Article  CAS  PubMed  Google Scholar 

  20. Ieiri S, Uemura M, Konishi K, Souzaki R, Nagao Y, Tsutsumi N et al (2012) Augmented reality navigation system for laparoscopic splenectomy in children based on preoperative CT image using optical tracking device. Pediatr Surg Int 28:341–346

    Article  PubMed  Google Scholar 

  21. Nam WH, Kang DG, Lee D, Lee JY, Ra JB (2012) Automatic registration between 3D intra-operative ultrasound and pre-operative CT images of the liver based on robust edge matching. Phys Med Biol 57:69–91

    Article  PubMed  Google Scholar 

  22. Shekhar R, Dandekar O, Bhat V, Philip M, Lei P, Godinez C et al (2010) Live augmented reality: a new visualization method for laparoscopic surgery using continuous volumetric computed tomography. Surg Endosc 24:1976–1985

    Article  PubMed  Google Scholar 

  23. Hostettler A, Nicolau SA, Remond Y, Marescaux J, Soler L (2010) A real-time predictive simulation of abdominal viscera positions during quiet free breathing. Prog Biophys Mol Biol 103:169–184

    Article  CAS  PubMed  Google Scholar 

  24. Urbanavicius L, Pattyn P, de Putte DV, Venskutonis D (2011) How to assess intestinal viability during surgery: a review of techniques. World J Gastrointest Surg 3:59–69

    Article  PubMed Central  PubMed  Google Scholar 

  25. Diana M, Noll E, Diemunsch P, Dallemagne B, Benahmed MA, Agnus V, et al (2013) Enhanced-reality video fluorescence: a real-time assessment of intestinal viability. Ann Surg

Download references

Disclosures

Patrick Pessaux, Michele Diana, Luc Soler, Tullio Piardi, Didier Mutter, and Jacques Marescaux have no conflicts of interest or financial ties to disclose.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Patrick Pessaux.

Additional information

This work is part of the eHealth project PASSPORT funded by the ICT program of the European Community within the 7th Framework Program.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (WMV 223180 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pessaux, P., Diana, M., Soler, L. et al. Robotic duodenopancreatectomy assisted with augmented reality and real-time fluorescence guidance. Surg Endosc 28, 2493–2498 (2014). https://doi.org/10.1007/s00464-014-3465-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-014-3465-2

Keywords

Navigation