Skip to main content
SpringerLink
Log in

Evaluation of a marker-less, intra-operative, augmented reality guidance system for robot-assisted laparoscopic radical prostatectomy

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Robot-assisted laparoscopic radical prostatectomy (RALRP) using the da Vinci surgical robot is a common treatment for organ-confined prostate cancer. Augmented reality (AR) can help during RALRP by showing the surgeon the location of anatomical structures and tumors from preoperative imaging. Previously, we proposed hand-eye and camera intrinsic matrix estimation procedures that can be carried out with conventional instruments within the patient during surgery, take < 3 min to perform, and fit seamlessly in the existing surgical workflow. In this paper, we describe and evaluate a complete AR guidance system for RALRP and quantify its accuracy.

Methods

Our AR system requires three transformations: the transrectal ultrasound (TRUS) to da Vinci transformation, the camera intrinsic matrix, and the hand-eye transformation. For evaluation, a 3D-printed cross-wire was visualized in TRUS and stereo endoscope in a water bath. Manually triangulated cross-wire points from stereo images were used as ground truth to evaluate overall TRE between these points and points transformed from TRUS to camera.

Results

After transforming the ground-truth points from the TRUS to the camera coordinate frame, the mean target registration error (TRE) (SD) was \(4.56\pm 1.57\) mm. The mean TREs (SD) in the x-, y-, and z-directions are \(1.93\pm 1.26\) mm, \(2.04\pm 1.37\) mm, and \(2.94\pm 1.84\) mm, respectively.

Conclusions

We describe and evaluate a complete AR guidance system for RALRP which can augment preoperative data to endoscope camera image, after a deformable magnetic resonance image to TRUS registration step. The streamlined procedures with current surgical workflow and low TRE demonstrate the compatibility and readiness of the system for clinical translation. A detailed sensitivity study remains part of future work.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Adebar T, Salcudean S, Mahdavi S, Moradi M, Nguan C, Goldenberg L (2011) A robotic system for intra-operative trans-rectal ultrasound and ultrasound elastography in radical prostatectomy. In: Taylor RH, Yang GZ (eds) Information processing in computer-assisted interventions. Springer, Berlin, pp 79–89

    Chapter  Google Scholar 

  2. Chen EC, Morgan I, Jayarathne U, Ma B, Peters TM (2017) Hand-eye calibration using a target registration error model. Healthc Technol Lett 4(5):157–162

    Article  CAS  Google Scholar 

  3. Cooperberg MR, Lubeck DP, Meng MV, Mehta SS, Carroll PR (2004) The changing face of low-risk prostate cancer: trends in clinical presentation and primary management. J Clin Oncol 22(11):2141

    Article  Google Scholar 

  4. DiMaio S, Hasser C (2008) The da Vinci research interface. MIDAS J. http://hdl.handle.net/10380/1464

  5. Edgcumbe P, Singla R, Pratt P, Schneider C, Nguan C, Rohling R (2016) Augmented reality imaging for robot-assisted partial nephrectomy surgery. In: International conference on medical imaging and augmented reality. Springer, pp 139–150

  6. Groves L, Vanberlo B, Peters T, Chen E (2019) A deep learning approach for automatic out-of-plane needle localization for semi-automatic ultrasound probe calibration. Healthc Technol Lett 6:204–209

    Article  Google Scholar 

  7. Inoue S, Shiina H, Hiraoka T, Mitsui Y, Sumura M, Urakami S, Igawa M (2009) Retrospective analysis of the distance between the neurovascular bundle and prostate cancer foci in radical prostatectomy specimens: its clinical implication in nerve-sparing surgery. BJU Int 104(8):1085–1090

    Article  Google Scholar 

  8. Kalia M, Mathur P, Navab N, Salcudean T (2019) A marker-less real time intra-operative camera and hand-eye calibration procedure for surgical augmented reality. Healthc Technol Lett 6:255–260

    Article  Google Scholar 

  9. Lanchon C, Custillon G, Moreau-Gaudry A, Descotes JL, Long JA, Fiard G, Voros S (2016) Augmented reality using transurethral ultrasound for laparoscopic radical prostatectomy: preclinical evaluation. J Urol 196(1):244–250

    Article  Google Scholar 

  10. Linte CA, Davenport KP, Cleary K, Peters C, Vosburgh KG, Navab N, Jannin P, Peters TM, Holmes DR III, Robb RA (2013) On mixed reality environments for minimally invasive therapy guidance: systems architecture, successes and challenges in their implementation from laboratory to clinic. Comput Med Imaging Graph 37(2):83–97

    Article  Google Scholar 

  11. Mohareri O, Ischia J, Black PC, Schneider C, Lobo J, Goldenberg L, Salcudean SE (2015) Intraoperative registered transrectal ultrasound guidance for robot-assisted laparoscopic radical prostatectomy. J Urol 193(1):302–312

    Article  Google Scholar 

  12. Novara G, Ficarra V, Rosen RC, Artibani W, Costello A, Eastham JA, Graefen M, Guazzoni G, Shariat SF, Stolzenburg JU, Van Poppelk H, Zattonia F, Montorsil F, Mottrieb A, G. Wilson T (2012) Systematic review and meta-analysis of perioperative outcomes and complications after robot-assisted radical prostatectomy. Eur Urol 62(3):431–452

    Article  Google Scholar 

  13. Pachtrachai K, Allan M, Pawar V, Hailes S, Stoyanov D (2016) Hand-eye calibration for robotic assisted minimally invasive surgery without a calibration object. In: 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 2485–2491

  14. Pratt P, Bergeles C, Darzi A, Yang GZ (2014) Practical intraoperative stereo camera calibration. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 667–675

  15. Samei G, Tsang K, Kesch C, Lobo J, Hor S, Mohareri O, Chang S, Goldenberg SL, Black PC, Salcudean S (2019) A partial augmented reality system with live ultrasound and registered preoperative MRI for guiding robot-assisted radical prostatectomy. Med Image Anal 60:101588

    Article  Google Scholar 

  16. Shao J, Luo H, Xiao D, Hu Q, Jia F (2017) Progressive hand-eye calibration for laparoscopic surgery navigation. In: Cardoso J, Arbel T, Luo X, Wesarg S, Reichl T, González Ballester MÁ, McLeod J, Drechsler K, Peters TM, Erdt M, Mori K, Linguraru MG, Uhl A, Oyarzun Laura C, Shekhar R (eds) Computer assisted and robotic endoscopy and clinical image-based procedures. Springer, New York, pp 42–49

    Chapter  Google Scholar 

  17. Silberstein JL, Eastham JA (2014) Significance and management of positive surgical margins at the time of radical prostatectomy. Indian J Urol IJU J Urol Soc India 30(4):423

    Article  Google Scholar 

  18. Thompson S, Stoyanov D, Schneider C, Gurusamy K, Ourselin S, Davidson B, Hawkes D, Clarkson MJ (2016) Hand-eye calibration for rigid laparoscopes using an invariant point. Int J Comput Assist Radiol Surg 11(6):1071–1080

    Article  Google Scholar 

  19. Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22:1330–1334

    Article  Google Scholar 

Download references

Acknowledgements

The authors are thankful for financial support provided by the Canadian Institutes of Health Research (CIHR), the Natural Sciences and Engineering Research Council (NSERC), and the Charles Laszlo Chair in Biomedical Engineering held by Professor Salcudean. The authors thank the Canadian Research Chairs (CRC) and Canada Foundation of Innovation (CFI) for their support. The authors would also like to thank Intuitive Surgical for providing the research API and support.

Author information

Author notes

  1. Megha Kalia and Prateek Mathur are co-first authors and contributed equally to the manuscript.

Authors and Affiliations

  1. Electrical and Computer Engineering, University of British Columbia, 2329 West Mall, Vancouver, BC, V6T 1Z4, Canada

    Megha Kalia, Prateek Mathur, Keith Tsang & Septimiu Salcudean

  2. Computer Aided Medical Procedures, Technical University of Munich, Boltzmannstraße 15, 85748, Garching bei München, Germany

    Megha Kalia & Nassir Navab

  3. Vancouver Prostate Centre, Department of Urologic Sciences, University of British Columbia, Vancouver, BC, V5Z 1M9, Canada

    Peter Black

Authors
  1. Megha Kalia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Prateek Mathur
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Keith Tsang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peter Black
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nassir Navab
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Septimiu Salcudean
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Megha Kalia or Prateek Mathur.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical standard

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

This article does not contain patient data.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kalia, M., Mathur, P., Tsang, K. et al. Evaluation of a marker-less, intra-operative, augmented reality guidance system for robot-assisted laparoscopic radical prostatectomy. Int J CARS 15, 1225–1233 (2020). https://doi.org/10.1007/s11548-020-02181-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-020-02181-4

Keywords

Search

Navigation