Skip to main content
SpringerLink
Log in

Smartphone- versus smartglasses-based augmented reality (AR) for percutaneous needle interventions: system accuracy and feasibility study

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

To compare the system accuracy and needle placement performance of smartphone- and smartglasses-based augmented reality (AR) for percutaneous needle interventions.

Methods

An AR platform was developed to enable the superimposition of annotated anatomy and a planned needle trajectory onto a patient in real time. The system accuracy of the AR display on smartphone (iPhone7) and smartglasses (HoloLens1) devices was evaluated on a 3D-printed phantom. The target overlay error was measured as the distance between actual and virtual targets (n = 336) on the AR display, derived from preprocedural CT. The needle overlay angle was measured as the angular difference between actual and virtual needles (n = 12) on the AR display. Three operators each used the iPhone (n = 8), HoloLens (n = 8) and CT-guided freehand (n = 8) to guide needles into targets in a phantom. Needle placement error was measured with post-placement CT. Needle placement time was recorded from needle puncture to navigation completion.

Results

The target overlay error of the iPhone was comparable to the HoloLens (1.75 ± 0.59 mm, 1.74 ± 0.86 mm, respectively, p = 0.9). The needle overlay angle of the iPhone and HoloLens was similar (0.28 ± 0.32°, 0.41 ± 0.23°, respectively, p = 0.26). The iPhone-guided needle placements showed reduced error compared to the HoloLens (2.58 ± 1.04 mm, 3.61 ± 2.25 mm, respectively, p = 0.05) and increased time (87 ± 17 s, 71 ± 27 s, respectively, p = 0.02). Both AR devices reduced placement error compared to CT-guided freehand (15.92 ± 8.06 mm, both p < 0.001).

Conclusion

An augmented reality platform employed on smartphone and smartglasses devices may provide accurate display and navigation guidance for percutaneous needle-based interventions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Availability of data and material

The data that support the findings of this study are available from the corresponding author, ML, upon reasonable request.

References

  1. Appelbaum L, Sosna J, Nissenbaum Y, Benshtein A, Goldberg SN (2011) Electromagnetic navigation system for CT-guided biopsy of small lesions. AJR Am J Roentgenol 196(5):1194–1200. https://doi.org/10.2214/ajr.10.5151

    Article  PubMed  Google Scholar 

  2. Krücker J, Xu S, Glossop N, Viswanathan A, Borgert J, Schulz HJWB (2007) Electromagnetic tracking for thermal ablation and biopsy guidance: clinical evaluation of spatial accuracy. J Vasc Interv Radiol 18(9):1141–1150. https://doi.org/10.1016/j.jvir.2007.06.014

    Article  PubMed  PubMed Central  Google Scholar 

  3. Kloeppel R, Weisse T, Deckert F, Wilke W, Pecher S (2000) CT-guided intervention using a patient laser marker system. Eur Radiol 10(6):1010–1014. https://doi.org/10.1007/s003300051054

    Article  CAS  PubMed  Google Scholar 

  4. Kettenbach J, Kronreif G (2015) Robotic systems for percutaneous needle-guided interventions. Minim Invasive Ther Allied Technol 24(1):45–53. https://doi.org/10.3109/13645706.2014.977299

    Article  PubMed  Google Scholar 

  5. Deng W, Li F, Wang M, Song Z (2014) Easy-to-use augmented reality neuronavigation using a wireless tablet PC. Stereotact Funct Neurosurg 1:17–24. https://doi.org/10.1159/000354816

    Article  Google Scholar 

  6. Watanabe E, Satoh M, Konno T, Hirai MTY (2016) The trans-visible navigator: a see-through neuronavigation system using augmented reality. World Neurosurg 87:399–405. https://doi.org/10.1016/j.wneu.2015.11.084

    Article  PubMed  Google Scholar 

  7. Pratt P, Ives M, Lawton G, Simmons J, Radev N, Spyropoulou L, Amiras D (2018) Through the HoloLens™ looking glass: augmented reality for extremity reconstruction surgery using 3D vascular models with perforating vessels. Eur Radiol Exp. https://doi.org/10.1186/s41747-017-0033-2

    Article  PubMed  PubMed Central  Google Scholar 

  8. Solbiati M, Passera KM, Rotilio A, Oliva F, Marre I, Goldberg SN, Ierace T, Solbiati L (2018) Augmented reality for interventional oncology: proof-of-concept study of a novel high-end guidance system platform. Eur Radiol Exp 2:18. https://doi.org/10.1186/s41747-018-0054-5

    Article  PubMed  PubMed Central  Google Scholar 

  9. Hecht R, Li M, de Ruiter QMB, Pritchard WF, Li X, Krishnasamy V, Saad W, Karanian JW, Wood BJ (2020) Smartphone augmented reality CT-based platform for needle insertion guidance: a phantom study. Cardiovasc Intervent Radiol. https://doi.org/10.1007/s00270-019-02403-6

    Article  PubMed  Google Scholar 

  10. Park BJ, Hunt SJ, Martin C, Nadolski GJ, Wood BJ, Gade TP (2020) Augmented and mixed reality: technologies for enhancing the future of IR. J Vasc Interv Radiol. https://doi.org/10.1016/j.jvir.2019.09.020

    Article  PubMed  Google Scholar 

  11. Kuzhagaliyev T, Clancy NT, Janatka M, Tchaka K, Vasconcelos F, Clarkson MJ, Gurusamy K, Hawkes DJ, Davidson B, Stoyanov D (2018) Augmented reality needle ablation guidance tool for irreversible electroporation in the pancreas. In: SPIE Medical Imaging, Houston, Texas, p 1057613. https://doi.org/10.1117/12.2293671

  12. Al-Nimer S, Hanlon A, Cho K, Kalra-Lall A, Weunski C, Yanof J, West K, Martin C 3rd (2020) 3D holographic guidance and navigation for percutaneous ablation of solid tumor. J Vasc Interv Radiol 31(3):526–528. https://doi.org/10.1016/j.jvir.2019.09.027

    Article  PubMed  Google Scholar 

  13. Heinrich F, Joeres F, Lawonn K, Hansen C (2019) Comparison of projective augmented reality concepts to support medical needle insertion. IEEE Trans Vis Comput Graph. https://doi.org/10.1109/tvcg.2019.2903942

    Article  PubMed  Google Scholar 

  14. Held RT, Hui TT (2011) A guide to stereoscopic 3D displays in medicine. Acad Radiol 18(8):1035–1048. https://doi.org/10.1016/j.acra.2011.04.005

    Article  PubMed  Google Scholar 

  15. Condino S, Carbone M, Piazza R, Ferrari M, Ferrari V (2020) Perceptual limits of optical see-through visors for augmented reality guidance of manual tasks. IEEE Trans Biomed Eng 67(2):411–419. https://doi.org/10.1109/TBME.2019.2914517

    Article  PubMed  Google Scholar 

  16. Grubert J, Itoh Y, Moser K, Swan JE (2018) A survey of calibration methods for optical see-through head-mounted displays. IEEE Trans Vis Comput Graph 24(9):2649–2662. https://doi.org/10.1109/TVCG.2017.2754257

    Article  PubMed  Google Scholar 

  17. Fichtinger G, Deguet A, Masamune K, Balogh E, Fischer GS, Mathieu H, Taylor RH, Zinreich SJ, Fayad LM (2005) Image overlay guidance for needle insertion in CT scanner. IEEE Trans Biomed Eng 52(8):1415–1424. https://doi.org/10.1109/tbme.2005.851493

    Article  PubMed  Google Scholar 

  18. Racadio JM, Nachabe R, Homan R, Schierling R, Racadio JM, Babic D (2016) Augmented reality on a C-Arm system: a preclinical assessment for percutaneous needle localization. Radiology 281(1):249–255. https://doi.org/10.1148/radiol.2016151040

    Article  PubMed  Google Scholar 

  19. Vovk A, Wild F, Guest W, Kuula T (2018) Simulator sickness in augmented reality training using the Microsoft HoloLens. In: CHI Conf. Hum. Factors Comput. Syst., Montreal QC, Canada, ACM, pp 1–9. https://doi.org/10.1145/3173574.3173783

  20. Hua H, Javidi B (2014) 3D integral imaging optical see-through head-mounted display. Opt Express 22(11):13484–13491. https://doi.org/10.1364/OE.22.013484

    Article  PubMed  Google Scholar 

  21. Lanman D, Luebke D (2013) Near-eye light field displays. ACM Trans Graph. https://doi.org/10.1145/2508363.2508366

    Article  Google Scholar 

  22. Singh G, Ellis SR, Swan JE II (2018) The effect of focal distance, age, and brightness on near-field augmented reality depth matching. J IEEE Trans Vis Comput Graph. https://doi.org/10.1109/TVCG.2018.2869729

    Article  Google Scholar 

  23. Rosenthal M, State A, Lee J, Hirota G, Ackerman J, Keller K, Pisano E, Jiroutek M, Muller K, Fuchs H (2002) Augmented reality guidance for needle biopsies: an initial randomized, controlled trial in phantoms. Med Image Anal 6(3):313–320

    Article  PubMed  Google Scholar 

  24. Wacker FK, Vogt S, Khamene A, Jesberger JA, Nour SG, Elgort DR, Sauer F, Duerk JL, Lewin JS (2006) An augmented reality system for MR image-guided needle biopsy: initial results in a swine model. Radiology 238(2):497–504. https://doi.org/10.1148/radiol.2382041441

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This research was supported by the Intramural Research Program of the NIH Center for Interventional Oncology.

Funding

This work was supported by the Center for Interventional Oncology in the Intramural Research Program of the National Institutes of Health (NIH) by intramural NIH Grants NIH Z01 1ZID BC011242 and CL040015.

Author information

Authors and Affiliations

  1. Center for Interventional Oncology, Radiology and Imaging Sciences, Clinical Center, National Institutes of Health, Bethesda, MD, 20892, USA

    Ming Li, Reza Seifabadi, Dilara Long, Quirina De Ruiter, Nicole Varble, Rachel Hecht, Ayele H. Negussie, Venkatesh Krishnasamy, Sheng Xu & Bradford J. Wood

  2. Philips Research of North America, Cambridge, MA, USA

    Nicole Varble

Authors
  1. Ming Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Reza Seifabadi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dilara Long
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Quirina De Ruiter
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nicole Varble
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Rachel Hecht
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ayele H. Negussie
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Venkatesh Krishnasamy
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Sheng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Bradford J. Wood
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ming Li.

Ethics declarations

Conflicts of interest

This research was supported in part by the Center for Interventional Oncology and the Intramural Research Program of the NIH. NV is an employee of Philips Research North America, Cambridge, MA, USA.

Code availability

The code that support the findings of this study are available from the corresponding author, ML, upon reasonable request.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, M., Seifabadi, R., Long, D. et al. Smartphone- versus smartglasses-based augmented reality (AR) for percutaneous needle interventions: system accuracy and feasibility study. Int J CARS 15, 1921–1930 (2020). https://doi.org/10.1007/s11548-020-02235-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-020-02235-7

Keywords

Search

Navigation