Skip to main content

Advertisement

SpringerLink
Log in

Using game controller as position tracking sensor for 3D freehand ultrasound imaging

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Position tracking has been widely used in medical applications, especially in 3D ultrasound imaging, where it has transformed the 2D slice limitation into 3D volume with bigger clinical impacts. As a game controller can also produce position tracking information, it has the potential to act as a low-cost and portable position tracker for ultrasound probes. This paper aims to investigate the feasibility of a game controller to perform as a position tracker and to design its implementation in 3D ultrasound imaging. The study consists of data acquisition and 3D ultrasound reconstruction for visualization. The data acquisition is accomplished by capturing the 2D ultrasound frame and its relative positional and orientation data by using an ultrasound probe and game controller respectively. These data are further reconstructed to produce 3D ultrasound volume for visualization. Our experiments include game controller position tracker testing and 3D ultrasound reconstruction on baby phantom. The results have confirmed that the game controller performance was closely aligned with that of in a robot arm. Also, the 3D ultrasound reconstruction implementation has revealed promising outcomes. With these features, the function of the currently available ultrasound probes can be prospectively improved using a game controller position tracker effectively.

Graphical Abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Sippel S, Muruganandan K, Levine A, Shah S (2011) Review article: use of ultrasound in the developing world. Int J Emerg Med 4(1):72

    Article  PubMed  PubMed Central  Google Scholar 

  2. Harvey CJ, Pilcher JM, Eckersley RJ, Blomley MJK, Cosgrove DO (2002) Advances in ultrasound. Clin Radiol 57(3):157–177

    Article  PubMed  Google Scholar 

  3. Gee A, Prager R, Treece G, Berman L (2003) Engineering a freehand 3D ultrasound system. Pattern Recogn Lett 24(4-5):757–777

    Article  Google Scholar 

  4. De Fiori E, Rampinelli C, Turco F, Bonello L, Bellomi M (2010) Role of operator experience in ultrasound-guided fine-needle aspiration biopsy of the thyroid. Radiol Med 115(4):612–618

    Article  PubMed  Google Scholar 

  5. Downey DB, Fenster A, Williams JC (2000) Clinical Utility of three-dimensional US. RadioGraphics 20(2):559–571

    Article  CAS  PubMed  Google Scholar 

  6. Fenster A, Parraga G, Bax J (2011) Three-dimensional ultrasound scanning. Interface Focus 1 (June):503–519

    Article  PubMed  PubMed Central  Google Scholar 

  7. Fenster A, Downey DB, Cardinal HN (2001) Three-dimensional ultrasound imaging. Phys Med Biol 46 (5):R67

    Article  CAS  PubMed  Google Scholar 

  8. Scharf A, Ghazwiny MF, Steinborn A, Baier P, Sohn C (2001) Evaluation of two-dimensional versus three-dimensional ultrasound in obstetric diagnostics: a prospective study. Fetal Diagn Ther 16(6):333–341

    Article  CAS  PubMed  Google Scholar 

  9. Gee A, Prager R, Treece G, Cash C, Berman L (2004) Processing and visualizing three-dimensional ultrasound data. Br J Radiol, 77 (SPEC. ISS. 2)

  10. Deb S, Campbell BK, Clewes JS, Raine-Fenning NJ (2010) Quantitative analysis of antral follicle number and size: a comparison of two-dimensional and automated three-dimensional ultrasound techniques. Ultrasound Obstet Gynecol 35(3):354– 360

    Article  CAS  PubMed  Google Scholar 

  11. Hata T, Tanaka H, Noguchi J, Hata K (2011) Three-dimensional ultrasound evaluation of the placenta. Placenta 32(2):105–115

    Article  CAS  PubMed  Google Scholar 

  12. Housden RJ, Gee AH, Treece GM, Prager RW (2007) Sensorless reconstruction of unconstrained freehand 3D ultrasound data. Ultrasound Med Biol 33(3):408–419

    Article  PubMed  Google Scholar 

  13. Gee AH, Housden RJ, Hassenpflug P, Treece GM, Prager RW (2006) Sensorless freehand 3D ultrasound in real tissue: speckle decorrelation without fully developed speckle. Med Image Anal 10(2):137–149

    Article  PubMed  Google Scholar 

  14. Prager RW, Ijaz UZ, Gee AH, Treece GM (2010) Three-dimensional ultrasound imaging. Proc Inst Mech Eng H J Eng Med 224(2):193–223

    Article  CAS  Google Scholar 

  15. Yu H, Pattichis MS, Agurto C, Goens MB (2011) A 3D freehand ultrasound system for multi-view reconstructions from sparse 2D scanning planes. BioMedical Engineering Online 10(1):7

    Article  PubMed  PubMed Central  Google Scholar 

  16. Chen X, Wen T, Li X, Qin W, Lan D, Pan W, Gu J (2014) Reconstruction of freehand 3D ultrasound based on kernel regression. BioMedical Engineering OnLine 13(1):124

    Article  PubMed  PubMed Central  Google Scholar 

  17. Wein W, Röper B, Navab N (2007) Integrating diagnostic B-mode ultrasonography into CT-based radiation treatment planning. IEEE Trans Med Imaging 26(6):866–879

    Article  PubMed  Google Scholar 

  18. Zhang H, Banovac F, Lin R, Glossop N, Wood B, Lindisch D, Levy E, Cleary K (2006) Electromagnetic tracking for abdominal interventions in computer aided surgery. In: Computer aided surgery, vol 11, pp 127–136

  19. Cleary K, Peters T (2010) Image-guided interventions: review technology and clinical applications. Annu Rev Biomed Eng 12:119–142

    Article  CAS  PubMed  Google Scholar 

  20. Moon H, Ju G, Park S, Shin H (2016) 3D freehand ultrasound reconstruction using a piecewise smooth Markov random field. Comput Vis Image Underst 151:101–113

    Article  Google Scholar 

  21. Chung S-W, Shih C-C, Huang C-C (2017) Freehand three-dimensional ultrasound imaging of carotid artery using motion tracking technology. Ultrasonics 74:11–20

    Article  PubMed  Google Scholar 

  22. Khatib F, Cooper S, Tyka MD, Xu K, Makedon I, Popovic Z, Baker D (2011) Algorithm discovery by protein folding game players. Proc Natl Acad Sci 108(47):18949–18953

    Article  CAS  PubMed  Google Scholar 

  23. Kitsunezaki N, Adachi E, Masuda T, Mizusawa JI (2013) KINECT applications for the physical rehabilitation. In: MeMea 2013 - IEEE International Symposium on Medical Measurements and Applications, Proceedings, pp 294–299

  24. Giovanni S, Choi YC, Huang J, Khoo ET, Yin KK (2012) Virtual try-on using Kinect and HD camera motion in games SE-6, pp 55–65

  25. Bailey SW, Bodenheimer B (2012) A comparison of motion capture data recorded from a Vicon system and a Microsoft Kinect sensor. In: Proceedings of the ACM Symposium on Applied Perception, pp 121–121

  26. Ha S, Bai Y, Liu CK (2011) Human motion reconstruction from force sensors. In: Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation - SCA ’11, pp 129–128

  27. Aldrich JE (2007) Basic physics of ultrasound imaging

  28. Steiner H, Staudach A, Spitzer D, Schaffer H (1994) Three-dimensional ultrasound in obstetrics and gynaecology: technique, possibilities and limitations. Hum Reprod (Oxford, England) 9(9):1773–1778

    Article  CAS  Google Scholar 

  29. Salim R, Woelfer B, Backos M, Regan L, Jurkovic D (2003) Reproducibility of three-dimensional ultrasound diagnosis of congenital uterine anomalies. Ultrasound Obstet Gynecol 21(6):578–582

    Article  CAS  PubMed  Google Scholar 

  30. Bonilla-Musoles F, Raga F, Osborne NG (1995) Three-dimensional ultrasound evaluation of ovarian masses. Gynecol Oncol 59(1):129–135

    Article  CAS  PubMed  Google Scholar 

  31. Kyei-Mensah A, Maconochie N, Zaidi J, Pittrof R, Campbell S, Tan SL (1996) Transvaginal three-dimensional ultrasound: reproducibility of ovarian and endometrial volume measurements. Fertil Steril 66 (5):718–722

    Article  CAS  PubMed  Google Scholar 

  32. Purnama KE, Wilkinson MHF, Veldhuizen AG, Van Ooijen PMA, Lubbers J, Burgerhof JGM, Sardjono TA, Verkerke GJ (2010) A framework for human spine imaging using a freehand 3D ultrasound system. Technol Health Care 18(1):1–17

    Article  PubMed  Google Scholar 

  33. Nguyen DV, Vo QN, Le LH, Lou EHM (2015) Validation of 3D surface reconstruction of vertebrae and spinal column using 3D ultrasound data – a pilot study. Med Eng Phys 37(2):239–244

  34. Elliot TL, Downey DB, Tong S, McLean CA, Fenster A (1996) Accuracy of prostate volume measurements in vitro using three-dimensional ultrasound. Acad Radiol 3(5):401–406

    Article  CAS  PubMed  Google Scholar 

  35. Leen E, Kumar S, Khan SA, Low G, Ong KO, Tait P, Averkiou M (2009) Contrast-enhanced 3D ultrasound in the radiofrequency ablation of liver tumors. World J Gastroenterol 15(3):289–299

    Article  PubMed  PubMed Central  Google Scholar 

  36. Perl T (2012) Cross-platform tracking of a 6dof motion controller. PhD Thesis Vienna University of Technology

  37. Ioannou D, Huda W, Laine AF (1999) Circle recognition through a 2D Hough transform and radius histogramming. Image Vis Comput 17(1):15–26

    Article  Google Scholar 

  38. Bradley D, Roth G (2005) Natural interaction with virtual objects using vision-based six DOF sphere tracking natural interaction with virtual objects using vision-based six DOF sphere tracking. In: Proceedings of the 2005 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, pp 19–26

  39. Dawei L, Qingming H, Shuqiang J, Hongxun Y, Wen G (2006) Mean-shift blob tracking with adaptive feature selection and scale adaptation. Proceedings - International Conference on Image Processing, ICIP 3:369–372

    Google Scholar 

  40. Miletitch R, de Courville R, Rébulard M, Danet C, Doan P, Boutet D (2012) Real-time 3D gesture visualisation for the study of Sign Language. In: Electronic Visualisation & the Arts (EVA), pp 275–280

  41. Shoemake K (1985) Animating rotation with quaternion curves. ACM SIGGRAPH Computer Graphics 19 (3):245–254

    Article  Google Scholar 

  42. Skehan DP (2011) Virtual retraining system for diagnostic ultrasound. PhD thesis, Worchester Polytechnic Institute

  43. Kong MX, Ji C, Chen ZS, Li RF (2013) Application of orientation interpolation of robot using unit quaternion. In: 2013 IEEE International Conference on Information and Automation, ICIA 2013, pp 384–389

  44. Solberg OV, Lindseth F, Bø LE, Muller S, Bakeng JBL, Tangen GA, Hernes TAN (2011) 3D ultrasound reconstruction algorithms from analog and digital data. Ultrasonics 51(4):405–419

    Article  PubMed  Google Scholar 

  45. Rohling R, Gee A, Berman L (1999) A comparison of freehand three-dimensional ultrasound reconstruction techniques. Med Image Anal 3(4):339–359

    Article  CAS  PubMed  Google Scholar 

  46. Nelson TR, Pretorius DH (1998) Three-dimensional ultrasound imaging. Ultrasound Med Biol 24 (9):1243–1270

    Article  CAS  PubMed  Google Scholar 

  47. Dewi DEO, Mengko TLR, Purnama AG, Veldhuizen IKE, Wilkinson MHF (2010) An improved olympic hole-filling method for ultrasound volume reconstruction of human spine. International Journal of E-Health and Medical Communications 1(3):28–40

    Article  Google Scholar 

  48. Wan MH, Tan K, Supriyanto E (2010) 3D ultrasound image reconstruction based on VTK

Download references

Funding

This work is supported by the Fundamental Research Grant Scheme of Ministry of Education of Malaysia under the grant FRGS/1/2014/ICT07/UTM/03/1 and Research University Grant of Universiti Teknologi Malaysia – Potential Academic Staff Grant Q.J130000.2709.01K87.

Author information

Authors and Affiliations

  1. School of Computing, Faculty of Engineering, Universiti Teknologi Malaysia, Johor Bahru, Malaysia

    Vei Siang Chan, Farhan Mohamed, Yusman Azimi Yusoff & Wey Sheng Mong

  2. School of Biomedical Engineering & Health Sciences, Faculty of Engineering, Universiti Teknologi Malaysia, Johor Bahru, Malaysia

    Dyah Ekashanti Octorina Dewi

  3. Media and Games Innovation Centre of Excellence, Institute of Human Centre Engineering, Universiti Teknologi Malaysia, Johor Bahru, Malaysia

    Alfiera Anuar

  4. School of Electrical Engineering, Faculty of Engineering, Universiti Teknologi Malaysia, Johor Bahru, Malaysia

    Mohamad Amir Shamsudin

Authors
  1. Vei Siang Chan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Farhan Mohamed
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yusman Azimi Yusoff
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dyah Ekashanti Octorina Dewi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Alfiera Anuar
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mohamad Amir Shamsudin
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Wey Sheng Mong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vei Siang Chan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chan, V.S., Mohamed, F., Yusoff, Y.A. et al. Using game controller as position tracking sensor for 3D freehand ultrasound imaging. Med Biol Eng Comput 58, 889–902 (2020). https://doi.org/10.1007/s11517-019-02044-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-019-02044-4

Keywords

Search

Navigation