Paulo Bala
Valentina Nisi
Nuno Nunes
ABSTRACT
Visually Induced Motion Sickness (VIMS), when the visual system detects motion that is not felt by the vestibular system, is a deterrent for first-time Virtual Reality (VR) users and can impact its adoption rate. Constricting the field-of-view (FoV) has been shown to reduce VIMS as it conceals optical flow in peripheral vision, which is more sensitive to motion. Additionally, several studies have suggested the inclusion of visual elements (e.g., grids) consistent with the real world as reference points. In this paper, we describe a novel technique dynamically controlled by a video’s precomputed optical flow and participants’ runtime head direction and evaluate it in a within-subjects study (N = 24) on a 360° video of a roller coaster. Furthermore, based on a detailed analysis of the video and participant’s experience, we provide insights on the effectiveness of the techniques in VIMS reduction and discuss the role of optical flow in the design and evaluation of the study.
Supplemental Material
Available for Download
pdf
supplementarymaterial_imx2020__final_.pdf (2.2 MB)
Supplementary Material for "Staying on Track: a Comparative Study on the Use of Optical Flow in 360° Video to Mitigate VIMS"
- 2015. [Extreme] 360° RollerCoaster at Seoul Grand Park. https://www.youtube.com/watch?v=8lsB-P8nGSMGoogle Scholar
- 2017. Introduction to Best Practices. https://developer.oculus.com/design/latest/concepts/bp_intro/Google Scholar
- 2017. Tunneling Demo | Google VR. https://developers.google.com/vr/elements/tunnelingGoogle Scholar
- Majed Al Zayer, Isayas B. Adhanom, Paul MacNeilage, and Eelke Folmer. 2019. The Effect of Field-of-View Restriction on Sex Bias in VR Sickness and Spatial Navigation Performance. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems(CHI ’19). ACM, New York, NY, USA, 354:1–354:12. https://doi.org/10.1145/3290605.3300584Google ScholarDigital Library
- Simon Baker and Iain Matthews. 2004. Lucas-Kanade 20 Years On: A Unifying Framework. International Journal of Computer Vision 56, 3 (Feb. 2004), 221–255. https://doi.org/10.1023/B:VISI.0000011205.11775.fdGoogle ScholarDigital Library
- Paulo Bala, Raul Masu, Valentina Nisi, and Nuno Nunes. 2019. ”When the Elephant Trumps”: A Comparative Study on Spatial Audio for Orientation in 360° Videos. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems(CHI ’19). ACM, New York, NY, USA, 695:1–695:13. https://doi.org/10.1145/3290605.3300925Google ScholarDigital Library
- A. Berthoz, B. Pavard, and L. R. Young. 1975. Perception of linear horizontal self-motion induced by peripheral vision (linearvection) basic characteristics and visual-vestibular interactions. Experimental Brain Research 23, 5 (Nov. 1975), 471–489.Google ScholarCross Ref
- Jiwan Bhandari, Paul MacNeilage, and Eelke Folmer. 2018. Teleportation without spatial disorientation using optical flow cues. In Proceedings of Graphics Interface, Vol. 2018.Google Scholar
- Roger Bivand, L. Anselin, O. Berke, A. Bernat, M. Carvalho, Y. Chun, C. F. Dormann, S. Dray, R. Halbersma, and N. Lewin-Koh. 2011. spdep: Spatial dependence: weighting schemes, statistics and models. R package version 0.5-31, URL http://CRAN. R-project. org/package= spdep.Google Scholar
- Roger Bivand, Tim Keitt, Barry Rowlingson, and E. Pebesma. 2014. rgdal: Bindings for the geospatial data abstraction library. R package version 0.8-16(2014).Google Scholar
- Mark Bolas, J. Adam Jones, Ian McDowall, and Evan Suma. 2017. Dynamic field of view throttling as a means of improving user experience in head mounted virtual environments.Google Scholar
- Doug A. Bowman, Ernst Kruijff, Joseph J. LaViola, and Ivan Poupyrev. 2004. 3D User Interfaces: Theory and Practice. Addison Wesley Longman Publishing Co., Inc., Redwood City, CA, USA.Google ScholarDigital Library
- Th. Brandt, Johannes Dichgans, and Ed Koenig. 1973. Differential effects of central versus peripheral vision on egocentric and exocentric motion perception. Experimental Brain Research 16, 5 (March 1973), 476–491. https://doi.org/10.1007/BF00234474Google ScholarCross Ref
- Mi-Hyun Choi, Soo-Jeong Lee, Hyo-Sung Kim, Jae-Woong Yang, Jin-Seung Choi, Gye-Rae Tack, Bongsoo Lee, Soon-Cheol Chung, Soo-Young Min, and Byung-Chan Min. 2009. Long-term study of simulator sickness: differences in psychophysiological responses due to individual sensitivity. In 2009 International Conference on Mechatronics and Automation. IEEE, 20–25.Google ScholarCross Ref
- Henry E. Cook, Justin A. Hassebrock, and L. James Smart. 2018. Responding to Other People’s Posture: Visually Induced Motion Sickness From Naturally Generated Optic Flow. Frontiers in Psychology 9 (2018), 1901. https://doi.org/10.3389/fpsyg.2018.01901Google ScholarCross Ref
- Simon Davis, Keith Nesbitt, and Eugene Nalivaiko. 2015. Comparing the onset of cybersickness using the Oculus Rift and two virtual roller coasters. In Proceedings of the 11th Australasian Conference on Interactive Entertainment (IE 2015), Vol. 27. 30. http://crpit.com/confpapers/CRPITV167Davis.pdfGoogle Scholar
- M. H. Draper, E. S. Viire, T. A. Furness, and V. J. Gawron. 2001. Effects of image scale and system time delay on simulator sickness within head-coupled virtual environments. Human Factors 43, 1 (2001), 129–146. https://doi.org/10.1518/001872001775992552Google ScholarCross Ref
- Huiyu Duan, Guangtao Zhai, Xiongkuo Min, Yucheng Zhu, Wei Sun, and Xiaokang Yang. 2017. Assessment of Visually Induced Motion Sickness in Immersive Videos. In Advances in Multimedia Information Processing —PCM 2017(Lecture Notes in Computer Science). Springer, Cham, 662–672. https://doi.org/10.1007/978-3-319-77380-3_63Google Scholar
- Henry Been-Lirn Duh, Donald E. Parker, and Thomas A. Furness. 2001. An “Independent Visual Background” Reduced Balance Disturbance Envoked by Visual Scene Motion: Implication for Alleviating Simulator Sickness(CHI ’01). Association for Computing Machinery, New York, NY, USA, 85–89. https://doi.org/10.1145/365024.365051Google ScholarDigital Library
- Henry Been-Lirn Duh, Donald E. Parker, and Thomas A. Furness. 2004. An Independent Visual Background Reduced Simulator Sickness in a Driving Simulator. Presence: Teleoperators and Virtual Environments 13, 5(2004), 578–588. https://doi.org/10.1162/1054746042545283 arXiv:https://doi.org/10.1162/1054746042545283Google ScholarDigital Library
- Natalia Dużmańska, Paweł Strojny, and Agnieszka Strojny. 2018. Can Simulator Sickness Be Avoided? A Review on Temporal Aspects of Simulator Sickness. Frontiers in Psychology 9 (Nov. 2018). https://doi.org/10.3389/fpsyg.2018.02132Google Scholar
- Sheldon M. Ebenholtz. 2001. Oculomotor systems and perception. Cambridge University Press, New York.Google Scholar
- Gunnar Farnebäck. 2003. Two-frame motion estimation based on polynomial expansion. Image analysis (2003), 363–370.Google ScholarDigital Library
- Ajoy S. Fernandes and Steven K. Feiner. 2016. Combating VR sickness through subtle dynamic field-of-view modification. In 2016 IEEE Symposium on 3D User Interfaces (3DUI). 201–210. https://doi.org/10.1109/3DUI.2016.7460053Google Scholar
- Jos Feys. 2016. Nonparametric tests for the interaction in two-way factorial designs using R. The R Journal 8, 1 (2016), 367–378.Google ScholarCross Ref
- Andy Field, Jeremy Miles, and Zoë Field. 2012. Discovering statistics using R. Sage publications.Google ScholarDigital Library
- Moira B. Flanagan, James G. May, and Thomas G. Dobie. 2005. Sex differences in tolerance to visually-induced motion sickness. Aviation, space, and environmental medicine 76, 7 (2005), 642–646.Google Scholar
- Augusto Garcia-Agundez, Aiko Westmeier, Polona Caserman, Robert Konrad, and Stefan Göbel. 2017. An Evaluation of Extrapolation and Filtering Techniques in Head Tracking for Virtual Environments to Reduce Cybersickness. In Serious Games(Lecture Notes in Computer Science). Springer, Cham, 203–211. https://doi.org/10.1007/978-3-319-70111-0_19Google Scholar
- John F Golding. 1998. Motion sickness susceptibility questionnaire revised and its relationship to other forms of sickness. Brain Research Bulletin 47, 5 (Nov. 1998), 507–516. https://doi.org/10.1016/S0361-9230(98)00091-4Google ScholarCross Ref
- Peter A. Howarth and Simon G. Hodder. 2008. Characteristics of habituation to motion in a virtual environment. Displays 29, 2 (2008), 117 – 123. https://doi.org/10.1016/j.displa.2007.09.009 Health and Safety Aspects of Visual Displays.Google ScholarCross Ref
- Haikun Huang, Michael Solah, Dingzeyu Li, and Lap-Fai Yu. 2019. Audible Panorama: Automatic Spatial Audio Generation for Panorama Imagery. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems(CHI ’19). ACM, New York, NY, USA, 621:1–621:11. https://doi.org/10.1145/3290605.3300851Google ScholarDigital Library
- J. Adam Jones, David M. Krum, and Mark T. Bolas. 2017. Vertical field-of-view extension and walking characteristics in head-worn virtual environments. ACM Transactions on Applied Perception (TAP) 14, 2 (2017), 9.Google Scholar
- Nupur Kala, Kyungmin Lim, Kwanghyun Won, Jaesung Lee, Tammy Lee, Sehoon Kim, and Wonhee Choe. 2017. P-218: An Approach to Reduce VR Sickness by Content Based Field of View Processing. SID Symposium Digest of Technical Papers 48, 1 (May 2017), 1645–1648. https://doi.org/10.1002/sdtp.11956Google ScholarCross Ref
- Shunichi Kasahara, Shohei Nagai, and Jun Rekimoto. 2015. First Person Omnidirectional Video: System Design and Implications for Immersive Experience(TVX ’15). Association for Computing Machinery, New York, NY, USA, 33–42. https://doi.org/10.1145/2745197.2745202Google ScholarDigital Library
- Robert S. Kennedy, Norman E. Lane, Kevin S. Berbaum, and Michael G. Lilienthal. 1993. Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness. The International Journal of Aviation Psychology 3, 3 (July 1993), 203–220. https://doi.org/10.1207/s15327108ijap0303_3Google ScholarCross Ref
- Behrang Keshavarz, Heiko Hecht, and Lisa Zschutschke. 2011. Intra-visual conflict in visually induced motion sickness. Displays 32, 4 (Oct. 2011), 181–188. https://doi.org/10.1016/j.displa.2011.05.009Google ScholarCross Ref
- Nam-Gyoon Kim and Beom-Su Kim. 2019. The Effect of Retinal Eccentricity on Visually Induced Motion Sickness and Postural Control. Applied Sciences 9 (05 2019), 1–9. https://doi.org/10.3390/app9091919Google Scholar
- Young Youn Kim, Eun Nam Kim, Min Jae Park, Kwang Suk Park, Hee Dong Ko, and Hyun Taek Kim. 2008. The Application of Biosignal Feedback for Reducing Cybersickness from Exposure to a Virtual Environment. Presence 17, 1 (Feb. 2008), 1–16. https://doi.org/10.1162/pres.17.1.1Google ScholarDigital Library
- Joseph J. LaViola, Jr.2000. A Discussion of Cybersickness in Virtual Environments. SIGCHI Bull. 32, 1 (Jan. 2000), 47–56. https://doi.org/10.1145/333329.333344Google ScholarDigital Library
- Jiun-Yu Lee, Ping-Hsuan Han, Ling Tsai, Rih-Ding Peng, Yang-Sheng Chen, Kuan-Wen Chen, and Yi-Ping Hung. 2017. Estimating the Simulator Sickness in Immersive Virtual Reality with Optical Flow Analysis. In SIGGRAPH Asia 2017 Posters(SA ’17). ACM, New York, NY, USA, 16:1–16:2. https://doi.org/10.1145/3145690.3145697Google ScholarDigital Library
- James Jeng-Weei Lin, Habib Abi-Rached, Do-Hoe Kim, Donald E. Parker, and Thomas A. Furness. 2002. A “Natural” Independent Visual Background Reduced Simulator Sickness. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 46, 26 (2002), 2124–2128. https://doi.org/10.1177/154193120204602605 arXiv:https://doi.org/10.1177/154193120204602605Google Scholar
- Wen-Chih Lo, Ching-Ling Fan, Jean Lee, Chun-Ying Huang, Kuan-Ta Chen, and Cheng-Hsin Hsu. 2017. 360° Video Viewing Dataset in Head-Mounted Virtual Reality. In Proceedings of the 8th ACM on Multimedia Systems Conference(MMSys’17). ACM, New York, NY, USA, 211–216. https://doi.org/10.1145/3083187.3083219Google ScholarDigital Library
- Ville Mäkelä, Tuuli Keskinen, John Mäkelä, Pekka Kallioniemi, Jussi Karhu, Kimmo Ronkainen, Alisa Burova, Jaakko Hakulinen, and Markku Turunen. 2019. What Are Others Looking at? Exploring 360° Videos on HMDs with Visual Cues about Other Viewers(TVX ’19). Association for Computing Machinery, New York, NY, USA, 13–24. https://doi.org/10.1145/3317697.3323351Google ScholarDigital Library
- Mark McGill, Alexander Ng, and Stephen Brewster. 2017. I Am The Passenger: How Visual Motion Cues Can Influence Sickness For In-Car VR. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems(CHI ’17). ACM, New York, NY, USA, 5655–5668. https://doi.org/10.1145/3025453.3026046Google ScholarDigital Library
- Doug McIlroy, Ray Brownrigg, Thomas P. Minka, and Roger Bivand. 2014. Mapproj: map projections. R package version (2014), 1–2.Google Scholar
- Jason D. Moss and Eric R. Muth. 2011. Characteristics of Head-Mounted Displays and Their Effects on Simulator Sickness. Human Factors: The Journal of the Human Factors and Ergonomics Society 53, 3 (June 2011), 308–319. https://doi.org/10.1177/0018720811405196Google ScholarCross Ref
- Justin Munafo, Meg Diedrick, and Thomas A. Stoffregen. 2017. The virtual reality head-mounted display Oculus Rift induces motion sickness and is sexist in its effects. Experimental brain research 235, 3 (2017), 889–901.Google Scholar
- Mahdi Nabiyouni and Doug A. Bowman. 2016. A Taxonomy for Designing Walking-based Locomotion Techniques for Virtual Reality. In Proceedings of the 2016 ACM Companion on Interactive Surfaces and Spaces(ISS Companion ’16). ACM, New York, NY, USA, 115–121. https://doi.org/10.1145/3009939.3010076Google ScholarDigital Library
- Anh Nguyen and Zhisheng Yan. 2019. A Saliency Dataset for 360-degree Videos. In Proceedings of the 10th ACM Multimedia Systems Conference(MMSys ’19). ACM, New York, NY, USA, 279–284. https://doi.org/10.1145/3304109.3325820Google ScholarDigital Library
- Cuong Nguyen, Stephen DiVerdi, Aaron Hertzmann, and Feng Liu. 2017. Vremiere: In-Headset Virtual Reality Video Editing. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems(CHI ’17). ACM, New York, NY, USA, 5428–5438. https://doi.org/10.1145/3025453.3025675Google ScholarDigital Library
- George D. Park, R. Wade Allen, Dary Fiorentino, Theodore J. Rosenthal, and Marcia L. Cook. 2006. Simulator sickness scores according to symptom susceptibility, age, and gender for an older driver assessment study. In Proceedings of the human factors and ergonomics society annual meeting, Vol. 50. SAGE Publications Sage CA: Los Angeles, CA, 2702–2706.Google ScholarCross Ref
- Edzer Pebesma. 2018. Simple Features for R: Standardized Support for Spatial Vector Data. The R Journal 10, 1 (2018), 439–446. https://doi.org/10.32614/RJ-2018-009Google ScholarCross Ref
- J. D. Prothero, M. H. Draper, T. A. Furness, D. E. Parker, and M. J. Wells. 1999. The use of an independent visual background to reduce simulator side-effects. Aviation, Space, and Environmental Medicine 70, 3 Pt 1 (March 1999), 277–283.Google Scholar
- R Core Team. 2014. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org/Google Scholar
- Sharif Razzaque, Zachariah Kohn, and Mary C Whitton. 2001. Redirected Walking. Technical Report. University of North Carolina at Chapel Hill, Chapel Hill, NC, USA.Google ScholarDigital Library
- Lisa Rebenitsch and Charles Owen. 2014. Individual Variation in Susceptibility to Cybersickness. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology(UIST ’14). ACM, New York, NY, USA, 309–317. https://doi.org/10.1145/2642918.2647394Google ScholarDigital Library
- Lisa Rebenitsch and Charles Owen. 2016. Review on cybersickness in applications and visual displays. Virtual Reality 20, 2 (June 2016), 101–125. https://doi.org/10.1007/s10055-016-0285-9Google ScholarDigital Library
- Gary E. Riccio and Thomas A. Stoffregen. 1991. An ecological Theory of Motion Sickness and Postural Instability. Ecological Psychology 3, 3 (Sept. 1991), 195–240. https://doi.org/10.1207/s15326969eco0303_2Google ScholarCross Ref
- Sylvia Rothe and Heinrich Hußmann. 2018. Spatial statistics for analyzing data in cinematic virtual reality. In Proceedings of the 2018 International Conference on Advanced Visual Interfaces - AVI ’18. ACM Press, Castiglione della Pescaia, Grosseto, Italy, 1–3. https://doi.org/10.1145/3206505.3206561Google ScholarDigital Library
- Santeri Saarinen, Ville Mäkelä, Pekka Kallioniemi, Jaakko Hakulinen, and Markku Turunen. 2017. Guidelines for Designing Interactive Omnidirectional Video Applications. In 16th IFIP TC 13 International Conference on Human-Computer Interaction — INTERACT 2017 - Volume 10516. Springer-Verlag, Berlin, Heidelberg, 263–272. https://doi.org/10.1007/978-3-319-68059-0_17Google ScholarDigital Library
- Timothy Scaffidi. 2018. ofxOpticalFlowFarneback: openFrameworks addon for gunnar-farneback dense optical flow method. https://github.com/timscaffidi/ofxOpticalFlowFarneback original-date: 2012-11-13T08:43:28Z.Google Scholar
- Thomas Schubert, Frank Friedmann, and Holger Regenbrecht. 2001. The Experience of Presence: Factor Analytic Insights. Presence: Teleoper. Virtual Environ. 10, 3 (June 2001), 266–281. https://doi.org/10.1162/105474601300343603Google ScholarDigital Library
- Mel Slater and Maria V. Sanchez-Vives. 2016. Enhancing Our Lives with Immersive Virtual Reality. Frontiers in Robotics and AI 3 (Dec. 2016). https://doi.org/10.3389/frobt.2016.00074Google Scholar
- Kay M. Stanney, D. Susan Lanham, Robert S. Kennedy, and Robert Breaux. 1999. Virtual Environment Exposure Drop-Out Thresholds. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 43, 22 (1999), 1223–1227. https://doi.org/10.1177/154193129904302212Google ScholarCross Ref
- Sam Tregillus. 2016. VR-Drop: Exploring the Use of Walking-in-Place to Create Immersive VR Games. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems(CHI EA ’16). ACM, New York, NY, USA, 176–179. https://doi.org/10.1145/2851581.2890374Google ScholarDigital Library
- M. Treisman. 1977. Motion sickness: an evolutionary hypothesis. Science (New York, N.Y.) 197, 4302 (July 1977), 493–495. https://doi.org/10.1126/science.301659Google Scholar
- Michael Venturino and Maxwell J. Wells. 1990. Head Movements as a Function of Field-of-View Size on a Helmet-Mounted Display. Proceedings of the Human Factors Society Annual Meeting 34, 19 (Oct. 1990), 1572–1576. https://doi.org/10.1177/154193129003401932Google ScholarCross Ref
- David Matthew Whittinghill, Bradley Ziegler, T Case, and B Moore. 2015. Nasum virtualis: A simple technique for reducing simulator sickness. In Games Developers Conference (GDC). 74.Google Scholar
- Hadley Wickham. 2009. ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York. http://ggplot2.orgGoogle ScholarDigital Library
- Bob G. Witmer and Michael J. Singer. 1998. Measuring Presence in Virtual Environments: A Presence Questionnaire. Presence: Teleoperators and Virtual Environments 7, 3 (June 1998), 225–240. https://doi.org/10.1162/105474698565686Google ScholarDigital Library
- Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only Anova Procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’11). ACM, New York, NY, USA, 143–146. https://doi.org/10.1145/1978942.1978963Google ScholarDigital Library
- Robert Xiao and Hrvoje Benko. 2016. Augmenting the Field-of-View of Head-Mounted Displays with Sparse Peripheral Displays. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems(CHI ’16). ACM, New York, NY, USA, 1221–1232. https://doi.org/10.1145/2858036.2858212Google ScholarDigital Library
Recommendations
Mixing in Reverse Optical Flow to Mitigate Vection and Simulation Sickness in Virtual Reality
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing SystemsSimulator sickness has been one of the major obstacles toward making virtual reality (VR) widely accepted and used. For example, virtual navigation produces vection, which is the illusion of self-motion as one perceives bodily motion despite no movement ...
Dynamic Field of View Restriction in 360° Video: Aligning Optical Flow and Visual SLAM to Mitigate VIMS
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing SystemsHead-Mounted Display based Virtual Reality is proliferating. However, Visually Induced Motion Sickness (VIMS), which prevents many from using VR without discomfort, bars widespread adoption. Prior work has shown that limiting the Field of View (FoV) ...
Adaptive Field-of-view Restriction: Limiting Optical Flow to Mitigate Cybersickness in Virtual Reality
VRST '22: Proceedings of the 28th ACM Symposium on Virtual Reality Software and TechnologyDynamic field-of-view (FOV) restriction is a widely used software technique to mitigate cybersickness in commercial virtual reality (VR) applications. The classical FOV restrictor is implemented using a symmetric mask that occludes the periphery in ...
Comments