Martin Misiak
Arnulph Fuhrmann
Marc Erich Latoschik
ABSTRACT
This paper presents an image-based rendering approach to accelerate rendering time of virtual scenes containing a large number of complex high poly count objects. Our approach replaces complex objects by impostors, light-weight image-based representations leveraging geometry and shading related processing costs. In contrast to their classical implementation, our impostors are specifically designed to work in Virtual-, Augmented- and Mixed Reality scenarios (XR for short), as they support stereoscopic rendering to provide correct depth perception. Motion parallax of typical head movements is compensated by using a ray marched parallax correction step. Our approach provides a dynamic run-time recreation of impostors as necessary for larger changes in view position. The dynamic run-time recreation is decoupled from the actual rendering process. Hence, its associated processing cost is therefore distributed over multiple frames. This avoids any unwanted frame drops or latency spikes even for impostors of objects with complex geometry and many polygons. In addition to the significant performance benefit, our impostors compare favorably against the original mesh representation, as geometric and textural temporal aliasing artifacts are heavily suppressed.
Supplemental Material
Available for Download
- 2018. Unity - Octahedral Impostors. https://github.com/xraxra/IMP. Accessed: 2021-07-05.Google Scholar
- 2020. Godot - Octahedral Impostors. https://github.com/wojtekpil/Godot-Octahedral-Impostors. Accessed: 2021-07-05.Google Scholar
- Carlos Andújar, Javier Boo, Pere Brunet, Marta Fairén, Isabel Navazo, Pere Vazquez, and A Vinacua. 2007. Omni-directional Relief Impostors. Computer Graphics Forum 26, 3, 553–560.Google ScholarCross Ref
- Lionel Baboud and Xavier Décoret. 2006. Rendering geometry with relief textures. In Proceedings of Graphics Interface 2006. Canadian Information Processing Society, 195–201.Google ScholarDigital Library
- Kevin Boos, David Chu, and Eduardo Cuervo. 2016. Flashback: Immersive virtual reality on mobile devices via rendering memoization. In Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services. 291–304.Google ScholarDigital Library
- Ryan Brucks. 2018. Octahedral Impostors. https://www.shaderbits.com/blog/octahedral-impostors. Accessed: 2021-07-05.Google Scholar
- Jonathan Cohen, Marc Olano, and Dinesh Manocha. 1998. Appearance-preserving simplification. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques. 115–122.Google ScholarDigital Library
- Xavier Décoret, Frédo Durand, François X Sillion, and Julie Dorsey. 2003. Billboard clouds for extreme model simplification. In ACM SIGGRAPH 2003 Papers. 689–696.Google ScholarDigital Library
- Jon Hasselgren, Jacob Munkberg, Marco Salvi, Anjul Patney, and Aaron Lefohn. 2020. Neural temporal adaptive sampling and denoising. In Computer Graphics Forum, Vol. 39. Wiley Online Library, 147–155.Google Scholar
- Jozef Hladky, Hans-Peter Seidel, and Markus Steinberger. 2019. Tessellated Shading Streaming. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 171–182.Google Scholar
- Lloyd A Jeffress. 1937. Anomalous Parallax in Anaglyphs and Stereograms. The Journal of General Psychology 17, 2 (1937), 389–394.Google ScholarCross Ref
- Zeqi Lai, Y Charlie Hu, Yong Cui, Linhui Sun, Ningwei Dai, and Hung-Sheng Lee. 2019. Furion: Engineering high-quality immersive virtual reality on today’s mobile devices. IEEE Transactions on Mobile Computing(2019).Google Scholar
- Puneet Lall, Silviu Borac, Dave Richardson, Matt Pharr, and Manfred Ernst. 2018. View-Region Optimized Image-Based Scene Simplification. Proceedings of the ACM on Computer Graphics and Interactive Techniques 1, 2(2018), 1–22.Google ScholarDigital Library
- Kyungmin Lee, David Chu, Eduardo Cuervo, Johannes Kopf, Yury Degtyarev, Sergey Grizan, Alec Wolman, and Jason Flinn. 2015. Outatime: Using speculation to enable low-latency continuous interaction for mobile cloud gaming. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services. 151–165.Google ScholarDigital Library
- Yong Li and Wei Gao. 2019. DeltaVR: Achieving high-performance mobile VR dynamics through pixel reuse. In Proceedings of the 18th International Conference on Information Processing in Sensor Networks. 13–24.Google ScholarDigital Library
- Paulo WC Maciel and Peter Shirley. 1995. Visual navigation of large environments using textured clusters. In Proceedings of the 1995 symposium on Interactive 3D graphics. ACM, 95–ff.Google ScholarDigital Library
- Alexander Majercik, Cyril Crassin, Peter Shirley, and Morgan McGuire. 2018. A ray-box intersection algorithm and efficient dynamic voxel rendering. Journal of Computer Graphics Techniques Vol 7, 3 (2018).Google Scholar
- Stephan Mantler, Stefan Jeschke, and Michael Wimmer. 2007. Displacement mapped billboard clouds. In Proceedings of symposium on interactive 3D graphics and games. Citeseer.Google Scholar
- William R Mark, Leonard McMillan, and Gary Bishop. 1997. Post-rendering 3D warping. In Proceedings of the 1997 symposium on Interactive 3D graphics. 7–ff.Google ScholarDigital Library
- Joerg H Mueller, Philip Voglreiter, Mark Dokter, Thomas Neff, Mina Makar, Markus Steinberger, and Dieter Schmalstieg. 2018. Shading atlas streaming. ACM Transactions on Graphics (TOG) 37, 6 (2018), 1–16.Google ScholarDigital Library
- Manuel M Oliveira, Gary Bishop, and David McAllister. 2000. Relief texture mapping. In Proceedings of the 27th annual conference on Computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co., 359–368.Google ScholarDigital Library
- Edward M Peek, Burkhard C Wünsche, and Christof Lutteroth. 2014. Image warping for enhancing consumer applications of head-mounted displays. In Proceedings of the Fifteenth Australasian User Interface Conference-Volume 150. 47–55.Google ScholarDigital Library
- Fabio Policarpo and Manuel M Oliveira. 2006. Relief mapping of non-height-field surface details. In Proceedings of the 2006 symposium on Interactive 3D graphics and games. ACM, 55–62.Google ScholarDigital Library
- Fábio Policarpo, Manuel M Oliveira, and João LD Comba. 2005. Real-time relief mapping on arbitrary polygonal surfaces. In Proceedings of the 2005 symposium on Interactive 3D graphics and games. ACM, 155–162.Google ScholarDigital Library
- Voicu Popescu, Kyle Hayward, Paul Rosen, and Chris Wyman. 2009. Non-Pinhole Imposters. Technical Report 09-006. Purdue University.Google Scholar
- Bernhard Reinert, Johannes Kopf, Tobias Ritschel, Eduardo Cuervo, David Chu, and Hans-Peter Seidel. 2016. Proxy-guided image-based rendering for mobile devices. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 353–362.Google Scholar
- Eric Risser, Musawir Shah, and Sumanta Pattanaik. 2007. Faster relief mapping using the secant method. Journal of graphics tools 12, 3 (2007), 17–24.Google ScholarCross Ref
- Gernot Schaufler. 1996. Exploiting frame-to-frame coherence in a virtual reality system. In Proceedings of the IEEE 1996 Virtual Reality Annual International Symposium. IEEE, 95–102.Google ScholarCross Ref
- Gernot Schaufler. 1998. Per-object image warping with layered impostors. In Eurographics Workshop on Rendering Techniques. Springer, 145–156.Google ScholarCross Ref
- Gernot Schaufler and Wolfgang Stürzlinger. 1996. A three dimensional image cache for virtual reality. In Computer Graphics Forum, Vol. 15. Wiley Online Library, 227–235.Google Scholar
- Jonathan Shade, Steven Gortler, Li-wei He, and Richard Szeliski. 1998. Layered depth images. In Proceedings of the 25th annual conference on Computer graphics and interactive techniques. 231–242.Google ScholarDigital Library
- Jonathan Shade, Dani Lischinski, David H Salesin, Tony DeRose, and John Snyder. 1996. Hierarchical image caching for accelerated walkthroughs of complex environments. In Proceedings of the 23rd annual conference on Computer graphics and interactive techniques. ACM, 75–82.Google ScholarDigital Library
- Ferdi Smit, Robert van Liere, Stephan Beck, and Bernd Fröhlich. 2009. An image-warping architecture for vr: Low latency versus image quality. In 2009 IEEE Virtual Reality Conference. IEEE, 27–34.Google ScholarCross Ref
- Jan-Philipp Stauffert, Kristof Korwisi, Florian Niebling, and Marc Erich Latoschik. 2021. Ka-Boom!!! Visually Exploring Latency Measurements for XR. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems(CHI EA ’21). Association for Computing Machinery, New York, NY, USA.Google ScholarDigital Library
- Jan-Philipp Stauffert, Florian Niebling, and Marc Erich Latoschik. 2018. Effects of Latency Jitter on Simulator Sickness in a Search Task. In 25th IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR). 121–127.Google ScholarCross Ref
- Jan-Philipp Stauffert, Florian Niebling, and Marc Erich Latoschik. 2020. Latency and Cybersickness: Impact, Causes, and Measures. A Review. Frontiers in Virtual Reality 1 (2020), 31.Google Scholar
- László Szirmay-Kalos and Tamás Umenhoffer. 2008. Displacement Mapping on the GPU—State of the Art. In Computer graphics forum, Vol. 27. Wiley Online Library, 1567–1592.Google Scholar
- Natalya Tatarchuk. 2006. Dynamic parallax occlusion mapping with approximate soft shadows. In Proceedings of the 2006 symposium on Interactive 3D graphics and games. 63–69.Google ScholarDigital Library
- Franco Tecchia and Yiorgos Chrysanthou. 2000. Real-time rendering of densely populated urban environments. In Rendering Techniques 2000. Springer, 83–88.Google ScholarCross Ref
- Severin Todt, Christof Rezk-Salama, Andreas Kolb, and K-D Kuhnert. 2008. GPU-Based Spherical Light Field Rendering with Per-Fragment Depth Correction. In Computer Graphics Forum, Vol. 27. Wiley Online Library, 2081–2095.Google Scholar
- JMP Van Waveren. 2016. The asynchronous time warp for virtual reality on consumer hardware. In Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology. 37–46.Google ScholarDigital Library
- Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13, 4 (2004), 600–612.Google ScholarDigital Library
- Sven Widmer, D Pająk, André Schulz, Kari Pulli, Jan Kautz, Michael Goesele, and David Luebke. 2015. An adaptive acceleration structure for screen-space ray tracing. In Proceedings of the 7th Conference on High-Performance Graphics. 67–76.Google ScholarDigital Library
- Graham Wihlidal. 2016. Optimizing the graphics pipeline with compute. In Game Developers Conference. 54.Google Scholar
- Niko Wißmann, Martin Mišiak, Arnulph Fuhrmann, and Marc Erich Latoschik. 2020. Accelerated Stereo Rendering with Hybrid Reprojection-Based Rasterization and Adaptive Ray-Tracing. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 828–835.Google Scholar
- Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. 2018. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition. 586–595.Google ScholarCross Ref
Recommendations
Image-based rendering of diffuse, specular and glossy surfaces from a single image
SIGGRAPH '01: Proceedings of the 28th annual conference on Computer graphics and interactive techniquesIn this paper, we present a new method to recover an approximation of the bidirectional reflectance distribution function (BRDF) of the surfaces present in a real scene. This is done from a single photograph and a 3D geometric model of the scene. The ...
Sample-Based Cameras for Feed Forward Reflection Rendering
This paper presents sample-based cameras for rendering high quality reflections on convex reflectors at interactive rates. The method supports change of view, moving objects and reflectors, higher order reflections, view-dependent lighting of reflected ...
Relief mapping of non-height-field surface details
I3D '06: Proceedings of the 2006 symposium on Interactive 3D graphics and gamesThe ability to represent non-height-field mesostructure details is of great importance for rendering complex surface patterns, such as weave and multilayer structures. Currently, such representations are based on the use of 3D textures or large datasets ...
Comments