Walkthroughs of complex environments using image-based simplification

Abstract

We present an image-based technique to accelerate the navigation in complex static environments. We perform an image-space simplification of each sample of the scene taken at a particular viewpoint and dynamically combine these simplified samples to produce images for arbitrary viewpoints. Since the scene is converted into a bounded complexity representation in the image space, with the base images rendered beforehand, the rendering speed is relatively insensitive to the complexity of the scene. The proposed method correctly simulates the kinetic depth effect (parallax), occlusion, and can resolve the missing visibility information. This paper describes a suitable representation for the samples, a specific technique for simplifying them, and different morphing methods for combining the sample information to reconstruct the scene. We use hardware texture mapping to implement the image-space warping and hardware affine transformations to compute the view-dependent warping function.

Introduction

In contrast to a conventional geometry-based rendering pipeline image-based renderers use pre-rendered images of the scene as the basic primitives to render a scene—in place or in conjunction with the usual 3D models. This makes it possible to achieve much higher levels of realism, not only when modeling with real world photographs, but also when using synthetically generated imagery, since much more complex rendering algorithms can be used in the pre-navigation step. Image-based rendering also decouples the navigation frame rate from the complexity of the models, since only the fixed resolution images of the models are used, making this approach potentially faster than traditional ones as the model complexity continues to increase.

The trade-off of geometry complexity for images is not new. Texture mapping was introduced in 1974 by Catmull[8], and has been used extensively since then. Blinn and Newell[7]used pre-rendered images to map the surrounding environment on reflective surfaces.

Image-based rendering has been used with two different but closely related aims: to enable the navigation of environments modeled by real-world photographs and to accelerate the navigation of synthetic environments. In the first case the scene is not modeled on the computer and parameters such as depth information are not easily available. In the second case, all the information necessary for an image-based renderer can be retrieved from the original model.

Traditional approaches to graphics acceleration for the navigation of a three-dimensional environment have involved:

— reducing the rendering complexity by using texture mapping6, 7, and by using various levels of complexity in shading and illumination models[4].

— reducing the geometric complexity of the scene, by using level-of-detail hierarchies11, 17, 23, 24, 36, 37, 46, and by visibility-based culling2, 20, 22, 29, 44

— exploiting frame-to-frame coherence with one of the above5, 49.

However, as the complexity of the three-dimensional object–space has increased beyond the bounded image–space resolution, image-based rendering has begun to emerge as a viable alternative to the conventional three-dimensional geometric modeling and rendering, in specific application domains. Image-based rendering has been used to navigate (although with limited freedom of movement) in environments modeled from real-world digitized images9, 33, 43. More recent approaches19, 26generalize the idea of plenoptic modeling by characterizing the complete flow of light in a given region of space. This can be done for densely sampled arrays of images (digitized or synthetic) without relying on depth information, resulting in a large amount of information that limits its applicability to current environments without further research. Promising results towards making these approaches feasible in real time appear in[42].

Combining simple geometric building blocks with view-dependent textures derived from image-based rendering3, 16, 30has resulted in viable techniques for navigation in environments that can be described by those simple blocks. Also, Harry et al.[25]derived simple 3D scene models from photographs that allowed navigation in a single image. They use a spidery mesh graphical user interface, enabling the user to easily specify a vanishing point and background and foreground objects in an existing picture. An interesting use of image-based rendering for distributed virtual environments has been presented in[31]. In this approach only a subset of the scene information that can not be extrapolated from the previous frame is transmitted in a compressed form from the server to the client, thereby dramatically reducing the required network band-width. The potential of image-based rendering specifically for the navigation in generic synthetic environments on single graphics machines however has been investigated in fewer instances10, 32, 40.

We have presented a conceptual discussion and an implemented system for the problem of image-based rendering using image–space simplification and morphing[14]. In this paper, we discuss the technique in more detail, present the image–space simplification algorithm used, and compare the results of different blending techniques. Given a collection of z-buffered images representing an environment from fixed viewpoints and view directions, our approach first constructs an image–space simplification of the scene as a pre-process, and then reconstructs a view of this scene for arbitrary viewpoints and directions in real-time. We achieve speed through the use of the commonly available texture mapping hardware, and partially rectify the visibility gaps (“tears”) pointed out in previous work on image-based rendering9, 10through morphing.

In Section 2, we present an overview of the image-based rendering area; in Section 3, we discuss the relation between the morphing problem and image-based rendering. Section 4describes the image–space simplification technique in detail, and 5 Single node navigation, 6 Multiple node navigationdiscuss the navigation problem, comparing various forms of node combination. Some results are presented in Section 7.