The computer representation of a 3D digital (or voxel-based) scene is heavily influenced by the processing for which it is used. In medical imaging applications, the commonly used types of processing include data format transformation, segmentation of the regions of interest, display, and manipulation, and quantitative measurement of the objects contained in the scene. In this paper, with all these types of processing taken into account, we propose a new data structure called “quadtree-segment encoding” to represent a 3D digital binary scene. We also propose some associated algorithms for those types of processing, including: 1. An algorithm to generate quickly a distance-only shaded image of the 3D scene; 2. A new image space shading method, called “interence-driven shading”, which is computationally inexpensive and provides good image quality; 3. A simple image space anti-aliasing method; 4. An algorithm to make interactively some quantitative measurements, manipulate interactively the organs, and redisplay the organs after manipulation. 5. An algorithm which is simple conceptually and cheap computationally to compute ray-object intersections for displaying transparent objects and slicing the objects. If we take all the factors of data compactness, speed, image quality, and efficiency of display and manipulation into account, the object representation and associated algorithms seem superior to others proposed in the literature, particularly for medical applications.