Virtual Liver Surgical Simulator by Using Z-Buffer for Object Deformation

Abstract

Virtual surgical simulator which is using computer graphics is much popular system than before. It is generally used in the medical areas, such as medical hospital or medical university. The simulator uses virtual organ models like liver, brain and so on. These models are usually based on the scanning data from patients and are used as volume models. Fortunately, the volume model is familiar with its cutting or deforming operation in a surgical system. For this reason, there are many kinds of surgical simulation or navigation systems using the volume model. However, visual reality of the volume model is not sufficient for human being including doctors. This means that the doctors cannot identify shape or location of a target organ from volume objects. In order to overcome this, we should use the translating method, such as marching cubes method and so on, for getting precisely polygon models which is included normal vectors of volume object. However, the method is quite time consuming and consequently the doctors cannot operate the virtual model in real-time.

On these observations, we propose the virtual surgical simulator for operating the human liver in a virtual environment, which is based on the cooperation of polygon models and Z-buffer in GPU. By using parallel processing of GPU, the simulator allows uses to cut or deform a virtual liver model by using several kinds of medical tools like a scalpel in this system. In addition, visual reality of polygon model is wonderful for a doctor to identify its shape or location because this model maintains their precise normal vector.

1 Introduction

Surgical simulation is becoming popular theme in the medical operation. In these simulators, the system use organ models and allow user to cutting or deforming it for training of surgical method [1]. And in the field of computer graphics, hardware of graphics processing Unit (GPU) is becoming high efficiency and can carry out the parallel processing of 3D virtual space.

In this paper, we describe about our surgical simulator which is aimed at liver surgery. Our simulator uses virtual liver model which is included blood vessels. And it allows user to cutting or deforming these models by haptic devices. It is enable to display the any part of vessel such as that closed to a scalpel.

Virtual organ model such as liver and so on, is generated from patient scan data by MRI/CT. the model is typically generated as volume model. Volume model is included whole structured of the organ. It means that it is included inside blood vessels. Therefore, the volume model is used in such as surgical simulator. Some of the simulators use organ model generated from patient in real time by using MRI [2]. But these system is almost used in case of neurosurgery. Because the brain is surrounded by skull bones and it is almost not moving or deforming its shapes. Therefore, it is adequate to process by volume data. On the other hand, the liver is not same situation as the brain. It is in human body. And the liver surgery navigator or simulator is needed to measure some parameter for recognizing its posture which varies in real-time. Therefore it is adequate to use the virtual liver model as polygon model in the simulation system. For this reason, some system use Marching cubes algorithm [3] for translating into polygon models, which are surface models included position and normal vector data, from volume models in real-time. But it is difficult to show the detailed polygon model to users in real-time because of [4–6].

Therefore, our system uses only the virtual liver models constructed by polygon data for operating surgical tasks in real-time. Our system render the liver models by using Z-buffer included in GPU. By using parallel processing of GPU, the simulator allows uses to cut or deform a virtual liver model by using several kinds of medial tools like a scalpel in this system.

2 Deformation Algorithm

2.1 Overview

In our system, it is used a deformation algorithm for rendering a liver model to operating surgical tasks, cutting, deforming and so on. The liver model is created from volume image data (Fig. 1(a)). Figure 1(b) shows the liver model that our system have used. This polygon model has the body and blood vessels, the hepatic artery and vein. And our system use a scalpel model to operate surgical tasks (Fig. 1(c)). The deformation algorithm is based on that our previous study which is used for our dental surgical simulator [7]. In order to reduce the computation time to rendering it, this method uses a coordinate system organized the depth direction as an operation direction. And our system use Z-buffer that is stored depth map in the coordinate system. Figure 2 shows the image of our algorithm overview.

Fig. 1.

An image of virtual liver models.

Fig. 2.

The abstract of our algorithm.

2.2 Procedure of Deformation

Z-buffer is used to rendering only the model showed from a viewpoint in 3D graphic scene. To use the Z-buffer, the system is enable to process many polygons simultaneously and reducing the rendering time. In our algorithm, it is used to measure the deforming volume by user’s operation. At first, the region of manipulation is defined from view volume by a camera for operation. And the coordinate system is defined in this region and stored the depth map. Figure 3 shows summary of this process.

Fig. 3.

Our deformation procedure.

In order to generate the depth map after the deformation by operating with a virtual tool, it is stored amount of changing the depth map when the virtual scalpel slink to the liver model. This is used to change the shape of liver model and calculate parameters to generate tactile feedback. This depth map is a simple two-dimensional array.

The deformation of liver model is generated by the following process. In order to be assigned vertex of the liver model to Z-buffer, it uses a conversion method to change the vertex with world coordinate system, which constructs Z-buffer. And it acquires z value from the pixel, which contains the vertex in Z-buffer. The deformation volume is defined from this volume and it is reconverted to the vertex position in world coordinate system. All of this process is executed in the high-speed transaction by using the parallel processing in GPU.

Finally, to adapt this deformation process, it changes the vertex point of the liver model to deform it. In this process, some surfaces of the model is extended to a direction of deformed region and it is not enable to deform to other direction. To solve this problem, our system uses a subdivision surface algorithm to divide the extended surfaces. A summary of this method is that generates middle point to the edge between 2 vertexes with length more than a certain threshold.

3 Approach Notification Algorithm

Our simulator has been required to have a function of alert notification. It is that the system has some notification event, when a virtual scalpel model is come closed to blood vessels. In order to realize this function, it is using the Z-buffer to measure the distance between the scalpel and blood vessels. While the scalpel model is moved in world coordinate space, vertex point of the scalpel edge and blood vessels is converted to the depth value in the region of manipulation. And it is calculated the distance between the scalpel edge and the vertex of blood vessels in the depth map. This calculation is executed by using the parallel processing in GPU. Therefore it uses mutual exclusion by using the unit for distance.

4 Implementation

We implemented this simulator system on Windows PC. Figure 4 shows our prototype system. The user operate scalpel model by using a haptic feedback device. It is enable to display force feedback to the user, while they operate scalpel model. Figure 5 shows an example of user’s view image. Our system shows 4 different views And the user enable to select these views. The user enable to show the liver model and blood vessels included in the liver from any angle. Figure 6 shows an example scene of user’s cutting operation. We confirmed that the user enable to operate the surgical tasks in our system.

Fig. 4.

Our prototype simulation system.

Fig. 5.

An example of user’s view image.

Fig. 6.

An example of cutting operation.

The limitation of our system is that the user enable to treat the liver model through the scalpel model. In case of real surgery, the doctor have often changed the shape of the liver by their hands. But our system doesn’t support it. To realize it operation, the improvement of our deformation algorithm is needed. We consider it as our future work.

5 Conclusion

We proposed a virtual liver surgical simulator system by using Z-buffer for object deformation. Our system uses the polygon liver model and uses the deformation algorithm based on parallel processing in GPU. We constructed a prototype system and confirmed the capability and the limitation of this system.

In future work, we will consider the improvement of our deformation algorithm which is able to show the liver model more precisely, and consider the evaluation of our system.