Bone tumor segmentation from MR perfusion images with neural networks using multi-scale pharmacokinetic features
Introduction
Segmentation of bone tumor entails the distinction between viable and non-viable tumor tissue and is required for the follow-up of chemotherapeutical treatment. Monitoring the volume change of viable tumor can help in assessing the success of chemotherapy, or it may lead to abort its application. It is well known that most malignant tumors are highly vascularized tissues. Distinction between viable and non-viable tumor in post-chemotherapy studies can only be performed by dynamic Magnetic Resonance (MR) perfusion imaging using an intravenous contrast tracer [1], [2]. The dynamic MR-signal associated with each physical pixel characterizes perfusion properties of the tissue under study.
In this paper, we present a feature-based neural network for segmentation of dynamic MR-images. A pharmacokinetic model of the tissue perfusion is used to obtain a compact representation of the perfusion properties of each pixel. This allows us to reduce the MR image sequence into three parametric images. Furthermore, concepts of linear scale-space [3], [4] are used to analyze the parametric images at several scales in order to incorporate contextual (spatial) information into the segmentation approach. Finally, a classifier is used to combine both temporal and spatial information.
Haring et al. [5] have previously presented a method for image segmentation based on Kohonen networks using multi-scale features. This method, however, performed the multi-scale analysis on the original images since it was devised for static examinations. Our method performs a similar multi-scale analysis but takes advantage of a physically-motivated model of tissue perfusion. Another distinction is that our classifier is based on a supervised feed-forward neural network. Class labels used in the training and testing sets were inferred from post-operative histological studies. Although invasive, this technique is regarded as the gold standard given its high spatial resolution (a histologic slice is about 5 μm thin while a two-dimensional slice in a MR-perfusion acquisition can be 8 mm thick).
Given the uneven distribution of the different tissue types in our application, a two-stage cascaded classifier architecture was designed. A hierarchical classification between healthy and tumor tissue and, within the latter, between viable and non-viable tumor, is thereby obtained. This architecture is inspired by the decision-making process followed by a radiologist when confronted with this type of images.
The paper is structured as follows. In Section 2, multi-scale image features are introduced which are based on a pharmacokinetic model of the perfusion process. Section 3 describes the type of neural network architecture that we have adopted and the quality measures that we have used in designing the classifier. Section 4 describes our experiments in order to design an optimal classifier with our features. Finally, the paper is concluded in Section 5 with a discussion and issues for future research.
Section snippets
Methods
Our starting point is an MR image sequence, s(x,y;t), that characterizes the perfusion properties of each pixel, i.e. the up-take and secretion of a blood tracer over time. In the next two sections, we indicate how this image sequence can be reduced to three parametric images summarizing the perfusion properties of the tissue, and how to build a multi-scale feature vector based on the parametric images so as to incorporate spatial information.
Choice of classifier
In principle, a large variety of statistical classifiers are available which could be trained to segment the dynamic MR-images. Statistical classifiers such as Bayes rule, discriminant analysis, k-nearest neighbor and feed-forward neural networks were compared theoretically along several criteria in Ref. [11]. It was argued that if a classifier is required that has a small error rate and is fast in application, a feed-forward neural network is a good choice. It has namely been proven that
Image acquisition
All MR-examinations were performed on a 0.5 T NT Gyroscan (Philips Medical Systems, Best, The Netherlands) using a surface coil. One, two or three sections were selected for T1-weighted dynamic contrast-enhanced imaging using a magnetization prepared imaging gradient recalled echo technique. The MR-images were acquired with a repetition time of 12 ms, an echo time of 5.7 ms and a prepulse delay time of 741 ms. The chosen flip angle was 30°. The field of view varied per patient depending on the size
Discussion and conclusions
In this paper, a multi-scale feature-based neural network classifier for automatic segmentation of MR perfusion images of bone tumor is presented. The method classifies each pixel as either viable tumor, non-viable tumor or “rest” (healthy+background) tissue.
Features are computed in two steps. First, a pharmacokinetic model is used to summarize the temporal information in the perfusion sequence into three main parameters (a, m1 and m2). Subsequently, these parametric images are used to derive
Acknowledgements
We are grateful to Prof Dr J.L. Bloem (Department of Radiology) and Prof Dr P.C.W. Hogendoorn (Department of Pathology), both from the Leiden University Medical Center, for providing us with the patient material. This project was financially supported by a the Dutch Cancer Foundation (KWF), Grant RUL 97-1509 and by the Dutch Ministry of Economic Affairs, IOP Beeldverwerking Grant IBV-97009.
References (29)
- et al.
Kohonen networks for multiscale image segmentation
Image Vis. Comput.
(1994) - et al.
Detection of areas with viable remnant tumor in postchemotherapy patients with Ewing's sarcoma by dynamic constrast-enhanced MRI using pharmacokinetic modeling
Magn. Reson. Imaging
(2000) - et al.
Scale and the differential structure of images
Image Vis. Comput.
(1992) On the approximate realization of continuous mappings by neural networks
Neural Netw.
(1989)- et al.
Multilayer feedforward networks are universal approximators
Neural Netw.
(1989) - et al.
Optimal combinations of pattern classifiers
Pattern Recogn. Lett.
(1995) - et al.
Soft combination of neural classifiers: a comparative study
Pattern Recogn. Lett.
(1999) - et al.
Sequential selection of discrete features for neural networks — a bayesian approach to building a cascade
Pattern Recogn. Lett.
(1999) - et al.
On the quality of neural net classifiers
Artif. Intell. Med.
(1994) - et al.
Assessing the importance of features for multi-layer perceptrons
Neural Netw.
(1998)
A scaled conjugate gradient algorithm for fast supervised learning
Neural Netw.
Monitoring the effect of chemotherapy in ewing's sarcoma of bone with MR imaging
Skeletal. Radiol.
Osteosarcoma and ewing's sarcoma after neoadjuvant chemotherapy: value of dynamic MR imaging in detecting viable tumor before surgery
Am. J. Roentgenol.
Linear scale-space
J. Math. Image Vis.
Cited by (17)
An advanced W-shaped network with adaptive multi-scale supervision for osteosarcoma segmentation
2023, Biomedical Signal Processing and ControlCitation Excerpt :Other challenges include the heterogeneity of pixel textures within the tumor itself and the close similarity between the tumorous tissues and normal surrounding tissues. In the past, segmentation of osteosarcoma in medical images have been done by cluster-based methods [6–8] or learning-based methods [9–13]. The cluster-based methods iteratively partition the medical images into tumor tissue region and non-tumor tissue region by appending pixels to different seeds according specific criteria for pixel features.
Multiple supervised residual network for osteosarcoma segmentation in CT images
2018, Computerized Medical Imaging and GraphicsCitation Excerpt :Chen et al. (2013) proposed an approach based on Zernike moment and support vector machine (SVM) for the segmentation of osteosarcoma in MRI images. Frangi et al. (2001) applied neural network method to segment osteosarcoma in MRI perfusion images. They first built a pharmacokinetic model for the MRI perfusion images and then extracted pharmacokinetic features of MRI perfusion images from the model.
MSFCN-multiple supervised fully convolutional networks for the osteosarcoma segmentation of CT images
2017, Computer Methods and Programs in BiomedicineCitation Excerpt :In addition, due to the lack of object prior, they were only able to process the images with simple structures and orderly textures. (2) Traditional learning-based methods [7–9]. These methods considered the segmentation tasks as per-pixel classification tasks.
A new parametric model-based technique in bone tumour analysis
2014, Computerized Medical Imaging and GraphicsCitation Excerpt :An exemplary bone tumour, manually delineated in MR images is shown in Fig. 1.1 A limited number of studies directed to the bone tumour detection and segmentation is reported in the literature [26,30,21,40,13,17,10]. The analysis mostly target on one type of tumour or particular anatomical structure [26,30,17].
Auxiliary Segmentation Method of Osteosarcoma in MRI Images Based on Denoising and Local Enhancement
2022, Healthcare (Switzerland)