Bone tumor segmentation from MR perfusion images with neural networks using multi-scale pharmacokinetic features

https://doi.org/10.1016/S0262-8856(01)00044-0Get rights and content

Abstract

Bone tumor segmentation and the distinction between viable and non-viable tumor tissue is required during the follow-up of chemotherapeutical treatment. Monitoring viable tumor area over time is important in the ongoing assessment of the effect of preoperative chemotherapy. In this paper, features derived from a pharmacokinetic model of tissue perfusion are investigated. A multi-scale analysis of the parametric perfusion images is applied to incorporate contextual information. A feed-forward neural network is proposed to classify pixels into viable, non-viable tumor, and healthy tissue. We elaborate on the design of a cascaded classifier and analyze the contribution of the different features to its performance. Multi-scale blurred versions of the parametric images together with a multi-scale formulation of the local image entropy turned out to be the most relevant features in distinguishing the tissues of interest. We experimented with an architecture consisting of cascaded neural networks to cope with uneven class distributions. The classification of each pixel was obtained by weighting the results of five bagged neural networks with either the mean or median rules. The experiments indicate that both the mean and median rules perform equally well.

Introduction

Segmentation of bone tumor entails the distinction between viable and non-viable tumor tissue and is required for the follow-up of chemotherapeutical treatment. Monitoring the volume change of viable tumor can help in assessing the success of chemotherapy, or it may lead to abort its application. It is well known that most malignant tumors are highly vascularized tissues. Distinction between viable and non-viable tumor in post-chemotherapy studies can only be performed by dynamic Magnetic Resonance (MR) perfusion imaging using an intravenous contrast tracer [1], [2]. The dynamic MR-signal associated with each physical pixel characterizes perfusion properties of the tissue under study.

In this paper, we present a feature-based neural network for segmentation of dynamic MR-images. A pharmacokinetic model of the tissue perfusion is used to obtain a compact representation of the perfusion properties of each pixel. This allows us to reduce the MR image sequence into three parametric images. Furthermore, concepts of linear scale-space [3], [4] are used to analyze the parametric images at several scales in order to incorporate contextual (spatial) information into the segmentation approach. Finally, a classifier is used to combine both temporal and spatial information.

Haring et al. [5] have previously presented a method for image segmentation based on Kohonen networks using multi-scale features. This method, however, performed the multi-scale analysis on the original images since it was devised for static examinations. Our method performs a similar multi-scale analysis but takes advantage of a physically-motivated model of tissue perfusion. Another distinction is that our classifier is based on a supervised feed-forward neural network. Class labels used in the training and testing sets were inferred from post-operative histological studies. Although invasive, this technique is regarded as the gold standard given its high spatial resolution (a histologic slice is about 5 μm thin while a two-dimensional slice in a MR-perfusion acquisition can be 8 mm thick).

Given the uneven distribution of the different tissue types in our application, a two-stage cascaded classifier architecture was designed. A hierarchical classification between healthy and tumor tissue and, within the latter, between viable and non-viable tumor, is thereby obtained. This architecture is inspired by the decision-making process followed by a radiologist when confronted with this type of images.

The paper is structured as follows. In Section 2, multi-scale image features are introduced which are based on a pharmacokinetic model of the perfusion process. Section 3 describes the type of neural network architecture that we have adopted and the quality measures that we have used in designing the classifier. Section 4 describes our experiments in order to design an optimal classifier with our features. Finally, the paper is concluded in Section 5 with a discussion and issues for future research.

Section snippets

Methods

Our starting point is an MR image sequence, s(x,y;t), that characterizes the perfusion properties of each pixel, i.e. the up-take and secretion of a blood tracer over time. In the next two sections, we indicate how this image sequence can be reduced to three parametric images summarizing the perfusion properties of the tissue, and how to build a multi-scale feature vector based on the parametric images so as to incorporate spatial information.

Choice of classifier

In principle, a large variety of statistical classifiers are available which could be trained to segment the dynamic MR-images. Statistical classifiers such as Bayes rule, discriminant analysis, k-nearest neighbor and feed-forward neural networks were compared theoretically along several criteria in Ref. [11]. It was argued that if a classifier is required that has a small error rate and is fast in application, a feed-forward neural network is a good choice. It has namely been proven that

Image acquisition

All MR-examinations were performed on a 0.5 T NT Gyroscan (Philips Medical Systems, Best, The Netherlands) using a surface coil. One, two or three sections were selected for T1-weighted dynamic contrast-enhanced imaging using a magnetization prepared imaging gradient recalled echo technique. The MR-images were acquired with a repetition time of 12 ms, an echo time of 5.7 ms and a prepulse delay time of 741 ms. The chosen flip angle was 30°. The field of view varied per patient depending on the size

Discussion and conclusions

In this paper, a multi-scale feature-based neural network classifier for automatic segmentation of MR perfusion images of bone tumor is presented. The method classifies each pixel as either viable tumor, non-viable tumor or “rest” (healthy+background) tissue.

Features are computed in two steps. First, a pharmacokinetic model is used to summarize the temporal information in the perfusion sequence into three main parameters (a, m1 and m2). Subsequently, these parametric images are used to derive

Acknowledgements

We are grateful to Prof Dr J.L. Bloem (Department of Radiology) and Prof Dr P.C.W. Hogendoorn (Department of Pathology), both from the Leiden University Medical Center, for providing us with the patient material. This project was financially supported by a the Dutch Cancer Foundation (KWF), Grant RUL 97-1509 and by the Dutch Ministry of Economic Affairs, IOP Beeldverwerking Grant IBV-97009.

References (29)

  • M.F. Moller

    A scaled conjugate gradient algorithm for fast supervised learning

    Neural Netw.

    (1993)
  • H. van der Woude et al.

    Monitoring the effect of chemotherapy in ewing's sarcoma of bone with MR imaging

    Skeletal. Radiol.

    (1994)
  • H. van der Woude et al.

    Osteosarcoma and ewing's sarcoma after neoadjuvant chemotherapy: value of dynamic MR imaging in detecting viable tumor before surgery

    Am. J. Roentgenol.

    (1995)
  • L.M.J. Florack et al.

    Linear scale-space

    J. Math. Image Vis.

    (1994)
  • Cited by (17)

    • An advanced W-shaped network with adaptive multi-scale supervision for osteosarcoma segmentation

      2023, Biomedical Signal Processing and Control
      Citation Excerpt :

      Other challenges include the heterogeneity of pixel textures within the tumor itself and the close similarity between the tumorous tissues and normal surrounding tissues. In the past, segmentation of osteosarcoma in medical images have been done by cluster-based methods [6–8] or learning-based methods [9–13]. The cluster-based methods iteratively partition the medical images into tumor tissue region and non-tumor tissue region by appending pixels to different seeds according specific criteria for pixel features.

    • Multiple supervised residual network for osteosarcoma segmentation in CT images

      2018, Computerized Medical Imaging and Graphics
      Citation Excerpt :

      Chen et al. (2013) proposed an approach based on Zernike moment and support vector machine (SVM) for the segmentation of osteosarcoma in MRI images. Frangi et al. (2001) applied neural network method to segment osteosarcoma in MRI perfusion images. They first built a pharmacokinetic model for the MRI perfusion images and then extracted pharmacokinetic features of MRI perfusion images from the model.

    • MSFCN-multiple supervised fully convolutional networks for the osteosarcoma segmentation of CT images

      2017, Computer Methods and Programs in Biomedicine
      Citation Excerpt :

      In addition, due to the lack of object prior, they were only able to process the images with simple structures and orderly textures. (2) Traditional learning-based methods [7–9]. These methods considered the segmentation tasks as per-pixel classification tasks.

    • A new parametric model-based technique in bone tumour analysis

      2014, Computerized Medical Imaging and Graphics
      Citation Excerpt :

      An exemplary bone tumour, manually delineated in MR images is shown in Fig. 1.1 A limited number of studies directed to the bone tumour detection and segmentation is reported in the literature [26,30,21,40,13,17,10]. The analysis mostly target on one type of tumour or particular anatomical structure [26,30,17].

    View all citing articles on Scopus
    View full text