Published online Mar 20, 2024.
https://doi.org/10.3349/ymj.2023.0192
Prediction of High-Risk Neuroblastoma Among Neuroblastic Tumors Using Radiomics Features Derived from Magnetic Resonance Imaging: A Pilot Study
Abstract
Purpose
This study aimed to predict high-risk neuroblastoma among neuroblastic tumors using radiomics features extracted from MRI.
Materials and Methods
Pediatric patients (age≤18 years) diagnosed with neuroblastic tumors who had pre-treatment MR images available were enrolled from institution A from January 2010 to November 2019 (training set) and institution B from January 2016 to January 2022 (test set). Segmentation was performed with regions of interest manually drawn along tumor margins on the slice with the widest tumor area by two radiologists. First-order and texture features were extracted and intraclass correlation coefficients (ICCs) were calculated. Multivariate logistic regression (MLR) and random forest (RF) models from 10-fold cross-validation were built using these features. The trained MLR and RF models were tested in an external test set.
Results
Thirty-two patients (M:F=23:9, 26.0±26.7 months) were in the training set and 14 patients (M:F=10:4, 33.4±20.4 months) were in the test set with radiomics features (n=930) being extracted. For 10 of the most relevant features selected, intra- and inter-observer variability was moderate to excellent (ICCs 0.633–0.911, 0.695–0.985, respectively). The area under the receiver operating characteristic curve (AUC) was 0.94 (sensitivity 67%, specificity 91%, and accuracy 84%) for the MLR model and the average AUC was 0.83 (sensitivity 44%, specificity 87%, and accuracy 75%) for the RF model from 10-fold cross-validation. In the test set, AUCs of the MLR and RF models were 0.94 and 0.91, respectively.
Conclusion
An MRI-based radiomics model can help predict high-risk neuroblastoma among neuroblastic tumors.
Graphical Abstract
INTRODUCTION
Neuroblastic tumors encompass a variety of tumors, including neuroblastoma, ganglioneuroblastoma, and ganglioneuroma, ranging from mature or spontaneously regressing tumors to those that lead to patient death.1, 2 Thus, patients with neuroblastic tumors are stratified by risk from the very low- to high-risk group according to the International Neuroblastoma Risk Group (INRG) pre-treatment classification system based on a combination of pathological grade, patient age, disease stage, chromosome 11q status, ploidy, and MYCN gene amplification to predict prognosis and determine treatment plans.3 In particular, the 5-year survival rate of the high-risk neuroblastoma group is reported to be less than 50%.4
Radiomics refers to the method of extracting a very large amount of quantitative features from medical images through specific algorithms. These extracted features are called radiomics features, and their use is thought to enable radiologists to distinguish subtle differences that cannot be caught by the naked eye, and to quantitatively evaluate features that were previously only qualitatively evaluated visually.5, 6, 7 Many studies have already been conducted on CT or MRI-based radiomics targeting various organs or tumors; and in particular, meaningful tumor discrimination has been reported with radiomics models derived from T2-weighted images that have been developed as either two-dimensional or three-dimensional models.8, 9, 10
A few studies have reported on the use of CT-based radiomics to predict MYCN gene amplification in children with neuroblastic tumors, showing optimistic results.11, 12 Due to radiation concerns, MRI is sometimes preferred for initial tumor staging in pediatric patients. However, no studies have yet applied MR imaging for radiomics analysis in children with neuroblastic tumors. If high-risk neuroblastomas could be assessed quantitatively by MR-based radiomics, it would be helpful for planning diagnostic workup and treatment even before the invasive procedures. Unless the whole tumor is taken out for pathologic evaluation, needle biopsy for pathology has a limitation to obtain whole tissue of the tumor. Also, due to the heterogeneous nature of neuroblastoma, only needle biopsy results could underestimate the tumor aggressiveness. In these cases, quantitative radiomics evaluation could be helpful as additional diagnostic criteria for risk stratification. Therefore, the purpose of this study was to determine if high-risk neuroblastoma could be predicted among neuroblastic tumors using radiomics features extracted from MRI.
MATERIALS AND METHODS
Subjects
This retrospective study was approved by the Institutional Review Board at the authors’ institution (IRB number: 4-2020-1095), and the requirement for informed consent was waived. From two different institutions, pediatric patients (age ≤18 years) diagnosed with neuroblastic tumors who had pre-treatment T2-weighted MR images available were included in this study. For the training set, patients from institution A were enrolled from January 2010 to November 2019, and for the test set, patients from institution B were enrolled from January 2016 to January 2022. Exclusion criteria were 1) image corruption due to program errors and 2) incorrect risk stratification due to lack of information. The included patients were divided into two groups as high-risk group and non-high-risk group, with the latter consisting of patients with very low to intermediate risk. Data on clinical and biological factors, including age, sex, disease stage, MYCN gene amplification, and ploidy were collected from medical records. Variable MR protocols were used for the T2-weighted images according to the tumor location and size. The mean value was 2606±1372 msec for repetition time, 91±15 for echo time, and 5±2 mm for slice thickness (Supplementary Table 1, only online).
Development and testing of the radiomics model
Segmentation
Each tumor was semi-automatically segmented on T2-weighted MR images by two radiologists (J.K. with 6 years of experience, M.L. with 20 years of experience) using a commercial software package (syngo.via Frontier, version 1.3.0; Siemens Healthineers, Munich, Germany). The regions of interest (ROI) were drawn along the tumor margin, excluding encasing vascular structure on the cross-section selected from the largest plane available in the entire volume among the axial or coronal images (Fig. 1). Segmentation was independently implemented by two radiologists who were blinded to each other. One radiologist (J.K. with 6 years of experience) repeated segmentation once more a month later for all patients. The maximum diameter of a tumor was defined as the largest measured value among the T2-weighted images of the three axes.
Fig. 1
Regions of interest drawn along the tumor margin of the widest section of the T2-weighted axial images of 3-year-and-4-month-old girl assigned to non-high-risk group (A) and 3-year-and-1-month-old boy assigned to high-risk group (B).
Normalization
Using linear interpolation, MR images were resampled with a spatial resolution of 1×1×1 mm3. All images were processed to normalize the MR signal intensity using the same software, according to the following equation:
with f(x) as normalized intensity, x as original intensity, µx as mean, and σx as the standard deviation of the image signal intensity contained in each drawn ROI.13, 14, 15
Extraction, selection, and modeling of radiomics features
A software package (syngo.via Frontier, version 1.3.0; Siemens Healthineers) was used for the radiomics analysis, which was developed based on the PyRadiomics library, version 3.0.1 (https://github.com/Radiomics/pyradiomics) and scikit-learn machine learning library (https://scikit-
Fig. 2
A cluster map produced through agglomerative hierarchical clustering to visualize the association between the found clusters of subjects and features.
We selected multivariate logistic regression (MLR) for interpretability and random forest (RF) models from 10-fold cross-validation to evaluate non-linear relationships and avoid overfitting. Using classic minimum redundancy maximum relevance with R2 difference, the 10 most relevant features were selected from all the extracted radiomics features and clinical data. The best feature subset of MLR was determined with the forward selection method using adjusted R2. Also, a RF model was built using the 10 selected features. Specifically, 10-fold cross-validation was conducted with iterations of feature selection and model development for the RF model. The average area under the receiver operating characteristic curve (AUC) and average sensitivity, specificity, and accuracy with a threshold probability of 0.5 were provided as performance metrics for the training set. Both the trained MLR and RF models were tested in the external test set.
Statistical analysis
Statistical analyses were performed using SPSS software (version 25.0; IBM Corp., Armonk, NY, USA) and R software (R Foundation for Statistical Computing). Univariate analyses using clinical and biological data were performed to assess correlations with the high-risk group. Kaplan–Meier survival plots were analyzed to compare the survival rates between patient groups in the training set using the log-rank test. Intraclass coefficients of the features were calculated to evaluate intra- and inter-observer variability using the extracted features before building the MLR and RF models. p values less than 0.05 were considered statistically significant.
RESULTS
Patient populations
In the training set, a total of 32 patients (M:F=23:9, mean age 26.0±26.7 months) were enrolled with pre-treatment T2-weighted MR images after excluding only one patient with corrupted images due to program errors. Patients were divided into two groups: the high-risk group diagnosed with neuroblastoma (n=9) and the non-high-risk group diagnosed with either neuroblastoma (n=14), ganglioneuroblastoma (n=4), or ganglioneuroma (n=5), according to the INRG pre-treatment classification system (Table 1).
Table 1
International Neuroblastoma Risk Group Consensus Pretreatment Classification Schema
The mean age of the high-risk group (32.3±16.9 months) and non-high-risk group (23.5±29.7 months) was not significantly different (p=0.408) with no significant difference in sex between the two groups (p=0.185). MYCN gene amplification was found in six out of nine patients in the high-risk group, and distant metastasis was reported in six out of 23 patients in the non-high-risk group. The Kaplan–Meier survival plots are presented in Fig. 3. In the non-high-risk group, 18 out of 23 patients (78.3%) showed no evidence of disease (NED) status and two out of 23 (8.7%, follow-up period: 2.6±3.3 months) showed disease progression, whereas only one out of nine patients (11.1%) in the high-risk group showed NED status and six out of nine (66.7%) showed disease progression (p<0.001, follow-up period: 23.0±15.6 months). The primary tumor location of the non-high-risk group was the thorax (11/23, 47.8%), followed by the adrenal gland (7/23, 30.4%). In the high-risk group, seven out of nine patients had tumors located in the adrenal gland (77.8%), and there was one case with tumors in the retroperitoneum and thorax (11.1%, each). The location of the primary tumor did not statistically differ between the two risk groups (p=0.103) (Table 2).
Fig. 3
Kaplan–Meier survival plots comparing the high-risk and non-high-risk groups in the training set using the log-rank test (p<0.001).
Table 2
Clinical, Laboratory, and MRI Results of Patients in the Training Set from Institution A (n=32) and Test Set from Institution B (n=14)
The test set included 14 patients (M:F=10:4, mean age 33.4±20.4 months) after two patients were excluded for image corruption and two for incorrect risk stratification due to lack of information. Three out of 14 patients were included in the high-risk group (M:F=3:0, mean age 32.0±6.9 months). Eleven patients were included in the non-high-risk group (M:F=7:4, mean age 33.7±23.1 years) with three neuroblastomas, four ganglioneuroblastomas, and four ganglioneuromas. Of the high-risk group, two patients reached NED status and one was lost to follow-up. Eight patients of the non-high-risk group showed NED status, and three were receiving treatment at the time of data collection (Table 2).
Development and testing of the radiomics model
A total of 930 radiomics features were extracted from T2-weighted MRI images. The top 10 most relevant features were selected with three original and seven wavelet-based features, original glcm Maximum Probability, original glszm Gray Level Non Uniformity Normalized, original glszm Gray Level Variance, wavelet-HLL first order Energy, wavelet-LLL glszm Zone Entropy, wavelet-LLL glcm Joint Energy, wavelet-LLL glcm Maximum Probability, wavelet-LHL gldm Dependence Non Uniformity, wavelet-HHH gldm Dependence Non Uniformity, and wavelet-HHL gldm Dependence Non Uniformity.
The heatmap of these radiomics features showed a trend towards differences between the non-high-risk group and high-risk group (Fig. 4). Among the 10 selected features, two features were finally used to build the MLR model. The odds ratios (ORs) for each feature were as follows: wavelet-LLL_glcm_MaximumProbability: OR 7.088, confidence interval (CI) 1.329–37.821 (p=0.022) and wavelet-HHH_gldm_DependenceNonUniformity: OR 4.042, CI 0.612–26.68 (p=0.147). The AUC of the MLR model was 0.94 (sensitivity 67%, specificity 91%, and accuracy 84%). The average AUC of the RF model from 10-fold cross-validation was 0.83 (sensitivity 44%, specificity 87%, and accuracy 75%) (Fig. 5). The diagnostic performances of MLR and RF models were not significantly different (p=0.165).
Fig. 4
Feature extraction from T2-weighted MR images and selection of the 10 most relevant features.
Fig. 5
Receiver operating characteristic curves of the multivariate logistic regression (MLR) and random forest (RF) models from 10-fold cross-validation to predict high-risk neuroblastoma in the training set (p=0.165).
In the external test set, the AUC of the MLR model was 0.94 (sensitivity 33%, specificity 91%, and accuracy 79%) and the average AUC of the RF model was 0.91 (sensitivity 67%, specificity 91%, and accuracy 86%) (Fig. 6). In the test set as well, the diagnostic performances of the MLR and RF models were not significantly different (p=0.480).
Fig. 6
Receiver operating characteristic curves of the multivariate logistic regression (MLR) and random forest (RF) models from 10-fold cross-validation to predict high-risk neuroblastoma in the external test set (p=0.480).
Reproducibility of the radiomics features extracted from MRI
Intra-observer variability
When intraclass correlation coefficients (ICCs) were calculated for the 930 radiomics features extracted through two segmentations performed independently by a radiologist, ICCs showed poor (450/930, 48.4%), moderate (232/930, 24.9%), good (190/930, 20.4%), and excellent (58/930, 6.2%) repeatability.
The ICCs of the 93 original features were poor for 28 (<0.5), moderate for 35 (0.5 to <0.75), good for 28 (0.75 to <0.9), and excellent for two (0.9–1.0). Among the 744 wavelet-filtered features, the ICCs were poor for 394 features (<0.5), moderate for 193 (0.5 to <0.75), good for 149 (0.75 to <0.9), and excellent for eight (0.9–1.0). The ICCs of 93 square-filtered features were poor for (<0.5) 28, moderate for four (0.5 to <0.75), good (0.75 to <0.9) for 13, and excellent (0.9–1.0) for 48.
The ICC of the most relevant features chosen to create the prediction model was moderate (0.5 to <0.75) for six features (original glszm Gray Level Non Uniformity Normalized 0.633, original glszm Gray Level Variance 0.633, wavelet-HLL first order Energy 0.653, wavelet-LLL glszm Zone Entropy 0.671, wavelet-LHL gldm Dependence Non Uniformity 0.685, wavelet-HHL gldm Dependence Non Uniformity 0.718), good (0.75 to <0.9) for two features (wavelet-HHH gldm Dependence Non Uniformity 0.813, wavelet-LLL glcm Joint Energy 0.882), and excellent (0.9 to 1.0) for two features (original glcm Maximum Probability 0.909, wavelet-LLL glcm Maximum Probability 0.911) (Table 3).
Table 3
Reproducibility of the 10 Most Relevant Features Extracted from MRI
Inter-observer variability
When the ICCs of the 930 radiomics features extracted after the normalization were calculated to compare the two radiologists, the ICCs were widely distributed with poor (276/930, 29.7%), moderate (314/930, 33.8%), good (231/930, 24.8%), and excellent (109/930, 11.7%) reproducibility.
The ICCs of the 93 original features were poor for 14 (<0.5), moderate for 45 (0.5 to <0.75), good for 31 (0.75 to <0.9), and excellent for three (0.9–1.0). Among the 744 wavelet-filtered features, the ICCs were poor for 245 features (<0.5), moderate for 256 (0.5 to <0.75), good for 189 (0.75 to <0.9), and excellent for 54 (0.9–1.0). The ICCs of the 93 square-filtered features were poor (<0.5) for 17, moderate (0.5 to <0.75) for 13, good (0.75 to <0.9) for 11, and excellent (0.9–1.0) for 52.
The ICC of the most relevant features, which were used to create the prediction model, was moderate (0.5 to <0.75) for three features (wave-HLL first order Energy 0.695 original glszm Gray Level Non Uniformity Normalized 0.737, original glszm Gray Level Variance 0.737), good (0.75 to <0.9) for four features (wavelet-LLL glszm Zone Entropy 0.759, wavelet-LLL glcm Joint Energy 0.781, original glcm Maximum Probability 0.781, wavelet-LLL glcm Maximum Probability 0.82), and excellent (0.9 to 1.0) for three features (wavelet-LHL gldm Dependence Non Uniformity 0.962, wavelet-HHH gldm Dependence Non Uniformity 0.979, wavelet-HHL gldm Dependence Non Uniformity 0.985) (Table 3).
DISCUSSION
In our study, the prognosis actually differed between high-risk and non-high-risk patients, and a combination of several wavelet and original features could predict high-risk neuroblastic tumors using T2-weighted MRI. Even when tested in patients from the external institution, the good performance of the built model was still maintained.
To date, there have been several radiomics studies on neuroblastoma based on CT. Three studies evaluated whether MYCN amplification could be predicted, one tried to differentiate intermediate/high mitosis-karyorrhexis index (MKI) from low MKI, and all four of the abovementioned studies presented radiomics models capable of predicting MYCN amplification or MKI with high accuracy.12, 16, 17, 18
Although MRI-based radiomics have been applied to various oncology fields, such as breast cancer, hepatocellular carcinoma, and prostate cancer,19, 20, 21 none has made an effort to study MRI-based radiomics in neuroblastoma patients. Since MRI could obtain several images with subtle differences depending on the setting even with the same sequence, concerns have been raised about the reproducibility of radiomics in MRI. Scalco, et al.22 reported that the same radiomics model could be applied if images undergo pre-processing through the normalization process. MRI-based radiomics can play a clinically important role in pediatric patients, since there are many children in whom only MRI is taken instead of CT due to radiation issues.
The most relevant features selected for modeling included Maximum Probability, Gray Level Non Uniformity Normalized, Gray Level Variance, Energy, Zone Entropy, Joint Energy, and Dependence Non Uniformity with or without wavelet filtering. All of these features indicated tumor homogeneity or heterogeneity according to their calculated values. Our results showed that the high-risk group and the non-high-risk group had significant differences even when heterogeneity was quantitatively evaluated with radiomics, which has previously been evaluated only qualitatively.11 As some cases have pathology downgraded after chemotherapy, surgical pathology can differ from expectations.23 In these cases, if we can use imaging studies to predict risk, precise risk stratification will be possible just by analyzing pre-chemotherapy images.
In this study, we analyzed data using segmentation for cross-sections rather than the entire tumor volume. This is because segmentation of the total neuroblastoma tumor volume is inaccurate and labor-intensive, considering that neuroblastoma characteristically grows by creeping around the surrounding structures and that there are difficulties in distinguishing primary lesions from conglomerated lymph node metastasis.24 It will be easier to apply the studied model in practice if the texture features of cross-sections can yield meaningful results.
When the ICCs of 930 extracted radiomics features were compared to evaluate intra- and inter-observer variability, they ranged widely from poor to excellent reliability. However, since the most relevant features used to build the radiomics model showed only moderate to excellent agreement, we can assume that the reproducibility of the radiomics model developed in this study is more than moderate.
Our study had some limitations. First, the number of included patients was relatively small. In particular, since only three high-risk patients were included in the external test set, it is possible that the sensitivity was underestimated. However, given the rarity of the disease, the number of 32 patients collected over 10 years is not that small, and when analyzed with the RF model, a 10-fold cross-validation was performed. Even with these efforts, there was a risk of overfitting in this study. Also, a sample bias could exist as only cross-sections were segmented, not the entire volume. However, the widest cross-sectional area was selected and segmented, and this is considered a simpler method for practical use. Another point to consider is that despite the shape being considered a significant feature given the growth pattern of neuroblastoma, shape features were excluded from our analysis, since as a radiologist, it is easy for us to evaluate image-defined risk factors and body compartment in neuroblastoma patients. It would be helpful for us to detect high-risk group before obtaining histopathology or mutation by radiomics texture analysis. Additional studies with a larger number of cases to overcome overfitting concerns and validate repeatability and reproducibility of radiomics features are needed.
In conclusion, an MRI-based radiomics model showed potential for predicting high-risk neuroblastoma among neuroblastic tumors. This method could be considered as an additional option for risk stratification without increasing radiation exposure for pediatric patients.
SUPPLEMENTARY MATERIAL
Details about the Scanner and Protocol Used for T2-Weighted Images of Training Set PatientsSupplementary Table 1
The authors have no potential conflicts of interest to disclose.
AUTHOR CONTRIBUTIONS:
Conceptualization: Mi-Jung Lee.
Data curation: Mi-Jung Lee and Young Hun Choi.
Formal analysis: Mi-Jung Lee and Jisoo Kim.
Investigation: Mi-Jung Lee and Jisoo Kim.
Methodology: Mi-Jung Lee and Jisoo Kim.
Project administration: Mi-Jung Lee.
Resources: Mi-Jung Lee and Young Hun Choi.
Software: Jisoo Kim and Mi-Jung Lee.
Supervision: Mi-Jung Lee.
Validation: Jisoo Kim, Young Hun Choi, and Mi-Jung Lee.
Visualization: Jisoo Kim.
Writing—original draft: Jisoo Kim.
Writing—review & editing: all authors.
Approval of final manuscript: all authors.
ACKNOWLEDGEMENTS
We would like to thank Mingeon Kim (Siemens Healthineers Ltd, Korea) for technical support on radiomic analysis of pediatric imaging data.
References
-
Carré A, Klausner G, Edjlali M, Lerousseau M, Briend-Diop J, Sun R, et al. Standardization of brain MR images across machines and protocols: bridging the gap for MRI-based radiomics. Sci Rep 2020;10:12340
-
MeSH Terms
Figures
1 / 6
Tables
1 / 3
