Introduction

The paradigm is shifting from qualitative visual assessment of medical imaging to quantitative data analysis with the development of high-throughput mining of low- to high dimensional data. Radiomic features are considered to be an important alternative for interpretation and analysis of medical images and to predict lesion characteristics with numerous features, from first-order to high-order features1,2,3,4.

However, radiomic features can have limitations in their reproducibility or stability. The stability of radiomic features is still challenging with a lack of standardization during image acquisition, reconstruction, segmentation and analyses even though standardized image processing and feature computation have allowed radiomic features to be stable4. Among the various types of medical imaging, magnetic resonance imaging (MRI) has a variety of imaging acquisition methods and combinations of complicated parameters even in the same imaging sequences, which makes it difficult to apply radiomic features to MRI.

Fast scanned techniques are essential in the acquisition of MRI because of the major limitation of MRI, the need for a long scan time, particularly in emergency situations such as after a suspected cerebral acute ischemic stroke5. Fast scanned images have been realized by using echo-planar imaging (EPI), parallel imaging, echo train length (ETL) and recently introduced advanced techniques such as compressed sensing and simultaneous multi-slice acquisition, and so on6,7,8,9,10,11,12,13. The various techniques have resulted in a very complicated combination of imaging parameters, which can hamper the acquisition of stable radiomic features.

T2-Fluid attenuated inversion recovery (FLAIR) is very commonly used and essential sequence for the evaluation of cerebral acute ischemic stroke patients9,10,11,14,15,16,17,18,19,20. Therefore, T2-FLAIR is an important candidate for the application of radiomic features. However, there have been attempts to reduce the scan time of T2-FLAIR for a long time, which resulted in various parameters of T2-FLAIR. The EPI, parallel imaging, and ETL have been widely used6,9,10,11,12. Nevertheless, the stability of the radiomic features have been poorly investigated.

We hypothesized that radiomic features from fast scanned T2-FLAIR show variability relative to conventional T2-FLAIR. Therefore, the aim of our study was to investigate the stability of radiomic features from various fast scanned T2-FLAIR images in patients with acute ischemic stroke, and to compare the agreement of the radiomic features with conventional T2-FLAIR as a reference standard.

Results

Stability of radiomic features in fast scanned T2-FLAIR compared with conventional T2-FLAIR

The proportions of stable radiomic features in the first group were 13.8% (82/595) in EPI-T2-FLAIR and 5.5% (33/595) in ETL-T2-FLAIR for acute ischemic hyperintensity and 15.6% (93/595) in EPI-T2-FLAIR and 5.1% (30/595) in ETL-T2-FLAIR for chronic ischemic hyperintensity. Proportions of stable radiomic features in the second group were 16.8% (100/595) in TR-T2-FLAIR and 1.8% (11/595) in ETL-T2-FLAIR for acute ischemic hyperintensity and 13.0% (77/595) in TR-T2-FLAIR and 0.7% (4/595) in ETL-T2-FLAIR for chronic ischemic hyperintensity. EPI-T2-FLAIR and TR-T2-FLAIR showed significantly higher proportions of stable radiomic features than those of ETL-T2-FLAIR (p < 0.001). Proportions of stable radiomic features in the first group were 9.7% (58/595) in the EPI-T2-FLAIR intersection and 3.4% (20/595) in the ETL-T2-FLAIR intersection for acute ischemic hyperintensity and 18.0% (107/595) in the EPI-T2-FLAIR intersection and 11.6% (69/595) in the ETL-T2-FLAIR intersection for chronic ischemic hyperintensity. Proportions of stable radiomic features in the second group were 12.1% (72/595) in the TR-T2-FLAIR intersection and 3.9% (23/595) in the ETL-T2-FLAIR intersection for acute ischemic hyperintensity and 9.6% (57/595) in the TR-T2-FLAIR intersection and 1.2% (7/595) in the ETL-T2-FLAIR intersection for chronic ischemic hyperintensity. EPI-T2-FLAIR and TR-T2-FLAIR showed significantly higher proportions of stable radiomic features than those of ETL-T2-FLAIR (p < 0.002). The detailed results are listed in Table 1.

Table 1 Reproducibility of radiomics features in fast scanned T2-FLAIR compared with original T2-FLAIR as a reference standard.
Full size table

Stable radiomic features in fast scanned T2-FLAIR compared with conventional T2-FLAIR

There were no stable radiomic features across acute and chronic ischemic hyperintensities in the first and second groups. Stable radiomic features across acute and chronic ischemic hyperintensities in the first group were 1.01% (1/99) in gray-level run-length matrix (GLRLM) and 2.22% (1/45) in neighboring gray tone difference matrix (NGTDM). Stable radiomic features across acute and chronic ischemic hyperintensities in the second group were 1.01% (1/99) in GLRLM. Stable radiomic features across acute ischemic hyperintensities in the first group were 2.17% (1/46) in the first order, 2.53% (5/198) in gray-level co-occurrence matrix (GLCM), 4.04% (4/99) in GLRLM, 2.22% (1/45) in local binary pattern (LBP), 1.71% (2/117) in gray-level size zone matrix (GLSZM), and 6.67% (3/45) in NGTDM. Stable radiomic features across chronic ischemic hyperintensities in the first group were 4.44% (2/45) in the second order, 2.02% (4/198) in GLCM, 6.06% (6/99) in GLRLM, 4.27% (5/117) in GLSZM, and 4.44% (2/45) in NGTDM. Stable radiomic features across acute ischemic hyperintensities in the second group were 2.17% (1/46) in the first order, 2.02% (4/198) in GLCM, 2.02% (2/99) in GLRLM, 2.22% (1/45) in LBP, 0.85% (1/117) in GLSZM, and 4.44% (2/45) in NGTDM. Stable radiomic features across chronic ischemic hyperintensities in the second group were 1.01% (1/99) in GLRLM, 2.22% (1/45) in LBP, and 2.22% (1/45) in NGTDM.

There were no stable radiomic features across acute and chronic ischemic hyperintensities using intersection ROIs in the first group. Stable radiomic features across acute and chronic ischemic hyperintensities in the second group were 1.01% (1/99) in GLRLM and 2.22% (1/45) in LBP. Stable radiomic features across acute ischemic hyperintensities in the first group were 4.35% (2/46) in the first order, 2.22% (1/45) in the second order, 1.01% (2/198) in GLCM, 4.04% (4/99) in GLRLM, 2.22% (1/45) in LBP, and 1.71% (2/117) in GLSZM. Stable radiomic features across chronic ischemic hyperintensities in the first group were 2.22% (1/45) in the second order, 2.53% (5/198) in GLCM, 6.06% (6/99) in GLRLM, 5.13% (6/117) in GLSZM, and 4.44% (2/45) in NGTDM. Stable radiomic features across acute ischemic hyperintensities in the second group were 6.52% (3/46) in the first order, 3.03% (6/198) in GLCM, 5.05% (5/99) in GLRLM, 2.22% (1/45) in LBP, 2.56% (3/117) in GLSZM, and 4.44% (2/45) in NGTDM. Stable radiomic features across chronic ischemic hyperintensities in the second group were 1.01% (1/99) in GLRLM and 4.44% (2/45) in LBP. All of the details mentioned above are shown in Figs. 1, 2 and 3.

Figure 1
figure 1

Heat maps of radiomics features extracted from whole- and intersection ROI masks in the first and second groups. Features with CCC > 0.85 were regarded as stable. (a) Features from whole ROI masks in the first group. (b) Features from whole ROI masks in the second group. (c) Features from intersection ROI masks in the first group. (d) Features from intersection ROI masks in the second group. AIH acute ischemic hyperintensity, CIH chronic ischemic hyperintensity, CCC concordance correlation coefficients, ROI region-of-interest, EPI echo-planar imaging, ETL echo train length, TR repetition time.

Full size image
Figure 2
figure 2

Proportions of stable radiomic features extracted from whole- and intersection ROI masks on each fast scanned T2-FLAIR sequence in the first and second groups. (a,b) Proportions of stable radiomic features from whole ROI masks (a) and intersection ROI masks (b) according to each fast scanned T2-FLAIR. (c,d) Numbers of stable radiomic features from whole ROI masks (c) and intersection ROI masks (d) according to each fast scanned T2-FLAIR. AIH acute ischemic hyperintensity, CIH chronic ischemic hyperintensity, ROI region-of-interest, EPI echo-planar imaging, ETL echo train length, TR repetition time, GLCM gray-level co-occurrence matrix, GLRLM gray-level run-length matrix, LBP local binary pattern, GLSZM gray-level size zone matric, NGTDM neighboring gray tone difference matrix.

Full size image
Figure 3
figure 3

Proportions of stable radiomic features throughout both fast scanned T2-FLAIR sequences extracted from whole- and intersection ROI masks in the first and second groups. (a,b) Proportions of stable radiomic features from whole ROI masks (a) and intersection ROI masks (b). (c,d) Numbers of stable radiomic features from whole ROI masks (c) and intersection ROI masks (d). AIH acute ischemic hyperintensity, CIH chronic ischemic hyperintensity, ROI region-of-interest, EPI echo-planar imaging, ETL echo train length, TR repetition time, GLCM gray-level co-occurrence matrix, GLRLM gray-level run-length matrix, LBP local binary pattern, GLSZM gray-level size zone matric, NGTDM neighboring gray tone difference matrix.

Full size image

Discussion

This study showed a consistent tendency of higher proportions of reliable features in EPI-T2-FLAIR and TR-T2-FLAIR than ETL-T2-FLAIR in both acute and chronic ischemic hyperintensities and for both whole- and intersection-ROI mask. Therefore, various image acquisitions of T2-FLAIR resulted in unstable radiomic features, which may lead to different radiomic features’ outcomes, such as prediction modeling.

MRI is a useful and sometimes essential imaging modality to identify the infarct core on DWI, and additional useful information can be obtained from various images such as T2-FLAIR or gradient echo (GRE) images, and also allow acquisition of vessel information without the need for contrast media during the evaluation of cerebral acute ischemic stroke patients; however, MRI has a lesser availability and a longer scan time compared to CT5,21,22,23,24,25. Therefore, there have been many attempts to reduce the scan time of MRI in cerebral acute ischemic stroke situations, which has resulted in various MRI sequences and parameters being applied in clinical practice6,9,10,11,12.

Previous studies on fast-scanned T2-FLAIR in acute ischemic stroke showed a consistent superior reliability when compared with that of conventional images6,9,10,11. However, those studies only showed repeatability or reliability in qualitative scoring systems or simple quantitative comparisons, such as signal intensity. In contrast, our study based on radiomic features showed a lower reliability than that of conventional images even though the data was originated from the same registry in the previous study6.

The diversity in the image acquisition makes it difficult to apply radiomic features to MRI for cerebral acute ischemic stroke. Imaging acquisition, segmentation, and feature extraction can affect the stability of radiomic features2. Ford et al. demonstrated that changes of imaging parameters could lead to variable radiomic features in a phantom study26. Minjae et al. only showed that the change in acceleration factors on the same images can affect the stability of radiomic features, and two different under-sampling methods on the same images can show different radiomic features even under the same acceleration factors27. Therefore, different imaging parameters even on the same FLAIR sequences may reduce the stability of radiomic features, as in this study. In addition, these obstacles may affect a few published studies on the predictive models developed using MRI radiomic features in cerebral acute ischemic stroke28,29,30. The results from this study showed some stable radiomic features across variable acquisition of T2-FLAIRs and acute and chronic ischemic hyperintensities, but a substantial proportion showed variability. To our knowledge, there is no previous report on the stability of MRI radiomic features according to various imaging parameters in cerebral acute ischemic stroke. The segmentation can affect the stability of radiomic features. Manual segmentation, compared with semi- or automated segmentations, may lead to lower reproducibility in radiomic features2. However, many previous studies on radiomic features or high dimensional quantitative analyses using artificial intelligence relied on manual segmentations. In addition, Haarburger et al. showed poor reproducibility of some radiomic features even under automated segmentation methods31. The segmentation reproducibility can be influenced by the anatomic location and lesion types27,32. In our study, there were some different results between whole and intersection ROI masks, which may be also owing to different sizes of ROI masks and thus different numbers of pixels. Feature extraction can also affect the stability of radiomic features. Li et al. demonstrated the poor stability of radiomic features (no features > 0.85 in concordance correlation coefficient [CCC]) across different extraction combinations33.

Studies evaluating stable radiomic features in cerebral acute or chronic ischemic lesions based on multiparameteric variances appear to be lacking, and the reproducibility of radiomic features in brain tumors, including glioblastoma, has been reported and several features belonging to GLRLM were identified as reproducible features33,34. GLRLM was also the most reproducible feature in cine balanced steady-state free procession and first-order, and GLCM had the most reproducible features on both T1 and T2 maps in the myocardial radiomic features35. However, a phantom study for test–retest reproducibility reported that GLRLM was neither the most robust nor least robust feature class, while GLCM was one of the least robust feature classes across MRI sequences: FLAIR, T1-weighted, and T2-weighted imaging36. Our study also showed that GLCM and GLRLM are common stable features in the numbers but some variability was seen in the proportions of stable radiomic features.

This study has several limitations. First, this study was designed as a retrospective study with a small population in a single center. A study population cannot be free from selection bias, which may have affected the deviations in sex and age, and a specific MR machine was adopted. Therefore, further multi-center studies with a larger sample size are necessary. Second, we did not compare all of the T2-FLAIRs simultaneously because it is hard to obtain all of the T2-FLAIRs at the same time in a cerebral acute ischemic stroke situation. Third, this study presented only the stabilities of radiomic features in T2-FLAIRs of acute ischemic stroke and an evaluation of the stability of identification or prediction models influencing the treatment options for stroke outcomes using radiomic features from variable T2-FLAIRs is necessary.

In conclusion, the fast-scanned T2-FLAIR showed small proportions of stable radiomic features and variable stability of radiomic features among those in terms of the agreements with conventional T2-FLAIR. Therefore, even if the images in the same sequence have different parameters, the radiomic features obtained from the images may be used with caution.

Methods

Study population

From May 2015 to June 2016, data on 296 patients undergoing MRI at a single tertiary hospital for symptoms of acute ischemic stroke were retrospectively collected. Among them, 118 patients underwent echo-planar imaging (EPI)-T2-FLAIR and echo train length (ETL)-T2-FLAIR and 178 patients underwent ETL-T2-FLAIR and repetition time (TR)-T2-FLAIR simultaneously. In total, 79 patients showed acute ischemic hyperintensity and 89 patients showed chronic ischemic hyperintensity on simultaneous acquisition of EPI- and ETL-T2-FLAIR, who were classified to the first group, and 112 patients showed acute ischemic hyperintensity and 127 patients showed chronic ischemic hyperintensity on simultaneous acquisition of ETL- and TR-T2-FLAIR, who were classified to the second group for comparable paired data analysis. The detailed demographics of the patients are presented in Table 2. The data on patients were collected from the fast stroke MRI registry in our institute6. The institutional review board of Asan Medical Center approved the present study, and the requirement for informed consent was waived. The data was analyzed in compliance with the International Council for Harmonization of Technical Requirements for Registration of Pharmaceutical for Human Use: Guideline for Good Clinical Practice (ICH GCP).

Table 2 Demographics of the patients.
Full size table

Image acquisition

All T2-FLAIR was scanned on a 1.5-T scanner (Magnetom Avanto; Siemens Healthineers). The scan times were 128 s for conventional T2-FLAIR, 45 s for EPI-T2-FLAIR, 74 s for ETL-T2-FLAIR, and 79 s for TR-T2-FLAIR. The detailed scan parameters for the conventional and fast T2-FLAIR were as previously reported6 and are listed in Table 3 and representative images are presented in Fig. 4.

Table 3 T2-FLAIR protocols.
Full size table
Figure 4
figure 4

Acute and chronic ischemic hyperintensities on conventional T2-FLAIR and fast scanned T2-FLAIR with respect to DWI. (a) First group with conventional T2-FLAIR, EPI-T2-FLAIR, and ETL-T2-FLAIR. (b) Second group with conventional T2-FLAIR, ETL-T2-FLAIR and TR-T2-FLAIR. DWI diffusion-weighted image, FLAIR fluid attenuated inversion recovery, EPI echo-planar imaging, ETL echo train length, TR repetition time.

Full size image

Image analysis

The segmentations of acute- and chronic ischemic hyperintensities were conducted as described in a previous report6. We defined acute ischemic hyperintensity as a T2-FLAIR high signal intensity within acute infarcts on diffusion weighted images (DWI)15,17, and chronic ischemic hyperintensity as a hyperintensity outside of acute infarcts on DWI. The segmentation of region-of-interest (ROI) mask was done by one research assistant (K.M.C. with 5 years of experience in stroke imaging) using an in-house program for semiautomatic segmentation (based on ImageJ software; National Institutes of Health, Bethesda, MD). The intersection ROI mask between the respective ROI mask of the conventional and fast T2-FLAIR was obtained after the coregistration and then each intersection ROI mask was transferred into the respective conventional and fast T2-FLAIR images. The intersection ROI mask was used to compare the radiomic features from different T2-FLAIRs without ROI mask differences. A total of 14 kinds of ROI masks were obtained as follows: 3 ROIs from conventional-, EPI-, and ETL-T2-FLAIR in the first group; 2 intersection ROIs from conventional and EPI-T2-FLAIR, and 2 intersection ROIs from conventional and ETL-T2-FLAIR in the first group; 3 ROIs from conventional-, ETL-, and TR-T2-FLAIR in the second group; 2 intersection ROIs from conventional and ETL-T2-FLAIR, and 2 intersection ROIs from conventional and TR-T2-FLAIR in the second group (Fig. 5). From the ROIs, 595 radiomics features were extracted and concordance correlation coefficients (CCC) for radiomic features were calculated between fast scanned and conventional T2-FLAIR in each group.

Figure 5
figure 5

The outline of the image analysis focused on the types of ROI masks used in this study. Feature extraction was performed from each conventional and fast scanned T2-FLAIR using both whole-(× 3) and intersection ROI masks (× 4) for both the first and second groups (× 2). FLAIR fluid attenuated inversion recovery, ROI region-of-interest, EPI echo-planar imaging, ETL echo train length, TR repetition time.

Full size image

Radiomic features

Radiomic features were extracted with Matlab R2016a (The Mathworks, Natick, MA): first-order features, texture features, and wavelet-transformed features37,38. The first-order features were acquired based on the histogram analyses of pixel values within the region-of-interest. The second-order features or texture features were as follows: gray-level co-occurrence matrix (GLCM), gray-level run-length matrix (GLRLM), local binary pattern (LBP), gray-level size zone matric (GLSZM), and neighboring gray tone difference matrix (NGTDM). The wavelet transformations extracted additional features. The radiomic features extraction was done under the Imaging Biomarker Standardization Initiative39. Finally, 595 radiomic features were extracted with 46 first-order features, 61 texture features (5 s order, 22 GLCM, 11 GLRLM, 5 LBP, 13 GLSZM, 5 NGTDM) and 488 wavelet features (× 8; 40 s order, 176 GLCM, 88 GLRLM, 40 LBP, 104 GLSZM, 40 NGTDM).

Statistical analysis

The stability of the radiomic features was evaluated using CCC between the features extracted from the conventional- and fast scanned T2-FLAIR based on Lin’s definition40. The proportions of stable radiomic features were compared between EPI and ETL in the first group and between TR and ETL in the second group using McNemar’s test. Radiomic features with a CCC of higher than 0.85 were considered to be stable. All statistical analyses were performed using the MedCalc 15.6.1 software package (MedCalc Software) and R version R 3.3.3 (R Foundation for Statistical Computing; http://www.R-project.org, 2016).