Published online Jun 16, 2023.
https://doi.org/10.13104/imri.2022.1102
Artificial Intelligence and Deep Learning in Musculoskeletal Magnetic Resonance Imaging
Abstract
The application of artificial intelligence (AI) and deep learning (DL) in radiology is rapidly evolving. AI in healthcare has benefits for image recognition, classification, and radiological workflows from a clinical perspective. Additionally, clinical triage AI can be applied to triage systems. This review aims to introduce the concept of DL and discuss its applications in the interpretation of magnetic resonance (MR) images and the DL-based reconstruction of accelerated MR images, with an emphasis on musculoskeletal radiology. The most recent developments and future directions are also discussed briefly.
INTRODUCTION
Artificial intelligence (AI) is rapidly evolving in the field of radiology. Deep learning (DL) is a subfield of AI and machine learning (Fig. 1), and is characterized by an algorithm that uses a neural network with multiple layers. This technique is used to extract a hierarchy of structures and higher-level features from raw input data [1, 2, 3]. Additionally, from a clinical perspective, AI in healthcare has benefits for radiological workflow, such as improving risk prediction and intervention, advising medical decision-making, and assisting with early triage [4] and radiological reports [5].
Fig. 1
Venn diagram of the relationship between artificial intelligence (AI), machine learning (ML), deep learning (DL), and a convolutional neural network (CNN). AI is a branch of science and engineering concerned with making intelligent systems perform tasks based on external data in the same way that humans do. ML is a subfield of AI that enables computers to perform tasks and learn without explicit programming. DL is a subset of ML in which the algorithm studies comprehensive features that reflect a structural hierarchy in the data. CNN is a deep learning architecture distinguished by structured multiple data processing arrays.
Magnetic resonance imaging (MRI) is a valuable imaging tool for diagnosing and treating musculoskeletal and spinal disorders by visualizing the anatomy and pathology ranging from the bones and cartilages to muscles [6]. However, MRI has several drawbacks, including image quality issues, radiologist errors, and long acquisition times, which can result in patient discomfort [7, 8]. Herein, we discuss recent methods and future directions to overcome these drawbacks from the perspective of DL applications in the image interpretation and reconstruction of musculoskeletal MRI.
DL APPLICATIONS ON IMAGING DIAGNOSIS IN MUSCULOSKELETAL MRI
The DL interpretation of musculoskeletal MRI has emerged. In situations with a gradually increasing number of examinations, radiologists expect DL to reduce workloads. In this context, we describe DL applications for the interpretation of musculoskeletal images, especially MRI images.
Knee Anterior Cruciate Ligament
For diagnosing knee injuries, MRI is a useful and effective noninvasive imaging diagnostic tool with high spatial resolution and excellent soft tissue resolution that can clearly visualize the overall structure of the knee joint. DL has been used to identify anterior cruciate ligament (ACL) injuries. DL has been demonstrated to have a statistically equivocal or slightly lower performance than experienced radiologists [9, 10]; it has a sensitivity of 96%–96.1%, specificity of 93.5%–96%, and an area under the receiver operating characteristic curve (AUC) of 0.935–0.98, whereas radiologists have a sensitivity of 97.5%–98%, specificity of 98%–100%, and an AUC of 0.98–0.99. However, in another study employing binary detection (i.e., presence or absence of tears), DL outperformed radiologists in the detection of ACL tears [11]; radiologists had a sensitivity of 0.804–0.957 and specificity of 0.820–0.860, whereas DL had a sensitivity of 0.976 and specificity of 0.944. However, previous studies did not differentiate between partial- and full-thickness tears, and were restricted to the binary detection of ACL tears. More classification and multiple abnormality detection are required in future studies, not only for ACL tears, but also for cartilage defects and meniscal tears.
Several preprocessing techniques have been introduced for the detection of ACL tears to enhance DL performance. Equipped with cropped and additional randomly cropped image techniques, DL had the best performance in cropped and additional five-slice image settings, with a sensitivity and specificity of 100% and 93.3%, respectively [12]. In another study, two preprocessed images of non-cropped whole images and manually segmented images of the ACL demonstrated a sensitivity, specificity, and AUC of 97.6%, 94.4%, and 0.960, respectively [11]. This result had an increased sensitivity of 4.4% and specificity of 2.2% compared to preprocessed images of whole images only. Recent studies using 3D convolutional neural networks (CNNs) have used a preprocessing algorithm in which DL categorizes distinct anatomic components of the knee and crops the image automatically to isolate the ACL [13, 14].
Conventionally, DL has the typical weakness of a poor performance on an external dataset. Some researchers have compared the performance on internal and external datasets and identified a strategy to boost DL performance by adding more training to external datasets. In both internal and external MRI datasets, DL demonstrated unsatisfactory outcomes for external MRI, in which DL had a decreased sensitivity of 6.5%, specificity of 7.3%, and AUC of 0.069 in the outside MRI dataset [10]. In the internal dataset, DL had a high AUC value of 0.965 for ACL tear detection, whereas the AUC in the external dataset was 0.824 [15]. After additional training on the external training set, DL achieved an increased AUC (0.911). One option for actual clinical applications is additional training using an external dataset. Although many studies have focused on the standalone AI/DL performance, AI/DL may also aid radiologists and clinicians in image interpretation. The usefulness of DL-assisted image interpretation is crucial in clinical practice. General radiologists and orthopedic surgeons with DL assistants considerably improved the specificity of identifying ACL tears (4.8%) [15]. Inexperienced trainees significantly improved the agreement between experienced radiologists in the interpretation of cartilage, meniscus, and ACL abnormalities using DL-assisted grading [14]. In the future, DL-assisted diagnosis and grading in radiological reading rooms will be beneficial.
Knee Meniscus
Meniscal and cartilage abnormalities are frequently identified as pathologies on knee MRI scans. However, few studies on the application of DL have been conducted to date [15, 16, 17, 18, 19, 20, 21, 22] because of the challenging training process for detection and classification (tear orientation, such as horizontal or vertical) compared with ACL tear detection. Automatic detection of meniscal tears had a poorer performance than the detection of ACL tears, in which the AUC of meniscal tear detection and classification was 0.7791–0.906 (Fig. 2). Using arthroscopy as a reference standard, the authors evaluated the sensitivity of DL in the medial meniscus and observed that it was significantly lower than that of the radiologist by 9%–12%, in which the sensitivities of the two radiologists was 93.0%–96.5% and that of DL was 84.2% [15, 16]. Another study observed excellent diagnostic performance in both meniscal tear detection and localization with AUCs of 0.94 and 0.92, respectively [17].
Fig. 2
Magnetic resonance imaging of the knee of a 30-year-old male. A: A coronal short-tau inversion recovery image displays both menisci (arrow). B: A sagittal intermediate-weighted image with fat-suppression at the junction of the body to the posterior horn of the medial meniscus (arrow). C: Probability of a meniscal tear calculated by a convolutional neural network is represented by a heatmap. A horizontal tear with extension to the posterior horn is observed at the body of the medial meniscus. The convolutional neural network estimated the probability of a tear at 99.9%. Adapted from Fritz et al. [16], Skeletal Radiology 2020;49:1207-1217, used under CC BY 4.0 license. The legend has been modified from the original version.
Shoulder
DL applications in shoulder MRI have focused on the detection of rotator cuff tears and the evaluation of rotator cuff muscle atrophy. Few studies on the detection and classification of rotator cuff tears using DL have been reported [23, 24, 25]. DL has a relatively high performance for binary detection (tear or no-tear) of rotator cuff tears, with an accuracy of 87%–92.5% [24, 25]. In the classification of partial thickness tears, full-thickness tears, and normal tendons, DL demonstrated a poor performance; the sensitivity in the partial- and full-thickness groups was 72.5% and 100%, respectively [23]. The limited diagnostic performance in detecting partial-thickness tears may be due to misclassification between tendinosis and partialthickness tears. High performance in the segmentation of certain muscles and fractions of fat/muscle content was observed in the analyses of shoulder muscles [26, 27, 28, 29]. For the supraspinous fossa and muscle regions, the dice similarity coefficient, which evaluates the similarity of two datasets (predicted by DL and ground truth in these studies), ranged from 0.93 to 0.99 [27, 28, 29].
Spine
In the field of spine imaging, DL has been applied to vertebral segmentation (separation of vertebrae from intervertebral discs), spine detection (localization and identification of intervertebral discs and vertebrae), pathology detection (central canal stenosis and neural foraminal stenosis), and improvement of workflow efficiency. DL has slowly and steadily improved its diagnostic performance in detecting central canal stenosis [30, 31], neural foraminal stenosis [32, 33], and disc degeneration [34, 35]. In a recent study, DL demonstrated comparable agreement with experienced radiologists (recall of > 99%) and statistically lower agreement with foraminal stenosis (recall of 84.5%) [36]. An impressive time reduction and improvement in inter-observer agreement were recorded in a recent study that utilized DL to detect central canal, lateral recess, and neural foraminal stenosis [37]. With DL assistance, the image interpretation time per spine MRI was reduced from a mean of 124–127 s to 47–71 s, and the interobserver agreement was improved from a kappa value of 0.39 to 0.70–0.71.
DL-BASED IMAGE RECONSTRUCTION
Before the advent of AI/DL, parallel imaging (PI) [38] and compressed sensing (CS) [39] were commonly used to accelerate magnetic resonance (MR) acquisition. PI and CS are techniques based on undersampling k-space data. The major drawback of undersampled k-space data is that the sparsity of the reconstructed image results in image noise (low signal-to-noise ratio, SNR) and aliasing. Although CS preserves SNR better than PI, compression methods may blur information and oversimplify the image [40]. DL-based reconstruction, a new and different approach to accelerated MRI, is an emerging method for overcoming the disadvantages of PI and CS (Supplementary Fig. 1). The DL-based reconstruction is generally based on supervised or unsupervised learning algorithms. The majority of applications use supervised learning, in which fully sampled and paired undersampled data are coupled with machine training. Unsupervised learning remains a topic of currently active research [41]. In the clinical field, recent studies on DL-based reconstructed MRI images have focused on image quality for diagnostic accuracy, artifacts, and time reduction.
Ultrafast MRI can be performed with DL-based image reconstruction, which is helpful for patients with claustrophobia. This imaging technique demonstrated comparable image quality and noise while maintaining diagnostic performance (Fig. 3). A 5-minute 3D quantitative double-echo steady-state sequence of the knee with AI image quality enhancement demonstrated strong inter-reader agreement with the 20-minute conventional knee MRI and near-equivalent diagnostic performance with an arthroscopic reference [42]. Recht et al. [40] compared the diagnostic performance of DL-based reconstructed accelerated knee MRI with conventional MRI. Axial fat-suppressed T2-weighted images, sagittal proton density (PD)-weighted images, sagittal fat-suppressed T2-weighted images, and coronal PD-weighted images with and without fat suppression were all included in the DL-based reconstruction for knee imaging, which was performed within a four-fold acceleration of 5 min. Several studies on faster MRI with deep-learning applications in musculoskeletal MRI are summarized in Supplementary Table 1.
Fig. 3
Magnetic resonance imaging of the shoulder of a 66-year-old male. A: A coronal fat-suppressed T2-weighted with an acceleration factor 3 image indicates a supraspinatus tendon tear and fluid in the subacromial–subdeltoid bursa. The image demonstrates noisy patterns of the bone marrow and muscles. B: A corresponding deep learning-based compressed sensing reconstruction image demonstrates image quality enhancement with less noise. The torn supraspinatus tendon and vascular structures are clearly delineated.
DL-based reconstructed MRI have been evaluated in the shoulder and lumbar spine [43, 44]. The examination times for accelerated sequences were reduced by 67% in the former (scan time: 3 min 5 s vs. 9 min 23 s) [44]. In a spine study [43], DL-based reconstructed 3D sequences had a higher image quality score than the two standard sequences and similar inter-observer agreement for pathologies such as foraminal and central stenoses. These studies concluded that DL-based reconstruction has advantages in terms of time reduction, fewer artifacts, and diagnostic accuracy similar to conventional image reconstruction. However, DL-based reconstruction produces unrealistic [45] and oversmoothed images [46] (Fig. 4), which can hinder the gradual adoption of new methods in the clinical field [46]. The banding artifact produced by cartesian DL reconstruction was strong, especially in the low-SNR region of the reconstructed images [45, 47]. These artifacts and unrealistic over-smoothed images that radiologists are reluctant to obtain should be thoroughly overcome in clinical practice.
Fig. 4
Magnetic resonance imaging of the knee of a 67-year-old female. A: A coronal fat-suppressed T2-weighted with an acceleration factor 2 image indicates subchondral cysts and bone marrow edema with an osteochondral lesion in the medial femoral condyle. B: A corresponding deep learning-based compressed sensing reconstruction image demonstrates image quality enhancement with less noise. However, the image textures of the bone marrow are slightly blurred and appear over-smoothened.
Low-field (LF) MRI is another target of DL applications because of the increasing demands of LF MRI owing to its reduced maintenance cost, fewer susceptibility artifacts, and higher T1 contrast [48]. DL applications have inherent drawbacks such as a low SNR and relatively long scan time in LF MRI image acquisition [49]. Although DL techniques have grown rapidly in MRI, they still have several drawbacks, including the limitations of DL algorithms, large training data, and generalizability to different datasets or applications, such as LF and ultra-high-field MRI applications.
Resolution and scan time in MRI have a tradeoff. By using fast MR technology, the scan time can be shortened, and the spatial resolution can be improved. Enhancements in spatial resolution beyond fast imaging, such as image super-resolution, have been investigated [50]. A promising method for radiologic images is DL-based image super-resolution or super-resolution generative adversarial networks because DL can predict high-resolution images from lower-resolution images [51, 52]. Super-resolution imaging techniques are promising for musculoskeletal MRI when considering musculoskeletal joint imaging, which visualizes tiny structures in the joints.
RADIOLOGIC WORKFLOW
Recently, milestone DL models with extremely low error rates and high computational efficiency have demonstrated remarkable performance in lesion detection, classification, and segmentation tasks. However, the applications of AI in radiology are not limited to visual tasks. AI is expected to improve the efficiency of the radiological workflow beyond imaging acquisitions and reconstructions, including initial patient scheduling, optimized protocol, MRI reconstruction, image enhancement, medical image-to-image translation, and AI-assisted image interpretation [53]. Radiology is facing increasing pressure to improve productivity [54]. Radiologists can work more efficiently with intelligent hanging protocols in a picture archiving communication system (PACS), including appropriate preferred position, size syncing, and cross-referencing settings. AI has the potential to enhance PACS viewers using smart tools that process various available data [55]. These are essential applications for optimizing imaging workflows and improving noninterpretive tasks. For example, in clinical studies that require time, triage AI can be applied to automated DL-based triage systems for acute neurologic events [56]. It can generate a framework using computer-assisted surveillance of cranial imaging by prioritizing more emergent imaging studies. This process reduces the time to triage, improves early treatment, and improves patient outcomes. Similarly, the radiological worklist to be read can be sorted using a higher-priority image study.
CONCLUSION
DL-based image reconstruction has already achieved image quality comparable to that of conventional imaging of the knee, shoulder, and spine. Realistic and non-oversmoothed images are challenging but conquerable issues that need to be resolved. Future studies should be directed toward enhancing pathology detection rates and improving training processes.
The rapid image acquisition and increased spatial resolution of musculoskeletal MRI permit the noninvasive evaluation of tiny morphological changes using PI, CS, and other accelerated imaging techniques [57]. Recent advancements in DL and CNNs can help to generalize super-resolution imaging using natural 2D images for applications in 3D medical imaging [58, 59]. Furthermore, generative adversarial networks can generate realistic data and have received considerable attention in the field of DL [60]. AI/DL in musculoskeletal radiology is anticipated to be the next step in future radiology because of its capacity to go beyond detection and classification and move toward efficient and fast image reconstruction capabilities. Although DL-based image reconstructions have a promising future in musculoskeletal radiology, their real-world application still requires large-scale clinical validation. Therefore, radiologists need to understand the inherent properties of specific data within DL image reconstruction and to collaborate with other radiologists, MR physicists, MR engineers, and data scientists in the MRI world.
Supplementary Materials
The online-only Data Supplement is available with this article at https://doi.org/10.13104/imri.2022.1102.
Summary of Studies on Accelerated Magnetic Resonance Imaging (MRI) with Deep Learning Applications in Musculoskeletal MRISupplementary Table 1
Magnetic resonance imaging of the ankle of a 42-year-old male. Coronal T1-weighted and intermediate-weighted turbo spin echo images with spectral presaturation with inversion recovery, using compressed sensing (CS) (A, D), CS artificial intelligence (CSAI) with an acceleration factor of 4.6–5.3 (CSAI 2×) (B, E), and CSAI with an acceleration factor of 6.9–7.7 (CSAI 3×) (C, F). An osteochondral defect (arrowheads) and degenerative changes of the subtalar joint (arrows) were detected. The depicted bone was scored equally on CS and CSAI 2× images (Likert score of 5, excellent), although it rated slightly lower on CSAI 3× images (Likert score of 4, good). The scan duration was 12:44, 6:45, and 4:46 min for CS, CSAI 2×, and CSAI 3×, respectively. Adapted from Foreman et al. [6], Eur Radiol 2022;32:8376-8385, used under CC BY 4.0 license. The legend was modified from the original version.Supplementary Fig. 1
REFERENCES
Click here to view.(17K, pdf)
Conflicts of Interest:The authors have no potential conflicts of interest to disclose.
Author Contributions:
Conceptualization: Young Han Lee.
Funding acquisition: Young Han Lee.
Writing—original draft: Seung Dae Baek, Young Han Lee.
Writing—review & editing: Joohee Lee, Sungjun Kim, Ho-Taek Song.
Funding Statement:This work was supported by a National Research Foundation (NRF) grant funded by the Korea government, Ministry of Science and ICT (MSIP, 2022R1F1A1071702).
References
-
Germann C, Marbach G, Civardi F, et al. Deep Convolutional Neural Network–based diagnosis of anterior cruciate ligament tears: performance comparison of homogenous versus heterogeneous knee MRI cohorts with different pulse sequence protocols and 1.5-T and 3-T magnetic field strengths. Invest Radiol 2020;55:499–506.
-
-
Pedoia V, Norman B, Mehany SN, Bucknor MD, Link TM, Majumdar S. 3D convolutional neural networks for detection and severity staging of meniscus and PFJ cartilage morphological degenerative changes in osteoarthritis and anterior cruciate ligament subjects. J Magn Reson Imaging 2019;49:400–410.
-
-
Irmakci I, Anwar SM, Torigian DA, Bagci U. In: Matthews MB, editor. 2019 53rd Asilomar Conference on Signals, Systems, and Computers; 2019 Nov 3-6; Pacific Grove, USA. Pacific Grove: IEEE; 2019. pp. 1481-1485.
-
-
Tsai CH, Kiryati N, Konen E, Eshed I, Mayer A. In: Arbel T, Ayed IB, Bruijne M, Descoteaux M, Lombaert H, Pal C, editors. International Conference on Medical Imaging with Deep Learning; 2020 July 6-8; Montréal, Canada. 2020. pp. 784-794.
-
-
Kim M, Park H, Kim JY, Kim SH, Hoeke S, Neve WD. In: Doshi-Velez F, Fackler J, Jung K, et al., editors. MLHC 2020: Machine Learning for Healthcare Conference; 2020 Aug 7-8; 2020. pp. 292-308.
-
-
Al-Kafri AS, Sudirman S, Hussain A, et al. Boundary delineation of MRI images for lumbar spinal stenosis detection through semantic segmentation using deep neural networks. IEEE Access 2019;7:43487–43501.
-
-
Han Z, Wei B, Leung S, Chung J, Li S. In: Frangi A, Schnabel J, Davatzikos C, Alberola-López C, Fichtinger G, editors. Medical Image Computing and Computer Assisted Intervention – MICCAI 2018; 2018 Sep 16-20; Granada, Spain. Cham: Springer; 2018. pp. 185-193.
-
-
Jamaludin A, Lootus M, Kadir T, et al. ISSLS PRIZE IN BIOENGINEERING SCIENCE 2017: automation of reading of radiological features from magnetic resonance images (MRIs) of the lumbar spine without human intervention is comparable with an expert radiologist. Eur Spine J 2017;26:1374–1383.
-
-
Matcuk GR Jr, Gross JS, Fritz J. Compressed sensing MRI: technique and clinical applications. Adv Clin Radiol 2020;2:257–271.
-
-
Defazio A, Murrell T, Recht MP. In: Larochelle H, Ranzato M, Hadsell R, Balcan MF, Lin H, editors. 34th Conference on Neural Information Processing Systems (NeurIPS 2020); 2020 Dec 6-12; Vancouver, Canada. 2020. pp. 7660-7670.
-
-
Sarracanie M, Salameh N. Low-field MRI: how low can we go? A fresh view on an old debate. Front Phys 2020;8:172
-
-
Dong C, Loy CC, He K, Tang X. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T, editors. Computer Vision -- ECCV 2014 13th European Conference; 2014 Sep 6-12; Zurich, Switzerland. Cham: Springer; 2014. pp. 184-199.
-
-
Goodfellow IJ, Pouget-Abadie J, Mirza M, et al. In: Ghahramani Z, Welling M, Cortes C, Lawrence N, Weinberger KQ, editors. 28th International Conference on Neural Information Processing Systems; 2014 Dec 8-13; Montréal, Canada. MIT Press; 2014. pp. 2672-2680.
-