Skeletal age evaluation using hand X-rays to determine growth problems

Image processing

The majority of conventional automated systems use image processing techniques or manual design using automatic ways to segment the ROIs. Several image preprocessing techniques were used by the researcher (Pietka et al., 2001) to generate ROIs from the tibia and metaphysis and discover any relationships between skeletal bone age and features. After two years, another study combined the bone age evaluation and clinical PACS and offered a new bone age assessment method (Pietka et al., 2003). In 2007, the researcher determined the bone age for each of the four categories of X-ray images based on features of ROIs of seven hand bones. Both studies designed regression models based on automated carpal bone segmentation (Hsieh et al., 2007). The study provided a computerized bone age estimation technique based on geometric features of carpal bones  (Zhang, Tang & Li, 2007). The “BoneXpert” methodology, which draws on both the G&P and TW approaches, was introduced by a study in 2009 (Thodberg et al., 2009).

Giordano designed an automatic bone age assessment system using the TW2 technique in 2010 in Giordano et al. (2010) and Giordano, Kavasidis & Spampinato (2016). A bone age evaluation model using a support vector machine (SVM) classifier was proposed by the researcher (Kashif et al., 2016) after examining the impacts of five image processing approaches for extracting features. In addition, Seok et al. (2016) picked 17 ROIs of the hand bones to construct five groups based on anatomical similarities and designed a bone age evaluation technique in 2016 using the weight of associated groups.

A researcher proposed a bone extraction method that used trigonometric concepts with knowledge of the anatomy of the hand and obtained TW2 staging by combining bone segmentation with Gaussian filtering augmentation (Giordano et al., 2007). Epiphyseal or metaphyseal ROI identification was suggested in the study from which the graphical model retrieved information (Pietka et al., 2001). The digital hand’s atlas was made with 1,400 X-ray images for evenly distributed normal youngsters and produced an automated method for calculating bone age (Gertych et al., 2007). Cao et al. (2000) proposed a method based on the web for estimating bone age that includes an atlas created from a huge database of healthy images from different racial groups.

Knowledge-based approaches

Decision rules or ML techniques are the main foundations of knowledge-based methodologies. Mahmoodi et al. (1997) showed how to segment the bones of a child’s hand radiological image and gauge the developmental process using decision-theory-based methods. Aja-Fernández et al. (2004) developed a fuzzy technique for transforming the plain language description of the TW3 method into an artificial model for bone age estimation. To mechanically reconstruct 15 bones’ borders and examine the bone age from 13 bones, Thodberg et al. (2009) suggested utilizing BoneXpert. A technique developed by researchers allowed for the automated location of phalanx bones and the derivation of various shapes using segmentation of bones (Mahmoodi et al., 2000).

Deep learning models

Modern methods for image classification, segmentation, object identification, and many other areas, such as medicine and genomics, have been greatly improved by DL (LeCun, Bengio & Hinton, 2015). DL algorithms have also been utilized for several clinical applications such as the categorization of skin cancer (Andre et al., 2019), detection of diabetic retinopathy from fundus images (Kermany et al., 2018; Gulshan et al., 2016), and the evaluation of lung nodules from CT scans (Setio et al., 2016). DL is a suitable application for skeletal maturity or bone age determination through automated radiograph processing. The age estimation method which involves the comparison of one or more radiological images with a reference standard is being used by researchers for many years. DL methods for bone age determination have already demonstrated extraordinary effectiveness in clinical settings, with accuracy on par with that of human experts (Larson et al., 2018; Kim, 2017; Spampinato et al., 2017; Lee et al., 2017; Mutasa et al., 2018; Wittschieber et al., 2013). State-of-the-art methods like VGG-16 (Simonyan & Zisserman, 2015), Inception-V3 (Szegedy et al., 2016), VGG-19 (Simonyan & Zisserman, 2015), Inception-ResNet-V2 (Szegedy et al., 2017), and Xception (Chollet, 2017), employing deep CNN models were proposed by several researchers.

The DL strategy instead aims to directly encode visual properties (Goodfellow, Bengio & Courville, 2016). Numerous DL techniques have been proposed and tested by the authors for automatically identifying the age of skeletal bones (Spampinato et al., 2017). The results showed that there was an average difference of 9.6 months between the results and the expert interpretation. A method that made use of several DL designs, such as U-Net (Ronneberger, Fischer & Brox, 2015), ResNet-50 (He et al., 2016), and VGG-style NN (Simonyan & Zisserman, 2015), superior to other standard methods (Iglovikov et al., 2018). AlexNet, VGG16, and GoogLeNet as transfer learning models have all been used in the ImageNet Large Scale Visual Recognition Competition and received acknowledgment. In terms of accuracy, VGG16 is the most accurate of the three, whereas AlexNet is the least accurate (Canziani, Alfredo & Culurciello, 2016). A study (Tang, Chan & Chan, 2019) used the RSNA dataset and employed the transfer learning strategy to train their TjNet architecture created using CNNs. The study focuses on calculating the bone age from the carpal areas utilizing features rather than on the characteristics of the finger (De Luca et al., 2016). On the other hand, methods in Tang, Chan & Chan (2019) and Pahuja & Kumar Garg (2018) used an ANN regressor to determine bone age.

Artificial neural networks and SVM regressors are the most often used conventional machine learning techniques to predict bone age (Dallora et al., 2019). The authors employed MRI rather than X-ray images in this study, in contrast to earlier ones (Tang, Chan & Chan, 2019). The study has also investigated three other characteristics: texture data, number of features, and a histogram of gradients. These features were all extracted from the phalanges regions of bones and used with the Random Forest predictor (Simu & Lal, 2018). Spampinato et al. (2017) developed the first automated DL system utilizing BoNet architecture. A simple CNN architecture with five convolutional layers is used in the model. The designed BoNet outperformed other contemporary DL models including OverFeat, GoogLeNet, and OxfordNet. Similar to this, Lee et al. (2017) discovered that GoogLeNet was the most effective over various CNN architectures at determining bone age. Additionally, to enhance the model’s accuracy, they used pre-processing steps as opposed to BoNet, which does not employ any pre-processing steps, to normalize the input images and eliminate extraneous noise. A high number of images are typically required when training a DL model to prevent underfitting and overfitting problems.

Wibisono & Mursanto (2020) developed a comparable ROIs division in support of their claim that getting particular attributes enhances the regressor’s ability to forecast bone age. Then, DenseNet-121, Inception-V3, and InceptionResnet-V2 were used to train each of these areas, and the training data was passed into a Random Forest regressor. Another study developed an automated method that concentrates on the index finger’s X-ray images rather than the entire hand for bone age prediction (Reddy et al., 2020; Manzoor et al., 2021). Despite having a small test dataset, their method was able to determine the age of the bones from just one index finger. CNNs were also used to develop an end-to-end technique for forecasting bone age (Marouf et al., 2020). The authors took advantage of CNNs’ capacity to calculate bone age based on gender information, even though the gender categorization used was less accurate than ideal (79.60 percent).

Overview

This study uses hand X-ray image collection for the diagnosis of growth problems as well as a variety of other ailments, including bone cancer. At first, data is preprocessed by applying image resizing and image augmentation. The data is split into train and test sets in an 80:20 ratio. A customized CNN model is proposed for skeletal growth disorder. The accuracy of the proposed mechanism is then compared to that of the existing approaches. A comparison of the current and suggested approaches is performed to assess the proposed strategy. Figure 1 depicts the proposed methodology for this study.

Dataset description

An X-ray of a child’s hand was used in a competition to determine the age at the 2017 RSNA. The RSNA Bone Age database (Mader, 2017) is publicly accessible on Kaggle, making it simple to use. This dataset was initially made available as a CloudApp RSNA challenge. The dataset comprises numerous digitally scanned photo files and a CSV with the gender and age of each participant. The datasets used in the Pediatric Bone Age Challenge were contributed by Stanford University, the University of Colorado, and the University of California–Los Angeles. The dataset is comprised of age-related X-ray images of males and females.

Evaluation parameters

Several parameters are used in this study to assess the performance of machine learning models. These assessment measures include accuracy, recall, F1 score, and precision. Using the values from the confusion matrix, we calculate these performance metrics. The four values in the confusion matrix are true positive (TP), false positive (FP), true negative (TN), and false negative (FN).

The percentage of cases that were accurately estimated is known as accuracy. The accuracy score is computed by dividing the number of all predictions by the overall quantity of correct assumptions. The accuracy score has a scale from 0 to 1, with 0 denoting the lowest accuracy and 1 denoting the highest accuracy. The accuracy score is determined as follows (1)${\displaystyle Accuracy=\frac{TP+TN}{TP+TN+FP+FN}}$

The accuracy of models in use is measured using precision. Precision is computed by dividing the amount of TP by the sum of TP and FP. (2)${\displaystyle Precision=\frac{TP}{TP+FP}}$

The recall is evaluated by dividing the TP by the total of the TP and FN. The recall score is a numeric value between 0 and 1, with 1 being the greatest possible score. (3)${\displaystyle Recall=\frac{TP}{TP+FN}}$

The F1 measure sometimes referred to as the F1 score, is a harmonic mean of recall and accuracy. An F1 score is a number between 0 and 1, where 0 is the lowest and 1 is the greatest. Additionally, the F1 score demonstrates accuracy and recall balance. (4)${\displaystyle F1score=2\ast \frac{Precsion\ast Recall}{Precision+Recall}}$

Results of customized CNN

Table 3 presents the results of customized CNN. To get the best results, customized CNN is applied using several epochs. Different epochs 10, 20, 30, 40, 50, 60, and 70 were used to analyze results and with 50 epochs CNN attained the maximum accuracy, precision, and F1 score of 97%. Customized CNN, on the other hand, had the highest recall, likewise with 50 epochs, at 96%. Following 97%, the model attained the second-highest accuracy of 96% from 40 epochs and the highest precision of 95% from the same 40 epochs. However, the 95% recall is obtained from 60 epochs and the 95% F1 score is obtained from 70 epochs.

Figure 2 uses a bar graph to compare the achieved outcomes for all epochs using customized CNN. From all epochs, customized CNN obtained the best results from 50 epochs. For example, customized CNN obtained 97% accuracy, 97% precision, and 97% F1 score from 50 epochs.

Experimental results with VGG16

Table 4 contains results using VGG. To get the best results, different epochs are employed with VGG; the best results are obtained using 50 epochs. The VGG model attained the highest accuracy and precision of 96%. With a 97% F1 score, VGG achieved the best F1 score among the other assessment criteria with 50 epochs while simultaneously achieving the highest 95% recall.

Figure 3 uses a bar graph to compare the VGG outcomes for all epochs. VGG outperformed all other epochs with results from 50 epochs, including 96% accuracy, 96% precision, 95% recall, and 97% F1-score. This demonstrates that VGG works optimally with a 97% F1 score obtained from 50 epochs.

Comparison between customized CNN and VGG16

The results of the two models are compared in this section. Table 5 compares the performance of CNN with VGG based on accuracy, precision, recall, and F1 score. Results show that with a 97% accuracy, customized CNN outperforms the VGG model.

A line graph in Fig. 4 compares CNN’s prediction of age with the actual values. The line graph demonstrates that the model performed well.

Comparison with state-of-the-art approaches from literature

For many years, computer vision and radiological studies have been working towards fully automated bone age assessment. The majority of earlier methods included classification or regression utilizing manually extracted features that were taken from segmented areas of interest (ROIs) for certain bones. Table 6 compares our technique to four previous efforts from the literature. In order to extract visual descriptors and produce fixed-size feature vectors that could be fed into a fully connected neural network, Seok et al. (2012) used a Scale-invariant feature transform and singular value decomposition. Due to the limited amount of images they utilized, their model was not efficient for images that were completely different from their actual dataset. Somkantha, Theera-Umpon & Auephanwiriyakul (2011) determined the carpal bone area utilizing projections in the vertical and horizontal axes. Using the segmented carpal bones, they derived five morphological characteristics and used them in SVM. This technique is similar to the approach of Zhang et al. (Cao et al., 2000) in that hand-engineered features retrieved from carpal bones were utilized as input for a fuzzy logic classifier. The carpal bones are normally fully developed by ages 5 to 7 and do not allow for meaningful distinction after that age, therefore this method is not appropriate for younger children. BoneXpert (Zhang, Gertych & Liu, 2007), the first commercially available and only software-based medical device certified for use in Europe, is the most successful effort to date. In order to autonomously segment bones and to identify the GP or TW2 bone age based on form, intensity, and textural aspects, BoneXpert uses a generative model called the active appearance model. The forecast made by BoneXpert is dependent on the link between chronological and bone ages, therefore it cannot directly determine bone age (Seok et al., 2012). Because of its fragility, the system will not detect radiographs when there is too much noise. According to earlier research, out of 5,161 unique bones, BoneXpert did not detect around 235 of them Thodberg et al. (2008). In spite of the carpal bones’ discriminatory characteristics for young children, BoneXpert does not use them.

In conclusion, the majority of previous attempts at automated bone age estimation have relied on hand-crafted characteristics, which limits the algorithms’ capacity to generalize to the intended application. Our method uses a customized deep CNN to automatically extract key characteristics from all bones on an ROI that was automatically segregated by a detection CNN. Abbreviations used in the whole article are presented in Table 7.