Development of a multi-fusion convolutional neural network (MF-CNN) for enhanced gastrointestinal disease diagnosis in endoscopy image analysis

Data collection

The study utilized publicly accessible datasets from reputable repositories, including KVASIR (Pogorelov et al., 2017) and ETIS-Larib Polyp DB (Silva et al., 2014). Due to the complexities associated with obtaining endoscopic data related to gastrointestinal disorders, these datasets proved invaluable. The KVASIR dataset was derived using the wireless capsule endoscopy (WCE) technique from a diverse population of gastrointestinal patients at the Vestre Viken Health Trust in Norway. Upon comprehensive review and data acquisition, medical specialists ensured each image within the dataset was correctly labeled, making it suitable for most deep-learning studies. Similarly, the verification process was applied to the ETIS-Larib Polyp DB dataset. In this inquiry, a conscious decision was made to utilize a stochastic approach in order to choose and organize the datasets carefully. This was done to minimize the possibility of data leakage and any potential biases. Figure 3 of the dataset showcases samples from each category.

Comprehensive data overview

In this study, we employed a meticulously prepared dataset comprising a total of 6,000 images. This dataset was strategically divided into distinct subsets to facilitate a systematic approach in model training and evaluation: training, validation, and testing. The distribution was set as 3,200 images for training, 2,000 for validation, and 800 for testing. Such a structured segmentation assures a proportionate exposure of the model to varied data while also allowing for a comprehensive evaluation of its performance. One of the primary concerns in machine learning, particularly in image classification tasks, is the challenge posed by data instability and class dominance. Our methodology adeptly addresses and minimizes these possible drawbacks by assuring an even distribution of images across various subsets. For a detailed dissection of the dataset, including class categorization, quantity, and specific distribution, refer to Table 1. A noteworthy characteristic of the images within the dataset is the variance in their dimensions, attributable to their diverse sources of origin. Recognizing the need for uniformity in input data for deep learning models, we standardized the training images to a consistent dimension of 224 × 224 pixels. This was accomplished using an automatic image data generator available within the Keras framework (Chollet, 2015). Such a standardization facilitates the computational process and ensures efficient resource allocation. By adopting this technique, we optimized the training speed while concurrently preventing computational memory overload, ensuring smooth and efficient experimentation.

Data preprocessing

Image enhancement (adaptive histogram equalization)

In our study, we recognized the potential benefits of enhancing the clarity and contrast of the images to obtain more precise results. To this aim, we employed the adaptive histogram equalization (AHE) technique (Pizer et al., 1987), applying specific parameters of ‘clip_limit = 2.0’ and ‘tile_grid_size = (8, 8)’ to adjust the process to our needs. AHE is renowned for its capacity to amplify the local contrast of an image, particularly in areas that are closer in color or intensity. The fundamental principle of AHE involves performing histogram equalization within small, contextual regions or tiles of the image rather than the entire image. For each tile, defined by the ‘tile_grid_size’ parameter as 8 × 8 blocks, the histogram of pixel intensities denoted as $H\left(i\right)$, is calculated for each tile. It provides the frequency at which each intensity level $i$ arises.

(2) $$H\left(i\right)=numberofpixelswithintensityi.$$

Following this, the cumulative distribution function (CDF) is computed for each histogram using the formula $C\left(i\right)$,

(3) $$C\left(i\right)={\sum }_{j=0}^{i}H\left(j\right).$$

In this equation, $j$ is an index runs from 0 to $i$, summing the histogram values up to the current intensity level $i$, to calculate the cumulate frequency. After calculating the CDF, it is normalized with the ‘clip_limit’ parameter set to 2.0. This normalization process is represented as $C_{norm}\left(i\right)$, which adjusts the intensity values within each tile to enhance local contrast,

(4) $$C_{norm}\left(i\right)=NormalizedValueofC\left(i\right).$$

The notation $C_{norm}\left(i\right)$ refers to the value of the normalized CDF for the intensity level $i$. Applying AHE to our dataset with these parameters improved the images by increasing the visibility of identifiable features. This improvement facilitated a more comprehensive representation and set an environment for our model to effectively detect and learn from complex, intricate patterns in the image. The effectiveness of AHE in the study is demonstrated in Fig. 4, which clearly displays the enhancement of contrast in local regions of the images. The histograms display the distribution of pixel intensities across the RGB color channels for both the original and AHE-enhanced images, providing a visual representation of the contrast adjustments. This processing phase ensured that our model was trained on data that accurately reflected the depth and clarity of the original endoscopic images, increasing the probability of accurate diagnosis. Algorithm 1 illustrates the entire AHE procedure.

Algorithm 1:

Adaptive histogram equalization (AHE) for image enhancement

1:	procedure AdaptiveHistEqualize (Image $I$, ClipLimit $C$, TileGridSize $T$)
2:	$I_{YCbCr}$ ← Convert $I$ from RGB to YCbCr color space
3:	Extract $Y$ channel as $I_Y$
4:	Divide $I_Y$ into non-overlapping tiles of size $T$
5:	for each tile in $I_Y$ do
6:	Initialize histogram array $H\left[0\dots \mathrm{m}\mathrm{a}\mathrm{x}\mathrm{\_}intensity\right]$ to zeros
7:	Initialize CDF array $CDF\left[0\dots \mathrm{m}\mathrm{a}\mathrm{x}\mathrm{\_}intensity\right]$ to zeros
8:	for each pixel $p$ in the tile do
9:	$H\left[intensityofp\right]$ ← $H\left[intensityofp\right]$ + 1
10:	end for
11:	$CDF\left[0\right]$ ← $H\left[0\right]$
12:	for $1to\mathrm{m}\mathrm{a}\mathrm{x}\mathrm{\_}intensity$ do
13:	$CDF\left[i\right]$ ← $CDF\left[i-1\right]$ + H $\left[i\right]$
14:	end for
15:	Normalize CDF using $C$
16:	for each pixel $p$ in the tile do
17:	$p_{newintensity}$ ← $normalizedCDF\left[intensityofp\right]$
18:	end for
19:	end for
20:	Apply bilinear interpolation between adjacent tiles in $I_Y$
21:	Replace $Y$ channel in $I_{YCbCr}$ with the equalized $I_Y$
22:	Convert $I_{YCbCr}$ back to RGB color space
23:	return Enhanced color image $I$
24:	end procedure

DOI: 10.7717/peerj-cs.1950/table-40

Development approach

This research explored an extensive variety of deep convolutional neural networks (DCNNs) for their efficacy in medical image analysis. Here, we present a systematic review and integration of various models, leading to the development of the proposed MF-CNN for diagnosing gastrointestinal (GI) conditions.

Selection and evaluation of DCNN models

Convolutional neural networks (CNNs), a form of artificial neural network, play a crucial role in deep learning, particularly when analyzing visual data (Hossain et al., 2022). These networks consist of many connected layers of artificial neurons. Their design focuses on feature extraction and classification, with a composition of an input layer, convolutional layer, pooling layer, fully connected layer, hidden layer, and activation function. The crucial aspect of this study included utilizing the capabilities of DCNNs, known for their ability to analyze medical images. The research used well-established models, including EfficientNetB0 (Wu et al., 2020; Nigam et al., 2021), MobileNetV2 (Sandler et al., 2018; Buiu, Dănăilă & Răduţă, 2020), ResNet50V2 (Santos-Bustos, Nguyen & Espitia, 2022; Praveen et al., 2022), DenseNet121 (Li et al., 2020), VGG16 (Great Learning, 2022), and Xception (Chollet, 2016), as a component of our technique.

• 1)

  EfficientNetB0: Originating from Google, this model is acclaimed for its exceptional scalability and adeptly accommodating various image dimensions. The MBConv block, enhanced with a squeeze-and-excitation component, functions as an inverted residual block, streamlining performance without amplifying the number of parameters. Such efficiencies render it an optimal selection for applications in medical imaging (Buiu, Dănăilă & Răduţă, 2020; Santos-Bustos, Nguyen & Espitia, 2022; Praveen et al., 2022).

• 2)

  MobileNetV2: The architecture of MobileNet is based on using a pointwise convolution approach. This approach speeds up the procedure while lowering computing requirements. Its improved version, V2, streamlines the design further, enhancing efficiency (Great Learning, 2022; Chollet, 2016).

• 3)

  ResNet50V2: ResNet50V2 incorporates a revised inter-block deep residual network structure designed to flow information between blocks seamlessly for improved model accuracy. The main idea behind residual blocks, referred to as “skip connections,” emphasizes the resilience and effectiveness of the CNN architecture (Das, Santosh & Pal, 2020; He et al., 2016; Akter et al., 2021).

• 4)

  DenseNet121: The DenseNet-121 model has 121 layers and is part of the DenseNet series. Its classification subnetwork includes the 7 × 7 global average pooling and the 1000D fully connected layer (Akter et al., 2021).

• 5)

  VGG16: VGG16 is a deep convolutional neural network model introduced by the Visual Geometry Group (VGG) from the University of Oxford. It comprises 16 layers, including 13 convolutional layers and three fully connected layers. VGG16 is known for its simplicity, utilizing only 3 × 3 convolutional layers stacked on top of each other in increasing depth, and has proved to be extremely effective in image recognition tasks (Minaee et al., 2020).

• 6)

  Xception: Xception was designed to outperform other architectures by exploiting depthwise separable convolutions, which replace standard convolutions. The primary insight behind Xception is that the cross-channel and spatial correlations in the feature maps of convolutional neural networks can be mapped separately, leading to improved performance and efficiency. It has been notably successful in various image classification tasks (Chollet, 2016).

The selection of EfficientNetB0, MobileNetV2, ResNet50V2, DenseNet121, VGG16, and Xception for the proposed Multi-Fusion Convolutional Neural Network (MF-CNN) model was based on a systematic strategy that highlighted the architectural variety and showed effectiveness in image processing tasks. The selection of each model and the components (AuxFL, αDO, and FuRB) was based on its distinct advantages:

• EfficientNetB0 was selected due to its optimal balance of accuracy and efficiency. Moreover, its scalable architecture enables flexible model sizing, rendering it a highly suitable candidate for the wide variety of image resolutions that are commonly encountered in endoscopy data.

• MobileNetV2 was selected based on its lightweight architecture and exceptional efficiency, which are essential for real-time analytic applications. The depthwise separable convolutions provide an ideal balance between computational workload and prediction accuracy.

• ResNet50V2 addresses the issue of vanishing gradient, enabling the training of far deeper neural networks. The MF-CNN incorporates it to guarantee strong feature extraction skills, which are crucial for capturing the intricate patterns that are characteristic of GI diseases.

• DenseNet121 was chosen based on its densely linked convolutional networks, which enable efficient feature reuse across the network. This architecture is particularly effective for learning complex attributes from endoscopic images. Its inclusion enhances the model’s sensitivity to subtle features indicative of GI diseases.

• VGG16 offers very reliable feature extraction capabilities. The consecutive convolutional layers of the model effectively capture a diverse range of image features, including basic textures and complex patterns. This contributes to the MF-CNN’s capacity to do thorough and comprehensive analysis.

• Xception model incorporates depthwise separable convolutions, which enhance computing efficiency without compromising accuracy. The layout of the system is especially advantageous for handling the diverse and sophisticated images seen in GI diagnostic procedures, providing an optimal combination of performance and resource efficiency.

• Auxiliary Fusing Layers (AuxFL), Fusion Residual Block (FuRB), and Alpha Dropouts (αDO) are specifically developed to improve the integration process, enhance regularization, and facilitate effective feature fusion. They handle prevalent issues such as overfitting and the optimal combination of features from multiple sources, which are essential for attaining high diagnostic accuracy.

The diversified combination ensures an extensive selection of learning features, which include efficient computation for precise recognition of patterns. A balanced combination of efficiency, accuracy, and complexity led to choosing these methods to ensure that the fusion is robust and effective across various applications. This strategic selection is in contrast to selecting alternative deep learning models, which might not provide the same range of complimentary features and demonstrated performance in various scenarios, a critical consideration in aiming for a model that is both high-performing and broadly applicable in real-world scenarios.

Strategy for fusion model and tuning

The models outlined previously in the paper have individually exhibited daunting performance metrics in the domain of medical image diagnosis. In our study, we delved into the model fusion domain to extract even more complex and subtle characteristics from the data, capitalizing on the distinct efficiency of each model.

The underlying principle of the proposed MF-CNN is simple yet powerful, as it exploits the individual strengths of each model, creating a more all-inclusive and resilient diagnostic tool. Integrating multiple CNN models is an approach to incorporate knowledge obtained from diverse domains. The implied collaborative approach has the potential to unveil a wider array of features from medical imaging, potentially increasing prediction accuracy. The integration of each chosen model contributes systematically to the extensive framework of the MF-CNN, specifically designed for GI diagnosis. It is important to emphasize that our proposed model excels in mere integration; it involves careful consideration and circumspect modifications, such as the inclusion of previously recognized FuRB, Partial Layer Freezing and AuxFL. These incorporations are consciously executed to ensure that the final MF-CNN achieves optimal performance. However, the process of combining several models with varied architectures and approaches is not devoid of difficulties. A significant concern emerges when a model becomes overly familiar with the training data too well, including its noise and outliers, making it perform less efficiently on unseen data. The vulnerability becomes more prominent when combining models, especially on limited datasets. In their endeavor to capture every minute detail, the models might just “ memorize” the training data, undermining their generalizability to new, unseen data (Olson, Wyner & Berk, 2018).

Recognizing this potential drawback, our study incorporated a strategic countermeasure. We integrated the residual learning strategy, a signature methodology from the ResNet model, into our fusion approach. This gave origin to the FuRB. Residual learning primarily involves using “shortcut” or “skip” connections that bypass one or more layers. This approach helps mitigate the vanishing gradient issue, ensuring that the model can still learn well even when more layers are added, particularly in the context of our fusion. Figure 5 depicts the complicated fusing process in a visual manner. It shows how each selected model contributes to the overall framework of the MF-CNN built particularly for gastrointestinal (GI) diagnosis step by step. The illustration highlights the integration of several models and the meticulous planning and thoughtful modifications, such as the addition of FuRB, that ensure the final MF-CNN achieves optimal performance while avoiding risks such as overfitting.

The initial phase used transfer learning to empower models with ImageNet’s pre-trained features, increasing diagnostic precision (Orsic et al., 2019). Following that, model layer truncation was performed to reduce the number of trainable parameters. The layers are put into a ‘freezing’ process during training to confirm the integrity of these enhanced pre-trained weights. The transformation of these fine-tuned models to AuxFLs assured consistent output dimension. Furthermore, these improved models were aggregated with the FuRB, resulting in an undeviating feature fusion suitable for diagnostic applications. Our research, which combines these insights and models, is a testimony to medical imaging improvements, placing the MF-CNN as a potentially transformational tool in GI diagnosis.

Truncation of freezing layers

A systematic refinement of each designated model was undertaken in the construction of the MF-CNN. Initially, each model’s upper section (often referred to as the ‘head’) was detached. A strategic pruning of certain layers followed this, while a substantial number of the remaining layers were suspended during the training process. The reason for these measures was to mitigate the risk of potential parameter inflation that might occur post-fusion. An excessively complex model, particularly when trained on limited datasets, is susceptible to overfitting, compromising its predictive ability (Das, Santosh & Pal, 2020; He et al., 2016).

Table 2 provides a detailed analysis of each model’s parameters—both in their original and truncated states—along with the preserved features, training status, and the specific layer at which truncation occurred. It’s worth noting that the original architectures of these models were designed with the intent to classify up to 1,000 distinct objects. However, the scope of this analysis was narrowed down to identifying four specific categories of GI abnormalities. This disparity in classification objectives made truncation an essential step. It became more streamlined and efficient by paring down the model’s architecture. An empirical approach was adopted to determine the optimal cut points, ensuring that an extensive set of features was retained despite the pruning.

Harmonizing fusion with auxiliary fusing layers

The truncation technique introduced a challenge of mismatched cut-point dimensions among the selected models. As a result, direct integration of these models became problematic. To overcome this challenge and attain effective fusion, AuxFLs were accumulated to ensure dimensional compatibility, providing a controlled number of trainable parameters for the fused model.

Figure 6 depicts the overall construction of the AuxFLs, which are distinguished by their simplified yet strong layout. Each AuxFL is made up of three layers that are arranged sequentially: a convolutional layer (Conv), an average pooling layer (AP), and a dropout layer (αDO). Each layer has a distinct purpose. The Conv layer, which employs a h × w convolutional filter, may extract and highlight additional information. This replaces any parameters that may have been lost during truncation and guarantees that the parameter count does not increase excessively. The AP layer then enters the image and begins pooling the retrieved features to restructure and standardize their proportions, making them suitable for fusion. The decision to use average pooling rather than maximum pooling was intended since it affirms that the resulting feature maps are more representative of the underlying data, allowing for a more unified fusion. Finally, the αDO layer is incorporated for an improved regularization.

Table 3 itemizes the specific configurations and specifications of each AuxFL tailored to the requirements of the truncated variants. This detail ensures that every model’s unique features are preserved and harmonized before the fusion process.

Optimized feature fusion and FuRB implementation

AuxFLs integration ensured that all models achieved suitable cut-points, paving the road for effective feature fusion (Salau & Jain, 2019; Dhiman et al., 2023; Liu et al., 2023; Montalbo, 2023). On the other hand, the inherent complexity produced by this fusion triggered worries about excessive adherence to training data. This study employed a recognized integrated FuRB solution to address this concern. This FuRB, inspired by the ResNetV2 block’s “full pre-activation” approach, was upgraded with a dropout layer (αDO), assuring optimized performance without increasing computing costs. Significantly, adding the FuRB allowed the model to avoid performance saturation, improving overall efficacy. Figure 7 depicts the FuRB as well as the extensive MF-CNN framework in detail.

MF-CNN hyperparameter optimization

The MF-CNN’s hyperparameters and loss function were specified before beginning the training process. Table 4 shows the hyperparameters that were employed during training. While pre-trained networks, which are frequently trained on large datasets, excel at extracting hierarchical features, they are especially useful for smaller datasets. The majority of the models mentioned before are included in Keras. Keras-tune was first used for model fine-tuning and hyperparameter optimization, employing grid search-a prominent parameter-tuning approach. The following were the preliminary choices:

• Size of batch: 16, 24, 32, 64, 100

• Count of epochs: 30, 50, 100, 150, 250, 300

• Rate of learning: 0.0000001 to 0.1

• Optimizers: ‘SGD’, ‘RMSprop’, ‘Adadelta’, ‘Nadam’, ‘Adam’, ‘Adamax’

The grid search approach was used to determine optimal parameter values for deep learning models. All the previously mentioned optimizers were tried for the unique MF-CNN technique. To avoid memory difficulties, the batch size for this study was fixed at 32, which was changed based on dataset size and system parameters. For its efficiency, the Adam optimizer (Akter et al., 2021) with a learning rate (LR) of 0.0001 was used. The model utilized Adam’s adaptive capabilities to attain optimal performance in just 100 epochs, even with a low learning rate.

MF-CNN hyperparameter and learning rate optimization

This study employed the ReduceLROnPlateau callback to ensure optimal performance without surpassing optimum accuracy levels. This is particularly beneficial if a model’s LR becomes too aggressive during training. If there is no improvement in accuracy over two consecutive epochs, the ReduceLROnPlateau function automatically reduces the LR by 0.5 (50%), updating it with the new LR as per Eq. (5).

(5) $$LRnew=LR\times 0.5.$$

A deep learning (DL) model’s performance is measured not only by its accuracy but also by its error rates. The Categorical Crossentropy Loss function, as shown in Eq. (6), was used in this study. This function minimizes differences between expected and actual classes, providing a loss measure. It is especially useful in circumstances with several classes since, unlike its binary version, it delivers a probability for simply the relevant classes (GitHub, 2020).

(6) $$CC{E}_{loss}=-\sum _{c=1}^m{Y}_{o,c}\log \left(P_{o,c}\right).$$

Ablation analysis

A commonly employed technique for fine-tuning parameters is the implementation of the grid search method. The aforementioned approach is utilized to determine parameters including the optimizer, learning rate, loss function, batch size, and dropout. Furthermore, an ablation study was performed to validate the robustness of the proposed MFCNN model. The ablation study has impacted the following elements: model combination, layers, optimizer, learning rate, and adaptive histogram equalization (AHE).

Experimental setup

To develop and test our MF-CNN, we used a computing system with an AMD Ryzen 7 3800X CPU, 32 GB of RAM, and an NVIDIA GeForce RTX 2080 Ti GPU. The hardware arrangement was selected for its superior computing power and efficiency in handling extensive datasets and performing complex deep-learning computations. The use of the NVIDIA GeForce RTX 2080 Ti GPU, loaded with its superior CUDA cores and huge memory bandwidth, facilitated the efficient training of our model, resulting in a reduction in the time needed for both model training and assessment. This configuration represents a widely accessible yet effective computing environment that achieves a balance between accessibility and performance, thereby assuring the replication and applicability of our research across diverse contexts.

Cost and complexity analysis

Table 9 thoroughly analyzes various deep learning models compared to the proposed optimized MF-CNN, highlighting significant factors, including inference efficiency, computational demands, and model complexity.

The MF-CNN, with 150 × 10⁶ parameters and 20 × 10⁹ floating point operations (FLOPs), stands out as one of the most complex and computationally intensive models in the comparison. The network’s extensive parameter count signifies its fundamental and intricate nature, enabling it to capture a wide array of critical features for achieving optimal performance in complex tasks. The MFCNN needs a significant number of computations for each inference, a requirement that not only ensures thorough data processing but also requires considerable computational resources. This value is much higher than that of models such as MobileNetV2 and EfficientNetB0, renowned for their minimal computational demands and high efficiency. The MFCNN’s complexity is described as an “optimized fusion of models with reduced layers and efficient architectures,” indicating that a conscious decision was made during design to maintain a balance between the practical efficiency of the fusion model and its inherent complexity. The optimization above plays a critical role in effectively handling the computational cost and model’s size (580 MB), rendering it a robust yet resource-intensive solution.

The MFCNN has a greater size and more FLOPs than other models, such as DenseNet121, VGG16, and Xception, indicating a higher resource consumption. Still, part of what makes it function so well is precisely this intricacy. While this may be more than for more lightweight models, the MF-CNN’s projected inference time of 20–80 ms per image shows its broad processing capabilities, which are essential for attaining cutting-edge outcomes in demanding applications. Although the proposed MFCNN model demands the most resources among the models, its improved performance capabilities justify its demands. This solution maintains a balance between intensive computational requirements and sophisticated features, rendering it well-suited for situations in which optimal performance is essential and adequate computational resources are accessible.

Evaluation of diagnostic performance metrics

The conventional methods depend primarily on the anticipated elements of the confusion matrix, i.e., true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN), derived from the validation and test datasets. These components are used to evaluate the overall diagnostic accuracy of vision-based models in the field of medical imaging. The suggested model’s overall accuracy, precision, recall, and F1-score are evaluated using the following Eqs. (7) to (10) based on the values of the elements of the confusion matrix (Minaee et al., 2020).

(7) $$Accuracy={\displaystyle \frac{TP+TN}{TP+TN+FP+FN}}$$

(8) $$Precision={\displaystyle \frac{TP}{TP+FP}}$$

(9) $$Recall/Sesitivity={\displaystyle \frac{TP}{TP+TN}}$$

(10) $$f_1-\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}={\displaystyle \frac{2\times Precision\times Recall}{Precision+Recall}}.$$

Diagnostic performance analysis

The dataset in this study was divided into training, validation, and test sets, allowing for a more detailed performance comparison with other cutting-edge approaches. The confusion matrices for both the validation and test datasets are shown in Figs. 8 and 9, demonstrating the predictive accuracy of the proposed MF-CNN model. As shown in Fig. 8, the model made just 28 mistakes out of 2,000 samples in the validation set, attaining a remarkable accuracy rate of 98.6% and an error rate of 1.4%. Similarly, in Fig. 9, the model misclassified only six of the 800 samples in the test set, resulting in an accuracy of 99.25% and an error rate of 0.75%.

The findings in Table 10 indicate the performance measures of the MF-CNN model, such as accuracy, precision, recall, and F1-score, as determined from the three-way split approach. These values were obtained by analyzing the data in the associated confusion matrix. The MF-CNN model attained an impressive accuracy of 98.60% on the validation set and 99.25% on the test set. Furthermore, the model displayed exceptional consistency across additional criteria. Precision, recall, and F1-score for the validation set were 98.65%, 98.6%, and 98.6%, respectively. In comparison, the test set reported precision, recall, and F1-score values of 99.27%, 99.25%, and 99.25%, respectively.

Tables 11 and 12 provide an in-depth evaluation of the model’s diagnostic accuracy for each distinct instance. The MF-CNN’s proficiency was slightly lower when diagnosing ulcer and polyp diseases compared to the normal and esophagitis situations. As shown in Table 11, among the validation data, polyp diagnosis had the lowest accuracy at 94.60%, whereas ulcer diagnoses had a higher rate of 99.80%. When the model’s accuracy for the polyps diagnosis was evaluated using the test data from Table 12, it was recorded at 97.00%. Such findings imply that the MF-CNN’s capacity to distinguish polyp cases is considerably limited when compared to other situations. Despite this minor variance, the MF-CNN provided an excellent overall diagnostic performance.

ROC curve analysis

The receiver operating characteristics (ROC) curve was used by the MF-CNN model to explore its diagnostic capabilities more deeply, analyzing sensitivity and specificity across multiple thresholds (Nour & Polat, 2020). As Eq. (11) specified, the model’s sensitivity examined its accuracy in identifying instances with mucosal abnormalities. Similarly, as defined in Eq. (12), specificity assessed the model’s accuracy in identifying situations where the mucosa showed no abnormalities.

(11) $$Sensitivity={\displaystyle \frac{Instancesofcorrectlydiagnosedsampleswithabnormalities}{Totalnumberofsampleswithabnormalities}}$$

(12) $$Specificity={\displaystyle \frac{Instancesofcorrectlydiagnosedsampleswithoutabnormalities}{Totalnumberofsampleswithoutabnormalities}}.$$

Figures 10 and 11 demonstrate the ROC curve for the MF-CNN model, derived from the sensitivity and specificity metrics of the validation and test datasets. The AUC values averaged 1.00 for both the validation set represented in Fig. 10 and the test set in Fig. 11. This highlights the model’s consistent diagnostic precision across disparate thresholds. The findings confirm the model’s outstanding performance on both datasets (Jeni, Cohn & De La Torre, 2013).

PR curve analysis

Similar to the ROC curve, the Precision-Recall (P-R) curve is formulated based on the recall or sensitivity metric. This curve precisely balances precision, frequently referred to as the positive predictive value, and recall. Its ability to adjust for varying thresholds grants the P-R curve a unique edge in providing a holistic assessment of the model’s efficacy with the dataset (Pizer et al., 1987). Figures 12 and 13 vividly portray the P-R curve of the advanced MF-CNN model, distilled from insights collected from both validation and test datasets. Observations from the validation set, represented in Fig. 12, revealed a micro-average P-R curve value of 0.996. In comparison, the test set, illustrated in Fig. 13, reported a marginally superior value of 0.998. Such findings accentuate the model’s prowess in rapidly diagnosing an array of GI diseases. An essential conclusion from this analysis is the model's unwavering performance consistency. This is evident from the nearly uniform P-R curve across both datasets, suggesting minimal anomalies. Such consistent outcomes reinforce the model’s reliability, simultaneously mitigating any identified weaknesses.

Comparative analysis

Figures 14 and 15 provide a detailed visualization of the accuracy and loss metrics associated with the proposed MF-CNN model. This model’s robust performance is evident from its consistently increasing accuracy and decreasing loss as training progresses through successive epochs. Figure 14, in particular, highlights the training and validation accuracy statistics. The model’s progression indicates a consistent improvement in accuracy, reflecting its effective learning capability. On the other hand, the primary objective during the model’s training phase is to minimize the loss. This scalar measure quantifies the difference between the model’s predictions and the actual labels. A lower loss score implies more accurate predictions. This relationship is depicted in Fig. 15, which compares training loss against validation loss. Both accuracy and loss metrics are displayed against the epoch count. At the onset, during the initial epochs, the model presents its most fundamental performance, characterized by the lowest accuracy and the steepest loss. However, as training persists, both metrics exhibit significant improvement. By the 90th epoch, a significant milestone is achieved: validation accuracy maximizes, and the validation loss is at its lowest concurrently. This convergence signifies that the model’s pinnacle of performance is attained at the 90th epoch, marked by peak accuracy and minimized loss.

To assess the MF-CNN’s ability to diagnose certain gastrointestinal disorders, it is essential to compare its performance to various top DCNN models. This section compares the MF-CNN to well-known DCNN models using the same dataset. Montalbo (2022) compared their MFuRe-CNN against 11 of the best Keras pre-trained models. This work is consistent with Montalbo’s comparison, as it draws on his performance assessments of these DCNNs given the common dataset. Table 13 shows that the MF-CNN scored highest with a validation accuracy of 98.60%. In comparison, the highest-ranking DCNN, MobileNetV2, attained an accuracy of 90.90%. The MF-CNN achieves a considerable performance improvement of 7.7%. While the MF-CNN outperforms traditionally trained DCNNs, MobileNetV2 has advantages from the proposed fusion strategy. Furthermore, our MF-CNN model outperformed Montalbo’s MFuRe-CNN by 1.95%, with a validation accuracy of 96.65%. Similarly, Table 13 also shows, MF-CNN outperformed MobileNetV2 with a 99.25% accuracy rating from the test dataset, where MobileNetV2 had 92.25%. An improvement of 7% showcases the supremacy of MF-CNN on this dataset. Even if we look at the customized MFuRe-CNN model, it had an accuracy of 97.75%. Compared to Montalbo’s model, MF-CNN had an accuracy rate of 1.5% higher. This further illustrates the significant improvement in gastrointestinal tract diagnosis achieved through the implementation of the suggested MF-CNN.

Feature map analysis of MF-CNN

Feature map analysis involves interpreting and understanding the outputs (feature maps) produced by the convolutional layers in a CNN. These feature maps depict the activations of the neurons in the layer, signifying the features in the input data (such as an image) that the neurons have detected. Figure 16 presents a visual representation of the feature maps generated by the layers of the MF-CNN. Due to the model’s extensive depth, the figure selectively showcases only certain layers and blocks. In the earliest layers of the MF-CNN, as demonstrated in the feature maps, there is a distinct focus on the basic, low-level features of the input. These foundational elements can be compared to the basic building blocks of the image. For instance, the feature maps capture and exhibit aspects such as boundaries, textures, and distinct color regions. These foundational elements give us an overview of what the neural network first identifies and discerns in its initial layers. The early phases of interpretation are fundamental and relatively straightforward, concentrating on the primary structures and patterns in the data. A significant prominence can be seen in the upper left quadrant of Fig. 16. This particular section provides a visualization of the feature maps from the initial layers of the MF-CNN. The patterns emerging here graphically highlight the network’s ability to detect fundamental visual elements. We can observe distinct indications of contours and individual color regions, signifying the network’s focus on these rudimentary components.

Comprehensive assessment of state-of-the-art methods

In the following section, we systematically compare the efficacy of various CNN models in diagnosing gastrointestinal (GI) diseases, including our proposed MF-CNN. These models are compared comprehensively in Table 14, with an emphasis on their diagnostic accuracy. Upon analysis of the data, it becomes evident that the proposed method consistently outperforms other models in its domain. The outstanding performance metrics observed in this framework can be attributed to its advanced design and cutting-edge development methodologies. The table serves as a visible depiction of the advances made in the current study and supports the claim that our proposed method has established a new standard. We have demonstrated our model’s exceptional accuracy through testing and evaluation, making it a leading tool for detecting GI diseases.