Introduction

The world is anxious about a submicroscopic agent known as the coronavirus. Before the declaration of COVID-19 by the WHO in February 2020, the old virus belonged to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). COVID-19 is a zoonotic virus that is positive and mono-stranded RNA. It has four main structural proteins: the spike protein, the envelope protein, the membrane protein, and the nucleocapsid protein [1]. It is a highly infectious virus spread by aerosol and respiratory droplets [2]. Initially, there were no significant clinical symptoms when first infected by COVID-19. However, gradually the general symptoms became more severe for those who do not have a better immune system, especially aged people. In such a crucial situation, fast, reliable, and sensitive testing is required. In that place, RT-PCR plays a vital role in identifying COVID-19 suspects within a few days or hours [3, 4].

Coronaviruses belong to the Coronaviridae family, and their virion architecture resembles that of a crown when seen under an electron microscope. In December 2019, a novel coronavirus was identified in Wuhan, China, that could potentially infect humans (2019 novel coronavirus) [5]. SARS-CoV-2 (severe acute respiratory syndrome coronavirus) was named by the International Committee on Virus Taxonomy (ICTV) on February 11, 2020, based on its genetic relationship to the coronavirus responsible for the SARS outbreak in 2002–2003 [6]. It differs from past coronavirus outbreaks, such as the SARS coronavirus in 2002–2003 and the Middle East respiratory syndrome (MERS) coronavirus in 2012. The transmission rates of these viruses are extremely fast [5]. COVID-19 is highly infectious because it contains spike glycoprotein in its outer envelope. A unique amino acid stretch in the spike protein of the coronavirus SARS-CoV-2 is located in the S1/S2 region. It is speculated that this region is responsible for viral pathogenicity and transmissibility. After infection, the glycoprotein binds to the ACE-II receptor protein of the human airway tract. Most of SARS-CoV-2 patients have shown a spectrum of symptoms, ranging from mild flu-like symptoms to high fever. The WHO (World Health Organization) officially designated the disease associated with this virus as COVID-19 in the International Classification of Diseases (ICD) [6]. The emergence of COVID-19 has resulted in millions of deaths, industries' destruction, and economies' collapse [7, 8].

Tests are one of the most challenging aspects of identifying COVID-19. A patient infected with COVID-19 can be identified and treated with a list of contacts who need to be found and quarantined to prevent transmission. Therefore, the initial screening and quarantine of COVID-19 patients can save many lives. RT-PCR (real-time reverse transcriptase-polymerase chain reaction) is the most confirmatory test for diagnosing this type of virus [9]. RT-PCR testing is insufficient to manage this pandemic because preparing RT-PCR testing kits takes time. Radiographic images are alternative diagnostic tools for the identification of lung disease. But one disadvantage of radiographic images is that an expert doctor is required to identify the infected region in the lung.

This solution also required time and an expert doctor. Expert doctors are rare compared to the number of patients in this case. Thus, healthcare experts are searching for the fastest and most automated way to detect COVID-19 disease. Therefore, AI/ML can provide support using radiographic images in parallel with traditional methods. It is possible for doctors to analyze X-rays and CT scans more quickly and efficiently by integrating AI/ML techniques.

ML and DL have added value in detecting various lung diseases [10, 11] with higher accuracy and efficiency. It is well-known that radiological imaging, mainly X-ray and CT, helps in predicting the disease signature in a non-invasive way. From this point of view, X-ray imaging is the safest and most readily available diagnostic tool for COVID-19 like other diseases. On the other side, CT provides a high-resolution image with more prominent microstructural information related to lung damage. Here, Fig. 1 shows about how ML/DL can effectively identify lung image samples such as normal, COVID-19, and different types of pneumonia by automatically analyzing CT and X-ray images [12, 13]. Deep learning, a type of machine learning, integrates concepts of artificial neural networks (ANN) [17] as a result of substantial advancements in radiology imaging technology [14, 15]. DL and ML assist healthcare experts with the tracing, controlling, and managing of COVID-19 cases within quarantined periods through the use of mobile devices. Using LSTM (long short-term memory) [16], the susceptible–exposed–infected–recovered (SEIR) epidemiological model is used to estimate the COVID-19 peaks accurately. The method utilizes time series data to conduct the analysis.

Fig. 1
figure 1

General flow of an AI system for detecting infected segments and training the model accordingly

Full size image

To potentially reduce the transmission of COVID-19, strategies such as patient quarantine, contact tracing, and positive counseling were used [18, 19]. Supporting an essential technology data network through a terminal tracking system and data transfer to improve urban management may also improve viral detection efficacy and forecast the location of the next epidemic [20]. The researchers applied a bidirectional gated recurrent unit (GRU) and attentional techniques to predict tachypnea, which might be a first-order diagnostic feature for COVID-19 screening [21]. IoT connects different networking gadgets in today's environment, giving people comfort and simplicity of living [22]. As new technological advances are made, the Internet of things (IoT) provides us with intelligent solutions based on X-ray images [23]. These solutions will range from IoT-based industrial development of ventilation systems, masks, and other medical devices to safely tracking patient conditions in hospitals or self-isolation at home, designing new passive, privacy-preserving contact-tracing techniques, and diagnosing IoT and cell-phone data collection and analytics-based COVID-19 conditions [24].

Many of those vaccinations, if they are active at all, may not be able to elicit an immunological response in some populations. The effects of vaccination may vary among people due to the different reactions of their bodies to the elements in vaccines that activate virus-fighting T cells. There is a major part to the vaccination puzzle that goes beyond how many people have received a vaccine, but also how many will mount an immune response in the future [22]. AI might be used to build a platform that analyses clinical and immunological data automatically and effectively using the supervised classification model-based Vaxign reverse vaccinology machine learning platform for COVID-19. Other responsive approaches like artificial neural networks (ANN), convolutional neural networks (CNN), deep generative neural networks (DGNN), and other responsive approaches are used.

The primary objective of this paper is to discuss current advances in ML/DL-based diagnostic systems that use radiographic images such as X-rays, CTs, MRIs, and ultrasound. This study focuses on summarizing various databases used and class sizes, ML and DL algorithms, and customized CNN frameworks. Here, 36 customized solutions using CNN methods having their unique architectural patterns have been reported.

For better readability, the paper is organized as follows: “Introduction” Section describes the introduction of the paper. “X-Ray and CT imaging modality” Section will demonstrate various medical imaging modalities and their advantages and disadvantages, as well as digital image processing. “Available benchmark datasets” Section represents various manifestations of different imaging modalities. “Reported works” Section discusses reported work from the perspective of ML/DL using various classifiers and architectures. In “Discussion” Section, X-rays and CT imaging modalities are discussed along with their challenges and solutions. An overview of future directions and a conclusion are included.

To start with the notion of understanding the COVID-19 disease, it is worthwhile to know about the verities of clinical symptoms and its impact on disease severity, as discussed below.

Clinical Symptoms of COVID-19

There are no major symptoms associated with COVID-19, but it is a very serious condition that starts with minor symptoms. The twenty-first-century human society is confronted with a potentially fatal disease with numerous symptoms [25]. Patients can be separated into mild, moderate, extreme, and seriously ill stages depending on the progression of their illness [26, 27]. The condition has significant and harmful implications for the lungs as a whole. Fever, cough, shortness of breath, body pain, and fatigue are among the disease's most frequent fourteen clinical signs [28] in statistical analysis [29]. These patients' clinical care and effectiveness were recorded, with information on ICU assistance and length of stay, respiratory oxygen time, sputum NAT-positive time, health center stay time, and outcome in terms of full recovery, partial recovery with residual pulmonary injury, delayed recovery, and death gathered [30].

Figure 2 depicts different symptoms at different stages. A significant percentage of COVID-19 patients recover early with few generic symptoms. Moderate symptoms may not need hospitalization, but severe health concerns, such as muscle pain and a chronic cough, should be examined by a physician. Approximately 5% of patients require urgent hospitalization and oxygen support. Using demographic data with the assistance of machines and deep learning, researchers can make diverse predictions from multiple perspectives [31]. In clinical diagnostics, early signs based on machine learning lowered mortality risk [32].

Fig. 2
figure 2

Based on the severity of symptoms, the percentage share among COVID-19 infected patients

Full size image

Systematic Review Strategy

This systematic literature review used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) procedure for its preparation and reporting [33]. In Fig. 3, we go over the whole PRISMA analysis.

Fig. 3
figure 3

A four-phase search and selection strategy for a systematic review based on keywords published papers excluded or included in the review

Full size image

Search and Selection Strategy for Relevant Articles

We systematically searched the databases to identify COVID-19 using computer-based intelligent algorithms. For this purpose, we consider basically five full-text digital repositories, including Google Scholar, PubMed, DBLP and Scopus. Between November 1, 2020, and September 15, 2021, PubMed systematically searched for peer-reviewed publications written in English. Several queries were used to enhance the search-related quality of various deep and machine learning approaches for detecting COVID-19. The following were some of the most important search terms used to get data: “COVID-19 screening”, “machine learning”, “deep learning”, “X-ray using machine learning”, “X-ray using deep learning”, “CT using machine learning”, “CT using deep learning”, and “X-ray and CT using deep and machine learning”. These phrases were discovered in the titles, abstracts, and keywords of published publications. Furthermore, only original research publications relating to the identification of COVID-19 using ML/DL methods were included in the collection.

Inclusion–Exclusion Criteria

The articles were chosen based on the following inclusion criteria: (1) Research that used deep and machine learning for predicting COVID-19. (2) Several deep learning algorithms for COVID-19. (3) Transfer learning applied to a limited amount of data. (4) Different classifiers of machine learning algorithms. (5) Several image modalities for the chest.

The following studies were not included in the analysis: (1) Research that does not combine machine and deep learning with X-ray, CT, and other image modalities. (2) The classification methods of different lung diseases are not well described. (3) Articles that only use segmentation techniques without describing classification methods.

Data Segregation and Categorization

Based on two of the most common imaging modalities, the data were extracted and classified into four sections: X-ray images, CT images, machine learning classifiers, and deep learning comprehensive models. The CT and X-ray images analyzed in this research belong to distinct categories. However, it should be noted that the COVID-19 images are a common element among all of the datasets. Two other significant findings in a research paper are segmentation and transfer learning.

This comprehensive review study looks at some COVID-19 detection techniques that use machine learning and deep learning to help clinicians and doctors rapidly identify COVID-19 patients. We looked at chest X-ray and CT, a hybrid-COVID-19 network, and other deep learning methods.

X-Ray and CT Imaging Modality

Medical imaging is the practice of generating transparent representations of internal body structures for biomedical and therapeutic analysis and care, as well as a visual view of the role of internal tissues. The purpose of this treatment is to recognize and cure the condition [34]. This procedure creates a database of the organs' normal structure and function, making anomalies easier to spot [35]. This spotting can be achieved using a series of imaging tests, including X-ray, MRI, ultrasound, CT, mammography, nuclear medicine, fluoroscopy, bone mineral densitometry, and PET scan [36].

We are trying to represent a very informative comparison and contrast for medical image capture tools. Figure 4 shows those machines' brief overview and important parameters, which is very helpful for selecting machines according to patient conditions. Each section is divided into four different parts according to several imaging tool parameters. Here, yellow boxes indicate radiation exposure, while blue boxes represent the cost of different machines to capture a single image. Green boxes represent the time it takes to generate a single image, and red boxes show the suitable targeted organs for a better image.

Fig. 4
figure 4

Compare and contrast a variety of medical imaging instruments based on four criteria: radiation exposure, processing cost, image processing time, and appropriate organ for scanning [37]

Full size image

A CT scan provides a three-dimensional image with a detailed view of the lungs and soft tissues [38], whereas other medical imaging devices do not produce that high-quality image. It is more time saving and higher sensitivity than other radiological images [39]. Compared to X-ray-based assessment, CT provides a better diagnostic scope because of its clear visibility into organ tissues. But one of the significant drawbacks of CT scans is that different age groups and issues require different radiation doses [40].

Medical Image Preprocessing for Classification

After being collected from various medical devices, sample images must be enhanced. The most common image processing techniques include calibration, filter optimization, transformation, and registration [41]. There are 71 characteristics that may be examined to process images for analysis [42], but 3 significant features must be extracted. Those critical features are pixel-based, edge-based, and texture-based features [43]. Y. Wan shows more extracted features from differing perspectives [44]. Image segmentation based on characteristics is a significant step in medical image processing, and it contributes to the method of applying biological identifiers to image pixels [45, 46]. The main three image segmentation methods for biological purposes are pixel base, geometry base, and others [47]. A segmentation-based approach identifies three challenges: network design, training, and data [43]. A. Voulodimos [49] and A. Saood [50] describe a very efficient segmentation algorithm for the segmenting process of medical images.

Follow these four procedures in Fig. 5 to automatically classify COVID-19 using a segmentation algorithm and lung-based infections [51,52,53]. Medical image segmentation is useful for detecting sick regions and conducting further in-depth analysis [54, 55]. Image segmentation may be done on both two-dimensional [56,57,58] and three-dimensional [59] images. Most of the segmentation processes are done by the three-dimensional or CT images. Image conversion is necessary before a computer can understand it. The non-digital image needs to be converted to digital form and sent to the image processor for feature extraction. After extracting the features, a machine-based algorithm segments the functional area. For segmentation, the U-net is preferred. [60, 61]. Finally, various classes are determined based on algorithm interpretation and measurement [41]. Image segmentation and classification are essential jobs for diagnosing illness, and various researchers are working on it [62, 63]. Fourth, classification is used to detect or analyse patterns and insights in images. It comprises testing a series of classifiers, comparing their results, and selecting the best one. Some classifying algorithms are used for this purpose, such as the fuzzy-based segmentation method [64] and fuzzy-based color technique [65], nearest neighbor, neural networks, and learning vector quantization [66, 67].

Fig. 5
figure 5

Medical image processing covers four main areas: image formation, analysis, visualization, and management. The algorithms of image enhancement can be assigned as pre and post in all areas [41]

Full size image

Available Benchmark Datasets

Our collection comprises a variety of sources, so both X-rays and CTs were not accessible from a single source, and it currently contains large databases [68, 69]. A collection of datasets is distributed into two tables based on their categories. The images from these sources are used for two purposes. First, it is diverse, with images obtained from numerous sites, making it possible to design a sophisticated method to assist radiologists in diagnosing COVID-19 worldwide. Second, the images of the datasets are widely available to the scientific community and the general public. All public two-dimensional and three-dimensional datasets originate from three sources: Kaggle, GitHub, and Radiopaedia, presented in Fig. 6. In addition to these three, more sources are represented, including ultrasound [70, 71] and various dimensions of the image [72]. Every dataset image shows COVID infected and non-COVID infected cases.

Fig. 6
figure 6

Public datasets are available that contain primarily positive and negative labels on the images of COVID-19 suspects

Full size image

X-Ray Image Dataset

Basic medical support systems is very much needed due to the rapid growth of pandemic all around the world in minimum time. In almost all hospitals around the world, one of the low-cost radiography or X-ray machines can produce 2D (two-dimensional) images of the patient's chest. This chest X-ray machine is the standard for diagnosing problems with the lungs, valves, heart, blood vessels, chest, spinal cord, and bones [73]. X-ray produced radiographic images within less than half an hour with low-cost radiographic images [74], which is mentioned in Table 1.

Table 1 Dataset of X-ray image from different sources
Full size table

To highlight the various X-ray datasets in Table 2. Machines need more data for training purposes, but the current situation makes gathering correct and valid data extremely difficult. X-ray has more data globally than other radiographic data since it is used in almost every hospital. Most of the data were gathered from three separate sources: GitHub, Kaggle, and the Radiological Society of North America. In the X-ray image, we are trying to characterize five separate groups of data, including viral and bacterial pneumonia, and their highest accuracy using various deep learning and machine learning models.

Table 2 Dataset of CT image for different sources
Full size table

CT Image Dataset

CT is the gold standard for image processing because it provides better image quality. It is preferred over X-rays because it gives a more detailed three-dimensional (3D) overview of the lungs. In recent works [84], CT slices were used to detect early-stage infection signs like ground-glass opacity (GGO) and late-stage infection symptoms like lung consolidation [85, 86]. Table 3 identifies CT scan of the lungs as a clinical feature of COVID-19 patients. Bilateral lung lesions resemble "white lungs" as the infection spreads [87]. The density of the lesion steadily reduces, and the region of the lesion narrows in the latter stages. The density of the lesions increases significantly a few days later, and halo and reversed-halo symptoms emerge [88].

Table 3 Chest manifestation of different image modalities tools
Full size table

The CT imaging manifestations of COVID-19 patients are depicted in Fig. 7 in three categories: lesion density, lesion signs, and interstitial pulmonary involvement. The density of lesions is pure GGO consolidation, and the halo sign reversal indicates that the lesion is in its early stages. There is vascular enlargement, air bronchogram signs, and bronchial wall thickening in interstitial pulmonary involvement.

Fig.7
figure 7

Severity levels of COVID-19 disease indicate a variety of CT imaging manifestations [89]

Full size image

The amount of data in a CT scan is limited due to certain technical and economic drawbacks [40, 90], which affect the performance of deep and machine learning. On the other hand, a CT scan learns image features of multiscale inputs that allow it to outperform large-scale training data [91]. Table 2 focuses on various CT scan data sources, such as image types and capture parameters. Capturing parameters is vital for CT scan machines because different parameters provide different results, and radiation is one of the major concerns. Another factor of the CT image is ROI delineation [92,93,94].

Radiomic Signatures of COVID-19

In COVID-19, an extremely severe case of pneumonia-like disease that damages the lungs, the macro-imaging modalities are used to address different signs of infections. This manifestation machine learning algorithm detects different lung diseases like viral pneumonia and bacterial pneumonia, including lung cancer. This study, explores a new area of lung image analysis using ML. Table 3 shows four different manifestations of chest imaging modalities. Among four of them, two modalities are used explicitly in our review discussions.

Reported Works

Deep and machine learning is two effective approaches to speeding up testing performance in developing an autonomous diagnostic system. Among the four imaging modalities discussed in "Radiomic signatures of COVID-19" Section, when compared to other medical imaging modalities, X-rays and CT are both widely available medical imaging modalities that are useful for detecting COVID-19. Researchers all over the world have applied automated frameworks like ML and DL to detect lung diseases such as COVID-19, pneumonia, and others. In recent studies, many researchers have been experimenting with their work using two imaging modalities and two automated techniques, such as ML and DL. Figure 8 uses X-ray and CT images to illustrate the percentage share of an ML and DL used by several researchers. In ML, we put a lot of focus on different classifiers and approaches with their accuracy value in Table 4. Similarly, a number of ANN and CNN models are listed in Table 5, along with their accuracy, specificity, sensitivity, precision, recall, and f-measure.

Fig. 8
figure 8

Percentage distribution of ML/DL techniques for COVID-19 screening using X-ray and CT images

Full size image
Table 4 Summary of the conventional machine learning techniques applied on X-ray and/or CT images
Full size table
Table 5 Deep learning architecture specially used in COVID-19 and compared with other architecture
Full size table

The comparative study found some novel techniques to classify diseases with limited resources [45, 78]. One of the most problematic aspects of the COVID-19 study is the lack of reliable and precise data. Several studies employed transfer learning in both two-dimensional [116,117,118,119] and three-dimensional [120,121,122] images due to a shortage of appropriate datasets to improve better evaluation performance [123]. Another study used both X-ray and CT images to use transfer learning techniques [124]. The proposed approach is still in its early stages of adoption, and it will require technological enhancements before healthcare institutions can extensively use it. We also carefully evaluated several useful review articles [125,126,127,128,129,130] that used different classifiers of ML and several architectures of DL in our comprehensive review article.

Machine Learning Models

The main objective is to develop automated approaches by integrating and improving machine-learning models to detect and categorize coronavirus-infected patients from non-infected ones quickly. In the machine learning field, researchers discuss diverse areas of tracing, identifying, treatment, and monitoring [131]. Figure 9 depicts the general workflow of machine learning prediction and evaluated it for final output [132]. For the machine learning model, preparing data according to features for training and testing is critical. Different classifiers were applied for validation to select the appropriate features [133]. Because of its image features, the machine learning method can be used to detect COVID-19 disease in both two-dimensional and three-dimensional images [134].

Fig. 9
figure 9

Machine learning model depicts several states from input data through performance evaluation

Full size image

In this case, classification is an extremely important task in classifying normal, bacterial pneumonia, viral pneumonia, positive COVID-19, and negative COVID-19. Several machine learning classifiers were used for classification purposes after extracting features from traditional methods [135]. M. A. Elaziz et al. [136] introduce two new reliable classification methods, and the first focused on descriptors and the second based on feature selection [42]. The classifier represents the input data using feature sequences generated from feature extraction [137]. Table 4 shows machine learning classifiers, their maximum results, and the various datasets used by machine learning models. SVM and Random forest (RF) provide better results than all the classifiers, but it mainly depends on the dataset.

Shi et al. [138] presented the iSARF (infection Size Aware Random Forest technique), which classified individuals based on the size of infected lesions and used random forests to identify each class of people. Using five-fold cross-validation, the proposed procedure had a sensitivity of 0.907, precision of 0.833, and an accuracy of 0.879. This approach may be used as a decision support system but not relied on because the false-positive rate is relatively high, and the system performance increases slightly.

Hussain et al. [139] showed morphological and texture characteristics to immediately determine whether a patient is infected or not with COVID-19. The method's key benefit is identifying adaptive image features and classification concurrently. They evaluate the hypothesis of texture features in addition to optimizing machine learning classifiers. They attempted to demonstrate more significant 95% accuracy in five different machine learning classifiers. However, the fundamental flaw in this strategy was that it only used a limited images dataset. Mahdy et al. [140] proposed an automated lung X-ray image assessment method that combines multilevel thresholding and support vector machines. A multilevel threshold is a technique for dividing a gray image into several regions. The COVID-19 damaged lung X-ray images are classified using deep features by the support vector gadget. They describe with equation that two-class boundaries can be used to enhance hyperplane distance. SVM improves the interval between the hyperplane and the edge. Other researchers employed CNN with SVM classifiers [141], similarly to Mahdy's study. But CNN with SVM is more flexible from a future search perspective.

The survey article by Farhat et al. [142] analyzed medical image processing using deep learning in depth with five different sections. The second section covers a broad range of medical image modalities. However, for the final section, the remaining three sections use DL to process COVID-19 images and use it for pulmonary medical images. They clustered with various DL parameters on pulmonary medical image analysis. But they are not discussed the DL-based efficient framework and future work related to the DL model. In this paper [143], they show how ResNet152 and ML classifiers can be used to classify COVID-19 effectively. Since being developed on two publicly accessible datasets, the suggested technique has outperformed all groups. The synthetic minority oversampling technique (SMOTE) algorithm was also applied to balance intra-class heterogeneity among the datasets. ML algorithms are applied to the training dataset using SMOTE-based features, leading to the best possible result achieved by random forest. We observed that apart from ResNet152, some other models worked best in the feature extraction field.

Pereira et al. [144] have introduced a classification scheme that combines two forms of classification: multiclass classification and hierarchical classification. Since pneumonia follows a hierarchical classification structure, they placed a particular emphasis on it. To rebalance the distribution of classes in this domain, they proposed resampling techniques in the schema. One of the most important features of X-ray images is texture. Several well-known texture descriptors, as well as a pre-trained CNN model, are used in our classification scheme to extract features. Similar to this research, other researchers have used DL models for feature extraction and selection [145,146,147]. They looked at early and late fusion methods in the hierarchy to incorporate the strengths of various texture descriptors and base classification methods. Few papers introduced different fusion-related techniques for COVID-19 diagnosis [148].

Tuncer el al. [149] introduced a hybrid approach for extracting and selecting features. The main goal is to extract an essential feature and use the most discriminative one. Five classifiers classified the most discriminative functionality. Few papers introduced important correlations between laboratory extracted data and deep learning segmentation data using the machine learning classification technique [150]. Liu et al. [151] used an ensemble of bagged tree algorithms to explore a promising approach for distinguishing COVID-19 from general pneumonia. After selecting the feature, they used an ensemble of bagged trees, four ML classifiers, and ten-fold cross-validation [152].

Deep Learning-Based Work

Deep learning is the most popular category of artificial intelligence systems, which performs well for analyzing unstructured data using several layers. Data are processed across multiple layers in this model, with each layer using the output of the previous one to provide its specific output. There are many of DL algorithms that can be used for different purposes. Among all the deep learning algorithms, the convolution neural network (CNN) plays a significant role in analyzing visual imagery. Many frameworks have been proposed that use CNN algorithms for image classification, segmentation, and object detection, such as VGG16 [166], MobileNet [167], ResNet [168], InceptionV3[169]. Using this standard CNN framework, researchers have developed custom CNN models, which are represented in Table 5. Those frameworks are specially built for COVID-19 detection. With the help of CNN, researchers were able to find in-depth analyses of medical imaging modalities [170] using a variety of deep learning frameworks [171]. By analyzing biomedical and clinical data, healthcare experts and academics can learn about new ways to serve the patient community [21]. According to recent studies, DL approaches almost accurately identify a wide range of diseases using radiographic images, reducing the burden on medical practitioners while simultaneously improving diagnosis performance [172,173,174,175].

Data gathering, preparation, division, package selection, validation, and model performance are all phases in deep learning-based systems in general. It has been widely used in the prediction and diagnosis of COVID-19 in patients in recent years. Figure 10 depicts the general pipeline of a deep learning system. Data can be taken in various forms for deep learning training, such as X-ray, CT, MRI, and ultrasound. Due to some complicated structures in the medical system, handling unstructured data is a big issue. After evaluating image severity levels, an ANN was applied to the data to obtain a basic classification result. Compared to ANN (artificial neural networks), DNN (deep neural networks) performed well due to their multiple hidden layer features, which increased accuracy. However, because of the problem of vanishing gradients in DNNs, convolution neural networks (CNNs) were considered as an alternative [176,177,178,179]. Compared to other image recognition learning models, CNNs are proven to produce better outcomes. CNNs are recommended because they give better visual processing models for detecting noisy input. Therefore, DL approaches are increasingly being used to automatically extract related features and automatically identify the object of interest [180].

Fig.10
figure 10

The deep learning architecture depicts several states from pre-process data through performance monitoring of several classes

Full size image

Ensemble, fusion, and segmentation [181, 182] are three additional techniques that significantly enhance the overall classification performance in deep learning [183, 184]. Here we focus on an ensemble of classifiers that generally outperform a single classifier in categorizing many lung-related diseases, particularly pneumonia and COVID-19, by lowering variation in the final prediction [185, 186]. The performance of the COVID-19 disease classification is being boosted through ensemble-based learning [187]. Different deep transfer learning techniques may be used to build an ensemble of classifiers by altering the architecture of pre-trained CNN models in the setting of deep transfer learning [188]. Another method is the snapshot ensemble technique [189]. We will cover segmentation and fusion in the next section.

In Table 5, we explore several custom-built deep learning convolutional network architectures. The third column addresses two objectives, which depend on deep learning architecture. One is a custom model compared to a standard model, and the other is pre-trained architecture. We also addressed other pre-train custom models with their own names, which are as follows: MANet [223], MDCNN [71], CSEN [224], MSCovCNN [225], ToraxIA [226], FJCovNet [227], ReCoNet [228], ReCOV-101 [229], COVID-CLNet [230], FCONet [231], CNR-IEMN [232], COVID-NET CT-S [233], and CovFrameNet [234].

Apart from the custom network mentioned above, several researchers are experimenting with pre-training deep learning architecture by modifying specific parameters or implementing transfer learning techniques. Nanochest-net [235] experimented with five datasets, and the base architecture is ResNet50, Xception, and DenseNet121. F. Saiz et al. [236] are working on two different approaches, one is histogram-based, and the other is decoder based. K. Hammoudi et al. [237] split the input images and fine-tuned them by Inception, ResNetV2, and predicting the final image using the directional LSTM method. CT and X-ray of suspected patient cases were classified as infected or non-infected COVID-19 using VGG16 [238, 239] and VGG19 [240], including other architectures [241]. It is necessary to fine-tune VGG16 because it always does not offer better results [242]. Another fine-tune process was done for this pre-train network AlexNet, GoogleNet, and SqueezeNet [243]. Several researchers have been experimenting with various ResNet architectural variants as a result of their findings [244,245,246]. Along with pre-train deep learning architectures, some other articles used only CNN models with different types of layer sizes [247,248,249]. To improve the convolution neural network performance for identifying COVID-19, geometric image augmentation is one of the essential factors [250, 251]. Rahhal et al. [252]’s article is divided into four parts. To begin, the EfficientNet backbone of the CNN model was used. In the second step, multiscale feature extraction was used. Third, the extracted features from the multiscale feature map extractor were combined using a weighted average fusion operator. In the final phase, the modules were classified and the network was optimized. Besides optimizing the network, B. Sekeroglu and I. Ozsahin [253] used the extreme machine learning technique to achieve the first training. Other studies show extreme learning with different activation functions [254].

Khan et al. [13] article represents an extreme version of Inception, which falls under the CNN architecture and its name is CoroNet. This architecture prepared 71 layers with the help of deep CNN pre-trained on ImageNet. CoroNet is inspired by Xception, with two fully connected layers added at the last and a single dropout layer. This extended model provided a satisfactory result for an insufficient dataset to avoid the overfitting they used for transfer learning. In these custom CNN architectures, it does not perform well in the four-class X-ray images dataset. Like CoroNet, other experiments used various versions of Inception architecture [255] [256], experimenting with other versions of Inception architecture. Gupta et al. [2] proposed a DL classification model applying X-ray images. It implemented two image preprocessing or regeneration techniques which were used to achieve better functionality for its model. One method is a fuzzy color image enhancement, and another is image stacking. Fuzzy color is used to improve image quality and reduce image noise. This method divides the picture into fuzzy windows, assigning a weight to each pixel according to the distance between the window and the pixel. Focus blending, z-stacking, and central plane consolidation are used in image stacking to enhance accuracy to 99.08 percent. Because it can reduce the noise from low-quality X-ray images, this fuzzy color image processing method is highly efficient. Authors may use social mimic optimization in addition to fuzzy color and stacking to improve comparison.

Ozturk et al. [194] proposed five pre-trained convolutional neural network-based models for diagnosing coronavirus pneumonia-infected patients in this research. Three separate binary classifications of four different levels have been introduced. Out of the five models, ResNet50 provides the best classification results. Mahmud et al. [190] proposed CovXNet. It is a deep learning model for identifying pneumonia with distinct localization using chest X-rays. Active depth-wise convolution with different dilation frequencies is used instead of traditional convolution. A produced class activation map allows for discriminative localization of irregular areas, which may aid in diagnosing pneumonia's various clinical features. Z. Wang [257] discusses automatic discrimination and localization of areas using DL.

Zhang et al. [123] proposed a model, modified version of ResNet32, called COVID19XrayNet. A large database of pneumonia chest X-ray images was used to fine-tune a pre-trained deep residual network (DRN) model called ResNet34. Shiri et al. [258] introduced the ultra-low-dose CT imaging residual model, and few researchers worked on this model [259,260,261]. To diagnose coronavirus from chest X-ray, researchers Chowdhury et al. [192] created PDCOVIDNet, a CNN application. The proposed model is divided into two parts, the first of which is feature extraction and the second of which is classification and visualization. They created an architecture that implements a dilated convolution in a parallel stack of convolutional blocks to collect and distribute essential features in actual environments through the network, achieving good detection accuracy.

Ucar et al. [193] proposed a new architecture for detecting COVID-19 patients rapidly and reduced diagnostic complexity. The COVIDiagnosis-Net architecture is built on a deep Bayes-SqueezedNet model. SqueezeNet is a cutting-edge deep learning architecture based on the well-known AlexNet, but with a much smaller model scale. In the architecture, offline augmentation is performed. One of the main advantages of this network is that it can operate well in an embedded environment. Zhou et al. [1] research article implements a novel approach using the symmetry characteristics of the lungs and other tissues to break down the three-dimensional segment problem into two-dimensional one, which greatly improves segmentation accuracy while reducing the number of model parameters by order of magnitude.

For the comparative and critical analysis, Majeed et al. [3] used class activation maps (CAM) to examine the classification made by CNNs. In these cases, they used a total of 12 CNN architectures using natural images to assist radiologists in identifying COVID-19 disease that is dependent on X-ray images. In addition to comparative research, they also suggested a simplified CNN architecture with fewer parameters than many well-known CNN architectures. Wang et al. [82] used an automatic feature extraction method and directly extracted features from five pre-trained CNN models. They observe high accuracy with 99.33% for implementing the deep model with various classifiers. In the proposed method, they prepared in three steps. First step is preprocessing of the input dataset. In the second step, bottleneck features are extracted using deep learning models that have been pre-trained and the last step is machine learning classifier. However, none of the three approaches is optimized, and evaluation may take a long time.

Asraf et al. [21] introduce the battlefield of the world's fastest-spreading virus, and scientists are ready to fight with various technologies and methods, especially in artificial intelligence. They came up with several technologies, from disease tracking to drug discovery. With the help of CNN and LSTM, they used CT and X-ray images to predict the COVID-19 suspect with significant accuracy. A deep learning-based disease tracking system using camera footage, a gated recurrent unit (GRU), and a satellite for geographical risk assessment is also discussed. Identifying disease from an image is a challenging task in the medical domain. Since DL is a difficult area, long short-term memory (LSTM) networks play a key role. It may be used in conjunction with CNN and entropy feature extraction or independently with the feature section [262, 263].

Ahsan et al. [264] proposed a method that uses numerical and categorical data, as well as an X-ray image, to construct a COVID-19 diagnostic model that was evaluated on a balanced and imbalanced dataset using the multilayer perceptron and convolutional neural network (MLP–CNN). Hamadneh et al. [265] also applied MLP, which was based on single-country demographic data and used the prey–predator algorithm. Elzeki [266] employed a different technique for working with an imbalance dataset and two-layer image fusion in DL. This paper's [267] primary focus is deep feature fusion (DFF), which is the fusion of several deep feature representations from both convolutional neural networks (CNN) and graph convolutional networks (GCN). They selected the best method from eight suggested network models and then applied it to fifteen state-of-the-art methods. They experimented with batch normalization, dropout, average pooling to replace max pooling, and multi-way data augmentation. Zhang et al. [268] introduced stochastic pooling instead of average pooling. A few other papers introduced different fusion-related techniques for COVID-19 diagnosis [200, 269, 270] and other approaches to multimodal fusion learning [271].

In deep learning, convolutional neural networks were used to distinguish between COVID-19, non-COVID-19, and pneumonia. In most cases, pre-trained CNN methods are used. Many research articles [272,273,274,275,276] show their experiments with more than five pre-trained architectures. M. M. Rahaman et al. [277] experimented with 15 pre-trained CNN architectures with transfer learning and represented the confusion matrix very organized way.

As per Apostolopoulos et al. [76] research, deep learning combined with X-ray imaging could extract important biomarkers linked to the COVID-19 disease. They used two key strategies: function extraction through transfer learning with a pre-trained model parameter tuning and architecture improvements. The analysis recognizes two shortcomings in this field. Panwar et al. [77] introduced a transfer learning-based algorithmic model known as ‘nCOVnet’. The top layer of the VGG16 model is used as the base model in this algorithm, and is then added to the custom five layers as the head layers. They claim that the nCOVnet model is 97.62 percent accurate and that an RT-PCR test can replace it. Jain et al. [78] explain this using a comparative study of deep learning models based on CNN. In addition, the proposed algorithm is based on the three models and is explicitly clarified with each step evaluated using Inception Net V3, Xception Net, and ResNet. They tested their accuracy by activating LeakyReLU, a novel modification. Instead of the relu, this activation mechanism was used.

Albahli et al. [79] describe comparative analysis with existing three DL models and fine-tuning for automatically detecting COVID-19. They used ImageNet for transfer learning due to a lack of data. InceptionNet-V3 is the best approach for the overfitting problem. This paper discusses the average solution for automatically detecting COVID-19 from X-ray images. Authors [80] used a VGG16 model that was well-trained and evaluated with a vast range of X-ray image datasets from three different classes. In the image processing step, most of the diaphragm region was removed, the segmented grayscale image was converted, and a bilateral low-pass filter and histogram equalization was applied. They illustrate that their suggested graph of a transfer learning model effectively overcomes overfitting and underfitting.

Mohammadi et al. [81] explain using a comparative analysis of the CNN model with some modifications to achieve better accuracy. The picture is scaled to a normalized form, and the pixel value is changed from zero to one. The other is to tune and enhance the outcome by processing the transfer learning layer portion. They also mention that MobileNet performs better than the other three CNN models. According to Gao et al. [45], since it has been discovered that classification outcomes are strongly dependent on segmentation, authors have merged classification and segmentation processes. They used U-net for slice-level segmentation and classification. At the segmentation level, they proposed a novel weighted dice loss. They compared their model with five deep learning networks achieving better accuracy both in internal and external validation. N. Paluru et al. [278] introduced Anam-Net as state-of-the-art U-Net technology with a fully convolutional AD-block developed inside a symmetric encoder–decoder architecture. This architecture is also compared with heavyweight segmentation methods such as Enet, DeepLabV3, LED-Net etc. Similar to this work, other CNN-based segmentation methods help to detect COVID-19 diseases [279,280,281,282,283].

Ying et al. [96] proposed novel deep learning architecture for CT images. Comparison to the other common three deep learning models, VGG16, ResNet, and DenseNet, the details relation extraction neural network (DRENet) architecture performs well on CT images. DRENet outperformed other architectures because it has a powerful ability to retrieve data from images. They detach key areas of the lungs, fill in the vacant lung segment in the design to avoid noises, and extract top-K images for image-level predictions that result in patient-level diagnoses. Wang et al. [97] identify basic DL and radionics features related to COVID-19 pneumonia in an attempt to optimize ML algorithm interpretability and increase the overall efficiency of COVID-19 pneumonia imaging phenotypes. They also demonstrate the result of DL and radiomics models for COVID-19 pneumonia prediction using chest CT, showing the ability for ML predictions to enhance radiologist’s predictive sensitivity.

S. Karakanis and G. Leontidis [284] discuss lightweight DL models for detecting COVID-19 using chest X-ray. They try to prepare synthetic images to overcome the limitation of COVID-19, which is previously present in conditional generative adversarial networks, adapted for our purpose (cGANs). G. Ciano et al. [285] and D. Mahapatra et al. [286] elaborately explain synthetic image generation, including different types of GAN for segmentation purposes. Y. Karbhari et al. [287] proposed a novel model using synthetic image generation to overcome the limitation of COVID-19. They discuss the whole topic in four major points. First, they develop an auxiliary classifier generative adversarial network (ACGAN) to generate chest X-ray images. Apply a different CNN model to these synthetic X-ray images, then fine-tune the method for better accuracy. After that, using the harmony search algorithm to reduce the dimension of the feature vector, feature selection is made on the CNN extracted feature vector.

Other Reported Works

Arshadi et al. [11] survey article elaborately discusses drug discovery and vaccine development, which is one of the prime targets of the AI ecosystem in the coronavirus pandemic. The molecular mechanism section explains how viruses spread from one host to another host via the viral spike protein and its structures. The main aim of drugs and vaccine development is to manipulate and predict intertwined biological pathways or off-target biology of current proven safe drugs. New clinical trials will be easily conducted using deep learning and machine learning. Wu et al. [18] suggested a deep learning method focused on hybrid weak labels to learn infection and consolidation information from CT datasets with the help of expectation–maximization(EM). A UNet was first trained with supervised learning to predict the infected areas based on clear semantic labels. Single-class contours were used to train the network, which was then fine-tuned using poor patient-level labels. The proposed framework's feasibility is demonstrated by evaluations focused on databases from various hospitals worldwide. Training a semantic segmentation network is divided into two labels, strong label and weak label. The results section discusses severity segmentation, consolidation segmentation, and statistical testing of consolidation score.

Allam et al. [20] article discussed sharing pandemic data and preparing technological devices and labs around the world to exchange data and cooperate to create systems and cures. Related activities should be discussed by smart city practitioners on how collaboration initiatives could allow for the maximization of public safety in those and similar scenarios. Managing a huge amount of data, artificial intelligence can give added benefits to accurately and efficiently solve the current situation. Gupta et al. [24] show the great perspective of smart connectivity for quickly and efficiently taking steps according to fetching data. This smart connectivity fetches data from many areas such as smart city, E-health, smart locality, smart supply chain management, and even collects data from home. For collaboration, this proposed smart networking is separated into separate layers. IoT and CPS architecture are used in this communication scheme. The e-health ecosystem is now applied centrally with AWS, thanks to smart connectivity.

García et al. [26] discuss the details of antibody response and immunological profile of COVID-19 patients. It also shows clinical and immunological with different stages, including protective immunity and inflammatory spectra. It is important to remember that very little time has passed since the pandemic began and how little data have been gathered. The evidence presented in the papers examined strongly supports quantitative and qualitative variations in immune responses among SARS-CoV-2-infected people. Mohamed et al. [35] discuss all aspects of medical imaging, and also explain certain fundamental features of digital images and their applications. They describe four major types of diagnostic images in the medical image section: X-ray, CT, nuclear medicine, and ultrasound. Image analysis, such as enhancement, segmentation, thresholding, edge detections, and k-mean segmentation, is the main objective of this paper. This research paper covers any important aspect of medical imaging in great detail.

Lightweight design provides several advantages, particularly in the area of overfitting. H. Mukherjee et al. [288] proposed shallow CNN architecture with only four layers. Because of the multiple layers and a large number of parameters, most CNN architectures take longer to train, even though they suffer from overfitting. However, the architecture developed by Mukherjee and co-researchers performs exceptionally well with unseen data, with 99.69 percent accuracy. Zhang et al. [98] say that using a CT image, the uAI (intelligent assistant analysis system) can correctly classify COVID-19 pneumonia. The capacity of the uAI to rapidly and reliably localize and measure infection locations from CT can help practitioners not only diagnose COVID-19 but also evaluate the disease and direct treatment plans. Multifocal bilateral GGOs are typical CT traits of COVID-19 pneumonia, with the dorsal section of the right lower lobe being one of the most usual sites of infection.

Fan et al. [289] suggested approach works on CT slices, but CT slices have several challenges due to their low strength and high variation. Most recent studies on three-dimensional image slices used GGO in the early stages and pulmonary consolidation in the later stages. But proposed methods named Inf-Net and Semi-Inf-Net identify infected sections using reverse and edge attention, which is an explicit and implicit method, respectively. To build a global map from the high-level attributes, Inf-Net is used as a parallel partial decoder. But actual practice is classifying the COVID-19 patients before segmentation. Shan et al. [95] built a DL-based method to overcome the significant challenges of identifying COVID-19 from CT images due to low contrast, shape, and segment position. This system measures infection based on volume and contours infection regions automatically. Another necessary procedure was computed to show the entire pipeline for quantitative segmentation and infection assessment [290, 291].

Discussion

Our study included 265 research papers encompassing various medical image analyses using ML and DL. In Fig. 11, 118 research articles solely look at X-rays, 96 articles look at CT images, and 36 articles look at CT, X-ray, and other related radiography images. In the years 2020 and 2021, most of the articles were published. We observed that these infectious diseases could be distinguished by two visual indicators: ground-glass opacity and crazy pavement patterns. Based on these two most significant visual indications, ML and DL methods classify disease. Most artificial intelligence researchers were interested in the CNN architecture for automatically identifying a disease from chest radiography. However, the CNN architecture has several drawbacks, such as overfitting, for a low number of image datasets. In medical research, researchers work with sensitive human data, and false-positive values may infect more people.

Fig. 11
figure 11

Papers published in the last 2 years based on various medical imaging formats, from February 2020 to July 2021

Full size image

Our study observed that medical image data and computer-based automated systems face several challenges [292]. In both ML and DL, a sufficient amount of high-quality data is required to train the model. The training time is considerably longer for poor-quality images [293]. Specific problems in the automated diagnostic process are directly or indirectly related to our review. However, our study might be incomplete if we did not highlight such challenges. First, we observed that, for the limited number of sample sizes and imbalanced datasets, DL performance was degraded in a few cases. The researchers used augmentation techniques and transfer learning to overcome this issue [153, 294]. Second, one of the essential areas is the DL method for radiographic segmentation. However, since segmentation requires numerous annotated datasets, in the case of COVID-19, it is challenging for the current situation. On the other hand, performing annotation on new images will also be a tedious task. Another problem with medical image segmentation is that the target organs are heterogeneous [48]. The target organ or lesion's scale, shape, and place can vary significantly from patient to patient [295]. Third, due to the viral mutation of the virus, it is now difficult to predict the virus with the previously standard trained model [296]. Forth, the lack of a chest X-ray orientation (posterior or anteroposterior, upright or supine) may have delayed X-ray-based research. Another disadvantage is that the timing of chest X-rays about symptoms is not documented, which is a severe flaw in this X-ray image-based study [297, 298]. Fifth, X-rays and CT produce images based on radiation technology. The virus is particularly prevalent in people of all ages, so the radiation dose is essential, and the slightest variation in radiation can cause significant harm to the patient [299]. Sixth, since the target organ is heterogeneous, the idea of feature hierarchy grows, and the complexity of the neural network continues to grow [176]. The time consumption of feature engineering increases the difficulty of specific models [300].

During this pandemic, the radiologists in spite of their significant shortage [301] were constantly engaged to provide the diagnostic support to the huge amount of the patients. In effect, sufficient number of X-ray, CT, MRI image data were not properly stored and also not available to the researchers for the development of the AI/ML algorithms. To overcome the limited amount of data, most researchers applied transfer learning techniques to increase accuracy and reduce overfitting [302,303,304,305]. There are two main problems with transfer learning: negative transfer and overfitting. Most studies deal with overfitting in a generic approach and do not discuss negative transfer challenges. Negative transfer learning only works if both models' beginning and target problems are sufficiently comparable. However, because we are still in the experimental stage, our target problem does not give precise or pure data, which might lead to unexpected results. Synthetic image generation [102] is another approach for dealing with this problem, although it is still in its early stages and is unsuitable for sensitive image data.

Some studies used only two classes of images to train their deep learning models: normal and COVID-19, which is confusing because pneumonia and COVID-19 images are quite similar. If the DL model is not adequately trained with pneumonia and COVID-19 images, typical pneumonia patients may experience needless fear as a result of this outcome [306]. To overcome this misclassification, deep learning architecture should be trained with three class images and added extra features like stack ensemble learning [307]. This technique makes the model robust to classify diseases more elegantly. A few researchers implemented feature extraction techniques to extract relevant features from radiographic and applied several ML approaches to reduce misclassification as much as possible. In the context of misclassification, the DL model works like a black box, which does not represent why these images are misclassified. In these situations, visualization is essential to display the infected region of the lung to the clinician in a more transparent manner. To overcome the transparency problem for the DL researcher, they applied Grad-CAM [308] visualization to show the infected region of the chest X-ray image.

Conclusions and Possible Future Work

This article reviews the various DL and ML approaches for detecting COVID-19 from various radiographic images. Moreover, the most popular pre-trained and custom-built CNN architectures were discussed. The datasets used in various research are explained, as well as the shortcomings of current techniques. Our study has found that custom CNN approaches have a lot of promise for automatic COVID-19 diagnosis from the already available datasets. Here, we discussed how to deal with a variety of problems. We hope that this article will assist researchers in selecting the appropriate network structure for their topic, and be aware of potential problems and solutions. Medical professionals and computer scientists, on the other hand, should collaborate closely to confirm the use of CNN approaches by combining their respective skills.

Several researchers applied conventional transfer learning algorithms to COVID-19 research using radiographic images. Hybrid heterogeneous transfer learning will be used in the future to bridge the gap with the small radiography dataset. Apart from the dataset's limited number of images, multiclass lung disease might be another useful, efficient, and accurate COVID-19 diagnosis method. In some cases, radiological instruments may not be able to produce high-quality images suitable for DL training. Another technique for detecting COVID-19 from noisy and unclear images in this situation is to employ synthetic image synthesis. It is far better than their previous version in synthetic image generation, with the advantage of a smaller number of images for COVID-19 research. Other approaches can benefit from the scarcity of training data for the COVID-19 class that is few-shot, zero-shot, and deep reinforcement learning. These approaches ensure minimal human intervention in the diagnosis. Aside from working with a few images in several datasets to avoid overfitting or undefining, the well-defined DL architecture is also a significant concern in this research. In the future, analyzing the several DL and ML architectures and their supporting methods will make them reliable and efficient models for diagnosis in a more resource-friendly and time-efficient manner.