Deep learning-assisted ultra-accurate smartphone testing of paper-based colorimetric ELISA assays

Highlights

• This deep learning-assisted smartphone platform is unaffected by ambient lighting.

• A fully automated “image in, answer out” operation fashion.

• A 2048 custom image dataset is used to test 4 mainstream deep learning algorithms.

• GoogLeNet provides >97% accuracy in quantitative rabbit IgG testing.

• The area under the curve (AUC) is 4% higher than that of conventional methods.

Abstract

Smartphone has long been considered as one excellent platform for disease screening and diagnosis, especially when combined with microfluidic paper-based analytical devices (μPADs) that feature low cost, ease of use, and pump-free operations. In this paper, we report a deep learning-assisted smartphone platform for ultra-accurate testing of paper-based microfluidic colorimetric enzyme-linked immunosorbent assay (c-ELISA). Different from existing smartphone-based μPAD platforms, whose sensing reliability is suffered from uncontrolled ambient lighting conditions, our platform is able to eliminate those random lighting influences for enhanced sensing accuracy. We first constructed a dataset that contains c-ELISA results (n = 2048) of rabbit IgG as the model target on μPADs under eight controlled lighting conditions. Those images are then used to train four different mainstream deep learning algorithms. By training with these images, the deep learning algorithms can well eliminate the influences of lighting conditions. Among them, the GoogLeNet algorithm gives the highest accuracy (>97%) in quantitative rabbit IgG concentration classification/prediction, which also provides 4% higher area under curve (AUC) value than that of the traditional curve fitting results analysis method. In addition, we fully automate the whole sensing process and achieve the “image in, answer out” to maximize the convenience of the smartphone. A simple and user-friendly smartphone application has been developed that controls the whole process. This newly developed platform further enhances the sensing performance of μPADs for use by laypersons in low-resource areas and can be facilely adapted to the real disease protein biomarkers detection by c-ELISA on μPADs.

Introduction

Microfluidic paper-based analytical devices (μPADs) have been demonstrated as a promising tool for detecting disease-relevant biomarkers at point-of-care testing (POCT), thanks to their low-cost, ease of operation, and self-driven capillary fluidic flow [[1], [2], [3], [4], [5]]. In the last decade, many biosensing mechanisms, such as electrochemistry, chemiluminescence, fluorescence immunoassay, and colorimetric enzyme-linked immunosorbent assay (c-ELISA) have been developed and realized on μPADs [[6], [7], [8], [9], [10]]. Among them, colorimetric μPADs are particularly suitable for being used in low resources settings, especially when using smartphones as colorimetric reading and analyzing tools, and no further complicated equipment is needed [11,12]. The smartphones feature the broadest user community, a friendly operator interface, and excellent data transfer capability. It is a powerful computation platform capable of fully automating the colorimetric results collection, analysis, and displaying process, using its camera, processor, and screen, respectively [[13], [14], [15]]. Meanwhile, it is usually cheaper than the dedicated medical-grade colorimetric reader [16]. Hence, the smartphone-based colorimetric μPADs well meet all the criteria outlined by the World Health Organization (WHO) for POCT devices in low-resource settings, abbreviated as “ASSURED”; affordable, sensitive, specific, user-friendly, rapid and robust, equipment-free, and delivered [17]. And it can significantly improve healthcare worldwide by providing low-cost, timely, accurate disease screening.

Recently, smartphone-based colorimetric μPADs have been extensively studied and developed to detect various disease biomarkers such as glucose, cholesterol, and uric acid [15,[18], [19], [20]]. The c-ELISA is the most widely used analytical assay as it is the gold standard for detecting protein biomarkers in disease-related clinical samples [10,[21], [22], [23]]. The c-ELISA produces color signals that are quantitatively correlated with the different concentrations of the sensing target molecules, and those signals can be easily picked up by smartphones; however, smartphone cameras are subject to various camera settings and, more severely, to different environmental lighting conditions [[24], [25], [26], [27]]. Therefore, the accuracy (the ratio between the number of true positives and negatives to the total number of experimental results) and sensitivity (the ratio between the number of true positives to the number of true positives and false negatives) of the smartphone-based colorimetric μPADs are greatly jeopardized, and the smartphone has shown the lowest sensing performance when compared with the standard colorimetric μPADs analysis methods: the RGB sensor and RGB scanner, with or without the use of ambient lighting prevention tools [23,28]. Thus, effectively minimizing the influence of internal camera optics/settings and external ambient light conditions is still highly desired to improve the analytical performance of smartphone-based colorimetric μPADs, thereby facilitating the development of POCT in resource-limited areas.

Most recently, machine learning has been demonstrated as one of the most potent image data processing tools [29,30], and it can be readily implemented on smartphone platforms. However, the smartphone is not powerful enough to run the machine learning algorithms to process the data, which can be wirelessly uploaded to/downloaded from the cloud server before and after the processing [14,31,32]. Several works have applied machine learning algorithms to analyze the colorimetric signal for the lateral flow assays (LFA), and highly reliable qualitative and quantitative results were obtained [[33], [34], [35], [36], [37]]. However, LFA is much simpler than c-ELISA μPADs and usually cannot enable high sensitivity and high specificity biosensing [38]. Machine learning has also been used on smartphone-based colorimetric μPADs and provides higher accuracy [14,[39], [40], [41]]. But they all used conventional machine learning algorithm that requires human feature extraction, further increasing human intervention and hindering its usage by laypersons. Subsequently, two works have combined deep learning algorithms with smartphone paper devices for disease diagnosis, which can automatically extract features and achieve high accuracy, thanks to its higher computation power than conventional machine learning. But in these works, an additional homemade cassette is needed to prevent the ambient light [32,42], which increases the device complexity. Most recently, Ning et al. employed a deep learning algorithm for the colorimetric detection of C-reactive protein (CRP) and provided good accuracy of 96% [43]. However, this work relied on the manual image and data transfer between the smartphone and deep learning servers, which hinders the wide accessibility of the POCT devices. Therefore, a fully automated smartphone-based colorimetric μPAD that is free of the influences of human inventions and ambient lighting conditions is still highly desired and yet to develop.

In this work, we present a deep learning-assisted smartphone colorimetric μPADs platform for rapid, sensitive, and accurate detection of protein markers using c-ELISA. Unlike the previous work, there is no need to manually copy the resultant images extracted from the smartphone to the local computer, and the data transfer can be performed directly from the smartphone. In addition, a deep learning model is used to further avoid manual screening of features. Furthermore, an Android application with a friendly user interface is developed to interact with the whole process, and it enables the fully automated, streamlined manner of “image in, answer out” colorimetric sensing. Here, the complete automation is contributed by the deep learning algorithms. Specifically, we first constructed a comprehensive dataset (number = 2048) using Rabbit IgG as the c-ELISA sensing target on μPADs. Then, the three light sources and two different smartphones were used to enrich the complexity of the dataset for training the algorithm, with the objective of increasing the robustness and adaptability of the algorithm. By training all the images of experimental results under different lighting conditions, the deep learning algorithm can learn itself and eliminate the effect of illumination on the experimental results. In this work, we compared the performance of the four mainstream deep learning algorithms (AlexNet, GoogLeNet, ResNet34, and MobileNet_V2). Among them, GoogLeNet provides the highest accuracy (>97%), which also provides a 4% higher area under curve (AUC) value than that of the conventional smartphone sensing results analysis method that uses curve fitting. The overview of deep learning-assisted ultra-accurate smartphone testing of paper-based c-ELISA is shown in Fig. 1. In this work, we validated the feasibility of our platform by using rabbit IgG protein as a sensing target for c-ELISA, which is the most common sensing target protein and has been widely used for validation experiments in many works [10,23]. In future work, we will apply this approach to clinically relevant biomarkers. To sum up, the unique capabilities of our platform further enhance the advantages of μPAD for general use in resource-poor settings and hold great promise for real-world use.