Deep learning-assisted ultra-accurate smartphone testing of paper-based colorimetric ELISA assays
Graphical abstract
Introduction
Microfluidic paper-based analytical devices (μPADs) have been demonstrated as a promising tool for detecting disease-relevant biomarkers at point-of-care testing (POCT), thanks to their low-cost, ease of operation, and self-driven capillary fluidic flow [[1], [2], [3], [4], [5]]. In the last decade, many biosensing mechanisms, such as electrochemistry, chemiluminescence, fluorescence immunoassay, and colorimetric enzyme-linked immunosorbent assay (c-ELISA) have been developed and realized on μPADs [[6], [7], [8], [9], [10]]. Among them, colorimetric μPADs are particularly suitable for being used in low resources settings, especially when using smartphones as colorimetric reading and analyzing tools, and no further complicated equipment is needed [11,12]. The smartphones feature the broadest user community, a friendly operator interface, and excellent data transfer capability. It is a powerful computation platform capable of fully automating the colorimetric results collection, analysis, and displaying process, using its camera, processor, and screen, respectively [[13], [14], [15]]. Meanwhile, it is usually cheaper than the dedicated medical-grade colorimetric reader [16]. Hence, the smartphone-based colorimetric μPADs well meet all the criteria outlined by the World Health Organization (WHO) for POCT devices in low-resource settings, abbreviated as “ASSURED”; affordable, sensitive, specific, user-friendly, rapid and robust, equipment-free, and delivered [17]. And it can significantly improve healthcare worldwide by providing low-cost, timely, accurate disease screening.
Recently, smartphone-based colorimetric μPADs have been extensively studied and developed to detect various disease biomarkers such as glucose, cholesterol, and uric acid [15,[18], [19], [20]]. The c-ELISA is the most widely used analytical assay as it is the gold standard for detecting protein biomarkers in disease-related clinical samples [10,[21], [22], [23]]. The c-ELISA produces color signals that are quantitatively correlated with the different concentrations of the sensing target molecules, and those signals can be easily picked up by smartphones; however, smartphone cameras are subject to various camera settings and, more severely, to different environmental lighting conditions [[24], [25], [26], [27]]. Therefore, the accuracy (the ratio between the number of true positives and negatives to the total number of experimental results) and sensitivity (the ratio between the number of true positives to the number of true positives and false negatives) of the smartphone-based colorimetric μPADs are greatly jeopardized, and the smartphone has shown the lowest sensing performance when compared with the standard colorimetric μPADs analysis methods: the RGB sensor and RGB scanner, with or without the use of ambient lighting prevention tools [23,28]. Thus, effectively minimizing the influence of internal camera optics/settings and external ambient light conditions is still highly desired to improve the analytical performance of smartphone-based colorimetric μPADs, thereby facilitating the development of POCT in resource-limited areas.
Most recently, machine learning has been demonstrated as one of the most potent image data processing tools [29,30], and it can be readily implemented on smartphone platforms. However, the smartphone is not powerful enough to run the machine learning algorithms to process the data, which can be wirelessly uploaded to/downloaded from the cloud server before and after the processing [14,31,32]. Several works have applied machine learning algorithms to analyze the colorimetric signal for the lateral flow assays (LFA), and highly reliable qualitative and quantitative results were obtained [[33], [34], [35], [36], [37]]. However, LFA is much simpler than c-ELISA μPADs and usually cannot enable high sensitivity and high specificity biosensing [38]. Machine learning has also been used on smartphone-based colorimetric μPADs and provides higher accuracy [14,[39], [40], [41]]. But they all used conventional machine learning algorithm that requires human feature extraction, further increasing human intervention and hindering its usage by laypersons. Subsequently, two works have combined deep learning algorithms with smartphone paper devices for disease diagnosis, which can automatically extract features and achieve high accuracy, thanks to its higher computation power than conventional machine learning. But in these works, an additional homemade cassette is needed to prevent the ambient light [32,42], which increases the device complexity. Most recently, Ning et al. employed a deep learning algorithm for the colorimetric detection of C-reactive protein (CRP) and provided good accuracy of 96% [43]. However, this work relied on the manual image and data transfer between the smartphone and deep learning servers, which hinders the wide accessibility of the POCT devices. Therefore, a fully automated smartphone-based colorimetric μPAD that is free of the influences of human inventions and ambient lighting conditions is still highly desired and yet to develop.
In this work, we present a deep learning-assisted smartphone colorimetric μPADs platform for rapid, sensitive, and accurate detection of protein markers using c-ELISA. Unlike the previous work, there is no need to manually copy the resultant images extracted from the smartphone to the local computer, and the data transfer can be performed directly from the smartphone. In addition, a deep learning model is used to further avoid manual screening of features. Furthermore, an Android application with a friendly user interface is developed to interact with the whole process, and it enables the fully automated, streamlined manner of “image in, answer out” colorimetric sensing. Here, the complete automation is contributed by the deep learning algorithms. Specifically, we first constructed a comprehensive dataset (number = 2048) using Rabbit IgG as the c-ELISA sensing target on μPADs. Then, the three light sources and two different smartphones were used to enrich the complexity of the dataset for training the algorithm, with the objective of increasing the robustness and adaptability of the algorithm. By training all the images of experimental results under different lighting conditions, the deep learning algorithm can learn itself and eliminate the effect of illumination on the experimental results. In this work, we compared the performance of the four mainstream deep learning algorithms (AlexNet, GoogLeNet, ResNet34, and MobileNet_V2). Among them, GoogLeNet provides the highest accuracy (>97%), which also provides a 4% higher area under curve (AUC) value than that of the conventional smartphone sensing results analysis method that uses curve fitting. The overview of deep learning-assisted ultra-accurate smartphone testing of paper-based c-ELISA is shown in Fig. 1. In this work, we validated the feasibility of our platform by using rabbit IgG protein as a sensing target for c-ELISA, which is the most common sensing target protein and has been widely used for validation experiments in many works [10,23]. In future work, we will apply this approach to clinically relevant biomarkers. To sum up, the unique capabilities of our platform further enhance the advantages of μPAD for general use in resource-poor settings and hold great promise for real-world use.
Section snippets
Reagents and materials
Potassium periodate (>99.5%, 80,106,916) was purchased from Sinopharm Chemical Reagent Co., Ltd (Shanghai, China). Rabbit IgG (I5006) was purchased from Sigma-Aldrich (Shanghai, China). Alkaline phosphatase (ALP) conjugated anti-rabbit IgG (A0239), 5-bromo-4-chloro-3-indolyl phosphate/nitro blue tetrazolium (BCIP/NBT) alkaline phosphatase color development kit (C3206) was purchased from the Beyotime Institute of Biotechnology (Haimen, China). 10 × phosphate-buffered salines (PBS, G4207),
Lighting conditions disturb c-ELISA results
In this work, a direct c-ELISA was used to detect the concentration of Rabbit IgG in PBS. We prepared seven different concentrations (N = 16 for each concentration) in 10-fold dilutions (6.7 pM–6.7 μM) ofrabbit IgG as the sensing target for c-ELISA, and PBS buffer (0 pM) was used as a negative control. After each test, we captured images of the same microwells detection zones under different lighting conditions with two different smartphones (iPhone 11 and Huawei P40 Pro), respectively. The
Conclusion
This paper reports a deep learning-assisted smartphone platform for ultra-accurate detection of protein markers by μPAD c-ELISA. The platform uses a deep learning algorithm trained from images acquired using two different smartphones under eight lighting conditions to improve the platform's robustness to lighting variations and camera optics. Unlike existing platforms, our platform can fully automatically extract c-ELISA features from the original images regardless of smartphone brand, time,
CRediT authorship contribution statement
Sixuan Duan: Conceptualization, Writing – original draft, Writing – review & editing. Tianyu Cai: Visualization. Jia Zhu: Investigation. Xi Yang: Methodology. Eng Gee Lim: Conceptualization. Kaizhu Huang: Conceptualization. Kai Hoettges: Conceptualization. Quan Zhang: Visualization. Hao Fu: Methodology. Qiang Guo: Methodology. Xinyu Liu: Supervision. Zuming Yang: Writing – review & editing. Pengfei Song: Conceptualization, Writing – review & editing.
Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgment
The authors thank the financial support from the programs of Natural Science Foundation of the Jiangsu Higher Education (20KJB460024, 22KJB460033), Jiangsu Science and Technology Programme - Young Scholar (BK2020041995), Jiangsu Province High-level Innovation and Entrepreneurship Talent Plan (2020–30803), XJTLU Key Programme Special Fund – Exploratory Research Programme (KSF-E-39), and XJTLU Research Development Fund (RDF-18-02-20). The authors also acknowledge the financial support from Xi'an
References (63)
- et al.
Fabrication techniques for microfluidic paper-based analytical devices and their applications for biological testing: a review
Biosens. Bioelectron.
(2016) - et al.
ZnSe quantum dot based ion imprinting technology for fluorescence detecting cadmium and lead ions on a three-dimensional rotary paper-based microfluidic chip
Sensor. Actuator. B Chem.
(2020) - et al.
A novel 3D paper-based microfluidic electrochemical glucose biosensor based on rGO-TEPA/PB sensitive film
Anal. Chim. Acta
(2020) - et al.
Detection methods and applications of microfluidic paper-based analytical devices
Trends Anal. Chem.
(2018) - et al.
Machine learning-based colorimetric determination of glucose in artificial saliva with different reagents using a smartphone coupled μPAD
Sens. Actuator B-Chem.
(2021) - et al.
Smartphone-based mobile biosensors for the point-of-care testing of human metabolites
Mater. Today Bio
(2022) - et al.
Smartphone-based, sensitive μPAD detection of urinary tract infection and gonorrhea
Biosens. Bioelectron.
(2015) - et al.
Point-of-care colorimetric analysis through smartphone video
Sensor. Actuator. B Chem.
(2019) - et al.
A paper-based microfluidic platform with shape-memory-polymer-actuated fluid valves for automated multi-step immunoassays
Microsyst. Nanoeng.
(2019) - et al.
Development of a smartphone-based lateral-flow imaging system using machine-learning classifiers for detection of Salmonella spp
J. Microbiol. Methods
(2021)
Quantifying colorimetric tests using a smartphone app based on machine learning classifiers
Sensor. Actuator. B Chem.
Machine learning-based colorimetric determination of glucose in artificial saliva with different reagents using a smartphone coupled μPAD
Sensor. Actuator. B Chem.
Comparative analysis of image classification algorithms based on traditional machine learning and deep learning
Pattern Recogn. Lett.
Deep learning for biological image classification
Expert Syst. Appl.
Diagnostics for the Developing World: Microfluidic Paper-Based Analytical Devices
Microfluidic paper-based analytical devices: from design to applications
Chem. Rev.
A paper-based microfluidic analytical device with a highly integrated on-chip valve for autonomous ELISA
Proc. IEEE Int. Conf. Micro Electro Mech. Syst. (MEMS), IEEE
Recent developments in paper-based microfluidic devices
Anal. Chem.
A flow chemiluminescence paper-based microfluidic device for detection of chromium (III) in water
J. Innov. Opt. Health. Sci.
Colorimetric determination of acidity constant using a paper-based microfluidic analytical device
Chem. Pap.
Angew. Chem.
Smartphone-based colorimetric detection system for portable health tracking
Anal. Methods
Smartphone-based analytical biosensors
Analysis
Low-cost, user-friendly, all-integrated smartphone-based microplate reader for optical-based biological and chemical analyses
Anal. Chem.
Mapping the Landscape of Diagnostics for Sexually Transmitted Infections: Key Findings and Recommendations
Three-dimensional paper-based microfluidic analysis device for simultaneous detection of multiple biomarkers with a smartphone
Biosensors
Smartphone-based colorimetric detection systems for glucose monitoring in the diagnosis and management of diabetes
Analysis
A colorimetric enzyme-linked immunosorbent assay (ELISA) detection platform for a point-of-care dengue detection system on a lab-on-compact-disc
Sensors
A portable microfluidic paper-based device for ELISA
Optimization of a paper-based ELISA for a human performance biomarker
Anal. Chem.
Simple telemedicine for developing regions: camera phones and paper-based microfluidic devices for real-time, off-site diagnosis
Anal. Chem.
Cited by (8)
Smartphone-based paper microfluidic detection implementing a versatile quick response code conversion strategy
2024, Sensors and Actuators B: ChemicalCNN-assisted accurate smartphone testing of μPAD for pork sausage freshness
2024, Journal of Food EngineeringReview of paper-based microfluidic analytical devices for in-field testing of pathogens
2023, Analytica Chimica ActaMachine learning-assisted image label-free smartphone platform for rapid segmentation and robust multi-urinalysis
2024, Analytical and Bioanalytical Chemistry