FormalPara Key Summary Points

FebriDx is a finger prick point-of-care test that detects an antiviral protein.

We compared FebriDx & RT-PCR accuracy to a composite clinical reference standard.

Sensitivity of FebriDx for COVID-19 in hospitalised adults was comparable to RT-PCR.

Sensitivity of FebriDx plus RT-PCR was superior to either test alone

FebriDx may be useful in rapidly identifying patients with COVID-19 who test negative by RT-PCR.

Introduction

The diagnosis of COVID-19 is routinely made using reverse transcription polymerase chain reaction (RT-PCR) on upper respiratory tract samples; however, the suboptimal sensitivity of this technique has been recognised [1]. This is especially relevant in hospitalised patients with COVID-19 pneumonia as they have generally been unwell for 7–10 days prior to presentation [2, 3], and so have lower viral loads in the upper respiratory tract or may even have cleared the virus from this site completely [4]. Strategies to improve on the sensitivity of RT-PCR for detection of COVID-19 have included serial upper respiratory tract swabbing for RT-PCR, chest CT scanning, and the use of SARS-CoV-2 serology [5,6,7]. Unfortunately, all these strategies have drawbacks: serial swabbing is insensitive and associated with long delays, CT scanning is a limited resource in most settings and involves radiation exposure, and serology is unreliable early in the disease [6, 7]. Urgent identification of patients with RT-PCR negative COVID-19 at presentation is needed for infection control purposes, as patients may still represent an infection risk from viral replication in the lower respiratory tract, and for consideration of treatment with antivirals or immunomodulatory agents [8]. The lack of an adequate gold standard test to compare the accuracy of other testing modalities is a major challenge and so use of a composite clinical reference standard, based on radiological and other clinical features, has been suggested as the optimal approach [9, 10], and has been endorsed by the UK National Institute for Health and Care Excellence (NICE) [11].

FebriDx (Lumos diagnostics, Sarasota, Florida, USA) is a CE-marked point-of-care test (POCT) that detects two host response proteins, myxovirus resistance protein A (MxA) and C-reactive protein (CRP), in finger prick blood samples, and was originally designed to distinguish viral from bacterial respiratory infection [12, 13]. MxA is a marker of interferon-induced antiviral host response and, in our previous work, the detection of MxA by FebriDx had high sensitivity for the detection of influenza in hospitalised adults, during influenza season [14]. We subsequently demonstrated that detection of MxA by FebriDx also has high sensitivity for the identification of hospitalised patients with COVID-19 during the first wave of the pandemic, using RT-PCR as the reference standard [15]. The aim of this study was to assess the diagnostic accuracy of RT-PCR, MxA detection by FebriDx and both combined, compared to a composite clinical reference standard in hospitalised patients with suspected COVID-19.

Methods

Study Design, Participants, and Ethics Committee Approval

This study was nested within the CoV-19POC study, a trial assessing the clinical impact of molecular POCT for COVID-19 and the study including the full protocol has been published [16]. Adults presenting to hospital with suspected COVID-19 were enrolled; full details of the inclusion and exclusion criteria can be found in the protocol, linked below. The study was approved by the South Central—Hampshire A Research Ethics Committee: reference 20/SC/0138, on 16 March 2020. The study was conducted in accordance with the Helsinki Declaration of 1964 and its later amendments. This study was prospectively registered with the ISRCTN (ISRCTN14966673) on 18 March 2020. This trial has completed, and the protocol is available at https://eprints.soton.ac.uk/439309/2/CoV_19POC_Protocol_v2_0_eprints.pdf.

Procedures

Participants gave informed written consent prior to any procedures. Combined nose and throat swabs were obtained from patients by research staff and tested by RT-PCR immediately using the QIAstat-Dx platform with the Respiratory SARS-CoV-2 Panel at the point-of-care [16]. The QIAstat-Dx Respiratory SARS-CoV-2 Panel tests for 19 respiratory viruses including SARS-CoV-2, and three atypical bacteria using real-time RT-PCR [17, 18]. In addition, laboratory RT-PCR testing for SARS-CoV-2 on combined nose and throat swabs was performed, in the on-site Public Health England (PHE) microbiology laboratory, using the PHE RdRp and E gene reference assays [16]. Demographic and clinical data was collected at enrolment and outcome data collected retrospectively from case note and electronic hospital systems, using an electronic case report form.

For this sub-study, patients were approached for testing using the FebriDx host response POCT on finger prick blood samples, taken at the same time as nose and throat swabs for RT-PCR. Detailed instructions for use of FebriDx are available at https://www.febridx.com/how-to-use#testing.

The FebriDx test is read after 10 min and generates results in the form of the presence or absence of three lines, assessed by visual inspection: a CRP line (grey), a MxA line (red) and a control line (blue).

Composite Clinical Reference Standard

Patients were classified as having COVID-19 or not on the basis of retrospective review of RT-PCR results, radiological reports, and discharge summaries. Patients were classified as having COVID-19 if they were RT-PCR positive (as defined above), or RT-PCR negative with: either chest X-ray reported as classic/probable COVID-19 (as per British Society of Thoracic Imaging reporting guidelines), or chest CT reported as showing classic/probable changes of COVID-19, or radiology (chest X-ray or CT) reported as indeterminate for COVID-19 with a senior physician discharge diagnosis of COVID-19 and no other explanation for radiological finding [19]. Chest X-rays and CT scans were reported by radiologists and reporting radiographers who were independent of the study. Case review and classification was performed independently by three clinician researchers who were blinded to FebriDx result. Where there was discrepancy between assessments this was adjudicated by a fourth clinician researcher.

Sample Size

The sample size of 500 patients in the parent study was chosen pragmatically on the basis of the availability of the QIAstat-Dx Respiratory SARS-CoV-2 Panel test kits. Although not formalised in the study design, to estimate a sensitivity of 85% to within ± 5% (based on the score method for a 95% confidence interval) with 80% power, 196 positive cases were required. With a prevalence of COVID-19 of 45% in those tested, 456 patients in total were required [16].

Statistical Analysis

Baseline characteristics were summarised for all those recruited to the study, and also presented for those who were SARS-CoV-2 RT-PCR positive, SARS-CoV-2 RT-PCR negative with a clinical/radiological diagnosis of COVID-19, and those who were COVID-19 negative. Measures of sensitivity, specificity, predictive values, and likelihood ratios were calculated for FebriDx MxA alone, RT-PCR alone, and for both combined, compared to the composite clinical reference standard. Sensitivity was compared between RT-PCR, MxA, and the combination of the two using chi-square test for equality of proportions. The 95% confidence intervals (95% CI) were calculated using the ‘exact’ Clopper–Pearson method. Analysis was carried out in Prism version 7.05 (GraphPad Software Inc; La Jolla, California).

Role of Funding Source

The funder of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. All authors had full access to all of the data and the final responsibility to submit for publication.

Results

Between 20 March and 29 April 2020, 500 patient-participants were recruited during the first wave of the pandemic [16]. All 500 patients were approached for FebriDx testing and 22 patients declined or their clinical team felt it was inappropriate for them to have finger prick testing. Three patients had a FebriDx test that failed and could not be repeated. Sixteen had a test that failed initially but produced a valid result on repeating. Tests failed because of blood visibly clotting in the collection tube and not progressing though the device or because the CRP line was not visible when read despite a concurrent serum CRP over 20 mg/L. Overall, 475 patients had a valid FebriDx result. Of these 475 patients, 222 (46.7%) were classified as having COVID-19 on the basis of the composite clinical reference standard. A total of 192 of the 475 (40.4%) patients were RT-PCR positive for SARS-CoV-2 (187 at presentation and five on repeat RT-PCR testing within 7 days of admission) and 30/475 (6.3%) patients were RT-PCR negative but classified as having COVID-19 on the basis of clinical/radiological features. The flow of patients in the study is shown in Fig. 1. Table 1 shows baseline characteristics for all patients and for those RT-PCR positive for SARS-CoV-2, those RT-PCR negative but clinically/radiologically diagnosed with COVID-19, and those who were classified as COVID-19 negative. Patients classified as RT-PCR negative COVID-19 had higher median blood neutrophil counts (9.9 [IQR 7.5–14.8] versus 5.4 [4.0–8.2], difference of 4.5, 95% CI 2.8–6.6; p < 0.0001) and higher median [IQR] levels of biochemical markers of inflammation, including LDH (731 [533–987] versus 540 [475–689], difference of 191, 95% CI 29–292; p = 0.0129), ferritin (504 [245–1213] versus 151 [88–807], difference of 354, 95% CI 50–466; p = 0.0085) and D-dimer (635 [362–1048] versus 388 [230–610], difference of 247, 95% CI 0–413; p = 0.0381), compared to patients with RT-PCR positive COVID-19. In addition, a higher proportion of patients with RT-PCR negative COVID-19 had initial chest X-rays reported as classic/probable COVID-19 compared to patients with RT-PCR positive COVID-19 although this did not reach statistical significance, 21/30 (70%) versus 98/191 (51%), difference of 19%, 95% CI 2–40; p = 0.075.

Fig. 1
figure 1

Flow of participants through the study

Full size image
Table 1 Clinical characteristics of all patients, patients with COVID-19 and a positive RT-PCR, patients with COVID-19 and a negative RT-PCR (with clinical/radiological diagnosis of COVID-19) and COVID-19 negative patients
Full size table

Diagnostic Accuracy

Using the composite reference standard, the prevalence of COVID-19 during the study was 222/475 (46.7%). Compared to the composite clinical reference standard, RT-PCR had a sensitivity of 192/222 (86.5%, 95% CI 81.3–90.7). FebriDx MxA had a sensitivity of 186/222 (83.8%, 95% CI 78.3–88.4) and a specificity of 236/253 (93.3%, 95% CI 89.5–96.0). There was no significant difference in sensitivity between RT-PCR and FebriDx MxA (difference 2.7, 95% CI − 3.9 to 9.3; p = 0.42). The combination of RT-PCR and FebriDx MxA together had a sensitivity of 208/222 (93.7%, 95% CI 89.7–96.5) and a specificity of 236/253 (93.3%, 95% CI 89.5–96.0). The sensitivity of combined RT-PCR and FebriDx MxA detection was superior to the sensitivity of either alone (difference of 7.2, 95% CI 1.6–12.9; p = 0.011 for MxA, and difference of 9.9, 95% CI 4.1–15.9; p = 0.001 for RT-PCR). Table 2 shows the measures of diagnostic accuracy for RT-PCR, FebriDx MxA and the two combined, compared to the composite clinical reference standard. Table S1 (Supplementary Material) shows measures of diagnostic accuracy for FebriDx compared to RT-PCR as the reference standard. The negative predictive value of combined FebriDx and RT-PCR testing was 236/250 (94.4%, 95% CI 91.0–96.6) at a prevalence of 46.7% and was 99.3% at a prevalence of 10%. Table S2 (Supplementary Material) shows the positive and negative predictive values for combined RT-PCR and FebriDx MxA testing at different levels of SARS-CoV-2 prevalence.

Table 2 Diagnostic accuracy measures of RT-PCR, FebriDx MxA, and the composite clinical reference standard for COVID-19
Full size table

Few non-SARS-CoV-2 respiratory viruses were detected during the study. In COVID-19 positive (RT-PCR positive) patients 2/193 (1.0%) had another virus detected 1 × human coronavirus HKU1, 1 × adenovirus (both were FebriDx MxA negative). In COVID-19 positive (RT-PCR negative) patients 2/30 (6.7%) had non-SARS-CoV-2 viruses detected: 2 × rhinovirus (one was FebriDx MxA positive). In COVID-19 negative patients 17/253 (6.7%) had a non-SARS-CoV-2 virus detected: 10 × rhinovirus, 3 × human metapneumovirus, 3 × human coronavirus OC43 and 1 × human coronavirus NL63 (5 were FebriDx MxA positive, 3 with human metapneumovirus and 2 with rhinovirus).

Discussion

This large diagnostic accuracy study shows equivalent sensitivity of the finger prick FebriDx MxA test and upper respiratory tract RT-PCR for the diagnosis of COVID-19 in hospitalised adults using a composite clinical reference standard. The sensitivity of the combination of FebriDx MxA and RT-PCR was superior to that of each test alone. The ability to detect patients with possible RT-PCR negative COVID-19 in near real-time with FebriDx would allow early appropriate infection control and therapeutic decisions without the need for other time-consuming, costly and inaccurate diagnostic strategies such as serial upper respiratory tract swabbing or CT scanning. The high negative predictive value (NPV) associated with a negative FebriDx MxA result, across a range of COVID-19 prevalence rates, allows the confident rejection of COVID-19 in RT-PCR negative patients with an ongoing clinical suspicion of COVID-19, thus allowing de-isolation and other management decisions.

A large retrospective study also conducted by our group has shown that use of FebriDx MxA in the emergency department safely reduced the time that SARS-CoV-2 RT-PCR negative patients spent in high-risk areas alongside RT-PCR positive patients. This demonstrates the practical use of FebriDx in a real-world acute hospital setting [20]. The negative predictive value and positive predictive value of the FebriDx MxA in the implementation study were comparable to this study.

The sensitivity of the FebriDx MxA test is superior to that of the best rapid antigen lateral flow tests for the detection of COVID-19 in patients presenting to hospital [21, 22]. With its ease of use and faster time to generating results, the FebriDx has several advantages over lateral flow tests. Our finding of high FebriDx sensitivity as compared to RT-PCR in adults hospitalised with suspected COVID-19 is supported by other, smaller studies [23,24,25].

Alternative methods of establishing or refuting a diagnosis of COVID-19 in patients who test negative by nose and throat swab RT-PCR include RT-PCR testing of lower respiratory tract specimens obtained by bronchoscopy or endotracheal intubation. However, the process of obtaining these specimens requires specialist resources, and is highly invasive and time-consuming, compared to the FebriDx. Laboratory blood testing for SARS-CoV-2 nucleocapsid antigen by ELISA has been proposed as less complex and less resource-intensive than molecular assays; however, the accuracy of such an approach is not yet established and it requires specialist laboratory facilities and is likely to have a substantially longer turnaround time to results than FebriDx [26].

There were few differences in baseline characteristics between patients with RT-PCR positive and RT-PCR negative (but radiologically/clinically diagnosed) COVID-19 except for their laboratory results on admission. The higher levels of inflammatory markers in patients with RT-PCR negative COVID-19 are likely to relate to the preferential identification of RT-PCR negative COVID-19 in patients with very clinically overt and advanced disease, as supported by the higher proportion of patients with classic radiological changes of COVID-19. The longer length of hospital stay in patients with RT-PCR positive COVID-19 is difficult to explain but there were no differences in other outcome measures including critical care admission and all-cause mortality.

The strengths of the study include that is a large, adequately powered, prospectively recruited study. Owing to its setting in a typical large teaching hospital emergency department and acute admissions unit, and its pragmatic study design, the results are highly generalisable to other acute care settings.

There are limitations to this study including its single centre nature and the findings should be confirmed in multicentre studies. In addition, these results are not generalisable to children, immunocompromised patients, asymptomatic patients, or patients managed in the community with less severe symptoms, although studies in these settings are now warranted. The relative scarcity of non-SARS-CoV-2 respiratory viruses in this study, including influenza and respiratory syncytial virus [3, 16], may have influenced the specificity of FebriDx MxA for COVID-19. As radiological and clinical features are known to be inaccurate for the diagnosis of COVID-19, the composite reference standard itself is likely to be imperfect and may lack sensitivity in select patient groups. As RT-PCR may remain positive for many weeks after COVID-19 infection, RT-PCR positive patients may present to hospital with a subsequent unrelated acute respiratory illness but be included in the RT-PCR positive group in this analysis. However, as this study recruited patients during the first wave of the pandemic in the UK, it is unlikely that patients with prolonged RT-PCR detection were included. Although the QIAstat-Dx Respiratory SARS-CoV-2 panel is highly accurate for the detection of SARS-CoV-2, it may not be as sensitive as some RT-PCR tests with very low lower limits of detection, and therefore our study may have missed a small number of patients with a very low SARS-CoV-2 viral load. Further studies of FebriDx MxA detection of non-SARS-CoV-2 respiratory virus infections during autumn–winter and influenza seasons are warranted. More reliable and sensitive SARS-CoV-2-specific tests than this surrogate test may be available [27]; however, as the FebriDx generates a result in 10 min, requires neither specialist facilities nor laboratory-trained personnel, and is only as invasive as a finger prick, it is likely to be a useful tool for clinicians in rapid clinical decision-making.

In this study, RT-PCR and FebriDx had similar sensitivity for the identification of COVID-19 cases in hospitalised adults and the combination of the two had superior sensitivity to either one alone. Negative predictive value of combination testing was high across a broad range of SARS-CoV-2 prevalence. A strategy of combined point-of-care RT-PCR and host response testing using FebriDx allows the early identification or rejection of patients with RT-PCR negative COVID-19 and may help inform infection control and therapeutic decisions in near real-time.