Evaluation of the Practicability of Biosynex Antigen Self-Test COVID-19 AG+ for the Detection of SARS-CoV-2 Nucleocapsid Protein from Self-Collected Nasal Mid-Turbinate Secretions in the General Public in France

Abstract

Due to their ease-of-use, lateral flow assay SARS-CoV-2 antigen-detecting rapid diagnostic tests could be suitable candidates for antigen-detecting rapid diagnostic self-test (Ag-RDST). We evaluated the practicability of the Ag-RDST BIOSYNEX Antigen Self-Test COVID-19 Ag+ (Biosynex Swiss SA, Freiburg, Switzerland), using self-collected nasal secretions from the turbinate medium (NMT), in 106 prospectively included adult volunteers living in Paris, France. The majority of the participants correctly understood the instructions for use (94.4%; 95% confidence interval (CI): 88.3–97.4), showing a great ability to perform the entire self-test procedure to obtain a valid and interpretable result (100%; 95% CI: 96.5–100), and demonstrated the ability to correctly interpret test results (96.2%; 95% CI: 94.2–97.5) with a high level of general satisfaction. About one in eight participants (# 15%) needed verbal help to perform or interpret the test, and only 3.8% of test results were misinterpreted. By reference to multiplex real-time RT-PCR, the Ag-RDST showed 90.9% and 100% sensitivity and specificity, respectively, and high agreement (98.1%), reliability (0.94), and accuracy (90.9%) to detect SARS-CoV-2 antigen. Taken together, our study demonstrates the high usability and accuracy of BIOSYNEX Antigen Self-Test COVID-19 Ag+ for supervised self-collected NMT sampling in an unselected adult population living in France.

1. Introduction

Coronaviruses are RNA viruses that cause common respiratory infections in humans, and they are an infectious agent responsible for mild respiratory tract infections [1]. Since December 2019, the coronavirus “SARS-CoV-2”, which causes severe bilateral pneumonia, has spread throughout the world, starting in Wuhan, China. The world continues to face the 2019 coronavirus pandemic (COVID-19). By October 2021, the World Health Organization (WHO) had noted more than 200,000,000 confirmed cases of COVID-19 infection and nearly 5,000,000 deaths due to this virus.

Testing is one of the principal strategies to fight infectious diseases, as it is the gateway to prevention and treatment strategies. Furthermore, massive screening of people potentially infected with SARS-CoV-2 is essential to limit the spread of SARS-CoV-2 [2,3].

While currently recommended nucleic acid amplification tests, such as real-time reverse transcription-polymerase chain reaction assays, remain the gold-standard cornerstone for the diagnosis of SARS-CoV-2 infection [4,5], immunological methods can also be used to detect viral antigens [5,6,7]. Thus, antigen-detecting rapid diagnostic tests (Ag-RDT) enable new testing strategies for SARS-CoV-2 infection diagnosis and control due to their short turn-around time and ease-of-use [6,8,9]. However, most SARS-CoV-2 Ag-RDTs rely on nasopharyngeal sampling, and their use is hampered by the necessity to deploy qualified healthcare workers and protective equipment to collect nasopharyngeal secretions, perform the tests and interpret the test results. In addition, a nasopharyngeal swab is often considered uncomfortable, which limits the ability of frequently performing screening tests from this type of nasopharyngeal sampling [10].

Alternative sampling methods, such as nasal mid-turbinate (NMT) sampling (swab inserted about 2–3 cm into nostril parallel to the palate until resistance is met at turbinates), is well tolerated and can be performed by the person himself either unsupervised at home or supervised on-site [10,11,12,13]. NMT sampling was equivalent to nasopharyngeal sampling for a WHO-listed SARS-CoV-2 Ag-RDT, which lays the foundation for its use as a self-sampling technique under supervision [12,13,14,15]. Widespread use of self-testing for SARS-CoV-2 infection could help improve the control of the spread of the SARS-CoV-2 epidemic [16,17,18,19,20,21]. Previous experience with HIV self-testing showed that a self-test can be performed perfectly well by a layperson and gives reliable and accurate results, and that HIV self-testing is considered as an acceptable and useful tool to improve HIV testing coverage worldwide to improve HIV epidemic control [22,23]. Lateral flow tests for the SARS-CoV-2 antigen are appropriate for use as an antigen-detecting rapid diagnosis self-test (Ag-RDST) because of their low cost, ease of use, and prompt result [12,14,24].

Recent reports have established the achievability of NMT self-sampling under supervision [12,14]. Data on the clinical performance of self-testing with Ag-RDT are, however, limited to comparisons with quantitative real-time RT-PCR (rtRT-PCR) detection of SARS-CoV-2 RNA [25,26].

This study aimed to evaluate the usability and clinical performance in the field of the Ag-RDST BIOSYNEX Antigen Self-Test COVID-19 Ag+ (Biosynex Swiss SA, Freiburg, Switzerland; reference 859271) for COVID-19 antigenic testing from self-collected NMT secretions in adults living in the Paris region during the third wave of the COVID-19 epidemic in France. For the first time to our knowledge, the usability of this novel Ag-RDST was evaluated according to a systematic approach based on our previous experience in assessing HIV and SARS-CoV-2 self-testing using capillary blood samples [27,28,29,30].

2. Materials and Methods

2.1. SARS-CoV-2 Antigen-Detecting Rapid Diagnostic Test for Self-Testing

The CE IVD-labeled Ag-RDST BIOSYNEX Antigen Self-Test COVID-19 Ag+ (Biosynex Swiss SA) is a rapid qualitative membrane-based immunochromatographic test that uses highly sensitive monoclonal antibodies to detect the SARS-CoV-2 N nucleocapsid protein in nasopharyngeal secretions sample. The Ag-RDST BIOSYNEX Antigen Self-Test COVID-19 Ag+ is packaged for individual use with seven components placed in an aluminum pouch containing the test cassette, a diluent tube pre-filled with buffers (sample, reagent, and absorbent), a dropped teat, swab (Jiangsu Changfeng Medical Industry Co., Yangzhou, China; CE marking: CE0197), and a sachet of desiccant (to be thrown away). The box has a cut-out hole to place the pre-filled diluent tube. The test includes a reaction membrane containing colloidal gold particles conjugated to monoclonal antibodies and secondary antibodies directed against the N protein of SARS-CoV-2 that are placed inside a plastic cassette.

The self-test was performed according to manufacturer instruction. In brief, the top of the pouch is removed following the indent, in order to remove the cassette. The seal is removed from the pre-filled diluent tube, which is placed in the cut-out hole in the box. Then, the swab is removed from its packaging. The swab should be vertically inserted into a nostril, until it starts to resist (at approximately 3 cm deep). The swab is rolled 5 times along the inner walls of each nostril, to ensure that the nasal secretions and cells are collected. Afterwards, the swab is inserted into the pre-filled diluent tube and is rotated for at least 10 s whilst the end of the swab is pressed against the bottom of the extraction tube. It should be mentioned that as much secretion as possible should be collected from the swab by “squeezing it out” using the edge of the tube, by pressing the swab against the edge of the tube several times. Then, the swab is removed from the diluent tube, which is closed by the dropper teat. Using the dropper, 4 drops of solution are slowly deposited into well “S”.

If SARS-CoV-2 antigens are present in the sample, a red line appears in the test line area (T). The result is negative when no red line appears in the test line area. An internal procedural control is assured by a red line appearing in the control line area (C). Visual interpretation of results is performed 10 min after. The results should not be read after 15 min.

The test must be performed at room temperature (15–30 °C) and in an environment free from excessive humidity, within 1 h after opening.

The analytical performances of the self-test Ag-RDT BIOSYNEX Antigen Self-Test COVID-19 Ag+ are given in the instructions for use (Figure S1), with a sensitivity of 96.0% (95% confidence intervals (CI): 90.9–98.7%) and a specificity of 99.2% (95%CI: 95.4–99.9%).

The simplified instructions for use of the BIOSYNEX Antigen Self-Test COVID-19 Ag+ were comprised of an easy-to-read leaflet in French and other languages (English, German, Dutch, and Italian) in A3 format color printing. The printable instruction for use in French is depicted as supporting information (Figure S1). The online instruction in the video for use was available online from YouTube (in French: https://www.youtube.com/watch?v=R9fxagzoFSs&list=PLQqQgOWEz169GgQECgZIMW04qcvnpTC1-&index=1 (accessed on 3 November 2021); in English: https://www.youtube.com/watch?v=MVDFQZJZTig&list=PLQqQgOWEz169GgQECgZIMW04qcvnpTC1-&index=3 (accessed on 3 November 2021)).

The BIOSYNEX Antigen Self-Test COVID-19 Ag+ has obtained the CE-IVD mark in 2021, and is sold in many European countries (Austria, Belgium, Denmark, France, Germany, Italy, Sweden, and The Netherlands) (around 800,000 tests per month). Biosynex Swiss SA (Freiburg, Switzerland) is the Own Brand Labeler which has designed this self-test, and ensures all the packaging, and national quality controls of manufacturing.

2.2. Practicability Evaluation

The practicability evaluation of the BIOSYNEX Antigen Self-Test COVID-19 Ag+ is a prospective study performed during the third wave of the COVID-19 epidemic between April and May 2021 by recruitment of adult volunteers aged ≥18 years receiving testing for SARS-CoV-2, at the Laboratoire Paris XV, Paris, France, consisting of face-to-face, paper-based, semi-structured, and self-administrated questionnaires.

2.3. Study Design and Recruitment of Participants

In France, the molecular test for SARS-CoV-2 RNA detection is carried out voluntarily or on medical prescription and the test is fully covered by the health insurance [31,32,33]. The study site offered SARS-CoV-2 testing to anyone in the community who wanted testing by conventional molecular diagnosis with a nasopharyngeal swab for suspected COVID-19, for travel, as part of a preoperative assessment, exposure as a contact case of an individual infected with SARS-CoV-2, or control of past infection with SARS-CoV-2 within 30 days. A questionnaire capturing demographic information (sex and age), reasons for testing, and current and past–14-day symptoms (headache, fatigue, fever, or upper or lower respiratory symptoms) for symptomatic patients was administered to all participants. Asymptomatic individuals were defined as those not reporting any of these symptoms in the past month.

For rtRT-PCR, a health care professional collected nasopharyngeal secretions in one nostril, using a flocked swab. The nasopharyngeal swab sample was discharged in 1000 μL of physiological serum (NaCl 0.9%), and stored at +4 °C before processing. The laboratory gives the diagnosis of SARS-CoV-2 RNA detection within 24 h by SMS, internet or in the form of a paper report.

After the nasopharyngeal swab sampling for SARS-CoV-2 RNA molecular testing, the objectives of the practicability and analytical evaluation of the BIOSYNEX Antigen Self-Test COVID-19 Ag+ were explained to the participants. Eligible participants were of an age ≥18 years, wanted to use on-site for the first time for a self-test Ag-RDT using nasal secretions for SRAS-CoV-2 detection, were capable to speak and read in French, and gave their informed consent to participate in the study. Anyone already trained in the use of rapid diagnostic tests, such as doctors, nurses and technicians or biologists, were excluded.

2.4. Practicability Study Outcomes

The practicability evaluation was divided into four sub-studies carried out by trained health care professionals, based on previously acquired experience from WHO recommendations for evaluating the practicability of HIV and COVID-19 self-tests [27,28,29,30]. While all participants were included in the study concerning the understanding of labeling (sub-study 1), they were randomized into two groups for studies concerning manipulation of the test (sub-study 2) and evaluating the interpretation of COVID-19 self-test results (sub-study 3), using block randomization of 4. A cross-over was then made, allowing the participants of sub-study 2 to participate in sub-study 3 and vice versa. Finally, all participants were asked to fill the satisfaction questionnaires (sub-study 4) (Figure 1).

2.5. Data Collection and Procedures

Paper-based, structured questionnaires were completed by each participant in order to collect their socio-demographic characteristics, their medical history, to assess their understanding of the instructions for use, and to obtain their opinions or levels of satisfaction about the practicability of the BIOSYNEX Antigen Self-Test COVID-19 Ag+. All data relating to the observation of manipulation and the interpretation of test results were recorded on the standardized sheets by the observers.

2.5.1. Sub-Study 1

Comprehension of labeling. After receiving a brief explanation of the objectives and conduct of the study, the participants were asked to give their oral informed consent. In a private setting, the participants had the choice between a paper-based instruction for use and a video-based instruction for use (Biosynex autotest antigénique COVID-19 Ag+; available at: https://youtu.be/R9fxagzoFSs (Accessed on: 3 November 2021)), which they were asked to read or watch and understand independently. After declaring that the instructions for use were understood, participants were asked to complete a questionnaire to assess their understanding. This questionnaire included 11 questions concerning the key information of the instruction for use with closed answers (true, false, or do not know). That key information included the manipulation of the kit, the interpretation of test results, and the consequence of the test results. The participants who correctly answered all 11 questions were considered to have correctly understood the instructions for use.

After this survey, participants were randomized into two groups for evaluation on performing the self-test and the interpretation of test results, using a sealed randomization envelope.

2.5.2. Sub-Study 2

Observation of manipulation. In a private setting supervised by an observer, each participant received a box containing the BIOSYNEX Antigen Self-Test COVID-19 Ag+. Participants were then asked to carry out the self-test by themselves in front of a trained observer. The observer was responsible for recording the respect or not of each step, appeal for verbal assistance (mimicking telephone support), difficulty, and errors on a standardized sheet. The successful performance of the SARS-CoV-2 antigenic self-test was conditioned by the presence of the control band on the test strip, and the results of the Ag-RDT were read and recorded independently by participants and observers.

2.5.3. Sub-Study 3

Interpretation of test results. In a private setting supervised by an observer, seven standardized test results including three positive tests (positive, weak positive, and strong positive), two negative tests, and two invalid tests were proposed to the participants for interpretation after successive random selection of five tests (Figure 2).

2.5.4. Sub-Study 4

Satisfaction questionnaire. Finally, the participants fulfilled the satisfaction questionnaire concerning their experiences with the BIOSYNEX Antigen Self-Test COVID-19 Ag+.

2.6. Laboratory Procedure

Nucleic acid extraction was performed from 300 μL thawed aliquots of the eluted nasopharyngeal swab samples using automated nucleic acid extraction EX3600 extractor (Liferiver & Shanghai ZJ Bio-Tech Co., Shanghai, China) with the Liferiver^® Viral RNA Extraction kit (Liferiver & Shanghai ZJ Bio-Tech Co.; reference NE-0044). The BIOSYNEX AmpliQuick^® SARS-CoV-2 (Biosynex Swiss SA) is a multiplex rtRT-PCR assay for the detection of SARS-CoV-2 RNA, according to the manufacturer’s instructions. The kit is composed of a 96-well microplate pre-filled with the master mix containing dNTPs, MgCl2, fluorescent primers and probes, Taq polymerase and reverse transcriptase enzymes, and reaction buffer. The assay can simultaneously detect 2 coronavirus target genes, including the SARS-like (including SARS-CoV-2, SARS-CoV, and bat SARS-like coronavirus) conserved region of envelope protein gene (E), and RNA-dependent RNA polymerase gene (ORF1ab of RdRP gene), providing individual cycle threshold (Ct) values for each target gene. This assay was performed on the CFX96™ Real-Time PCR Detection System (Bio-Rad Laboratories, Hercules, CA, USA), according to the manufacturer’s instructions.

The results of SARS-CoV-2 RNA detection obtained by the multiplex rtRT-PCR were considered as the reference.

2.7. Statistical Analysis

The minimum sample size was calculated using Fisher’s test with the assumption of 80% expected frequency corresponding to the success of practicability (sub-studies 1, 2, and 3), 5%-alpha risk and 20%-beta risk. Thus, the minimum sample size was estimated to be 105 participants.

All data (Excel file) were analyzed using SPSS 20.0 (Chicago, IL). Descriptive statistical analyses were carried out using mean, standard deviation, median, interquartile range, frequency, and percent. The 95% confidence interval (CI) was calculated using Wilson score bounds [34]. The results of SARS-CoV-2 RNA detection by the multiplex rtRT-PCR were used as the reference standard to estimate the sensitivity and specificity of the study self-test Ag-RDT, with corresponding 95% CI. The percent agreement and concordance between study self-test and reference test corresponded to the observed proportion of identical results between Ag-RDT compared to rtRT-PCR detection. The reliability between the study self-test Ag-RDT and the multiplex molecular detection of SARS-CoV-2 RNA was estimated by Cohen’s κ coefficient [35], and the degree of agreement was determined as ranked by Landlis and Koch [36]. Youden’s J index (J = sensitivity + specificity − 1) was used to estimate the accuracy of the study self-test [37].

2.8. Ethics Statement

The study is part of the continuous laboratory quality improvement program for the diagnosis of SARS-CoV-2 infection, in accordance with European guidelines on the accreditation of medical biology laboratories [38]. The dataset was completely anonymous. In addition, all participants fulfilled the recommendations of the Haute Autorité de Santé (Saint-Denis, France) of 21 April 2021, on the use of self-tests for SARS-CoV-2 detection, i.e., in asymptomatic persons over 15 years of age, within the framework of large-scale iterative screening campaigns, as well as in the private setting for individuals [39].

3. Results

3.1. Study Population

A total of 116 individuals were assessed for eligibility, but 10 were excluded because they were less than 18 years old (n = 4), and not consenting (n = 6) (Figure 1). Finally, 106 were successfully enrolled in all practicability sub-studies and clinical performance evaluation.

The demographic characteristics and medical history of study participants are shown in

Table 1. Demographic characteristics and medical history of the 106 study participants.

Variable	Items	Number (%)
Sex
	Male	38 (35.8)
	Female	68 (64.2)
Age (years)
	18–20	4 (3.7)
	21–59	96 (90.6)
	≥60	6 (5.7)
Educational level
	No diploma	17 (16.1)
	College level	40 (37.7)
	High school level	22 (20.7)
	Post-graduate level	27 (25.5)
Previous use of other
	self-tests (not for COVID-19)
	Yes	13 (12.3)
	No	93 (87.7)
Asymptomatic for COVID-19
	Yes	76 (71.7)
	Contact case	10 (9.4)
Symptomatic for COVID-19 #
	Yes	30 (28.3)
Day from the onset
	0–1	7 (6.6)
	2–4	20 (18.8)
	5–7	3 (2.9)
	>7	0 (0.0)

^# Participants who reported having at least one of the following major symptoms associated or not with minor symptoms were considered to have a COVID-19 symptom: fever, fatigue, dry cough, anosmia, and dyspnea. Minor symptoms were: pain, nasal congestion, runny nose, sore throat or diarrhea.

. Overall, 68 (64.2%) were female. The median age was 40 years, and the majority of participants were aged between 21 and 59 years. The majority (83.9%) of participants possessed at least a college or higher education level. The majority (87.7%) of them possessed no previous experience of self-testing for diseases other than COVID-19.3.1. Main reasons for testing were air travel (n = 41; 38.7%), case-contact exposure of an individual infected with SARS-CoV-2 (n = 10; 9.4%), suspected COVID-19 (n = 30; 28.3%), pre-operative assessment (n = 18; 17.0%), and control of SARS-CoV-2 infection in the 30 days preceding (n = 7; 6.6%). At the time of testing, the majority (71.7%) of participants had reported no symptoms of COVID-19 in the past month, including all contact-cases from SARS-CoV-2-infected individuals. Around one-third of participants reported at least one COVID-19-compatible symptom. Among symptomatic patients, the median time of symptom duration before sampling was 4 days (range, 0–7 days).

3.2. Practicability Evaluation

3.2.1. Sub-Study 1

This sub-study evaluated the ability of the 106 study participants to understand the instructions for use of the BIOSYNEX Antigen Self-Test COVID-19 Ag+. The majority (n = 76; 71.7%) of participants preferred to use the paper-based instructions whereas 30 (28.3%) participants used the video-based instructions. The analytical results of the evaluation questionnaire are shown in

Table 2. Analytical results of the evaluation questionnaire concerning the ability of the 106 study participants to understand the instruction for use of the BIOSYNEX Antigen Self-Test COVID-19 Ag+ (Biosynex Swiss SA).

Comprehension of Labeling Checklist *	Participants’ Responses
	True [Number (%)]	False [Number (%)]	Do Not Know [Number (%)]
Q1: “The same swab is used to take both nostrils one after the other”	106 (100)	0 (0.0)	0 (0.0)
Q2: “The nasal swab is inserted into one nostril until silk resistance is reached (approximately 3 cm)”	104 (98.2)	0 (0.0)	2 (1.8)
Q3: “The nasal swab must touch the walls of the nostril”	100 (94.3)	1 (0.9)	5 (4.7)
Q4: “The swab has to be turned 6 times in the tube filled with diluent (during at least 10 s)”	106 (100)	0 (0.0)	0 (0.0)
Q5: “The swab should be properly wrung out in the tube to let a maximum of nasal sample”	100 (94.3)	1 (0.9)	5 (4.7)
Q6: “A nipple must be placed on the tube before depositing the 4 drops of solution in the well “S””	104 (98.2)	1 (0.9)	1 (0.9)
Q7: “Lack of band by test results is interpreted as a negative test”	1 (0.9)	102 (96.2)	3 (2.8)
Q8: “Lack of control band by test results should be interpreted as an invalid test”	101 (95.3)	2 (1.8)	3 (2.8)
Q9: “Having symptoms of fever, cough and aches, the negativity of BIOSYNEX Antigen Self-test COVID-19 Ag+ confirm the lack of COVID-19 infection”	2 (1.8)	103 (97.2)	1 (0.9)
Q10: “If the test is positive, it means that I have to see the doctor and additional tests will be required to confirm the diagnosis”	106 (100)	0 (0.0)	0 (0.0)
Q11: “The BIOSYNEX Antigen Self-test COVID-19 Ag+ detects the presence of the virus”	104 (98.2)	0 (0.0)	2 (1.8)
Labeling index for understanding (% [95% CI]) £	96.8 [91.5–98.8]
Correct understanding of the instruction for use (n; % [95% CI]) #	100; 94.3 [88.3–97.3]

* Overall, 76 (71.7%) participants preferred to use the paper-based instruction whereas 30 (28.3%) participants used the video-based instruction; ^£ The labeling index for understanding was defined as the mean of the correct answers for each question; # The participants who correctly answered all 11 questions were considered to have correctly understood the instructions for use. CI: Confidence interval; COVID-19: Coronavirus disease 2019; Q: Question.

. Overall, 100 (94.4%; 95% CI: 88.3–97.4) participants correctly understood the instructions for use, thus correctly answering all 11 questions. The labeling index for participant understanding, which measured the mean of the correct answers for each question was 96.8% (95% CI: 91.5–98.8). The question concerning the necessity that the nasal swab must touch the inner walls of the nostrils and that concerning the lack of control band showed the highest rate (5.6% and 4.6%, respectively) of incorrect responses.

3.2.2. Sub-Study 2

This sub-study evaluated the ability of participants to use the study self-test Ag-RDT BIOSYNEX Antigen Self-Test COVID-19 Ag+ in a supervised setting. Overall, all participants (100%; 95% CI: 96.5–100) performed the self-test and succeeded in obtaining a valid test result with an overall usability index of 99.1% (95% CI: 94.9–99.9) (

Table 3. Analytical results of the manipulation observation concerning the ability of Table, 106 study participants to correctly use each step of the BIOSYNEX Antigen Self-Test COVID-19 Ag+ (Biosynex Swiss SA) autonomously or with verbal help.

Usability Checklist µ	Successful Manipulation		Need for Verbal Help
	Yes [Number (%)]	No [Number (%)]	Yes [Number (%)]
Did the participant open the box?	106 (100)	-	-
2. Did the participant read the instructions for use (written part and drawn part)?	106 (100)	-	-
3. Did the participant wash his hands?	104 (98.2)	2 (1.8)	1 (0.9)
4. Did the participant easily identify the different components of the kit?	106 (100)	-	-
5. Did the participant properly remove the test cassette from the aluminum pouch?	105 (99.1)	1 (0.9)	-
6. Did the participant open the lid and place the tube on the rack correctly?	104 (98.2)	2 (1.8)	2 (1.8)
7. Did the participant introduce his nasal swab correctly in both nostrils?	106 (100)	-	8 (7.5)
8. Did the participant turn the swab 6 times in the tube?	101 (95.3)	5 (4.7)	3 (2.8)
9. Did the participant wait for 1 min after turning the swab?	104 (98.2)	2 (1.8)	1 (0.9)
10. Did the participant properly wring out the tightness of the swab?	104 (98.2)	2 (1.8)	6 (5.7)
11. Did the participant correctly add the nipple to the tube?	106 (100)	-	-
12. Did the participant correctly deposit 4 drops into the “S” well of the test cassette?	106 (100)	-	-
13. Did the participant observe the correct time after deposition of the sample drops into the “S” well of the cassette?	106 (100)	-	-
14. Did the participant obtain an interpretable result at the end of the process? #	106 (100)	-	3 (2.8)
Usability index (% [95% CI]) £	99.1 [94.9–99.8]
Correct use without difficulties, errors, or helps (n; % [95% CI])	92; 86.8 [79.1–92.0]
Correct use with help (n; % [95% CI])	14; 13.2 [8.0–20.9]
Average time of manipulation (minutes [SD])	8.1 [1.3]

^µ A total of 30 (28.3%) participants used the video-based instruction for use; among them the usability index was estimated to 100% without any difficulties, errors, or help; ^# The result was considered interpretable when a control strip was readable after the migration time recommended by the manufacturer; in the present series, 21 (27.3%) participants had a positive self-test result; ^£ The usability index was defined as the mean of the correct answers for each question. CI: Confidence interval; SD: Standard deviation.

). A total of ninety-two (86.8%; 95% CI: 79.1–92.0) participants correctly used the self-test without any difficulties, errors, or help, whereas 14 (13.2%; 95% CI: 8.0–20.9) had asked for verbal help. The introduction of the nasal swab in both nostrils, and the necessity to turn the swab 6 times in the diluent tube were the steps requiring the most frequent verbal help in 7.5%, and 5.7%, respectively (Table 3). Interestingly, all participants (n = 30; 28.3%) using the video instructions performed the self-test easily (usability index of 100%) without any difficulties, errors, or help. Overall, the mean time of antigenic self-test performance (since the opening of the box until the migration step) was 8.1 (SD: 1.3) minutes.

3.2.3. Sub-Study 3

This sub-study evaluated the ability of participants to read and interpret the COVID-19 antigenic self-test results after successive random selection of five tests from a panel of seven standardized tests. The results are depicted in Figure 3. Overall, 530 standardized tests were read and interpreted by the 106 participants, including 216 positive, 166 negative, and 148 invalid test results. A total of 510 (96.2%; 95% CI: 94.2–97.5) tests were correctly interpreted, whereas 20 (3.8%; 95% CI: 2.8–5.8) tests were misinterpreted.

Misinterpretation occurred in 0.5% (n = 1) of positive tests (result weakly positive falsely interpreted as negative), in 1.8% (n = 3) of negative tests (result negative falsely interpreted as positive) and in 10.8% (n = 16) of invalid tests falsely interpreted as positive. Cohen’s κ coefficient between the results of reading by participants and the expected results was 0.95, demonstrating an excellent concordance.

3.2.4. Sub-Study 4

This sub-study assessed the post-test satisfaction of participants concerning the instructions for use (sub-study 1), performing the COVID-19 antigenic self-test (sub-study 2), and the interpretation of test results (sub-study 3). The results of the questionnaire are shown in

Table 4. Items and results of the satisfaction questionnaire and concerning the instruction notice (sub-study 1), the performing of the BIOSYNEX Antigen Self-Test COVID-19 Ag+ (Biosynex Swiss SA) (sub-study 2), and the interpretation of test results.

Satisfaction Questionnaire	Post-Test Satisfaction [Number (%)]
How did you find the understandability of instructions for use of self-test?
Very easy	61 (57.5)
Rather easy	39 (36.8)
Rather difficult	6 (5.7)
Very difficult	0 (0)
How did you find the overall implementation of the self-test kit?
Very easy	53 (50.0)
Rather easy	43 (40.6)
Rather difficult	9 (8.5)
Very difficult	1 (0.9)
How did you find the sample collection with nasal swabbing in both nostrils?
Very easy	37 (34.9)
Rather easy	59 (55.7)
Rather difficult	8 (7.7)
Very difficult	2 (1.8)
How did you find the use of the component of the self-test kit and the sample deposition into the “S” well of the test cassette?
Very easy	53 (50.0)
Rather easy	44 (41.4)
Rather difficult	8 (7.7)
Very difficult	1 (0.9)
How did you find the reading of strips after migration?
Very easy	89 (83.9)
Rather easy	12 (11.4)
Rather difficult	5 (4.7)
Very difficult	0 (0)
How did you find the interpretation of self-test results?
Very easy	95 (89.6)
Rather easy	11 (10.4)
Rather difficult	0 (0)
Very difficult	0 (0)
How did you find the video on YouTube to help with performing the self-test? (n = 30)
Very easy	24 (80.0)
Rather easy	6 (20.0)
Rather difficult	0 (0)
Very difficult	0 (0)
How did you find the overall handling of the self-test component?
Very easy	80 (75.5)
Rather easy	17 (16.0)
Rather difficult	9 (8.5)
Very difficult	0 (0)
How did you find your ability to surmount the difficulties encountered?
Very easy	100 (94.4)
Rather easy	6 (5.6)
Rather difficult	0 (0)
Very difficult	0 (0)
You have just performed the nasal COVID-19 self-test, do you repeat it if necessary?
Yes	83 (78.3)
No	23 (21.7)
You just took the COVID-19 nasal self-test; would you recommend it to someone else?
Yes	99 (93.4)
No	7 (6.6)

. The majority of items were considered very easy or easy. The use of the video on YouTube to help with the realization of the self-test was appreciated as very easy by 80% of participants, having chosen this option. Regarding their ability to overcome the difficulties encountered during the COVID-19 self-test, all (100%) participants responded by saying that it was easy (94.4% very easy; 5.6% rather easy). The majority of participants would repeat such antigenic self-test if necessary (78.3%) and would recommend it to someone else (93.4%).

3.3. Analytical Performance by Lay Users

Analytical performance by lay users. Using rtRT-PCR as the standard, two false-negative antigenic self-test results occurred, among specimens from asymptomatic (n = 1) or symptomatic (n = 1) participants. No false-positive antigenic self-test result was observed. The mean level of viral excretion assessed by the mean Ct values of E and RdRP genes by the reference rtRT-PCR was higher in symptomatic than asymptomatic participants (23.1 ± 2.8 versus 28.2 ± 2.7 arbitrary units; p = 0.021 by Mann–Whitney U test). Overall, the Ag-RDT BIOSYNEX Antigen Self-Test COVID-19 Ag+ showed high sensitivity and specificity of 90.9% and 100%, respectively, and high or almost perfect agreement (98.1%), reliability (0.94), and accuracy (90.9%) to detect SARS-CoV-2 (

Table 5. Analytical performance of the BIOSYNEX Antigen Self-Test COVID-19 Ag+ for the qualitative detection of the N protein of SARS-CoV-2 using 106 prospectively collected nasopharyngeal swab samples by reference rRT-PCR#.

				BIOSYNEX COVID-19 Ag BSS §
		Mean Genes Ct *,# (Median; Range)	N	TN (n)	FN (n)	TP (n)	FP (n)	Sensitivity (% [95% CI]) µ	Specificity (% [95% CI])	Agreement a	Concordance b	Youden’s J Index c
PCR	Symptomatic (n = 30)	23.1 (19.5–30.0)	16	0	1	15	0	93.8 (79.3–98.4)	100 (88.6–100)	96.7 (83.4–99.4)	0.93 (0.78–0.98)	93.8 (79.3–98.4)
		Negative	14	14	0	0	0
	Asymptomatic (n = 76)	28.2 (25.0–33.0)	6	0	1	5	0	83.3 (73.4–90.0)	100 (95.2–100)	98.7 (92.0–99.6)	0.91 (0.82–0.96)	83.3 (73.4–90.0)
		Negative	70	70	0	0	0
	Whole study population (n = 106)	26.2 (19.5–33.0)	22	0	2	20	0	90.9 (83.9–95.0)	100 (96.5–100)	98.1 (93.4–99.5)	0.94 (0.88–0.97)	90.9 (83.9–95.0)
		Negative	84	84	0	0	0

^§ Nasopharyngeal samples in one nostril were collected with a flocked swab for each volunteer participant by trained healthcare personnel (nurses or biologists), while the swab contained in the BIOSYNEX Antigen Self-Test COVID-19 Ag+ kit was further used for nasal secretions self-sampling by the participant himself. Molecular testing as well COVID-19 antigen detection were carried out on fresh samples; ^a Agreement = TP + TN/TP + FP + TN + FN, expressed in percentage; ^b The Cohen’s κ coefficient calculation was used to estimate the concordance [35] and interpreted according the Landis and Koch scale [36], as follows: < 0 as indicating no agreement, 0–0.20 as slight, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1 as almost perfect concordance; ^c The accuracy of the test BIOSYNEX COVID-19 Ag BSS to correctly diagnose SARS-CoV-2 infection was estimated by Youden’s J index (J = sensitivity + specificity − 1) [37]; ^µ 95% confidence intervals in brackets were calculated by using the Wilson score bounds; ^# The CE IVD-marked BIOSYNEX AmpliQuick^® SARS-CoV-2 (Biosynex, Strasbourg, France) constituted the reference multiplex rRT-PCR for SARS-CoV-2 RNA detection. This assay detects two target genes of SARS-CoV-2 (E and RdRP genes); * Mean genes Ct = mean of Ct for E gene and Ct for RdRP gene. Ct: Cycle threshold; FN: False negative; FP: False positive; NA: Not attributable; rRT-PCR: real-time reverse transcription-polymerase chain reaction; TP: True positive; TN: True negative.

).

4. Discussion

We herein report on our recent experience during the third COVID-19 epidemic peak period in France of the usability and clinical performance in the field of a novel Ag-RDST for COVID-19 antigenic testing from self-collected NMT secretions in adult volunteers living in the Paris region. The majority of participants were able to correctly understand the instructions for use of the self-test, to obtain a reliable result and to interpret the final test results. About one out of eight (#15%) participants needed verbal help, and only 3.8% of test results were misinterpreted. Taken together, our study demonstrates the feasibility and accuracy of COVID-19 self-testing from self-collected NMT sampling in an unselected adult population in a supervised setting. These observations lay the foundations for the potential large-scale use of COVID-19 Ag-RDST in lay adults, at least Europeans, to complete the arsenal of available diagnostic tests for SARS-CoV-2.

4.1. Practicability Evaluation

The overall usability of the BIOSYNEX Antigen Self-Test COVID-19 Ag+ was evaluated by four successive sub-studies that allow a good analysis of all the steps of the test realization, from the understanding of the instructions of use with the identification of the test components, to the realization of the Ag-RDST and the final interpretation of the results. The satisfaction sub-study is also essential to assess the acceptability of the self-test, which will necessarily have to be repeated in an epidemic context.

The sub-study 1 tested whether the instructions for use can be read and understood by all participants. Our findings showed that 94.4% of participants correctly answered all 11 questions, indicating a generally correct understanding of the key messages delivered by the instructions for use of the BIOSYNEX Antigen Self-Test COVID-19 Ag+, with an overall rate of good responses of 96.8%. These satisfactory results may be explained in part by a sufficient education level of the majority of study participants. Indeed, previous experience with HIV self-diagnosis has shown that an insufficient level of education is a great challenge to understanding the instructions for use [22,40]. Although systematic reviews and meta-analysis have shown that HIV self-diagnosis can be successfully conducted by untrained users without help [22], our observations underscore the need to supplement the traditional paper instructions with other educational tools such as a short video, which was preferred by 28.3% of study participants for better understanding of the instructions for use. These findings are reminiscent of previous WHO recommendations for HIV self-testing, that all self-testers should have the option of accessing assistance via telephone, the internet, or additional instructions such as videos, animations, or diagrams [41]. The difficulty in explaining that the swab must touch the bottom of the inner wall of the nostril for correct NMT sampling must be emphasized. This difficulty, encountered in 5.6% of participants, could be the cause of false negative results.

In sub-study 2, all study participants carried out the COVID-19 Ag-RDST and succeeded in obtaining a valid test result with an overall usability index estimated at 99.1%. The time of realization of the test was particularly short (8.1 min). Some difficulty in the correct introduction of the nasal swab in both nostrils and the necessity to turn the swab 6 times in the diluent tube were the principal reported concerns encountered and were the most common reason for oral help. All participants using the video instructions performed the self-test easily. These features underscore the importance of video instructions, when available. The use of a hotline could also provide direct remote assistance.

The ability to correctly read and interpret the results of the self-test is considered a sensitive step in self-testing [42]. This refers not only to the visual ability related to good visual acuity when reading and interpreting the results, but also to the intensity of the bands to be read on the strip. In our series, the rate (96.2%) of correct interpretation of COVID-19 Ag-RDST results was high, as usually observed for example with HIV self-test using a similar cassette [28,29,30]. In one case, the misinterpreted test results concerned a weak positive band, read as negative. This difficulty in reading some weak positive bands and definitively interpreting the test results can occur in non-professional users as well as in users trained in professional testing [43]. The misreading of a weakly positive strip is obviously restrictive because the subject falsely believes he is negative. However, the instructions for use clearly state that a negative test does not mean that the Ag-RDST is not contagious. The misreading of invalid tests interpreted as positive should in principle be corrected by further COVID-19 molecular testing, which must be systematic in case of positivity, as indicated in the instructions for use.

The post-test answers to the satisfaction questionnaire concerning the instructions for use (sub-study 1), performing the self-test (sub-study 2), and the interpretation of the results (sub-study 3), showed that the large majority of the COVID-19 self-testing steps were considered easy or very easy by the majority of participants, as previously reported for HIV self-testing using a similar rapid test cassette [28,30]. Watching a video before the test was particularly appreciated. The willingness to repeat the use of the study Ag-RDST when necessary is important for widespread use, because the Ag-RDT for SARS-CoV-2 detection must be repeated to identify infectious individuals, or for private personal use so that it becomes routine.

Taken together, our observations highlight in adult profanes the usability of the BIOSYNEX Antigen Self-Test COVID-19 Ag+ using painless NMT self-sampling as a novel approach to assess SARS-CoV-2 infection by using Ag-RDT test and self-interpretation of the results in a supervised setting. The feasibility of nasal swabs self-collection in the community was previously emphasized for the diagnosis of respiratory virus infection, such as influenza A and B and other common respiratory viruses, including common coronaviruses [44,45], and these swabs appear to be both easy to perform [42,43,44,46,47,48] and well accepted [46,48]. Self-collection of nasal secretions for respiratory viruses offers potential significant benefits by reducing the requirement for personal protective equipment, limiting exposure of patients and staff to infection, increased convenience and access for patients and timeliness of a sample receipt [45,48,49]. In the present series, the handling of the test itself did not pose any particular problem, as previously demonstrated for other self-tests with similar cassettes [28,29,30,50]. NMT self-sampling was acceptable and achievable over a range of education levels, and the majority of participants had a preference for this method over health care collection, likely because of the ability of patients to control the comfort level of nasal collection better than a trained collector can, as has previously been reported for other respiratory viruses [46,47,48]. Finally, the interpretation of the two bands on the strip of the study Ag-RDST was generally correct, as previously shown for rapid diagnostic tests of comparable forma in lay users from general adult population [28,29].

4.2. Analytical Performance by Lay Users of the BIOSYNEX Antigen Self-Test COVID-19 Ag+

The analytical performance of the study Ag-RDST was evaluated by reference to multiplex rtRT-PCR for SARS-CoV-2 RNA detection as gold standard in a real-life community setting. In this evaluation, the sensitivity of the BIOSYNEX Antigen Self-Test COVID-19 Ag+ was lower among specimens from asymptomatic persons (83.3%) than among specimens from symptomatic persons (93.8%). Specificity (>99.0%) was high in specimens from both asymptomatic and symptomatic groups. The prevalence of having SARS-CoV-2 rtRT-PCR positive test results in this population was relatively high (20.7% overall; 7.9% for asymptomatic participants and 53.3% for symptomatic participants). The high viral load of SARS-CoV-2 across the upper and lower respiratory tracts, including nasal sites [41,47,48,51] and sensitive molecular techniques may explain the equivalent sensitivity of nasal self-collection to health care collection samples in COVID-19 patients.

The study Ag-RDST fulfilled the current WHO’s recommendations for a screening Ag-RTD stating that, at minimum, Ag-RDTs would need to correctly identify significantly more cases than they would miss (sensitivity ≥80%) and would have very high specificity (≥97–100%) [52]. Furthermore, analytical performances must be of a comparable order, as those participants in our study of Ag-RDT were previously reported for some Ag-RDTs in lateral flow immunoassay format [9,53,54,55,56,57,58,59,60,61], whereas several studies have reported much lower sensitivity levels contrasting with always high specificity [6,62,63,64,65]. In addition, the Ag-RDST BIOSYNEX Antigen Self-Test COVID-19 Ag+ fulfilled the current recommendations of the Haute Autorité de Santé (Saint-Denis, France) for a screening Ag-RTD, stating that, at minimum, Ag-RDTs would need to correctly identify significantly more cases than they would miss in symptomatic patients (sensitivity ≥80%) as well as in asymptomatic individuals (sensitivity ≥50%) and should have very high specificity (≥99%) [66].

Our results clearly show that the analytical performances of the study Ag-RDST were better in the event of a high viral load, i.e., in the case of significant viral excretion, especially in symptomatic individuals. These observations confirm the differential interest of Ag-RDT for the detection of SARS-CoV-2 antigens depending on the level of viral load of the sample analyzed, underlined by several authors [67,68,69], and international [52] and national [66,70] recommendations. As stated for Ag-RDT [52,66,70], Ag-RDSTs may be used at best to detect SARS-CoV-2 infected symptomatic individuals suffering from COVID-19-like symptoms with high viral loads, thus highly contagious individuals.

Previous studies to date have looked at the performance of unobserved as well as supervised self-collected specimens for molecular detection of SARS-CoV-2 [45,71,72,73,74,75,76,77]. Thus, Tu and colleagues compared SARS-CoV-2 detection using a variety of swab types self-collected (under health care professional observation) by 530 symptomatic child and adult outpatients (or their parents or guardians), with nasopharyngeal swabs collected by health care professionals as the comparator gold standard [72]. Of the nasopharyngeal swabs collected by the health care professional, 51 (9.6%) were positive. Mid-turbinate self-collected specimens showed high sensitivity of 96.2% (87.0% to 100%) [72]. McCulloch and colleagues have shown that unsupervised home mid-nasal swab collection was comparable to clinician-collected nasopharyngeal swab collection for molecular detection of SARS-CoV-2 in symptomatic patients, particularly those with higher viral loads [71]. The high analytical performance of self-collected samples, including NMT samples, for the molecular detection of common respiratory viruses was similarly reported [44,45,46,47,48,78]. A meta-analysis of nine studies comparing self-sampling and health care worker sampling for influenza testing reported a pooled sensitivity of 87% and specificity of 99% for self-collection [79].

Finally, the performances of supervised self-collected specimens for detection of SARS-CoV-2 using a WHO-listed SARS-CoV-2 Ag-RDT was evaluated in adult volunteers in Charité hospital, Universitätsmedizin Berlin, Germany [12,13,14,15]. Thus, nasal sampling (including self-sampling) assessed against nasopharyngeal sampling led to comparable performance using SARS-CoV-2 Ag-RDT [12,13,15]. Klein and colleagues extended these studies by evaluating the clinical performances of a supervised, self-collected NMT swab and a professional-collected naso-pharyngeal swab, using Panbio™ Ag-RDT (distributed by Abbott) by reference to molecular testing [14]. Participants were able to reliably carry out the NMT-self-sampling. A SARS-CoV-2 infection was diagnosed by rtRT-PCR in 45 of 290 participants (15.5%). Comparing the NMT and naso-pharyngeal sampling, the positive percent agreement of the Ag-RDT was 88.1% (95% CI 75.0–94.8%). The sensitivity of Panbio™ Ag-RDT was between 84.4% and 88.9%. Specificity was >99.0% for NMT and naso-pharyngeal sampling. The sensitivity of the Panbio™ Ag-RDT in participants with high viral load (>7 log10 SARS-CoV-2 RNA copies/mL) was 96.3% (95% CI 81.7–99.8%) for both NMT and naso-pharyngeal sampling. Supervised NMT self-sampling yielded comparable results to NP sampling. Overall, our findings and the Klein’s study demonstrate that NMT self-sampling leads to results comparable to nasopharyngeal sampling when using a convenient Ag-RDT. The possible reduction in SARS-CoV-2 viral load present in the nasal region compared to the nasopharyngeal region may be counterbalanced by the ease-of-sampling of nasal self-collection, as hypothesized [13]. In the same setting, Nikolai and colleagues further showed that professional anterior nasal and NMT sampling were of equivalent accuracy for a SARS-CoV-2 Ag-RDT in ambulatory symptomatic adults, but did not evaluate anterior nasal self-sampling for Ag-RDST [13]. Interestingly, the Centers for Disease Control and Prevention (Atlanta, GA, USA) have added recently both self-sampling for analysis in a reference laboratory and home, self NMT-sampling as an acceptable alternative to professional nasopharyngeal-sampling in their guidance for SARS-CoV-2 testing [11]. Direct COVID-19 self-tests and home tests that have received Emergency Use Authorization (EUA) status from the U.S. Food and Drug Administration (FDA) and are on the market, and are even available in common pharmacies in the USA. Oral fluid-based self-tests have also been authorized by the FDA under an EUA, although further clinical studies are warranted on the sensitivity of saliva as sample material for COVID-19 self-testing.

Other technologies under development could be used as a self-test. For example, loop-mediated isothermal amplification (LAMP) is a DNA amplification method that allows rapid and sensitive detection of a specific gene. LAMP merged with reverse transcription (RT-LAMP) has been successfully used for the detection of several respiratory RNA viruses, including SARS-CoV-2 [80]. Recently, a low cost, ease of use, paper-based device has been developed for extraction-free detection of SARS-CoV-2 in whole saliva using RT-LAMP with a colorimetric response visible to the human eye, that could be used as a self-test or in home diagnostics because of its simplicity [81].

4.3. Strengths and Limitations

Our study is original by highlighting both the usability assessed by several complementary sub-studies and the clinical performance of a novel NMT self-sampling-based Ag-RDST to diagnose SARS-CoV-2 infection in a real-setting in Paris during the third COVID-19 epidemic in France. Using written and illustrated instructions, participants were able to easily carry out NMT-self-sampling. Furthermore, procedural sampling-deviations might be reduced by video instructions. NMT collection was performed just after health care professional collection, allowing accurate comparisons. All samples for routine molecular diagnosis of SARS-CoV-2 infection were tested via the same rtRT-PCR assay. However, the study has some limitations. First, there was only one inclusion site, that could be the basis for allocation bias during randomization. Second, the presence of an observer may lead to a bias in our observations concerning the participants’ ability to perform the tests and to interpret the results. Third, participants who performed self-sampling were predominantly young and educated adults. The study was however limited by the lack of adolescents or children included. Fourth, the low sample size of included participants could reduce the study’s power, especially to evaluate the analytical performance of the Ag-RDST, as previously shown for test at a Tier 1 performance level (e.g., sensitivity ≥90% and specificity ≥95%) [82]. Finally, the study Ag-RDST characteristics might be different depending on whether an individual had previously tested positive, or has knowledge on self-testing for other conditions.

5. Conclusions

Ag-RDST may be used to detect SARS-CoV-2 infected symptomatic individuals suffering from COVID-19-like symptoms with high viral loads and has the potential to determine highly contagious individuals [52,66,70]. Large screening of populations with active SARS-CoV-2 infection is a possible way to break chains of transmission to limit the current pandemic and deconfine societies. While repeated population screening of asymptomatic individuals can be used to limit the spread of SARS-CoV-2, the speed of reporting is much more important than sensitivity, because the results concerning the screening of infected and asymptomatic individuals must always be known quickly [18]. Thus, despite a lower sensitivity to detect infection, Ag-RDST for SARS-CoV-2 antigen detection can be an important tool for screening because of their quick turnaround time and lower costs [67,68]. The rapid realization of Ag-RDST can help limit the transmission of SARS-CoV-2 by more quickly identifying infectious people to isolate, especially when used in the setting of serial testing strategies.

The need for health services has grown worldwide during the COVID-19 pandemic. Innovative ways to address this crisis are required. Self-collection of nasal swabs for SARS-CoV-2 antigenic testing offers an acceptable and reliable alternative to health worker collected samples. Thus, self-swab collection associated with Ag-RDST presents several advantages, which are even more important at a time of global health crisis, including accessibility outside of the health care system, minimizing personal protective equipment use, limiting exposure of patients and staff to infection, and providing a more comfortable patient experience. Self-collection for SARS-CoV-2-infected or COVID-19 patients is well-accepted, safe, and scalable in the pandemic setting, permitting widespread testing of both asymptomatic and symptomatic participants early in the illness and the potential for prompt self-quarantine and contract tracing [16]. The high sensitivities of Ag-RDST, especially in symptomatic patients, alleviate concerns of increased false negatives in this context. As societies reopen, expansion of testing is become critical for preventing a global resurgence in COVID-19. Nasal self-sampling with Ag-RDST has the potential to play a pivotal role in increasing testing access across the broader population.