Skip to main content
Advertisement
  • Loading metrics

Standardized evaluation of Zika nucleic acid tests used in clinical settings and blood screening

  • Mars Stone ,

    Roles Conceptualization, Data curation, Methodology, Supervision, Validation, Writing – original draft, Writing – review & editing

    MStone@vitalant.org

    Affiliations Vitalant Research Institute, San Francisco, California, United States of America, Department of Laboratory Medicine, University of California San Francisco, San Francisco, California, United States of America

  • Sonia Bakkour,

    Roles Resources, Validation, Writing – review & editing

    Affiliations Vitalant Research Institute, San Francisco, California, United States of America, Department of Laboratory Medicine, University of California San Francisco, San Francisco, California, United States of America

  • Eduard Grebe,

    Roles Data curation, Formal analysis, Investigation, Writing – review & editing

    Affiliations Vitalant Research Institute, San Francisco, California, United States of America, Department of Laboratory Medicine, University of California San Francisco, San Francisco, California, United States of America

  • Devy M. Emperador,

    Roles Conceptualization, Funding acquisition, Methodology, Supervision, Writing – review & editing

    Affiliation Pandemic Threats Program, FIND, Geneva, Switzerland

  • Camille Escadafal,

    Roles Methodology, Supervision, Validation, Writing – review & editing

    Affiliation Pandemic Threats Program, FIND, Geneva, Switzerland

  • Xutao Deng,

    Roles Resources, Validation, Writing – review & editing

    Affiliation Vitalant Research Institute, San Francisco, California, United States of America

  • Honey Dave,

    Roles Data curation, Investigation, Project administration

    Affiliation Vitalant Research Institute, San Francisco, California, United States of America

  • Cassandra Kelly-Cirino,

    Roles Conceptualization, Funding acquisition, Supervision, Writing – review & editing

    Affiliation Elizabeth Glaser Pediatric AIDS Foundation, Geneva, Switzerland

  • Eve Lackritz,

    Roles Conceptualization, Funding acquisition, Methodology, Writing – review & editing

    Affiliation World Health Organization, Geneva, Switzerland

  • Diana P. Rojas,

    Roles Supervision, Writing – review & editing

    Affiliation World Health Organization, Geneva, Switzerland

  • Graham Simmons,

    Roles Methodology, Validation, Writing – review & editing

    Affiliations Vitalant Research Institute, San Francisco, California, United States of America, Department of Laboratory Medicine, University of California San Francisco, San Francisco, California, United States of America

  • Ingrid B. Rabe,

    Roles Conceptualization, Methodology, Supervision, Writing – review & editing

    Affiliation World Health Organization, Geneva, Switzerland

  • Michael P. Busch

    Roles Conceptualization, Supervision, Writing – review & editing

    Affiliations Vitalant Research Institute, San Francisco, California, United States of America, Department of Laboratory Medicine, University of California San Francisco, San Francisco, California, United States of America

Abstract

Early detection of Zika virus (ZIKV) transmission within geographic regions informs implementation of community mitigation measures such as vector reduction strategies, travel advisories, enhanced surveillance among pregnant women, and possible implementation of blood and organ donor screening or deferral. Standardized, comparative assessments of ZIKV assay and testing lab performance are important to develop optimal approaches to ZIKV diagnostic testing and surveillance. We conducted an expanded blinded panel study to characterize and compare the analytical performance of fifteen diagnostic and blood screening ZIKV NAT assays, including detection among single- and multiplex assays detecting ZIKV, dengue virus (DENV) and chikungunya virus (CHIKV). A 300 member blinded panel was constructed, consisting of 11 serial half-log dilutions ranging from ~104 to 10−1 genome equivalents/mL in 25 replicates each of the Tahitian Asian ZIKV isolate in ZIKV-negative human serum. Additionally, clinical samples from individuals with DENV-like syndrome or suspected ZIKV infection in Brazil were evaluated. The majority of assays demonstrated good specificity. Analytical sensitivities varied 1–2 logs, with a substantially higher limit of detection (LOD) in one outlier. Similar analytical sensitivity for ZIKV RNA detection in singleplex and multiplex assays of the Grifols and ThermoFisher tests were observed. Coefficient of Assay Efficiency (CE), calculated to characterize assays’ RNA extraction and amplification efficiency, ranged from 0.13 for the Certest VIASURE multiplex and 0.75 for the Grifols multiplex assays. In general, assays using transcription mediated amplification (TMA) technology had greater CE compared to assays using conventional PCR technology. Donor screening NAT assays were significantly more sensitive than diagnostic RT-qPCR assays, primarily attributable to higher sample input volumes. However, ideal assays to maximize sensitivity and throughput may not be a viable option in all contexts, with other factors such as cost, instrumentation, and regulatory approval status influencing assay availability and selection, particularly in resource constrained settings.

Author summary

Having standardized, comparative studies of Zika virus (ZIKV) tests are important for implementing optimal diagnostic testing and disease surveillance. Detection of ZIKV nucleic acid is ideal for confirming infection; however, knowing the sensitivity and specificity of a test is needed for selecting the right test in a given laboratory. We looked at the analytical performance of 15 diagnostic and blood screening ZIKV molecular tests, including tests that detect ZIKV only and tests that detect ZIKV plus other arboviruses. We found donor screening NAT assays were significantly more sensitive than diagnostic RT-qPCR assays, mostly due to the higher sample input volumes. We also found no difference in analytical sensitivity between tests that only detected ZIKV compared to tests that detected ZIKV and related viruses. The ideal molecular test will depend on the testing context, with other factors such as cost, instrumentation, and regulatory approval status influencing assay availability and selection.

Introduction

Zika virus (ZIKV) spread rapidly throughout most of Latin America during 2015 and early 2016, resulting in severe neurological consequences in fetuses and adults. Although ZIKV is mostly transmitted through the bite of Aedes mosquitoes, [1] other routes of transmission have been identified. [18] Of particular public health concern, vertical transmission during pregnancy can cause Congenital ZIKV Syndrome in a proportion of cases (CZS) [913] and this can occur whether maternal ZIKV infection is symptomatic or asymptomatic. In addition, ZIKV has the potential for transfusion transmission (TT) with a high proportion of documented ZIKV RNAemia in donors during the 2013–2014 French Polynesia outbreak and the 2016 outbreak in the Americas [14,15] and several cases of probable transfusion transmission reported in Brazil. ZIKV ribonucleic acid (RNA) and infectious ZIKV has been found in whole blood, [15], urine, [3] breast milk, [16] saliva and semen [17,18].

Timely public health interventions and appropriate clinical management depend on accurate detection of acute ZIKV infections. Early detection of ZIKV transmission activity within geographic regions serve to guide implementation of community mitigation measures such as vector reduction strategies, travel advisories, enhanced surveillance among pregnant women, and possible implementation of blood and organ donor screening, donor deferral, or pathogen reduction techniques in blood products.

Most ZIKV infections are asymptomatic or mildly symptomatic with non-specific symptoms, limiting the ability of presenting clinical cases to assess ZIKV infection rates/incidence. Testing for the presence of ZIKV RNA is a more accurate and specific means of diagnosing ZIKV infections during outbreaks and, given the risk for transfusion transmission, may be implemented for screening the blood supply. ZIKV antibody testing is impacted by extensive antibody cross-reactivity between ZIKV and dengue viruses (DENV) and other antigenically similar flaviviruses and detects seroreactivity after acute viremia is resolved and thus more limited in value. For this reason, the US Food and Drug Administration (FDA), [19] European Center for Disease Control (ECDC) [20] and the World Health Organization (WHO) [21] have recommended nucleic acid technology (NAT) tests for detection of acute ZIKV infection, and particularly for blood donor screening [22,23].

When the first cases of ZIKV infection in the Americas were reported in 2015 there were no commercial assays available to test for the presence of ZIKV RNA or antibodies. Government, academic laboratories and industry responded rapidly to this emerging threat to develop sensitive and specific assays, including high throughput assays essential to ensure a safe and adequate blood supply should donor screening be recommended. However, NAT performance can vary widely depending on assay characteristics, thus comparative assessments of assay and testing lab performance relative to a reference standard is critical.

Although ZIKV transmission in the Americas has greatly declined since late 2017, the inter-epidemic period was identified as a critical time to advance research, development, and availability of ZIKV diagnostics to continue public health programs in endemic areas and prepare for future outbreaks. Standardized, comparative assessments of performance of different ZIKV diagnostic assays and testing labs are important for clinicians, public health organizations, and emergency response planners to develop optimal approaches to ZIKV diagnostic testing and surveillance.

At the beginning of the ZIKV epidemic, with the impending mandate for ZIKV screening of blood donations in Puerto Rico and US states, there was an urgent need to evaluate the performance of newly developed, donor NAT screening assays to be implemented under IND authorization, relative to other diagnostic ZIKV assays. We executed a study to compare the performance of blood screening assays demonstrating their excellent sensitivities and appropriateness for use, but that study was not powered for significant precision and did not include many ZIKV RNA assays that were developed for diagnostic use. [24]

We therefore executed an expanded blinded panel study to robustly characterize and compare the analytical performance of diagnostic and blood screening NAT assays for ZIKV RNA using well-characterized and precisely quantified viral stocks to generate serially diluted virus panels. The panels were sufficiently powered to establish analytical sensitivity and precise estimation of 50% and 95% limits of detection (LOD50 and LOD95) of molecular assays for ZIKV, as well as multiplexed assays for ZIKV, DENV, and CHIKV virus [15].

Subsequent to initial execution of the expanded blinded panel study, a number of other ZIKV NAT assays received approval from regulatory agencies. The WHO ZIKV Task Force sought to advance global capacity for ZIKV preparedness and response through support of FIND, the global alliance for diagnostics, to advance diagnostics for emerging global health threats, including ZIKV and other arboviral diseases. The scope of work for the ZIKV Diagnostics Evaluation Project encompassed a standardized evaluation of currently available immunologic and molecular diagnostic tests for ZIKV infection, and the molecular evaluation built off our previous work to evaluate performance of blood screening and diagnostic assays. The results were compiled from both phases and compared across testing labs to evaluate the relative sensitivities of molecular assays currently used in various settings including clinical diagnosis, research and blood donor screening.

Methods

Ethics statement

This work was approved by the University of Sao Paulo Medical School Ethical Review Board (CAAE number 89342418.1.0000.0065). Since samples were anonymized prior to testing and participants underwent no additional procedures, individual informed consent was deemed unnecessary for virus detection.

Panel development

A 300-member blinded ZIKV analytical performance panel was constructed consisting of 11 serial half-log dilutions in 25 replicates each of the Tahitian Asian ZIKV isolate (which was chosen as the WHO international standard) with initial concentrations of ZIKV RNA quantified by RT-qPCR [25]. Serial dilutions were prepared using defibrinated ZIKV-negative human serum (Gemini Biosciences) resulting in panels with ZIKV viral loads (VLs) ranging from ~104 to 10−1 genome equivalents/mL. The dilutions were performed in a single procedure at VRI in order to generate a sufficient number of 1mL aliquots of each dilution to generate identical frozen panels with a total of 25 uniquely labelled vials for each member panel to perform 25 replicate runs of the panel; the panel also included 25 replicate negative control diluent samples. The 300-member panels with unique identifiers on each tube were stored at -80°C and shipped on dry ice until testing. Panels were sent to manufacturers, research and commercial laboratories experienced in performing the assays they were evaluating, in Table 1. Results were reported to VRI for unblinding and analysis of % reactivity at each dilution and probit regression analysis to estimate LOD50 and LOD95.

thumbnail
Table 1. Summary of key characteristics, including extraction methods, input sample volumes, cutoffs values, and instruments, of the 15 assays performed at the 10 laboratories that participated in this study.

https://doi.org/10.1371/journal.pntd.0011157.t001

Panel calibration and quantification

The concentration of viral genome equivalents of the French Polynesian ZIKV isolate was calculated from serial dilutions and real-time RT-qPCR data using a computational method based on maximum-likelihood estimation (MLE) for viral genome equivalents number estimation [26]. The ZIKV RNA end-point was first determined based on 10-fold dilutions of the viral stock in plasma. Subsequently, eight 3-fold dilutions around the end-point were prepared by diluting in ZIKV-negative plasma. Twenty replicate RNA extractions were performed for each of the two highest and two lowest dilutions, whereas 40 replicate RNA extractions were performed for each of the four intermediate dilutions. RNA was reverse transcribed and amplified in quadruplicate wells using two-step real-time RT-PCR, and the number of positive replicates was recorded for each dilution. MLE was used to combine the Poisson distribution estimators for all dilutions and generate a viral genome equivalents number estimate that optimally fit the overall serial dilution data.

An aliquot of the highest concentration in the ZIKV NAT panel was quantified using a confirmatory RT-qPCR assay developed by Vitalant Research Institute [25]. Viral load was extrapolated from a standard curve using four 10-fold serial dilutions of the First International Standard (SI) for ZIKV virus NAT assays (PEI: 11468/16) [2729]. The viral load from triplicate assay runs (performed by three operators on two instruments) was averaged to generate the measured concentration in IU/mL. Nominal concentrations of the further serial dilutions in the ZIKV NAT panel were calculated based on the dilution factor.

Identification of candidate assays and manufacturers

Manufacturers of NAT assays that received FDA EUA approvals for use in blood donor or diagnostic settings were initially invited to participate in a comparative evaluation to establish analytical sensitivity and precise estimation of 50 and 95% LODs of molecular assays for ZIKV.

To expand the initial comparative evaluation study, a call for partners, managed by FIND, was conducted in October 2020 for ZIKV test manufacturers interested in a head-to-head performance evaluation of their assay. Submissions for inclusion in the head-to-head performance evaluation were evaluated based on the following inclusion criteria: 1) the assay must be design-locked and commercially available; 2) the manufacturer must have a quality management system to manufacture tests in place (e.g., ISO certification, good manufacturing practice); and 3) the assay must be approved for use by a regulatory body or have an emergency use authorization. Manufacturers of assays meeting these criteria were invited to be included in the evaluation. Manufacturers who accepted inclusion of their product in the study signed a material transfer agreement to receive panels for the evaluation that included a commitment to allow publication of the study findings.

Participating labs

Table 1 summarizes the participating assay manufacturers, testing labs and key characteristics of the assays evaluated in this study. S1 Text provides more detailed summaries of methods and citations for each assay.

Comparison of diagnostic performance on clinical samples

Residual plasma samples were obtained from individuals with DENV like syndrome or suspected ZIKV virus infection from November 2015 to May 2016 in the states of Tocantins and Alagoas in the North and Northeast regions of Brazil, respectively. To investigate the impact of input sample volume on diagnostic sensitivity, samples were tested for DENV, CHIKV, and ZIKV virus RNA using the research-use Arboplex target-capture transcription-mediated amplification (TC-TMA) assay on the automated Panther system (Grifols Diagnostic Solutions) and the research-use VRI triplex real-time RT-qPCR (rRT-PCR) assay which incorporates the small volume manual RNA extraction method, oligonucleotide sequences and one-step RT-qPCR master mix used for the CDC Trioplex RT-qPCR assay, with amplification performed on a real-time PCR instrument (Roche Lightcycler 480 System). After parallel testing using the Arboplex TC-TMA and VRI triplex RT-qPCR assays, samples with discordant results were re-tested using a primer-probe set and positive control set from the CDC Trioplex RT-qPCR assay (ATCC Item Number NR-50417) on a real-time PCR instrument (Roche Lightcycler 480 System).

Statistical methods

Probit regression was performed using positive/negative results as the dependent variable and log10 concentration (in IU/mL or viral genome equivalents/mL) as the independent variable, with a probit link function. Regression analysis was performed in R (version 3.1.1, R Core Team 2014 and version 4.0.4 R Core Team 2021) using the glm module for generalized linear regression modeling. The fitted probit regression model and its standard errors were used to estimate LOD50 and LOD95 and confidence intervals. In order to characterize assays’ RNA extraction and amplification efficiency, we computed a novel metric which we term Coefficient of Assay Efficiency (CE). The CE is the product of the LoD in IUs/mL and the derived plasma input in mL, and therefore expresses the number of ZIKV RNA IUs required in the plasma subjected to the RNA extraction and amplification reactions in order to achieve 50% (CE50) or 95% (CE95) probability of detection.

Results

Analytic performance

Results of replicate testing for all 15 individual assays on the unblinded analytical performance panels are reported in Table 2, with assays grouped into categories based on similar methods, targets, and findings (LODs). Raw data expressed as number of replicates reactive for each assay are reported in S1 Table. Assays demonstrated good specificity based on analysis of the 25 negative control samples, with only one assay yielding a single reactive result on the negative control. Analytical sensitivities of the assays varied by 1–2 logs based on comparisons of % reactivity on serial dilutions and probit analyses to derive LOD50 and LOD95, with one outlier that had substantially higher limits of detection than the other evaluated assays (Fig 1).

thumbnail
Fig 1. Analytic performance of plasma blood screening and diagnostic assays.

Probit curves comparing analytical sensitivity of assays evaluated in this study by replicate testing of Polynesia isolate blinded panels, grouped by assay category or laboratories. The fitted probit regression model curve (solid lines) and standard errors were used to estimate LOD50 and LOD95 and confidence intervals presented in Table 2.

https://doi.org/10.1371/journal.pntd.0011157.g001

thumbnail
Table 2. Probit analysis of 25 replicate serial dilution panel. Results of replicate testing of Polynesia isolate blinded panels presented as combinations of % reactivity and LODs for related assays.

https://doi.org/10.1371/journal.pntd.0011157.t002

The Roche and Grifols blood screening qualitative NAT assays had similar single viral genome equivalents sensitivities at LOD50. The Grifols assay had somewhat higher analytic sensitivity based on a LOD95 of 7.4 IU/mL (95% CI: 4.1,11.3) compared to 20.7 IU/mL (95% CI: 11.2,31.9) for the Roche assay.

Diagnostic PCR assays performed similarly with analytical sensitivities in the range of 6.0 to 55.0 IU/ml at the 50% probability of detection level. The Certest singleplex assay had a substantially higher LOD95 than the other assays at 1,931 IU/mL. The 95% limits of detection of the Altona and MTC lower input volume assays were 252 IU/ml (95% CI: 126,415) and 109 IU/mL (95% CI: 56,176), respectively.

The analytical sensitivity for detection of ZIKV RNA of the Grifols singleplex assay was similar to that of the Grifols multiplex ZIKV/DENV/CHIKV assay, with LOD95 of 7.4 IU/mL (95% CI: 4.1,11.3) and 13.2 IU/mL (95% CI: 6.5,22.1), respectively, which was not statistically significant. The ThermoFisher singleplex assay (LOD95 26.6 IU/mL, 95% CI: 15.5,38.6) also had similar sensitivity to the ThermoFisher multiplex assay (LOD95 46.4 IU/mL, 95% CI: 23.5,75.9). By contrast, the Certest singleplex assay had substantially poorer sensitivity (LOD95 1,931 IU/mL, 95% CI: 686,4,193) than the Certest arbovirus multiplex assay (LOD95 31.2 IU/mL, 95% CI: 15.3,52.4), owing to optimizations made to the multiplex assay that had not been incorporated in the ZIKV assay (personal communication from Certest).

CE50 values ranged from 0.13 for the Certest VIASURE multiplex and 0.75 for the Grifols multiplex assays, indicating very high RNA extraction and amplification efficiency, to 6.65 for the Abbott RealTime assay. In general assays using transcription mediated amplification (TMA) technology had greater efficiency than assays using conventional PCR technology for ZIKV RNA detection. Some assays, like the CDC Trioplex, with very good efficiency nevertheless had relatively poor analytical sensitivity as a result of low effective input volumes.

Impact of diagnostic assay performance

To investigate the impact of input sample volume on diagnostic sensitivity of two of the multiplex assays detecting DENV, CHIKV and ZIKV RNA on clinical specimens, we tested 1017 plasma samples collected from individuals with dengue like syndrome or suspected ZIKV infection in early 2016 in Brazil. Samples were tested using the Grifols Arboplex TC-TMA assay and the VRI small volume triplex RT-qPCR assay. Samples with discordant results between the two assays were further tested using the small volume manual extraction protocol and oligonucleotides from the CDC Trioplex RT-qPCR assay on the Roche Lightcycler 480 instrument. Both the TC-TMA and RT-qPCR assays detected DENV RNA in 2% (21/1017), CHIKV RNA in 19% (189/1017) and ZIKV RNA in 7% (72/1017) of the plasma specimens. Rare (0.1%) coinfections were also detected by both assays (1/1017 each for DENV-CHIKV, DENV-ZIKV and CHIKV-ZIKV). In addition, the TC-TMA assay detected DENV RNA in 1.7% (17/1017), CHIKV RNA in 4.3% (44/1017) and ZIKV RNA in 4.6% (47/1017) of the plasma specimens that were not detected by rRT-PCR. Furthermore, the TC-TMA assay detected 0.2% (2/1017) DENV-CHIKV and 0.4% (4/1017) CHIKV-ZIKV coinfections that were not detected by rRT-PCR, as well as 0.2% (2/1017) CHIKV-ZIKV and 0.1% (1/1017) DENV-ZIKV coinfections that were only detected as ZIKV infections by rRT-PCR. In no cases was viral RNA detected by RT-qPCR but not by TC-TMA. Overall, the data showed that testing by TC-TMA had increased diagnostic sensitivity relative to testing by RT-qPCR of 47% for DENV, 21% for CHIKV and 40% for ZIKV.

Discussion

Appropriate diagnostic preparedness includes rapid deployment of validated assays with optimized performance characteristics. This study used blinded replicate testing of serial dilutions of well characterized ZIKV viral stocks to robustly characterize the relative analytical sensitivities of ZIKV molecular assays used for clinical diagnosis, research and blood donor screening. When performed according to manufacturer instructions for use (IFU), assays demonstrated consistently high specificity, however differences in sensitivity were noted.

Enhancing sensitivities of diagnostic assays may be achievable by increasing sample input per assay, as demonstrated by the relative sensitivities of CDC LI and HI assays performed at the two participating CDC laboratories. Donor screening NAT assays and a Grifols research use Triplex (ZIKV/DENV/CHIKV) assay were comparable in sensitivity and significantly more sensitive than diagnostic RT-qPCR assays, due to higher input volumes of plasma. There were substantial differences in volumes of plasma subjected to amplification in the blood screening versus diagnostic assays. More impactful in performance the volume of plasma subjected to extraction, which correlated with analytical sensitivity demonstrating that enhanced sensitivity was associated with using a greater sample volume of plasma input for initial RNA extraction. The significantly poorer sensitivities of diagnostic PCR assays are primarily attributable to the lower volumes of plasma extracted and consequently of eluted RNA amplified by different assays. Ideal assays to maximize sensitivity and throughput may not be a viable option in all contexts and other factors such as cost, instrumentation and regulatory approval status may influence assay availability and selection, particularly in resource constrained settings.

Although the input volumes of plasma processed in different assays is a primary driver of sensitivity, RNA extraction and amplification reaction efficiency also contribute to assay performance. The coefficient of efficiency (CE50) is a measure of input target viral genome equivalents number as a ratio of the 50% LOD relative to the volume of plasma subjected to amplification. The Grifols TMA assays demonstrated very high CE50 performance, likely due to the efficiency of single tube target capture RNA concentration and the isothermal amplification step. Conventional RT-qPCR assays had somewhat lower and variable CE50 performance, which could be impacted by both extraction and application efficiency. The CDC Trioplex had relatively poor analytical sensitivity despite high CE50, highlighting the importance of input volumes in driving assay sensitivity.

Expanded multiplexed arbovirus assays are needed for clinical diagnostics, blood screening and surveillance, particularly in areas where multiple arboviruses co-circulate. However differential sensitivities of diagnostic assays have implications for diagnosis and management of clinical cases and monitoring of travelers and pregnant women. This is particularly significant in arbovirus endemic settings where in the absence of laboratory confirmation, clinical diagnosis is challenged because of similar clinical presentation of infection with ZIKV, DENV, CHIKV and other regionally endemic infections; specific identification of ZIKV infections is particularly critical for prenatal and postnatal counseling and clinical management. [30,31].

The higher sensitivity and CE50 performance observed on the ZIKV molecular panel for high volume plasma input multiplexed TMA assay relative to low volume plasma input multiplexed RT-qPCR assay was corroborated on clinical samples, with 40% increased diagnostic sensitivity for ZIKV using high effective input volume TMA compared to low effective input volume RT-PCR. Likewise, the diagnostic sensitivity for DENV and CHIKV were 47% and 21% higher, respectively, using high input TMA relative to low input RT-PCR. This data supports the use of high input multiplexed TMA as a sensitive test for clinical screening of plasma in geographic areas with arbovirus outbreaks that may include co-circulation of ZIKV, DENV and CHIKV.

Some limitations of this evaluation exist. The molecular panels did not include clinical specimens and are thus limited to determination of limit of detection of a single ZIKV strain. Panels based on a single viral isolate do not represent all epidemiologic scenarios or diverse global geographic areas where ZIKV is known to occur or where ZIKV transmission may emerge or re-emerge (e.g., Africa, southeast Asia). The performance of assays on different ZIKV variants is not well understood, and although the target region of many commercial assays is propriety, assays are generally designed to target conserved regions to enable detection across variants. This study was executed under optimal conditions as assays were performed by the manufacturers and hence the findings may not translate directly to real world performance. Ideally, it is the combination of rigorous assay evaluation and execution of laboratory proficiency programs that will ensure the most reliable diagnostic interpretation; in this context ongoing External Quality Assurance (EQA) and proficiency testing should be considered [32]. Although this evaluation focuses on high throughput, commercially available assays, it should be noted that well-established in-house protocols may also perform well. Finally, this panel focuses on assay performance using serum, comparative evaluation of assays using alternative sample types for use in different diagnostic contexts should be considered.

Opportunities exist for additional external evaluations of commercialised ZIKV tests as remaining panels from the current study exist that can be used in the future for analytical evaluation of commercialised molecular assays used by laboratories for ZIKV diagnosis but for which the test developers did not respond to FIND’s call for partners. Based on this ongoing work, WHO is updating priority use cases and target product profiles for diagnostics. This should include consideration of contexts for recommendations, sample acquisition, and selection of assays, e.g., sample acquisition linked to blood donor screening assays where ample volumes may be available, versus clinical samples with lower volumes, and less resourced areas with more rudimentary molecular capabilities. Additionally, these panels could be expanded to include samples from other epidemiologically diverse endemic regions (sub-Saharan Africa, south-east assaysoutheast Asia). Outcomes of this evaluation will serve to accelerate knowledge on test performance and inform public health policy, surveillance programs, blood donor screening and clinical management of patients with suspected ZIKV infection.

Supporting information

S1 Table. Probit analysis of 25 replicate serial dilution panel.

Results of replicate testing of Polynesia isolate blinded panels presented as number of replicates reactive for related assays.

https://doi.org/10.1371/journal.pntd.0011157.s001

(XLSX)

Acknowledgments

The authors would like to acknowledge the invaluable technical assistance of Kristen Valanoski, Inderdeep Singh and Cydni Samuels.

References

  1. 1. Richard V, Paoaafaite T, Cao-Lormeau V-M. Vector Competence of French Polynesian Aedes aegypti and Aedes polynesiensis for Zika Virus. PLOS Neglected Tropical Diseases. 2016;10(9):e0005024. pmid:27654962
  2. 2. Barjas-Castro ML, Angerami RN, Cunha MS, Suzuki A, Nogueira JS, Rocco IM, et al. Probable transfusion-transmitted Zika virus in Brazil. Transfusion. 2016;56(7):1684–8. pmid:27329551
  3. 3. Besnard M, Lastere S, Teissier A, Cao-Lormeau V, Musso D. Evidence of perinatal transmission of Zika virus, French Polynesia, December 2013 and February 2014. Euro Surveill. 2014;19(13). pmid:24721538
  4. 4. Foy BD, Kobylinski KC, Chilson Foy JL, Blitvich BJ, Travassos da Rosa A, Haddow AD, et al. Probable non-vector-borne transmission of Zika virus, Colorado, USA. Emerg Infect Dis. 2011;17(5):880–2. pmid:21529401
  5. 5. Motta IJF, Spencer BR, Cordeiro da Silva SG, Arruda MB, Dobbin JA, Gonzaga YBM, et al. Evidence for Transmission of Zika Virus by Platelet Transfusion. New England Journal of Medicine. 2016;375(11):1101–3. pmid:27532622
  6. 6. Musso D, Nhan T, Robin E, Roche C, Bierlaire D, Zisou K, et al. Potential for Zika virus transmission through blood transfusion demonstrated during an outbreak in French Polynesia, November 2013 to February 2014. Eurosurveillance. 2014;19(14):20761. pmid:24739982
  7. 7. Oliveira Melo AS, Malinger G, Ximenes R, Szejnfeld PO, Alves Sampaio S, Bispo de Filippis AM. Zika virus intrauterine infection causes fetal brain abnormality and microcephaly: tip of the iceberg? Ultrasound Obstet Gynecol. 2016;47(1):6–7. pmid:26731034
  8. 8. Sarno M, Sacramento GA, Khouri R, do Rosário MS, Costa F, Archanjo G, et al. Zika Virus Infection and Stillbirths: A Case of Hydrops Fetalis, Hydranencephaly and Fetal Demise. PLoS Negl Trop Dis. 2016;10(2):e0004517. pmid:26914330
  9. 9. Brasil P, Nielsen-Saines K. More pieces to the microcephaly-Zika virus puzzle in Brazil. The Lancet infectious diseases. 2016;16(12):1307–9. pmid:27641776
  10. 10. Brasil P, Pereira JP, Moreira ME Jr., Ribeiro Nogueira RM, Damasceno L, Wakimoto M, et al. Zika Virus Infection in Pregnant Women in Rio de Janeiro. The New England journal of medicine. 2016;375(24):2321–34. pmid:26943629
  11. 11. Mlakar J, Korva M, Tul N, Popović M, Poljšak-Prijatelj M, Mraz J, et al. Zika Virus Associated with Microcephaly. New England Journal of Medicine. 2016;374(10):951–8. pmid:26862926
  12. 12. Nunes ML, Carlini CR, Marinowic D, Neto FK, Fiori HH, Scotta MC, et al. Microcephaly and Zika virus: a clinical and epidemiological analysis of the current outbreak in Brazil. J Pediatr (Rio J). 2016;92(3):230–40. pmid:27049675
  13. 13. Wang JN, Ling F. Zika Virus Infection and Microcephaly: Evidence for a Causal Link. Int J Environ Res Public Health. 2016;13(10). pmid:27775637
  14. 14. Williamson PC, Linnen JM, Kessler DA, Shaz BH, Kamel H, Vassallo RR, et al. First cases of Zika virus-infected US blood donors outside states with areas of active transmission. Transfusion. 2017;57(3pt2):770–8. pmid:28229475
  15. 15. Stone M, Bakkour S, Lanteri MC, Brambilla D, Simmons G, Bruhn R, et al. Zika virus RNA and IgM persistence in blood compartments and body fluids: a prospective observational study. The Lancet infectious diseases. 2020;20(12):1446–56. pmid:32673593
  16. 16. Musso D, Roche C, Nhan TX, Robin E, Teissier A, Cao-Lormeau VM. Detection of Zika virus in saliva. Journal of clinical virology: the official publication of the Pan American Society for Clinical Virology. 2015;68:53–5. pmid:26071336
  17. 17. Atkinson B, Hearn P, Afrough B, Lumley S, Carter D, Aarons EJ, et al. Detection of Zika Virus in Semen. Emerg Infect Dis. 2016;22(5):940. pmid:27088817
  18. 18. Musso D, Roche C, Robin E, Nhan T, Teissier A, Cao-Lormeau VM. Potential sexual transmission of Zika virus. Emerg Infect Dis. 2015;21(2):359–61. pmid:25625872
  19. 19. FDA. Donor Screening Recommendations to Reduce the Risk of Transmission of Zika Virus by Human Cells, Tissues, and Cellular and Tissue-Based Products; Guidance for Industry https://www.fda.gov/regulatory-information/2018 [updated May 2018.
  20. 20. Stockholm. European Centre for Disease Prevention and Control. Zika virus and safety of substances of human origin: a guide for preparedness activities in Europe. ECDC. 2017.
  21. 21. World Health Organization. Maintaining a safe and adequate blood supply during Zika virus outbreaks: interim guidance. Geneva: World Health Organization; 2016 2016. Contract No.: WHO/ZIKV/HS/16.1.
  22. 22. Musso D, Stramer SL, Busch MP. Zika virus: a new challenge for blood transfusion. Lancet. 2016;387(10032):1993–4. pmid:27156435
  23. 23. Lanteri MC, Kleinman SH, Glynn SA, Musso D, Keith Hoots W, Custer BS, et al. Zika virus: a new threat to the safety of the blood supply with worldwide impact and implications. Transfusion. 2016;56(7):1907–14. pmid:27282638
  24. 24. Stone M, Lanteri MC, Bakkour S, Deng X, Galel SA, Linnen JM, et al. Relative analytical sensitivity of donor nucleic acid amplification technology screening and diagnostic real-time polymerase chain reaction assays for detection of Zika virus RNA. Transfusion. 2017;57(3pt2):734–47. pmid:28194799
  25. 25. Galel SA, Williamson PC, Busch MP, Stanek D, Bakkour S, Stone M, et al. First Zika-positive donations in the continental United States. Transfusion. 2017;57(3pt2):762–9. pmid:28164310
  26. 26. Deng X, Custer BS, Busch MP, Bakkour S, Lee TH. Simultaneous estimation of detection sensitivity and absolute copy number from digital PCR serial dilution. Comput Biol Chem. 2017;68:1–5. pmid:28213308
  27. 27. Baylis SA, Nübling CM. Nucleic Acid Testing of Viral Pathogens: Traceability of the International Unit. Clinical chemistry. 2021;68(1):257–8. pmid:35265967
  28. 28. Baylis SA, Hanschmann KO, Schnierle BS, Trösemeier JH, Blümel J. Harmonization of nucleic acid testing for Zika virus: development of the 1(st) World Health Organization International Standard. Transfusion. 2017;57(3pt2):748–61. pmid:28229462
  29. 29. Baylis SA, Wallace P, McCulloch E, Niesters HGM, Nübling CM. Standardization of Nucleic Acid Tests: the Approach of the World Health Organization. Journal of clinical microbiology. 2019;57(1). pmid:30257900
  30. 30. Brooks T, Roy-Burman A, Tuholske C, Busch MP, Bakkour S, Stone M, et al. Real-Time Evolution of Zika Virus Disease Outbreak, Roatán, Honduras. Emerg Infect Dis. 2017;23(8):1360–3.
  31. 31. Sharp TM, Fischer M, Muñoz-Jordán JL, Paz-Bailey G, Staples JE, Gregory CJ, et al. Dengue and Zika Virus Diagnostic Testing for Patients with a Clinically Compatible Illness and Risk for Infection with Both Viruses. MMWR Recommendations and reports: Morbidity and mortality weekly report Recommendations and reports / Centers for Disease Control. 2019;68(1):1–10. pmid:31194720
  32. 32. Fischer C, Pedroso C, Mendrone A, Bispo de Filippis AM Jr., Vallinoto ACR, Ribeiro BM, et al. External Quality Assessment for Zika Virus Molecular Diagnostic Testing, Brazil. Emerg Infect Dis. 2018;24(5):888–92. pmid:29470164