Figures
Abstract
Objectives
Validation of a cost effective in-house method for HIV-1 drug resistance genotyping using plasma samples.
Design
The validation includes the establishment of analytical performance characteristics such as accuracy, reproducibility, precision and sensitivity.
Methods
The accuracy was assessed by comparing 26 paired Virological Quality Assessment (VQA) proficiency testing panel sequences generated by in-house and ViroSeq Genotyping System 2.0 (Celera Diagnostics, US) as a gold standard. The reproducibility and precision were carried out on five samples with five replicates representing multiple HIV-1 subtypes (A, B, C) and resistance patterns. The amplification sensitivity was evaluated on HIV-1 positive plasma samples (n = 88) with known viral loads ranges from 1000–1.8 million RNA copies/ml.
Results
Comparison of the nucleotide sequences generated by ViroSeq and in-house method showed 99.41±0.46 and 99.68±0.35% mean nucleotide and amino acid identity respectively. Out of 135 Stanford HIVdb listed HIV-1 drug resistance mutations, partial discordance was observed at 15 positions and complete discordance was absent. The reproducibility and precision study showed high nucleotide sequence identities i.e. 99.88±0.10 and 99.82±0.20 respectively. The in-house method showed 100% analytical sensitivity on the samples with HIV-1 viral load >1000 RNA copies/ml. The cost of running the in-house method is only 50% of that for ViroSeq method (112$ vs 300$), thus making it cost effective.
Conclusions
The validated cost effective in-house method may be used to collect surveillance data on the emergence and transmission of HIV-1 drug resistance in resource limited countries. Moreover, the wide applications of a cost effective and validated in-house method for HIV-1 drug resistance testing will facilitate the decision making for the appropriate management of HIV infected patients.
Citation: Chaturbhuj DN, Nirmalkar AP, Paranjape RS, Tripathy SP (2014) Evaluation of a Cost Effective In-House Method for HIV-1 Drug Resistance Genotyping Using Plasma Samples. PLoS ONE 9(2): e87441. https://doi.org/10.1371/journal.pone.0087441
Editor: Mark Wainberg, McGill University AIDS Centre, Canada
Received: September 26, 2013; Accepted: December 25, 2013; Published: February 12, 2014
Copyright: © 2014 Chaturbhuj et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: Intramural funded by NARI, Pune. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Close to 60% of those who require anti-retroviral treatment (ART) are already under ART [1]. The long term success of this programme will depend on the continued virus suppression as a result of anti-retroviral treatment. However, in resource-limited countries, HIV drug resistance testing is not generally available or it is too costly to be used in the routine monitoring of patients receiving ART. Monitoring the trends of HIV drug resistance among treatment failures is crucial as HIV drug resistance is a major threat with ability to negate the benefits accrued by the free ART programme. Detection and monitoring of HIV drug resistance by molecular genotyping is pivotal to ensure ongoing regimen efficacy. Therefore, the World Health Organization (WHO) recommends population-based surveillance and monitoring of HIV drug resistance in resource-limited settings [2], [3]. The reported pattern and rates of transmitted and acquired drug-resistant HIV variants will collectively form the regional and global recommendations on which ART is to be maintained or changed in the first and second-line antiretroviral regimens [4]. The ViroSeq Genotyping System 2.0 (Celera Diagnostics, US) and TruGene are the two FDA-approved commercially available methods for HIV drug resistance testing [5], [6]. The majority of the HIV-1 infected patients cannot afford the commercial HIV-1 drug resistance testing when they experience ART failure because of its high cost. Thus, there is a need for the development of a cost effective and efficient in-house method for HIV-1 drug resistance testing for application in resource limited settings.
The development of an in-house method has been done independently in each lab, usually requiring investment of considerable effort to optimize the procedures used [7]–[11]. In larger studies and surveys, it is important to have confidence that the results generated from different participating labs are of high quality and comparable to each other. One solution to this problem would be to recommend all labs to use the same method, but this is not practical, given that the local difference in reagent supply, HIV subtypes, personnel training and the requirement to change the established procedure may vary. An alternative approach is to use a validated method that will ensure quality results.
The validation of an in-house method is now a prerequisite for testing any sample for World Health Organization (WHO) recommended surveillance and studies [12]. Most of the laboratories have validated the in-house method by the minimal criteria strategy which includes the comparison of the sequence of large numbers of specimens (e.g. 50 to 200) obtained by an in-house method with that obtained by previously validated commercially available FDA approved HIV genotyping method as “Gold Standard” [7], [9], [10]. Whereas this minimal validation procedure can describe the accuracy of the method, further assessment of the other analytical characteristics such as reproducibility, precision and analytical sensitivity is also required to ensure the reliable results. The acceptability of the performance of a method is determined by the analytical performance characteristics such as accuracy, reproducibility, precision, amplification sensitivity and specificity.
In this study, previous in-house method [13] was modified and evaluated by the analytical performance characteristics such as accuracy, reproducibility, precision, analytical sensitivity.
Materials and Methods
Samples
The validation of an in-house method was done according to the WHO guidelines [12] including participation in Virological Quality Assessment (VQA) HIV Genotypic Drug Resistance proficiency testing panels (VQA contract # NO1-AI-50044). The accuracy of the in-house method was assessed by comparing 26 paired VQA HIV Genotypic Drug Resistance proficiency testing panel sequences. The nucleotide/amino acid sequence obtained by the ViroSeq method (Celera Diagnostics, US) was considered as the gold standard with which the nucleotide/amino acid sequences obtained by the in-house method were compared.
The samples used in the validation study had similar characteristics to the samples which are tested routinely (sample type: plasma, genetic subtype: HIV-1 subtype C, A and B, viral load range >1000 copies/ml, resistance pattern: reverse transcriptase and protease inhibitor mutations). The reproducibility and the precision were carried out on five samples with five replicates each. All these samples were previously tested for HIV-1 drug resistance genotyping using the ViroSeq method and aliquots were stored at −70±5°C.
Validation criteria
Accuracy was defined as detection of 99% of known HIV-1drug resistance mutations when compared with the results obtained by the ViroSeq Genotyping System 2.0 (Celera Diagnostics, US). The reproducibility and precision were defined as ≥98% nucleotide identities in ≥90% of pairwise comparisons, with the mixtures being counted as partially discordant. The nucleotide/amino acid sequences were analyzed under three categories: [10] (i) Concordant (if both the ViroSeq and in-house method gave the same nucleotide/amino acid). (ii) Partially discordant (if nucleotide base/amino acid mixture by one method but not by the other) and (iii) Discordant (if the two methods detected different nucleotide bases/amino acids).
Analytical Sensitivity
Since HIV-1 subtype C is the predominant subtype in India, the analytical sensitivity of the in-house method was tested by using two HIV-1 subtype C clinical samples with the viral load 75,520 copies/ml and 18,500 copies/ml determined by the COBAS Amplicor HIV-1 Monitoring Kit version 1.5 (Roche Diagnostics, Branchburg, New Jersey). The samples were serially diluted (the created copy number ranged from 4720 to less than 1000 HIV-1 RNA copies/ml) using HIV negative human plasma. Viral RNA was then extracted and the pol gene was amplified by the in-house method as described below.
The protocol was finalized before initiation of the evaluation of the in-house HIV drug resistance assay. As per the WHO guidelines, an acceptance criterion for each analytical performance characteristic was established in advance [12]. The protocol for the in-house method used for the validation is described below.
RNA Extraction
RNA extraction from plasma samples were performed using the NucliSENS® easyMAG™ (Biomerieux, Durham, NC) automated nucleic acid extraction system according to the manufacturer's recommendations (NucliSENS easyMAG user manual, v 1.1; BioMérieux, Boxtel, Netherlands). Five hundred microlitres of each sample was placed in the disposable sample vessel and the sample vessel was loaded onto the extractor. After the initial lysis incubation for 10 min, 50 µl of magnetic silica was added to each sample and the extractor was restarted. The samples were eluted in 50 µl of extraction buffer 3. All samples were transferred to a 1.5-ml microcentrifuge tube and stored at −70°C.
RT-PCR and Nested PCR
RT-PCR was performed as described previously [13]. The nested PCR was performed using the inner primer pair as mentioned in Table 1 to get an amplified fragment of 1204 bp. The amplification was done using Taq DNA polymerase (Genei, Bangalore India) 3 U/µl (1 µl) in a 10× PCR buffer B (5 µl), 25 mM MgCl2 (4 µl), 2 mM dNTPs mix (Fermentas) (5 µl), 10 pmol of each primer and 6 µl of RT-PCR product. The final volume of 50 µl reaction was made up using DNase/RNase free water. The reaction conditions used for nested PCR were: initial denaturation at 94°C for 2 min followed by DNA amplification: 25 cycles at 94°C for 20 sec, 59°C for 45 sec, 72°C for 3 min and a final extension for 10 min at 72°C.
A GeneAmp PCR system 9700 thermal cycler (Applied Biosystem, CA, USA) was used for all PCRs. The results were checked by electrophoresis of the nested PCR products on 1% Agarose (GeNei, Bangalore, India) gels containing ethidium bromide (Sigma Aldrech, USA) and visualization of the amplified bands under UV light.
The amplified PCR products were purified using the QIAquick PCR purification kit (Qiagen Hilden, Germany) and eluted in 30 µl elution buffer. The purified PCR product was directly sequenced in both directions using BigDye Terminator Cycle Sequencing Ready Reaction kit version 3.1 (Applied Biosystem, CA, USA).
Sequence analysis
DNA sequencing was performed on 3100 DNA genetic analyzer (Applied Biosystem, CA, USA) with the six specific primers mentioned in Table 1. These sequencing primers provided overlapping, bidirectional sequences covering the whole protease region and partial reverse transcriptase region. The sequencing reaction was performed in 10 µl volume containing 2 µl ready reaction mix, 4 pmol primers, 2 µl of 5× sequencing buffer, 2 µl of purified PCR product (30 ng/µl) and the volume was adjusted to 10 µl with DNase/RNase free water. The sequencing reaction was carried out using 25 linear amplification cycles of 96°C for 10 sec, 50°C for 5 sec, 60°C for 4 min. Unincorporated dNTPs were then removed by precipitation with 80% isopropanol and the dried pellet was resuspended in 10 µl of Hi-DiTM formamide (Applied Biosystem, CA, USA). The raw sequence data from ABI 3100 genetic analyzer was assembled, aligned and edited with the SeqScape v2.0 software (Applied Biosystem, CA, USA).
To rule out PCR contamination, the phylogenetic tree was generated by using the PhyML software to create a maximum likelihood tree [14]. The reference sequences were obtained from the Los Alamos HIV Database (www.hiv.lanl.gov). The sequence quality was also checked by Sequence Quality Assessment Tool (SQUAT) analysis software [15]. The final edited sequences were submitted to the “HIVdb Program: Sequence Analysis” in the Stanford University HIV drug resistance database for drug resistance interpretation [16]. Pairwise nucleotide sequence identity and discrepancy were analyzed using BioEdit v 7.0.9.0 [17]. The Rega HIV-1 subtyping tool Version 2.0 was used to determine the HIV-1 subtype (http://dbpartners.stanford.edu/RegaSubtyping/ Accessed on 14th May 2012).
Results
Accuracy
All 26 paired VQA HIV Genotypic Drug Resistance proficiency testing panel samples were amplified and sequenced successfully using both the in-house and ViroSeq methods. The mean nucleotide identity was 99.41±0.46% (mean ± SD) among paired nucleotide sequences (Table 2). Wilcoxon signed-rank test was used to compare the in-house and ViroSeq method basecalling for mixed bases and revealed no significant difference (P, 0.382).
A total of 135 drug resistance-associated amino acid positions in protease (36 mutations) and reverse transcriptase (99 mutations) region were detected among 26 paired sequences. There was no complete nucleotide or amino acid discordance. The overall amino acid codon agreement was 99.68±0.35% among paired amino acid sequences.
The partial discordance due to synonymous/non-synonymous substitutions was observed at 15 amino acid positions including 2 positions in protease and 13 positions in the reverse transcriptase region (Table 3). Out of 15 partial discordant positions, in 13 positions mixtures of the wild type and the mutant virus codons were detected by the in-house method only, while the ViroSeq method detected either the wild type or mutated codon. In the remaining two positions, mixtures of the wild type and the mutant virus codons were detected by the ViroSeq method only, while the in-house method could detect only the wild type or mutated codon.
Reproducibility
All five replicates of five samples were amplified and sequenced successfully. The drug resistance mutation pattern and subtype distribution of samples selected for reproducibility study are given in Table 4. A total of 25 sequences were generated from five samples. The phylogenetic tree for all 25 sequences along with HIV-1 reference sequences (n = 56) was constructed by using the PhyML software to create a maximum likelihood tree [14]. The sequences confirmed the absence of cross-contamination or sample mix-up. The paired sequences obtained for each sample were more closely related to one another than to the sequences of any other sample (Fig. 1). Moreover, as expected, sequences from the same sample clustered together.
The phylogenetic tree was generated by using the PhyML software to create a maximum likelihood tree [13]. The reference sequences were obtained from the Los Alamos HIV Database (www.hiv.lanl.gov). IHDR- sequences generated by an in-house method; VSQ- sequences generated by the ViroSeq method; R1 to R5- sample used in the reproducibility study (hilighted in blue); P1 to P5- sample used in the precision study (hilighted in red); A, B, C, D, E are the replicates of the same sample.
We found high nucleotide sequence identities ranging from 99.72% to 100.00% (99.88±0.10) (Table 4). The minor differences observed in sequence identity were caused by base mixtures (nucleotide mixture by one method but not by another). However, this difference did not translate into differences in amino acids.
A total of 35 drug resistance-related positions [PI mutations (7), NRTI mutations (16) and NNRTI mutations (12)] for each sequence and its replicates were also analyzed for reproducibility. All 35 drug resistance mutations were found to be reproducible with 100% concordance.
Precision
The precision study also showed high nucleotide sequence identity ranging from 98.49% to 100.00% (99.82±0.20) (Table 4). A total of 34 drug resistance-related positions [PI mutations (16), NRTI mutations (12) and NNRTI mutations (6)] were seen for five replicates of five samples analyzed for precision study. In addition, one sample showed an insertion at amino acid 35 in the protease for all replicates. All 34 drug resistance mutations and one insertion at amino acid 35 in the protease were found to be reproducible with 100% concordance. The drug resistance mutation pattern and subtype distributions of samples selected for precision study is given in Table 4.
Amplification Sensitivity
The amplification sensitivity of the in-house method was evaluated on HIV-1 positive plasma samples (n = 88) with known viral load ranging from 1000 to 1.8 million RNA copies/ml (see Table S1) for the viral copy number range, % amplification and subtype distribution). All the samples with viral load >1000 HIV-1 RNA copies/ml were amplified and sequenced successfully.
We have also evaluated the amplification sensitivity by serial dilution of two samples with high viral loads to achieve a range of copy numbers (4720, 2360, 1180, 590 RNA copies/ml) using plasma from HIV-1 negative donor followed by four replicate testings of each dilution. The amplification was seen in all four replicates of the sample with >1000 RNA copies/ml, whereas amplification was observed only in two out of four replicates with <1000 RNA copies/ml for both samples.
Discussion
In this study, the validation of an in-house method for HIV-1 drug resistance genotyping developed at the National AIDS Research Institute (ICMR), Pune, India was orchestrated. Accuracy of the in-house method was assessed by comparing the results of VQA HIV Genotypic Drug Resistance proficiency testing panel samples generated by the in-house method with the results of the ViroSeq method. The VQA HIV Genotypic Drug Resistance proficiency testing panel samples used in this study consisted of HIV-1 group M subtypes with known viral loads (viral load ranges 3000 to 60,633 copies/ml). All these samples (n = 26) were amplified and sequenced successfully by both in-house method and ViroSeq method with high nucleotide sequence identity (99.41±0.46%). All the clinically relevant mutations were concordant by both methods and reproducible. Despite the minimal differences seen by the partial discordance in nucleotide base callings, the in-house method demonstrated an ability to identify clinically relevant mutations correctly (99.41±0.46) when compared with the ViroSeq method.
In every case of nucleotide partial discordance, one method detected a mixture of the wild type and the mutant virus, the other method detected either the wild type or the mutant virus. We performed sequence editing for all validation samples using the minor peak default nucleotide mixture calling setting at 30% of the major peak in bidirectional sequences. The minor peaks at partial discordant nucleotide could be seen, but were below the nucleotide mixture cutoff (30%) on the chromatogram by SeqScap software and ViroSeq Software. Thus, these mixtures were not counted, resulted in the partial discordance. These results indicated that the sequences generated by population-based sequencing were highly reproducible but the sensitivity at detecting low viral variants was very challenging. Many factors could contribute to the detection of mixed nucleotide sites including viral quasispecies, primer binding preference and location, Taq polymerase misincorporation and sequence quality. The sequence identity and codon concordance are challenging when mixed nucleotide bases are present [18]. It has been reported that the ViroSeq method detected more mixtures of the wild type with the mutant virus (78%) than seen in the in-house assay (22%) [10]. In contrast, our results revealed no significant difference (P value, 0.382) in basecalling for nucleotide mixed bases. The variability in detecting nucleotide mixtures of the wild type or the mutant virus was likely due to the first-round RT-PCR in sampling of quasispecies strains rather than technical errors in the sequencing process [19], [20].
Further, the in-house method was assessed for inter and intra-assay variability (reproducibility and precision) on multiple aliquots of the same sample in different test runs or within a test run respectively. The samples selected for inter and intra-assay variability were from multiple HIV subtypes (C, A and B) and contained the most common NRTI mutations (M184V & D67N), NNRTI mutations (K103N, G190A) and PR mutations (M46I, I54V & L90M). The inter-assay variability was analyzed over time (24 weeks), among different lots of critical reagents (Extration buffers, RT PCR kit and Magnetic Silica). The high degree of pairwise nucleotide sequence identity for inter and intra-assay comparisions (99.88±0.10 and 99.82±0.20 respectively) showed a high degree of reproducibility of the result.
The laboratory has already been certified for HIV Genotypic Drug Resistance using the ViroSeq method and is under VQA HIV Genotypic Drug Resistance proficiency testing program. However, the validated in-house method also successfully completed a total of five VQA HIV Genotypic Drug Resistance proficiency panels, resulting in the in-house method described here being certified by the Virology Quality Assessment (VQA) Program funded by National Institute of Health (NIH). The successful testing of VQA HIV Genotypic Drug Resistance proficiency panels indicated that the in-house method could genotype HIV-1 group M subtypes.
The manpower and turnaround time for both the ViroSeq and the in-house methods were similar. The cost per test incurred for the in-house method (US 112$) was approximately 50% of the cost incurred using the ViroSeq method (US 300$).
In conclusion, we have evaluated a cost effective in-house method for HIV-1 drug resistance testing using plasma samples. The validated in-house method was broadly sensitive in genotyping multiple HIV-1 group M subtypes. The validation analyses indicate 100% amplification sensitivity for samples >1000 HIV-1 copies/ml, and high accuracy (99.41±0.46) when compared with the ViroSeq method. In the present study, high degree of reproducibility (99.88±0.10) and precision (99.82±0.20) were also observed with the in-house method. The validated in-house method may be used to effectively monitor patients failing ARV therapy, as well as to collect surveillance data on the emergence and transmission of HIV-1 drug resistance isolates in resource limited countries. Moreover, the wide applications of a cost effective and validated in-house method for HIV-1 drug resistance testing will facilitate the decision making for the appropriate management of HIV infected patients and thereby reduce the risk of onset of further drug resistance-related mutations.
Supporting Information
Table S1.
Supporting Information for the amplification sensitivity of an in-house method using known viral copy number samples.
https://doi.org/10.1371/journal.pone.0087441.s001
(DOCX)
Acknowledgments
The authors are grateful to their colleagues Mr. Nitin Hingankar and Ms. Smita Thorat at the National AIDS Research Institute (NARI), Pune for their support and cooperation. The authors thank NARI Clinic and Armed Forces Medical College, Pune for reffering the samples for HIV-1 Drug Resistance testing at NARI, Pune. The authors also thank Dr. James W. Bremer and Dr. Cheryl Jennings from Rush University, USA for providing the VQA HIV Genotypic Drug Resistance proficiency panels.
Author Contributions
Conceived and designed the experiments: DNC RSP SPT. Performed the experiments: DNC. Analyzed the data: DNC APN RSP SPT. Wrote the paper: DNC APN RSP SPT.
References
- 1. WHO HIV/AIDS in the South-East Asia region progress towards MDG 6A, 2012. 2012: 35–42.
- 2. Bennett DE (2006) The requirement for surveillance of HIV drug resistance within antiretroviral rollout in the developing world. Curr Opin Infect Dis 19: 607–614.
- 3. Jordan MR, Bennett DE, Bertagnolio S, Gilks CF, Sutherland D (2008) World Health Organization surveys to monitor HIV drug resistance prevention and associated factors in sentinel antiretroviral treatment sites. Antivir Ther 13 Suppl 2: 15–23.
- 4. Gilks CF, Crowley S, Ekpini R, Gove S, Perriens J, et al. (2006) The WHO public-health approach to antiretroviral treatment against HIV in resource-limited settings. Lancet 368: 505–510.
- 5. Eshleman SH, Hackett J Jr, Swanson P, Cunningham SP, Drews B, et al. (2004) Performance of the Celera Diagnostics ViroSeq HIV-1 Genotyping System for sequence-based analysis of diverse human immunodeficiency virus type 1 strains. J Clin Microbiol 42: 2711–2717.
- 6. Grant RM, Kuritzkes DR, Johnson VA, Mellors JW, Sullivan JL, et al. (2003) Accuracy of the TRUGENE HIV-1 genotyping kit. J Clin Microbiol 41: 1586–1593.
- 7. Peduzzi C, Pierotti P, Venturi G, Romano L, Mazzotta F, et al. (2002) Performance of an in-house genotypic antiretroviral resistance assay in patients pretreated with multiple human immunodeficiency virus type 1 protease and reverse transcriptase inhibitors. J Clin Virol 25: 57–62.
- 8. Steegen K, Demecheleer E, De Cabooter N, Nges D, Temmerman M, et al. (2006) A sensitive in-house RT-PCR genotyping system for combined detection of plasma HIV-1 and assessment of drug resistance. J Virol Methods 133: 137–145.
- 9. Chen JH, Wong KH, Chan K, Lam HY, Lee SS, et al. (2007) Evaluation of an in-house genotyping resistance test for HIV-1 drug resistance interpretation and genotyping. J Clin Virol 39: 125–131.
- 10. Saravanan S, Vidya M, Balakrishnan P, Kumarasamy N, Solomon SS, et al. (2009) Evaluation of two human immunodeficiency virus-1 genotyping systems: ViroSeq 2.0 and an in-house method. J Virol Methods 159: 211–216.
- 11. Wallis CL, Papathanasopoulos MA, Lakhi S, Karita E, Kamali A, et al. (2010) Affordable in-house antiretroviral drug resistance assay with good performance in non-subtype B HIV-1. J Virol Methods 163: 505–508.
- 12.
WHO. HIV drug resistance laboratory training package, 2009: HIV Genotyping Assay Validation. 2009. Available at: http://www.who.int/hiv/pub/drugresistance/HIVDR_Mod_7_Assay_Validation.ppt
- 13. Chaturbhuj DN, Deshmukh PS, Hingankar NK, Siddhaarth K, Deshpande SN, et al. (2013) Evaluations of an in-house drug resistance method for HIV-1 drug resistance using ViroSeq™ 2.0 genotyping system as a gold standard. J Virol Methods 189: 87–92.
- 14. Guindon S, Gascuel O (2003) A simple, fast, and accurate algorithm to estimate large phylogenies by maximum likelihood. Syst Biol 52: 696–704.
- 15. Delong AK, Wu M, Bennett D, Parkin N, Wu Z, et al. (2012) Sequence quality analysis tool for HIV type 1 protease and reverse transcriptase. AIDS Res Hum Retroviruses 28: 894–901.
- 16. Liu TF, Shafer RW (2006) Web resources for HIV type 1 genotypic-resistance test interpretation. Clin Infect Dis 42: 1608–1618.
- 17. Hall TA (1999) BioEdit: a user-friendly biological sequence alignment editor and analysis program for Windows 95/98/NT. Nucleic Acids Symposium Series 41: 95–98.
- 18. Parkin NT, Fitzgibbon J, Bremer J, Bertagnolio S (2009) HIV drug resistance genotyping external quality assurance (EQA) for laboratories in the World Health Organization (WHO) ResNet during 2007–2009. Antivir Ther 14 Suppl 1: A162.
- 19. Galli RA, Sattha B, Wynhoven B, O'Shaughnessy MV, Harrigan PR (2003) Sources and magnitude of intralaboratory variability in a sequence-based genotypic assay for human immunodeficiency virus type 1 drug resistance. J Clin Microbiol 41: 2900–2907.
- 20. Shafer RW, Hertogs K, Zolopa AR, Warford A, Bloor S, et al. (2001) High degree of interlaboratory reproducibility of human immunodeficiency virus type 1 protease and reverse transcriptase sequencing of plasma samples from heavily treated patients. J Clin Microbiol 39: 1522–1529.