Background:  The World Health Organization (WHO) has developed a global laboratory network to support an HIV drug-resistance prevention and assessment strategy at national, regional, and global levels. A quality assurance program is being implemented to ensure the reliability of genotyping data generated by the various laboratories. Proficiency panels were developed in collaboration with NIH and sent to 58 network member laboratories in Europe, North America, Asia, Africa, and the Caribbean during 2007 and 2008. Appropriate and widely accepted evaluation criteria have not been described previously.

Methods:  Each panel was composed of 5 clinical samples (subtypes B, C, D, F, CRF01_AE, and CRF02_AG) with viral load between 3500 and 57,000 copies/mL. Panel 1 was distributed in 2007 to 21 labs, and included only samples with no major drug resistance-associated mutations. Panel 2 was used in 2008 for 26 labs and included 1 sample with several major drug resistance-associated mutations. Some labs used more than one type of assay. Consensus sequences (PR codons 10 to 99 and RT 38 to 240) for each sample were generated based on >80% concordance across multiple labs; individual test results were compared to that sample's consensus sequence. An overall sequence identity score as well as concordance at drug resistance-associated mutation codons were used to assess lab performance. Initial analysis was performed based on arbitrarily-set pass/fail criteria of 99% nucleotide identity for both scores.

Results:  Overall, 41 of 58 submissions passed using the 99% criteria. Success rates were higher with panel 1 (80%) than panel 2 (64%). Specific reasons for failure included:  editing errors leading to frame-shifts, missing sequence, amplification failure, and a low concordance rate in codons with mixed bases in the consensus. Samples with naturally occurring mixtures at drug resistance-associated mutation sites were more challenging than others. For example, no lab obtained >99% concordance for sample 4 in panel 1 (median 98.6%); for sample 4 in panel 2, (subtype B, drug resistance-associated mutations for all 3 drug classes) only 18 of 33 labs reported sequence with >99% concordance, compared to 26 to 30 labs for the other 4 samples. Use of a 98% identity criteria overall would result in >85% success rate overall.

Conclusions:  The use of a single and stringent criterion for evaluation of sequence-based assays may be unrealistic when using clinical samples containing mixed bases at several positions. Acceptance criteria may need to be relaxed for such samples (e.g., 98% identity), or be flexible and based on the number of mixed positions in each sample.