Clinical Evaluation of an Affordable Qualitative Viral Failure Assay for HIV Using Dried Blood Spots in Uganda

Background WHO recommends regular viral load (VL) monitoring of patients on antiretroviral therapy (ART) for timely detection of virological failure, prevention of acquired HIV drug resistance (HIVDR) and avoiding unnecessary switching to second-line ART. However, the cost and complexity of routine VL testing remains prohibitive in most resource limited settings (RLS). We evaluated a simple, low–cost, qualitative viral–failure assay (VFA) on dried blood spots (DBS) in three clinical settings in Uganda. Methods We conducted a cross–sectional diagnostic accuracy study in three HIV/AIDS treatment centres at the Joint Clinical Research Centre in Uganda. The VFA employs semi-quantitative detection of HIV–1 RNA amplified from the LTR gene. We used paired dry blood spot (DBS) and plasma with the COBASAmpliPrep/COBASTaqMan, Roche version 2 (VLref) as the reference assay. We used the VFA at two thresholds of viral load, (>5,000 or >1,000 copies/ml). Results 496 paired VFA and VLref results were available for comparative analysis. Overall, VFA demonstrated 78.4% sensitivity, (95% CI: 69.7%–87.1%), 93% specificity (95% CI: 89.7%–96.4%), 89.3% accuracy (95% CI: 85%–92%) and an agreement kappa = 0.72 as compared to the VLref. The predictive values of positivity and negativity among patients on ART for >12 months were 72.7% and 99.3%, respectively. Conclusions VFA allowed 89% of correct classification of VF. Only 11% of the patients were misclassified with the potential of unnecessary or late switch to second–line ART. Our findings present an opportunity to roll out simple and affordable VL monitoring for HIV–1 treatment in RLS.


Introduction
HIV/AIDS remains one of the world's critical public health challenges with 36.9 million people living with HIV and 1.2 million AIDS-related deaths at the end of 2014 [1,2]. Sub-Saharan Africa, which represents 2.1% of the global Gross domestic product (GDP) [3], is disproportionately affected and holds 70% (25.8 million) of the world's HIV/AIDS burden [1,2]. Nonetheless, recent evidence demonstrates unprecedented milestones in the global AIDS response with a decline in the number of new infections and deaths [1,2].
Indeed there has been an exponential increase in ART coverage since 2003, with 41% (15 million people) of eligible Persons Living With HIV/AIDS (PLWHAs) accessing therapy in sub-Saharan Africa as of march 2015 [1,2]. This rapid expansion in ART coverage creates an urgent need for a strengthened laboratory support network for early diagnosis of HIV, timely monitoring of HIV treatment and early detection of resistance due to failing ART regimens.
Despite existing evidence and the 2013 WHO recommendations that VL testing is crucial in predicting clinical outcomes among PLWHAs taking ART [4], implementation considerations are inhibiting the scale up of this technology in sub-Saharan Africa. A recent report indicates that less than 20% of African patients on ART have access to a VL testing [5]. Costs and complexity are often prohibitive due to expensive VL detection equipment, the need for human resource training and laboratory infrastructure as well as operational challenges in sample collection, transport, storage and processing. Notwithstanding, the WHO makes a strong recommendation that VL is the preferred monitoring approach to diagnose and confirm ART failure.
A previous systematic review published in 2009 indicated that DBS are a practical alternative specimen source to liquid plasma for HIV testing, in terms of a stable specimen matrix, ease of sample collection, storage and transportation [6]. We have previously reported the development of a qualitative VFA which is simple optimizes an open platform and is compatible with finger or heel prick DBS collection [7]. This assay was specifically designed to function as a tie-breaker for a subsequent HIV-1 drug resistance test [8]. In the current paper, we report the performance of this VFA as a screening tool to determine treatment failure using DBS among PLWHAs.

Ethical considerations
We obtained ethical clearance for the use of patient sample material was obtained through the Ethics review committees of JCRC, the Uganda National Council of Science &Technology (UNCST), and the Academic Medical Center of the University of Amsterdam, Netherlands. All adult participants, and parent(s) or guardian(s) of all eligible children provided written informed consent. Children above the age of 8 years who were aware of their HIV status provided written informed assent.

Setting
The Joint Clinical Research Centre (JCRC) is a pioneer HIV/AIDS care, research and training institute in Africa founded in 1991, (www.jcrc.org.ug). The JCRC operates a network of 7 Regional Centres of Excellence in Uganda, which provide comprehensive AIDS care and advanced laboratory tests including VL measurements. Mbale, Fort Portal and Kampala have high patient loads, are geographically representative and therefore selected for this study. In 2012, the 3 centres attended to at least 10,000 PLWHAs, whilst over 100,000 PLHWAs have ever accessed services at JCRC sites countrywide.

Design
In a cross-sectional diagnostic accuracy study between 2012 and 2013, we compared the performance of the index VFA to the standard reference VL test (VL ref ). We report these findings in line with the Standards for Reporting Diagnostic Accuracy Studies statement (STARD) [9].

Participants, sampling and sample size estimation
We collected DBS samples from HIV-1 positive individuals participating in the PASER-M program [10] (36 month follow-up) and MARCH [11] studies (baseline, 6, 12 months). Using Burderer's formula [12] we estimated that a sample size of at least 490 would be sufficient to test for diagnostic accuracy, anticipating VFA sensitivity of 85%, a true virological failure of 10% and 10% precision for a 95% Confidence Interval. These parameters were informed by our previous studies, [7,13].

Sample storage and Transportation
We stored DBS and plasma samples at -20°C and -80°C respectively, tested on site using robust affordable real-time PCR instruments (Mini-opticon). We transported plasma samples in dryice to the reference laboratory in Kampala for the reference testing (VL ref ) using the COBA-SAmpliPrep/ COBASTaqManSystem,Roche version 2.

Nucleic Acid Isolation
We described details of the laboratory procedures in our previous work published elsewhere [7]. We performed nucleic acid isolation from DBS using the QIAamp RNA kit (Qiagen GmbH, Germany), according to manufactures instructions. Briefly, we excised DBS samples by hand using scissors and placed two of the spots in lysis buffer and incubated at room temperature with gentle rotation for 30 min, subsequent washes and a final elution of 50μl.

Reverse Transcription
We conducted the reverse transcription as described elsewhere [6]. In short, we used the eluent comprising both HIV-1 RNA and internal control RNA, and reverse transcribed using the TaqMan Real-time PCR system Random Hexamers RT kit (Life Technologies, Foster City, CA) according to the manufacturer's instructions.

HIV-1 Viral subtype
We determined the HIV-1 viral subtype in our laboratories in Amsterdam (based on baseline HIVDR sequence data) using REGA HIV-1 subtyping tool version 3.0.

Outcome measures
These included the following measures of diagnostic accuracy: sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV), area under curve (AUC), kappa agreement, likelihood ratios (LR) and Diagnostic Odds Ratios (DOR).

Data management and statistical considerations
We captured clinical and laboratory data onsite (single data entry) during routine patient visits, entered these into a central database and controlled for quality using query systems. We used Stata1 version 11.2 (College Station, Texas, USA) for analysis. We tabulated patient characteristics into frequencies, proportions and appropriate measures of central tendency. Thereafter, we employed the Stata1 commands "roctab", "roccomp", "rocgold" and "diagti" to estimate the diagnostic accuracy outcome measures compared to the reference standard, with 95% confidence intervals where appropriate. We used two definitions of viral failure as greater than 5,000 viral copies/mL (in line with the initial WHO cut-off for switching to second-line antiretroviral therapy) and greater than 1000 copies (in line with the most recent WHO guidelines [4]). We conducted sensitivity analyses by varying age strata (children, adults), duration on antiretroviral therapy (<6 versus 12-36 months), HIV-1 viral subtype and study site (Kampala, Fort Portal and Mbale). Children tend to have higher VL and their adherence patterns largely depend on their adult guardians [14]; duration on ART reflects the durability of treatment effect, whilst HIV-1 viral subtype may vary the treatment effect and user dependence may explain variation of assay performance across the study sites. We computed costs for equipment and kits from multiple sources including local facility procurement records, published studies or conference presentations, or websites of the manufacturer (http://www.bio-rad.com/ and http://www.roche.com/). We obtained salary information for Laboratory Technicians from the Uganda Public Service Single Spine Structure. We converted all costs to 2015 US dollars, (One US dollar~Uganda shillings 2,940).

Clinical and socio-demo graphic characteristics of study participants
Four hundred and ninety six samples of paediatric (MARCH) and adult (PASER) HIV patients on ART were consecutively submitted for VL testing. We tested 168 (33.9%) from the MARCH cohort with a mean age of 5.6 years (SD: 3.5), of whom 54% were girls. The PASER cohort constituted 328 adults (66.2%) with a mean age of 38.2 (9) and majority were female (57.6%). The 496 samples were geographically from Mbale (147), Fort Portal (151) and Kampala (198). One hundred twenty four baseline samples were included and 372 participants (75%) had been on ART for at least 6 months allowing for assessment of VFA. Subtypes A (54.5%) and D (33.2%) were the predominant HIV-1 viral subtypes. Details of baseline characteristics are in Table 1.

Overall performance of the Viral Failure Assay
The prevalence of VL 5,000 copies/mL, our primary cut-off point, was 27.0% (95% CI: 22.9%-32.9%) as determined by the gold standard VLref assay and depicted in Table 2 Fig 1).

Sensitivity analyses
The prevalence of VL 1,000 copies/mL, our primary cut-off point, was 33.1%. At this lower HIV treatment viral failure threshold of 1,000 cp/mL, we recorded a higher yield of 95.7% sensitivity (95% CI: 91.8%-99.7%), but lower specificity of 38.6% (95% CI: 32-45%) and 57.4% accuracy (95% CI: 51.9%-63%) for the VFA as compared to VL ref . These results are shown in  3) correspondingly. One hundred and twenty four samples were naïve to ART and the AUC for this sub-population was 0.75 compared to 0.90 for those exposed to ART (Fig 2). Each point on the ROC curve represents a sensitivity/specificity pair corresponding to a particular decision threshold. A test with perfect discrimination (no overlap in the two distributions) has a ROC curve that passes through the upper left corner (100% sensitivity, 100% specificity). Therefore the closer the ROC curve is to the upper left corner, the higher the overall accuracy of the test. An area of 1 represents a perfect test whilst 0.5 is a worthless test.  Table 2).
The VFA had a better sensitivity for HIV-1 viral subtype A at 81.4% compared to 75% for subtype D, at a cut-off of >5,000 copies/ml. When we lowered the cut off to >1,000 copies/ml, the sensitivity results were similar at 97.5% (subtype A) and 95.1% (subtype D).

Costing analysis
In

Discussion
The exponential increase to nearly 15 million PLWHAs on ART by march 2015 and the shift from access to quality care, makes laboratory monitoring of HIV treatment increasingly important [1,2]. In light of the progressively shrinking donor basket, HIV treatment programs in RLS are expected to do more with less funding [15,16,17] and thus practical solutions to this problem are urgently needed. The VFA test was developed as part of an affordable HIVDR test algorithm, (ART-A). The VFA is unique, as it does not attempt to amplify target sequences irrelevant to HIV drug resistance monitoring in RLS (protease gene). Noteworthy, greater than 95% of PLWHAs in RLS are on first-line ART, which do not contain the Protease Inhibitors and are preserved for second-line in PLWHAs failing first-line Reverse Transcriptase inhibitor regimen. In addition, the VFA does not attempt to provide an absolute copy number of viral genomes/ml but rather focuses on a given threshold that is clinically significant (5000 or 1000 copies/ml). We report a Viral Failure Assay on DBS that is practical, feasible, and highly specific, with moderate sensitivity, and is fairly accurate (9 out of 10 patients with VF were accurately identified). This VFA had a mixed performance with generally better overall results at a higher threshold of 5000 copies/ml. Nonetheless, the VFA had varying results with a higher sensitivity at a lower threshold of 1,000 copies/ml and at a higher specificity at a higher threshold of 5000 copies/ml. This variation in performance was also seen in adults compared to children, across HIV-1 viral subtypes, study sites and with ART status.
Although, the VFA overall performance was lower in terms of sensitivity and specificity than the gold standard, this shortcoming may be outweighed by the benefits of its feasibility and cost. The VFA performance could be considered in those situations where otherwise only clinical and immunological (CD4) monitoring would be available, which is suboptimal as compared to routine VL monitoring [18,19]. Further, the VFA showed a predictive value of positivity of 81.3% in the field among cohorts of PLWHAs on ART. Research literature demonstrates that CD4 testing has a low positive predictive value of viral failure [19]. It is thus prudent to suggest that the VFA provides an advantage in identifying PLWHAs on ART who require switching to 2 nd line therapy. The VFA generally demonstrated better receiver operating characteristics at a higher viral load threshold of 5,000 copies/mL, consistent with a recent systematic review [20]. In fact, this review showed that specificity was close to 100% at DBS VL above 5000 copies/ml, and this threshold would be the most reliable for predicting true virologic failure using DBS [20]. This puts into perspective the current definition of VF by WHO set at 1000 copies/ml [4] and questions whether this decision results in ineligibility of (promising) in house VL tests and in fact slows down roll-out of VL monitoring in RLS.
The finding that the VFA performed better in adults than children at our higher primary threshold of >5,000 copies/ml remains a paradox; particularly in light of the documented higher VL in Children as compared to adults [14]. A practical explanation could be that the volume and quality of DBS taken from children is less compared to adults suggesting an issue of volume input contrary to the viral load.
Our study was not without limitations. First the VFA performance was sub-optimal with lowering of the VF threshold to 1,000 copies/mL, increasing the possibility of false negatives. Hence, refining a simple technique for nucleic acid extraction from the DBS to allow maximum yield of nucleic acids, concentration of reagents, standardizing the volume of blood during sample collection could be considered in the future development of the VFA. These modifications would however be made at higher cost. Secondly, the VFA showed variability in performance across the Ugandan study sites, which highlights the need for further standardisation of the assay. Genetic variability is unlikely to explain these cross-site differences since Aitken and colleagues [7] previously optimized the assay for different HIV subtypes, which performed equally well across the various HIV-1 subtypes. Indeed there was no variation in the distribution of the HIV-1 viral subtypes across the study sites (55.2%, 55.8% and 61.4% for subtype D in Mbale, Mbarara and Kampala respectively) and among the samples that failed. Lastly, we did not consider maintenance costs, depreciation of equipment and costs due to electricity and waste disposal in our costing analysis. However, such costs would likely be much higher for the reference standard with bigger and more sophisticated equipment.