Advertisement

Development of Methods for Cross-Sectional HIV Incidence Estimation in a Large, Community Randomized Trial

  • Oliver Laeyendecker ,

    Contributed equally to this work with: Oliver Laeyendecker, Michal Kulich

    olaeyen1@jhmi.edu

    Affiliations: National Institute of Allergy and Infectious Diseases, National Institutes of Health, Bethesda, Maryland, United States of America, Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States of America

  • Michal Kulich ,

    Contributed equally to this work with: Oliver Laeyendecker, Michal Kulich

    Affiliation: Department of Probability and Statistics, Faculty of Mathematics and Physics, Charles University in Prague, Prague, Czech Republic

  • Deborah Donnell,

    Affiliation: Statistical Center for HIV/AIDS Research and Prevention, Fred Hutchinson Cancer Research Center, Seattle, Washington, United States of America

  • Arnošt Komárek,

    Affiliation: Department of Probability and Statistics, Faculty of Mathematics and Physics, Charles University in Prague, Prague, Czech Republic

  • Marek Omelka,

    Affiliation: Department of Probability and Statistics, Faculty of Mathematics and Physics, Charles University in Prague, Prague, Czech Republic

  • Caroline E. Mullis,

    Affiliation: Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States of America

  • Greg Szekeres,

    Affiliation: UCLA Program in Global Health, University of California Los Angeles, Los Angeles, California, United States of America

  • Estelle Piwowar-Manning,

    Affiliation: Department of Pathology, Johns Hopkins University, School of Medicine, Baltimore, Maryland, United States of America

  • Agnes Fiamma,

    Affiliation: UCLA Program in Global Health, University of California Los Angeles, Los Angeles, California, United States of America

  • Ronald H. Gray,

    Affiliations: Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, United States of America, Rakai Health Science Program, Entebbe, Uganda

  • Tom Lutalo,

    Affiliation: Rakai Health Science Program, Entebbe, Uganda

  • Charles S. Morrison,

    Affiliation: Clinical Sciences, Family Health International, Durham, North Carolina, United States of America

  • Robert A. Salata,

    Affiliations: Division of Infectious Diseases and HIV Medicine, Case Western Reserve University, Cleveland, Ohio, United States of America, University Hospitals, Case Medical Center, Cleveland, Ohio, United States of America

  • Tsungai Chipato,

    Affiliation: University of Zimbabwe, Harare, Zimbabwe

  • Connie Celum,

    Affiliation: Department of Global Health, Medicine and Epidemiology, University of Washington, Seattle, Washington, United States of America

  • Erin M. Kahle,

    Affiliation: Department of Epidemiology, University of Washington, Seattle, Washington, United States of America

  • Taha E. Taha,

    Affiliation: Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, United States of America

  • Newton I. Kumwenda,

    Affiliation: Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, United States of America

  • Quarraisha Abdool Karim,

    Affiliations: Department of Epidemiology, Mailman School of Public Health, Columbia University, New York, New York, United States of America, Centre for the AIDS Programme of Research In South Africa (CAPRISA), Doris Duke Medical Research Institute, Nelson R Mandela School of Medicine, University of KwaZulu-Natal, Congella, South Africa

  • Vivek Naranbhai,

    Affiliation: Centre for the AIDS Programme of Research In South Africa (CAPRISA), Doris Duke Medical Research Institute, Nelson R Mandela School of Medicine, University of KwaZulu-Natal, Congella, South Africa

  • Jairam R. Lingappa,

    Affiliation: Departments of Global Health, Medicine and Pediatrics, University of Washington, Seattle, Washington, United States of America

  • Michael D. Sweat,

    Affiliation: Department of Psychiatry and Behavioral Sciences, the Medical University of South Carolina, Charleston, South Carolina, United States of America

  • Thomas Coates,

    Affiliation: UCLA Program in Global Health, University of California Los Angeles, Los Angeles, California, United States of America

  •  [ ... ],
  • Susan H. Eshleman

    Affiliation: Department of Pathology, Johns Hopkins University, School of Medicine, Baltimore, Maryland, United States of America

  • [ view all ]
  • [ view less ]

Development of Methods for Cross-Sectional HIV Incidence Estimation in a Large, Community Randomized Trial

  • Oliver Laeyendecker, 
  • Michal Kulich, 
  • Deborah Donnell, 
  • Arnošt Komárek, 
  • Marek Omelka, 
  • Caroline E. Mullis, 
  • Greg Szekeres, 
  • Estelle Piwowar-Manning, 
  • Agnes Fiamma, 
  • Ronald H. Gray
PLOS
x
  • Published: November 13, 2013
  • DOI: 10.1371/journal.pone.0078818

Abstract

Background

Accurate methods of HIV incidence determination are critically needed to monitor the epidemic and determine the population level impact of prevention trials. One such trial, Project Accept, a Phase III, community-randomized trial, evaluated the impact of enhanced, community-based voluntary counseling and testing on population-level HIV incidence. The primary endpoint of the trial was based on a single, cross-sectional, post-intervention HIV incidence assessment.

Methods and Findings

Test performance of HIV incidence determination was evaluated for 403 multi-assay algorithms [MAAs] that included the BED capture immunoassay [BED-CEIA] alone, an avidity assay alone, and combinations of these assays at different cutoff values with and without CD4 and viral load testing on samples from seven African cohorts (5,325 samples from 3,436 individuals with known duration of HIV infection [1 month to >10 years]). The mean window period (average time individuals appear positive for a given algorithm) and performance in estimating an incidence estimate (in terms of bias and variance) of these MAAs were evaluated in three simulated epidemic scenarios (stable, emerging and waning). The power of different test methods to detect a 35% reduction in incidence in the matched communities of Project Accept was also assessed. A MAA was identified that included BED-CEIA, the avidity assay, CD4 cell count, and viral load that had a window period of 259 days, accurately estimated HIV incidence in all three epidemic settings and provided sufficient power to detect an intervention effect in Project Accept.

Conclusions

In a Southern African setting, HIV incidence estimates and intervention effects can be accurately estimated from cross-sectional surveys using a MAA. The improved accuracy in cross-sectional incidence testing that a MAA provides is a powerful tool for HIV surveillance and program evaluation.

Introduction

Accurate methods for estimating HIV incidence are needed to monitor the epidemic and evaluate interventions for HIV prevention [1]. In clinical trials, HIV incidence is usually assessed by enrolling HIV-uninfected individuals and following them over time to detect HIV acquisition. An alternate approach is to assess HIV incidence by analyzing specimens from cross-sectional surveys without longitudinal follow-up [2]. This approach may be needed for evaluation of population-level interventions for HIV prevention, particularly when HIV testing is part of a combination prevention strategy [3], [4]. In this report, we describe the development of methods that were used to analyze HIV incidence in a large, Phase III community randomized trial: National Institute of Mental Health (NIMH) Project Accept (HIV Prevention Trials Network 043 [HPTN 043]) [5]. Project Accept is one of the largest randomized, controlled trial performed to date, and is the first randomized controlled trial with a primary study endpoint based solely on cross-sectional estimation of HIV incidence.

Project Accept evaluated the impact of integrated behavioral interventions on HIV incidence in 48 paired communities (34 in Africa, 14 in Thailand) [16]. Control communities received standard, clinic-based, voluntary counseling and testing services; intervention communities received enhanced, community-based voluntary counseling and testing services. After a 3-year intervention period, samples were collected from individuals in the communities (aged 18 to 32 years) in a single cross-sectional survey. When the trial was designed, the study plan was to estimate HIV incidence using the BED capture immunoassay (BED-CEIA, Calypte Biomedical Corporation, Lake Oswego, OR, USA) [6]. That approach was not used because the BED-CEIA was later found to overestimate incidence in many settings [7].

In this report, we describe the laboratory and statistical analysis that was used to identify an alternate testing strategy for HIV incidence estimation in Project Accept. The testing strategies that were evaluated used multiple biomarkers to assess HIV incidence [8]. This approach was based on recent success using a multi-assay algorithm (MAA) to estimate HIV incidence in populations in the United States (clade B settings) [9][11]. That MAA combines serologic assays (the BED-CEIA and an antibody avidity assay) with non-serologic biomarkers (CD4 cell count and HIV viral load) to identify individuals who were likely to have been recently infected at the time of sample collection (referred to in this report as MAA positive). In Project Accept, because HIV prevalence in the communities in Thailand was low (<1%, [12]), data from Thailand were not included in the primary endpoint analysis. Therefore, we focused on identifying a MAA that could be used to estimate incidence in the African communities of the trial, using validation samples obtained from seven African cohorts.

Development of methods for cross-sectional HIV incidence estimation is challenging for several reasons. First, an assay or MAA must have a suitable mean window period; this term refers to the average period of time that individuals are identified as positive by a specific assay or MAA. If the window period is too short, fewer individuals will be classified as positive, resulting in higher variance and lower precision of incidence estimates, reducing the power to determine an intervention effect. Conversely, if the window period is too long, the precision of incidence estimates will be reduced because of high bias; furthermore, if too many individuals with long-term infection (e.g, infected >1 year) test positive by an assay or MAA, the incidence estimates will not reflect the current epidemic. Bias is reduced when the probability of being classified as positive approaches zero as the time since infection increases [13], [14]. The performance of serologic assays used for cross-sectional incidence estimation may also be affected by HIV viral load, frequency and duration of antiretroviral treatment (ART), the stage of HIV disease, HIV subtype, and race [15][21]. Finally, the performance of assays and MAAs for incidence estimation varies by the stage of the epidemic. For example, a given test method may perform well in an emerging epidemic with high incidence, but may not perform well in waning epidemic where incidence is low and many individuals have advanced HIV disease. In this report, we describe the laboratory and statistical methods used to identify a MAA for incidence analysis in Project Accept.

Methods

Samples used for analysis

Samples obtained from seven African cohort studies and clinical trials (Table 1) were used for validation. Samples were selected based on availability of stored plasma, known infecting subtype, known duration of infection (known date of a prior positive and/or negative HIV test), and available CD4 cell count data from the time of sample collection. Of the samples that did not have a known prior HIV negative time point, 99.3% were from individuals who were HIV seropositive for >1 year and 52.7% were known to be from individuals who were HIV seropositive for >2 years. Infection times were either interval-censored (the dates of the first positive and the last negative HIV tests were available) or right-censored (the date of the last negative test was unknown). In the latter case, we assigned the 14th birthday as the date of the last negative test and treated the infection time as interval-censored. Interval-censored infection times were randomly imputed in the simulations. For almost all of the samples, the infection time was generated from the uniform distribution between the last negative and first positive test dates. However, for 147 samples from subjects who had another visit following the current sample date, the infection time was generated from a posterior Weibull density truncated to the interval between the last negative and first positive test dates. We assumed a Weibull distribution on the survival of HIV-infected subjects and a uniform prior distribution of the infection time, and calculated the posterior distribution of the infection time given the date of the last known visit. We used the Weibull survival distribution with the shape parameter 1.856 and rate parameter 4358.5 days. The truncated Weibull density puts more weight on the more recent infection times if the subject is known to have survived for a long time after the first positive test date and so improves the precision of the imputed infection times compared to the uniform distribution. More details on these methods are presented elsewhere [22]. Samples from Botswana, Malawi, South Africa, and Zimbabwe were assumed to come from individuals infected with HIV subtype C [23]. The HIV subtypes of samples from Uganda and Kenya were determined previously [24][28].

thumbnail
Table 1. Samples used for analysis.

doi:10.1371/journal.pone.0078818.t001

Serologic testing

Samples were tested by the BED-CEIA; [6] samples were run in duplicate and the average normalized optical density (OD-n) value was used for analysis. Antibody avidity was measured using a modified version of the Genetic Systems 1/2+O ELISA (BioRad, Hercules, CA) [29]. For this assay, duplicate sample aliquots were diluted 1:10 and incubated at 4°C for 30 minutes (initial antibody-binding step). Samples were then incubated for 30 minutes at 37°C with or without the chaotropic agent, diethylamine (antibody disassociation step). The avidity index (AI) was calculated as follows: AI = [optical density of the diethylamine-treated well]/[optical density of the non-treated well] ×100.

HIV viral load testing

If a viral load result was not available, viral load testing was performed using the Amplicor HIV-1 Monitor test version 1.5 (Roche Diagnostics, Indianapolis, IN). Viral load testing was only performed for samples that had a BED-CEIA result <1.5 OD-n, an AI <90%, and a CD4 cell count >150 cells/mm3.

Statistical methods

To compare the performance of various testing algorithms, we constructed 403 MAAs that included one or more of the following assays: BED-CEIA, the avidity assay, CD4 cell count, and HIV viral load. The ranges of cutoffs used for each assay were as follows: BED-CEIA: 0.5 to 1.5 OD-n (steps of 0.1 OD-n); the avidity assay: 30% to 90% (steps of 10% AI); CD4 cell count: 150, 200, and 250 cells/mm3; all MAAs that included HIV viral load used a cutoff of >400 copies/mL. The MAAs included the BED-CEIA alone, the avidity assay alone, or these two assays in combination with or without inclusion of CD4 cell count; HIV viral load was only included for MAAs that included a CD4 cell count with a cutoff >200 cells/mm3.

The mean window period for each MAA was calculated by integrating estimated sensitivity of the MAA, as described [9], with the following caveat: the maximum duration of HIV infection was assumed to be 12 years [22]. The performance of the MAAs for estimating HIV incidence was evaluated in simulated populations with 10% prevalence using three epidemic scenarios (emerging, stable, and waning epidemics). Detailed descriptions of the statistical methods and simulation exercises are described elsewhere [22], The three scenarios were simulated using data from the sample set described in Table 1. For each scenario, data from 200 individuals with different durations of HIV infection were repeatedly sampled. The bias, variance, and root mean square error (RMSE) of the incidence estimate for each of the 403 MAAs was determined for each of the three scenarios. The RMSE measures the overall precision of estimated incidence by combining the bias and variance. The same performance measures were used to evaluate incidence estimates calculated from simulated 6-month and one-year follow-up assessments. In the final evaluation, the data were used to simulate the capacity of selected MAAs to accurately estimate a difference in HIV incidence in the control and intervention communities of Project Accept [22], Finally, the power to detect a 35% decrease in incidence and coverage of confidence intervals for the intervention effect were determined for a stable epidemic setting. The results were compared to simulated intervention effect estimates obtained by 6-month follow-up.

Human subjects protection

Written informed consent was obtained from study participants and all studies were reviewed and approved by relevant institutional review boards. The study for cross sectional incidence testing on stored study samples was approved by the institutional review board of the Johns Hopkins University. The primary studies for the collection of the samples evaluated were approved by the University of KwaZulu-Natal Biomedical Research Ethics Committee, Medical Research Council of Zimbabwe, Family Health International Protection of Human Subjects Committee, the South African Medicines Control Council, the institutional review board at the University of Malawi, Ethics Committee of the Uganda Virus Research Institute and the Committee for Human Research at Johns Hopkins University and the University of Washington Human Subjects Review Committee.

Results

Performance of the BED-CEIA and avidity assay in subtypes A, C, and D

Because HIV subtype can affect performance of the BED-CEIA and avidity assays [18], [30], we first considered the potential impact of HIV subtype on incidence estimation in Project Accept which was conducted in South Africa, Zimbabwe, and Tanzania. Most HIV infections in South Africa and Zimbabwe are subtype C, while Tanzania has three prevalent subtypes: A, C, and D [23]. To evaluate incidence algorithms for use in Project Accept, we obtained validation samples from seven African studies conducted in countries where subtypes A, C, and D are prevalent (Table 1). This sample set included 5,325 samples from 3,436 individuals with known duration of HIV infection (from 1 month to >10 years, see Methods).

First, we evaluated the performance of the BED-CEIA and avidity assay as a function of duration of infection for subtypes A, C, and D (Figure 1). The mean window period for each assay and each subtype were obtained using the standard assay cutoffs (Table 2). For all three subtypes, the mean window period was longer for the BED-CEIA alone than for the avidity assay alone. For both assays, the mean window periods varied considerably among subtypes and were longest for subtype D (>2.5 years for the BED-CEIA, >1.5 years for the avidity assay). We also compared the proportion of individuals infected >2 years who had BED-CEIA results <0.8 OD-n or avidity assay results <40% AI (Table 2). Previous reports indicated that the frequency of subtype D in Tanzania was low [12], [23]; this was confirmed by subtyping a subset of the samples from Project Accept [31]. Because subtype D infections were not likely to have a significant impact on HIV incidence estimates in Project Accept, subsequent analyses did not include subtype D validation samples. After removing the subtype D samples, the validation sample set included 4,166 samples from 2,882 individuals (median age: 27.8 years, interquartile range [IQR]: 24.1 to 32.5 years; 88% women). The performance of the BED-CEIA and the avidity assay was similar for subtypes A and C (Figure 1, Table 2).

thumbnail
Figure 1. BED-CEIA and avidity assay results for HIV subtypes A, C, and D.

Samples from the validation sample set were analyzed using the BED-CEIA (Panels A–C) and the avidity assay (Panels D–F). Results are shown for each assay for subtypes A, C, and D as a function of duration of HIV infection (years after HIV seroconversion). Data are shown for 50 randomly-selected samples for each 6-month interval after seroconversion. The HIV incidence testing algorithms evaluated in this report only included algorithms with BED-CEIA results ≤1.5 OD-n or avidity results ≤90% AI (dashed lines).

doi:10.1371/journal.pone.0078818.g001

thumbnail
Table 2. Window periods and classification of individuals with long-standing infection as positive, for the BED-CEIA alone, the avidity assay alone, a two-assay multi-assay algorithm (MAA) and two 4-assay MAAs*.

doi:10.1371/journal.pone.0078818.t002

Performance of MAAs for identifying individuals with recent HIV infection

We evaluated the performance of 403 MAAs (see Methods). The mean window period for each MAA is presented in Table S1. Below, we present more detailed information for four of the testing algorithms: the BED-CEIA alone (using the standard assay cutoff of 0.8 OD-n), the avidity assay alone (using the standard assay cutoff of 40% AI)], and two MAAs that include the BED-CEIA, the avidity assay, CD4 cell count and viral load. One of these two MAAs was shown to accurately estimate HIV incidence in subtype B settings (BED-CEIA <1.0 OD-n + AI <80% + CD4 cell count >200 cells/mm3 + viral load >400 copies/mL) when testing the samples at the end of follow-up in three clinical studies and comparing the cross-sectional incidence estimates to the incidence observed during study follow-up [9][11]. Based on its performance described below, the other MAA was ultimately selected for endpoint analysis in Project Accept (BED-CEIA <1.2 OD-n + AI <90% + CD4 cell count >200 cells/mm3 + viral load >400 copies/mL) [22].

The proportion of samples positive for each of the four testing algorithms was determined as a function of duration of infection (Figure 2). Consistent with results shown in Figure 1, a high proportion of individuals infected >2 years were positive by the BED-CEIA alone (7.9%). The other three testing approaches identified a lower proportion of these long-term infections as positive (1.9% for the avidity assay, 0.8% and 1.5% for the 4-assay MAAs). For individuals infected <6 months, 68.5% were positive using the avidity assay alone, 75.6% were positive using the MAA that was previously optimized for incidence estimation in clade B epidemics, and 81.5% were positive using the MAA that was ultimately selected for analysis of the Project Accept endpoint. The MAA that was ultimately selected for use in Project Accept identified 27.5% of those infected 0.5–1 year and 8.8% of those infected 1–2 years as positive.

thumbnail
Figure 2. Proportion of samples classified as positive using the BED-CEIA alone, the avidity assay alone, and two MAAs.

Subtype A and C samples were analyzed using the BED-CEIA alone (using the standard assay cutoff of 0.8 OD-n, black bars), the avidity assay alone (using the standard assay cutoff of 40% AI, dark grey bars), and two MAAs that included multiple biomarkers, (BED-CEIA <1.0 OD-n + AI <80% + CD4 cell count >200 cells/mm3 + viral load >400 copies/mL, medium grey bars; BED-CEIA <1.2 OD-n + AI <90% + CD4 cell count >200 cells/mm3 + viral load >400 copies/mL, light grey bars). For each test method, the percentage of samples classified as positive was determined as a function of the duration of HIV infection (years after HIV seroconversion). N indicates the number of samples analyzed for each time period (e.g., 0–0.5 years after seroconversion).

doi:10.1371/journal.pone.0078818.g002

Performance of MAAs for estimating HIV incidence in different epidemic scenarios

The overall performance of incidence assays and MAAs is affected by the distribution of infection times in a population. Because the stage of the HIV epidemic in the Project Accept communities was not known, we evaluated the performance of the 403 MAAs for estimating HIV incidence in three simulated epidemic scenarios: emerging, stable, and waning epidemics. These scenarios were constructed by randomly selecting samples from the validation data set so that the distribution of durations of infection corresponded to the desired scenario (see Methods, Figure 3). In the stable epidemic, 5.5% of individuals were infected <0.5 years, 13.3% were infected 0.5–2 years, 29.0% were infected 2–5 years, 33.4% were infected 5–10 years, and 17.8% were infected >10 years. For the emerging epidemic, these percentages were 17.7%, 42.9%, 38.4%, and 1.0%, respectively; for the waning epidemic, these percentages were 0.7%, 1.6%, 44.0%, and 53.7%, respectively.

thumbnail
Figure 3. Simulated epidemic scenarios.

HIV incidence testing algorithms were assessed using three simulated epidemic scenarios: an emerging epidemic (black bars), a stable epidemic (dark grey bars), and a waning epidemic (light grey bars). The plot shows the percentage of HIV-positive samples included in each scenario for different time periods (years after HIV seroconversion).

doi:10.1371/journal.pone.0078818.g003

The bias, variance, and RMSE were calculated for each MAA in the three epidemic scenarios. Results obtained using the BED-CEIA alone, the avidity assay alone, the best performing 2-assay MAA (BED-CEIA <0.8 OD-n + AI <70%) and the two 4-assay MAAs described above are shown in Table 3; results for all 403 MAAs are shown in Supplemental Table 1. The bias reflects the difference between the estimated incidence and the true incidence. Among the 403 MAAs, the bias ranged from −6.1% to −63.3% in the emerging epidemic scenario, from −9.2% to −26.6% in the stable epidemic scenario, and from 40.6% to 346.7% in the waning epidemic scenario. In the stable and emerging epidemic scenarios, the bias was lower for MAAs that used serologic assays in combination with CD4 cell count and viral load (Table S1). The RMSE ranged from 0.20 to 1.01 in the emerging epidemic scenario, from 0.28 to 0.55 in the stable epidemic scenario, and from 0.65 to 1.50 in the waning epidemic scenario. The best performing two assay MAA was ranked 186 out of the 403 algorithms tested. Though this MAA was comparable to the best 3- and 4-assay MAAs in the waning epidemic scenario, it performed poorly in stable and emerging scenarios, In the emerging and waning epidemic scenarios, the lowest precision values (highest RMSEs) were obtained using the BED-CEIA alone or the avidity assay alone.

thumbnail
Table 3. Accuracy of incidence estimates obtained using the BED-CEIA alone, the avidity assay alone, a two-assay multi assay algorithm (MAA), and two four-assay MAAs in three epidemic scenarios*.

doi:10.1371/journal.pone.0078818.t003

Finally, we compared the capacity of the four testing algorithms to accurately detect a 35% difference in HIV incidence in the control and intervention communities of Project Accept (Table 4) [22]. The intervention effect was accurately estimated by both of the 4-assay MAAs, but was underestimated using BED-CEIA alone or the avidity assay alone. The percentage of 95% confidence intervals that covered the true intervention effect was >93% for both 4-assay MAAs, but was unacceptably low for the BED-CEIA alone or the avidity assay alone. The MAA that was ultimately selected for primary endpoint analysis in Project Accept (BED-CEIA <1.2 OD-n + avidity index <90% + CD4 cell count >200 cells/mm3 + viral load >400 copies/mL) provided the largest power for detecting a 35% reduction in HIV incidence and had minimal bias in estimating incidence across differing epidemic scenarios. This MAA had better precision, power, and negligible bias compared to a simulated 6-month follow-up study using the same validation sample set. In a separate simulation, we showed that both of the 4-assay MAAs maintained the required probability of type I error, provided that the scenarios in the paired communities were the same.

thumbnail
Table 4. Capacity to estimate and detect a 35% reduction in HIV incidence in the Southern African communities of Project Accept using the BED-CEIA alone, the avidity assay alone, and two multi-assay algorithms (MAAs)*.

doi:10.1371/journal.pone.0078818.t004

Discussion

We evaluated the performance of incidence assays and MAAs using data from a large set of validation samples from Africa. These samples were from individuals with a broad range of infection times who had CD4 cell count data available from the time of sample collection. We found that testing algorithms that included multiple assays were superior to single serologic assays; the incidence estimates obtained using multiple assays had lower bias and better precision. We used simulation exercises to demonstrate that the 4-assay MAA that was selected for use in Project Accept provided a more precise estimate of the ratio between incidence in the intervention and control communities than would have been obtained by following a cohort for seroconversion over a 6-month period.

Our findings demonstrate the importance of including samples from very long-term infections when validating methods for cross-sectional HIV incidence estimation. Using a sample set that included individuals infected >10 years, the mean window period for the BED-CEIA was 1.63 years, which is approximately three times longer than the mean window period previously reported for this assay [18]. We found that the BED-CEIA and avidity assay frequently identified individuals infected >2 years as positive. This effect was most pronounced for subtype D. Previous studies have shown that misclassification of subtype D samples using these assays reflects differences in the serologic response to subtype D infection compared to subtype A infection; differences in the sequences of subtype D viruses in the region corresponding to the BED-CEIA target antigen also negatively affect assay performance [30]. We recommend against using the BED-CEIA or Bio-Rad avidity assays for cross-sectional incidence estimation in populations that include a substantial proportion of subtype D-infected individuals. Additional studies should be performed using different test methods [e.g., different assay (s) and/or different cutoff (s)] to identify an effective method for cross-sectional incidence estimation in subtype D endemic areas. Fortunately, the prevalence of subtype D was low in the Tanzanian communities in Project Accept [31]. Therefore, we felt it was reasonable to use a MAA that was optimized for subtypes A and C for estimating incidence in the four African sites in Project Accept.

In this report, we used a novel approach to compare the performance of a large set of MAAs that employed different combinations of assays and assay cutoffs in three different simulated epidemic scenarios. The MAAs that included both serologic assays and non-serologic biomarkers (CD4 count and viral load) had lower bias and variance for estimating incidence than algorithms based on a single assay. The MAA that provided the greatest power to detect a reduction in incidence in Project Accept used relatively high cutoffs for both the BED-CEIA (<1.2 OD-n) and the avidity assay (<90% AI), which increased the identification of individuals infected within 1 year. Use of these higher cutoffs for increased the frequency of misclassification, though it was still <1.5% for individuals infected >2 years.

HIV viral load was included in 77 of the 403 MAAs evaluated in this study. This biomarker identifies both elite suppressors and individuals who are virally suppressed from ART. It is important to identify both of these groups, since viral suppression is associated with down-regulation of the humoral immune response to HIV infection and with increased rates of false-recent misclassification using serologic HIV incidence assays [2]. ART also serves as an independent surrogate for non-recent HIV infection, since individuals with recent HIV infection are not likely to be identified or to access ART in many settings. We recognize that the 4-assay MAA selected for endpoint analysis in Project Accept may misclassify some individuals on ART who are not virally suppressed (e.g., those with low viral loads that are >400 copies/ml). This was addressed in Project Accept using a two-step approach: first, the MAA was used to identify samples from individuals who were likely to have had recent infection; second, these samples were tested for the presence of antiretroviral drugs [31]. When using antiretroviral drug testing in an HIV incidence assessment, it is important to consider the test results in the context of antiretroviral drug regimens that were used in the communities at the time the incidence survey was conducted; this should take into account use of antiretroviral drugs for HIV prevention (e.g., for prevention of mother-to-child transmission, where those receiving prophylaxis may have been recently infected).

The novel approach that was used to select an optimal testing algorithm for HIV incidence assessment in Project Accept is of general relevance to HIV prevention trials. We demonstrate that a large validation data set from individuals with known duration of infection can be used to assess the performance of various testing algorithms in terms of estimating incidence, providing estimates of bias and precision, and comparing the cross-sectional estimates to cohort-based estimates. We also used an empirical approach to determine assay cutoffs that optimized the precision of cross-sectional incidence estimates using MAAs. The methods described in this report could be used for cross-sectional incidence assessment in non-subtype D epidemics of Southern Africa for HIV prevention studies, surveillance, and other purposes.

Supporting Information

Table S1.

Window periods, bias, root square mean error (RMSE) for stable, emerging and waning epidemics for 403 multi-assay algorithms (MAAs). *

*BED: BED capture immunoassay (results expressed as normalized optical density units); AI: avidity assay (results expressed as a percentage, avidity index); CD4: CD4 cell count (results expressed as cells/mm3); VL: viral load (results expressed as HIV RNA copies/mL); yrs: years;

doi:10.1371/journal.pone.0078818.s001

(XLS)

Acknowledgments

We thank the communities that partnered with us in conducting this research, and all study participants for their contributions. We also thank study staff and volunteers at all participating institutions for their work and dedication. A portion of this research was presented at the 20th Conference on Retroviruses and Opportunistic Infections, Atlanta GA, March 2013. Disclaimer, the findings and conclusions in this article are those of the authors and do not necessarily represent the views of the National Institutes of Health (NIH). Furthermore, the use of trade names is for identification purposes only and does not constitute endorsement by the NIH. Views expressed are those of the authors, and not necessarily those of sponsoring agencies.

Author Contributions

Conceived and designed the experiments: OL MK DD SHE. Performed the experiments: OL CEM EP-M SHE. Analyzed the data: MK AK MO DD. Contributed reagents/materials/analysis tools: OL MK DD AK MO CEM GS EP-M AF RHG TL CSM RAS TC CC EMK TET NIK QAK VN JRL MDS TC SHE. Wrote the paper: OL MK DD AK MO CEM GS EP-M AF RHG TL CSM RAS TC CC EMK TET NIK QAK VN JRL MDS TC SHE.

References

  1. 1. Brookmeyer R (2010) Measuring the HIV/AIDS epidemic: approaches and challenges. Epidemiol Rev 32: 26–37. doi: 10.1093/epirev/mxq002
  2. 2. Busch MP, Pilcher CD, Mastro TD, Kaldor J, Vercauteren G, et al. (2010) Beyond detuning: 10 years of progress and new challenges in the development and application of assays for HIV incidence estimation. AIDS 24: 2763–2771. doi: 10.1097/qad.0b013e32833f1142
  3. 3. Chang LW, Serwadda D, Quinn TC, Wawer MJ, Gray RH, et al. (2013) Combination implementation for HIV prevention: moving from clinical trial evidence to population-level effects. Lancet Infect Dis 13: 65–76. doi: 10.1016/s1473-3099(12)70273-6
  4. 4. Padian NS, McCoy SI, Karim SS, Hasen N, Kim J, et al. (2011) HIV prevention transformed: the new prevention research agenda. Lancet 378: 269–278. doi: 10.1016/s0140-6736(11)60877-5
  5. 5. Sweat M, Morin S, Celentano D, Mulawa M, Singh B, et al. (2011) Community-based intervention to increase HIV testing and case detection in people aged 16–32 years in Tanzania, Zimbabwe, and Thailand (NIMH Project Accept, HPTN 043): a randomised study. Lancet Infect Dis 11: 525–532. doi: 10.1016/s1473-3099(11)70060-3
  6. 6. Dobbs T, Kennedy S, Pau CP, McDougal JS, Parekh BS (2004) Performance characteristics of the immunoglobulin G-capture BED-enzyme immunoassay, an assay to detect recent human immunodeficiency virus type 1 seroconversion. J Clin Microbiol 42: 2623–2628. doi: 10.1128/jcm.42.6.2623-2628.2004
  7. 7. UNAIDS Reference Group on Estimates MaP (2006) Statement on the use of the BED assay for estimation of HIV-1 incidence or epidemic monitoriing. Weekly Epidemiological Record 81: 33–40.
  8. 8. Mastro TD (2012) Determining HIV Incidence in Populations: Moving in the Right Direction. J Infect Dis.
  9. 9. Laeyendecker O, Wang L, Hughes J, Cummings V, Piwowar-Manning E, et al.. (2013) Use of a multi-assay algorithm for cross-sectional HIV incidence estimation in HPTN 061 (the Brothers Study). 20th Conf on Retroviruses and Opportunistic Infections. Atlanta, GA. Abstract# 1053.
  10. 10. Laeyendecker O, Brookmeyer R, Cousins MM, Mullis CE, Konikoff J, et al. (2013) HIV Incidence determination in the United States: A multi-assay approach. Journal of Infectious Diseases 207: 232–239. doi: 10.1093/infdis/jis659
  11. 11. Eshleman S, Hughes J, Laeyendecker O, Wang J, Brookmeyer R, et al. (2013) Comparison of three methods for assessing HIV incidence in a cohort study of women in the United States: HPTN 064. Journal of Infectious Diseases 207: 223–231. doi: 10.1093/infdis/jis658
  12. 12. Piwowar-Manning E, Fiamma A, Laeyendecker O, Kulich M, Donnell D, et al. (2011) HIV surveillance in a large, community-based study: results from the pilot study of Project Accept (HIV Prevention Trials Network 043). BMC Infect Dis 11: 251. doi: 10.1186/1471-2334-11-251
  13. 13. Brookmeyer R (2009) Should biomarker estimates of HIV incidence be adjusted? AIDS 23: 485–491. doi: 10.1097/qad.0b013e3283269e28
  14. 14. Brookmeyer R (2010) On the statistical accuracy of biomarker assays for HIV incidence. J Acquir Immune Defic Syndr 54: 406–414. doi: 10.1097/qai.0b013e3181dc6d2c
  15. 15. Laeyendecker O, Rothman RE, Henson C, Horne BJ, Ketlogetswe KS, et al. (2008) The effect of viral suppression on cross-sectional incidence testing in the Johns Hopkins Hospital Emergency Department. J Acquir Immune Defic Syndr 48: 211–215. doi: 10.1097/qai.0b013e3181743980
  16. 16. Laeyendecker O, Brookmeyer R, Mullis C, Donnell D, Lingappa J, et al. (2012) Specificity of four laboratory approaches for cross-sectional HIV incidence determination: Analysis of samples from adults with known non-recent HIV infection from five African countries. AIDS Res Hum Retroviruses 28: 1177–83. doi: 10.1089/aid.2011.0341
  17. 17. Young CL, Hu DJ, Byers R, Vanichseni S, Young NL, et al. (2003) Evaluation of a sensitive/less sensitive testing algorithm using the bioMerieux Vironostika-LS assay for detecting recent HIV-1 subtype B' or E infection in Thailand. AIDS Res Hum Retroviruses 19: 481–486. doi: 10.1089/088922203766774522
  18. 18. Parekh BS, Hanson DL, Hargrove J, Branson B, Green T, et al. (2011) Determination of Mean Recency Period for Estimation of HIV Type 1 Incidence with the BED-Capture EIA in Persons Infected with Diverse Subtypes. AIDS Res Hum Retroviruses 27: 265–273. doi: 10.1089/aid.2010.0159
  19. 19. Marinda ET, Hargrove J, Preiser W, Slabbert H, van Zyl G, et al. (2010) Significantly diminished long-term specificity of the BED capture enzyme immunoassay among patients with HIV-1 with very low CD4 counts and those on antiretroviral therapy. J Acquir Immune Defic Syndr 53: 496–499. doi: 10.1097/qai.0b013e3181b61938
  20. 20. Hayashida T, Gatanaga H, Tanuma J, Oka S (2008) Effects of low HIV type 1 load and antiretroviral treatment on IgG-capture BED-enzyme immunoassay. AIDS Res Hum Retroviruses 24: 495–498. doi: 10.1089/aid.2007.0150
  21. 21. Hallett TB, Ghys P, Barnighausen T, Yan P, Garnett GP (2009) Errors in ‘BED’-derived estimates of HIV incidence will vary by place, time and age. PLoS One 4: e5720. doi: 10.1371/journal.pone.0005720
  22. 22. Kulich M, Laeyendecker O, Eshleman SH, Komarek A, Omelka M, et al.. (2013) Evaluation of multi-assay algorithms for cross-sectional HIV incidence estimation. Submitted.
  23. 23. Hemelaar J, Gouws E, Ghys PD, Osmanov S (2011) Global trends in molecular epidemiology of HIV-1 during 2000–2007. AIDS 25: 679–689. doi: 10.1097/qad.0b013e328342ff93
  24. 24. Morrison CS, Chen PL, Nankya I, Rinaldi A, Van Der Pol B, et al. (2011) Hormonal contraceptive use and HIV disease progression among women in Uganda and Zimbabwe. J Acquir Immune Defic Syndr 57: 157–164. doi: 10.1097/qai.0b013e318214ba4a
  25. 25. Campbell MS, Mullins JI, Hughes JP, Celum C, Wong KG, et al. (2011) Viral linkage in HIV-1 seroconverters and their partners in an HIV-1 prevention clinical trial. PLoS One 6: e16986. doi: 10.1371/journal.pone.0016986
  26. 26. Arroyo MA, Sateren WB, Serwadda D, Gray RH, Wawer MJ, et al. (2006) Higher HIV-1 incidence and genetic complexity along main roads in Rakai District, Uganda. J Acquir Immune Defic Syndr 43: 440–445. doi: 10.1097/01.qai.0000243053.80945.f0
  27. 27. Conroy SA, Laeyendecker O, Redd AD, Collinson-Streng A, Kong X, et al. (2010) Changes in the distribution of HIV type 1 subtypes D and A in Rakai District, Uganda between 1994 and 2002. AIDS Res Hum Retroviruses 26: 1087–1091. doi: 10.1089/aid.2010.0054
  28. 28. Valley-Omar Z, Sibeko S, Anderson J, Goodier S, Werner L, et al. (2012) CAPRISA 004 tenofovir microbicide trial: no impact of tenofovir gel on the HIV transmission bottleneck. J Infect Dis 206: 35–40. doi: 10.1093/infdis/jis305
  29. 29. Masciotra S, Dobbs T, Candal D, Hanson D, Delaney K, et al.. (2010) Antibody avidity-based assay for identifying recent HIV-1 infections based on genetic systems TM 1/2 plus O EIA. 17th Conference on Retroviruses and Opportunistic Infections, abstract 937. San Francisco, CA; Feb 16–19.
  30. 30. Mullis CE, Munshaw S, Grabowski MK, Serwadda D, R B, et al.. (2012) Differential misclassification of HIV-1 cross-sectional incidence assays by subtype in Rakai, Uganda. 18th Conference on Retroviruses and Opportunistic Infections, abstract 541. Seattle, WA.
  31. 31. Laeyendecker O, Piwowar-Manning E, Fiamma A, Kulich M, Donnell D, et al. (Submitted 2013) Estimation of HIV incidence in a large, community-based, randomized clinical trial: NIMH Project Accept (HIV Prevention Trials Network 043).
  32. 32. Abdool Karim Q, Abdool Karim SS, Frohlich JA, Grobler AC, Baxter C, et al. (2010) Effectiveness and safety of tenofovir gel, an antiretroviral microbicide, for the prevention of HIV infection in women. Science 329: 1168–1174. doi: 10.1126/science.1193748
  33. 33. Celum C, Wald A, Hughes J, Sanchez J, Reid S, et al. (2008) Effect of aciclovir on HIV-1 acquisition in herpes simplex virus 2 seropositive women and men who have sex with men: a randomised, double-blind, placebo-controlled trial. Lancet 371: 2109–2119. doi: 10.1016/s0140-6736(08)60920-4
  34. 34. Celum C, Wald A, Lingappa JR, Magaret AS, Wang RS, et al. (2010) Acyclovir and transmission of HIV-1 from persons infected with HIV-1 and HSV-2. N Engl J Med 362: 427–439. doi: 10.1056/nejmoa0904849
  35. 35. Kumwenda NI, Hoover DR, Mofenson LM, Thigpen MC, Kafulafula G, et al. (2008) Extended antiretroviral prophylaxis to reduce breast-milk HIV-1 transmission. N Engl J Med 359: 119–129. doi: 10.1056/nejmoa0801941
  36. 36. Wawer MJ, Sewankambo NK, Serwadda D, Quinn TC, Paxton LA, et al. (1999) Control of sexually transmitted diseases for AIDS prevention in Uganda: a randomised community trial. Rakai Project Study Group. Lancet 353: 525–535. doi: 10.1016/s0140-6736(98)06439-3