We hypothesized that the percent adherence to antiretroviral therapy necessary to maintain HIV suppression would decrease with longer duration of viral suppression.
Eligible participants were identified from the REACH cohort of marginally housed HIV infected adults in San Francisco. Adherence to antiretroviral therapy was measured through pill counts obtained at unannounced visits by research staff to each participant's usual place of residence. Marginal structural models and targeted maximum likelihood estimation methodologies were used to determine the effect of adherence to antiretroviral therapy on the probability of virologic failure during early and late viral suppression.
A total of 221 subjects were studied (median age 44.1 years; median CD4+ T cell nadir 206 cells/mm3). Most subjects were taking the following types of antiretroviral regimens: non-nucleoside reverse transcriptase inhibitor based (37%), ritonavir boosted protease inhibitor based (28%), or unboosted protease inhibitor based (25%). Comparing the probability of failure just after achieving suppression vs. after 12 consecutive months of suppression, there was a statistically significant decrease in the probability of virologic failure for each range of adherence proportions we considered, as long as adherence was greater than 50%. The estimated risk difference, comparing the probability of virologic failure after 1 month vs. after 12 months of continuous viral suppression was 0.47 (95% CI 0.23–0.63) at 50–74% adherence, 0.29 (CI 0.03–0.50) at 75–89% adherence, and 0.36 (CI 0.23–0.48) at 90–100% adherence.
The risk of virologic failure for adherence greater than 50% declines with longer duration of continuous suppression. While high adherence is required to maximize the probability of durable viral suppression, the range of adherence capable of sustaining viral suppression is wider after prolonged periods of viral suppression.
Citation: Rosenblum M, Deeks SG, van der Laan M, Bangsberg DR (2009) The Risk of Virologic Failure Decreases with Duration of HIV Suppression, at Greater than 50% Adherence to Antiretroviral Therapy. PLoS ONE 4(9): e7196. doi:10.1371/journal.pone.0007196
Editor: Gary Maartens, University of Cape Town, South Africa
Received: May 26, 2009; Accepted: July 29, 2009; Published: September 29, 2009
This is an open-access article distributed under the terms of the Creative Commons Public Domain declaration which stipulates that, once placed in the public domain, this work may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose.
Funding: This study was funded by NIMH 54907 and was conducted at the UCSF Clinical and Translational Science Initiative Tenderloin Clinical Research Center, NIH U54 RR023566-01. Dr. Bangsberg received additional funding from NIAAA 015287. HIV RNA kits were donated by Roche. Michael Rosenblum was supported by a Ruth L. Kirschstein National Research Service Award (NRSA) under NIH/NIMH grant 5 T32 MH-19105-19. Mark van der Laan was supported by NIH grant R01 A1074345-01. http://www.nih.gov/ The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Medication adherence is the most important predictor of viral suppression among HIV infected patients receiving combination antiretroviral therapy –. The adherence threshold required to achieve durable viral suppression has declined with more potent regimens, such that the majority of patients in clinical practice are now able to maintain undetectable viral loads at adherence proportions as low as 70% –. Although the relationship between adherence and the short-term virologic response to therapy has been well described, the impact of successful treatment duration on the relationship between adherence and viral suppression remains unexamined.
Several independent observations suggest that the degree of drug pressure necessary to initially achieve viral suppression may be higher than that needed to maintain viral suppression. During effective antiretroviral therapy, plasma HIV RNA levels decline in a characteristic multi-phasic manner. The rapid first phase decay likely reflects the death of actively turning over, short-lived CD4+ T cells, while each subsequent phase likely reflects the death of longer lived cellular reservoirs . After a period of 2 to 5 years, most patients reach a new steady-state in which the long-lived reservoir (presumed to be resting CD4+ T cells) continues to produce a steady-state level of viremia , . Since the size of the reservoir containing replication competent virus declines over time , it is reasonable to postulate that the amount of virus that is able to initiate new rounds of replication also declines, as does the reservoir of pre-existing drug-resistant variants. Long-term HAART (highly active antiretroviral therapy) is also associated with decline in the number of activated and/or proliferating CD4+ T cells , . Since these cells are the primary target cells for virus production, it is likely that the ability of virus to escape drug pressure declines proportionally with the decline in these cells . These observations have been used as a rationale for a series of induction-maintenance clinical trials, in which patients are initially treated with more potent regimens and then later switched to a better tolerated, less potent regimen. Although many of the earlier trials failed , , more recent studies using longer induction periods and/or better maintenance regimens have provided some support for this approach , .
Based on these theoretical considerations, we hypothesized that the impact of adherence on the probability of virologic failure would differ depending on how long a subject had maintained virologic suppression. To test this hypothesis, we examined the effect of adherence on viral load after different durations of viral suppression. Subjects for this study were enrolled in a systematic community-based sample of HIV infected urban poor individuals living in San Francisco (the REACH cohort) . Adherence was measured using unannounced pill counts at the participant's usual place of residence, as previously described . This method has a close association with concurrent viral load , electronic pill cap adherence assessment , development of resistance , , and progression to AIDS . We used marginal structural models – and targeted maximum likelihood estimation  to adjust for potential confounders of adherence and viral suppression.
The University of California San Francisco and Committee on Human Subjects Research and the Partner's Human Research Committee approved all procedures.
Study design and subject recruitment
Participants enrolled in the REACH cohort were invited to participate in a substudy focused on intensive adherence monitoring, as previously described , , . Briefly, subjects had unannounced visits by research staff at their usual place of residence every three to six weeks, over a one year period. Percent adherence was determined from the number of remaining or unused antiretroviral pills and number of pills refilled between visits . Three hundred and fifty seven HIV-positive subjects were monitored for medication adherence. The earliest monitoring period started in March 1998 and the latest monitoring date was in October 2007.
Confounding variable assessment
Confounders of the effect of current adherence on virologic failure included the following: prior adherence, prior duration of HAART, prior exposure to mono/dual nucleoside therapy, recent CD4+ T cell count (lagged 2 months), CD4+ T cell nadir (lagged 2 months), demographics (sex, ethnicity, age), years of education, past and current antiretroviral treatment characteristics, crack cocaine and alcohol use, calendar time, and homelessness. Current antiretroviral treatment was classified as one of five potential regimens: ritonavir boosted protease inhibitor containing regimen, unboosted protease inhibitor containing regimen, non-nucleoside reverse transcriptase inhibitor containing regimen (NNRTI), combined non-nucleoside reverse transcriptase inhibitor-protease inhibitor containing regimen (PI/NNRTI), and nucleoside reverse transcriptase inhibitor (NRTI) only containing regimen.
Blood was drawn monthly for HIV RNA levels (viral loads) and quarterly for CD4+ T cell levels. HIV-1 viral load was performed using the HIV-1 Amplicor Monitor Version 1.0 ultra sensitive assay (Roche Molecular Systems, Alameda, CA).
For every month of observation, each REACH cohort participant was classified as experiencing either virologic suppression (defined as HIV RNA less than 50 copies/ml) or virologic failure (defined as HIV RNA at least 50 copies/ml). Each participant in the REACH cohort who was suppressed for at least two consecutive months during adherence monitoring contributed data to the analysis; there were 221 participants who met this criterion. For each such participant, his/her data was included starting at the first month of viral suppression during adherence monitoring and continuing until virologic failure occurred or the adherence monitoring period ended.
Duration of suppression at any month was defined as the number of prior consecutive months of viral suppression. In calculating the duration of viral suppression, we used data on viral loads prior to the initiation of adherence monitoring.
We used marginal structural models to estimate the effect of adherence during a given month on the probability of virologic failure at the end of that month. These effects were estimated within subpopulations defined by duration of prior viral suppression. For example, among participants who had maintained viral suppression for exactly 4 consecutive months, we estimated the effect of adherence during month 5 on the probability of virologic failure at the end of month 5. For our primary analysis, adherence was stratified into the following four categories: 0−49%, 50−74%, 75−89%, or 90−100% pills taken. The marginal structural model was fit using a targeted maximum likelihood estimator (TMLE) , in order to adjust for the potential confounders listed above. The TMLE, in this application, relies on 1) a multinomial logistic regression model for predicting medication adherence given potential confounders and number of consecutive months virally suppressed and 2) a logistic regression model for the probability of virologic failure given adherence, confounders, and number of consecutive months virally suppressed. The TMLE is robust to model misspecification, in that it gives asymptotically unbiased estimates whenever at least one of the models (1) or (2) is correct. It also has several advantages over alternative estimation techniques, such as inverse probability of treatment weighting , g-computation , , and doubly robust estimators –; these advantages have been previously described .
Based on the marginal structural model fit using targeted maximum likelihood estimation, we estimated the probabilities of virologic failure at each set stratum of adherence, for duration of continuous suppression ranging from 1 to 12 months. Confidence intervals for the difference between the probability of failure after 1 month of continuous suppression and the probability of failure after 12 months of continuous suppression, at each adherence stratum, were calculated using 10,000 iterations of the nonparametric bootstrap (BCa method) .
We tested for effect modification based on the estimated coefficients of the marginal structural models. More precisely, we tested whether the effect of the proportion of medication taken on the probability of virologic failure in the current month, is impacted by the number of consecutive months one has been virally suppressed. This test involved first computing the logarithm of the causal relative risk of virologic failure, comparing high adherence (90–100%) vs. low adherence (0–49%), conditional on number of consecutive months virally suppressed. This causal relative risk was based on the fit of the marginal structural model discussed above. Next, we used a linear regression model to test the null hypothesis that the causal relative risk does not depend on the number of consecutive months virally suppressed. Standard errors were computed using the nonparametric bootstrap as above.
Time-lagged confounder measurements were used to ensure that confounders occurred before, and thus could not be influenced by, medication adherence. We imputed missing confounder values by carrying the most recent observation forward. Because subjects could contribute data at multiple time points, bootstrap resampling was based on subject rather than data-point.
Of the 357 subjects in the REACH cohort who received unannounced pill counts, 221 met our criteria for inclusion in the analysis. The median age at start of adherence monitoring was 44.1 years (IQR 10.5). The median CD4+ T cell count was 390 cells/mm3 and the median CD4+ T cell nadir was 206 cells/mm3. Fifty-six of the 221 participants (25%) were receiving an unboosted protease inhibitor based regimen. The mean observed duration of continuous viral suppression was 6.7 months. Of the 1201 subject-months of observation, 95 (8%) were missing adherence measurements and 159 (13%) were missing viral load measurements. See Tables 1 and 2 for a more extensive list of participant characteristics.
Adherence over time
Unannounced pill count adherence ranged from 0 to 100%, with a median of 92%. In univariate analyses, higher adherence was associated with higher past adherence, longer duration of viral suppression, higher CD4+ nadir, and white/Caucasian ethnicity, while lower adherence was associated with use of crack, intravenous drug use, and Black/African American ethnicity. Comparing observed adherence proportions among subjects who maintained viral suppression for different durations of time, there was a gradual decrease in the proportion with 0–49% adherence (from 11% of subject-months at 1 month of suppression to 5% of subject-months at 12 months of continuous suppression) and 50–74% adherence (from 19% of subject-months at 1 month of suppression to 15% of subject-months at 12 months of continuous suppression). There was a corresponding gradual increase in the proportion with 75–89% adherence (from 19% of subject-months at 1 month of suppression to 25% of subject-months at 12 months of continuous suppression) and 90–100% adherence (from 50% of subject-months at 1 month of suppression to 55% of subject-months at 12 months of continuous suppression). None of these changes in the proportion of participants at a given adherence level was statistically significant at the 0.05 level.
Duration of viral suppression and incidence of virologic failure
Of the 221 participants who achieved viral suppression during adherence monitoring, 108 (49%) subsequently experienced virologic failure during adherence monitoring. Based on univariate regression analyses, participants who had longer duration of suppression also had higher CD4+ counts, higher CD4+ nadir, and higher cumulative average adherence proportions. Participants who had longer duration of suppression were also more likely to be on an NNRTI or PI/NNRTI regimen than an unboosted protease inhibitor regimen. Virologic failure (defined as a plasma HIV RNA greater than 50 copies/ml) was associated with low recent CD4+ T cell count, low past adherence, shorter duration of prior suppression, and intravenous drug use.
A logistic regression of the probability of virologic failure on adherence, confounders, and number of months since initial suppression was used in the targeted maximum likelihood analysis (see Table 3). Based on this logistic regression, a lower risk of virologic failure was associated with longer duration of continuous suppression, lower past viral load, and being on an NNRTI-based regimen.
Duration of viral suppression modifies the effect of adherence on virologic failure
We first estimated the causal effect of different adherence proportions on the probability of virologic failure, conditioned on duration of continuous viral suppression (Figure 1). We stratified subject-months by corresponding duration of suppression and then estimated the probability of virologic failure setting the adherence proportion to be in the ranges (0–49%, 50–74%, 75–89%, and 90–100%). Because adherence proportion cannot be randomly assigned as an intervention, we relied on a marginal structural model and targeted maximum likelihood estimation of the causal effects of adherence on viral suppression at each time point.
At one month of suppression, the estimated probabilities of virologic failure at all four adherence categories were relatively high: 0.36 for 0–49% adherence, 0.49 for 50–74% adherence, 0.34 for 75–89% adherence, and 0.37 for 90–100%. The widths of the confidence intervals for these estimates were large (ranging from 0.24 to 0.43), and there was a substantial region of mutual overlap among all four confidence intervals.
The point estimates for the probability of virologic failure decreased with longer duration of suppression (Figure 1). The probabilities of virologic failure after 12 months of viral suppression were 0.24, 0.02, 0.06, and 0.01 for adherence categories of 0–49%, 50–74%, 75–89%, and 90–100%, respectively. Comparing the probability of failure just after achieving suppression versus after 12 consecutive months of suppression, there was a statistically significant decrease in the probability of virologic failure for adherence set above 50%. The estimated risk difference, comparing the probability of virologic failure after 1 month vs. after 12 months of continuous viral suppression was 0.47 (95% CI 0.23–0.63) for the 50–74% adherence stratum, 0.29 (95% CI 0.03–0.50) for 75–90% adherence stratum, and 0.36 (95% CI 0.23–0.48) for the 90–100% adherence stratum. The estimated risk of failure decreased by only 0.13 (95% CI −0.74–0.39) for the 0–49% adherence stratum, which was not statistically significant at the 0.05 level.
We carried out a test for effect modification by number of consecutive months virally suppressed. We considered the causal relative risk, comparing the effect of different adherence proportions on the probability of virologic failure. This causal relative risk was calculated conditioning on the number of months of viral suppression, using a marginal structural model. The null hypothesis that this causal relative risk does not depend on the number of consecutive months virally suppressed was rejected (p-value 0.001). Thus, the data provide evidence for duration of suppression being an effect modifier of the adherence-suppression relationship.
Treatment adherence is widely accepted as the primary determinant of long-term virologic outcomes among antiretroviral-treated patients. These data suggest that for adherence proportions greater than 50%, the probability of virologic failure decreases with longer duration of viral suppression. For example, we estimated the risk of virologic failure for adherence between 75–89% (which is the most common adherence range in most chronic diseases) to be 0.31 after 1 month of suppression. In contrast, we estimated the risk of virologic failure for adherence between 75–89% to be 0.06 after 12 months of suppression. Similar trends were observed in those with adherence in the ranges 50 to 74%, and 90 to 100%.
Consistent with our finding that the impact of adherence on viral suppression is modified by history of successful treatment are a number recent induction-maintenance studies. For example, lopinavir-ritonavir monotherapy leads to higher rates of viral suppression once patients achieve suppression on standard regimens than when lopinvavir-ritonavir monotherapy is used as de novo , . Similarly, the use of a triple nucleoside reverse transcriptase inhibitor appears to be more effective after viral suppression is achieved than when the regimen is used as an initial regimen , .
One important limitation in our analysis is the potential role of selection bias in explaining the decrease in probability of virologic failure among those with longer duration of suppression. That is, those susceptible to virologic failure may have been selected out at earlier months, leaving a group with lower probability of failure at later months. Thus, we cannot exclude selection bias as a potential explanation for these findings.
There are several other limitations that deserve comment. While we did have extensive and systematic adherence data, we did not have as extensive virologic suppression data on all individuals, since some people entered the cohort already on treatment. These individuals may have had additional periods of viral suppression that were not included in this analysis. The estimation method we used relied on having included all confounders of adherence and virologic failure in our analyses, and on our marginal structural model and other models used being correctly specified. While we included many of the known predictors of adherence and virologic failure, unmeasured confounders may lead to bias in our estimates. Finally we studied patients on a wide range of antiretroviral therapy and some regimens we studied are no longer commonly used. We did not have the statistical power to limit analyses to just individuals on NNRTI or ritonavir boosted protease inhibitor based regimens.
While the adherence proportion required to sustain viral suppression may decline over time, the goal of near perfect adherence should remain unchanged. While both more potent therapy and sustained viral suppression may lessen the virologic consequences of missed doses, improving adherence will increase the probability of durable and sustained viral suppression , .
Conceived and designed the experiments: DRB. Analyzed the data: MR SGD MJvdL. Wrote the paper: MR SGD MJvdL DRB. Conceived of statistical analysis: MR.
- 1. Paterson DL, Swindells S, Mohr J, Brester M, Vergis EN, et al. (2000) Adherence to protease inhibitor therapy and outcomes in patients with HIV infection. Ann Intern Med 133: 21–30.
- 2. Bangsberg DR, Hecht FM, Charlebois ED, Zolop AR, Holodniy M, et al. (2000) Adherence to protease inhibitors, HIV-1 viral load, and development of drug resistance in an indigent population. AIDS 14: 357–366.
- 3. Arnsten JH, Demas PA, Farzadegan H, Grant RW, Gourevitch MN, et al. (2001) Antiretroviral therapy adherence and viral suppression in HIV-infected drug users: comparison of self-report and electronic monitoring. Clin Infect Dis 33: 1417–1423.
- 4. Bangsberg DR (2006) Less than 95% adherence to nonnucleoside reverse-transcriptase inhibitor therapy can lead to viral suppression. Clin Infect Dis 43: 939–941.
- 5. Shuter J (2008) Forgiveness of non-adherence to HIV-1 antiretroviral therapy. J Antimicrob Chemother 61: 769–773.
- 6. Maggiolo F, Ravasio L, Ripamonti D, Gregis G, Quinzan G, et al. (2005) Similar adherence rates favor different virologic outcomes for patients treated with nonnucleoside analogues or protease inhibitors. Clin Infect Dis 40: 158–163.
- 7. Nachega J, Hislop M, Dowdy D, Chaisson R, Regensberg L, et al. (2007) Adherence to nonnucleoside reverse transcriptase inhibitor-based HIV therapy and virologic outcomes. Ann Intern Med 146: 564–573.
- 8. Perelson AS, Essunger P, Cao Y, Vesanen M, Hurley A, et al. (1997) Decay characteristics of HIV-1-infected compartments during combination therapy. Nature 387: 188–191.
- 9. Palmer S, Maldarelli F, Wiegand A, Bernstein B, Hanna GJ, et al. (2008) Low-level viremia persists for at least 7 years in patients on suppressive antiretroviral therapy. Proc Natl Acad Sci U S A 105: 3879–3884.
- 10. Siliciano JD, Kajdas J, Finzi D, Quinn TC, Chadwick K, et al. (2003) Long-term follow-up studies confirm the stability of the latent reservoir for HIV-1 in resting CD4+ T cells. Nat Med 9: 727–728.
- 11. Strain MC, Gunthard HF, Havlir DV, Ignacio CC, Smith DM, et al. (2003) Heterogeneous clearance rates of long-lived lymphocytes infected with HIV: Intrinsic stability predicts lifelong persistence. Proc Natl Acad Sci U S A 100: 4819–4824.
- 12. Lempicki RA, Kovacs JA, Baseler MW, Adelsberger JW, Dewar RL, et al. (2000) Impact of HIV-1 infection and highly active antiretroviral therapy on the kinetics of CD4+ and CD8+ T cell turnover in HIV-infected patients. Proc Natl Acad Sci U S A 97: 13778–13783.
- 13. Deeks SG, Kitchen CM, Liu L, Guo H, Gascon R, et al. (2004) Immune activation set point during early HIV infection predicts subsequent CD4+ T-cell changes independent of viral load. Blood 104: 942–947.
- 14. Zhang ZQ, Wietgrefe SW, Li Q, Shore MD, Duan L, et al. (2004) Roles of substrate availability and infection of resting and activated CD4+ T cells in transmission and acute simian immunodeficiency virus infection. Proc Natl Acad Sci U S A 101: 5640–5645.
- 15. Havlir DV, Marschner IC, Hirsch MS, Collier AC, Tebas P, et al. (1998) Maintenance antiretroviral therapies in HIV infected patients with undetectable plasma HIV RNA after triple-drug therapy. AIDS Clinical Trials Group Study 343 Team. N Engl J Med 339: 1261–1268.
- 16. Reijers MH, Weverling GJ, Jurriaans S, Wit FW, Weigel HM, et al. (1998) Maintenance therapy after quadruple induction therapy in HIV-1 infected individuals: Amsterdam Duration of Antiretroviral Medication (ADAM) study. Lancet 352: 185–190.
- 17. Markowitz M, Hill-Zabala C, Lang J, DeJesus E, Liao Q, et al. (2005) Induction with abacavir/lamivudine/zidovudine plus efavirenz for 48 weeks followed by 48-week maintenance with abacavir/lamivudine/zidovudine alone in antiretroviral-naive HIV-1-infected patients. J Acquir Immune Defic Syndr 39: 257–264.
- 18. Mallolas J, Pich J, Penaranda M, Domingo P, Knobel H, et al. (2008) Induction therapy with trizivir plus efavirenz or lopinavir/ritonavir followed by trizivir alone in naive HIV-1-infected adults. Aids 22: 377–384.
- 19. Moss AR, Hahn JA, Perry S, Charlebois ED, Guzman D, et al. (2004) Adherence to highly active antiretroviral therapy in the homeless population in San Francisco: a prospective study. Clin Infect Dis 39: 1190–1198.
- 20. Bangsberg D, Hecht F, Charlebois E, Chesney M, Moss A (2001) Comparing Objectives Measures of Adherence to HIV Antiretroviral Therapy: Electronic Medication Monitors and Unannounced Pill Counts. AIDS and Behavior 5: 275–281.
- 21. Bangsberg DR, Acosta EP, Gupta R, Guzman D, Riley ED, et al. (2006) Adherence-resistance relationships for protease and non-nucleoside reverse transcriptase inhibitors explained by virological fitness. Aids 20: 223–231.
- 22. Bangsberg DR, Charlebois ED, Grant RM, Holodniy M, Deeks SG, et al. (2003) High levels of adherence do not prevent accumulation of HIV drug resistance mutations. Aids 17: 1925–1932.
- 23. Bangsberg D, Perry S, Charlebois E, Clark R, Roberston M, et al. (2001) Adherence to highly active antiretroviral therapy predicts progression to AIDS. AIDS 15: 1181–1183.
- 24. Petersen ML, Wang Y, van der Laan MJ, Bangsberg DR (2006) Assessing the effectiveness of antiretroviral adherence interventions. Using marginal structural models to replicate the findings of randomized controlled trials. J Acquir Immune Defic Syndr 43: Suppl 1S96–S103.
- 25. Petersen ML, Wang Y, van der Laan MJ, Guzman D, Riley E, et al. (2007) Pillbox organizers are associated with improved adherence to HIV antiretroviral therapy and viral suppression: a marginal structural model analysis. Clin Infect Dis 45: 908–915.
- 26. Robins J (1998) Marginal Structural Models. Proceedings of the American Statistical Association 1–10.
- 27. van der Laan MJ, Rubin D (2006) Targeted Maximum Likelihood Learning. International Journal of Biostatistics 2: Article 11.
- 28. Robertson M, Clark R, Charlebois E, Tulsky J, Long H, et al. (2004) HIV Seroprevalence and risk factors in a representative sample of homeless and marginally housed adults in San Francisco. Am J Public Health 94: 1207–1217.
- 29. Zolopa A, Hahn J, Gorter R, Miranda J, Wlodarczyk D, et al. (1994) HIV and tuberculosis infection in San Francisco's homeless adults: Prevalence and risk factors in a representative sample. JAMA 272: 455–461.
- 30. Robins J, Finkelstein D (2000) Correcting for non-compliance and dependent censoring in an aids clinical trial with inverse probability of censoring weighted (ipcw) log-rank tests. Biometrics 56: 779–788.
- 31. Robins J (1986) A new approach to causal inference in mortality studies with sustained exposure periods - application to control of the healthy worker survivor effect. (with errata). Mathematical Modelling 7: 1393–1512.
- 32. Robins JM (1987) A graphical approach to the identication and estimation of causal parameters in mortality studies with sustained exposure periods. Journal of Chronic Disease Supplement 2 40: 1390161.
- 33. Neugebauer R, van der Laan MJ (2002) Why prefer double robust estimates? illustration with causal point treatment studies. UC Berkeley Division of Biostatistics Working Paper Series.
- 34. Robins JM (2000) Robust estimation in sequentially ignorable missing data and causal inference models. Proceedings of the American Statistical Association Section on Bayesian Statistical Science 6–9:
- 35. Robins JM, Rotnitzky A (2001) Comment on the bickel and kwon article, “inference for semiparametric models: Some questions and an answer”. Statistica Sinica 11: 920–936.
- 36. van der Laan MJ, Robins JM (2002) Unified methods for censored longitudinal data and causality. New York: Springer.
- 37. Efron B, Tibshirani RJ (1993) An Introduction to the Bootstrap. New York: Chapman and Hall.
- 38. Cameron D, daSilva B, Arribas J, Myers R, Bellos N, et al. (2008) A 96-week comparison of lopinavir-ritonavir combination therapy followed by lopinavir-ritonavir monotherapy versus efavirenz combination therapy. J Infect Dis 198: 234–240.
- 39. Delfraissy J, Flandre P, Delaugerre C, Ghosn J, Horban A, et al. (2008) Lopinavir/ritonavir monotherapy or plus zidovudine and lamivudine in antiretroviral-naive HIV-infected patients. AIDS 22: 385–393.
- 40. Gulick RM, Ribaudo HJ, Shikuma CM, Lustgarten S, Squires KE, et al. (2004) Triple-nucleoside regimens versus efavirenz-containing regimens for the initial treatment of HIV-1 infection. N Engl J Med 350: 1850–1861.
- 41. Martin M, Del Cacho E, Codina C, Tuset M, De Lazzari E, et al. (2008) Relationship between Adherence Level, Type of the Antiretroviral Regimen, and Plasma HIV Type 1 RNA Viral Load: A Prospective Cohort Study. AIDS Research and Human Retroviruses 24: 1263–1268.