There is limited evidence on the optimal timing of antiretroviral therapy (ART) initiation in children 2–5 y of age. We conducted a causal modelling analysis using the International Epidemiologic Databases to Evaluate AIDS–Southern Africa (IeDEA-SA) collaborative dataset to determine the difference in mortality when starting ART in children aged 2–5 y immediately (irrespective of CD4 criteria), as recommended in the World Health Organization (WHO) 2013 guidelines, compared to deferring to lower CD4 thresholds, for example, the WHO 2010 recommended threshold of CD4 count <750 cells/mm3 or CD4 percentage (CD4%) <25%.
Methods and Findings
ART-naïve children enrolling in HIV care at IeDEA-SA sites who were between 24 and 59 mo of age at first visit and with ≥1 visit prior to ART initiation and ≥1 follow-up visit were included. We estimated mortality for ART initiation at different CD4 thresholds for up to 3 y using g-computation, adjusting for measured time-dependent confounding of CD4 percent, CD4 count, and weight-for-age z-score. Confidence intervals were constructed using bootstrapping.
The median (first; third quartile) age at first visit of 2,934 children (51% male) included in the analysis was 3.3 y (2.6; 4.1), with a median (first; third quartile) CD4 count of 592 cells/mm3 (356; 895) and median (first; third quartile) CD4% of 16% (10%; 23%). The estimated cumulative mortality after 3 y for ART initiation at different CD4 thresholds ranged from 3.4% (95% CI: 2.1–6.5) (no ART) to 2.1% (95% CI: 1.3%–3.5%) (ART irrespective of CD4 value). Estimated mortality was overall higher when initiating ART at lower CD4 values or not at all. There was no mortality difference between starting ART immediately, irrespective of CD4 value, and ART initiation at the WHO 2010 recommended threshold of CD4 count <750 cells/mm3 or CD4% <25%, with mortality estimates of 2.1% (95% CI: 1.3%–3.5%) and 2.2% (95% CI: 1.4%–3.5%) after 3 y, respectively. The analysis was limited by loss to follow-up and the unavailability of WHO staging data.
The results indicate no mortality difference for up to 3 y between ART initiation irrespective of CD4 value and ART initiation at a threshold of CD4 count <750 cells/mm3 or CD4% <25%, but there are overall higher point estimates for mortality when ART is initiated at lower CD4 values.
Please see later in the article for the Editors' Summary
Infection with HIV, the virus that causes AIDS, contributes substantially to the burden of disease in children. Worldwide, more than 3 million children younger than 15 years old (90% of whom live in sub-Saharan Africa) are HIV-positive, and every year around 330,000 more children are infected with HIV. Children usually acquire HIV from their mother during pregnancy, birth, or breastfeeding. The virus gradually destroys CD4 lymphocytes and other immune system cells, leaving infected children susceptible to other potentially life-threatening infections. HIV infection can be kept in check, with antiretroviral therapy (ART)—cocktails of drugs that have to be taken daily throughout life. ART is very effective in children but is expensive, and despite concerted international efforts over the past decade to provide universal access to ART, in 2011, less than a third of children who needed ART were receiving it.
Why Was This Study Done?
For children diagnosed as HIV-positive between the ages of two and five years, the 2010 World Health Organization (WHO) guidelines for the treatment of HIV infection recommended that ART be initiated when the CD4 count dropped below 750 cells/mm3 blood or when CD4 cells represented less than 25% of the total lymphocyte population (CD4 percent). Since June 2013, however, WHO has recommended that all HIV-positive children in this age group begin ART immediately, irrespective of their CD4 values. Earlier ART initiation might reduce mortality (death) and morbidity (illness), but it could also increase the risk of toxicity and of earlier development of drug resistance. In this causal modeling analysis, the researchers estimate the mortality associated with starting ART at different CD4 thresholds among children aged 2–5 years using observational data collected in cohort studies of ART undertaken in southern Africa. Specifically, they compared the estimated mortality associated with the WHO 2010 and WHO 2013 guidelines. Observational studies compare the outcomes of groups (cohorts) with different interventions (here, the timing of ART initiation). Data from such studies are affected by time-dependent confounding: CD4 count, for example, varies with time and is a predictor of both ART initiation and the probability of death. Causal modeling techniques take time-dependent confounding into account and enable the estimation of the causal effect of an intervention on an outcome from observational data.
What Did the Researchers Do and Find?
The researchers used g-computation (a type of causal modeling) adjusting for time-dependent confounding of CD4 percent, CD4 count, and weight-for-age z-score (a measure of whether a child is underweight for their age that provides a proxy indicator of the clinical stage of HIV infection) to estimate mortality for ART initiation at different CD4 thresholds in 2,934 ART-naïve, HIV-positive children aged 2–5 years old at their first visit to one of eight study sites in southern Africa. The average initial CD4 values of these children were a CD4 count of 592 cells/mm3 and a CD4 percent of 16%. The estimated cumulative mortality after three years was 3.4% in all children if ART was never started. If all children had started ART immediately after diagnosis irrespective of CD4 value or if the 2010 WHO-recommended threshold of a CD4 count below 750 cells/mm3 or a CD4 percent below 25% was followed, the estimated cumulative mortalities after three years were 2.1% and 2.2%, respectively (a statistically non-significant difference).
What Do These Findings Mean?
These findings suggest that, among southern African children aged 2–5 years at HIV diagnosis, there is no difference in mortality for up to three years between children in whom ART is initiated immediately and those in whom ART initiation is deferred until their CD4 value falls below a CD4 count of 750 cells/mm3 or a CD4 percent of 25%. Although causal modeling was used in this analysis, the accuracy of these results may be affected by residual confounding. For example, the researchers were unable to adjust for the clinical stage of HIV disease at HIV diagnosis and instead had to use weight-for-age z-scores as a proxy indicator of disease severity. Other limitations of the study include the large number of children lost to follow-up and a possible lack of generalizability—most of the study participants were from urban settings in South Africa. Importantly, however, these findings suggest that the recent change in the WHO guidelines for ART initiation in young children is unlikely to increase or reduce mortality, with the proviso that the long-term effects of earlier ART initiation such as toxicity and the development of resistance to ART need to be explored further.
Please access these websites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.1001555
- Information is available from the US National Institute of Allergy and Infectious Diseases on HIV infection and AIDS
- NAM/aidsmap provides basic information about HIV/AIDS and summaries of recent research findings on HIV care and treatment
- Information is available from Avert, an international AIDS charity, on many aspects of HIV/AIDS, including information on HIV and AIDS in Africa and on children and HIV/AIDS (in English and Spanish)
- The UNAIDS World AIDS Day Report 2012 provides up-to-date information about the AIDS epidemic and efforts to halt it; the 2013 Progress Report on the Global Plan provides information on progress towards eliminating new HIV infections among children by 2015
- The World Health Organization provides information about universal access to AIDS treatment (in several languages); its 2010 guidelines for ART in infants and children and its 2013 consolidated guidelines on the use of ART can be downloaded
- The researchers involved in this study are part of the International Epidemiologic Databases to Evaluate AIDSSouthern Africa collaboration, which develops and implements methodology to generate the large datasets needed to address high-priority research questions related to HIV/AIDS
- Personal stories about living with HIV/AIDS, including stories from young people infected with HIV, are available through Avert, through NAM/aidsmap, and through the charity website Healthtalkonline
Citation: Schomaker M, Egger M, Ndirangu J, Phiri S, Moultrie H, Technau K, et al. (2013) When to Start Antiretroviral Therapy in Children Aged 2–5 Years: A Collaborative Causal Modelling Analysis of Cohort Studies from Southern Africa. PLoS Med 10(11): e1001555. doi:10.1371/journal.pmed.1001555
Academic Editor: Steven G. Deeks, San Francisco General Hospital, United States of America
Received: June 8, 2013; Accepted: October 9, 2013; Published: November 19, 2013
Copyright: © 2013 Schomaker et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work was supported by the National Institute of Allergy and Infectious Diseases (NIAID) and the Eunice Kennedy Shriver National Institute for Child Health and Human Development (NICHD) [IeDEA Southern Africa, grant number U01 AI069924] and the World Health Organization. The opinions expressed herein are those of the authors and do not necessarily reflect the views of any of the funders. The National Institutes of Health, NIAID and NICHD, had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: LM, a WHO staff member, and MP and SE, WHO consultants, have been involved in the 2013 WHO Guidelines development process. They were involved in designing this study and writing the manuscript. MS and MAD received funding from WHO to support the data analysis. However, the opinions expressed in the manuscript are solely those of the authors and do not necessarily represent the official views or policies of WHO.
Abbreviations: ART, antiretroviral therapy; CD4%, CD4 percentage; CDC, US Centers for Disease Control and Prevention; HAZ, height-for-age z-score; IeDEA-SA, International Epidemiologic Databases to Evaluate AIDS–Southern Africa; LTFU, lost to follow-up; PREDICT, Pediatric Randomized to Early versus Deferred Initiation in Cambodia and Thailand; RCT, randomized controlled trial; WAZ, weight-for-age z-score; WHO, World Health Organization
HIV infection continues to contribute substantially to the burden of disease in children, with an estimated 330,000 new paediatric infections worldwide in 2011. Paediatric antiretroviral combination therapy (ART) is highly effective, but ART coverage for children who need it was below 30% in 2011 . Furthermore, there is still limited evidence from randomized controlled trials (RCTs) regarding the optimal timing of ART initiation in children aged 2–5 y ,. While earlier ART initiation may reduce mortality and morbidity, it could also increase the risk of toxicity and earlier development of drug resistance.
The Children with HIV Early Antiretroviral Therapy trial demonstrated a 76% (95% CI: 49%–89%) reduction in mortality associated with immediate compared to deferred ART in infants less than 3 mo of age; however, results of studies in older children are less clear –. The Pediatric Randomized to Early versus Deferred Initiation in Cambodia and Thailand (PREDICT) trial – enrolled children aged 1–12 y with initial CD4 percentage (CD4%) values of 15%–24% without US Centers for Disease Control and Prevention (CDC) category C disease. These children were randomly assigned to start ART immediately or defer therapy until their CD4% dropped below 15% or a CDC category C event occurred. There was no difference between the two arms in mortality, AIDS-free survival, number of new category B and C events, neurodevelopmental outcomes, rate of hospital admission, or number of drug-related adverse events ,. However, less than a third of the children included in the trial (96 in total) were in the 2–5-y age group. This group may experience more rapid disease progression than older children in the absence of ART, and so the benefits of immediate ART might be greater –. In addition, because of the low event rate for the primary outcomes (five CDC category C events, one death), the PREDICT study authors concluded that the study was underpowered to detect a difference between the two randomization groups .
In the absence of evidence from RCTs, observational data from cohort studies may be used to address this question. Such analyses are not straightforward, however, because of time-dependent confounding. The CD4 count, for example, is a time-dependent confounder that varies with time, predicts the initiation of ART, but also predicts the probability of dying. Causal modelling techniques such as g-computation enable estimation of the causal effect of an intervention on an outcome, taking into account such time-dependent confounding –. We used data from the International Epidemiologic Databases to Evaluate AIDS–Southern Africa (IeDEA-SA) collaboration to estimate mortality for up to 3 y of follow-up associated with starting ART at a range of different CD4 values in children aged 2–5 y. Our primary study aim was to determine whether there is any difference in estimated mortality associated with starting ART at the World Health Organization (WHO) 2010 recommended threshold of CD4 count <750 cells/mm3 or CD4% <25%  in comparison to starting ART immediately as recommended in the WHO 2013 guidelines.
The IeDEA-SA collaboration study was approved by the University of Cape Town and University of Bern human research ethics committees. The requirement for informed consent was waived, as only anonymized data that were already collected as part of routine monitoring contributed to the collaborative dataset.
Cohort Description and Selection of Children
This study is based on HIV treatment cohorts in southern Africa that took part in the IeDEA-SA collaboration (http://www.iedea-sa.org/). The collaboration has been described in detail elsewhere ,. In brief, data were collected at each site as part of routine monitoring and were transferred to data centres at the University of Cape Town, South Africa, or the University of Bern, Switzerland, at regular intervals. All programmes obtained ethical approval from the relevant local institutions before contributing anonymized patient data to IeDEA-SA. The present analysis is restricted to eight cohorts that routinely captured both pre-ART data and post-ART data of HIV-infected children: Lighthouse Trust Clinic in Malawi; Newlands Clinic in Zimbabwe; and McCord Hospital, Harriet Shezi Children's Clinic, Gugulethu Community Health Centre, Khayelitsha ART Programme, Hlabisa HIV Treatment and Care Programme, and Rahima Moosa Mother and Child Hospital in South Africa. All clinics except McCord Hospital were public health sector programmes with research support providing primary care to urban populations, but rural populations were included in the Hlabisa HIV Treatment and Care Programme . At Harriet Shezi Children's Clinic and Rahima Moosa Mother and Child Hospital, secondary and tertiary care are available in addition to primary care.
In South Africa, the recommended first-line regimen was two nucleoside reverse transcriptase inhibitors plus either lopinavir/ritonavir (children <3 y of age or <10 kg) or efavirenz (children >3 y of age and >10 kg). In Malawi and Zimbabwe the recommended first-line regimen for all children was two nucleoside reverse transcriptase inhibitors plus nevirapine. The nucleoside reverse transcriptase inhibitors used were mainly stavudine or zidovudine plus lamivudine.
All children who were at least 24 mo of age at enrolment into HIV care, enrolled between 1 January 2000 and 30 June 2012, and were not older than 60 mo were included. Children who were not ART-naïve at their time of entry into the HIV care programmes or who had no follow-up data were excluded from the analysis. The detailed process of selection of children eligible for this analysis is provided in Figure 1.
Variable Definitions and Loss to Follow-Up
Baseline characteristics included CD4 count, CD4%, weight-for-age z-score (WAZ), and height-for-age z-score (HAZ), as well as the child's sex and age at first visit. All z-scores were calculated using WHO standards . Follow-up data consisted of CD4 count, CD4%, and WAZ, evaluated 1 mo after the first clinic visit and in 3-mo intervals starting 3 mo after the first visit, for up to 36 mo in care. If two or more observations per interval were available, the one closest to the middle of the interval was used. The date of ART initiation (if applicable) was captured as well. If there were no data for a child in a particular time interval, data were defined to be missing. The outcome was mortality and was ascertained by the individual cohorts. Children were defined as being lost to follow-up (LTFU) and were censored if they had no contact with their health care facility for 9 mo before database closure (visits are scheduled up to 6 mo apart and clinic visits being delayed for up to 3 mo are possible).
Multiple Imputation of Missing Baseline and Follow-Up Data
To account for missing follow-up data for CD4 count, CD4%, and WAZ, a child's last observation was carried forward for a maximum of 9 mo. For the remaining missing follow-up data, as well as missing baseline data for CD4 count, CD4%, WAZ, and HAZ, longitudinal multiple imputation by means of the expectation maximisation bootstrap algorithm  was used to create ten imputed datasets. Imputations were utilised only after 9 mo without any visit data, as from there on it is plausible that CD4/WAZ measurements that determine treatment assignment were taken (and are thus needed to adjust for time-dependent confounding) but not electronically recorded, probably because of administrative or clerical errors. The missing at random assumption is therefore likely to hold. The imputation model included all measured baseline and follow-up variables, mortality, follow-up time, a variable indicating which observations were carried forward, and the region (Zimbabwe, Malawi, South Africa urban, South Africa rural). The longitudinal structure of the data was explicitly considered in the expectation maximisation bootstrap algorithm, nonlinear time trends were allowed, and lag and lead variables of CD4 count, CD4%, and HAZ were added to the imputation model. All results of the multivariate analysis are based on the imputed datasets and were combined using Rubin's rules .
Baseline characteristics were described using proportions or medians (reported with first; third quartile) overall and for different ART initiation thresholds. Follow-up data were described at 12 mo and 24 mo after the first clinic visit.
We then estimated mortality over 3 y of follow-up for ART initiation at different CD4 thresholds using causal modelling (g-computation) , adjusting for time-dependent confounding of CD4%, CD4 count, and WAZ, with 95% bootstrap confidence intervals based on 200 bootstrap samples. WAZ served as a proxy variable for WHO stage, which was not routinely available in our data: many clinical events that define advanced clinical WHO stage, and thus disease progression and severity, relate to a child's WAZ—for example, malnutrition, tuberculosis, persistent diarrhoea, severe wasting, recurrent bacterial pneumonia, recurrent bacterial infection, candidiasis, and chronic lung disease –. WAZ was therefore expected to approximately adjust for the time-dependent confounding induced by stage-defining severe clinical events. The analysis was also adjusted with respect to baseline CD4 count, CD4%, WAZ, HAZ, sex, age, and region.
The primary comparison we considered was (i) giving ART immediately irrespective of CD4 criteria or (ii) deferring ART until a child's CD4 cell count falls below 750 cells/mm3 or CD4% falls below 25%; further strategies we considered were (iii) deferring ART until the threshold of CD4 count <500 cells/mm3 or CD4% <20% is reached, (iv) deferring ART until the threshold of CD4 count <250 cells/mm3 or CD4% <15% is reached, and (v) no ART is given.
The details of our g-computation algorithm are contained in Box 1. Briefly, the observed associations in the data were used to simulate and evaluate counterfactual datasets that would have been observed under each of the above initiation rules. This mimics an RCT that compares the effect of the different initiation rules. These datasets were standardised with respect to the time-dependent confounders (CD4 count, CD4%, WAZ) that influence both mortality and ART initiation in children . To implement this, we first modelled the association of these time-dependent confounders at all follow-up visits (at time t = 1, 3, 6,… mo) with disease progression history (follow-up data at time t−1), baseline characteristics (CD4 count, CD4%, WAZ), demographics (age, sex, region), and ART (at time t−1, included under an intention to treat assumption) using additive linear models. We used additive logistic regression to model the association of death with the same variables at each follow-up interval. For all these models nonlinear relationships and interactions with baseline characteristics were explored and included if they improved the generalised cross-validation score. In the second step we used the observed baseline data and predictions of the fitted models under a specific treatment initiation rule to simulate the time-dependent confounders and the outcome “forward in time”. These two steps were repeated for all the above specified ART initiation rules, and mortality in the simulated datasets was evaluated thereafter. All these steps were repeated for all bootstrap samples in each imputed dataset and combined appropriately.
Box 1. Details of the g-Computation Algorithm
Step 1: Model fitting. For the 3-y period of follow-up we considered the follow-up times t = 0, 1, 3, 6, 9,…, 36 mo after the first clinic visit.
- Time-dependent confounders. We used additive linear models to estimate the association of the time-dependent confounders (CD4 count, CD4%, WAZ) with
- disease progression history (CD4 count, CD4%, and WAZ at time t−1)
- baseline characteristics (CD4 count, CD4%, WAZ, HAZ)
- demographics (age, sex, region)
- the intervention (ART at time t−1)
- Outcome. We used a logistic additive model to estimate the association of the outcome (death) with the time-dependent confounders at time t, disease progression history, baseline characteristics, demographics, and intervention for t = 1, 3, 6,…, 36 mo. This corresponds to fitting one model for 12 points in time.
- Model building. The models were built as follows:
- CD4 count was square root transformed in the models where CD4 count was the outcome variable.
- Generalized-cross-validation-based model selection ensured a good bias–variance tradeoff of the models.
- Nonlinear interactions of continuous variables (both time-dependent and baseline) with baseline characteristics (represented in categories) were explored to allow for flexible disease progression depending on how sick children present at their first visit. They were included if it improved the generalized cross-validation score of the model.
- The effect of continuous variables on the outcomes was modelled in a nonlinear manner via p-splines. The complexity of the splines was determined by means of generalized cross-validation.
Step 2: Simulation. We simulated data for the children for one specific intervention forward in time.
At the first visit, t = 0, the data corresponded to the observed data of all children.
- Simulation of the covariates. For t = 1, 3, 6,…, 36 mo, CD4 count at time t was predicted by a random draw from a normal distribution where mean and standard error were obtained from the prediction of the additive linear model fitted in Step 1(a). The additive linear model corresponds to the model fitted at time t, and the intervention was set as specified by the treatment rule (e.g., ART = 0 if the intervention of interest was “never give ART”). Prediction for CD4% and WAZ were obtained in the same manner.
The simulated values of CD4 count, CD4%, and WAZ at time t−1 were used when predicting the CD4 count, CD4%, and WAZ at time t.
- Simulation of the outcome. The hypothetical outcome was simulated based on a draw from a Bernoulli distribution with the probability obtained from the logistic additive model fitted in Step 1(b). Again, as in (a), we artificially set our dynamic intervention to the value corresponding to the strategy of interest. If the simulated outcome at time t was equal to 1 (death), then there was no more follow-up at time t+1.
Step 3: Estimation of mortality. We estimated the relative mortality for t = 1, 3, 6,…, 36 mo in evaluating our simulated dataset created in Step 2.
Step 4: Intervention repetitions. We repeated Steps 2 and 3 for all interventions of interest. These interventions are the following:
- Give a child ART immediately, irrespective of his/her CD4 count and CD4%
- Give a child ART when his/her absolute CD4 count falls below 750 cells/mm3 or his/her CD4% falls below 25%
- Give a child ART when his/her absolute CD4 count falls below 500 cells/mm3 or his/her CD4% falls below 20%
- Give a child ART when his/her absolute CD4 count falls below 250 cells/mm3 or his/her CD4% falls below 15%
- Never give a child ART
Step 5: Bootstrap repetitions. We repeated steps 1 to 4 for 200 bootstrap samples. The bootstrapping was utilized to obtain confidence intervals. The bounds of the 95% confidence intervals were set at the 2.5th and 97.5th percentiles of the distribution of the bootstrap estimates.
Step 6: Multiple imputation. Steps 1 to 5 were repeated for ten imputed sets of data. The multiple imputation was utilized with the Amelia II package in R. The imputation model included all measured variables and explicitly accounted for the longitudinal structure of the data.
Diagnostics. Imputation diagnostics (comparing imputed and observed densities, overimputation, convergence of expectation maximization chains, and time-series plots ) were evaluated to ensure the convergence of the algorithm and the appropriateness of the imputations.
We further conducted a sensitivity analysis, linking data from three of the South African cohorts (Hlabisa HIV Treatment and Care Programme, Khayelitsha ART Programme, McCord Hospital) with the national vital registry to obtain the vital status of children LTFU. We then imputed remaining events and survival times for children LTFU and unascertained  and estimated mortality using the g-computation algorithm described above.
We also calculated the percentage of children who presented with CD4 count above the threshold of 750 cells/mm3 or CD4% above 25% and estimated the probability of falling below one of these thresholds during the 3 y of follow-up using the Kaplan-Meier estimator. This analysis was restricted to pre-ART data, and we censored both children who died and children who were LTFU. The same probability was also estimated by means of cumulative incidence functions treating ART initiation, death, and LTFU as competing events. All analyses were conducted using R 2.15 .
From all 15,852 children in the database, there were 3,654 children in the eligible age range from cohorts with regular pre-ART data capturing. Of these, 110 children who were not ART-naïve and 610 ART-naïve children with no follow-up data were excluded. Most of the 2,934 children included in the study were from South Africa (85%; Figure 1). Median follow-up duration was 928 d (first; third quartile: 314; 1,082), during which 2,227 (75.9%) children started ART. In total there were 83 recorded deaths; 30 of these (36.1%) occurred in the first 3 mo, 40 (48.2%) in the first 6 mo. Overall, 662 (22.6%) children were defined to be LTFU.
The median (first; third quartile) age at enrolment into HIV care was 3.3 y (2.6; 4.1), and 1,501 (51.2%) were male (Table 1). The median CD4 count at enrolment was 592 cells/mm3 (356; 895), median CD4% was 16% (10; 23), and median WAZ and HAZ were −1.4 (−2.3; −0.5) and −2.6 (−3.5; −1.6), respectively. Among those who initiated ART, 1,716 (77%) started with a CD4 count below 750 cells/mm3 or CD4% below 25%, and 100 (4.5%) with a CD4 count above 750 cells/mm3 or CD4% above 25%. For 411 (18.5%) children, CD4 values at ART initiation were not available. The 707 (24.1%) children who never initiated ART had higher median CD4 count, CD4%, HAZ, and WAZ at their first clinic visit than children who initiated ART late (for example, at CD4 count below 750 cells/mm3 or CD4% below 25%) during follow-up. Among children who initiated ART, those who had lower CD4 counts or CD4% at ART initiation also had lower CD4 counts (and CD4%, WAZ, HAZ) at their first clinic visit (Table 1). Children who started ART had large CD4 and WAZ gains, up to >100%, compared to their first visit when presenting with a CD4 count below 250 cells/mm3 or CD4% below 15% at ART start (Table 2). The characteristics of children at enrolment were overall similar for the different cohorts, though there was some variation with respect to the median CD4 count (between 528 and 734 cells/mm3) and median WFA (between −0.9 and −1.8) (Table 3).
The results of the g-computation analysis provided higher mortality point estimates for initiating ART at lower CD4 values or not initiating ART at all (Table 4). This pattern became evident from 2 y after the first visit onwards. There was no mortality difference between initiating ART immediately, irrespective of CD4 values, or starting ART according to the WHO 2010 threshold of CD4 count <750 cells/mm3 or CD4% <25%: estimates of cumulative mortality after 3 y of follow-up were 2.1% (95% CI: 1.3%–3.5%) and 2.2% (95% CI: 1.4%–3.5%), respectively. Figure 2 illustrates these estimates in detail, highlighting the almost identical estimated cumulative mortality (and 95% confidence intervals) for both ART initiation rules during the entire follow-up period.
Estimated cumulative mortality (including 95% bootstrap CI, dashed lines) over 3 y if ART was given irrespective of CD4 count and CD4% (blue line, “always ART”) and if ART was given if the CD4 count was below 750 cells/mm3 or the CD4% was below 25% (red line, “750,25%”).
The sensitivity analysis included 851 patients, 245 of whom were LTFU. We were able to link 32 (13.1%) patients LTFU to vital registration data for the Hlabisa HIV Treatment and Care Programme, Khayelitsha ART Programme, and McCord Hospital cohorts, and found three to be dead (one per cohort). After 3 y of follow-up the percentage of children who died was estimated as 6% (95% CI: 4.7%–7.4%) for children who received ART immediately irrespective of their CD4 count, 5.9% (95% CI: 4.6%–7.4%) for children starting ART at CD4 count <750 cells/mm3 or CD4% <25%, and 7.8% (95% CI: 5.1%–10.4%) for children receiving no ART at all.
When considering the pre-ART data of the 322 children (11% of all children) who presented with a CD4 count above 750 cells/mm3 and CD4% above 25%, an estimated 26.0% (95% CI: 20.4%–32.7%) fell below this threshold after 1 y, 54.5% (47.2%–62.2%) after 2 y, and 72.2% (64.3%–79.6%) after 3 y (Figure 3A). The estimated probabilities of falling below the threshold were slightly higher when ART initiation, death, and LTFU were treated as competing events (Figure 3B).
(A) Time to fall below the CD4 threshold of a CD4 count of 750 cells/mm3 or a CD4% of 25% estimated via the Kaplan-Meier estimator. The figure is based on 322 children presenting with a CD4 count of 750 cells/mm3 or above and a CD4% of 25% or above. Only pre-ART follow-up is considered. 95% confidence intervals are visualised via the grey shaded area. (B) Time to fall below the CD4 threshold of a CD4 count of 750 cells/mm3 or a CD4% of 25%. The figure is based on 322 children presenting with a CD4 count of 750 cells/mm3 or above and a CD4% of 25% or above. ART initiation, death, and LTFU were treated as competing risks. The probability of falling below the threshold was estimated via the cumulative incidence of falling below the threshold before any other event occurred divided by one minus the probability that any other event occurred before the threshold was reached. 95% confidence intervals were obtained via bootstrapping and are visualised via the grey shaded area.
Statement of Principal Findings
Our results indicate no mortality difference for up to 3 y between immediate ART initiation irrespective of CD4 values and deferred ART initiation using a CD4 threshold of CD4 count <750 cells/mm3 or CD4% <25%, though point estimates for mortality were higher when ART was initiated at lower CD4 values (or not at all).
Strengths of the Study
To our knowledge, this is the first attempt to estimate the effect on mortality of starting ART at different CD4 values in children aged 2–5 y. Both RCTs and previous causal modelling studies of observational data included a wider age range of children, with relatively small numbers of children in the 2–5-y age group ,,,. Although post hoc subgroup analyses of the RCT data restricted to this age range have been conducted, results were inconclusive . Moreover, the observational studies had few children without either clinically or immunologically severe disease and did not compare different CD4 thresholds but rather concentrated on the effect of ART itself ,.
Our analysis included a large dataset of nearly 3,000 children with relatively long follow-up from three resource-limited countries with high HIV burden. Because of the good availability of CD4 and WAZ data, which is unusual in this setting, as well as the use of g-computation, we were able to adjust for measured time-dependent confounding of CD4 count, CD4%, and WAZ. In addition, we accounted for missing data using multiple imputation.
The use of a causal modelling technique was essential to accurately estimate mortality associated with starting ART at different CD4 values in the presence of time-dependent confounding by indication, whereby the illest children have the greatest risk of mortality and are also most likely to be started on ART. There was clear evidence for time-dependent confounding: a naïve analysis of the data simply adjusting for demographic and disease severity characteristics at enrolment showed that ART initiation was associated with higher mortality than no ART initiation (hazard ratio = 1.80; 95% CI: 1.03–3.15). Other causal modelling techniques such as inverse probability of treatment weighting, g-estimation, or targeted maximum likelihood estimation could also have been used to derive results that adjust for time-dependent confounding, though g-computation is probably the most intuitive when dealing with dynamic treatment comparisons .
Although we used causal modelling, this analysis is based on observational data, and the results may be affected by residual confounding. For example, WHO clinical stage data were not routinely available throughout follow-up, and so we may not have adequately adjusted for the effect of clinical disease severity. We addressed this possibility by adjusting for WAZ, which is a surrogate of disease severity and should at least partly capture staging information . Some stage-defining events such as HIV encephalopathy or HIV-associated nephropathy may, however, not be reflected by WAZ.
When interpreting estimates of mortality associated with starting ART at high CD4 values, it should be noted that relatively few children initiated ART before their CD4 count or CD4% declined. Since country and WHO guidelines at the time did not recommend ART initiation at high CD4 values unless children had WHO clinical stage 3 or 4 disease, it is plausible that these children may have initiated therapy on the basis of advanced clinical disease. The median WAZ in children starting ART with CD4 >750 cells/mm3 or CD4% >25% was −1.42 (Table 1), indicating that a proportion of these children had clinically severe disease. Our analysis may therefore underestimate mortality reductions associated with starting ART at high CD4 values in a child without symptomatic disease. Nevertheless, our mortality estimates associated with immediate ART initiation or starting ART at a high CD4 value are consistent with the probability of experiencing AIDS or death found in both the immediate and deferred ART arms of the PREDICT trial . It would be useful to repeat this analysis using data from other settings, for example, in high-income settings where more children start ART earlier in the course of their disease.
Further limitations of our study include not examining outcomes beyond 3 y of follow-up. This limitation is especially relevant with respect to evaluating toxicity risk and the effect of development of resistance in a context of limited second-line options. In addition, children included in our study were mostly from urban settings with facilities for electronic data collection in southern Africa, especially from South Africa. Generalisability to rural areas and sub-Saharan African countries with less well-resourced health care systems may be limited. In addition, country-specific differences regarding drug regimens, public health programmes, and breastfeeding  and its implications for the timing of HIV infection may contribute to both mortality figures and the success of different treatment strategies.
Our analysis was specifically targeted at examining the WHO 2010 guidelines for ART initiation for children aged 2–5 y and how mortality would be affected by the WHO 2013 guideline recommendations. There are, however, other interesting comparisons that remain unexplored: for example, we did not include children between 1 and 2 y of age. There are currently no data that show mortality benefits of early ART in this age group ; a comparison with the age groups in this analysis would therefore be useful.
Mortality Estimates in Context
The absolute mortality estimates for all ART initiation strategies in our study are relatively low ,. This finding may partly be explained by the analysis being restricted to children aged at least 2 y at their first visit to an HIV care and treatment facility. Our data therefore comprise a cohort of survivors who have not died and have not been referred for HIV care or ART prior to reaching 2 y of age, and so may have less rapidly progressing disease ,,. However, it is precisely in this group of children that the optimal timing of ART initiation needs to be determined, as WHO 2010 guidelines already recommend immediate ART in children presenting at less than 2 y of age . Moreover, our mortality estimates are comparable to the results of the PREDICT trial, which reported a 2.1% and 1.3% risk of death/AIDS events after 144 wk in the early and deferred ART arms, respectively .
We may still have underestimated mortality. Children with no visits after enrolment were excluded from the analysis, as at least one follow-up visit is needed in order for children to have the possibility to initiate ART and thus allow the implementation of our methodology. The excluded children had higher CD4 counts (median [first; third quartile]: 694 cells/mm3 [382; 1,073]) and CD4% values (median [first; third quartile]: 21% [13%; 28%]) but lower WAZ values (median [first; third quartile]: −1.9 [−0.7; −3.3]), and their mortality may be different from those included in the analysis.
In addition, among the 2,934 children included in the analysis, those LTFU were censored at their last visit, and mortality may also be higher among those LTFU compared to those remaining in care ,,. To address this potential problem we conducted a sensitivity analysis, linking data from three of the South African cohorts with the national vital registry and imputing survival times and events of children LTFU and unascertained with multiple imputation as described previously . This analysis confirmed that mortality might be higher than reported, though the relative differences between the strategies in the sensitivity analysis after 3 y of follow-up remained similar to those in our initial results ignoring LTFU that yielded much lower mortality estimates. It is worth mentioning that our estimate of mortality for children receiving no ART (7.8% [95% CI: 5.1%–10.4%]) is consistent with estimates of mortality from previous studies among children in this age group in the absence of ART . However, the linkage information we had was sparse, the sample size was small, and data were available only from South Africa, so mortality estimates derived after linking data with the death registry should be interpreted with caution.
Implications and Considerations for Clinicians and Policymakers
The WHO guidelines changed in 2013 and now recommend ART initiation in all children less than 5 y of age irrespective of clinical or immunological disease severity. Our results suggest that this change in ART initiation criteria would neither reduce nor increase mortality. There may, however, be other considerations regarding the optimal timing of ART. For example, while the PREDICT trial showed that the immune response to ART was equivalent in both groups, children in the deferred group spent more time at a lower CD4 count, which may result in more chronic inflammation and a higher risk of clinical events. This consideration is especially important for the older children in our age group, as their thymic responses are not as good as in children under 2 y . However, there was no difference in the number of CDC category B or C events between the two arms in the PREDICT trial ,. The long-term effects of earlier ART initiation on immune function, growth, neurological development, toxicity, and development of resistance remain to be explored. In addition, maintaining long-term adherence can be particularly challenging in young children because of the need for a caregiver to administer medication and possible caregiver changes, lack of child-friendly formulations, and the need for frequent dose changes in growing children.
Programmatically it may be simpler to have a single treatment initiation approach for all children <5 y of age that does not require regular pre-ART follow-up and CD4 measurement, especially in settings where access to CD4 measurement is limited . Coordination of recommended regimen choices with age thresholds for starting ART would also ease implementation. In our study only about 11% of children aged 2–5 y had CD4 count >750 cells/mm3 and CD4% >25% at presentation, and we estimated 72.2% (see Figure 3A) to have progressed below these values within 3 y, though the proportion might be even larger during early follow-up, given that children who were censored because they were LTFU or started ART may have had faster disease progression than others. Deferring treatment until CD4 count drops below 750 cells/mm3 or CD4% drops below 25% may not delay the onset of therapy very much, and may risk a child becoming LTFU and/or not having his/her repeat CD4 measured, thus not starting on treatment promptly after dropping below the initiation threshold. However, it would be important to ensure that immediate ART initiation in children aged 2–5 y does not happen at the expense of prioritising early infant diagnosis and rapid initiation, which has been shown to reduce mortality but may be more challenging to implement. This consideration is especially important in resource-limited settings, where an increasing number of patients often overburdens health care facilities, and clinics are therefore not able to trace children who miss their infant diagnostic testing and results or CD4 testing appointments ,.
It has also been suggested that early ART initiation may reduce the high loss to follow-up frequently observed in pre-ART care . However, loss to follow-up has not consistently been shown to be lower in children commenced on ART compared to those not on ART . In addition, there may be selection bias, as children more likely to remain in care are more likely to get started on therapy.
Our results indicate that in children aged 2–5 y in southern Africa, there is no difference in mortality between starting ART immediately and deferring until CD4% drops below 25% or CD4 count drops below 750 cells/mm3. The overall clinical and programmatic implications of earlier ART initiation, such as effects on morbidity-related outcomes, retention in care, cost-effectiveness, adherence, and drug resistance, remain to be explored.
The authors are grateful to all patients and staff at the HIV care programmes included in this analysis and to the staff at the data centres. We are also grateful to Leigh Johnson and Francesca Little for their valuable feedback concerning details of the statistical analysis.
IeDEA-SA Steering Group: James Ndirangu, Africa Centre for Health and Population Studies, University of Kwazulu-Natal, Somkhele, South Africa; Christopher Hoffmann, Aurum Institute for Health Research, Johannesburg, South Africa; Benjamin Chi, Centre for Infectious Disease Research in Zambia, Lusaka, Zambia; Denise Naniche, Centro de Investigação em Saúde de Manhiça, Manhiça, Mozambique; Robin Wood, Desmond Tutu HIV Centre (Gugulethu and Masiphumelele clinics), Cape Town, South Africa; Diana Dickinson, Independent Surgery, Gaborone, Botswana; Kathryn Stinson, Khayelitsha ART Programme and Médecins Sans Frontières, Cape Town, South Africa; Geoffrey Fatti, Kheth'Impilo Programme, South Africa; Sam Phiri, Lighthouse Trust Clinic, Lilongwe, Malawi; Janet Giddy, McCord Hospital, Durban, South Africa; Maureen Wellington, Newlands Clinic, Harare, Zimbabwe; Kennedy Malisita, Queen Elizabeth Hospital, Blantyre, Malawi; Brian Eley, Red Cross War Memorial Children's Hospital and School of Child and Adolescent Health, University of Cape Town, Cape Town, South Africa; Jara Llenas, SolidarMed SMART Programme, Pemba Region, Mozambique; Christiane Fritz, SolidarMed SMART Programme, Masvingo, Zimbabwe; Matthew Fox and Mhairi Maskew, Themba Lethu Clinic, Johannesburg, South Africa; Hans Prozesky, Tygerberg Academic Hospital, Stellenbosch, South Africa; Karl Technau, Empilweni Clinic, Rahima Moosa Mother and Child Hospital, Johannesburg, South Africa; Harry Moultrie, Harriet Shezi Children's Clinic, Chris Hani Baragwanath Hospital, Soweto, South Africa.
Conceived and designed the experiments: MS MAD ME TG OK TG LM MP SE. Performed the experiments: JN SP HM KT VC JG CC RW CBM BE HR. Analyzed the data: MS MAD. Contributed reagents/materials/analysis tools: MS TG. Wrote the first draft of the manuscript: MS MAD. Contributed to the writing of the manuscript: MS ME JN SP HM KT VC JG CC RW TG CBM HR BE LM MP SE OK MAD. ICMJE criteria for authorship read and met: MS ME JN SP HM KT VC JG CC RW TG CBM HR BE LM MP SE OK MAD. Agree with manuscript results and conclusions: MS ME JN SP HM KT VC JG CC RW TG CBM HR BE LM MP SE OK MAD. Enrolled patients: JN SP HM KT VC JG CC RW CBM HR BE.
- 1. Joint United Nations Programme on HIV/AIDS (2012) Report on the global AIDS epidemic. Geneva: Joint United Nations Programme on HIV/AIDS.
- 2. Leroy V, Dabis F (2012) When should therapy begin for children infected with HIV? Lancet Infect Dis 12: 900–902. doi: 10.1016/s1473-3099(12)70266-9
- 3. Prendergast AJ, Penazzato M, Cotton M, Musoke P, Mulenga V, et al. (2012) Treatment of young children with HIV infection: using evidence to inform policymakers. PLoS Med 9 (7) e1001273 doi:10.1371/journal.pmed.1001273.
- 4. Violari A, Cotton MF, Gibb DM, Babiker AG, Steyn J, et al. (2008) Early antiretroviral therapy and mortality among HIV-infected infants. N Engl J Med 359: 2233–2244. doi: 10.1056/nejmoa0800971
- 5. Penazzato M, Prendergast A, Tierney J, Cotton M, Gibb D (2012) Effectiveness of antiretroviral therapy in HIV-infected children under 2 years of age. Cochrane Database Syst Rev 2012: CD004772. doi: 10.1002/14651858.cd004772.pub3
- 6. Ananworanich J, Kosalaraksa P, Siangphoe U, Engchanil C, Pancharoen C, et al. (2008) A feasibility study of immediate versus deferred antiretroviral therapy in children with HIV infection. AIDS Res Ther 5: 24. doi: 10.1186/1742-6405-5-24
- 7. Puthanakit T, Ananworanich J, Vonthanak S, Kosalaraksa P, Hansudewechakul R, et al. (2013) Cognitive function and neurodevelopmental outcomes in HIV-infected children older than 1 year of age randomized to early versus deferred antiretroviral therapy: the PREDICT neurodevelopmental study. Pediatr Infect Dis J 32: 501–508. doi: 10.1097/inf.0b013e31827fb19d
- 8. Puthanakit T, Bunupuradah T (2010) Early versus deferred antiretroviral therapy in children in low-income and middle-income countries. Curr Opin HIV/AIDS 5: 12–17. doi: 10.1097/coh.0b013e3283339b27
- 9. Puthanakit T, Saphonn V, Ananworanich J, Kosalaraksa P, Hansudewechakul R, et al. (2012) Early versus deferred antiretroviral therapy for children older than 1 year infected with HIV (PREDICT): a multicentre, randomised, open-label trial. Lancet Infect Dis 12: 933–941. doi: 10.1016/s1473-3099(12)70242-6
- 10. Cross Continents Collaboration for Kids (3Cs4kids) Analysis and Writing Committee (2008) Markers for predicting mortality in untreated HIV-infected children in resource-limited settings: a meta-analysis. AIDS 22: 97–105. doi: 10.1097/01.aids.0000302262.51286.a5
- 11. Dunn D (2003) HIV Paediatric Prognostic Markers Collaborative Study Group (2003) Short-term risk of disease progression in HIV-1-infected children receiving no antiretroviral therapy or zidovudine monotherapy: a meta-analysis. Lancet 362: 1605–1611. doi: 10.1016/s0140-6736(03)14793-9
- 12. Dunn D, Woodburn P, Duong T, Peto J, Phillips A, et al. (2008) Current CD4 cell count and the short-term risk of AIDS and death before the availability of effective antiretroviral therapy in HIV-infected children and adults. J Infect Dis 197: 398–404. doi: 10.1086/524686
- 13. Daniel RM, Cousens SN, De Stavola BL, Kenward MG, Sterne JA (2013) Methods for dealing with time-dependent confounding. Stat Med 32: 1584–1618. doi: 10.1002/sim.5686
- 14. Robins JM, Hernan MA, Brumback B (2000) Marginal structural models and causal inference in epidemiology. Epidemiology 11: 550–560. doi: 10.1097/00001648-200009000-00011
- 15. Gsponer T, Petersen M, Egger M, Phiri S, Maathuis MH, et al. (2012) The causal effect of switching to second-line ART in programmes without access to routine viral load monitoring. AIDS 26: 57–65. doi: 10.1097/qad.0b013e32834e1b5f
- 16. World Health Organization (2010) Antiretroviral therapy for HIV infection in infants and children: towards universal access. Recommendations for a public health approach: 2010 revision. Geneva: World Health Organization.
- 17. Egger M, Ekouevi DK, Williams C, Lyamuya RE, Mukumbi H, et al. (2012) Cohort profile: the international epidemiological databases to evaluate AIDS (IeDEA) in sub-Saharan Africa. Int J Epidemiol 41: 1256–1264. doi: 10.1093/ije/dyr080
- 18. Fenner L, Brinkhof MWG, Keiser O, Weigel R, Cornell M, et al. (2010) Early mortality and loss to follow-up in HIV-infected children starting antiretroviral therapy in southern Africa. J Acquir Immune Defic Syndr 54: 524–532. doi: 10.1097/qai.0b013e3181e0c4cf
- 19. World Health Organization (2006) The WHO child growth standards. Geneva: World Health Organization.
- 20. Honaker J, King G (2010) What to do about missing values in time-series cross-section data. Am J Pol Sci 54: 561–581. doi: 10.1111/j.1540-5907.2010.00447.x
- 21. Rubin DB (1996) Multiple imputation after 18+ years. J Am Stat Assoc 91: 473–489. doi: 10.1080/01621459.1996.10476908
- 22. Schmidt WP, Boisson S, Genser B, Barreto ML, Baisley K, et al. (2010) Weight-for-age z-score as a proxy marker for diarrhoea in epidemiological studies. J Epidemiol Community Health 64: 1074–1079. doi: 10.1136/jech.2009.099721
- 23. World Health Organization (2007) WHO case definitions of HIV for surveillance and revised clinical staging and immunological classification of HIV-related disease in adults and children. Geneva: World Health Organization.
- 24. Curtis AJ, Marshall CS, Spelman T, Greig J, Elliot JH, et al. (2012) Incidence of WHO stage 3 and 4 conditions following initiation of anti-retroviral therapy in resource limited settings. PLoS ONE 7 (12) e52019 doi:10.1371/journal.pone.0052019.
- 25. van der Wal WM, Prins M, Lumbreras B, Geskus RB (2009) A simple G-computation algorithm to quantify the causal effect of a secondary illness on the progression of a chronic disease. Stat Med 28: 2325–2337. doi: 10.1002/sim.3629
- 26. Schomaker M, Gsponer T, Estill J, Fox M, Boulle A (2013) Non-ignorable loss to follow-up: correcting mortality estimates based on additional outcome ascertainment. Stat Med E-pub ahead of print. doi:10.1002/sim.5912.
- 27. R Development Core Team (2010) R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.
- 28. Edmonds A, Yotebieng M, Lusiama J, Matumona Y, Kitetele F, et al. (2011) The effect of highly active antiretroviral therapy on the survival of HIV-infected children in a resource-deprived setting: a cohort study. PLoS Med 8 (6) e1001044 doi:10.1371/journal.pmed.1001044.
- 29. Edmonds A, Yotebieng M, Lusiama J, Matumona Y, Kitetele F, et al. (2012) Quantification of CD4 responses to combined antiretroviral therapy over 5 years among HIV-infected children in Kinshasa, Democratic Republic of Congo. J Acquir Immune Defic Syndr 61: 90–98. doi: 10.1097/qai.0b013e31825bd9b7
- 30. Siegfried N, Davies MA, Penazzato M, Muhe LM, Egger M (2013) Optimal time for initiating antiretroviral therapy (ART) in HIV-infected, treatment-naive children aged 2 to 5 years old. Cochrane Database Syst Rev E-pub ahead of print. doi:10.1002/14651858.CD010309.pub2.
- 31. Marston M, Becquet R, Zaba B, Moulton LH, Gray G, et al. (2011) Net survival of perinatally and postnatally HIV-infected children: a pooled analysis of individual data from sub-Saharan Africa. Int J Epidemiol 40: 385–396. doi: 10.1093/ije/dyq255
- 32. Dunn DT, Gibb DM, Duong T, Babiker AG, Aboulker JP, et al. (2006) Predictive value of absolute CD4 cell count for disease progression in untreated HIV-1-infected children. AIDS 20: 1289–1294. doi: 10.1097/01.aids.0000232237.20792.68
- 33. Braitstein P, Songok J, Vreeman RC, Wools-Kaloustian KK, Koskei P, et al. (2011) “Wamepotea” (they have become lost): outcomes of HIV-positive and HIV-exposed children lost to follow-up from a large HIV treatment program in western Kenya. J Acquir Immune Defic Syndr 57: e40–e46. doi: 10.1097/qai.0b013e3182167f0d
- 34. McGuire M, Munyenyembe T, Szumilin E, Heinzelmann A, Le Paih M, et al. (2010) Vital status of pre-ART and ART patients defaulting from care in rural Malawi. Trop Med Int Health 15 (Suppl 1) 55–62. doi: 10.1111/j.1365-3156.2010.02504.x
- 35. Lewis J, Walker AS, Castro H, De Rossi A, Gibb DM, et al. (2012) Age and CD4 count at initiation of antiretroviral therapy in HIV-infected children: effects on long-term T-cell reconstitution. J Infect Dis 205: 548–556. doi: 10.1093/infdis/jir787
- 36. Leroy V, Malateste K, Rabie H, Lumbiganon P, Ayaya S, et al. (2013) Outcomes of antiretroviral therapy in children in Asia and Africa: a comparative analysis of the IeDEA pediatric multiregional collaboration. J Acquir Immune Defic Syndr 62: 208–219. doi: 10.1097/qai.0b013e31827b70bf
- 37. Luque-Fernandez MA, Van Cutsem G, Goemaere E, Hilderbrand K, Schomaker M, et al. (2013) Effectiveness of patient adherence groups as a model of care for stable patients on antiretroviral therapy in Khayelitsha, Cape Town, South Africa. PLoS ONE 8 (2) e56088 doi:10.1371/journal.pone.0056088.
- 38. Mugglin C, Wandeler G, Estill J, Egger M, Bender N, et al. (2013) Retention in care of HIV-infected children from HIV test to start of antiretroviral therapy: systematic review. PLoS ONE 8 (2) e56446 doi:10.1371/journal.pone.0056446.
- 39. Anaky MF, Duvignac J, Wemin L, Kouakoussui A, Karcher S, et al. (2010) Scaling up antiretroviral therapy for HIV-infected children in Cote d'Ivoire: determinants of survival and loss to programme. Bull World Health Organ 88: 490–499. doi: 10.2471/blt.09.068015
- 40. Braitstein P, Katshcke A, Shen CY, Sang E, Nyandiko W, et al. (2010) Retention of HIV-infected and HIV-exposed children in a comprehensive HIV clinical care programme in Western Kenya. Trop Med Int Health 15: 833–841. doi: 10.1111/j.1365-3156.2010.02539.x
- 41. Honaker J, King G, Blackwell M (2011) Amelia II: a program for missing data. J Stat Softw 45: 1–47.