http://orcid.org/0000-0002-2726-9929
Figures
Abstract
Several health interventions have been put in place to improve health outcomes of people living with HIV/AIDS (PLHIV) in Ghana. We evaluated the impact of Antiretroviral Therapy (ART) on all-cause mortality in Ghana using matching procedures. This was a retrospective cohort study of 12,881 HIV/AIDS patients initiated on ART at 40 sentinel sites and 199 treatment centers between 2013 and 2016 countrywide. Patients were included if they had date of ART initiation and if they had mortality outcome recorded. Mahalanobis distant metric matching within propensity score caliper and other matching procedures were used to evaluate the effectiveness of ART in reducing the risk of all-cause mortality among PLHIV in Ghana. We performed sensitivity analysis using different matching procedures including Kernel weighting adjustment and Mahalanobis distance metric matching with nearest neighbour to ascertain the robustness of our results in the presence of unmeasured covariates. The proportion of patients on ART was 60.3% (95% CI: 59.5–61.1). The total number of mortalities reported was only 734 representing 4.6% (95% CI: 4.2–4.9) of the studied population. The risk of all-cause mortality has reduced by 11.6 percentage point among HIV/AIDS patients who were on ART compared to those who were not on ART (95% CI: 9.6–13.4). ART was associated with a decreased risk of all-cause mortality. Effort being made by Government and non-Governmental organizations in support of ART treatment in Ghana should continue unabated to help reduce mortality rate and improve health outcomes among HIV/AIDS. To reduce bias to the barest minimum between treatment and intervention group when evaluating the effectiveness of health interventions, it is recommended to use matching procedures especially when the study design is not a randomized control trial.
Citation: Dwomoh D, Tambaa C, Ayisi Addo S, Wiah E, Abdulai M, Bosomprah S (2018) Effect of antiretroviral therapy on all-cause mortality among people living with HIV/AIDS in Ghana using Mahalanobis distant metric matching within propensity score caliper analysis: A retrospective cohort study. PLoS ONE 13(9): e0203461. https://doi.org/10.1371/journal.pone.0203461
Editor: Richard John Lessells, University of KwaZulu-Natal, SOUTH AFRICA
Received: January 24, 2018; Accepted: August 21, 2018; Published: September 7, 2018
Copyright: © 2018 Dwomoh et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data are available from the database of National Aids Control Programme in Ghana upon request and undergoing thorough Ethics Committee review for researchers who meet the criteria for access to this confidential data. Please contact this address for access to the data: info@nacp.org.gh.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Human Immunodeficiency Virus (HIV) is a life threatening chronic viral infection, which compromises the body’s immune system [1]. The virus renders infected persons susceptible to other infections usually referred to as Opportunistic Infections (OI). People Living with HIV (PLHIV) are therefore susceptible to other diseases that uninfected persons may usually not have. It is estimated that over 36 million people, worldwide, are living with HIV of which most are in Sub-Saharan Africa as at the year 2015 [1].
HIV has, since its diagnosis in Ghana in 1986, recorded 3.8% (250,232) of the 6.5 million cases in Western and Central Africa with adult prevalence rate of 1.47% as at December 2015 [2], [3]. The National AIDS/STI Control Programme (NACP), Ghana Health Service 2015 HIV Sentinel Survey report indicated that HIV type 1 is the most prevalent in the country. The infection is reported to be more prevalent in urban areas, border towns, among female commercial sex workers, men having sex with men and among people within 45–49 years age group [2], [3]. HIV & AIDS related mortality is persistently among the leading causes of mortalities in Ghana [3],[4],[5]. Causes of mortalities could be attributed to late diagnosis and initiation of treatment, poor access to healthcare, TB/HIV co-infection and poor environmental factors which increase the risk of opportunistic infections in resource poor countries[6],[7].
Antiretroviral Therapy (ART) was introduced in Ghana in 2003 and assessing its impact on mortality is an important consideration. Several studies have assessed the impact of ART on mortality but these studies have relied primarily on traditional regression models (logistic, Poisson and Cox proportional hazard models). These traditional models have been shown to lead to biased results when the covariate distributions in the groups are very different[8–10]. Selection and regression models have been shown to perform poorly in situations where there is insufficient overlap among covariates, but their standard diagnostics do not involve checking this overlap[11–13]. However, matching procedures have straightforward diagnostics by which their performance can be assessed (Stuart, 2010). Matching procedures mirror randomized experiment as closely as possible using techniques that make PLHIV who are on ART and those who are not on ART as similar as possible on covariate distributions. This then reduces bias due to the observed covariates.
Motivated by nationally representative HIV and AIDS data, we illustrated how to use matching procedures and sensitivity analysis in evaluating causal effect of ART. Specifically, we evaluated the effectiveness of ART on all-cause mortality for 2013–2016 cohort using propensity score matching and Mahalanobis distance metric matching with propensity score caliper. The latter have been proven to yield matches that are relatively well matched on the propensity score and particularly well matched on Mahalanobis covariates especially when the continuous covariates used in generating the Mahalanobis distance are well matched [14].
Methods
Ethical statement
The study was approved by the Institutional Review Board of Noguchi Memorial Institute for Medical Research (NMIMR) of the University of Ghana, Accra. The National AIDS/STI Control Programme (NACP) gave the authorisation to use the data. The personal identification code which include name of the HIV patient and the health facility where the subject obtained ARV therapy were excluded from the data before data were handed over to the authors. That is, the data were de-identified and therefore cannot be linked to any patients.
Study design and data management
This was a retrospective cohort study of HIV patients enrolled in 40 sentinel sites and 199 treatment centres from 2013 to 2016 in the NACP national database. NACP is a Ghana Health Service agency under the ministry of health that captures, processes and stores the country’s data on HIV. NACP collates data on services provided to HIV patients through 40 sentinel sites and 199 treatment centers countrywide. In Ghana, patients are scheduled monthly for services at allocated treatment centers across the country. Each treatment center is equipped with data capturing tools including patient folders, computers and an offline database where registration records and updates on treatment from visits are entered daily by trained data managers or by nurses and supervised by a zonal data manager. Data is stored on a server at the national level and updated quarterly with databases from the treatment centers. NACP uses a Microsoft access database and application program to collect and process data at all levels (treatment centers, regional and national).
The entered data is validated by a regional and national team and copy taken by the team to the national database on quarterly basis. The national data repository comprises data on all HIV patients captured and reported by all treatment centers. The data collected at community or treatment centers are validated and sent by electronic mail in a pass worded database form to the data manager at the national level where they are merged and stored in the server. The data for this study was extracted from the national database into excel for preliminary cleaning, error entry and assessment of missing values. A total of 16,311 HIV/AIDS patients data were extracted from the database within the study period. Patients were included if they had date of ART initiation and if they had mortality outcome recorded as died or alive. A total of 12,881 patients were eligible for analysis in this study. The excel data was then imported into Stata 14 (StataCorp, College Station, Texas, USA) for further cleaning and statistical analysis. Table 1 shows the variables used in the study.
Statistical analysis
The primary outcome of interest was all-cause mortality defined as all of the death that occurred in the sample regardless of the cause. We compared baseline sociodemographic characteristics and biomarkers between patients who are on ART and patients who are not on ART using Pearson’s chi-square test of independence and Cochran Armitage trend test where appropriate. Modified Poisson and Cox-proportional hazard models with robust standard errors were used to estimate effect of ART (defined as the percentage reduction in the number of mortalities as a result of ART calculated as 100 (1 − R), where R is the hazard ratio estimated by Cox regression or incidence rate ratio estimated by Poisson regression models (Cheung et al. 2010). Date of treatment initiation was considered as when the patient became at risk of death and was therefore set as the origin in calculating the survival time. Patients were considered as censored at the date they were last seen without the outcome (i.e. lost to follow up) or at the end of data cut-off point without having the event of death. We elected one year data cut-off point for this study. The modified Poisson and Cox-proportional hazard models were used for purposes of comparison of estimates from traditional methods with those from the matching procedures.
We estimated average treatment effect among PLHIV (ATET) using Mahalanobis matching within propensity score callipers proposed by Rubin and Thomas (2000). The propensity score for ith patients was then defined as the probability of receiving the ART treatment given the observed covariates based on the probit regression model. Thus, the probability to receive ART given the observed covariates was given by ei(Xi) = P(ARTi|Xi) where ei(Xi) is the estimated propensity score, Xi represent individual’s age in years, baseline CD4 count (cells/mm3), educational level, sex, marital status, clinical stage of the disease at diagnosis and time on treatment.
The Mahalanobis matching within propensity score callipers proposed by Rubin and Thomas [15] defines the distance between patients i and j as:
where c is the caliper, Z is the age and baseline CD4 count before treatment, Σ is the variance covariance matrix of Z.
We used numerical and graphical diagnosis to evaluate the common support of the distribution of propensity score between patients on ART and those who are not ART. We compared the multidimensional histograms and Kernel density plots of the covariates in the matched ART and non-ART groups. We assumed that the standardized differences greater than 10% in absolute value indicate serious imbalance in the covariate of interest between the two groups[16]. The standardized difference d is given as follows for continuous and binary indicator variables:
where S is the standard deviation of the continuous covariate, PT is the proportion of patients on ART, PC is the proportion of patients who are not on ART.
Sensitivity analysis of matching procedures
We also performed sensitivity analysis of the ignorability assumption under propensity score matching. This assumption states that there may be other unobserved factors that influence ART treatment enrollment, all-cause mortality or both. We calculated the Rosenbaum bounds for average ART effects on the treated in the presence of unobserved heterogeneity (hidden bias) between patients who are on ART and those who are not on ART[9]. We also assessed whether the effectiveness of ART were robust to the type of matching procedure used. Since different matching technique could produce slightly different results (impact estimates), Kernel weighting adjustment and Mahalanobis distance metric matching with nearest neighbour were performed to confirm the robustness of the ART estimate. We further compared our impact estimate of ART with results that could have been obtained from various regression adjustments including Cox-proportional hazard model and the modified Poisson regression with robust standard error. The proportional hazard assumption required for the Cox-proportional hazard model was tested using the Schoenfeld residuals. We further tested for over-dispersion of our outcome variable under the Poisson regression model. All analyses were performed using Stata 15 (StataCorp, College Station, Texas, USA).
Results
Description of the study participants
A total of 12,881 patients were included in the analysis. Of these, 74.2% were females and 25.8% were males. The median age was 37 years (IQR = 30, 45) (Table 2) with the youngest person aged 1 year and oldest aged 92 years. More than half (56.7%) of the patients were married with only 925 (5.8%) having attained tertiary education. Almost all the patients were HIV type 1 (96.9%) with 71% being TB positive. The median CD4 count at baseline was 245 (IQR = 90, 445) cells per cubic millimeter. The standard WHO classification of stage of the infection at diagnosis is as follows: stage I (33.3%), stage II (17.7%), stage III (44.0%) and stage IV (5.1%) (Table 2). We have a total of 385,247.5 persons-month at risk and the longest follow up was 48.7 months. The median time at risk of mortality in month is 23.6 months (IQR = 10.7, 36.5). The median duration of treatment is 18.6 months. With the exception of HIV type (p = 0.648) and TB/HIV coinfection status (p = 0.641), patients on ART were significantly different from those who were not ART in terms of age (p<0.001), sex (p = 0.002), educational attainment (p<0.001), marital status (p<0.001), CD4 count at baseline (p<0.001) and clinical stage of the disease (p<0.001) (Table 2).
Effect of ART on all-cause mortality using modified Poisson and Cox regression models
The proportional hazard assumption test based on Schoenfeld residuals showed that there was no violation of this assumption (χ2 = 10.8,p = 0.544). Besides, the likelihood ratio test showed that there was no statistically significant difference between the conditional variance and the conditional mean (p = 0.899) which authenticate the used of Poisson regression model. A total of 548 (4.3%) out of the 12,881 patients died during the study period (Table 3). 13 (0.2%) out of 7,771 on ART died compared to 535 (10.5%) out of 5110 not on ART died (Table 3). In multivariable analyses, the hazard ratio estimated using modified Poisson regression model was 40.0 (95% CI: 17.7–90.2) compared to 52.3 (95% CI: 21.5–126.9) as estimated by Cox Proportional Hazard model (Table 3). All analyses adjusted for duration on ART, age, sex, CD4 count at diagnosis, stage of HIV at diagnosis, educational attainment and marital status.
Matching procedures
The average probability for patients to receive ART was 63.1% (95% CI: 62.9–63.3). Using the propensity-score with nearest-neighbour matching it was possible to generate a control group (non-ART) which was similar enough to the treatment group (patients on ART) to be used in estimating the average ART treatment effect on patients who actually received treatment. Fig 1 shows that there is an overlap of the propensity scores of the patients who are on ART and those who are not on ART which clearly shows that the assumption of common support holds in this study.
The diagnostic graph assessment of the covariate balance between ART and non-ART patients showed that standardized percentage bias among covariates between the two groups reduced drastically after matching (Fig 2).
Covariates distributions were found to be similar in both groups after matching. For instance, bias in CD4 count before matching between patients on ART and those not on ART was 19.8% but reduced to 5.8% after matching. Similar bias reduction were observed for other covariates. The overall mean bias before matching was 6.90% and this reduced 1.9% after matching as shown in Table 4.
Causal effect of ART on all-cause mortality
The different matching procedures used in this study showed that patients on ART have reduced risk of mortality compared to those who are not on ART (Table 5). PSM with 1:1 nearest neighbour matching showed that the risk of mortality if every patient is on ART is 10.9 (95% CI: 8.8–13.1; p<0.0001) less compared to if no patient is on ART (Table 5). Using Mahalanobis with 1:1 nearest neighbour, risk of mortality if every patient is on ART is 11.6 (95% CI: 9.6–13.4; p<0.0001) less compared to if no patient is on ART (Table 5). Similar results were obtained with 2:1 nearest neighbour and Kernel weighting adjustment (Table 5).
Sensitivity analysis to the ignorability assumption under PSM based on Rosenbaum (1983) bounds
Rosenbaum’s method of sensitivity analysis relies on the sensitivity parameter gamma (Γ) which measures the degree of departure from random assignment of ART. This assumption states that given the observed covariates, there are no unobserved differences between patients on ART and those not on ART (control group).
Two patients with the same observed characteristics may differ in the odds of receiving the ART by at most a factor of Γ. In a randomized experiment, randomization of the ART ensures that Γ = 1. In an observational study, if Γ = 1.5 and two patients are identical on matched covariates then one person might be 1.5 times as likely as the other to receive the ART because they differ in terms of an unobserved covariate[17]. Under the assumption of no hidden bias (Γ = 1), the Hodges-Lehmann test-statistic gives a similar result, indicating a significant ART effect on all-cause mortality (p<0.001). The median difference in mortality if there is no hidden bias is 11.0% lower for ART patients.
Looking at the bounds under the assumption that we have over-estimated the ART effect (upper bound Hodges-Lehmann point estimate) reveals that already at relatively small levels of Γ (1.1–2.5), the result becomes significant at alpha = 5% indicating no hidden bias. For instance for Γ = 2.5, mortality among patients receiving ART might be as low as (9.6%-12.4%) compared to those who are not on ART (Table 6).
Discussion
This study was designed to evaluate the effectiveness of antiretroviral therapy on all-cause mortality among people living with HIV/AIDS in Ghana. The three years cohort study involved 12,881 HIV patients on ART (7,771) and not on ART (5,110), of which 4.3% (548) died within the period of study. Comparing these two groups, we found that those who were on Antiretroviral Therapy (ART) were of reduced risk of mortality. For example, our findings showed that the risk of mortality if every patient is on ART is 11.6 (95% CI: 9.6–13.4; p<0.0001) less compared to if no patient is on ART.
Our findings are consistent with what has been found across the globe. Lesko and colleagues [18] using national surveillance data in the USA, found that patients on ART had a reduced risk of mortality compared to those not on ART (5-year mortality risk difference = -17.7% (95%CI: -27.0%, -8.4%). Similarly, in a collaborative analyses of prospective cohort studies in South Africa, Europe and North America, Boulle and Colleagues [19]] also found that mortality was much higher in South Africa compared to Europe and North America but this reduced and reversed with longer duration on ART in South Africa [20–24].
We demonstrated the effectiveness of ART using various statistical procedures that compared the traditional methods of estimating effect of intervention and methods that minimises the effect of bias when evaluating the effectiveness of an intervention. The effect of ART on all-cause mortality estimated via the traditional procedures (Poisson and Cox-proportional hazard models) is 390% and 5130% respectively. A more robust measure of the effectiveness of ART was obtained after removing bias in the covariates. Patients on ART had 10.0–11.0 percentage points reduced risk of mortality compared to what was obtained using the traditional methods. This, suggests that the traditional regression models over-estimated the effectiveness of ART in reducing risk of mortality especially when the difference in the means of the propensity scores in the two groups being compared is large. As indicated by Cochran and Rubin [25], and Rosenbaum and Rubin [9], the three basic conditions that must generally be met for regression analyses to be trustworthy, in the case of approximately normally distributed covariates include: “the distributions of the covariates in both groups are nearly symmetric, the distributions of the covariates in both groups have nearly the same variances, and the sample sizes are approximately the same, the ratio of the variances of the propensity score in the two groups must be close to one (e.g., 1/2 or 2 are far too extreme), the ratio of the variances of the residuals of the covariates after adjusting for the propensity score must be close to one”. These conditions are rarely met in a non-randomized study design and therefore the use of matching procedures is more appropriate. It is important to note that matching methods and regression-based model adjustments should also not be seen as competing methods but rather as complementary. Austin, Cochran and Rubin [25], [26], Rosenbaum and Rubin [10] as well as Stuart [14] have all shown that the best approach is to combine the two methods. Regression analysis on those matched samples can adjust for small remaining differences and increase efficiency of our impact estimates as was the case in this study.
We highlight few limitations of this study. Several other factors might have contributed to all-cause mortality among the HIV/AIDs patients but the database used for this study did not have information on those variables and therefore the matching procedures excluded these unobserved variables. Matching methods assume that there are no unobserved differences between patients on ART and those who are not on ART (control group), conditional on the observed covariates. To satisfy this assumption, it is always advisable to include in the matching procedure all variables known to be related to both treatment assignment and the outcome[27],[28]. Although series of sensitivity analysis were conducted to assess the validity of this assumption, excluding important confounder could increase bias of impact estimate.
The study analysed secondary data that are pulled from all treatment centers into a national database and are thus subject to varying quality, incompleteness and under reporting. Also, not all events of HIV morbidity and mortality are captured at the treatment centres.
Nonetheless, we have illustrated the use of matching procedures in evaluating the causal effect of health intervention in an observational study.
Results from this study reiterate the importance of ART in reducing risk of all-cause mortality among HIV/AIDS patients. The various interventions put in place by Government and non-Governmental organizations in support of ART treatment in Ghana should continue unabated to help reduce mortality rate and improve health outcomes among HIV/AIDS patients in the country. There is the need to continue the education programme by NACP and its allied agencies on the importance of having continuous ART treatment for all HIV and AIDs patients in Ghana at all time.
Conclusion
The use of causal inference methodologies such as Mahalanobis distant metric matching within propensity score caliper has shown efficient estimates of the effectiveness of ART in reducing all-cause mortality among people living with HIV. Future observational studies should consider use of these methodologies to enhance estimates from such studies.
Acknowledgments
The authors acknowledge Dr. Alfred E. Yawson, Head of the Biostatistics Department, School of Public Heath University of Ghana for leading correspondence between the research team and NACP. We also acknowledge all HIV data managers across the country.
References
- 1.
UNAIDS, “Global AIDS Update,” pp. 1–16, 2016.
- 2.
P. A. Akwara, G. B. Fosu, P. Govindasamy, S. Alayón, and A. Hyslop, “An In-Depth Analysis of HIV Prevalence in Ghana,” 2005.
- 3. Sturge C., “Commentary. [References].,” Child Adolesc. Ment. Health, vol. 11(1) Feb, no. 2, pp. 46–48, 2006.
- 4.
CDC, “HIV/AIDS Factsheet Impact in Ghana CDC in Ghana,” no. Cdc, pp. 1–2, 2010.
- 5.
WHO, “Ghana: Life Expectancy,” Ghana WHO Stat. profile, pp. 1–3, 2016.
- 6. Manosuthi W., Chottanapand S., and Thongyen S., “Survival Rate and Risk Factors of Mortality Among HIV / Tuberculosis-Coinfected Patients With and Without Antiretroviral Therapy,” vol. 43, no. 1, pp. 42–46, 2006. pmid:16885778
- 7.
G. Y. Afful, “Factors influenicing low access to antiretrovirals for pregnant women living with HIV in Ghana to reduce mother to child transmission of HIV,” 2012.
- 8. Baser O., “Choosing propensity score matching over regression adjustment for causal inference: when, why and how it makes sense,” J. Med. Econ., vol. 4, no. 10, pp. 379–391, 2007.
- 9. Rosenbaum D. B., Rubin P. R., “The central role of the propensity score in observational studies for causal effects,” Biometrika, vol. 1, no. 70, pp. 41–55, 1983.
- 10. Rosenbaum D. B., Rubin P. R., “Constructing a control group using multivariate matched sampling methods that incorporate the propensity score,” Am. Stat., vol. 1, no. 39, pp. 33–38, 1985.
- 11. Dehejia S., Wahba R. H., “Causal effects in nonexperimental studies: Reevaluating the evaluation of training programs,” J. Am. Stat. Assoc., vol. 448, no. 94, pp. 1053–1062, 1999.
- 12. Dehejia S., Wahba R. H, “Propensity score-matching methods for nonexperimental causal studies,” Rev. Econ. Stat., vol. 1, no. 84, pp. 151–161, 2002.
- 13. Glazerman D., Levy S., Myers D. M., “Nonexperimental versus experimental estimates of earnings impacts,” Ann. Am. Acad. Pol. Soc. Sci., vol. 1, no. 589, pp. 63–93, 2003.
- 14. Stuart E. A., “Matching methods for causal inference: A review and a look forward. Statistical science: a review,” J. Inst. Math. Stat., vol. 1, no. 25, p. 1, 2010.
- 15. Rubin N., Thomas D. B., “Combining propensity score matching with additional adjustments for prognostic covariates,” J. Am. Stat. Assoc., vol. 450, no. 95, pp. 573–585, 2000.
- 16. Normand B. J., Landrum S. L. T., Guadagnoli M. B., Ayanian E., Ryan J. Z., Cleary T. J., P. D., et al. “Validating recommendations for coronary angiography following acute myocardial infarction in the elderly: a matched analysis using propensity scores,” J. Clin. Epidemiol., vol. 4, no. 54, pp. 387–398, 2001.
- 17. Rosenbaum P. R., “Heterogeneity and causality: Unit heterogeneity and design sensitivity in observational studies,” Am. Stat., vol. 2, no. 59, pp. 147–152, 2005.
- 18. Mugavero M. J., Catherine R, Lesko Stephen R, Cole H, Hall Irene, Westreich Daniel, et al. “The effect of antiretroviral therapy on all-cause mortality, generalized to persons diagnosed with HIV in the USA, 2009–11,” Int. J. Epidemiol., vol. 45, no. 1, pp. 140–150, 2016. pmid:26772869
- 19. Boulle A S. B., Schomaker M, May MT, Hogg RS, “Mortality in Patients with HIV-1 Infection Starting Antiretroviral Therapy in South Africa, Europe, or North America: A Collaborative Analysis of Prospective Studies.,” PLoS Med, vol. 11, no. 9.
- 20. Brechtl J. R., Breitbart W., Galietta M., Krivo S., and Rosenfeld B., “The use of highly active antiretroviral therapy (HAART) in patients with advanced HIV infection: impact on medical, palliative care, and quality of life outcomes.,” J. Pain Symptom Manage., vol. 21, no. 1, pp. 41–51, 2001. pmid:11223313
- 21. Davies M.-A., Gibb D., and Turkova A., “Survival of HIV-1 vertically infected children,” Curr. Opin. HIV AIDS, vol. 11, no. 5, pp. 455–464, 2016. pmid:27716730
- 22. Faizi Z., Rizwan B., Scott J., and Ahsan J., “Effectiveness of HIV Anti-retroviral Agents on the HIV/AIDS Epidemic,” Br. J. Med. Med. Res., vol. 15, no. 11, pp. 1–13, 2016.
- 23. Reniers G., Slaymaker E., Nakiyingi-Miiro J., Nyamukapa C., Crampin A. C., Herbst K., et al. “Mortality trends in the era of antiretroviral therapy,” Aids, vol. 28, pp. S533–S542, 2014. pmid:25406756
- 24. Barry O., Powell J., Renner L., Bonney E. Y., Prin M., Ampofo W., et al, “Effectiveness of first-line antiretroviral therapy and correlates of longitudinal changes in CD4 and viral load among HIV-infected children in Ghana,” 2013. pmid:24119088
- 25. Cochran D. B., Rubin W. G., “Controlling bias in observational studies: A review,” Sankhyā Indian J. Stat., no. Series A, pp. 417–446, 1973.
- 26. Austin P. C., “An introduction to propensity score methods for reducing the effects of confounding in observational studies,” Multivariate Behav. Res., vol. 3, no. 46, pp. 399–424, 2011.
- 27. Heckman P., Ichimura J. J., Todd H., “Matching as an econometric evaluation estimator,” Rev. Econ. Stud., vol. 65, no. 2, pp. 261–294, 1998.
- 28. Rubin N., Thomas D. B., “Matching using estimated propensity scores: relating theory to practice,” Biometrics, vol. 52, no. 1, pp. 249–264, 1996. pmid:8934595