Depression is highly prevalent among patients newly starting antiretroviral treatment (ART) in Malawi and many other countries. Untreated depression at ART initiation can disrupt the HIV care continuum. Effective approaches for depression screening and treatment exist for low-resource settings, but they are rarely applied. Identifying effective implementation strategies are critical.
A pilot program integrated depression screening and treatment into routine HIV care using existing staff at two public health clinics in Malawi in two phases; a screening-only “control” phase and an active “intervention” phase. During the intervention phase, providers prescribed antidepressants or referred patients for Friendship Bench problem-solving therapy. We evaluated the program’s impact on retention in HIV care, viral suppression, and depression remission at 6 months using tabular comparisons and log-binomial models to estimate adjusted risk ratios and mean differences among the intervention group relative to the control group.
Nearly all consenting participants were screened for depression appropriately and 25% had mild to severe depressive symptoms. During the intervention phase, 86% of participants with mild depressive symptoms started Friendship Bench therapy and 96% of participants with moderate to severe depressive symptoms started antidepressants. Few participants in the intervention group received consistent depression treatment over their first 6 months in care. In the adjusted main analysis, program exposure did not demonstrably affect most HIV or mental health outcomes, though the probability of currently being on ART at 6 months was significantly lower among the intervention group than the control group [RR 0.6(95%CI: 0.4–0.9)].
While it is feasible to integrate depression screening and treatment initiation into ART initiation, providing ongoing depression treatment over time is challenging. Similar implementation science studies focused on maintaining depression management will be increasingly important as we strive to understand and test the best ways to implement evidence-based depression treatment within HIV care.
Citation: Stockton MA, Udedi M, Kulisewa K, Hosseinipour MC, Gaynes BN, Mphonda SM, et al. (2020) The impact of an integrated depression and HIV treatment program on mental health and HIV care outcomes among people newly initiating antiretroviral therapy in Malawi. PLoS ONE 15(5): e0231872. https://doi.org/10.1371/journal.pone.0231872
Editor: Kwasi Torpey, University of Ghana College of Health Sciences, GHANA
Received: October 17, 2019; Accepted: April 1, 2020; Published: May 6, 2020
This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Data Availability: The data underlying the results presented in the this manuscript will be available in a USAID-managed repository.
Funding: This work was supported by Project SOAR (Cooperative agreement AID-OAA-140060), made possible by the generous support of the American people through the United States President’s Emergency Plan for AIDS Relief (PEPFAR) and United States Agency for International Development (USAID). The contents of this paper are the sole responsibility of the authors and do not necessarily reflect the views of PEPFAR, USAID, or the United States Government.
Competing interests: The authors have declared that no competing interests exist.
The burden of depression among people living with HIV in Malawi is high, ranging between 1–19% [1–4], as in other sub-Saharan African countries . Depression is associated with poor HIV care engagement and ultimately increased HIV-related mortality and morbidity [5–10]. The mechanisms through which depression undermines HIV care engagement are not fully clear. However, loss of interest, poor concentration, poor motivation, reduced self-efficacy, fatigue, hopelessness, and suicidality–key characteristics of depression–are all factors which can impair adherence and appointment attendance [8, 10, 11]. Fortunately, depression treatment programs have been developed that have the potential to improve both HIV care and mental health outcomes, with a small but growing number of interventions adapted for low- and middle-income countries [12–17]. In places such as Malawi, where nearly 10% of the adult population is living with HIV  and nearly a quarter of people living with HIV are lost to care in the first year of treatment [19–21], incorporating depression screening and treatment into HIV care may be key to improving engagement in care across the HIV care continuum.
There are limited resources for mental health care in Malawi, and a dearth of mental health infrastructure and specialists. Malawi treats mental healthcare as a specialized service, offered only by specialists in tertiary settings. There are only three psychiatrists and three functioning psychiatric hospitals in the country . In such settings where it is unlikely that the number of specialists and infrastructure could grow rapidly enough to meet the demands of the population, task-shifting programs–models of care that shift specialized services to non-specialists –are a popular, and often cost effective strategy for providing mental health services [24–26]. Of the few depression treatment interventions developed for the sub-Saharan region, most employ a task-shifting model [16, 27–32].
Two notable task-shifting interventions include algorithm-based care for depression [31–33] and the Friendship Bench behavioral activation and problem-solving therapy . Algorithm-based care is a resource-efficient, task-sharing model for prescribing antidepressant management in non-psychiatric settings. This model of care has demonstrated safety, feasibility, and acceptability when adapted for HIV care and delivered by general practice medical providers in Cameroon, Tanzania and Uganda [31–33]. Developed over many years of formative research in Zimbabwe, the Friendship Bench is patient-centered counseling that teaches patients how to identify triggers and effectively manage stressful life events by learning or reactivating problem solving skills [29, 34]. These programs as well as others have proven efficacy in improving depression outcomes, though evidence on improvements in HIV care outcomes has been mixed [27, 31–33, 35–37]. Further investment in understanding the feasibility of task-shifting program implementation and the effectiveness of these models of depression care is needed to meet the mental health care needs of people living with HIV.
The Malawi Ministry of Health (MOH) has recognized the importance of addressing the burden of depression among people living with HIV as a means of improving engagement in HIV care. The MOH has also prioritized the integration of mental health services into other general health services and the development of mental health capacity of general providers [38–40]. Expanding the growing nidus of investment in mental health care programming and capacity development, the MOH implemented a pilot program integrating depression management into HIV care at two public clinics in Lilongwe, Malawi, using both algorithm-guided depression treatment and adapted Friendship Bench therapy . Inspired by the key principles of implementation science, we use a multiple-baseline design to investigate the program’s impact on HIV care and depression outcomes.
The main objective of this study was to evaluate the program’s impact on retention, viral suppression, and depression remission among patients with elevated depressive symptoms at antiretroviral therapy (ART) initiation after 6 months in care.
This study employed a staggered multiple baseline (pre-post) design to evaluate a pilot program integrating depression screening and treatment into routine HIV primary care using existing staff at two public health clinics in Lilongwe, Malawi . Multiple baseline studies use a time-series design that can be used for studies with multiple sites in which each site intentionally receives the intervention at a different time point [42, 43]. This design can also provide evidence of causal relationships where randomization is not possible and provides stronger control for temporal trends [42, 43]. As such, the program rolled out in two phases, a screening-only “control” phase and an active “intervention” phase. The screening control phase launched at both clinics in April 2017. However, the launch of the intervention phase was staggered, launching at Clinic A in November 2017 and at Clinic B in April 2018. (Fig 1) Additional information on the study design is available in the published protocol paper .
During the control phase, providers screened patients for depression using the Patient Health Questionnaire-9 (PHQ-9). The PHQ-9, a nine-item screening questionnaire that assesses the presence and frequency of the nine Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition symptoms of major depression . It has been widely used in the region [2, 45, 46] and was recently validated in Malawi among patients with diabetes . A total score of 5–9 and ≥10 are considered indicative of mild depressive symptoms and moderate to severe depressive symptoms, respectively. Patients who endorsed the last question of the PHQ-9, which asks about thoughts of being better off dead or hurting oneself, completed a suicide risk assessment protocol (SRAP) with the providers. The SRAP guided providers’ evaluation of whether such thoughts were passive or active. During the control phase, ART providers were re-oriented to existing options for depression care within the Malawi primary care system. This options theoretically include counseling and antidepressants, although in actuality counseling consists of informal counseling by ART providers and antidepressants are rarely prescribed. For patients with suicidality, providers were trained to respond based on the level of severity, ranging from a brief safety assessment to immediate transport to the outpatient psychiatric department at the district hospital.
During the intervention phase, ART providers (nurses or clinicians) used PHQ-9 scores at ART initiation to triage patients by depressive severity. ART providers were trained to refer participants with mild depressive symptoms (PHQ-9 score 5–9) to clinic-based lay health workers or a project-employed counselor trained to provide an adapted problem-solving intervention called Friendship Bench therapy [27, 29]. These counselors provided patient-centered counseling that guided patients through recognizing problems, identifying their own solutions, implementing those solutions, and assessing the outcome[29, 34]. Patients referred to the Friendship Bench would ideally receive at least six counseling sessions. While these patients were encouraged to return weekly for Friendship Bench therapy, patients set their own appointments in line with the protocol’s patient-centered approach. For participants with moderate to severe depressive symptoms (PHQ-9 score ≥10), ART providers were trained to prescribe antidepressants (fluoxetine or amitriptyline) with appropriate dosing and dose changes, using an algorithm-based care protocol. Under this protocol, providers used PHQ-9 scores and drug side effects to guide antidepressant prescription, titrating the antidepressant dose by monitoring depressive symptoms and antidepressants side effects at each clinic visit [31–33] (Table 1). Patients who started antidepressants were meant to be reevaluated monthly at their ART visits when they would be re-prescribed antidepressants for at least three months. Additional information on the combined medication and counseling depression treatment program has previously been published .
From April 2017 through November 2018, all non-pregnant adults newly initiating ART at the study sites were eligible to be included in the program evaluation. Pregnant women were excluded as they typically test for HIV at a specific antenatal care clinic and continue to receive ART as part of their antenatal care. This analysis is restricted to participants who completed depression screening and had elevated depressive symptoms (PHQ-9 score ≥ five) at ART initiation. Participants were placed in the “control” or “intervention” groups based on the phase during which they initiated ART.
Control group: Includes participants with elevated depressive symptoms who initiated ART at either clinic during the screening “control” phase (e.g. at Clinic A between April 2017 and November 2017 and at Clinic B between April 2017 and April 2018).
Intervention group: Includes participants with elevated depressive symptoms who initiated ART at either clinic after the launch of the treatment program during the active “intervention” phase (e.g. at Clinic A after November 2017 and at Clinic B after April 2018).
As an implementation science study, the primary objective was to assess the impact of an intervention that could readily (and rapidly) be adopted and integrated into routine care in the public sector setting [49, 50]. Just as the program made use of existing clinical staff and systems, the evaluation was intentionally designed to influence and disrupt the provision of care as little as possible by collecting only data that was already routinely captured during clinical care and using research staff only to consent patients and abstract data. Specifically, we did not conduct baseline interviews or schedule six-month follow-up interviews, which would have limited enrollment and potentially brought participants back into care, biasing our primary outcome measure of HIV care engagement. Research assistants at the study sites approached potential participants during ART initiation to invite them to allow their clinical data to be used in the program evaluation. Research assistants abstracted routinely collected clinic data on depression and HIV care from consenting participants’ clinical records over a 13-month period, starting at ART initiation. Abstracted data included appointment dates, expected return dates, ART pills dispensed, PHQ-9 scores, and depression treatment provided. At ART initiation and each subsequent ART appointment, providers give patients a follow-up appointment date and a sufficient supply of ART. Generally, for the first six months of care, new ART patients receive a 30-day supply of ART and a return appointment date in 30 days at each appointment. Most often, patients need to attend monthly ART refill appointments for their first six months of care in order to maintain their ART supply. Additionally, we obtained electronic medical record data from the clinical sites to ensure the quality and completeness of the abstracted data.
Adequate Friendship Bench therapy should consist of six sessions over the first six months in care and a standard course of antidepressants should consist of at least three consecutive months of antidepressant prescription with dosage adjustments as necessary. For the analysis purposes, we have operationalized the intervention exposure as follows:
Intent-to-treat: Intervention group participants were considered “exposed” to the intervention if they initiated ART during the intervention phase. Control group participants were considered “unexposed” to the intervention if they initiated ART during the control phase.
Treatment started: The intervention group included participants who either had: 1) mild depressive symptoms at ART initiation who started Friendship Bench therapy at ART initiation or 2) moderate to severe depressive symptoms who were prescribed antidepressants at ART initiation. The control group participants did not start Friendship Bench therapy or antidepressants at ART initiation.
As treated: The intervention group included participants who attended at least their first follow-up appointment and either had: 1) mild depressive symptoms at ART initiation and at least two Friendship Bench therapy sessions over their first six months in care or 2) moderate to severe depressive symptoms at ART initiation and were prescribed at least two months of antidepressants over their first six months in care. The control group included participants who attended at least their first follow-up appointment, but did not have at least two friendship bench sessions or two months of antidepressant prescription over their first six months in care. Those participants who did not attend at least their first follow-up visit were excluded. While this definition was less stringent than the ideal course of treatment, this definition allowed for a more refined comparison of those who received early and consistent depression treatment compared to those who had not.
Retention was defined as never being more that 14 days late to an appointment through 6 months in care. Maintaining a consistent ART supply was defined as never more than 5 days without ART in the first 6 months, calculated from the cumulative days’ supply of ART dispensed at each appointment in the first 6 months and the time between appointments. In alignment with the PEPFAR definition of “currently on treatment” , participants were currently on ART 6 months after ART initiation if they attended an appointment and received a supply of ART that would last through the 6 month mark. Viral suppression was defined as a viral load of less than 1,000 copies/mL drawn after at least five and a half months (166 days) in care. Depression remission was defined as a PHQ-9 score of less than five taken after at least five and a half months (166 days) in care, to allow for completion of the acute phase of treatment and maintenance of that response . Additionally, we examined the proportion of scheduled HIV appointments attended within one week during the first 6 months of care and the ART pill possession ratio, the proportion of the first 6 months (183 days) since ART initiation with ART in hand, calculated from cumulative pills dispensed at each ART appointment through 6 months.
Due to the implementation science nature of this study, we only abstracted routinely collected clinical data. Measured covariates included the healthcare facility where patients received ART (Clinic A or B), months since program launch at ART initiation, gender, age, baseline depressive severity at ART initiation (mild or moderate to severe), and baseline presence of suicidal thoughts at ART initiation. Presence of suicidal thoughts was defined as any endorsement of the ninth question of the PHQ-9, “During the past two weeks, how many days have you been bothered by thoughts that you would be better off dead or of hurting yourself in some way?”
The main analysis followed an “intent-to-treat” approach, classifying participants according to screening “control” vs. active “intervention” phase (unexposed vs. exposed to the program) without regard to actual treatment received. Although patients identified with elevated depressive symptoms late in the control phase could theoretically have received depression care from providers during the intervention phase during their first six months of care, in practical terms, depression treatment was only provided to patients newly entering care. Furthermore, we monitored crossover between groups. We first completed unadjusted, tabular comparisons of HIV care and depression outcomes in the intervention group relative to the control group. We then used log-binomial models to estimate adjusted risk ratios comparing the probability of the primary outcome, retention with viral suppression, as well as the probability of other secondary HIV care outcomes in the intervention group compared to the control group. Since the evaluation employed a staggered multiple baseline design, with the intervention launching at different dates at the two clinics, all models were adjusted for clinic and months since program launch to allow for the assessment of and potential correction for any secular trend in outcomes (“confounding by history”). While the study design should have produced comparable control and intervention groups, we did additionally consider controlling for measured covariates. To separately evaluate the impact of the Friendship Bench and the antidepressant treatment, we then stratified the primary analysis by depressive severity at baseline (mild or moderate to severe depressive symptoms). Other secondary analyses used a “treatment started” approach, comparing those who actually started Friendship Bench therapy or antidepressants to those that did not in both phases and then an “as treated” approach, comparing those who received at least two Friendship Bench therapy sessions or two months of antidepressants to patients who did not in both phases, restricted to only those who attended at least their first follow-up visit.
Outcome data for those who transferred or died in the first 6-months of care were treated as missing. To address missing outcome data, we used multiple imputation by chained equations (MICE) to fill missing values using logistic regression imputation methods . We imputed values based on phase, healthcare facility, months since program launch at ART initiation, age, sex, baseline depressive severity at ART initiation and baseline presence of suicidality at ART initiation. We generated 18 imputed datasets to ensure the number of imputed datasets was at least as large as the percentage of incomplete information for the main outcome, “retention with viral suppression” . To verify that this was sufficient, we confirmed that the number of imputed datasets was also larger than the parameter-specific fraction of missing information for all parameters included in the final model .
All analyses were performed using STATA IC 14.
The National Health Sciences Research Committee of Malawi (NHSRC) and the Biomedical Institutional Review Board (IRB) of the University of North Carolina at Chapel Hill approved the study protocol. All participants gave written informed consent to allow the abstraction of their clinical data. The evaluation only used de-identified clinic data to ensure the protection of participants’ identities and confidentiality. Patients with depression received depression care regardless of whether or not they provided consent for their data to be abstracted.
Over the course of the program, 2414 patients newly initiated ART (Fig 2). Of these new initiators, 2067 (86%) consented to allow their data to be abstracted to evaluate the program. Nearly all of those who consented (96%) were screened with the PHQ-9 appropriately. Among those who completed the PHQ-9 screening, the prevalence of mild depressive symptoms (PHQ-9 score 5–9) was 19% and the prevalence of moderate to severe depressive symptoms (PHQ-9 score ≥ 10) was 6%. Only those participants with elevated depressive symptoms (PHQ-9 score ≥ 5) are included in this analysis (N = 501).
Of the 501 consenting participants with elevated depressive symptoms, the control group consisted of the 290 who initiated ART during the control phase and the intervention group consisted of the 211 who initiated during the intervention phase (Table 2). As the program launched later at Clinic B, a larger proportion (75%) of the intervention group were from Clinic A. Otherwise, the two groups appear very similar. About 43% of participants were male. The average age of participants was 33.8 years. Nearly all participants were classified as asymptomatic (Stage I) for HIV at ART initiation, according to the World Health Organization’s (WHO) HIV clinical stages for HIV surveillance . At ART initiation, 74% of participants had mild depressive symptoms, only around 1% of participants had severe depressive symptoms and 21% of participants reported suicidality.
Depression treatment exposure
Initiating depression treatment.
Among the control group, 89% of participants with mild depressive symptoms and 80% participants with moderate to severe depressive symptoms received some additional counseling by ART clinicians (“standard depression care” during the control phase) (Table 3). During this phase, only 18% (n = 14) of those with moderate to severe depressive symptoms started amitriptyline during the control phase and most (n = 11) started at a subtherapeutic dose of 25 mg.
Most of the intervention group started the appropriate treatment as defined by the intervention protocol during the intervention phase. During this phase, 86% of participants with mild depressive symptoms started Friendship Bench therapy and nearly all (96%) participants with moderate to severe depressive symptoms started antidepressants. All of the participants with moderate to severe depressive symptoms who started antidepressants at ART initiation started at a minimal effective dose (operationally defined as a daily oral dose of either 20 mg of fluoxetine or 50 mg of amitriptyline).
Provision of ongoing depression treatment.
Few patients in the intervention group continued to receive ongoing depression treatment during the intervention phase after ART initiation. Only 42% of participants referred to the Friendship Bench attended three or more therapy sessions (Table 4). Of those who were prescribed antidepressants, less than a third of participants (n = 17) received three or more months of antidepressant prescription. Only six of those participants received at least three consecutive months of the same antidepressant. Inadequate sustained treatment was in part due to poor appointment attendance, medication stock-outs, and clinicians’ failure to identify patients with elevated depressive symptoms at follow-up ART appointments. However, it did not appear that there was much crossover between groups. None of the control participants ever started Friendship Bench therapy and very few started antidepressants at ART initiation (n = 16) or at a follow-up visit (n = 6).
Most HIV and mental health outcomes did not appear to differ between groups. (Table 5). Retention in both arms was poor; approximately a third of participants in both groups remained in care through 6 months. Just over half of the control group and just under half of the intervention group were currently on ART at six months. Close to 40% of participants in both the control and intervention phases maintained a consistent ART supply (never more than 5 days without ART) through six months. Many participants had missing data for viral suppression and depression remission after 5.5 months; only 36% (n = 181) of participants had viral load draw after 5.5 months and 26% of participants (n = 132) had a measured PHQ-9 score. Among those with a measured viral load or PHQ-9 score after 5.5 months, respectively, viral suppression and depression remission were high (>91%) in both groups. Findings were similar when stratified by baseline depressive severity (mild or moderate to severe) and when using a “treatment started” and an “as treated” approach. S1 Table–S4 Table.
To assess potential bias due to missing data from transferring or attending care without getting a 6-month viral load or PHQ-9 assessment, we compared baseline information between those with and without missing data. While a larger proportion of transfers were from Clinic A, missing data did not appear to be associated with any other measured baseline characteristics. S5 Table–S7 Table.
The final adjusted model controlled for clinic, months since program launch at ART initiation, sex and baseline depressive severity. Both age and baseline presence of suicidality introduced model instability. As age did not appear to be associated with any of the outcomes or program exposure, we ultimately removed it from the model. We only retained baseline depressive severity in the final adjustment set as baseline presence of suicidality was highly correlated with baseline depressive severity.
In the adjusted “intent-to-treat” analysis, program exposure did not demonstrably affect the primary outcome, retention with viral suppression at 6 months (Table 6). While no difference was evident for most of the secondary outcomes, the probability of currently being on ART at 6 months and the ART pill possession ratio was significantly lower among the intervention group than among the control group [RR 0.6(95%CI: 0.4–0.9) and Mean Difference -0.1(95%CI: -0.3–0), respectively]. Correction for missing data with multiple imputation did not have a significant impact on any of the outcomes. When stratified by baseline severity, only the probability of being on ART at 6 months was significantly lower during the intervention phase than during the control phase [RR 0.33(95%CI: 0.13–0.83)] among those with moderate to severe depressive symptoms at baseline. S8 Table. The “treatment started” and “as treated” approaches generally showed similar results. S9 Table and S10 Table.
We investigated the impact of a program integrating depression management into HIV care initiation on 6-month HIV care and depression outcomes. Clinic staff screened nearly all patients for depression, documenting the prevalence of mild and moderate to severe depressive symptoms among people newly initiating ART. During the intervention phase of the program, they successfully started the majority of the intervention group on the appropriate depression treatment. However, providing ongoing treatment proved more challenging, and few patients received a standard course of antidepressants or attended a sufficient number of Friendship Bench therapy sessions. Retention was very low in both the intervention and control groups. Nearly all participants who did remain in care and had a 6-month viral load drawn and PHQ-9 assessment achieved viral suppression and depression remission. However, the evaluation did not yield evidence that the integrated depression treatment program improved 6-month HIV care or depression outcomes among the intervention group compared to the control group.
The successful integration of depression screening allowed this evaluation to document the prevalence of depression among people newly initiating ART. A growing body of research is beginning to establish the magnitude of mental health disorders in Malawi. In Malawian primary care settings, a third of all patients have a common mental disorder, most commonly depression . However, few studies have estimated the prevalence of depression among the general adult population of people living with HIV. We found that around a quarter of people newly initiating ART at the study sites had mild to severe depressive symptoms. This finding is only slightly higher than estimates from various subpopulations (adolescents, pregnant women, adults receiving HIV care) of people living with HIV in Malawi [1–3] and supports other sub-Saharan regional estimates of depression prevalence . Such evidence indicates a clear need for depression treatment among people living with HIV.
The integration of depression screening and treatment initiation at the time of ART initiation appeared feasible. During ART initiation, nearly all of the 2067 participants who consented to participate in the study were successfully screened for depression. Further, nearly all intervention participants started the treatment appropriate for their depressive severity at ART initiation. Factors that may have contributed to this success include the effective utilization of existing ART initiation processes and the creation of a collaborative training environment . For example, after careful study of patient flow, we designed the initial PHQ-9 screening to be shared by the HTC counselors and the ART providers, distributing the additional work and time burden. We also ensured that every HIV provider received training in how to administer and interpret the PHQ-9, provide depression treatment, and manage depression symptoms over time–creating opportunities for iterative learning and ongoing support . Another task-sharing study in Uganda found that both depression screening by lay workers and antidepressant treatment initiation by ART provider were feasible, a success the researchers attributed to ongoing training and appropriate mentorship . As such, using non-psychiatric specialists to screen and triage cases of depressions appears possible in this sub-Saharan setting.
While effective integration into existing processes was key to the success of depression screening and treatment initiation, the aspects of existing infrastructure and supply management that the program was unable to effectively utilize hampered the provision of sustained depression treatment. Providers at the study sites typically rely on an electronic medical records (EMR) system to provide ART, which did not incorporate PHQ-9 screening or depression treatment and thus did not alert providers to re-assess depressed patients returning for care. Despite developing a system of marking patients’ health passports so that those needing depression treatment re-evaluation could be identified, providers struggled to re-identify depressed patients returning for ART care. Very few studies on task-shifting models of care in the region have relied on existing staff (as opposed to study employed staff) or assessed the provision of depression treatment over time. For example, the study evaluating different task-sharing strategies for depression treatment in Uganda only reported on depression treatment initiation , making comparisons challenging. Other implementation science studies will need to track and assess the provision of treatment over time.
Antidepressant stock-outs were also common and problematic. While the Malawi Ministry of Health has committed to making antidepressants freely available at health center pharmacies , ensuring their availability was complicated and required substantial coordination, often beyond the scope of work of the health center pharmacist. Other countries such as Mozambique with similar policies on the provision of antidepressants have also experienced challenges stocking these medications at the district or clinic level . Another task-shifting depression program in Tanzania also experienced anti-depressant stock-outs . Greater engagement and investment of health sector stakeholders involved in the procurement, supply and distribution of drugs such as Malawi’s Central Medical Stores is required to ensure antidepressant stocks are maintained to ensure the success and safety of depression treatment programs.
The clinics also found it challenging to provide proper Friendship Bench therapy, in part due to community health care workers’ availability and in part due to patients’ ability to return to the clinic for therapy sessions, in light of financial, time, and transport barriers. As the community health workers already had a very high workload and often traveled off-site to run various Ministry of Health initiatives, the one program-employed counselor at each site ultimately provided the majority of the therapy sessions. Community health workers are often already overloaded with work and are at risk of becoming additionally overburdened when drawn into task-sharing models of care [60, 61], as was potentially the case at the study sites. Furthermore, while the original Friendship Bench protocol called for 6 weeks of weekly therapy sessions , patients were reluctant to make additional trips to the clinic primarily due to time and transportation barriers. This resulted in scheduling Friendship Bench sessions to coincide with the patients’ monthly ART refill appointments. While attending weekly sessions appeared acceptable in the original Friendship Bench program , it is difficult to tell from other adaptations as they may have incentivized attending sessions with reimbursement . These findings highlight the importance of implementation science research to establish the real-world feasibility of task-shifting depression care for people living with HIV.
As implemented, the program did not appear to improve HIV care or depression outcomes, as few patients received depression treatment as intended. Studies in the region have found that similar programs, albeit with higher fidelity to their treatment protocols, have been effective at treating depression [35, 36]. In our study, nearly all participants achieved depression remission, regardless of depression treatment. As many of these other evaluations of depression treatment programs were quite small or did not include control arms [35, 36], it is possible that depressive symptoms could have improved in the absence of depression treatment. However, it should be noted that we could have overestimated six-month depression remission if patients who did not remain in care were more likely to have persistent depressive symptoms. In regards to the program’s impact on HIV outcomes, the literature on linking depression treatment to improvements across the HIV care cascade is mixed [13–16, 62–69]. A recent meta-analysis of depression treatment interventions for people living with HIV in sub-Saharan Africa also did not find that interventions improve viral suppression, though they did conclude that the programs with an ART adherence component had the greatest impact on HIV outcomes . As it was impossible for the program to have a pure control group in which patients were denied depression treatment, it is possible that providers discussed the depression screening results with participants or provided brief additional ART adherence counseling. While even unstructured additional patient attention has the potential to impact engagement in care, it is no substitute for evidence-based depression treatment and adherence counseling interventions. Further research is needed to understand the factors that influenced adherence to the program protocol and to identify the more effective ways to implement depression treatment programs that will improve depression and HIV care outcomes.
Retention in care was remarkably low in both of the study phases; only a quarter of patients achieved retention and viral suppression at 6 months. Aligned with PEPFAR’s less stringent definition of being in care at 6 months (our “currently on ART” measure) , only half of patients were retained at six months. These estimates are much lower than national 12-month estimates from 2018, which found that 65% of adults who initiated ART were still in care at 12 months. However, as our entire study population had elevated depressive symptoms, it possible that depression is responsible for this difference in retention. The program did not appear to improve HIV care engagement, even when using the “treatment started” or “as treated” approach, and in the main analysis the intervention group was significantly less likely to currently be on ART at 6-months than the control group. It is possible that this population faced other barriers to care that were more relevant and difficult to overcome, such as human resource and institutional challenges, distance to the clinic, lack of support, stigma and fear of HIV status disclosure [71–74]. Nonetheless, there is a distinct need for urgent action to improve early retention in care for people living with HIV and depression.
The results should be considered in light of several limitations. Although the study’s design was not guided by a formal implementation science framework, the study was designed in congruence with the key principles of implementation science, using methods that would promote the systematic uptake of evidence-based practices (e.g. the Friendship Bench and algorithm-based care) into routine care [75, 76]. In line with the implementation science-inspired design of the study and our efforts not to unduly influence the provision of or engagement in care, all of the measures were drawn from routinely collected medical chart data. As we did not conduct baseline interviews or schedule six-month follow-up interviews, we captured limited information on potential confounders. A large proportion of patients did not have six-month viral loads or PHQ-9 scores due to either dropout or provider failure to refer patients for viral load testing or screen patients appropriately. Still, the multiple baseline design should have protected against confounding from unmeasured confounders by producing balanced characteristics across phases, as evident from similar characteristics of intervention and control groups seen in Table 2. Additionally, the program struggled to establish a sense of ownership among providers, develop capacity to manage depression treatment over time, and ensure the availability of antidepressants, which may have impacted providers’ adherence to the depression treatment protocol . As an extension of this challenge, ART providers administered the six-month PHQ-9, so part of the observed reduction in PHQ-9 scores could be due to less careful administration.
All of the HIV engagement measures may have been influenced by participants who sought care at another facility without formally transferring their records. These “silent transfers” would have been misclassified as being out of care. However, as the rate of silent transfers should not have varied between groups, this potential measurement error should not have introduced bias into the analyses.
This experience demonstrated that while it is feasible to integrate depression screening and treatment initiation into ART initiation, providing ongoing depression treatment over time is challenging; very few patients received ongoing depression treatment for many of the reasons discussed above. In this setting, it is clear that in order for such a program to be successful additional resources are needed to support ongoing capacity building, ensure the availability of Friendship Bench counselors and antidepressants, encourage patient engagement in the full-course of treatment, and further integrate into the electronic medical records system. Additionally, further research is needed to explore the other factors that potentially govern both the implementation of and engagement with such a program. While the evaluation did not yield evidence that the program improved HIV care or depression outcomes in a real world clinic setting, depression treatment in sub-Saharan Africa is efficacious for improving mental health and engagement in HIV care in more controlled research environments . Further, this program targeted patients newly initiating ART, often at the highest risk for loss to care. Further research could explore whether other patient populations may benefit from depression treatment, such as those returning for care with elevated depressive symptoms or whether depression treatment for people living with HIV may be more effective if it includes a specific component supporting ART adherence or HIV care engagement. Moving the field of mental health care in LMICs forward, similar implementation science studies will be increasingly important as we strive to understand and test the best ways to implement evidence-based depression treatment protocols for this vulnerable population.
S1 Table. Program impact on HIV and depression outcomes, among those with mild depression (N = 371).
S2 Table. Program impact on HIV and depression outcomes, among those with moderate to severe depression (N = 131).
S3 Table. Program impact on HIV and depression outcomes, “treatment started” approach* (N = 502).
S4 Table. Program impact on HIV and depression outcomes, “as treated” approach* (N = 358).
S5 Table. Participant characteristics, by transfer (N = 502).
S6 Table. Participant characteristics, by viral load data (N = 249).
S7 Table. Participant characteristics, by 6-month PHQ-9 data (N = 241).
S8 Table. Association between depression treatment and HIV care and depression outcomes, by depressive severity at baseline.
S9 Table. Association between depression treatment and HIV care and depression outcomes, “treatment started” approach*.
We would like to express our gratitude to the participants who allowed us to use their HIV and mental health clinical data to evaluate this program. We would also like to acknowledge the health facility staff at the program sites for their willingness to participate in trainings and provide mental health screening and treatment. As well, we are especially appreciative of the collaboration with the Lilongwe District Health Office, Lighthouse Trust, and Baylor, which facilitated comprehensive training. We are grateful to the Friendship Bench team who generously shared materials and led a training of trainers in Malawi to launch the Friendship Bench program. We would like to thank our program coordinator, Steve Mphonda, without whom the implementation of this project would not have been possible as well as our team of research assistants, who worked tirelessly to consent participants and ensure the highest quality of data.
- 1. Kim MH, Mazenga AC, Devandra A, Ahmed S, Kazembe PN, Yu X, et al. Prevalence of depression and validation of the Beck Depression Inventory-II and the Children's Depression Inventory-Short amongst HIV-positive adolescents in Malawi. Journal of the International AIDS Society. 2014;17(1).
- 2. Malava JK, Lancaster KE, Hosseinipour MC, Rosenberg NE, O’Donnell JK, Kauye F, et al. Prevalence and correlates of probable depression diagnosis and suicidality among patients receiving HIV care in Lilongwe, Malawi. Malawi Med Journal. 2018;30(4).
- 3. Harrington BJ, Hosseinipour MC, Maliwichi M, Phulusa J, Jumbe A, Wallie S, et al. Prevalence and incidence of probable perinatal depression among women enrolled in Option B+ antenatal HIV care in Malawi. J Affect Disorders. 2018.
- 4. Dow A, Dube Q, Pence BW, Van Rie A. Postpartum depression and HIV infection among women in Malawi. Journal of acquired immune deficiency syndromes. 2014;65(3):359–65. pmid:24189149.
- 5. Nakimuli-Mpungu E, Bass JK, Alexandre P, Mills EJ, Musisi S, Ram M, et al. Depression, Alcohol Use and Adherence to Antiretroviral Therapy in Sub-Saharan Africa: A Systematic Review. AIDS and behavior. 2011. Epub 2011/11/26. pmid:22116638.
- 6. Smillie K, Van Borek N, van der Kop ML, Lukhwaro A, Li N, Karanja S, et al. Mobile health for early retention in HIV care: a qualitative study in Kenya (WelTel Retain). Afr J AIDS Res. 2014;13(4):331–8. pmid:25555099.
- 7. Franke MF, Kaigamba F, Socci AR, Hakizamungu M, Patel A, Bagiruwigize E, et al. Improved retention associated with community-based accompaniment for antiretroviral therapy delivery in rural Rwanda. Clin Infect Dis. 2013;56(9):1319–26. pmid:23249611.
- 8. Gonzalez JS, Batchelder AW, Psaros C, Safren SA. Depression and HIV/AIDS treatment nonadherence: a review and meta-analysis. Journal of acquired immune deficiency syndromes. 2011;58(2):181–7. pmid:21857529; PubMed Central PMCID: PMC3858003.
- 9. Pence BW, Miller WC, Gaynes BN, Eron JJ Jr. Psychiatric illness and virologic response in patients initiating highly active antiretroviral therapy. Journal of acquired immune deficiency syndromes. 2007;44(2):159–66. Epub 2006/12/06. pmid:17146374.
- 10. Kidia K, Machando D, Bere T, Macpherson K, Nyamayaro P, Potter L, et al. 'I was thinking too much': experiences of HIV-positive adults with common mental disorders and poor adherence to antiretroviral therapy in Zimbabwe. Trop Med Int Health. 2015;20(7):903–13. pmid:25754063; PubMed Central PMCID: PMC4500937.
- 11. Losina E, Bassett IV, Giddy J, Chetty S, Regan S, Walensky RP, et al. The “ART” of linkage: pre-treatment loss to care after HIV diagnosis at two PEPFAR sites in Durban, South Africa. PloS one. 2010;5(3):e9538. pmid:20209059
- 12. Sin NL, DiMatteo MR. Depression treatment enhances adherence to antiretroviral therapy: a meta-analysis. Ann Behav Med. 2014;47(3):259–69. pmid:24234601; PubMed Central PMCID: PMC4021003.
- 13. Tsai AC, Weiser SD, Petersen ML, Ragland K, Kushel MB, Bangsberg DR. A marginal structural model to estimate the causal effect of antidepressant medication treatment on viral suppression among homeless and marginally housed persons with HIV. Archives of general psychiatry. 2010;67(12):1282–90. pmid:21135328; PubMed Central PMCID: PMC3208399.
- 14. Walkup J, Wei W, Sambamoorthi U, Crystal S. Antidepressant Treatment and Adherence to Combination Antiretroviral Therapy among Patients with AIDS and Diagnosed Depression. The Psychiatric quarterly. 2008;79(1):43–53. pmid:18095166.
- 15. Yun LW, Maravi M, Kobayashi JS, Barton PL, Davidson AJ. Antidepressant Treatment Improves Adherence to Antiretroviral Therapy Among Depressed HIV-Infected Patients. Journal of acquired immune deficiency syndromes. 2005;38(4):432–8. pmid:15764960.
- 16. Gaynes BN, Pence BW, Atashili J, O’Donnell JK, Njamnshi AK, Tabenyang ME, et al. Changes in HIV outcomes following depression care in a resource-limited setting: results from a pilot study in Bamenda, Cameroon. PLoS One. 2015;10(10):e0140001. pmid:26469186
- 17. Bockting C, Williams A, Carswell K, Grech A. The potential of low-intensity and online interventions for depression in low-and middle-income countries. Global Mental Health. 2016;3.
- 18. HIV/AIDS JUNPo. UNAIDS data 2017. Geneva, Switzerland: UNAIDS, 2017.
- 19. Fox MP, Rosen S. Retention of Adult Patients on Antiretroviral Therapy in Low- and Middle-Income Countries: Systematic Review and Meta-analysis 2008–2013. Journal of acquired immune deficiency syndromes. 2015;69(1):98–108. pmid:25942461; PubMed Central PMCID: PMC4422218.
- 20. Rasschaert F, Koole O, Zachariah R, Lynen L, Manzi M, Van Damme W. Short and long term retention in antiretroviral care in health facilities in rural Malawi and Zimbabwe. BMC Health Serv Res. 2012;12:444. pmid:23216919; PubMed Central PMCID: PMC3558332.
- 21. Massaquoi M, Zachariah R, Manzi M, Pasulani O, Misindi D, Mwagomba B, et al. Patient retention and attrition on antiretroviral treatment at district level in rural Malawi. Trans R Soc Trop Med Hyg. 2009;103(6):594–600. pmid:19298993.
- 22. Udedi M, Stockton MA, Kulisewa K, Hosseinipour MC, Gaynes BN, Mphonda SM, et al. Integrating depression management into HIV primary care in central Malawi: the implementation of a pilot capacity building program. BMC Health Serv Res. 2018;18(1):593. pmid:30064418; PubMed Central PMCID: PMC6069990.
- 23. Organization WH. Task shifting: rational redistribution of tasks among health workforce teams: global recommendations and guidelines. Geneva, Switzerland: WHO, 2007.
- 24. Chuah FLH, Haldane VE, Cervero-Liceras F, Ong SE, Sigfrid LA, Murphy G, et al. Interventions and approaches to integrating HIV and mental health services: a systematic review. Health policy and planning. 2017;32(suppl_4):iv27–iv47. pmid:29106512
- 25. World Health Organization. Consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection: Recommendations for a public health approach Geneva, Switzerland: World Health Organization 2016.
- 26. World Health Organization. Integrating mental health into primary care: a global perspective. Geneva, Switzerland: World Health Organization and World Organization of Family Doctors (Wonca), 2008.
- 27. Chibanda D, Weiss HA, Verhey R, Simms V, Munjoma R, Rusakaniko S, et al. Effect of a primary care–based psychological intervention on symptoms of common mental disorders in Zimbabwe: a randomized clinical trial. Jama. 2016;316(24):2618–26. pmid:28027368
- 28. Wagner GJ, Slaughter M, Ghosh-Dastidar B. Depression at Treatment Initiation Predicts HIV Antiretroviral Adherence in Uganda. Journal of the International Association of Providers of AIDS Care (JIAPAC). 2017;16(1):91–7.
- 29. Chibanda D, Mesu P, Kajawu L, Cowan F, Araya R, Abas MA. Problem-solving therapy for depression and common mental disorders in Zimbabwe: piloting a task-shifting primary mental health care intervention in a population with a high prevalence of people living with HIV. BMC public health. 2011;11(1):828.
- 30. Adams JL, Almond ML, Ringo EJ, Shangali WH, Sikkema KJ. Feasibility of nurse-led antidepressant medication management of depression in an HIV clinic in Tanzania. The International Journal of Psychiatry in Medicine. 2012;43(2):105–17. pmid:22849034
- 31. Adams JL, Almond ML, Ringo EJ, Shangali WH, Sikkema KJ. Feasibility of nurse-led antidepressant medication management of depression in an HIV clinic in Tanzania. International journal of psychiatry in medicine. 2012;43(2):105–17. pmid:22849034; PubMed Central PMCID: PMC3731063.
- 32. Pence BW, Gaynes BN, Atashili J, O'Donnell JK, Kats D, Whetten K, et al. Feasibility, Safety, Acceptability, and Preliminary Efficacy of Measurement-Based Care Depression Treatment for HIV Patients in Bamenda, Cameroon. AIDS and behavior. 2014. pmid:24558099.
- 33. Wagner GJ, Ngo V, Goutam P, Glick P, Musisi S, Akena D. A Structured Protocol Model of Depression Care versus Clinical Acumen: A Cluster Randomized Trial of the Effects on Depression Screening, Diagnostic Evaluation, and Treatment Uptake in Ugandan HIV Clinics. PLoS One. 2016;11(5):e0153132. pmid:27167852; PubMed Central PMCID: PMC4864192.
- 34. Pierce D. Problem solving therapy—use and effectiveness in general practice. Aust Fam Physician. 2012;41(9):676–9. pmid:22962642.
- 35. Lofgren SM, Nakasujja N, Boulware DR. Systematic review of interventions for depression for people living with HIV in Africa. AIDS and behavior. 2018;22(1):1–8. pmid:28900756
- 36. Passchier RV, Abas MA, Ebuenyi ID, Pariante CM. Effectiveness of depression interventions for people living with HIV in Sub-Saharan Africa: A systematic review & meta-analysis of psychological & immunological outcomes. Brain, behavior, and immunity. 2018;73:261–73. pmid:29768184
- 37. Chibanda D, Mesu P, Kajawu L, Cowan F, Araya R, Abas MA. Problem-solving therapy for depression and common mental disorders in Zimbabwe: piloting a task-shifting primary mental health care intervention in a population with a high prevalence of people living with HIV. BMC Public Health. 2011;11. pmid:22029430
- 38. Lupafya PC, Mwagomba BLM, Hosig K, Maseko LM, Chimbali H. Implementation of policies and strategies for control of noncommunicable diseases in Malawi: challenges and opportunities. Health Education & Behavior. 2016;43(1_suppl):64S–9S.
- 39. Government of Malawi Ministry of Health. Malawi Health Sector Strategic Plan II 2017–2022: Towards Universal Health Coverage. Lilongwe, Malawi: Government of Malawi Ministry of Health, 2017.
- 40. Government of Malawi Ministry of Health. The National Action Plan for NCDs and Mental Health (2012–2016). Lilongwe, Malawi.: Government of Malawi Ministry of Health, 2013.
- 41. Udedi M, Stockton MA, Kulisewa K, Hosseinipour MC, Gaynes BN, Mphonda SM, et al. The effectiveness of depression management for improving HIV care outcomes in Malawi: protocol for a quasi-experimental study. BMC Public Health. 2019;19(1):827. pmid:31242877
- 42. Hawkins NG, Sanson-Fisher RW, Shakeshaft A, D’Este C, Green LW. The multiple baseline design for evaluating population-based research. American journal of preventive medicine. 2007;33(2):162–8. pmid:17673105
- 43. Proeschold-Bell RJ, Swift R, Moore HE, Bennett G, Li X-F, Blouin R, et al. Use of a randomized multiple baseline design: rationale and design of the spirited life holistic health intervention study. Contemporary clinical trials. 2013;35(2):138–52. pmid:23685205
- 44. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4th (text revision) edition. Washington, DC: 2000.
- 45. Cholera R, Gaynes B, Pence B, Bassett J, Qangule N, Macphail C, et al. Validity of the Patient Health Questionnaire-9 to screen for depression in a high-HIV burden primary healthcare clinic in Johannesburg, South Africa. J Affect Disorders. 2014;167:160–6. pmid:24972364
- 46. Monahan PO, Shacham E, Reece M, Kroenke K, Ong’or WO, Omollo O, et al. Validity/reliability of PHQ-9 and PHQ-2 depression scales among adults living with HIV/AIDS in western Kenya. Journal of General Internal Medicine. 2009;24(2):189. pmid:19031037
- 47. Udedi M, Muula AS, Stewart RC, Pence BW. The validity of the patient health Questionnaire-9 to screen for depression in patients with type-2 diabetes mellitus in non-communicable diseases clinics in Malawi. BMC psychiatry. 2019;19(1):81. pmid:30813922
- 48. Kroenke K, Spitzer R, Williams J. The PHQ-9: Validity of a brief depression severity measure. Gen Intern Med. 2001; 16: 606–13.
- 49. Neta G, Brownson RC, Chambers DA. Opportunities for epidemiologists in implementation science: A primer. American journal of epidemiology. 2017;187(5):899–910.
- 50. Theobald S, Brandes N, Gyapong M, El-Saharty S, Proctor E, Diaz T, et al. Implementation research: new imperatives and opportunities in global health. The Lancet. 2018;392(10160):2214–28.
- 51. Relief PsEPfA. Monitoring, Evaluation, and Reporting (MER 2.0) Indicator Reference Guide Version 2.3. Washington (DC): PEPFAR, 2018.
- 52. Qaseem A, Barry MJ, Kansagara D. Nonpharmacologic versus pharmacologic treatment of adult patients with major depressive disorder: a clinical practice guideline from the American College of Physicians. Annals of internal medicine. 2016;164(5):350–9. pmid:26857948
- 53. Van Buuren S. Multiple imputation of discrete and continuous data by fully conditional specification. Statistical methods in medical research. 2007;16(3):219–42. pmid:17621469
- 54. White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Statistics in medicine. 2011;30(4):377–99. pmid:21225900
- 55. Von Hippel PT. How to impute interactions, squares, and other transformed variables. Sociological methodology. 2009;39(1):265–91.
- 56. World Health Organization. Interim WHO clinical staging of HVI/AIDS and HIV/AIDS case definitions for surveillance: African Region. Geneva: World Health Organization, 2005.
- 57. Kauye F, Jenkins R, Rahman A. Training primary health care workers in mental health and its impact on diagnoses of common mental disorders in primary care of a developing country, Malawi: a cluster-randomized controlled trial. Psychological medicine. 2014;44(3):657–66. pmid:23721658.
- 58. Wagenaar BH, Stergachis A, Rao D, Hoek R, Cumbe V, Napúa M, et al. The availability of essential medicines for mental healthcare in Sofala, Mozambique. Global health action. 2015;8(1):27942.
- 59. Stockton M, Ruegsegger L, Kulisewa K, Akiba C, Hosseinipour M, Gaynes B, et al. “Depression to me means…”: Knowledge and attitudes towards depression among HIV care providers and patients in Malawi. AIDS and Behaviour. Under Review. pmid:4418964
- 60. Surjaningrum ER, Minas H, Jorm AF, Kakuma R. The feasibility of a role for community health workers in integrated mental health care for perinatal depression: a qualitative study from Surabaya, Indonesia. International journal of mental health systems. 2018;12(1):27.
- 61. Jaskiewicz W, Tulenko K. Increasing community health worker productivity and effectiveness: a review of the influence of the work environment. Human resources for health. 2012;10(1):38.
- 62. Abas M, Nyamayaro P, Bere T, Saruchera E, Mothobi N, Simms V, et al. Feasibility and acceptability of a task-shifted intervention to enhance adherence to HIV medication and improve depression in people living with hiv in Zimbabwe, a low income country in sub-Saharan Africa. AIDS and behavior. 2018;22(1):86–101. pmid:28063075
- 63. Tsai AC, Karasic DH, Hammer GP, Charlebois ED, Ragland K, Moss AR, et al. Directly observed antidepressant medication treatment and HIV outcomes among homeless and marginally housed HIV-positive adults: a randomized controlled trial. American journal of public health. 2013;103(2):308–15. Epub 2012/06/23. pmid:22720766; PubMed Central PMCID: PMC3558777.
- 64. Pyne JM, Fortney JC, Curran GM, Tripathi S, Atkinson JH, Kilbourne AM, et al. Effectiveness of collaborative care for depression in human immunodeficiency virus clinics. Arch Intern Med. 2011;171(1):23–31. Epub 2011/01/12. 171/1/23 [pii] pmid:21220657.
- 65. Pence BW, Gaynes BN, Adams JL, Thielman NM, Heine AD, Mugavero MJ, et al. The effect of antidepressant treatment on HIV and depression outcomes: results from a randomized trial. AIDS. 2015;29(15):1975–86. pmid:26134881; PubMed Central PMCID: PMC4669218.
- 66. Safren SA, O'Cleirigh C, Tan JY, Raminani SR, Reilly LC, Otto MW, et al. A randomized controlled trial of cognitive behavioral therapy for adherence and depression (CBT-AD) in HIV-infected individuals. Health Psychol. 2009;28(1):1–10. Epub 2009/02/13. 2009-00026-001 [pii] pmid:19210012.
- 67. Safren SA, O'Cleirigh CM, Bullis JR, Otto MW, Stein MD, Pollack MH. Cognitive behavioral therapy for adherence and depression (CBT-AD) in HIV-infected injection drug users: a randomized controlled trial. Journal of consulting and clinical psychology. 2012;80(3):404–15. pmid:22545737; PubMed Central PMCID: PMC3365619.
- 68. Simoni JM, Wiebe JS, Sauceda JA, Huh D, Sanchez G, Longoria V, et al. A preliminary RCT of CBT-AD for adherence and depression among HIV-positive Latinos on the U.S.-Mexico border: the Nuevo Dia study. AIDS and behavior. 2013;17(8):2816–29. pmid:23812892; PubMed Central PMCID: PMC3788062.
- 69. Andersen LS, Magidson JF, O’Cleirigh C, Remmert JE, Kagee A, Leaver M, et al. A pilot study of a nurse-delivered cognitive behavioral therapy intervention (Ziphamandla) for adherence and depression in HIV in South Africa. Journal of health psychology. 2018;23(6):776–87. pmid:27121977
- 70. Joint United Nations Programme on HIV/AIDS. Global AIDS Monitoring 2019. Geneva, Switzerland: UNAIDS, 2019.
- 71. Wroe EB, Dunbar EL, Kalanga N, Dullie L, Kachimanga C, Mganga A, et al. Delivering comprehensive HIV services across the HIV care continuum: a comparative analysis of survival and progress towards 90-90-90 in rural Malawi. BMJ global health. 2018;3(1):e000552. pmid:29564158
- 72. Hall BJ, Sou K-L, Beanland R, Lacky M, Tso LS, Ma Q, et al. Barriers and facilitators to interventions improving retention in HIV care: a qualitative evidence meta-synthesis. AIDS and behavior. 2017;21(6):1755–67. pmid:27582088
- 73. Bilinski A, Birru E, Peckarsky M, Herce M, Kalanga N, Neumann C, et al. Distance to care, enrollment and loss to follow-up of HIV patients during decentralization of antiretroviral therapy in Neno District, Malawi: A retrospective cohort study. PloS one. 2017;12(10):e0185699. pmid:28973035
- 74. Bulsara SM, Wainberg ML, Newton-John TR. Predictors of adult retention in HIV care: a systematic review. AIDS and behavior. 2018;22(3):752–64. pmid:27990582
- 75. Eccles MP, Mittman BS. Welcome to implementation science. Springer; 2006.
- 76. Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An introduction to implementation science for the non-specialist. BMC psychology. 2015;3(1):32.