The majority of global cardiovascular disease (CVD) burden falls on people living in low- and middle-income countries (LMICs). In order to reduce preventable CVD mortality and morbidity, LMIC health systems and health care providers need to improve the delivery and quality of CVD care.
As part of the Disease Control Priorities Three (DCP3) Study efforts addressing quality improvement, we reviewed and summarized currently available evidence on interventions to improve quality of clinic-based CVD prevention and management in LMICs.
We conducted a narrative review of published comparative clinical trials that evaluated efficacy or effectiveness of clinic-based CVD prevention and management quality improvement interventions in LMICs. Conditions selected a priori included hypertension, diabetes, hyperlipidemia, coronary artery disease, stroke, rheumatic heart disease, and congestive heart failure. MEDLINE and EMBASE electronic databases were systematically searched. Studies were categorized as occurring at the system or patient/provider level and as treating the acute or chronic phase of CVD.
From 847 articles identified in the electronic search, 49 met full inclusion criteria and were selected for review. Selected studies were performed in 19 different LMICs. There were 10 studies of system level quality improvement interventions, 38 studies of patient/provider interventions, and one study that fit both criteria. At the patient/provider level, regardless of the specific intervention, intensified, team-based care generally led to improved medication adherence and hypertension control. At the system level, studies provided evidence that introduction of universal health insurance coverage improved hypertension and diabetes control. Studies of system and patient/provider level acute coronary syndrome quality improvement interventions yielded inconclusive results. The duration of most studies was less than 12 months.
The results of this review suggest that CVD care quality improvement can be successfully implemented in LMICs. Most studies focused on chronic CVD conditions; more acute CVD care quality improvement studies are needed. Longer term interventions and follow-up will be needed in order to assess the sustainability of quality improvement efforts in LMICs.
Citation: Lee ES, Vedanthan R, Jeemon P, Kamano JH, Kudesia P, Rajan V, et al. (2016) Quality Improvement for Cardiovascular Disease Care in Low- and Middle-Income Countries: A Systematic Review. PLoS ONE 11(6): e0157036. doi:10.1371/journal.pone.0157036
Editor: Gianpaolo Reboldi, University of Perugia, ITALY
Received: January 26, 2016; Accepted: May 24, 2016; Published: June 14, 2016
This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: AEM was supported by a grant from the U.S. National Heart, Lung and Blood Institute career development award (K08 HL089675-01; www.nhlbi.nih.gov). RV was supported by a grant from the Fogarty International Center of the U.S. National Institutes of Health (K01 TW 009218-05; www.flc.nih.gov). PJ is supported by a Wellcome Trust, Department of Biotechnology India Alliance clinical and public health intermediate career fellowship (www.wellcome.ac.uk). The views expressed in this article are those of the authors and do not necessarily represent the views of the National Heart, Lung, and Blood Institute, National Institutes of Health, or the U.S. Department of Health and Human Services. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Cardiovascular disease (CVD) is the leading cause of death globally. While the number of CVD deaths has not changed significantly in high-income countries since 1990, the number of deaths has increased by about 66% in low- and middle-income countries (LMICs) over the same interval. While attempting to address the burden of CVD, LMICs face the challenges of limited health care budgets and infrastructure as well as constrained professional health workforce capacity.
In the context of scarce resources available to support health care in LMICs, the imperative of clinical quality improvement is to get the most out of known effective interventions within the limitations of available resources rather than recommending unproven interventions that would require early phase studies or those that would require substantial financial and human resource investment for implementation. Clinical quality improvement can be implemented within any setting immediately, and need not be expensive.
Quality care delivery in lower resource settings does not necessarily mean dissemination and implementation of a universal set of standards formulated in high-income countries. Standards and interventions should be dictated by context and community capacity. Adaptation to local settings is necessary to achieve optimal clinical outcomes and patient satisfaction.
As part of the Disease Control Priorities Three (DCP3) Study efforts addressing quality improvement, we reviewed and narratively summarized currently available evidence on quality improvement for CVD prevention and management in LMICs. In keeping with the Institute of Medicine definition of clinical care quality, we focused on studies that compared specific clinical care quality improvement interventions with usual care and specified measurable health outcomes.
Materials and Methods
A conceptual framework guided the systematic review. We specified four domains, cross-cutting between two distinct phases of CVD (acute versus chronic) and two levels of intervention (health system versus patient/provider) (Fig 1). Acute phases of CVD, such as acute myocardial infarction, stroke, and limb ischemia, occur unpredictably. Good outcomes demand timely clinical responses, which require adequate and accessible facilities, functional transportation networks, providers prepared to treat cases that present at all hours, and patient awareness of when and how to seek medical attention. In contrast, chronic phases of CVD, such as diabetes mellitus, hypertension, and congestive heart failure (CHF), require screening for pre-clinical risk factors, systematic monitoring for complications, and substantial patient self-care and engagement in order to initiate and maintain treatment adherence. Good quality chronic phase care may prevent or delay onset of acute phase manifestations, thereby preventing or delaying disability or death. We also examined interventions at the health care system and patient/provider levels. Our team populated the four domains of this two-by-two framework a priori with potential quality improvement levers, based on our prior knowledge of the field and examples gleaned from other studies, including other DCP3 reviews.
Information Sources and Search Strategy
Two authors (EL and AEM) designed an electronic search of MEDLINE and EMBASE electronic databases for the purpose of capturing published reports of CVD care quality improvement studies carried out in LMICs and published in English between January 2000 and June 2014. Pertinent conditions selected for the search were myocardial infarction, acute coronary syndrome (ACS), hypertension, cardiovascular disease, diabetes, stroke, cerebrovascular disease, obesity, CHF, and rheumatic heart disease (RHD). Based on key words identified in an informal review of publications already known to the authors and a survey of the other DCP3 chapter authors, a second set of search terms related to quality improvement was identified (S1 and S2 Tables). Study designs were limited to controlled clinical trials, comparison studies, systematic reviews, and meta-analyses. Studies were limited to those published in peer-reviewed journals or in reports from institutes with rigorous internal reviews. The electronic search was refined and structured by a research librarian. Search results were considered valid if all of five key papers identified a priori by coauthors’ review were retrieved by the electronic search[3–7]. The review adhered to PRISMA recommendations for the conduct and reporting of systematic reviews (S1 Fig).
Eligibility Criteria and Study Selection
Two authors (EL and AEM) independently assessed identified studies for eligibility based on the title and abstract. Selected full text papers were then independently reviewed by both authors. Eligibility criteria included: 1) LMIC setting; 2) patient/provider or system level intervention aimed at improving CVD quality of care; 3) outcome included risk factor or disease screening, case-finding, clinic-based CVD prevention (medication adherence, clinic visits, vital signs, laboratory results, mortality), behavioral changes (diet or exercise), improved access to medical care, or clinical guideline implementation; and 4) study design involved a comparison between a clinical quality improvement intervention and usual care, regardless of whether the intervention group served as its own control (“before and after” study) or a separate control group was used.
Data Collection/Data Items/Summary Measures
Data were abstracted and each study was categorized into one or more quadrants of the chapter rubric (Fig 1). The following information was extracted from each study: study characteristics (setting, targeted disease, study design, participant number and baseline characteristics), type of intervention, observation interval, and outcome measures. The main summary measures were differences in means, proportions of the outcome, and outcome risk reduction. Three study types were observed most frequently and were therefore identified for potential pooling in a quantitative analysis: task-shifting in hypertension control, mobile health in diabetes management, and ACS clinical pathway interventions. Four out of eight task-shifting hypertension control studies did not report variance around the most common main effect estimate—systolic blood pressure (SBP) change [8–15]. Only three out of six studies analyzing mobile health in diabetes management reported variance around mean hemoglobin A1c (HbA1c) change [16–21]. Too few studies reported on changes in fasting plasma glucose or percent adherence to medications. ACS clinical pathways studies did not consistently report on the same quality improvement metrics [3, 5, 22]. Because of heterogeneity of outcomes and lack of reporting on measures of dispersion about main effect estimates (variance), the authors were unable to perform a quantitative analysis.
Study Selection and Characteristics
The search yielded 847 unique references and all five key papers identified a priori were captured. Based on a review of titles and abstracts, 273 potentially eligible papers were identified. Of these, two authors (EL and AEM) concurred that 49 papers reported population-based studies with clinically meaningful outcomes. The papers were selected for the detailed review (Fig 2). Table 1 presents a summary of study characteristics. All 49 studies are reported in Tables 2–5.
System level-acute phase CVD
Three studies examined interventions in the system level-acute phase category[22–24] (Table 2). Alexander et al. reported on a project being launched in a rural region within Tamil Nadu state, India that plans to implement a “hub-and-spoke” model using existing health care facility and ambulance transportation resources to improve the acute ST-elevation myocardial infarction care delivery system. Another multi-component intervention study, the Kerala ACS program included community-based health education programs that promoted self-detection of acute coronary disease symptoms, rapid self-referral for treatment, and timely self-administration of aspirin . The investigators concluded that improved patient awareness of ACS symptoms and treatments contributed to reduced time-to-thrombolysis achieved by the multi-component intervention. We found no studies assessing the impact of improved geographical and temporal coverage of acute care services, including the impact of building more hospitals within under-served areas, expanding service hours to include more overnight capacity, or making revascularization more widely available within the health system.
System level-chronic phase CVD
Most studies in the system level-chronic phase category studied expansion of health insurance coverage [25–32] (Table 3). Two studies evaluated the health impact of Seguro Popular insurance rolled out in 2002 as part of Mexico’s national universal health insurance plan. Seguro Popular covered approximately 50 million low-income people who previously had no formal health insurance–often because working family members participated in the informal economy . Based on data gathered in Mexican national health and nutrition surveys during step-wise introduction of insurance, Bleich et al. found that compared with matched hypertensive adults without insurance, hypertensive Seguro Popular enrollees had 1.5-fold higher odds of receiving hypertension treatment and 1.4-fold higher odds of having controlled blood pressure (BP) . A similar study of low-income diabetic patients found those with Seguro Popular insurance were more likely to receive regular blood glucose control monitoring and maintain adequate glucose control compared to their matched, uninsured counterparts .
In rural Nigeria, hypertensive patients living in a district where community-based health insurance was available had significantly lower SBP and diastolic blood pressure (DBP) not observed in the control group without insurance . In rural China, hypertensive patients receiving subsidies to defer medication costs had a 9% absolute increase in medication adherence and significantly lower annual out-of-pocket medical costs .
Patient/provider level-acute phase CVD
Most studies in the patient/provider level–acute phase category studied ACS clinical pathways[3, 5, 22, 33] (Table 4). Berwanger et al. randomized large urban hospitals in Brazil to either a multifaceted quality improvement program with educational materials, reminders, treatment algorithms and training visits or to usual care. The intervention group had 2.64 higher odds (95% confidence interval 1.28 to 5.45) of receiving evidence-based ACS treatment within the first 24 hours following symptom onset. However, there was no difference observed in 30-day mortality or rate of in-hospital repeat cardiovascular events.
In urban China, Du et al. randomized hospitals to implementation of a United States guideline-based ACS pathway, along with interval clinical performance audits and feedback throughout the intervention period and compared the results of these interventions to hospitals continuing usual care practices. Intervention hospitals had higher rates of discharge on recommended therapies compared to usual care hospitals, but no difference was observed in other indicators, including reperfusion within 12 hours of symptom onset in ST-elevation myocardial infarction cases, door-to-needle time, door-to-balloon time, or proportion of high-risk patients undergoing angiography. Similar to the Berwanger et al. study, there was no significant difference in mortality or cardiovascular morbidity events. In a third study, Prabhakaran et al. enrolled 34 hospitals in the Kerala region of India to serve as their own controls in a pre- and post-intervention comparison design. After the multi-faceted quality improvement intervention, there was a significant median reduction of 54 minutes in time-to-thrombolysis from 193.0 to 139.0 minutes and a significant increase in utilization of evidence-based medications.
Patient/provider level-chronic phase CVD
The majority of selected studies (33/49) were in the patient/provider level-chronic phase category [8–21, 35–54] (Table 5). CVD patients are often prescribed several medications for primary or secondary prevention, making medication adherence a key challenge to delivering quality CVD clinical care worldwide. Thus most interventions were related to chronic medication adherence. Types of medication adherence interventions included use of fixed-dose combination pills, “mobile” health care, task shifting, clinical guideline implementation, and patient education.
Improving patients’ medication adherence through combination pills.
Thom et al. randomized high-risk CVD patients in India (one of four countries studied) to receive combination pills or the usual, multiple-pill therapy. After a mean follow-up of about 15 months, Indian participants randomized to combination pills had a 25% higher absolute adherence compared to those randomized to usual care (89.1% vs 63.6%, unadjusted relative risk 1.4, 95% confidence interval 1.30 to 1.51, p<0.001) and small but significant reductions in both SBP and low-density lipoprotein (LDL) compared with participants randomized to multiple-pill treatments . In another study, Yusuf et al. also found small but significant improvement in adherence among high CVD risk Indian subjects randomized to a combination pill containing multiple BP medications, statins, and aspirin . A follow-up study showed that high-dose combination pills improved BP and lipid control in high CVD risk Indian subjects compared to low-dose combination pills, while both formulations had similar rates of tolerability . Zou et al. found that high CVD risk rural Chinese participants achieved an 11% higher absolute adherence rate after they were offered combination pills in a pre- and post-intervention comparison design (84% vs 73%, p = 0.000)
Improving patients’ medication adherence through mobile health interventions.
Twelve studies investigated the role of mobile health in prevention and management of CHF, diabetes, hypertension, and coronary artery disease in LMICs [16–21, 36–39, 41]. Chronic CHF patients in Argentina were randomized to a telemedicine intervention in which nurses called patients to adjust medications depending on their symptoms and provided counseling and education or to usual care. Patients in the intervention arm had a 20% relative risk reduction (RR = 0.80, 95% confidence interval 0.66 to 0.97, p = 0.026) in the primary composite outcome of all-cause mortality and CHF hospitalizations, driven mostly by the latter outcome component. A follow-up study showed that even after three years, CHF hospitalizations, medication adherence rates, and quality of life scores had improved significantly more in the intervention group . Similarly, Nesari et al. showed that Iranian diabetic patients randomized to a nurse telephone follow-up intervention had significant improvements in their HbA1c and adherence to healthy lifestyle changes compared to those randomized to usual care .
Four studies evaluated the effects of automated phone texts on diabetes and coronary artery disease prevention. Ramachandran et al. and Shetty et al. found that automated SMS (short message service) messages led to improved glycemic control in Indians with pre-diabetes and diabetes[17, 18]. Goodarzi et al. showed an absolute decrease in HbA1c in Iranian diabetic subjects who received automated SMS messages. Khonsari et al. revealed that automated SMS messages led to higher medication adherence rates in Malaysian patients after hospital discharge following ACS . It is important to note that all four studies only included participants with regular access to mobile phones with text receiving capabilities.
Task shifting to increase patients’ medication adherence.
Task shifting refers to the redistribution of tasks among health workforce teams, often from a few, highly trained health providers to a larger contingent of providers with less formal health care training. Our review identified six studies evaluating task shifting for improving patient adherence to prescribed medications [11–15, 42]. Some studies coupled task shifting with increased access to affordable or free medications [12, 13] or family-based home health education and supplemental training of general practitioners .
Five task-shifting studies targeted hypertensive patients[11–15]. Regardless of the approach, intensified, team-based care led to improved hypertension control. Kengne et al. carried out a large trial of hypertensive participants enrolled in a nurse-led clinic in Cameroon. Erhun et al. evaluated the role of pharmacist-led clinics in Nigerian patients with hypertension. Adeyemo et al. randomized Nigerian participants with hypertension to clinic-based care with home visits or to clinic-based care only. Jafar et al. was a cluster-randomized, controlled trial of two different interventions, home health education by health aides and training of general practitioners, in a population of Pakistani patients with hypertension.
Two studies focused on task shifting interventions in diabetic patients [16, 42]. Nesari et al., as mentioned above, showed that having nurses call patients regularly to reinforce lifestyle changes and adjust medication doses led to a significant decrease in HbA1c. The intervention group also showed increased rates of adherence to lifestyle changes and glucose monitoring.
Guideline implementation and provider education leading to improved patient outcomes.
Health care provider education and guideline implementation have the potential to standardize, improve, and sustain quality of care for vascular and other conditions in LMICs. Studies of the impact of physician education and guideline dissemination yielded mixed results[43–47]. Qureshi et al. showed that physician education through workshops and guideline dissemination led to significant improvements in quality of hypertension management. Anchala et al. revealed that providing physicians with a clinical decision support system to undertake guideline-based hypertension management led to significant reductions in SBP. Mendis et al. showed that dissemination of an evidence-based cardiovascular risk management package led to significant reductions in SBP in both Chinese and Nigerian primary care clinics compared to usual care (China: 13.3 vs 9.4mm Hg, p<0.0001; Nigeria: 11.0 vs 6.6mm Hg, p = 0.0002) . However, Reutens et al. and Steyn et al. showed conflicting results and highlighted the fact that guideline dissemination alone did not lead to implementation or improve health outcomes in South African clinics.
Educating patients to enhance health outcomes.
The review found eight studies that evaluated disease management education among patients with hypertension, diabetes, and CHF. [48–54]. Alhalaiqa et al. and Mohammadi et al. showed that hypertensive patients in Jordan and Iran, respectively, who were randomized to medication adherence education and motivational group meetings had improved SBP and medication adherence compared to those participants in the control groups. Likewise, Tan et al. revealed that Malaysian patients with poorly controlled diabetes who were randomized to monthly diabetes education programs resulted in 0.9% absolute reduction in HbA1c (8.75 vs 9.67%, p<0.001), changes not observed in the control group . Bocchi et al. reported Brazilian CHF patients randomized to disease management programs had a delay in median follow-up time to first hospitalization by 2.2 years (4.21 vs 2.06 years, p = 0.0041) compared to those in the control group, though mortality was similar in both groups . Conversely, Andryukhin et al. did not find that disease management led to decreased hospital admissions or mortality, but did lead to improved functional status and quality of life.
Our systematic review identified 49 original studies of CVD care quality improvement in LMICs. Taken together, the evidence suggests that quality improvement programs have the potential to improve CVD care in LMICs, whether for treating acute events or in long-term prevention and chronic disease management. Not all approaches reviewed were consistently effective, and we did not find evidence supporting all the approaches we intended to assess a priori (Figs 3 and 4).
Types of interventions targeting three levels of acute phase CVD prevention and management. Bullets in bold type are supported by evidence from the review. Bullets without bold type indicate that no supporting evidence was found in the review and are potential areas for further research.
Types of interventions targeting three levels of chronic phase CVD prevention and management. Bullets in bold are supported by evidence from the review. Bullet without bold type indicate that no supporting evidence was found in the review and are potential areas for further research.
System level-acute phase CVD
Our systematic review found limited evidence in the acute phase-system level category. Perhaps poor underlying infrastructure in low resource settings presents daunting challenges to re-organize complex health delivery systems . Just as likely, governments, non-governmental, and private sector organizations often introduce system improvements without rigorous systematic study. Therefore, health effects of system level changes may go unmeasured or unreported. Conducting randomized comparison studies in low- and middle-income settings may be limited by lack of research capacity, perception of causing unwanted delay in care delivery, “contamination” between intervention and control sites, and ethical concerns. The lack of evidence is unfortunate because timely intervention can dramatically improve acute CVD outcomes, and delays may result in unnecessary death or disability. Awareness, acceptability, affordability, and availability are all common barriers that can delay treatment of acute events. Community education and system-wide infrastructure, logistics, and human resource planning can overcome barriers to timely and appropriate care for acute CVD. Integration of private sector ambulance companies, hospitals, and providers is an important part of achieving improved quality of acute CVD care in LMICs, where the private sector provides a large proportion of health care and may be in the position to efficiently fill gaps in public health care systems .
System level-chronic phase CVD
Unfortunately, many CVD patients in LMICs remain untreated or incompletely treated with standard oral medications for secondary prevention . Our review suggests that system level quality improvement efforts can lead to measurable improvements in health status in LMIC patients with chronic CVDs. Implementation of universal health insurance led to significant improvements in chronic disease management in Latin America, Africa, and Asia [25–28]. Our review captured no studies of the impact of “essential medicines” designations and price controls or pharmaceutical market regulation on the quality of CVD clinical care.
In sub-Saharan Africa, the substantial infrastructure investment that turned the tide of the HIV epidemic is now being leveraged for chronic non-communicable disease management. Groups like the Kenya-based Academic Model Providing Access to Healthcare (AMPATH) have leveraged this health system platform to build a comprehensive, multi-level approach to quality improvement in order to improve cardiovascular care in the communities they serve[60, 61]. Results of this multi-level intervention are expected soon.
System level interventions have been systematically evaluated through pre- and post-intervention “natural experiment” comparisons, observational comparisons of intervention participants and matched usual care participants, or comparisons of geographic areas randomly assigned to receive or not receive the intervention. Stepped-wedge trials introduce interventions to couple step-wise active and systematic program implementation with evaluation . As in the Mexican Seguro Popular studies, repeated population-based surveys (national census or health surveys) can be leveraged to measure changes in chronic CVD risk factors and outcomes. Quality improvement studies will be most feasible where key outcomes are part of, or added to, ongoing surveys.
Patient/provider–acute phase CVD
Adopting clinical practice guidelines from high-income countries to LMICs appears to be a great opportunity both for implementing quality improvement standards and benchmarking significant practice and outcome improvements. The ACS and stroke quality improvement studies identified in our review showed some clinical process measure improvements, but like high-income region studies, quality did not consistently improve for all pre-specified outcomes [3, 5, 34]. Success of acute phase CVD clinical pathway interventions likely depends on the support of health care providers and administrators, and tailoring to the specific context of the participating health care system (i.e., availability of treatments and financial protection for patients) . Lessons learned from these programs may be helpful for the design of future acute, patient/provider level CVD quality improvement studies. Despite their limitations, these ambitious studies demonstrated that large-scale and complex quality improvement programs can be implemented in LMIC hospitals. Absent from our review were studies measuring the impact of physician education on diagnostic accuracy or clinical decision-making related to acute cardiovascular disorders.
Patient/provider–chronic phase CVD
Adherence to life saving medications and lifestyle changes is suboptimal world-wide, regardless of country income level . The most consistent general finding among selected LMIC studies was that increased intensity of care improved chronic disease and risk factor outcomes, regardless of how or through whom it was delivered. While care intensification inevitably requires up-front investment, this investment may be offset by improved downstream CVD outcomes, health gains, and cost-savings. A modeling study by Gaziano et al. projected that despite added costs of hiring community health workers to manage hypertension in South Africa, the intervention would be cost-effective due to health gains and cost savings from prevented hospital admissions and chronic disease complications . The trials reviewed also suggest that combination pills have the potential to improve medication adherence and improve risk factor control in high-risk patients without necessarily adding extra costs. Efforts to efficiently manufacture and gain regulatory approval for combination medications are underway .
Rapidly increasing access to mobile telecommunications technology may also be a promising tool for chronic disease management. There are now almost seven billion mobile phone users world-wide with more than three-quarters of these users living in LMICs . A 2012 Cochrane review cautiously concluded that there was some limited evidence that mobile phone messaging may be beneficial in self-management of chronic diseases . This was based on evidence from four studies, all performed within high-income countries. We found a number of studies in LMICs that successfully leveraged the ubiquity and low cost of mobile technology to improve CHF, hypertension, and diabetes care outcomes.
We found that efforts to improve quality care through physician education and guideline dissemination yielded inconsistent results [43, 44, 46, 47]. Guideline dissemination did not lead to actual implementation in all cases. Gaining buy-in from providers appears to be important for success. Allocating time for education and feedback and strategically inserting guideline information into the flow of clinical practice may increase the chance that guidelines are actually implemented.
Our review and the studies we reviewed had limitations, so our results should be interpreted with caution. Due to the diversity of interventions and conditions, we were unable to summarize effect sizes in a meta-analysis. It is possible that most of our included studies were published because of “positive” results—due to the heterogeneity in interventions and targets, we were not able to formally evaluate the results of our review for evidence of publication bias. Also important is that most study durations were relatively short term (<12 months) and sustaining intervention effects may be a big challenge when these programs are implemented in real clinical settings (Fig 5). As in high-income countries, most investigators studied clinical process measures, and did not report on hard clinical outcomes, which may be more vulnerable to “gaming” the system to produce only nominal improvements and other unintended consequences when these interventions are introduced into routine practice.
We identified good evidence that short-term quality improvement interventions improved the quality of cardiovascular care in low- and middle-income countries, especially for chronic conditions delivered at the patient/provider level. Most studies were short-term. At the patient/provider level, intensified, team-based care generally led to improved medication adherence and hypertension control. At the system level, studies provided evidence that introduction of universal health insurance coverage improved hypertension and diabetes control. Studies of system and patient/provider level ACS quality improvement interventions yielded inconclusive results.
What is needed now is more evidence that quality improvement can be sustained in the long term while remaining effective and affordable. Collecting data elements common to implementation research such as acceptability, sustainability, local context, and affordability and collecting and assessing qualitative and process data will help ensure that both positive and negative studies will guide implementation and future research.
The majority of CVD patients now live in LMICs, and demographic trends virtually guarantee that the number and proportion will grow in coming decades. In order to ensure that each of these patients receives high quality preventive and acute care, we will need to draw on promising research on clinical quality improvement in order to make the most of the resources directly at hand.
S1 Fig. PRISMA Checklist.
S1 Table. Search terms on PubMed (June 9, 2014).
S2 Table. Search terms on EMBASE (June 9, 2014).
We gratefully acknowledge Kristen Danforth and Vittoria Lutje for invaluable assistance with electronic review design and extraction of identifying data for the reviewed papers. We are grateful for Shuchi Anand’s coordination of the efforts of all contributors to the DCP3 Cardiometabolic and Respiratory Disease Volume chapters. We would also like to thank DCP3 Study members Rachel Nugent, John Peabody, and Riti Shimkhada for their helpful comments on earlier drafts of the paper.
Conceived and designed the experiments: ESL AEM VR PJ PK ME. Performed the experiments: ESL AEM. Analyzed the data: ESL AEM. Contributed reagents/materials/analysis tools: ESL AEM RV JHK. Wrote the paper: ESL AEM ME RV.
- 1. Roth GA, Huffman MD, Moran AE, Feigin V, Mensah GA, Naghavi M, et al. Global and Regional Patterns in Cardiovascular Mortality From 1990 to 2013. Circulation. 2015;132(17):1667–78. doi: 10.1161/circulationaha.114.008720. pmid:26503749
- 2. Institute of Medicine Committee to Design a Strategy for Quality R, Assurance in M. In: Lohr KN, editor. Medicare: A Strategy for Quality Assurance: Volume 1. Washington (DC): National Academies Press (US) Copyright (c) 1990 by the National Academy of Sciences.; 1990.
- 3. Du X, Gao R, Turnbull F, Wu Y, Rong Y, Lo S, et al. Hospital Quality Improvement Initiative for Patients With Acute Coronary Syndromes in China: A Cluster Randomized, Controlled Trial. Circulation: Cardiovascular Quality and Outcomes. 2014;7(2):217–26. doi: 10.1161/circoutcomes.113.000526. pmid:24619325
- 4. Glickman SW, Ou F, DeLong ER, et al. PAy for performance, quality of care, and outcomes in acute myocardial infarction. Jama. 2007;297(21):2373–80. doi: 10.1001/jama.297.21.2373. pmid:17551130
- 5. Berwanger O, Guimarães HP, Laranjeira LN, Cavalcanti AB, Kodama AA, Zazula AD, et al. Effect of a Multifaceted Intervention on Use of Evidence-Based Therapies in Patients With Acute Coronary Syndromes in Brazil. Jama. 2012;307(19). doi: 10.1001/jama.2012.413.
- 6. Galarraga O. Diabetes treatment and control: the effect of public health insurance for the poor in Mexico. Bulletin of the World Health Organization. 2009;87(7):512–9. doi: 10.2471/blt.08.053256. pmid:19649365
- 7. Walker RW, Jusabani A, Aris E, Gray WK, Unwin N, Swai M, et al. Stroke risk factors in an incident population in urban and rural Tanzania: a prospective, community-based, case-control study. The Lancet Global Health. 2013;1(5):e282–e8. doi: 10.1016/s2214-109x(13)70068-8. pmid:24748275
- 8. Thom S. Effects of a Fixed-Dose Combination Strategy on Adherence and Risk Factors in Patients With or at High Risk of CVD. Jama. 2013;310(9):918. doi: 10.1001/jama.2013.277064. pmid:24002278
- 9. Yusuf S, Pais P, Afzal R, Xavier D, Teo K, Eikelboom J, et al. Effects of a polypill (Polycap) on risk factors in middle-aged individuals without cardiovascular disease (TIPS): a phase II, double-blind, randomised trial. Lancet. 2009;373(9672):1341–51. Epub 2009/04/03. doi: 10.1016/s0140-6736(09)60611-5 pmid:19339045.
- 10. Zou G, Wei X, Gong W, Yin J, Walley J, Yu Y, et al. Evaluation of a systematic cardiovascular disease risk reduction strategy in primary healthcare: an exploratory study from Zhejiang, China. Journal of public health. 2014. Epub 2014/04/04. doi: 10.1093/pubmed/fdu013 pmid:24696086.
- 11. Ramanath K, Balaji D, Nagakishore C, Kumar SM, Bhanuprakash M. A study on impact of clinical pharmacist interventions on medication adherence and quality of life in rural hypertensive patients. Journal of young pharmacists: JYP. 2012;4(2):95–100. Epub 2012/07/04. doi: 10.4103/0975-1483.96623 pmid:22754261; PubMed Central PMCID: PMC3385224.
- 12. Kengne AP, Awah PK, Fezeu LL, Sobngwi E, Mbanya J-C. Primary Health Care for Hypertension by Nurses in Rural and Urban Sub-Saharan Africa. The Journal of Clinical Hypertension. 2009;11(10):564–72. doi: 10.1111/j.1751-7176.2009.00165.x. pmid:19817937
- 13. Erhun WO, Agbani EO, Bolaji EE. Positive benefits of a pharmacist-managed hypertension clinic in Nigeria. Public health. 2005;119(9):792–8. Epub 2005/07/02. doi: 10.1016/j.puhe.2004.11.009 pmid:15990127.
- 14. Adeyemo A, Tayo BO, Luke A, Ogedegbe O, Durazo-Arvizu R, Cooper RS. The Nigerian antihypertensive adherence trial: a community-based randomized trial. Journal of hypertension. 2013;31(1):201–7. Epub 2012/11/10. doi: 10.1097/HJH.0b013e32835b0842 pmid:23137954; PubMed Central PMCID: PMC3530610.
- 15. Jafar TH, Hatcher J, Poulter N, Islam M, Hashmi S, Qadri Z, et al. Community-based interventions to promote blood pressure control in a developing country: a cluster randomized trial. Annals of internal medicine. 2009;151(9):593–601. Epub 2009/11/04. doi: 10.7326/0003-4819-151-9-200911030-00004 pmid:19884620.
- 16. Nesari M, Zakerimoghadam M, Rajab A, Bassampour S, Faghihzadeh S. Effect of telephone follow-up on adherence to a diabetes therapeutic regimen. Japan journal of nursing science: JJNS. 2010;7(2):121–8. Epub 2010/11/26. doi: 10.1111/j.1742-7924.2010.00146.x pmid:21092015.
- 17. Ramachandran A, Snehalatha C, Ram J, Selvam S, Simon M, Nanditha A, et al. Effectiveness of mobile phone messaging in prevention of type 2 diabetes by lifestyle modification in men in India: a prospective, parallel-group, randomised controlled trial. The Lancet Diabetes & Endocrinology. 2013;1(3):191–8. doi: 10.1016/s2213-8587(13)70067-6.
- 18. Shetty AS, Chamukuttan S, Nanditha A, Raj RK, Ramachandran A. Reinforcement of adherence to prescription recommendations in Asian Indian diabetes patients using short message service (SMS)—a pilot study. The Journal of the Association of Physicians of India. 2011;59:711–4. Epub 2012/05/24. pmid:22616337.
- 19. Goodarzi M, Ebrahimzadeh I, Rabi A, Saedipoor B, Jafarabadi MA. Impact of distance education via mobile phone text messaging on knowledge, attitude, practice and self efficacy of patients with type 2 diabetes mellitus in Iran. J Diabetes Metab Disord. 2012;11(1):10. Epub 2012/01/01. doi: 10.1186/2251-6581-11-10 pmid:23497632; PubMed Central PMCID: PMCPmc3598175.
- 20. Chan JC, Sui Y, Oldenburg B, Zhang Y, Chung HH, Goggins W, et al. Effects of telephone-based peer support in patients with type 2 diabetes mellitus receiving integrated care: a randomized clinical trial. JAMA internal medicine. 2014;174(6):972–81. Epub 2014/05/02. doi: 10.1001/jamainternmed.2014.655 pmid:24781960.
- 21. Rotheram-Borus MJ, Tomlinson M, Gwegwe M, Comulada WS, Kaufman N, Keim M. Diabetes buddies: peer support through a mobile phone buddy system. The Diabetes educator. 2012;38(3):357–65. Epub 2012/05/02. doi: 10.1177/0145721712444617 pmid:22546740; PubMed Central PMCID: PMCPmc4059372.
- 22. Prabhakaran D, Jeemon P, Mohanan PP, Govindan U, Geevar Z, Chaturvedi V, et al. Management of acute coronary syndromes in secondary care settings in Kerala: impact of a quality improvement programme. The National medical journal of India. 2008;21(3):107–11. Epub 2008/11/14. pmid:19004139.
- 23. Alexander T, Victor SM, Mullasari AS, Veerasekar G, Subramaniam K, Nallamothu BK, et al. Protocol for a prospective, controlled study of assertive and timely reperfusion for patients with ST-segment elevation myocardial infarction in Tamil Nadu: the TN-STEMI programme. BMJ open. 2013;3(12):e003850. Epub 2013/12/05. doi: 10.1136/bmjopen-2013-003850 pmid:24302505; PubMed Central PMCID: PMC3855601.
- 24. Nazzal NC, Campos TP, Corbalan HR, Lanas ZF, Bartolucci JJ, Sanhueza CP, et al. [The impact of Chilean health reform in the management and mortality of ST elevation myocardial infarction (STEMI) in Chilean hospitals]. Revista medica de Chile. 2008;136(10):1231–9. Epub 2009/02/06. /S0034-98872008001000001. pmid:19194618.
- 25. Bleich SN, Cutler DM, Adams AS, Lozano R, Murray CJ. Impact of insurance and supply of health professionals on coverage of treatment for hypertension in Mexico: population based study. Bmj. 2007;335(7625):875. Epub 2007/10/24. doi: 10.1136/bmj.39350.617616.BE pmid:17954519; PubMed Central PMCID: PMC2043407.
- 26. Sosa-Rubi SG, Galarraga O, Lopez-Ridaura R. Diabetes treatment and control: the effect of public health insurance for the poor in Mexico. Bulletin of the World Health Organization. 2009;87(7):512–9. pmid:19649365; PubMed Central PMCID: PMC2704037.
- 27. Hendriks ME, Wit FW, Akande TM, Kramer B, Osagbemi GK, Tanovic Z, et al. Effect of health insurance and facility quality improvement on blood pressure in adults with hypertension in Nigeria: a population-based study. JAMA internal medicine. 2014;174(4):555–63. Epub 2014/02/19. doi: 10.1001/jamainternmed.2013.14458 pmid:24534947.
- 28. Yu B, Zhang X, Wang G. Full coverage for hypertension drugs in rural communities in China. The American journal of managed care. 2013;19(1):e22–9. Epub 2013/02/06. pmid:23379776.
- 29. Gaziano TA, Pandya A, Steyn K, Levitt N, Mollentze W, Joubert G, et al. Comparative assessment of absolute cardiovascular disease risk characterization from non-laboratory-based risk assessment in South African populations. BMC medicine. 2013;11:170. Epub 2013/07/25. doi: 10.1186/1741-7015-11-170 pmid:23880010; PubMed Central PMCID: PMC3734109.
- 30. Beaton A, Okello E, Lwabi P, Mondo C, McCarter R, Sable C. Echocardiography screening for rheumatic heart disease in Ugandan schoolchildren. Circulation. 2012;125(25):3127–32. Epub 2012/05/26. doi: 10.1161/CIRCULATIONAHA.112.092312 pmid:22626741.
- 31. Carapetis JR, Hardy M, Fakakovikaetau T, Taib R, Wilkinson L, Penny DJ, et al. Evaluation of a screening protocol using auscultation and portable echocardiography to detect asymptomatic rheumatic heart disease in Tongan schoolchildren. Nature clinical practice Cardiovascular medicine. 2008;5(7):411–7. Epub 2008/04/10. doi: 10.1038/ncpcardio1185 pmid:18398402.
- 32. Trujillo AJ, Vecino Ortiz AI, Ruiz Gomez F, Steinhardt LC. Health insurance doesn't seem to discourage prevention among diabetes patients in Colombia. Health affairs. 2010;29(12):2180–8. Epub 2010/12/08. doi: 10.1377/hlthaff.2010.0463 pmid:21134918.
- 33. Flather MD, Babalis D, Booth J, Bardaji A, Machecourt J, Opolski G, et al. Cluster-randomized trial to evaluate the effects of a quality improvement program on management of non-ST-elevation acute coronary syndromes: The European Quality Improvement Programme for Acute Coronary Syndromes (EQUIP-ACS). American heart journal. 2011;162(4):700–7 e1. Epub 2011/10/11. doi: 10.1016/j.ahj.2011.07.027 pmid:21982663.
- 34. Peng B, Ni J, Anderson CS, Zhu Y, Wang Y, Pu C, et al. Implementation of a structured guideline-based program for the secondary prevention of ischemic stroke in China. Stroke; a journal of cerebral circulation. 2014;45(2):515–9. Epub 2014/01/05. doi: 10.1161/STROKEAHA.113.001424 pmid:24385269.
- 35. Yusuf S, Pais P, Sigamani A, Xavier D, Afzal R, Gao P, et al. Comparison of risk factor reduction and tolerability of a full-dose polypill (with potassium) versus low-dose polypill (polycap) in individuals at high risk of cardiovascular diseases: the Second Indian Polycap Study (TIPS-2) investigators. Circulation Cardiovascular quality and outcomes. 2012;5(4):463–71. Epub 2012/07/13. doi: 10.1161/circoutcomes.111.963637 pmid:22787067.
- 36. Investigators G. Randomised trial of telephone intervention in chronic heart failure: DIAL trial. Bmj. 2005;331(7514):425. Epub 2005/08/03. doi: 10.1136/bmj.38516.398067.E0 pmid:16061499; PubMed Central PMCID: PMC1188105.
- 37. Ferrante D, Varini S, Macchia A, Soifer S, Badra R, Nul D, et al. Long-term results after a telephone intervention in chronic heart failure: DIAL (Randomized Trial of Phone Intervention in Chronic Heart Failure) follow-up. Journal of the American College of Cardiology. 2010;56(5):372–8. Epub 2010/07/24. doi: 10.1016/j.jacc.2010.03.049 pmid:20650358.
- 38. Khonsari S, Subramanian P, Chinna K, Latif LA, Ling LW, Gholami O. Effect of a reminder system using an automated short message service on medication adherence following acute coronary syndrome. European journal of cardiovascular nursing: journal of the Working Group on Cardiovascular Nursing of the European Society of Cardiology. 2014. Epub 2014/02/05. doi: 10.1177/1474515114521910 pmid:24491349.
- 39. Ortega KC, Gusmão JLd, Pierin AMG, Nishiura JL, Ignez EC, Segre CA, et al. How to avoid discontinuation of antihypertensive treatment: The experience in São Paulo, Brazil. Clinics. 2010;65(9):857–63. doi: 10.1590/s1807-59322010000900008. pmid:21049213
- 40. Piette JD, Datwani H, Gaudioso S, Foster SM, Westphal J, Perry W, et al. Hypertension Management Using Mobile Technology and Home Blood Pressure Monitoring: Results of a Randomized Trial in Two Low/Middle-Income Countries. Telemedicine and e-Health. 2012;18(8):613–20. doi: 10.1089/tmj.2011.0271. pmid:23061642
- 41. Labhardt ND, Balo J-R, Ndam M, Manga E, Stoll B. Improved retention rates with low-cost interventions in hypertension and diabetes management in a rural African environment of nurse-led care: a cluster-randomised trial. Tropical Medicine & International Health. 2011;16(10):1276–84. doi: 10.1111/j.1365-3156.2011.02827.x.
- 42. Obreli-Neto PR, Guidoni CM, de Oliveira Baldoni A, Pilger D, Cruciol-Souza JM, Gaeti-Franco WP, et al. Effect of a 36-month pharmaceutical care program on pharmacotherapy adherence in elderly diabetic and hypertensive patients. International journal of clinical pharmacy. 2011;33(4):642–9. Epub 2011/05/06. doi: 10.1007/s11096-011-9518-x pmid:21544559.
- 43. Qureshi NN, Hatcher J, Chaturvedi N, Jafar TH, Hypertension Research G. Effect of general practitioner education on adherence to antihypertensive drugs: cluster randomised controlled trial. Bmj. 2007;335(7628):1030. Epub 2007/11/10. doi: 10.1136/bmj.39360.617986.AE pmid:17991935; PubMed Central PMCID: PMC2078673.
- 44. Anchala R, Kaptoge S, Pant H, Di Angelantonio E, Franco OH, Prabhakaran D. Evaluation of effectiveness and cost-effectiveness of a clinical decision support system in managing hypertension in resource constrained primary health care settings: results from a cluster randomized trial. Journal of the American Heart Association. 2015;4(1):e001213. Epub 2015/01/07. doi: 10.1161/jaha.114.001213 pmid:25559011; PubMed Central PMCID: PMCPmc4330052.
- 45. Mendis S, Johnston SC, Fan W, Oladapo O, Cameron A, Faramawi MF. Cardiovascular risk management and its impact on hypertension control in primary care in low-resource settings: a cluster-randomized trial. Bulletin of the World Health Organization. 2010;88(6):412–9. Epub 2010/06/12. doi: 10.2471/BLT.08.062364 pmid:20539854; PubMed Central PMCID: PMC2878142.
- 46. Reutens AT, Hutchinson R, Van Binh T, Cockram C, Deerochanawong C, Ho L-T, et al. The GIANT study, a cluster-randomised controlled trial of efficacy of education of doctors about type 2 diabetes mellitus management guidelines in primary care practice. Diabetes Research and Clinical Practice. 2012;98(1):38–45. doi: 10.1016/j.diabres.2012.06.002. pmid:22784926
- 47. Steyn K, Lombard C, Gwebushe N, Fourie JM, Everett-Murphy K, Zwarenstein M, et al. Implementation of national guidelines, incorporated within structured diabetes and hypertension records at primary level care in Cape Town, South Africa: a randomised controlled trial. Global health action. 2013;6:20796. Epub 2013/09/28. doi: 10.3402/gha.v6i0.20796 pmid:24070181; PubMed Central PMCID: PMC3784670.
- 48. Alhalaiqa F, Deane KH, Nawafleh AH, Clark A, Gray R. Adherence therapy for medication non-compliant patients with hypertension: a randomised controlled trial. Journal of human hypertension. 2012;26(2):117–26. Epub 2011/02/18. doi: 10.1038/jhh.2010.133 pmid:21326328; PubMed Central PMCID: PMC3257548.
- 49. Mohammadi E, Abedi HA, Jalali F, Gofranipour F, Kazemnejad A. Evaluation of 'partnership care model' in the control of hypertension. International journal of nursing practice. 2006;12(3):153–9. Epub 2006/05/06. doi: 10.1111/j.1440-172X.2006.00563.x pmid:16674782.
- 50. Ahmadipour H, Farajzadegan Z, Kachoei A, Pirdehghan A. Secondary prevention by enhancing adherence in diabetic patients. Int J Prev Med. 2010;1(1):50–5. Epub 2010/01/01. pmid:21677766; PubMed Central PMCID: PMCPmc3075488.
- 51. Tan MY, Magarey JM, Chee SS, Lee LF, Tan MH. A brief structured education programme enhances self-care practices and improves glycaemic control in Malaysians with poorly controlled diabetes. Health education research. 2011;26(5):896–907. Epub 2011/07/01. doi: 10.1093/her/cyr047 pmid:21715653.
- 52. Arevian M. The significance of a collaborative practice model in delivering care to chronically ill patients: a case study of managing diabetes mellitus in a primary health care center. Journal of interprofessional care. 2005;19(5):444–51. Epub 2005/11/26. doi: 10.1080/13561820500215095 pmid:16308168.
- 53. Andryukhin A, Frolova E, Vaes B, Degryse J. The impact of a nurse-led care programme on events and physical and psychosocial parameters in patients with heart failure with preserved ejection fraction: a randomized clinical trial in primary care in Russia. The European journal of general practice. 2010;16(4):205–14. Epub 2010/11/16. doi: 10.3109/13814788.2010.527938 pmid:21073267.
- 54. Bocchi EA, Cruz F, Guimaraes G, Pinho Moreira LF, Issa VS, Ayub Ferreira SM, et al. Long-term prospective, randomized, controlled study using repetitive education at six-month intervals and monitoring for adherence in heart failure outpatients: the REMADHE trial. Circulation Heart failure. 2008;1(2):115–24. Epub 2008/07/01. doi: 10.1161/CIRCHEARTFAILURE.107.744870 pmid:19808281.
- 55. Organization WH. Task shifting: global recommendations and guidelines. 2008.
- 56. Macharia WM, Njeru EK, Muli-Musiime F, Nantulya V. Severe road traffic injuries in Kenya, quality of care and access. African health sciences. 2009;9(2):118–24. Epub 2009/08/05. pmid:19652745; PubMed Central PMCID: PMCPmc2707053.
- 57. Sekhri N, Feachem R, Ni A. Public-private integrated partnerships demonstrate the potential to improve health care access, quality, and efficiency. Health affairs. 2011;30(8):1498–507. Epub 2011/08/09. doi: 10.1377/hlthaff.2010.0461 pmid:21821566.
- 58. Yusuf S, Islam S, Chow CK, Rangarajan S, Dagenais G, Diaz R, et al. Use of secondary prevention drugs for cardiovascular disease in the community in high-income, middle-income, and low-income countries (the PURE Study): a prospective epidemiological survey. The Lancet. 2011;378(9798):1231–43. doi: 10.1016/s0140-6736(11)61215-4.
- 59. Rabkin M, Kruk ME, El-Sadr WM. HIV, aging and continuity care: strengthening health systems to support services for noncommunicable diseases in low-income countries. AIDS (London, England). 2012;26 Suppl 1:S77–83. Epub 2012/09/22. doi: 10.1097/QAD.0b013e3283558430 pmid:22781180.
- 60. Binanay CA, Akwanalo CO, Aruasa W, Barasa FA, Corey GR, Crowe S, et al. Building Sustainable Capacity for Cardiovascular Care at a Public Hospital in Western Kenya. Journal of the American College of Cardiology. 2015;66(22):2550–60. doi: http://dx.doi.org/10.1016/j.jacc.2015.09.086. pmid:26653630
- 61. Vedanthan R, Kamano JH, Bloomfield GS, Manji I, Pastakia S, Kimaiyo SN. Engaging the Entire Care Cascade in Western Kenya. Global heart. 2015;10(4):313–7. doi: 10.1016/j.gheart.2015.09.003. pmid:26704963
- 62. Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ. The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. Bmj. 2015;350:h391. Epub 2015/02/11. doi: 10.1136/bmj.h391 pmid:25662947.
- 63. Ranasinghe I, Rong Y, Du X, Wang Y, Gao R, Patel A, et al. System barriers to the evidence-based care of acute coronary syndrome patients in China: qualitative analysis. Circulation Cardiovascular quality and outcomes. 2014;7(2):209–16. Epub 2014/03/13. doi: 10.1161/CIRCOUTCOMES.113.000527 pmid:24619324.
- 64. Gaziano TA, Bertram M, Tollman SM, Hofman KJ. Hypertension education and adherence in South Africa: a cost-effectiveness analysis of community health workers. BMC public health. 2014;14:240. Epub 2014/03/13. doi: 10.1186/1471-2458-14-240 pmid:24606986; PubMed Central PMCID: PMCPmc3973979.
- 65. FDA. Briefing Document: Cardiovascular and Renal Drugs Advisory Committee Meeting. 2014.
- 66. International Telecommunications Union. Geneva S. The World in 2014: ICT Facts and Figures. 2014. Epub April 2014.
- 67. de Jongh T, Gurol-Urganci I, Vodopivec-Jamsek V, Car J, Atun R. Mobile phone messaging for facilitating self-management of long-term illnesses. The Cochrane database of systematic reviews. 2012;12:Cd007459. Epub 2012/12/14. doi: 10.1002/14651858.CD007459.pub2 pmid:23235644.
- 68. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Administration and Policy in Mental Health. 2011;38(2):65–76. doi: 10.1007/s10488-010-0319-7 pmid:PMC3068522.