Identifying the most effective essential medicines policies for quality use of medicines: A replicability study using three World Health Organisation data-sets

Background Poor quality use of medicines (QUM) has adverse outcomes. Governments’ implementation of essential medicines (EM) policies is often suboptimal and there is limited information on which policies are most effective. Methods We analysed data on policy implementation from World Health Organisation (WHO) surveys in 2007 and 2011, and QUM data from surveys during 2006–2012 in developing and transitional countries. We compared QUM scores in countries that did or did not implement specific policies and regressed QUM composite scores on the numbers of policies implemented. We compared the ranking of policies in this and two previous studies, one from the same WHO databases (2003–2007) the other from data obtained during country visits in South-East Asia (2010–2015). The rankings of a common set of 17 policies were correlated and we identified those that were consistently highly ranked. Findings Fifty-three countries had data on both QUM and policy implementation. Forty policies were associated with effect sizes ranging from +13% to -5%. There was positive correlation between the composite QUM indicator and the number of policies reported implemented: (r) = 0.437 (95% CI 0.188 to 0.632). Comparison of policy rankings between the present and previous studies showed positive correlation with the WHO 2003–7 study: Spearman’s rank correlation coefficient 0.498 (95% CI 0.022 to 0.789). Across the three studies, five policies were in the top five ranked positions 11 out of a possible 15 times: drugs available free at the point of care; a government QUM unit; undergraduate training of prescribers in standard treatment guidelines, antibiotics not available without prescription and generic substitution in the public sector. Interpretation Certain EM policies are associated with better QUM and impact increases with co-implementation. Analysis across three datasets provides a policy short-list as a minimum investment by countries trying to improve QUM and reduce antimicrobial drug misuse.


Methods
We analysed data on policy implementation from World Health Organisation (WHO) surveys in 2007 and 2011, and QUM data from surveys during 2006-2012 in developing and transitional countries. We compared QUM scores in countries that did or did not implement specific policies and regressed QUM composite scores on the numbers of policies implemented. We compared the ranking of policies in this and two previous studies, one from the same WHO databases (2003)(2004)(2005)(2006)(2007) the other from data obtained during country visits in South-East Asia (2010)(2011)(2012)(2013)(2014)(2015). The rankings of a common set of 17 policies were correlated and we identified those that were consistently highly ranked.

Findings
Fifty-three countries had data on both QUM and policy implementation. Forty policies were associated with effect sizes ranging from +13% to -5%. There was positive correlation between the composite QUM indicator and the number of policies reported implemented: (r) = 0.437 (95% CI 0.188 to 0.632). Comparison of policy rankings between the present and previous studies showed positive correlation with the WHO 2003-7 study: Spearman's rank correlation coefficient 0.498 (95% CI 0.022 to 0.789). Across the three studies, five policies were in the top five ranked positions 11 out of a possible 15 times: drugs available free at the PLOS  Introduction Suboptimal (irrational, incorrect, inappropriate) use of medicines is widespread, wasteful, and causes poor patient outcomes including anti-microbial drug resistance [1][2][3][4][5][6][7][8][9]. Interventions to improve quality use of medicines (QUM) in low/middle-income countries have mostly been small-scale, of limited duration, with small to modest effects [10][11].
Evidence from studies that we conducted in public healthcare sectors in developing and transitional countries suggests that implementation of WHO essential medicines (EM) policies is associated with better quality use (rational use) of medicines (QUM), including more appropriate use of anti-microbial agents [12][13][14]. The original WHO global data-set [12] covered the period 2003-2007 and there was uncertainty about how well EM policies were executed (based on country self-reports), with simultaneous deployment of multiple policies making it difficult to estimate individual impacts. We accessed a second source of data collected during 2-week visits to countries in South-East Asia during 2010-15, where policy implementation was observed independently [14]. The analyses of these data confirmed several of the findings of the earlier studies [12][13], including a correlation between the total numbers of EM policies implemented and composite measures of QUM. However, it remains unclear which policies are associated with the largest beneficial effects on medicines use.
The aims of the present work were to analyse an updated global WHO data-set (2007-2011), which included some policies not previously evaluated, and to test the consistency of our earlier findings of an increased impact with larger numbers of implemented EM policies. In addition, we wished to assess replicability of findings by correlating the rankings of policies that were common to the three studies to determine whether certain policies were consistently associated with the largest effects.

Methods
The analytical methods used have been described previously [12][13][14] and are summarised briefly here. QUM data (outcomes) were extracted from independent survey reports contained within the WHO medicines use database for the period 2006-2012 [3] and reported policy implementation data were obtained from WHO policy databases of surveys sent to Ministries of Health in 2007 and 2011 [15][16]. A dataset was created with one set of QUM and policy indicators for each country. Where the same QUM indicator was measured by more than one survey in the same country during 2006-2012, an average value was calculated. Where the same policy was reported differently in 2007 and 2011, the policy information reported from within one year of QUM survey was used or if this was not possible the data were excluded.

Indicators
Thirteen QUM indicators were extracted from the WHO medicines use database [3]. Only surveys using recommended validated measures estimated from at least 600 prescriptions and/ or three or more facilities were included [17][18]. The QUM indicators were all expressed as proportions and are described in Table 1, together with the directionality of better (or worse) QUM. Ten indicators were used in the previous analysis of WHO data [12].
Fifty-two indicators of reported policy implementation were extracted (49 from WHO questionnaires sent to Ministries of Health in 2007 and 2011 [19] and 3 from the WHO medicines use database [3]). The selected EM policies included all those that had been associated with better QUM in the previous two studies [12][13][14]; all were categorised as yes/no variables. Policies were excluded from analysis if there were fewer than six countries reporting implementation or non-implementation of the policy as was done previously [12].
Where policy indicators overlapped, only one was included. Where there was more than one indicator with a time-frame we included the one with the largest sample size. Through this process the number of policy indicators was reduced to 40 (reasons given in Table 2).
Eight policy indicators had not been analyzed in previous studies: availability of essential medicine list (EML) booklets at health facilities; existence of national legislation on drug promotion; prohibition of advertising prescription-only medicines to the public, a national task force to contain antimicrobial resistance and four policies concerning pharmacists-undergraduate training on the EML and standard treatment guidelines (STGs), continuing professional development and whether pharmacists prescribed in primary care (Table 2).

Analyses
As previously [12][13][14], we did not try to perform head-to-head comparisons of different policies. Countries implemented different combinations of policies, so the impact of a single policy could not be separated from those that were co-implemented.  [12][13][14]. However, assuming that overall measurement of QUM will be more robust with more individual QUM indicators, and due to the large number of studies measuring antimalarial use in recent years, it was decided to include this extra QUM indicator on antimalarial use in this study. https://doi.org/10.1371/journal.pone.0228201.t001 Essential medicine policies and quality use of medicines  For each QUM indicator we calculated the mean difference (expressed as a percentage) between countries reporting implementation (or not) of specific policies. For each policy, we estimated the average difference across all 13 QUM indicators, aligning directionality of better use (positive number) and worse use (minus number), and including only those QUM indicators where there were at least three countries with and three without the policy in question [12].
To assess the impact of multiple policies we generated a composite QUM score from 13 QUM indicators, which enabled all countries to be included in the analysis [12]. We calculated how far each country's value lay above or below the mean value from all countries for each QUM indicator expressed as standard deviation (SD) units. We then calculated the average of the SD unit increments across the thirteen QUM indicators for each country and used linear regression to assess correlation with the number of EM policies that were implemented [12]. We limited these analyses to policies that had a statistically significant association with better QUM in the univariate analyses.
Individual QUM indicators were also regressed on the number of implemented policies to determine whether specific aspects of QUM were influenced by the intensity of policy implementation. The impact of country wealth was assessed by including Gross National Income per capita [20] in multiple linear regression analyses and by repeating the regression analyses for countries with GNIpc above and below the median of USD 2315.

Testing the replicability of findings across three studies
Statistical analysis methods used in the present study were the same as those used in the earlier WHO analysis [12][13] and the SE Asia country visit analysis [14], enabling us to compare findings across three studies [12][13][14]. For each of the three data-sets we ranked the policies based on their estimated impact from the univariate analyses. We used non-parametric regression analysis to measure the correlation between the ranking of the policies that were common to the three studies [12][13][14]. We established the overall ordering of policies by calculating the sum of their ranks across the 3 studies. All analyses were done in Excel 2016, using either Epi Info (version 7.2.1.0)-or Stats Direct (version 3.1.20).
All work involved secondary analyses of data collected for other purposes. Data were aggregated at the level of countries or policies, not individuals, so research ethics board approval was not required.

Results
Fifty-three countries had data on both QUM and policy implementation. Regional distribution of countries was Africa (23), Eastern Mediterranean (7), Europe (2), Latin America (2) Table) describes the 13 QUM indicators and 3 policy indicators obtained from the WHO medicines use database, by country together with the survey references. Supporting information (S2 Table) describes information on the reported implementation of 52 policies by country. Supporting information (S3 Table) describes the impact of common policies in this study and the two previously published studies-this being the data used in the replicability analysis. Table 3 shows the estimates of policy effect on QUM by policy type, comparing results in countries that did, or did not, report implementation. Fig 1 shows these results in order of their estimated effect size.

Strength of associations for individual policies
Policies that were statistically significantly associated with 5-10% or higher effects on QUM included: drugs free at the point of care for children less than five years and the poor; generic substitution; a national strategy to contain antimicrobial resistance; a national body dedicated to QUM; booklets of the national essential medicines lists and standard treatment guidelines available at health facilities; not having systemic antibiotics available over-the-counter; an updated national formulary; no prescriber revenue from medicine sales; national legislation on drug promotion; public education; all drugs on the national Essential Medicines List (EML) provided free of charge in a national health or social insurance system; drug and therapeutic committees in hospitals; and undergraduate education of doctors and pharmacists on standard treatment guidelines.
Of the 27 policies that were associated with positive effects, the average estimated effects were: 9.3% (range 7.0 to 13.0)% for economic policies; 7.4% (2.3 to 10.5%) for managerial policies; 7.3% (range 5.6 to 10.2%) for structural policies; 4.9% (range 2.3 to 6.8%) for educational policies; 4.9% (range 1.7 to 8.6%) for regulatory policies, and 4.2% (range 3.2 to 5.1%) for human resource management policies. Correlation between the composite QUM indicator and the number of significantly effective policies reported as implemented (out of 18) was moderate (r = 0.437; 95% CI 0.188 to 0.6322) and strengthened when regression was limited to countries with more than two QUM indicators (r = 0.510; 95% CI 0.243 to 0.704). Inclusion of a national wealth measure (GNIpc) in the regression had no effect (r = 0.51; 95% CI 0.243 to 0.704) and the correlation coefficients were similar when analyses were conducted separately for countries with GNIpc levels above (r = 0.55, p = 0.018) and below the group median (r = 0.41, p = 0.048)).

Impacts of multiple policies and national wealth
When we examined the impact of multiple policies on individual QUM indicators (Supporting information S4 Table) we found that the percentage of all cases treated with antibiotics was significantly less with implementation of a greater number of policies (r = -0.375; 95% CI -0.624 to -0.059) as was the percentage of upper respiratory tract infection cases treated with antibiotics (r) = -0.554; 95% CI -0.796 to -0.161. Fig 3 shows a scatter-gram of the percentages of upper respiratory tract infection cases treated with antibiotics versus the number of policies reported implemented. The differences in the percentage of upper respiratory tract infection cases treated with antibiotics were large, ranging from 80-100% in countries implementing less than four EM policies to 30-70% in countries implementing more than 15 policies. Table 4 summarises the results for 17 policies that were common to the three studies ordered by the sum of the ranks across the three studies. The table also provides the individual study rankings and whether the univariate analyses of effect sizes had 95% CI that excluded zero. We found a significant correlation between the ranking (24 common policies) in the present study (2007)(2008)(2009)(2010)(2011) and that found in the previous analysis of WHO global data (2003)(2004)(2005)(2006)(2007): rank correlation coefficient Rho = 0.498 (95% CI 0.022 to 0.789). Correlation between the ranking (20 common policies) in the current analysis and that from the SE Asia country visits was weaker: Rho = 0.465 (95% CI -0.020 to 0.773). Nine policies had effect sizes of 4-10% that were Table 3 Table 4), appearing in the top 5 ranked positions 11 out of a possible 15 times. They were: medicines free at the point of care; the presence of a government QUM unit, undergraduate training of prescribers in STGs, antibiotics not available without prescription and generic substitution allowed in the public sector. Statistically significant better QUM associated with implementation of more policies was seen in all three studies [12][13][14].

Discussion
The main findings from the current study of the most recent WHO data-bases were three-fold. Firstly, some essential medicines policies were associated with better QUM. The strongest associations were for: medicines free at the point of care, implementation of STGs and the EML, a national body to promote QUM, a national strategy to contain AMR, disallowing antibiotic availability OTC, generic substitution in the public sector, hospital DTCs, and public education. Secondly, all policy categories had similar overall degrees of association with better QUM. Thirdly, there was a positive correlation between the number of policies that countries reported implementing and their measures of QUM. The WHO data have significant limitations, notably the reliance on self-report and the variable co-implementation of several policies, making it difficult to discern the true effects of individual policies. In addition, multiple policies and QUM measures make chance associations likely and limit the interpretation of statistical significance testing. In this situation a consistent finding of a relationship between intensity of policy implementation (number of policies) and a composite measure of QUM is important. In this and previous studies [12][13][14], there were  The strength of association seen in this study was like those seen in the previous analyses of WHO global data [12] and data from SE Asia [14]. Unlike the previous two studies the association between EM policies and QUM appeared to be weaker in poorer countries than in wealthier ones, although the association was stronger when regression analysis was limited to more robust QUM data (based on more than 2 QUM indicators). Although analyses of multiple policy exposure are valuable, these analyses have their own limitations. Most importantly, the exposure variable is the number of equally weighted policies and this does not assist in the identification of the most effective policies. With potentially large numbers of policies and co-variates, and modest number of countries, it was not possible to perform multi-variable analyses and conduct comparisons of individual policies. For these reasons, we assessed the replicability of the ordering of policies by estimated effects across the three studies we have completed. The correlations of the rank orders between the present and previous analyses were modest when measured across the full set of 17 policies that were common to each study. However, the five highest ranking policies (Table 4) occupied the top five places on 11 out of a possible 15 occasions.
In a situation defined by weak data we think the replicability we found across three separate studies, using almost identical methods, is the strongest evidence for identifying the most effective essential medicines policies. We are not suggesting that these policies are the only ones that should be considered for implementation. Countries with particular needs may Essential medicine policies and quality use of medicines choose from a larger basket of policies. However, five apparently strong policies, each from a different category, represents a minimum investment for countries seeking to improve QUM and optimize the consumption of antimicrobial drugs. The policies are: drugs free at the point of care; existence of a government QUM unit; undergraduate prescriber training in standard treatment guidelines; antibiotics not available over the counter without prescription and generic substitution allowed in the public sector. Because they come from different policy categories it is possible that they have complementary effects, although that couldn't be tested here.

Comparison with the broader literature
Previous reviews have recommended implementing similar policies to improve QUM [21][22]. Other studies reporting on actual policy effectiveness reported on: prescriber education [10][11]; public education [23][24]; an MOH body dedicated to promoting QUM [25]; hospital drug and therapeutic committees (DTCs) [26]; non-allowance of prescriber revenue from medicine sales [27][28][29]; non-allowance of antibiotic availability OTC [30], and national legislation and monitoring of drug promotional activities [31]. Greater effectiveness of multi-faceted interventions (which may involve multiple localised policies), as opposed to single-faceted ones, has also been found elsewhere [32][33][34]. Furthermore, the better QUM seen here with implementation of more policies was large and comparable with intervention effects reported elsewhere [10][11][32][33][34]. However, the sustainability of the better QUM achieved with national medicines policy implementation is likely to be much greater than that achieved with the discrete interventions implemented locally.

Limitations
We have extensively discussed the limitations of the WHO data-bases above and in previous reports [12][13]. The policy data used are reliant on self-reports of implementation, which may have been inaccurate, and the apparent effectiveness of individual policies may have been due to co-interventions. The small number of countries, large numbers of possible policy combinations, and other factors, including political will and economic stability, can reduce implementation effectiveness and hamper attempts to estimate the impacts of individual policies or specific policy combinations. Another weakness of the data was the assumption that policies may have remained the same over time. However, the fact that there were only small differences between estimates from countries reporting policy implementation for both 2007 and 2011 suggests that most policies generally remained constant. Furthermore, misreporting and misclassification would likely have weakened any associations seen between policy implementation and QUM.
There were weaknesses in the QUM data. Firstly, they come from surveys published in the literature. While only surveys using standard methodology and indicators [17][18] were used, they were often based on small sample sizes. Secondly, although standard QUM indicators were used, some were probably measured differently across studies. Thirdly, for some countries there were only one or two QUM indicators measured. This gave a less robust picture of overall QUM and was the reason for use of a composite QUM indicator that allowed all countries to be included in the regression analyses. A few countries had outlier QUM estimates based on only 1-2 indicators; this was a possible explanation for the stronger correlation between number of policies implemented and better QUM when the analysis was confined to Essential medicine policies and quality use of medicines countries reporting three or more QUM indicators. Finally, the clinical relevance of a composite QUM indicator is not clear, but the component indicators have relevance and we aligned each to ensure that directionality of change was constant. As with uncertainty over policy variables, any inaccuracies of medicine use estimates would likely have weakened any associations seen between policy implementation and QUM. Our results were limited to the public sector, since there were insufficient QUM data from the private sector. While the private sector may provide most health care in many low and middle-income countries, the findings are still important since many prescribers work in both sectors and many policies are aimed at both the private and public sectors.

Conclusions
In conclusion, repeated analyses of independent data-sets have shown replicability of two principal findings. The first is that five apparently robust essential medicines policies appear to represent the best choices for countries trying to improve medicines use, and the second one is that the implementation of multiple policies increases their effects. In 2016 The Lancet Commission on Essential Medicines identified five crucial areas of essential medicines policy. Three of these: paying for a basket of essential medicines, making essential medicines affordable and promoting quality use of medicines are strongly supported by the findings of this study [1].