A gap exists between evidence and practice regarding the management of cardiovascular risk factors. This gap could be narrowed if systematically developed clinical practice guidelines were effectively implemented in clinical practice. We evaluated the effects of a tailored intervention to support the implementation of systematically developed guidelines for the use of antihypertensive and cholesterol-lowering drugs for the primary prevention of cardiovascular disease.
Methods and Findings
We conducted a cluster-randomized trial comparing a tailored intervention to passive dissemination of guidelines in 146 general practices in two geographical areas in Norway. Each practice was randomized to either the tailored intervention (70 practices; 257 physicians) or control group (69 practices; 244 physicians). Patients started on medication for hypertension or hypercholesterolemia during the study period and all patients already on treatment that consulted their physician during the trial were included. A multifaceted intervention was tailored to address identified barriers to change. Key components were an educational outreach visit with audit and feedback, and computerized reminders linked to the medical record system. Pharmacists conducted the visits. Outcomes were measured for all eligible patients seen in the participating practices during 1 y before and after the intervention. The main outcomes were the proportions of (1) first-time prescriptions for hypertension where thiazides were prescribed, (2) patients assessed for cardiovascular risk before prescribing antihypertensive or cholesterol-lowering drugs, and (3) patients treated for hypertension or hypercholesterolemia for 3 mo or more who had achieved recommended treatment goals.
The intervention led to an increase in adherence to guideline recommendations on choice of antihypertensive drug. Thiazides were prescribed to 17% of patients in the intervention group versus 11% in the control group (relative risk 1.94; 95% confidence interval 1.49–2.49, adjusted for baseline differences and clustering effect). Little or no differences were found for risk assessment prior to prescribing and for achievement of treatment goals.
Citation: Fretheim A, Oxman AD, Håvelsrud K, Treweek S, Kristoffersen DT, Bjørndal A (2006) Rational Prescribing in Primary Care (RaPP): A Cluster Randomized Trial of a Tailored Intervention. PLoS Med 3(6): e134. doi:10.1371/journal.pmed.0030134
Academic Editor: Steve Jan, George Institute for International Health, United States of America
Received: August 18, 2005; Accepted: March 2, 2006; Published: June 6, 2006
Copyright: © 2006 Fretheim et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The study was funded by the Norwegian Ministry of Health, the Norwegian Directorate for Health and Social Affairs, and the Norwegian Knowledge Centre for Health Services. The funding agencies had no influence on the design or conduct of the study.
Competing interests: All the authors are, or have been, employed by the Norwegian government, which has a substantial interest in improving professional practice in Norway and in containing the costs of health care. The clinical practice guidelines and the study design have not been endorsed by the government or any other organization and represent the views of the authors. No other competing interests are declared.
Abbreviations: CI, confidence interval; GEE, generalized estimating equations; LDL, low-density lipoprotein
An important issue in health care is “getting research into practice,” in other words, making sure that, when evidence from research has established the best way to treat a disease, doctors actually use that approach with their patients. In reality, there is often a gap between evidence and practice.
An example concerns the treatment of people who have high blood pressure (hypertension) and/or high cholesterol. These are common conditions, and both increase the risk of having a heart attack or a stroke. Research has shown that the risks can be lowered if patients with these conditions are given drugs that lower blood pressure (antihypertensives) and drugs that lower cholesterol. There are many types of these drugs now available. In many countries, the health authorities want family doctors (general practitioners) to make better use of these drugs. They want doctors to prescribe them to everyone who would benefit, using the type of drugs found to be most effective. When there is a choice of drugs that are equally effective, they want doctors to use the cheapest type. (In the case of antihypertensives, an older type, known as thiazides, is very effective and also very cheap, but many doctors prefer to give their patients newer, more expensive alternatives.) Health authorities have issued guidelines to doctors that address these issues. However, it is not easy to change prescribing practices, and research in several countries has shown that issuing guidelines has only limited effects.
Why Was This Study Done?
The researchers wanted—in two parts of Norway—to compare the effects on prescribing practices of what they called the “passive dissemination of guidelines” with a more active approach, where the use of the guidelines was strongly promoted and encouraged.
What Did the Researchers Do and Find?
They worked with 146 general practices. In half of them the guidelines were actively promoted. The remaining were regarded as a control group; they were given the guidelines but no special efforts were made to encourage their use. It was decided at random which practices would be in which group; this approach is called a randomized controlled trial. The methods used to actively promote use of the guidelines included personal visits to the practices by pharmacists and use of a computerized reminder system. Information was then collected on the number of patients who, when first treated for hypertension, were prescribed a thiazide. Other information collected included whether patients had been properly assessed for their level of risk (for strokes and heart attacks) before antihypertensive or cholesterol-lowering drugs were given. In addition, the researchers recorded whether the recommended targets for improvement in blood pressure and cholesterol level had been reached.
Only 11% of those patients visiting the control group of practices who should have been prescribed thiazides, according to the guidelines, actually received them. Of those seen by doctors in the practices where the guidelines were actively promoted, 17% received thiazides. According to statistical analysis, the increase achieved by active promotion is significant. Little or no differences were found for risk assessment prior to prescribing and for achievement of treatment goals.
What Do These Findings Mean?
Even in the active promotion group, the great majority of patients (83%) were still not receiving treatment according to the guidelines. However, active promotion of guidelines is more effective than simply issuing the guidelines by themselves. The study also demonstrates that it is very hard to change prescribing practices. The efforts made here to encourage the doctors to change were considerable, and although the results were significant, they were still disappointing. Also disappointing is the fact that achievement of treatment goals was no better in the active-promotion group. These issues are discussed further in a Perspective about this study (DOI: 10.1371/journal.pmed.0030229).
Please access these Web sites via the online version of this summary at http://dx.doi.org/10.1371/journal.pmed.0030134.
• The Web site of the American Academy of Family Physicians has a page on heart disease.
• Another PLoS Medicine article has also addressed trends in thiazide prescribing.
Hypertension and hypercholesterolemia are common problems in general practice and have important consequences for patients and the use of health-care resources.
There is high-quality evidence from well-designed randomized trials of the effects of antihypertensive and cholesterol-lowering drugs for the primary prevention of cardiovascular disease [1,2]. However, a gap exists between evidence and practice regarding several aspects of cardiovascular risk factor management . This gap could be narrowed if systematically developed clinical practice guidelines were effectively implemented. Several randomized trials of guideline implementation strategies have been conducted, generally demonstrating only modest effects .
Based on a review of previous guidelines and the underlying evidence, we developed guidelines for prescribing antihypertensive and cholesterol-lowering drugs for the primary prevention of cardiovascular disease in Norwegian general practice [5–7].
We decided to address possible improvements for three key issues: cardiovascular risk is commonly not estimated before starting treatment , thiazides are underutilized when initiating treatment for hypertension , and treatment goals are frequently not achieved for both blood pressure and cholesterol [10,11]. We therefore developed an intervention aimed at improving performance in these three areas and conducted a trial to assess the effectiveness and costs of this intervention.
The results of evaluations of interventions to improve clinical practice differ, and large effects are not common [12,13]. As with any behavior, professional behavior is difficult to change. The underlying reasons for differences between clinical practice and recommendations in systematically developed guidelines vary from one clinical problem to another and from one clinician to another [14,15]. It is therefore logical to attempt to tailor strategies to support the implementation of guidelines to address identified barriers to change . However, the effectiveness of tailored interventions is uncertain .
The primary objective of this trial was to evaluate the effects of a tailored intervention to support the implementation of guidelines in Norwegian general practice for the use of antihypertensive and cholesterol-lowering drugs for the primary prevention of cardiovascular disease. Our hypothesis was that the tailored intervention would be more effective than passive dissemination of the guidelines.
We anticipated that passive dissemination of guidelines would have little effect on the outcomes of interest [12,13,18] and therefore was an appropriate control intervention. We decided that randomizing practices rather than physicians or patients would be the most feasible design for practical reasons, and also the most appropriate to avoid contamination from the intervention to the control group.
The study participants were all general practices in two geographically defined areas of Norway; the practices all used one or the other of two eligible electronic medical record systems.
Patients started on medication for hypertension or hypercholesterolemia during the study period and all patients already on treatment that consulted their physician during the trial were included in the analyses. The eligibility criteria for patients are described in more detail under “Outcomes.”
We developed our intervention through a process of identifying barriers to implementation of the recommendations and tailoring the intervention to address these. This is described in more detail elsewhere . Box 1 shows the various elements of our intervention.
The intervention was initiated through an educational outreach visit carried out between May and December 2002 by pharmacists recruited and trained specifically for this purpose.
During the outreach visit the main elements of the guidelines were presented, with special emphasis on cardiovascular risk estimation, choice of first-line drugs for hypertension, and treatment goals. A printed copy of the guidelines and a one-page version were given to the physicians, including a chart to aid the estimation of cardiovascular risk.
A software package was installed during the visit. This enabled us to extract data and immediately, during the visit, present the physicians with data on their performance of risk estimation, choice of antihypertensive drugs, and achievement of treatment goals (audit and feedback). The software package also included computerized reminders (“pop-ups”) on the computer screen. These were triggered at the patient's first visit following a recording of an elevated blood pressure (>140/90 mm Hg) or cholesterol level (total cholesterol > 5 mmol/l [190 mg/dl] or low-density lipoprotein [LDL] cholesterol > 3 mmol/l [115 mg/dl]).
If the patient had not been prescribed blood-pressure- or cholesterol-lowering drugs, the physician was reminded of the recommendation to carry out cardiovascular risk assessment and was given the option of starting a computer program for risk assessment. Recommendations on choice of drugs were also given, and the physician was given the choice of printing out patient information material.
If the patient was already on blood-pressure- or cholesterol-lowering drugs, the pop-up reminded the physician of recommended treatment goals and asked if the physician would like to print out patient information material.
Within 3 d after the outreach visit, a member of the research team called the clinic to confirm that they were not experiencing problems with their computers as a result of our visit.
The doctors who were invited to participate in the study were given information about the objectives of the study and the practical impact it might have on their practice. We obtained written consent from all practices. We submitted the research protocol to the Regional Committee of Research Ethics, which considered ethical approval unnecessary.
The Norwegian Data Inspectorate approved the handling of the data.
We chose three main outcome measures, all aimed at physician behavior regarding the pharmacological management of primary prevention of cardiovascular disease for the 12 mo following the outreach visit. All outcomes were recorded at the patient level. Because the eligibility criteria varied from one outcome to another, the number of patients included in the analysis varied across outcomes. Baseline data for the 12 mo preceding the intervention were also collected.
Patients with established cardiovascular disease were excluded, with the exception of the outcomes related to treatment goals for lipid-lowering therapy, since these are the same across patient groups. Because antihypertensive drugs are also prescribed for the treatment of thyrotoxicosis and migraine, we excluded data from patients with these diagnoses. We used extracted medical record data on prescribing and diagnoses to identify patients who were to be excluded. For example, patients who had been given a prescription for nitroglycerin were assumed to have coronary heart disease, and thus were excluded from most analyses.
Patients were considered to be previously untreated if they had hypertension (blood pressure > 140/90 mm Hg) or hypercholesterolemia (total cholesterol > 5 mmol/l [190 mg/dl] or LDL cholesterol > 3 mmol/l [115 mg/dl]) but no prescription for the corresponding medication had been recorded for 24 mo preceding the outreach visit.
The primary outcomes were the following: (1) the proportion of patients prescribed thiazides among patients prescribed antihypertensive drugs for the first time, (2) the proportion of patients with a cardiovascular risk assessment among all those started on antihypertensive or cholesterol-lowering treatment (excluding patients already on either type of medication), and (3) the proportion of patients with a recorded level of cholesterol (total or LDL) or blood pressure satisfying the specified treatment goals among all patients on the corresponding treatment for at least 3 mo. For cholesterol we also included patients on secondary prevention since the treatment goals are similar.
The secondary outcomes, as prespecified in the research protocol (Protocol S1; ), were the following: (1) the proportion of patients reporting that they were involved in the decision-making process before drug treatment for hypertension and/or elevated cholesterol was started, (2) the level of risk among patients started on treatment, (3) the proportion of patients with risk above 20% among those started on treatment, (4) the level of risk among patients not started on treatment for whom blood pressure and cholesterol level were recorded, (5) the proportion of prescriptions of thiazides or beta-blockers to patients who were prescribed antihypertensive drugs for the first time, (6) the proportion of prescriptions of angiotensin receptor blockers or alpha-blockers to patients who were prescribed antihypertensive drugs for the first time, (7) for patients with diabetes, the proportion of patients with a recorded level of cholesterol (total or LDL) or hypertension satisfying the specified treatment goals among all patients on the corresponding treatment (for cholesterol we also included patients on secondary prevention since the treatment goals are similar), (8) the proportion of patients reaching the specified treatment goal for blood pressure, and (9) the proportion of patients reaching the specified treatment goal for cholesterol level.
All outcomes were calculated from data extracted from the practices' medical record systems, with two exceptions: (1) the use of cardiovascular risk assessment tools by physicians before starting medication and (2) the level of patient involvement. Patients potentially eligible for inclusion in these analyses were identified from the medical record data, after which we interviewed the prescribing physician (or a colleague) per telephone. We enquired about the use of risk assessment tools for each patient they had started on treatment during the intervention period, and the physicians gave their answers based on notes from the medical records. The physicians assisted us by sending a questionnaire to the patients who had been started on medication. The questionnaire consisted of one question, asking to what extent the patients felt that they had taken part in the decision to start drug therapy. The answer was given on a five-point scale, from “none” to “fully.” Only patients reporting that they took no part at all were counted as not being involved in the decision to start treatment.
For two outcomes—(1) the proportion of patients for whom cardiovascular risk had not been estimated and (2) the level of risk among patients not started on treatment—the analysis was based on a random sample of eligible patients.
During interviews with physicians it was often possible to figure out whether a practice was in the intervention group. Investigators assessing outcomes and conducting analyses were blinded to the allocation of practices.
All analyses were by intention to treat. In response to comments from peer reviewers, we decided to deviate from our protocol and use adherence, rather than non-adherence, to recommendations as outcome measures. This had no impact on our findings.
In order to demonstrate a 25% relative reduction in non-adherence with the guidelines—with a power of 80% and a statistical significance level of 5%—in outcome measures between the control and intervention groups, we estimated that we needed a sample of approximately 140 practices in total (Cluster Randomisation Sample Size Calculator version 1.0.2, Health Services Research Unit, University of Aberdeen, Aberdeen, United Kingdom). The adjusting factor (intracluster correlation coefficient) was conservatively estimated to be 0.2, based on data from a previous study . More detail is available elsewhere .
Block randomization was done within two geographical areas (Oslo and Tromsø) to ensure balance in the number of practices in the intervention and control groups. The size of the blocks varied randomly between two, four, and six. A colleague not directly involved in our research project generated the allocation list using software from http://www.randomization.com. We gave her identification numbers representing each recruited practice, and she informed us whether the practice was allocated to the intervention or control group.
The generalized estimating equations (GEE) approach was used for the analysis of the binary outcomes, and the mixed effects linear regression model (a two-stage nested analysis of variance) for the continuous outcome (risk level), using the baseline log odds for compliance and the baseline mean risk level for each practice, respectively, as covariates . The analysis was performed using PROC GENMOD and PROC MIXED in SAS 9.1.3 (SAS Institute, Cary, North Carolina, United States).
The outcome from a GEE approach is an odds ratio; the adjusted relative risk was calculated from the odds ratio by using the overall proportion of compliance in the control group .
Originally we planned to use a parametric method, which depends on an underlying distribution of the sample observations. This is not the case for the GEE approach. We therefore used the GEE approach rather than the method we had specified in the protocol. This change was made before we analyzed the data.
Figure 1 illustrates the flow of practices (clusters) through the trial. Out of the 388 practices that were invited, 146 agreed to participate. One practice withdrew after randomization, before the outreach visit. The practices and patients in the intervention and control groups were similar with regards to baseline characteristics (Table 1).
aFor two of these practices no outreach visit was conducted (one withdrew after randomization and one had technical problems) but data collection was possible.
For seven of the 146 participating practices we were unable to collect medical record data for various reasons (Figure 1). One of the remaining 139 practices was not included in the analyses involving estimation of cardiovascular risk (three secondary outcomes) because of an error during data collection. The numbers of patients included in the analyses are shown in Figure 2.
aQuestionnaires were not sent to some patients included in the analysis of risk assessment (patient had died, unknown address, etc.), and some patients were excluded from the analysis of risk assessment after questionnaires had been sent to them.
We interviewed 339 physicians about their use of risk assessment tools. For another 108 physicians, information was collected via a colleague at the practice, covering a total of 89% of all physicians.
For the questionnaire, which was sent to 1,620 patients, we achieved a response rate of 59%.
The main findings are presented in Table 2. Thiazide prescribing increased from 5.8% to 17.3% in the experimental group, and from 8.8% to 11.1% in the control group. Thus, prescribing of thiazides was significantly higher in the experimental group (relative risk 1.94; 95% confidence interval [CI]1.49–2.49). Little or no effect was detected for the two other main outcomes.
The results for the secondary outcomes are reported in Table 3. There were no statistically significant differences for these outcomes, except for those related to choice of antihypertensive drug. The distribution of responses to the patient questionnaire was similar in the two groups.
The intervention had an impact on prescribing patterns, but for other outcomes no statistically significant effects were found.
Strengths and Weaknesses of the Study
Compared to other evaluations of quality improvement interventions, the size of the study and use of rigorous scientific methods are the main strengths of our study. The risk of selection bias was minimized by the randomized design. The impact of random errors was minimized by including 146 practices with 500 physicians and a large number of patients (between 950 and 33,800 per outcome).
Several assumptions had to be made during data analysis, such as assuming that patients were not on medication if no prior prescription for antihypertensive therapy was recorded. Some of the patients proved to be on medication that had been prescribed elsewhere. Similarly, we relied on routinely collected clinical data for coding patients as smokers or nonsmokers, and these data may be incomplete even though we searched the free-text notes area of the medical records for information on smoking. However, we can assume that these errors are evenly distributed between the intervention and control groups, and thus should not introduce biases into the analyses.
One outcome was based on self-report by physicians, and another on self-report by patients. It is possible that physicians in the experimental group tended to report a higher use of risk assessment tools since this had been promoted to them. However, the reported use of risk assessment tools was similar in the two groups.
Practices in only two areas of Norway were invited to participate in the trial, and only 38% of them were included. Some practices were not invited because they did not use a medical record system compatible with our software. Thus, the practices in the trial are not necessarily representative for all general practices in Norway. Compared to national statistics, the proportion of practices with only one or two physicians was higher in our study (48% versus a national average of 30%).
Study Findings in Relation to the Overall Evidence
Walsh and colleagues recently conducted a systematic review of studies evaluating quality improvement interventions for the management of hypertension . They included four randomized controlled trials of interventions targeting the choice of antihypertensive drug [24–27]. None of these trials reported statistically significant results. The point estimates for effect sizes ranged from −2.6% to 6%. The corresponding figure for our study is 9.3%. The review also included some nonrandomized studies, and the authors' overall conclusion was that “the median increase in percentage of providers adhering to recommendations was 3.0%” (interquartile range 1.0%–5.5%).
One randomized trial published too recently to be included in the systematic review was also aimed at increasing the prescribing of thiazides . The intervention involved case-based educational modules and personal prescribing feedback. The investigators reported an absolute effect size slightly better than ours: an increase in thiazide prescribing of 11.5% (95% CI 4.0%–18.8%). Thus, although the results from trials targeting professional behavior have often been disappointing, there are some encouraging findings, including those reported by us.
Walsh et al. also reviewed trials of interventions aimed at improving the achievement of treatment goals for patients on antihypertensive therapy. They found a median increase in the proportion of patients achieving recommended targets of systolic and diastolic blood pressure of 16.2% (interquartile range 10.3%–32.2%) and 6.0% (interquartile range 1.5%–17.5%), respectively. Nonrandomized trials were also included in this analysis. The greatest effect sizes were achieved with interventions involving organizational change and patient education. Our intervention included no such components, apart from giving physicians the ability to print out patient education materials incorporated into their medical record system.
Some specific areas of professional behavior may be more difficult to influence than others. For instance, physicians are under immense pressure from the pharmaceutical industry to choose antihypertensive drugs other than thiazides, which are off-patent drugs available at a low price. On the other hand, achieving treatment goals and conducting cardiovascular risk assessment are uncontroversial recommendations that are encouraged by practically all stakeholders. Attitudes and lack of knowledge among physicians are likely to be the main barriers to adhering to these recommendations, and our intervention was specifically designed to address these .
Despite our targeted intervention, we did not succeed in having an impact on these behaviors. One explanation may be that achieving treatment goals is a patient outcome, which depends on compliance of patients as well as physicians. Our intervention was mainly directed at physician behavior. We have no good explanation for the lack of effect on the use of risk assessment tools. A process evaluation is underway, to identify why the intervention did not have an effect on various outcomes across practices.
The outreach visits were conducted as group sessions with all the physicians in the practice. It is likely that visiting physicians individually is more effective , but this is also more costly to implement. Our intervention was multifaceted, and it is not possible to say which of the various components were the most important for the overall effectiveness. The planned process evaluation may provide some insight.
Although we managed to nearly double the prescribing of thiazides, it can be argued that the effect is far from satisfactory. The use of thiazides is still very limited in Norway, and although we achieved a rate of 17% for the prescribing of thiazides, this is low relative to other countries and relative to what might be considered appropriate .
An alternative strategy for quality improvement is the use of regulatory measures to enforce treatment protocols. Legislation went into effect in March 2004 that requires Norwegian doctors to prescribe thiazides as the first-choice antihypertensive drug if the patients are to have their drug costs reimbursed through the national health insurance system. However, the effect of such legislative measures is not well documented. Our results are in line with those of other randomized trials that have found outreach visits to be effective at changing prescribing, with similar effect sizes (median adjusted risk difference 3%, range 0%–7.5%, for 12 studies), while effects on other outcomes are much less consistent . We did not find our intervention to be effective for other behaviors. We report on the cost effectiveness of our intervention in a companion article .
This trial has the registration number ISRCTN48751230 in the International Standard Randomized Controlled Trial Number Register.
Protocol S1. Trial Protocol
(310 KB PDF)
Box 1. Components of the Tailored Intervention
Educational Outreach Visit
• Presentation focusing on three main messages: (1) relevance of risk estimation and how to do it, including strategies on how to communicate information about risk to patients, (2) evidence of the effectiveness of thiazides and that fears of adverse effects are unjustified, and pointing out the consensus that exists among guidelines (attention also directed to the importance of clinically relevant endpoints when studies are quoted), and (3) clear recommendations justified by referring to the high degree of consensus among guidelines
• Guidelines handed out, directing attention to the authors (opinion leaders)
Audit and Feedback at Outreach Visit
• To what extent treatment goals are currently being achieved
• Drug choice profile on antihypertensives
• Level of risk among patients on treatment, compared to a sample (men, 40–65 y) not on treatment
• Risk assessment
• First-choice antihypertensive drugs
• Treatment goals
Risk Assessment Tools (Software and Charts)
Patient Information Material
• The relationship between single risk factors and global risk
• Thiazides and beta-blockers.
• Treatment goals
• Telephone call within 1–3 d to check that software had not led to any difficulties
• Short telephone interview with each physician after 1–3 mo
We thank all the physicians who agreed to participate in our trial. We also thank our team of outreach pharmacists: Trine Klemetsrud, Angelica Kruse-Jensen, Kirsten Sørhus, and Tone Westergren, and the team conducting the physician interviews: Rupert Bose, Anne Lise Jørgensen, and Vidar Vang. Signe Flottorp, Cheryl Carling, and Morten Aaserud gave us helpful advice and support; Mediata AS prepared the software package used to audit practices, provide reminders and patient information, and collect data; and Pfizer provided SmartHeart (a computer program for calculating cardiovascular risk) without conditions other than being mentioned in this section of the article. Craig Ramsay generously shared his statistical expertise, and Martin Eccles gave valuable comments on a draft of this paper.
ADO and AB had the original idea for the study and wrote the project proposal. AF was the principle investigator, wrote the first draft of the trial protocol, was responsible for running the trial, and wrote the first draft of this report. KH was instrumental in the management and running of the trial, and participated in the interpretation of data. ST provided important input during the planning phase, and also played a key role in the implementation of the study. DTK arranged the data and performed the statistical analyses. All authors contributed to and approved the final manuscript.
- 1. Collins R, Peto R, MacMahon S, Hebert P, Fiebach NH, et al. (1990) Blood pressure, stroke, and coronary heart disease. Part 2, Short-term reductions in blood pressure: Overview of randomised drug trials in their epidemiological context. Lancet 335: 827–838.
- 2. LaRosa JC, He J, Vupputuri S (1999) Effect of statins on risk of coronary disease: A meta-analysis of randomized controlled trials. JAMA 282: 2340–2346.
- 3. Turnbull F (2005) Managing cardiovascular risk factors: The gap between evidence and practice. PLoS Med 2: e131. doi: 10.1371/journal.pmed.0020131.
- 4. Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, et al. (2004) Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 8: 1–72.
- 5. Fretheim A, Bjørndal A, Oxman AD, Dyrdal A, Golding M, et al. (2002) Hvilke kolesterolsenkende legemidler bør brukes for primærforebygging av hjerte- og karsykdommer? Tidsskr Nor Laegeforen 122: 2287–2288.
- 6. Fretheim A, Bjørndal A, Oxman AD, Dyrdal A, Golding M, et al. (2002) Hvilke blodtrykkssenkende legemidler bør brukes for primærforebygging av hjerte- og karsykdommer? Tidsskr Nor Laegeforen 122: 2283–2286.
- 7. Fretheim A, Bjørndal A, Oxman AD, Dyrdal A, Golding M, et al. (2002) Retningslinjer for medikamentell primærforebygging av hjerte- og karsykdommer—Hvem bør behandles? Tidsskr Nor Laegeforen 122: 2277–2281.
- 8. Fretheim A, Håvelsrud K, Flottorp S, Oxman AD (2003) Påvirker takster og refusjonsregler praksis? Tidsskr Nor Laegeforen 123: 795–796.
- 9. Rønning M (2001) Drug consumption in Norway 1996–2000. Oslo: World Health Organization Collaborating Centre for Drug Statistics Methodology. 237 p.
- 10. Svilaas A, Risberg K, Thoresen M, Ose L (2000) Lipid treatment goals achieved in patients treated with statin drugs in Norwegian general practice. Am. J Cardiol 86: 1250–1253. A6.
- 11. Westheim A, Klemetsrud T, Tretli S, Stokke HP, Olsen H (2001) Blood pressure levels in treated hypertensive patients in general practice in Norway. Blood Press 10: 37–42.
- 12. Grimshaw JM, Shirran L, Thomas R, Mowatt G, Fraser C, et al. (2001) Changing provider behavior: An overview of systematic reviews of interventions. Med Care 39: II2–II45.
- 13. Oxman AD, Thomson MA, Davis DA, Haynes RB (1995) No magic bullets: A systematic review of 102 trials of interventions to improve professional practice. CMAJ 153: 1423–1431.
- 14. Flottorp S, Oxman AD (2003) Identifying barriers and tailoring interventions to improve the management of urinary tract infections and sore throat: A pragmatic study using qualitative methods. BMC Health Serv Res 3: 3.
- 15. Oxman AD, Flottorp SSilagy C, Haines A (2001) An overview of strategies to promote implementation of evidence-based health care. Evidence-based practice in primary care. London: BMJ Books. pp. 101–119.
- 16. Shaw B, Cheater F, Baker R, Gillies C, Hearnshaw H, et al. (2005) Tailored interventions to overcome identified barriers to change: Effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2005: CD005470.
- 17. Freemantle N, Harvey EL, Wolf F, Grimshaw JM, Grilli R, et al. (1997) Printed educational materials: Effects on professional practice and health care outcomes. Cochrane Database Syst Rev 1997: CD000409.
- 18. Fretheim A, Oxman AD, Treweek S, Bjørndal A (2003) Rational Prescribing in Primary Care (RaPP-trial). A randomised trial of a tailored intervention to improve prescribing of antihypertensive and cholesterol-lowering drugs in general practice [ISRCTN48751230]. BMC Health Serv Res 3: 5.
- 19. Fretheim A, Oxman AD, Flottorp S (2004) Improving prescribing of antihypertensive and cholesterol-lowering drugs: A method for identifying and addressing barriers to change. BMC Health Serv Res 4: 23.
- 20. Flottorp S, Oxman AD, Håvelsrud K, Treweek S, Herrin J (2002) A cluster randomised trial of tailored interventions to improve the management of urinary tract infections and sore throat. BMJ 325: 367–370.
- 21. Donner A, Klar N (2000) Design and analysis of cluster randomization trials in health research. London: Arnold. 178 p.
- 22. Zhang J, Yu KF (1998) What's the relative risk? A method of correcting the odds ratio in cohort studies of common outcomes. JAMA 280: 1690–1691.
- 23. Walsh J, McDonald K, Shojania K, Sundaram V, Nayal S, et al. (2005) Hypertension care. Rockville (Maryland): Agency for Healthcare Research and Quality. 98 p.
- 24. Gullion DS, Tschann JM, Adamson TE (1988) Management of hypertension in private practice: A randomized controlled trial in continuing medical education. J Contin Educ Health Prof 8: 239–255.
- 25. Borenstein JE, Graber G, Saltiel E, Wallace J, Ryu S, et al. (2003) Physician-pharmacist comanagement of hypertension: A randomized, comparative trial. Pharmacotherapy 23: 209–216.
- 26. Goldberg HI, Wagner EH, Fihn SD, Martin DP, Horowitz CR, et al. (1998) A randomized controlled trial of CQI teams and academic detailing: Can they alter compliance with guidelines? Jt Comm J Qual Improv 24: 130–142.
- 27. Nilsson G, Hjemdahl P, Hassler A, Vitols S, Wallen NH, et al. (2001) Feedback on prescribing rate combined with problem-oriented pharmacotherapy education as a model to improve prescribing behaviour among general practitioners. Eur J Clin Pharmacol 56: 843–848.
- 28. Herbert CP, Wright JM, Maclure M, Wakefield J, Dormuth C, et al. (2004) Better Prescribing Project: A randomized controlled trial of the impact of case-based educational modules and personal prescribing feedback on prescribing for hypertension in primary care. Fam Pract 21: 575–581.
- 29. Figueiras A, Sastre I, Tato F, Rodriguez C, Lado E, et al. (2001) One-to-one versus group sessions to improve prescription in primary care: A pragmatic randomized controlled trial. Med Care 39: 158–167.
- 30. Fretheim A, Aaserud M, Oxman AD (2003) The potential savings of using thiazides as the first choice antihypertensive drug: Cost-minimisation analysis. BMC Health Serv Res 3: 18.
- 31. Thomson O'Brien MA, Jamtvedt G, Kristoffersen D, Oxman A (2006) Educational outreach visits: Effects on professional practice and health care outcomes. Cochrane Database Syst Rev. In press.
- 32. Fretheim A, Aaserud M, Oxman AD (2006) Rational Prescribing in Primary Care (RaPP): Economic evaluation of an intervention to improve professional practice. PLoS Med 3: e216. doi: 10.1371/journal.pmed.0030216.