Clinical practice guidelines have the potential to improve care, but are often not optimally implemented. Improving guideline use in clinical practice may improve care. The objective of this study was to identify the barriers and facilitators (determinants) of guidelines use among neurologists and to propose a strategy to improve guideline implementation.
This was a mixed-methods study design. A quantitative, population-based, cross-sectional survey of Canadian neurologists was conducted. Associations between guidelines use and determinants of guidelines use were examined. Focus groups and interviews were conducted using purposeful sampling of the population. Determinants of guideline use were mapped to interventions to establish a strategy for guideline implementation among neurologists.
38.7% (n = 311) of neurologists responded to the survey. Typically, respondents had been practicing for 16.6 years and worked in an academic institution in an urban setting. Being male and having an academic affiliation was associated with guideline use. Determinants of guideline use differed between guideline users and non-users; non-users consistently rating determinants lower than users, especially applicability. Two focus groups and one interview (n = 11) identified six main themes of determinants of guideline use: Credibility, knowledge, applicability, resources, motivation, and target audience; which was congruent with the quantitative data. The proposed knowledge translation strategy contains three pillars: guidelines development, dissemination, and interventions.
Several determinants of guideline use not commonly discussed in the literature were identified (applicability, target audience, credibility). The proposed implementation strategy is a valuable resource for guideline developers and policy/decision-makers to improve knowledge translation of guidelines among neurologists.
Citation: Sauro KM, Wiebe S, Holroyd-Leduc J, DeCoster C, Quan H, Bell M, et al. (2018) Knowledge translation of clinical practice guidelines among neurologists: A mixed-methods study. PLoS ONE 13(10): e0205280. https://doi.org/10.1371/journal.pone.0205280
Editor: Ka-Wai Tam, Taipei Medical University Shuang Ho Hospital, TAIWAN
Received: May 9, 2018; Accepted: September 22, 2018; Published: October 10, 2018
Copyright: © 2018 Sauro et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data cannot be shared publicly because of concerns around breaches of privacy (some cells contain small numbers, which when combined with other variables could identify participants). Data requests can be sent to: University of Calgary Conjoint Health Research Ethics Board firstname.lastname@example.org (403) 220-2297
Funding: This study was funded in part by a University of Calgary Thesis Research Grant (file 1033705) and salary support was provided to KM Sauro by an Alberta Innovates: Health Solutions graduate scholarship. S Wiebe was the holder of the Hopewell Professorship for Clinical Neurosciences Research at the Hotchkiss Brain Institute, University of Calgary. H Quan’s salary was supported by an Alberta Innovates: Health Solutions population health scholar award. N Jetté was the holder of a Canada Research Chair Tier 2 in Neurological Health Services Research. There was no additional external funding received for this study. The funders played no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Remaining up-to-date with evidence-based practices within and across neurological subspecialties is increasingly challenging, due to the growing and sometimes conflicting body of evidence. Clinical practice guidelines (CPG) are systematically created documents that contain recommendations to assist health professionals optimize care, based on the evidence.
In many settings, CPGs can improve care by improving processes of care and clinical outcomes.[2–4] Despite evidence of these potential benefits of CPGs in other settings, there is little evidence that this is the case in neurology. For example, the benefits of using CPGs in a common neurological condition, epilepsy, have not been observed, likely due to poor implementation of CPGs in clinical practice.[5,6] It is hypothesized that some determinants of CPG use for the care of people with neurological conditions are similar to those for CPG use in general. Namely, individuals’ lack of awareness, familiarity, agreement, and motivation; environmental factors; and CPG-related factors. However, the reason for the poor adherence to CPGs in neurology remains elusive. Applying knowledge translation (KT) methodologies and strategies to CPGs presents an opportunity to improve dissemination and implementation of CPGs. KT is the exchange, synthesis, and application of knowledge among researchers and users to accelerate the capture of the benefits of research.
The objective of this study was to identify barriers and facilitators (determinants) to the use of CPGs among Canadian neurologists. The determinants of CPG use among neurologists were then linked to theoretical and evidence-based behavior change constructs (i.e., the Theoretical Domain Framework [TDF]) to provide evidence-informed and theory-based suggestions for improving the use of CPGs. It is hypothesized that CPG use will be low and that novel determinants of CPG use will be identified.
Materials and methods
A mixed methods study design (quantitative survey and qualitative focus groups and interview) was used to explore the determinants of CPGs use among neurologists.
Population-based survey (quantitative)
A population-based, cross sectional survey of Canadian neurologists was conducted between April and November 2015. Demographic variables and variables related to determinants of CPG use in clinical practice were collected (S1 File). The survey was developed by the study authors based on the TDF for behavior change, which has been validated and used in similar studies (i.e., to identify barriers and facilitators to the use of CPGs in Australia).[11,12] Survey responses were dichotomous, continuous (seven-point Likert scale), or open text box.
The survey was pilot tested for face validity using a convenience sample of ten neurologists. Neurologists that participated in the pilot test provided feedback on: time to complete the survey, ease of the survey, and perceived objectives of the survey through open-ended questions at the end of the survey. The mean length of time to complete the survey was seven minutes. All participants rated the survey as moderately easy or very easy to complete, and correctly identified the objective of the survey.
Since Canada is a bilingual country (English and French) survey documents were provided to neurologists in their preferred language to minimize response bias based on language. The survey was developed in English and translated to French by an experienced translator who is also a neuroscience nurse.
All neurologists practicing in Canada were invited to participate in the study. Neurologists were excluded if they were retired, on sabbatical/leave (maternity or other), or no longer practicing in Canada.
Contact information (excluding email) for neurologists practicing in Canada was obtained from the Canadian Medical Directory. A modified Dillman method, using a minimum of three modes of contact (mail, email, fax, phone) was employed to ascertain participants.
The survey was initially distributed by mail to all neurologists. Neurologists were offered four methods to return the survey (mail, fax, email or electronically via Adobe Form Central) and were given four weeks to complete the survey, after which a reminder was sent either via email (if publically available) or fax. Neurologists were given another four weeks to complete the survey, after which they were contacted by phone to obtain their correct contact information, and their preferred contact method for the subsequent survey distribution.
Descriptive statistics were used to describe the respondent characteristics and determinants of CPG use. Non-responders were compared to responders. All comparisons were done using Students T-tests, or X2 when the data were non-parametric.
Regression analysis was used to examine the association between demographic factors (sex, age, years of practice, academic affiliation, urban or rural practice) and the determinants of CPG use, and between CPG users and non-users. Linear regression was employed to examine relationships between determinants of CPG use that were rated on the seven-point Likert scale (Likert scales were treated as a continuous variable). Logistic regression was used to examine relationships between determinants that were evaluated using dichotomous responses. Variables that were found to be significantly different between CPG users and non-users were included in the regression model to control for effect measure modifiers and confounders.
Using data from the responders, the proportion of CPG use and determinants of CPG use were stratified based on demographic profiles (years of practice, urban or rural, general or subspecialty, academic affiliation) of “responders” and “non-responders” to help quantify the direction and magnitude of potential response biases.
All quantitative data analysis was conducted using STATA 12. For all tests of significance, a p-value of <0.05 was considered statistically significant. In instances where there were multiple comparisons, an a priori decision was made to use a Bonferonni correction.
Qualitative data were analyzed according to the qualitative data analysis methods outlined below.
Focus groups and interview (qualitative)
A phenomenological approach was used to understand and conceptualize how the phenomena (CPGs) are perceived by neurologists. Based on our objective to expand our understanding of factors that influence the use of CPGs and to identify any novel determinants that were not probed using the quantitative survey among neurologists (a heterogeneous group), focus groups and interview were conducted.[15–17]
Participants were ascertained from survey respondents and were purposefully sampled based on survey responses. Participants were selected to be representative of the population on: age, years of practice, geographical location, patient population (adult vs. pediatric, and general vs. subspecialty), and setting (academic affiliation and rural vs. urban). All participants completed a written consent form prior to the focus group or interview.
A semi-structured, iterative script based on the TDF was used to guide the focus groups and interview (S2 File). The initial script was tailored to preliminary survey results. The script for the subsequent focus group and interview was tailored to the results of the initial focus group to attempt to elicit novel responses.
The same experienced facilitator moderated all focus groups and the interview. An assistant facilitator took notes and validated the transcript and analyses. A combination of a face-to-face focus group, an online/webinar focus group, and an interview were employed to maximize participation of neurologists from across the country (six time zones). Data from different methods of focus groups can be combined and is an acceptable means of data collection. All focus groups and the interview were audio recorded and transcribed verbatim.
A phenomenological approach was taken to identify themes in the lived experience of neurologists using CPGs in their practice.[19,20] The facilitator transcribed all audio recordings and the assistant facilitator reviewed the transcripts for accuracy. Deductive qualitative analysis was used. The codes or “nodes” in NVivo (Mac version 10.2.2) were informed by two previously published studies that examined determinants of CPG use and behaviour change (S3 File).[7,22] Data were coded into the evidence-informed nodes, which were then thematically analyzed to arrive at themes of determinants of CPG use.
The facilitator and an additional analyst with qualitative expertise independently coded the transcripts using standardized code definitions. The two analysts compared coding to ensure consistency and agreed on the themes. Quantitative data analysis was used to determine the frequency with which each theme was endorsed and the number of participants that endorsed each theme.
The two analysists were identified as subject matter experts based on their expertise in the area of KT and CPGs, or qualitative methods. Saturation was defined as the absence of additional themes and was discussed after each focus group/interview. Saturation was determined through consensus among the subject matter experts.
Knowledge translation strategy
Implementations strategies informed by theory may improve the success of the implementation;[23,24] therefore, the Knowledge to Action (KTA) framework was chosen for this study. Using the KTA as our framework, the TDF which was developed based on a literature review of psychological theories relevant to behavior change and consensus process among experts in the field, was employed to classify the identified determinants of CPG use to facilitate mapping the determinants to the Behaviour Change Wheel. The Behaviour Change Wheel is based on the TDF and maps behaviour change interventions to determinants of CPG use in order to tailor the KT strategies.
Population-based survey (quantitative)
The response rate for the national survey was 38.7% (n = 311; Fig 1). Typically, participants had been in practice for 16.6 years (SD = 12.2), practiced in an urban area, had an academic affiliation, and were subspecialized (Table 1). Non-responding neurologists differed from the responders on all variables except for sex and language (Table 1).
Participants were first contacted by mail, then email or fax. Participants that did not respond were called and asked to provide their preferred method of contact.
Of the respondents, 76.5% indicated they used CPGs in their practice. The most commonly reported CPGs used by the respondents are listed in S4 File. The median response on the seven-point Likert scale to the statement “I use CPGs in my clinical practice” was 5 (IQR = 4). Sex (52.2% of males vs. 26.9% of females, p = 0.01) and academic affiliation (academic affiliation = 69.5% vs. no affiliation = 9.3%, p = 0.02) were associated with CPG use (estimated power = 1.0).
Table 2 lists the determinants of CPG use examined. Overall, having the skills and resources to adhere to the recommendations in the CPGs, were not identified as barriers to the use of CPGs (4.1% and 13.8% responded yes, respectively). Conversely, lack of knowledge of CPGs was commonly (41.2%) endorsed as a barrier, as was time constraint (38.7%), and the applicability of CPGs to clinical practice (34.8%). Many of these determinants were corroborated when asked why they do not use CPGs (free-text box in the questionnaire). The barriers that were identified qualitatively included the following CPG characteristics: poor applicability (n = 24), unnecessary or unhelpful (n = 22), lack of credibility (n = 11), insufficient benefit (n = 11), unawareness or unfamiliarity with CPG (n = 7), rigidity of CPGs (n = 7), lack of accessibility (n = 5), outdated CPG (n = 5), impractical (n = 4), and time or resource constraints (n = 3).
Survey items probing determinants of CPG use in clinical practice (seven-point Likert scale ranging from barrier = 0 to facilitator = 7) were rated highly (median = 5), indicating that these variables were facilitators rather than barriers to CPG use (Table 2). The determinant rated as the greatest facilitator asked about the influence of CPGs on clinical practice (median = 6, IQR = 3). The item that had the lowest score asked about the benefits and cost of using CPGs (median = 4, IQR = 3), indicating that the benefit did not outweigh the cost of using CPGs (S1 Fig.).
Regression analysis (controlling for sex and academic affiliation since they were associated with CPG use) revealed that applicability of CPGs was the only determinant that differed between CPG non-users and CPG users (60.3% vs. 28.6%, p<0.001, S1 Table). However, CPG non-users consistently rated all determinants of CPG use lower than CPG users (Table 2).
Stratification of participants based on demographic characteristics of “responders” and “non-responders found a trend towards less CPG use among “non-responders”, but there were no differences in determinants of CPG use between the two groups (S2 Table).
Focus groups and interview (qualitative)
The first focus group was conducted in a face-to-face setting (n = 8), the second focus group was conducted using webinar software (n = 2), and the interview was conducted via telephone. The majority of focus group participants practiced in an urban setting (90.9%), had an academic affiliation (72.7%), were subspecialists (63.6%), and saw a predominantly adult population (63.6%). The mean number of years the participants were in practice was 25 year (SD = 13.4). Eighty-one percent (81%) of participants reported using CPGs in their clinical practice.
From the initial 33 nodes deduced from the literature (S3 File) our data identified six themes related to determinants of CPG use: credibility, knowledge, motivation, applicability, resources, and target audience (Table 3).
Three common discussion points related to credibility included the rigor of the development process, the reputation of the organization developing and endorsing the CPG, and the CPG being up-to-date.
Many of the references to knowledge were related to awareness of and access to CPGs, especially in a timely manner (i.e., at point of care and when they are still up-to-date). Access to CPGs was also included within the resources theme, such that in some settings the resources to access CPGs were not available, especially at the point of care. Other issues related to resources included the time required to provide the recommended care outlined in CPGs.
The discussion around applicability was commonly focused on the applicability to the complex, sub-population of patients commonly seen; and also to the practice setting. Discussion around applicability was linked to the discussion around the target audience, in that CPGs were better targeted to care settings where the CPG are appropriate to the patient population. The motivation for using or seeking a CPG was often in response to a patient that fell outside of the general area of the neurologist’s practice, and in cases where it was felt that not following a CPG’s recommendations could result in penalties.
Knowledge translation strategy
The evidence-based determinants of CPG use among neurologists (credibility, knowledge, motivation, applicability, resources, target audience) were linked to the Behavioural Change Wheel using the TDF (S2 Fig and S5 File) to guide the proposed KT strategy (Fig 2).[10,22] The proposed KT strategy was structured according to the KTA framework and has three main pillars: 1) CPG development, 2) implementation and 3) interventions; each of which targets many determinants of CPG use identified in our study (Fig 2 and S5 File).
Each of the 3 pillars of the KT strategy are outlined in colours that each pillar addresses, that correspond to the determinants of CPG use that have been identified. The need to evaluate knowledge use and clinical outcomes is highlighted by the bidirectional nature of the arrows.
A detailed description of the role of each component of the KTA in the proposed KT strategy is provided in S6 File.
This study provides a national evaluation of determinants of CPG among neurologists and provides a multi-dimensional, evidence-informed KT strategy to improve the use of CPGs in clinical practice. Insight into the unique barriers and facilitators of CPG use in day-to-day practice among neurologists were identified (credibility, applicability, target audience), along with the strength of their association to CPG use.
Of the six themes identified here (credibility, knowledge, motivation, applicability, resources, and target audience) some were previously reported in other clinical settings, such as knowledge, which is the most commonly cited determinant.[7,22] Importantly, our study identified novel determinants that may be specific to CPG use among neurologists (credibility, applicability, and target audience). Also, our study provides quantification of which determinants are most strongly associated with implementation of CPG recommendations among neurologists (strength of association).
Applicability (to the patient population, clinical setting, and end-user) has not been frequently reported as a barrier to CPG use, but was the barrier that was most strongly associated with CPG use in the quantitative component of our study. Applicability may be unique to neurologists because a high proportion of neurologists subspecialize and thus encounter complex patient populations that require individualized treatment where there is limited evidence to guide care, rendering CPG recommendations inapplicable. Applicability as a barrier to CPG use is echoed in a recent systematic review of CPGs for epilepsy in which the applicability of existing CPGs was poor according to the AGREE II tool. Improving CPG development methodology may present an opportunity to improve the applicability of CPGs.
Credibility was associated with CPG use in both the qualitative and quantitative data. Factors related to credibility as a barrier included biased synthesis and interpretation of the evidence, and lack of rigorous methodologies. Conversely, if the CPG was developed and disseminated by a reputable professional organization, it facilitated CPG use. For example, among our sample of neurologists, CPGs developed by the American Academy of Neurology were noted as credible and were cited as the CPGs more commonly used in clinical practice.
The present study used a mixed-methods design, which was a strength and provided rich understanding of the determinants of CPG use and the ability to examine congruency between the quantitative and qualitative data, as a means of validating each data set. Congruency between the quantitative and qualitative data was observed between many of the variables and themes in this study.
While the design is a study strength, there are also limitations to consider when interpreting our findings. First, we achieved a response rate of nearly 40%, which is arguably low, but higher than reported in other studies. Our ascertainment methods also yielded enough cases to achieve adequate power to examine the strength of the associations between CPG use and determinants of their utilization–a unique aspect of this study. Secondly, demographic differences between responders and non-responders were identified, indicating a potential bias. These differences could partially be explained by the fact that the demographic data for non-responders were taken from a database, while data for the responders were self-reported. Based on the analysis comparing demographic profiles of non-responders to demographic profiles of responders, we identified that our findings present a moderately optimistic picture of CPG use among neurologists–CPG use would likely be slightly lower among those who did not respond but determinants would be similar. This potential bias could limit the generalizability of our data beyond academic, subspecialty neurologists practicing in an urban area with more than 17 years of practice. A recent national survey of neurologists demonstrated that only a small portion of neurologists are in private, non-academic practices suggesting our sample represents the majority of Canadian neurologists. However, this minority were more likely to not respond and are an important target audience for CPG use, which highlights a gap in our knowledge of CPG use and determinants of CPG use among general neurologists with many years of experience in community practice in rural areas. Future research should aim to find methods to engage this population to identify CPG use and determinants of CPG use among this group.
While CPGs are not always viewed favorably among physicians, they have the capacity to improve quality of care when effectively implemented.[2,4] For example, mortality was reduced and treatment efficacy was increased among those who were compliant with stroke CPGs. However, CPGs are often not implemented or adopted in day-to-day practice in neurology.[5,6,32] Developing a CPG is a resource intensive undertaking that can be wasteful if recommendations within CPGs are not adopted into clinical practice. This study provides a multi-faceted, tailored KT strategy for implementing CPGs among neurologists; these factors have been found to increase the effectiveness of KT strategies.[23–25,33] Improving implementation of recommendations within high quality CPGs through KT interventions may decrease variation in practice, consequently improving the quality of care provided to patients with neurological conditions.
S1 File. Survey of neurologists regarding barriers and facilitators to using clinical practice guidelines (English and French).
S3 File. Code book for qualitative analysis of focus groups.
S4 File. Clinical practice guidelines used by neurologists.
S5 File. Expanded mapping of determinants of guideline use among neurologists to Theoretical Domains Framework and intervention functions to the proposed knowledge translation strategy.
S6 File. Detailed knowledge translation strategy.
S1 Fig. Determinants of clinical practice guideline use by neurologist.
S2 Fig. Mapping determinants of guideline use among neurologists to Theoretic Domains Framework and intervention functions.
S1 Table. Factors associated with guideline use among neurologists.
The authors would like to acknowledge Ms. Rachel Crooks’ assistance with transcription of the focus group audio recordings and analysis.
- 1. Clinical Practice Guidelines We Can Trust. Graham R, Mancher M, Wolman DM, Greenfield S, Steinberg E, editors: The National Academies Press; 2011.
- 2. Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet. 1993;342:1317–22. pmid:7901634
- 3. Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess. 2004;8:iii–iv, 1–72.
- 4. Lugtenberg M, Burgers JS, Westert GP. Effects of evidence-based clinical practice guidelines on quality of care: a systematic review. Qual Saf Health Care. 2009;18:385–92. pmid:19812102
- 5. Davis J, Roberts R, Davidson D, Norman A, Oqston S, Grimshaw J et al. Implementation strategies for a Scottish natioal epilepsy guideline in primary care: results of the Tayside Implementation of Guidelines in Epilepsy Randomized (TIGER) trial. Epilepsia. 2004;45:28–34.
- 6. Haneef Z, Stern J, Dewar S, Engel J Jr. Referral pattern for epilepsy surgery after evidence-based recommendations: a retrospective study. Neurology. 2010;75:699–704. pmid:20733145
- 7. Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PA et al. Why don't physicians follow clinical practice guidelines? A framework for improvement. Jama. 1999;282:1458–65. pmid:10535437
- 8. Wiebe S. Still and elusive target. Guiding practice for epilepsy surgery. Neurology. 2010;75:678–9. pmid:20733142
- 9. CIHR. [cited 2014]. Available from: http://www.cihr-irsc.gc.ca/e/39033.html.
- 10. Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005;14:26–33. pmid:15692000
- 11. Cane J, O'Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7:37. pmid:22530986
- 12. French SD, Green SE, O'Connor DA, McKenzie JE, Francis JJ, Michie S et al. Developing theory-informed behaviour change interventions to implement evidence into practice: a systematic approach using the Theoretical Domains Framework. Implement Sci. 2012;7:38. pmid:22531013
- 13. Dillman D. Mail and Telephone surveys. The total design method. 2nd ed. Hoboken NJ: John Wiley & Sons, Inc; 2000.
- 14. StataCorp. Stata Statistical Software: Release 11. College Station, TX: StataCorp LP; 2011.
- 15. Basch CE. Focus group interview: an underutilized research technique for improving theory and practice in health education. Health Educ Q. 1987;14:411–48. pmid:3319971
- 16. Green J, Thorogood N. Qualitative Methods for Health Research. London, UK: Sage; 2014.
- 17. Krueger RA, Casey MA. Focus Groups: A Practical Guide for Applied Research. London, UK: Sage; 2009.
- 18. Schneider SJ, Kerwin J, Frechtling J, Vivari BA. Characteristics of the Discussion in Online and Face-to-Face Focus Groups. Social Science Computer Review. 2002;20:31–42.
- 19. Giorgi A. The theory, practice, and evaluation of the phenomenological method as a qualitative research procedure. Journal of Phenomenological Psychology. 1997;28:235.
- 20. Creswell JW. Qualitative Inquiry and Research Design. 3rd ed. Washington, DC: SAGE; 2013.
- 21. QIP Ltd. NVivo qualitative data analysis Software. 10.2.2 ed2014.
- 22. Michie S, van Stralen MM, West R. The behaviour change wheel: A new method for characterising and designing behaviour change interventions. Implementation Science. 2011;6:42. pmid:21513547
- 23. Glanz K, Bishop DB. The role of behavioral science theory in development and implementation of public health interventions. Annu Rev Public Health. 2010;31:399–418. pmid:20070207
- 24. Taylor N, Conner M, Lawton R. The impact of theory on the effectiveness of worksite physical activity interventions: a meta-analysis and meta-regression. Health Psychology Review. 2012;6:33–73.
- 25. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, et al. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. 2006;26:13–24. pmid:16557505
- 26. Sauro K, Wiebe S, Dunkley C, Janszky J, Kumlien E, Moshe SL, et al. The current state of epilepsy guidelines: A systematic review. Epilepsia. 2015;57:13–23. pmid:26659723
- 27. Fielding NG. Triangulation and Mixed Methods Designs: Data Integration with New Research Technologies. J Mixed Meth Res. 2012;6:124–36.
- 28. Cunningham CT, Quan H, Hemmelgarn B, Noseworthy T, Beck CA, Dixon E, et al. Exploring physician specialist response rates to web-based surveys. BMC Med Res Methodol. 2015;15:32. pmid:25888346
- 29. Canadian Medical Association; [November 1, 2016]. Available from: https://www.cma.ca/Assets/assets-library/document/en/advocacy/profiles/neurology-e.pdf.
- 30. Farquhar C, Kofa E, Slutsky J. Clinicians' attitudes to clinical practice guidelines: a systematic review. Med J Aust. 2002;177:502–6. pmid:12405894
- 31. Micieli G, Cavallini A, Quaglini S. Guideline compliance improves stroke outcome: a preliminary study in 4 districts in the Italian region of Lombardia. Stroke. 2002;33:1341–7. pmid:11988613
- 32. Schroder S, Kuessner D, Arnold G, Zoliner Y, Jones E, Schaefer M. Do neurologists in Germany adhere to the national Parkinson's disease guideline? Neuropsychiatr Dis Treat. 2011;7:103–10. pmid:21552312
- 33. Oxman AD, Thomson MA, Davis DA, Haynes RB. No magic bullets: a systematic review of 102 trials of interventions to improve professional practice. CMAJ. 1995;153:1423–31. pmid:7585368