Figures
Abstract
Objectives
Scientific knowledge can help develop interventions that improve public health. The objectives of this review are (1) to describe the status of research on knowledge transfer strategies in the field of complex social interventions in public health and (2) to identify priorities for future research in this field.
Method
A scoping study is an exploratory study. After searching databases of bibliographic references and specialized periodicals, we summarized the relevant studies using a predetermined assessment framework. In-depth analysis focused on the following items: types of knowledge transfer strategies, fields of public health, types of publics, types of utilization, and types of research specifications.
Results
From the 1,374 references identified, we selected 26 studies. The strategies targeted mostly administrators of organizations and practitioners. The articles generally dealt with instrumental utilization and most often used qualitative methods. In general, the bias risk for the studies is high.
Citation: Dagenais C, Malo M, Robert É, Ouimet M, Berthelette D, Ridde V (2013) Knowledge Transfer on Complex Social Interventions in Public Health: A Scoping Study. PLoS ONE 8(12): e80233. https://doi.org/10.1371/journal.pone.0080233
Editor: Wolfgang Glanzel, Katholieke Universiteit Leuven, Belgium
Received: May 3, 2013; Accepted: October 1, 2013; Published: December 4, 2013
Copyright: © 2013 Dagenais et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This study was supported by a grant from the Canadian Institute of Health Research: RN # 74560. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Whether they are endeavoring to improve public health or reduce health-related social inequities, public health officials suggest and initiate increasingly complex interventions [1]. Indeed, the problems they face are most often multidisciplinary because of the diverse social determinants of health [2]. These problems therefore require responses that are adapted to local contexts and involve the participation of a number of people. They differ from clinical interventions, for which the objective is to prevent and treat illness in individuals and which can be handled in a standardized manner [1]. These complex social interventions: are based on the presumption that they will produce better results than standard individual interventions; involve the action of several players in the field; consist of a chain process involving several professionals and adapt to the social context in which they occur. The implementation of these interventions is not linear. It uses the bottom-up or top-down model and offers the ability to return to earlier stages of the implementation to adjust and adequately meet the needs of the supporting environment [3]–[5]. To illuminate the complexity of these interventions their characteristics are illustrated in Table 1 [6] by a concrete example based on recent policies in Africa intended to eliminate direct payment for health care. Given the nature of these interventions, knowledge transfer (KT) in such situations is challenging.
The popularity of KT has been growing since the 1880s, especially in the health sector. For example, the Canadian Institute of Health Research now funds projects that promote the use of research-based knowledge by potential users. There have also been many systematic and critical reviews of the literature. They generally deal with transfer of knowledge from clinical research, such as the efficiency of strategies that promote knowledge use [7]–[8], including practice guidelines [9]–[10]. None of these critical reviews deal with KT from research involving complex social interventions in public health, despite the major challenge in encouraging interventions based on convincing evidence.
The rapid development of the research field on the use of scientific knowledge has manifested itself in the emergence in recent decades of numerous terms to refer to the concept of “knowledge to action”. In a study of 33 funding organizations in 9 countries, Graham and his colleagues [11] identified 29 different terms used including knowledge transfer, knowledge translation and knowledge mobilization, among others. Most of these terms have been “promoted” by research funding agencies. For example, the Canadian Institutes for Health Research are using the term “knowledge translation”, while the Social Sciences and Humanities Research Council used instead “knowledge mobilization”. Although the definition of these terms may sometimes vary, different words are mostly used to designate more or less the same thing. To avoid this confusion, some organisations are now using the term K* (K star: for knowledge whatever…). In this study, we use the term knowledge transfer (KT), which is still the most frequently used term.
The knowledge transfer strategies are ultimately aimed at the use of knowledge. This study considers three types of knowledge use: instrumental utilization (i.e., changes in behavior or practice), conceptual utilization (i.e., changes in understanding or attitude), and persuasive utilization (i.e., arguments used to influence policies or practices) [12].
With a view to guiding KT research in the field of complex social interventions in public health, we examined existing literature on this topic to obtain an overview of the knowledge available. More precisely, we looked at two distinct aspects: (a) KT strategies and the way they are assessed and (b) the manner in which knowledge utilization is measured. The objectives of this scoping study are (1) to describe the status of research on knowledge transfer strategies in complex social interventions in the public health field and (2) to identify priorities for future research.
Methods
Our critical review of scientific literature was conducted as a scoping study [13]. This type of review differs from systematic reviews in that the intention is to obtain an overall picture of an issue or field of research in order to assess the feasibility of a systematic review and guide future research: it does not assess the effectiveness of an intervention. In this study, the choice to include or exclude a study was based not on the research specifications, but rather on relevance to the topic in question [13]. We also included gray literature. Our study was divided into four steps: 1) study identification; 2) choice and application of selection criteria; 3) data classification and 4) data analysis.
The review was carried out in accordance with a protocol developed in advance (http://equiperenard.ca/fr/protocole.html). The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist for this paper is presented as Table S1 (supporting information S1).
Study Identification
First, the following public health databases were consulted: MEDLINE, ERIC, PsycINFO, CINAHL and the French public health database, Banque de données santé publique. The search in each of these databases was limited to the period from 1960 to October 2010. Key words included the term “knowledge transfer” and its alternative expression as listed in Graham et al. [11], in English and French. The search strategy adopted for the various databases is described in Table S2 (Supporting information S2). The Tables of Contents of two periodicals, Evidence & Policy (2005 to January 2011) and Implementation Science (2006 to January 2011) were also thoroughly explored. Next, a snowball strategy was applied to the bibliographies of articles identified in the databases and selected for analysis (the inclusion criteria are provided below). This combination of research strategies was used to find the maximum number of documents on the topic.
Selection Criteria
The studies selected 1) focused on KT in public health; 2) dealt with the use of research-based knowledge; 3) covered complex social interventions in public health; 4) provided empirical data; 5) were written in English or French; and 6) were published in or after 1960, the decade in which the concept of KT appeared in scientific documentation [14]. Moreover, since the objective was to obtain a complete picture of this field of research, the type of study specification was not considered as a criterion for inclusion or exclusion.
With these criteria in mind, a first sort was done by consulting titles, then document abstracts. Two reviewers worked independently on this sort on the references obtained from the databases. The reviewers agreed 95.7% of the time. Differences were resolved by consensus. A second sort was then done in which the texts selected during the first sort were read in their entirety.
Data Classification
The articles selected were distributed to three members of the research team who classified the data and then checked all their work. For the specific requirements of this study, an assessment framework was developed [15]. For this framework, the team members looked at the following information: authors, publication year and location, sample type, types of knowledge users, fields of practice, research goals, methods, measuring instruments and important results. The Mixed Method Appraisal Tool (MMAT) developed by Pluye and his collaborators [16] (Table S3: Supporting information S3) was used to evaluate the quality of the articles selected on the basis of the method used. This tool links five types of research specifications (e.g. qualitative) with quality-criteria questions (e.g., does the process for analyzing data allow answering the research question?). All of the conclusions from the use of this framework were checked by at least one member of the team and differences were resolved by consensus. Because it is the only assessment framework that combines evaluation of five types of research specifications, quantitative, qualitative or mixed, this instrument appeared to us to be the most suitable for the study context.
Results
Our research identified 1,374 potentially relevant references (see the PRISMA flowchart Figure 1). Based on the titles and abstracts, 341 articles were read for a more in-depth examination. Of these, 26 empirical studies were included.
KT Strategies, Recipients and Users
Table 2 provides the characteristics of the studies on KT strategies. The field and type of study, strategies and type of utilization were examined [17], [32]. For the target publics, we differentiated the recipients—individuals or groups directly targeted by the KT strategy—and the publics concerned by the knowledge in question. Table 3 gives the characteristics of the studies that measure knowledge utilization [33], [42]. We examined the field of public health, the type of study specifications, the type of utilization measured, and the knowledge users.
In the articles assessing KT strategies, (Table 2), most of the strategies focus only on diffusion/dissemination: distribution of documentation (adapted or not), e-mailing, or development of Web sites, for example. These activities are sometimes accompanied by telephone assistance or training workshops. The strategies often involve several dissemination activities: distribution of policy briefs and workshops, for example. Only two articles deal with research made in partnership withwith stakeholders. One study discusses the knowledge transfer activities made by a liaison centre and one evaluates a KT strategy involving a knowledge broker. This strategy is designed to support evidence-based decision-making in the organization, management and execution of health services. The studies generally contain few details about the strategies, so relatively little is known about the knowledge concerned, the underlying conceptual or theoretical bases, or their objectives.
As for the knowledge users targetedtargeted, the two categories most often concerned are administrators or managers of organizations and practitioners. Few strategies specifically address policy makers or researchers. On the other hand, many have an organizational component and are intended to bring about a more systemic change. As for the target publics, they are usually the general public in the United States or Canada. Some of the strategies assessed deal with knowledge relating to preschoolers or school children, but few target women or actual patients.
In the articles that measure research utilization, (Table 3), we identified three main categories of users: administrators or managers (i.e., people in an organization), practitioners, and policy makers. Most of the studies deal with research utilization among policy makers; some with utilization by administrators or managers. One discusses research utilization by practitioners. Table 4 lists the number of studies by user type and target public.
What type of use for which field of public health?
We classified the studies according to the public health field and the type of utilization (Table 5). In general, most of the articles dealt with instrumental utilization, particularly in public health and health promotion. It is interesting to note that some studies specifically in these two areas discuss conceptual utilization. Persuasive utilization is very rarely considered in the studies.
Research specifications and study quality
Tables 6 and 7 show the results produced by the Mixed Method Appraisal Tool developed by Pluye et al. [16]. Table 6 shows the relationship between the research specifications and the three types of research utilization. Regardless of the research specifications, the studies deal mainly with instrumental utilization of knowledge, except for the qualitative studies that also take conceptual utilization into account.
Table 7 shows the research specifications regarding study quality. In decreasing order of frequency, the studies are based on qualitative (9/27), quantitative non-randomized (7/27), quantitative randomized (4/27), mixed methods (4/27) and descriptive (2/27) specifications.
Overall, the quantitative studies are based on data sources and analysis techniques that answer the research question and take the context into account. However, in half of the qualitative studies, the researchers did not specify their a priori considerations or the impacts of these assumptions on interpretation of the results.
Most of the quantitative non-randomized studies include participant recruitment mechanisms that reduce selection bias, have a response rate of at least 60% and explain the comparison between the groups sampled. Over half of the studies use measuring instruments whose psychometric properties are not well-documented.
As for the quantitative randomized studies, half do not provide a clear description of the selection method or attrition information, while three-quarters of them have less than 20% missing data.
In the studies using mixed methods, application of the Pluye et al. [16] criteria shows that the specifications chosen are relevant for meeting the research objectives. In addition, half of the studies explain the integration of the qualitative and quantitative results and only one article does not provide the limits of this integration.
Two studies use a quantitative descriptive specification. The sampling strategies appear relevant for answering the research questions, the samples seem representative of the population and the response rates are greater than 60%.
The measuring instruments are rarely described well enough to judge their quality. More specifically, reliability and validity are rarely reported.
Discussion and Conclusion
Limits of the study
One limitation is related to the study topic: the definition of complex social interventions was particularly difficult to pin down from the literature. On the one hand, this term incorporates two concepts that are still not well-developed, complexity and the social nature of an intervention, which makes it difficult to obtain a precise definition. Articles often comment superficially on the knowledge behind the KT strategy that is being assessed and we sometimes needed to do additional research to determine what this knowledge was. This was not always possible, so we decided to include false positives, i.e., studies for which the knowledge behind the KT strategy does not necessarily involve a complex social intervention. On the other hand, a more restrictive or broader definition of this expression might have made the composition of the pool of articles somewhat different.
With regard to Mixed Method Appraisal Tool analysis of study quality, it would be necessary to contact the authors of the articles when the response to one of the screening questions was incomplete after reading the articles. Owing to lack of resources, we were unable to do this. Our description of quality is therefore based exclusively on the information available in the article and the way in which the authors presented their study.
Status of research on KT and areas for future research
We found relatively few studies dealing with our research topic, KT strategies for complex social interventions in public health. The definition we chose may have reduced the pool of existing studies. These concepts are recent and the definition is still fuzzy. Furthermore, as we have seen, the vast majority of identified studies have been published over the past ten years; Thus, this field of research is recent, not yet well-developed and involves major methodological challenges for researchers. Since not much knowledge is available on the effectiveness of this type of intervention, it is not surprising that research on knowledge transfer to user groups remains limited.
Neither was it surprising that no studies were found on KT strategies for complex social interventions in environmental health or health administration. It is also possible that the search strategy was unable to track down specialized literature in these areas. KT in public health is therefore a field of study that remains to be explored.
In addition to these contextual comments, we observed that most of the included studies deal with instrumental utilization of knowledge, whether relating to assessment of KT strategies or measurement of knowledge use. Despite the methodological challenges involved in this type of study, we can suppose that they are not as great as for studies of conceptual or persuasive utilization. What method(s) should be preferred? What measurements are appropriate? What measuring instrument(s) should be used? What construct(s) should be studied? As Weiss noted in 1977, future research should pay attention to these questions as a number of authors [11], [43], mention the significance of these types of utilization in complex decision making.
Moreover, the effects of a KT strategy on instrumental utilization can be measured in the short term and is evident in observable behavior, even though knowledge use is a process that can continue in the medium and long terms. Researchers may therefore tend to prefer to study this type of use. When other types of utilization are considered, such as conceptual utilization, the effects are not evident in observable behavior and this makes their study more complex. Since the effects of KT strategies on knowledge use can appear in the short, medium and long terms, the choice of the appropriate time to assess a KT strategy also poses a challenge.
The question of what methods to use to assess KT strategies or measure knowledge use is closely tied to these challenges. Few studies use mixed-method design or descriptive quantitative design. Although descriptive quantitative design may impose limitations on evaluation of this type of relatively complex intervention, the use of mixed-method design appears to us to be worth exploring. Given the complexity of KT strategies and the extent of the potential effects, and considering the added value of these methods in enhancing understanding of a phenomenon, [4], [5], it seems obvious to us that mixed-method studies should be used more.
The issue of study quality is also pivotal. Aside from the aforementioned limitations of using the Mixed Method Appraisal Tool, our results clearly show that it is premature to conduct a systematic review, Cochrane-style for example, on this specific study topic. Firstly, few studies could be included in such a systematic review because of their methodological attributes. Secondly, the methodological weakness of most of the studies would limit the scope of the systematic review results. As for measurement of knowledge use, we noted that the studies devote little explanation to the measuring instruments used, which makes it difficult to evaluate the quality of these instruments and, by extension, the quality of the results produced.
Finally, this study has shown the preponderance of evaluations of strategies limited to diffusion/dissemination. Yet thanks to studies of factors that promote knowledge use, we now know that diffusion and dissemination alone have a very limited impact. It is therefore surprising to see the limited number of more complex strategies implemented [43]–[46]. We now know that one-way, push-pull-type strategies are often insufficient to promote knowledge application [10], [47] compared to two-way strategies [48]. There seems to be a gap between existing knowledge about effective strategies and the strategies that are actually implemented. This being said, the task here was to identify studies that evaluate KT strategies, and not to identify the strategies themselves. We can imagine that more elaborate strategies are now being implemented but have not yet been researched. We also wonder whether the methodological challenges and lack of funding are responsible for limiting researchers' interest in this type of intervention.
Finally, with regard to the second objective of this study, we think that future research on KT strategies involving complex social interventions in public health should consider that::
- Some reflection is required regarding the methodological issues raised by KT studies in general and more specifically on measurement of conceptual and persuasive utilization of knowledge.
- We encourage the organizations involved in KT to describe the conceptual and theoretical bases of their KT strategies. This would help researchers understand the logic behind the KT strategies and assess their plausibility, relevance and validity, which is rarely done.
- Future research should focus on two aspects: assessment of KT strategies used in complex social interventions with explicit descriptions in the articles of what knowledge is involved; and assessment of more complex, elaborate KT strategies (when they exist), not just diffusion/dissemination strategies.
- Lastly, a brief look at the Web shows a lot of activity surrounding knowledge transfer and a multitude of strategies of all kinds that are being used in the world. However, although much has been written on the theory of the potential effects of these strategies and the factors that promote their use, convincing evidence remains limited [43] [49]–[52]. Consequently, we think researchers should endeavor to develop assessment projects for various KT strategies and publish their results so that this knowledge can be utilized.
Author Contributions
Conceived and designed the experiments: CD MM ER MO DB VR. Performed the experiments: CD MM ER. Analyzed the data: CD MM ER MO VR. Wrote the paper: CD MM ER. Involved in critically revising the manuscript for important intellectual content: CD MO VR. Gave final approval of the version to be published: CD MM ER MO DB VR.
References
- 1. Rychetnik L, Frommer MS, Hawe P, Shiell A (2002) Criteria for evaluating evidence on public health interventions. J Epidemiol Community Health 56: 119–127 Available: http://jech.bmj.com/cgi/doi/10.1136/jech.56.2.119.
- 2. Marmot M (2005) Social determinants of health inequalities. Lancet 365: 1099–2104
- 3. Pawson R, Greenhalgh T, Harvey G, Walshe K (2005) Realist review–a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy 10 Suppl 1: 21–34
- 4.
Pawson R (2006) Evidence-based policy. A realist perspective. London: Sage Publications.
- 5. Egan M, Bambra C, Petticrew M, Whitehead M (2009) Reviewing evidence on complex social interventions: appraising implementation in systematic reviews of the health effects of organisational-level workplace interventions. J Epidemiol Community Health 63: 4–11
- 6. Ridde V, Robert É, Guichard A, Blaise P, Van Olmen J (2012) L'approche REALIST à l'épreuve du réel de l'évaluation des programmes. The Canadian ournal of Program Evaluation 26, 3: 37–59.
- 7. Brown JD (2004) Challenges facing Canadian federal offenders newly released to the community: a concept map. J Offender Rehabil 39: 19–35.
- 8. Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, et al. (1998) Getting research findings into practice: Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. Brit Med J 317: 465–468
- 9. Davis DA, Taylor-Vaisey A (1997) Translating guidelines into practice. A systematic review of theoretic concepts, practical experience and research evidence in the adoption of clinical practice guidelines. Can Med Assoc J 157: 408–416.
- 10. Grimshaw JM, Thomas RE, Maclennan G, Fraser C, Ramsay CR, et al. (2004) Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 4: 6.
- 11. Graham ID, Logan J, Harrison MB, Straus SE, Tetroe JM, et al. (2006) Lost in knowledge translation: time for a map? J Contin Educ Health Prof 26: 13–24 Available: http://www.ncbi.nlm.nih.gov/pubmed/16557505.
- 12. Straus SE, Tetroe JM, Graham ID, Zwarenstein M, Bhattacharyya O, et al. (2010) Monitoring use of knowledge and evaluating outcomes. CMAJ 182: E94–8 Available: http://www.ncbi.nlm.nih.gov/pubmed/20083566.
- 13. Arksey H, O'Malley L (2005) Scoping studies: towards a methodological framework. Int J Soc Res Methodol 8: 19–32
- 14. Estabrooks CA, Thompson DS, Lovely JJE, Hofmeyer A (2006) A guide to knowledge translation theory. J Contin Educ Health Prof 26: 25–36
- 15.
Malo M, Robert E (2011) Proposition: canevas pour une Scoping Study. Available: http://equiperenard.ca/fr/grille-danalyse.html
- 16.
Pluye P., Robert E, Cargo M, Bartlett G, O'Cathain A, et al.. (2011) Proposal: a mixed methods appraisal tool for systematic mixed studies reviews. Available: http://mixedmethodsappraisaltoolpublic.pbworks.com/f/MMAT%202011%20criteria%20and%20tutorial%202011-06-29.pdf
- 17. Adily A, Black D, Graham ID, Ward JE (2009) Research engagement and outcomes in public health and health services research in Australia. Aust N-Z J Public Health 33: 258–261
- 18. Armstrong R, Waters E, Crockett B, Keleher H (2007) The nature of evidence resources and knowledge translation for health promotion practitioners. Health Promot Int 22: 254–260
- 19. Brownson RC, Ballew P, Brown KL, Elliott MB, Haire-Joshu D, et al. (2007) The effect of disseminating evidence-based interventions that promote physical activity to health departments. Am J Public Health 97: 1900–1907
- 20. Dobbins M, Hanna SE, Ciliska D, Manske S, Cameron R, et al. (2009) A randomized controlled trial evaluating the impact of knowledge translation and exchange strategies. Implement Sci 4: 61
- 21. Kelly JA, Somlai AM, DiFranceisco WJ, Otto-Salaj LL, McAuliffe TL, et al. (2000) Bridging the gap between the science and service of HIV prevention: transferring effective research-based HIV prevention interventions to community AIDS service providers. Am J Public Health 90: 1082–1088.
- 22. Klein JD, Allan MJ, Elster AB, Stevens D, Cox C, et al. (2001) Improving adolescent preventive care in community health centers. Pediatr 107: 318–327
- 23. Kothari A, Birch S, Charles C (2005) “Interaction” and research utilisation in health policies and programs: does it work? Health Policy 71: 117–125
- 24. Lewis E, Mayer JA, Slymen D, Belch G, Engelberg M, et al. (2005) Disseminating a sun safety program to zoological parks: the effects of tailoring. Health Psychol 24: 456–462
- 25. Lia-Hoagberg B, Schaffer M, Strohschein S (1999) Public health nursing practice guidelines: an evalution of dissemination and use. Public Health Nurs 16: 397–404
- 26.
Manske SR (2001) Explaining knowledge use among clients of the Program Training & Consultation Centre. Thesis, Toronto: University of Toronto.
- 27. Masuda JR, Robinson K, Elliott S, Eyles J (2009) Disseminating chronic disease prevention “to or with” Canadian public health systems. Health Educ Behav 36: 1026–1050
- 28. McCormick L, Tompkins NO (1998) Diffusion of CDC's guidelines to prevent tobacco use and addiction. J Sch Health 68: 43–45
- 29. McFarlane WR, McNary S, Dixon L, Hornby H, Cimett E (2001) Predictors of dissemination of family psychoeducation in community mental health centers in Maine and Illinois. Psychiatr Serv 52: 935–942
- 30. McVey G, Gusella J, Tweed S, Ferrari M (2009) A controlled evaluation of web-based training for teachers and public health practitioners on the prevention of eating disorders. Eat Disord 17: 1–26
- 31. Naylor P-J, Macdonald HM, Reed KE, McKay HA (2006) Action Schools! BC: a socioecological approach to modifying chronic disease risk factors in elementary school children. Prev Chronic Dis 3: A60.
- 32. Schinke S, Brounstein P, Gardner S (2002) Science-Based Prevention Programs and Principles, 2002. Effective Substance Abuse and Mental Health Programs for Every Community Available: http://www.eric.ed.gov/ERICWebPortal/recordDetail?accno=ED474651. Accessed 24 January 2013.
- 33. Aarons GA, Sommerfeld DH, Walrath-Greene CM (2009) Evidence-based practice implementation: the impact of public versus private sector organization type on organizational support, provider attitudes, and adoption of evidence-based practice. Implement Sci 4: 83
- 34. Dobbins M, Thomas H, O'Brien MA, Duggan M (2004) Use of systematic reviews in the development of new provincial public health policies in Ontario. Int J Technol Assess Health Care 20
- 35. Ir P, Bigdeli M, Meessen B, Van Damme W (2010) Translating knowledge into policy and action to promote health equity: the Health Equity Fund policy process in Cambodia 2000–2008. Health Policy 96: 200–209
- 36. Lavis JN, Ross SE, Hurley JE (2002) Examining the role of health services research in public policymaking. Milbank Q 80: 125–154
- 37. Mackenzie M, Blamey A, Hanlon P (2006) Using and generating evidence: policy makers' reflections on commissioning and learning from the Scottish Health Demonstration Projects. Evid Policy: J Res Debate Pract 2: 211–226
- 38. Oh CH, Rich RF (1996) Explaining use of information in public policymaking. Knowl Policy 9: 3–35
- 39. Ouimet M, Landry R, Ziam S, Bédard P-O (2009) The absorption of research knowledge by public civil servants. Evid Policy: J Res Debate Pract 5: 331–350
- 40.
Patton MQ, Grimes PS, Guthrie CM, Brennan NJ, Dickey French B, et al.. (1975) In search of impact: an analysis of the utilization of federal health evaluation research. Minneapolis: Center for Social Research, University of Minnesota.
- 41. Toomey TL, Tramel S, Erickson DJ, Lenk KM (2009) Use of research-based information among leaders of public health agencies. Am J Health Educ 40: 66–70.
- 42. Waddell C, Lavis JN, Abelson J, Lomas J, Shepherd CA, et al. (2005) Research use in children's mental health policy in Canada: maintaining vigilance amid ambiguity. Soc Sci Med 61: 1649–1657
- 43.
Nutley SM (2011) Challenges and opportunities of studying research-based knowledge use.
- 44. Dagenais C, Lysenko L, Abrami PC, Bernard RM, Ramde J, et al. (2012) Use of research-based information by school practitioners and determinants of use: a review of empirical research. Evid Policy 8: 285–309 Available: http://openurl.ingenta.com/content/xref?genre=article&issn=1744-2648&volume=8&issue=3&spage=285.
- 45. Dagenais C, Ridde V, Laurendeau M-C, Souffez K (2009) Knowledge translation research in population health: establishing a collaborative research agenda. Health Res Policy Syst 7: 28
- 46. Dagenais C (2006) Vers une utilisation accrue des résultats issus de la recherche par les intervenants sociaux. Quels modèles de transfert de connaissances privilégier? Les sciences de l'éducation pour l'ère nouvelle 39: 23–35.
- 47. Landry R, Amara N, Lamari M (2001) Utilization of social science research knowledge in Canada. Res Policy 30: 333–349
- 48. Lavis JN, Robertson D, Woodsie JM, McLeod CB, Abelson J (2003) How can research organizations more effectively transfer research knowledge to decision makers? Milbank Q 81: 221–248
- 49. Estabrooks CA (2007) Prologue: a program of research in knowledge translation. Nurs Res 56: S4–6 Available: http://www.ncbi.nlm.nih.gov/pubmed/17625472.
- 50.
Morton S, Nutley SM, Jung T (2008) Seminar 2: The impact of the social sciences on public policy and the impact of evidence-based policy on the social sciences. In: NORFACE, editor. NORFACE seminar series on Evidence and Policy (2007 to 2009). Rotterdam. Available: internal-pdf:/Morton Nutley Jung 2008.pdf.
- 51. Mitton C, Adair CE, McKenzie E, Patten SB, Waye Perry B (2007) Knowledge transfer and exchange: review and synthesis of the literature. Milbank Q 85: 729–768 Available: http://www.ncbi.nlm.nih.gov/pubmed/18070335.
- 52.
Nutley SM, Walter I, Davies HTO (2007) Using Evidence: How Research Can Info Public Services. Bristol: Policy Press.