To examine and compare preferences of knowledge users for two different formats of summarizing results from systematic reviews: infographics and critical appraisals.
Annual members’ meeting of a Network of Centres of Excellence in Knowledge Mobilization called TREKK (Translating Emergency Knowledge for Kids). TREKK is a national network of researchers, clinicians, health consumers, and relevant organizations with the goal of mobilizing knowledge to improve emergency care for children.
Overall preference for infographic vs. critical appraisal format. Members’ rating of each format on a 10-point Likert scale for clarity, comprehensibility, and aesthetic appeal. Members’ impressions of the appropriateness of the two formats for their professional role and for other audiences.
Among 64 attendees, 58 members provided feedback (91%). Overall, their preferred format was divided with 24/47 (51%) preferring the infographic to the critical appraisal. Preference varied by professional role, with 15/22 (68%) of physicians preferring the critical appraisal and 8/12 (67%) of nurses preferring the infographic. The critical appraisal was rated higher for clarity (mean 7.8 vs. 7.0; p = 0.03), while the infographic was rated higher for aesthetic appeal (mean 7.2 vs. 5.0; p<0.001). There was no difference between formats for comprehensibility (mean 7.6 critical appraisal vs. 7.1 infographic; p = 0.09). Respondents indicated the infographic would be most useful for patients and their caregivers, while the critical appraisal would be most useful for their professional roles.
Infographics are considered more aesthetically appealing for summarizing evidence; however, critical appraisal formats are considered clearer and more comprehensible. Our findings show differences in terms of audience-specific preferences for presentation of research results. This study supports other research indicating that tools for knowledge dissemination and translation need to be targeted to specific end users’ preferences and needs.
Citation: Crick K, Hartling L (2015) Preferences of Knowledge Users for Two Formats of Summarizing Results from Systematic Reviews: Infographics and Critical Appraisals. PLoS ONE 10(10): e0140029. https://doi.org/10.1371/journal.pone.0140029
Editor: Roberta W. Scherer, Johns Hopkins Bloomberg School of Public Health, UNITED STATES
Received: December 3, 2014; Accepted: September 21, 2015; Published: October 14, 2015
Copyright: © 2015 Crick, Hartling. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This project was supported in part through the Network of Centres of Excellence in Knowledge Mobilization, TREKK (Translating Emergency Knowledge for Kids), and the Women and Children’s Health Research Institute in Edmonton, Canada. Dr. Hartling is supported by a New Investigator Salary Award from the Canadian Institutes of Health Research.
Competing interests: The authors have declared that no competing interests exist.
Health services research has consistently found a failure to translate healthcare research results into practice and policy. Despite the millions of dollars that are spent each year on health research, healthcare systems continue to fail to ensure that effective programs, services, and drugs get to all patients who need them.
Limited time is a frequently cited barrier to the use of evidence in practice.[2–6] Systematic reviews are considered a cornerstone of knowledge translation as they provide a comprehensive synthesis of available evidence on a clinical question for decision makers in the healthcare system.[1,7] Although systematic reviews are methodologically rigorous, users have requested that reviews be translated into formats that are shorter and more easily understood by different audiences. Knowledge users need knowledge translation tools that provide synopses of systematic reviews with key messages highlighted for quick reference.
Summarized evidence for clinicians exists in many different formats including structured abstracts and synopses published in secondary journals. Journals and other resources (e.g., UpToDate) that provide high-level summaries to inform evidence-based practice have used critical appraisal formats extensively. Critical appraisal formats summarize the research evidence and provide information on its validity and applicability for knowledge users.
Infographics have emerged recently as a novel method of graphically displaying information. Infographics are visual images such as charts or diagrams that are used to represent information and data. Research has shown that visualizations, as opposed to text, are intrinsically more memorable and effectively transferred with consistency across people.[12–14] High memorability scores are correlated with visualizations, including infographics, that contain pictograms, more color, low data-to-ink ratios, and high visual densities.[12,14] Infographics therefore have the potential to be an effective knowledge translation tool for the transfer of research results.
Little research has been done examining knowledge users' perspectives on different formats for summarizing research evidence. Despite the growing popularity of infographic formats, little research has examined whether infographics are an effective knowledge translation tool for systematic reviews. The goal of this study was to contribute to this body of research through an initial examination of knowledge user preferences. Our objective was to examine and compare preferences of knowledge users for two different formats of summarizing results from systematic reviews: infographics and critical appraisals.
This was a cross-sectional evaluation designed to compare the end user utility and preferences of two data presentation formats: critical appraisals and infographics. The target audience was stakeholders attending the 2013 annual meeting of the National Centre of Excellence in Knowledge Mobilization called TREKK (Translating Emergency Knowledge for Kids). TREKK is a national network of researchers, clinicians, health consumers and relevant organizations with the goal of mobilizing knowledge to improve emergency care for children. TREKK involves 39 general emergency departments across Canada. The purpose of the annual meeting was to update the stakeholders on TREKK activities. Sixty-four stakeholders attended the meeting and 58 participated in viewing the presentation formats and answering a short questionnaire. Stakeholders completed the activity voluntarily; no personal identifying information was collected.
We chose to summarize a systematic review of the efficacy and tolerability of different pharmacological therapies for acute migraine headaches in children and adolescents. The topic was selected because it was felt to be of general interest to a range of stakeholders including physicians, nurses, healthcare administrators, and parents. We engaged with a graphic artist to develop an infographic of the chosen review (Fig 1). We also developed a critical appraisal (Fig 2) based on the format used by Academic Emergency Medicine. This format was selected based on input from the TREKK steering committee. Information was summarized for the population, interventions, outcomes, study design, and the research evidence from the chosen systematic review. The summary of information followed the PICO format (i.e., population, intervention, comparison, outcome) that is widely accepted for the critical appraisal of evidence.
All stakeholders viewed both of the formats and completed a paper-based questionnaire answering the same set of questions for each of the two formats (S1 File). We compared the formats in terms of the stakeholders' impressions of the clarity, comprehensibility, and aesthetic appeal. These items were assessed using a 10-point Likert scale. We also compared stakeholders' preferences of the two formats,  whether they would use the format in their professional role, and for which audiences they believed each of the formats to be most useful. We asked stakeholders about their primary occupational role, and offered them the opportunity to provide additional comments about the two formats.
Differences in clarity, comprehensibility, and aesthetic appeal of the two formats within stakeholders were tested using Wilcoxon matched pairs signed rank test. Wilcoxon signed rank test was also used to compare stakeholders' paired responses to the usefulness of the two formats in their professional role, the usefulness to patients and their caregivers, and for which audiences the two formats would be most appropriate. Mann-Whitney U test was used to compare responses between primary roles. A thematic analysis approach was used to examine text responses to open-ended questions. Data underlying the results presented in this manuscript are available in S2 File.
A total of 58 stakeholders provided feedback (91%). A range of stakeholders were represented: physicians (n = 27, 48%); nurses (n = 13, 23%); other healthcare professionals, including administrators, allied health professionals, researchers, and research coordinators (n = 9, 16%); and parent advisors (n = 4, 7%). Forty-seven stakeholders reported their preferred format and were divided in their overall preference for the two formats: 24 (51%) preferred the infographic to the critical appraisal (see Table 1).
Despite there being no overall preference for either format, preferences of specific aspects of the two formats differed significantly. Stakeholders favored the critical appraisal format in terms of clarity (p = 0.03), whereas for aesthetic appeal, the infographic format was preferred (p<0.001). Stakeholders rated the two formats to be equally comprehensible (p = 0.09). Fifty (87.7%) stakeholders considered the critical appraisal format to be more useful in their professional role than the infographic, although the difference was not statistically significant (p = 0.07). The majority (88%) felt the infographic format would be useful for patients and their caregivers.
There were large differences for which audiences stakeholders believed the two display formats to be most appropriate. The infographic was believed to be most appropriate for the public by 39 (70%) stakeholders (p<0.001) and the media by 32 (57%) stakeholders (p<0.001). The critical appraisal was thought to be more appropriate for researchers (p<0.001) and research funders (p = 0.001). There were no differences in which format was thought to be most appropriate for other health practitioners (p = 0.98), decision-makers (p = 0.27), and policy makers (p = 0.13).
Respondents found the infographic to be visually engaging, attractive, easy to read, and they felt it captured a lot of information. Some respondents found the infographic to be too busy, difficult to interpret and follow, and difficult to determine the take home message. Respondents found the critical appraisal to be clear, directive, professional, and concise; however, they found it more technical and less visually appealing than the infographic.
We compared results for physicians and nurses, the most common primary professional roles represented in our audience. Physicians and nurses responded similarly to all questions except for one: more nurses indicated that the infographic would be useful in their professional role while physicians preferred the critical appraisal (p = 0.04).
Overall, the infographic and critical appraisal formats were equally preferred. Although both formats were well received, the two formats were preferred for different reasons and for different audiences. The critical appraisal format was preferred in terms of clarity and was found to be directive, professional, and concise. The infographic was preferred for aesthetic appeal; it was found to be visually engaging and easy to read while capturing a lot of information. In terms of preferred format for different audiences, it was felt that the infographic was more useful to patients and their caregivers, the public and the media, while the critical appraisal format was believed to be more appropriate for researchers and research funders. Our analyses by professional group showed that nurses preferred the infographic while physicians preferred the critical appraisal. This finding underscores the need to consider the target audience when developing evidence summaries, as well as knowledge translation tools more broadly.
This study is the first of its kind to compare infographic and critical appraisal formats as a knowledge translation tool for the results of systematic reviews. Infographics have become a popular format of data presentation; however, there is little research evidence to support their use or preferences among target audiences. There is a general sense that infographics may be an effective knowledge translation tool because they are aesthetically pleasing and popular. Further, there is evidence in the literature that visual formats are intrinsically more memorable and consistently transferred across people. However, there has been little formal evaluation of the infographic format itself as a knowledge translation tool in the health sciences.[12,14] Formal evaluations of different methods to summarize and disseminate health evidence are needed in order to understand their strengths and limitations, and for which audiences different methods are most appropriate. As demonstrated by this evaluation, there is no 'one fits all' tool. This evaluation involved a single target group and a select systematic review; therefore, results may not be generalizable to other clinical areas or target audiences.
The results of this research demonstrate the importance of understanding the preferences of the target audience when designing knowledge translation tools. Although the critical appraisal and infographic formats were equally preferred overall, there were specific audiences for which each of the two formats was believed to be more appropriate. This information may be helpful to others designing tools to share health information, including evidence from systematic reviews. Further research is needed on the effectiveness of these formats in terms of increasing knowledge and influencing behaviour in order to fully inform the utility of these tools as knowledge translation strategies.
S1 File. “Infographics Questionnaire” (contains the questionnaire that participants were asked to complete).
This study was conducted as part of the work of the Networks of Centres of Excellence in Knowledge Mobilization Initiative: Translating Emergency Knowledge for Kids (TREKK). TREKK is directed by Dr. Terry Klassen (University of Manitoba), Dr. Lisa Hartling (University of Alberta), Dr. Shannon Scott (University of Alberta), Dr. David Johnson (University of Calgary), and Dr. Mona Jabbour (University of Ottawa).
We thank Dr. Tariq AlShawabkeh for assistance with the critical appraisal, Sarah Oberik-Olivieri for assistance with design of the infographic, and Jocelyn Shulhan for assistance with data entry.
Conceived and designed the experiments: LH. Performed the experiments: LH. Analyzed the data: LH KC. Contributed reagents/materials/analysis tools: LH KC. Wrote the paper: LH KC.
- 1. Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implementation science: IS 2012;7:50. pmid:22651257
- 2. Coumou HC, Meijman FJ. How do primary care physicians seek answers to clinical questions? A literature review. Journal of the Medical Library Association: JMLA 2006;94(1):55–60. pmid:16404470
- 3. Ely JW, Osheroff JA, Ebell MH, Chambliss ML, Vinson DC, Stevermer JJ, et al. Obstacles to answering doctors' questions about patient care with evidence: qualitative study. Bmj 2002;324(7339):710. pmid:11909789
- 4. Gosling AS, Westbrook JI, Coiera EW. Variation in the use of online clinical evidence: a qualitative analysis. International journal of medical informatics 2003;69(1):1–16. pmid:12485700
- 5. Green ML, Ruff TR. Why do residents fail to answer their clinical questions? A qualitative study of barriers to practicing evidence-based medicine. Academic medicine: journal of the Association of American Medical Colleges 2005;80(2):176–82.
- 6. Leung GM, Yu PL, Wong IO, Johnston JM, Tin KY. Incentives and barriers that influence clinical computerization in Hong Kong: a population-based physician survey. Journal of the American Medical Informatics Association: JAMIA 2003;10(2):201–12. pmid:12595409
- 7. Scott NA, Moga C, Barton P, Rashiq S, Schopflocher D, Taenzer P, et al. Creating clinically relevant knowledge from systematic reviews: the challenges of knowledge translation. Journal of evaluation in clinical practice 2007;13(4):681–8. pmid:17683314
- 8. Rader T, Pardo Pardo J, Stacey D, Ghogomu E, Maxwell LJ, Welch VA, et al. Update of strategies to translate evidence from cochrane musculoskeletal group systematic reviews for use by various audiences. The Journal of rheumatology 2014;41(2):206–15. pmid:24293571
- 9. Rosenbaum SE, Glenton C, Nylund HK, Oxman AD. User testing and stakeholder feedback contributed to the development of understandable and useful Summary of Findings tables for Cochrane reviews. Journal of Clinical Epidemiology 2010;63(6):607–19. pmid:20434023
- 10. Straus SE, Glasziou P, Richardson WS. Evidence-Based Medicine: How to Practice and Teach It: Elsevier Churchill Livingstone, 2011.
- 11. Dictionaries O. "infographic". Oxford Dictionaries: Oxford University Press.
- 12. Hildon Z, Allwood D, Black N. Impact of format and content of visual display of data on comprehension, choice and preference: a systematic review. International Journal for Quality in Health Care 2012;24(1):55–64. pmid:22166856
- 13. Dur BIU. Analysis of data visualizations in daily newspapers in terms of graphic design. In: Ilhan AC, ed. World Conference on Design, Arts and Education, 2012:278–83.
- 14. Borkin MA, Vo AA, Bylinskii Z, Isola P, Sunkavalli S, Oliva A, et al. What makes a visualization memorable? IEEE Transactions on Visualization & Computer Graphics;19(12):2306–15.
- 15. Billinghurst L, Richer L, Russell K, Vandermeer B, Crumley E, Hartling L, et al. Drugs for treating acute migraine headaches in children and adolescents (Protocol). Cochrane Database of Systematic Reviews 2005, Issue 2. Art. No.: CD005220.