Figures
Abstract
Background
The epidemiological transition phenomena drive the attention to focus the scope on health literacy as it has an impact on patients’ health outcomes and quality of life.
Aim
This paper aims to explore the implemented interventions for improving users’ ability to identify trustworthy online health information.
Methodology
A comprehensive search of the literature will be conducted on the following electronic bibliographic databases: Ovid Medline, Embase, Cochrane database, Academic search complete and APA psycinfo. Further, manual search of eligible studies reference lists will be carried out to identify other eligible studies. The search strategy will include a combination of three key blocks of terms, namely: (adult OR adults) Or (patient OR patients) OR (layperson OR laypersons) OR (caregiver OR caregivers), (Intervention OR Interventions) OR Educational programs OR (health literacy And curriculum) OR Community outreach OR Interactive workshops OR (Online portal OR Patient Portals), and information seeking behavior OR consumer health information OR online information OR social media OR access to information. The results of these categories will then be combined using the AND connector. Two independent reviewers will screen and assess data quality. Disagreements will be resolved by consensus. Due to the anticipated methodological pluralism of the potentially eligible studies, a narrative synthesis of the findings on interventions aimed at improving users’ ability to identify trustworthy online information will be provided according to the pre-identified thematic areas. Furthermore, a narrative synthesis of the reported barriers and facilitators for applying these interventions by end users.
Expected results and impact
Given that the focus of our review findings is on understanding the breadth and depth of the global research into interventions to improve users’ ability to identify trustworthy online health information. The findings will be of great value to inform future innovative approaches to promote identification of trustable online sources for young people worldwide.
Citation: Mohamed H, Salsberg J, Kelly D (2023) An integrative review protocol on interventions to improve users’ ability to identify trustworthy online health information. PLoS ONE 18(4): e0284028. https://doi.org/10.1371/journal.pone.0284028
Editor: Ana Larissa Gomes Machado, Federal University of Piaui: Universidade Federal do Piaui, BRAZIL
Received: September 23, 2022; Accepted: March 13, 2023; Published: April 6, 2023
Copyright: © 2023 Mohamed et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: No datasets were generated or analysed during the current study. All relevant data from this study will be made available upon study completion.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The World Health Organization defines health literacy as "the cognitive and social skills which determine the motivation and ability of individuals to gain access to, understand and use information in ways which promote and maintain good health." Unfortunately, people with limited health literacy tend to have lower efficacy for maintaining healthy behaviours, leading to increased incidence of chronic diseases and health care costs. Nowadays, more and more information are distributed digitally, and the term digital health literacy can be used to describe health information in this context [1].
The value of digital health lies in the use of information and communication technologies for improving patients’ health and well-being and the delivery of health care services. Poor accessibility of health information in digital formats can lead to multiple diagnostic, therapeutic, and prognostic harms, which have an impact on patient safety [2].
Misinformation is defined as information, which is distributed with no intention to cause harm, in contrast to disinformation, where there is an intention to deceive. The spread of misinformation occurs through social contacts or social media [3]. Everyone has now become a potential dissemination source with the capacity to share whatever they believe regardless of its reliability [4]. Moreover, the interconnectivity of the digital era means that the rapid spread of misinformation has a corrosive power on the individual’s short-, medium- and long-term health outcomes [3].
Previous studies identified many factors that play essential roles in determining an individual’s online information-seeking behaviour, such as demographic characteristics (age, gender, educational, socioeconomic, and employment status) and health status [5]. Elderly patients are reluctant to use the Internet, which is consistent with offline gender-based health-seeking behaviour [6]. Individuals who are better educated, employed, and of a higher social class prefer to use online websites as a source of health information [7]. Those with poor or fair health, chronic conditions, and a long-term illness or disability were more likely to visit health sites than those who reported better health [8]. A study on rural digital immigrants in China [9] reported that poor information skills, lack of health information literacy, and low readability of online health information are significant barriers for digital immigrants to seeking health information via the Internet.
Although the Internet offers many advantages to consumers of online health information, searching for specific information can cause many issues concerning the credibility of the information provided [10]. Finding high-quality sites can be challenging, and there are problems with author identification, completeness, and accuracy of the information provided [11,12].
Evaluating the reliability of online health information has been a central issue in consumer health informatics for many years [13]. However, a recent review done in Finland (2020) [14] showed that many adolescents face a challenge in evaluating the credibility of health-related online information. Although they understand that online information is not always to be trusted, many remain unsure of how to evaluate its credibility [15–17].
A 2020 systematic review study [18] identified 17 tools for assessing the trustworthiness of online information, all published between 1997 and 2018 (S1 Table). The tools were developed for different purposes, from a general quality assessment of medical information to detailed analyses. However, the development process of the tools was poorly described. Some tools lacked essential criteria for assessing the trustworthiness of medical information and were not designed for a lay audience.
The criteria used for evaluating online website information in the literature vary depending on the population and context but often include website position in search results, picture quality; celebrity endorsement; website readability; website authorship; website credentials, and consistency with search intentions [19]. A study done in the United States [11] identified twenty-five evaluation criteria (S2 Table). However, using the evaluation criteria for online health information depends on consumers’ health literacy and educational level [19]. For example, Mackert et al. showed that individuals with low health literacy used "position in search results, quality of pictures, celebrity endorsement, and website authorship as criteria to evaluate online health information [13].
A 2019 systematic review [11] done in the United States identified the common indicators used by internet consumers to evaluate the quality of online health information, which are related to three aspects of online information: source, content, and design. Studies done by Fogg et al. (2003) [20] and Walthen and Burkell (2002) [21] reported that most consumers of online information depend on simple methods for credibility evaluation, relying on site presentation and visual design rather than considering content and source presentation.
Accordingly, many information processing theories have been used to explain information-seeking behaviours for users’ credibility assessment of web-based information [22]. For example, the dual processing model stated that internet consumers use either central or peripheral strategies for information evaluation depending on high or low motivational status, respectively [23]. This was supported by the limited capacity model of message processing, which stated that people usually process part of the message provided as they have a limited cognitive capacity for message processing [24]. Additionally, the information-foraging theory claims that whenever feasible, humans will choose behaviours that ’’tend to optimize the utility of information gained as a function of interaction cost’’ [25,26]. Conclusively and taken together, online information seekers tend to use strategies that minimise their cognitive and time investments to cope with the high load of available information.
Many interventions exist targeting a lay person’s ability to find reliable online information. For example, a 2014 study [27] identified the importance of educational programs, interactive workshops, collaborative learning, health literacy curriculum, community outreach, and an online portal with support via videoconferencing. All these interventions aim to teach people how to navigate online, then measure changes in knowledge and internet skills for evaluation purposes.
There is a need to think about a new form of social behaviour surrounding information-seeking that accounts for economic and social effects and requires a change in how science and information professionals approach the dissemination of knowledge. Above that, actions to combat misinformation must be planned and carried out with the support of patients and public.
Aim and objectives
Aim.
Explore the implemented interventions to improve users’ ability to identify trustworthy online health information.
Objectives.
- Understand the types of implemented interventions for improving users’ ability to identify trustworthy digital health information.
- Understand the outcomes of the implemented interventions for improving user’s attitude and behaviour to identify trustworthy online health information.
- Understand the outcomes of the implemented interventions for improving user’s knowledge about their health.
- Explore the measures by which these interventions are evaluated.
- Understand the barriers and facilitators for using these interventions by consumers
- Explore the effectiveness of these interventions for helping users to identify trustworthy online health information.
Materials and methods
Ethics approval and consent to participate is not required for conduction of this integrative systematic review. As this study is solely based on the analysis of previously published anonymous data and contain no identifiable individual person’s data, therefore consent to participate is not applicable.
This systematic review will be reported based on the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) S1 Checklist [28].
Review questions
PICO (population, intervention, comparator, outcome) framework was used to build our research question [29] as follows:
Population: Adult patients and their carer’s, and lay persons (19 to 65 years).
Intervention: Implemented interventions to improve users’ ability to identify trustworthy online health information. Examples of potential inclusions are educational programs, community outreach, health literacy curriculum, and university-based health programs.
Comparator: Studies with or without comparative groups will be considered for the review.
Primary outcome: Understand the outcomes of the implemented interventions for improving user’s attitude and behaviour to identify trustworthy online health information.
Secondary outcomes: Understand the outcomes of the implemented interventions for improving user’s knowledge about their health.
The systematic integrative review will address the following questions:
- What interventions have been implemented to improve users’ ability to identify trustworthy online health information?
- What are the outcomes of the implemented interventions for improving users’ ability to identify trustworthy online health information?
- What are the outcomes of the implemented interventions for improving users’ knowledge about their health?
- What are the reported barriers and facilitators for using those interventions by adults?
- How effective are those interventions for helping users to identify trustworthy online health information?
Study design
This integrative review will include peer-reviewed quantitative (experimental and observational), qualitative, and mixed-methods studies.
Context
Any study reporting an intervention conducted among adult people globally will be considered for the review (Community-based, Institutional, any specific setting based).
Eligibility criteria
Table 1 show the inclusion and exclusion criteria with the justification.
Information sources
We will explore several electronic databases, including Ovid Medline, Embase, Cochrane database, Academic search complete and APA psycinfo. In addition, a discussion with two additional research team members regarding exclusion and inclusion criteria at the outset of the systematic integrative review process will occur. Finally, we will manually search the references of the included studies and track their citations to identify other eligible studies.
Search strategy
A health sciences librarian helped to construct the search strategy. Medical Subject Headings (MESH) and free-text terms will be used. Truncation and adjacency searching are used to increase the sensitivity of the search, as appropriate.
A preliminary initial search was conducted on the Medline database to identify the mesh terms related to our research questions as follows:
(adult OR adults) OR (patient OR patients) OR (layperson OR laypersons) OR (caregiver OR caregivers).
(Intervention OR interventions) OR Educational programs OR (health literacy And
curriculum) OR Community outreach OR Interactive workshops OR (Online portal OR Patient Portals).
Information seeking behaviour OR consumer health information OR online information OR social media OR Access to information.
The results of these categories were then combined using the AND connector.
Data management
Duplicates will be removed using Endnote Reference Management Software (Clarivate), and additional duplicates not identified by the Endnote function can be removed manually. The deduplicated data can then be imported into Covidence (Veritas Health Innovation Ltd), a review-management software program that operates in partnership with Cochrane Collaboration and allows multiple reviewers to work on study selection simultaneously and independently.
Study selection
The eligible studies will be identified in 2 stages: title and abstract screening and full-text review. Abstracts and titles will be screened by two independent reviewers using the eligibility criteria outlined above. Conflicts will be resolved by discussion and adjudicated by a third independent reviewer when necessary. The full-text review of all studies selected during the screening will be independently conducted by two reviewers, with disagreement resolved as described previously.
Data extraction
Two reviewers will independently extract the data from each eligible study using a Microsoft Excel data extraction form. Other reviewers can review the extracted data to ensure accuracy and completeness of the data. The following study details will be extracted: authors, year of publication, country, affiliation, study aim, study design, and publication status. Data will also be extracted about types of implemented interventions to improve users’ ability to identify trustworthy digital health information, how they measure their success and effectiveness, and the barriers, facilitators, and outcomes of these interventions.
Quality assessment
The mixed methods appraisal tool will be adopted to integrate evidence from quantitative, qualitative, and mixed-methods studies [30]. Two reviewers will appraise the quality of each study independently. Disagreements between reviewers can be resolved by discussion. The results of the quality appraisal will be presented for each study, and any eligible study will not be excluded based on the quality appraisal in keeping with the integrative review methodology [31].
Data synthesis
Due to different methodological designs of potentially included reviews, the findings of the narrative synthesis will be provided from the included studies, structured around answering the main review questions. We will present the summarised information on interventions aimed at improving users’ ability to identify trustworthy online health information according to identified thematic areas such as the study setting, target population, type of intervention, intervention objectives, and how the interventions work. Furthermore, Whittemore and Knafl’s integrative review approach will be used as guidance for narrative synthesis [31].
Multiple different outcomes will be measured across various study designs; thus, a meta-analysis will not be feasible. The intervention’s outcomes will be summarised according to the reported change in the attitude and behaviour to identify trustworthy digital health information and health knowledge before and after the interventions. Furthermore, a narrative synthesis of the reported barriers and facilitators of the implemented interventions will be provided.
Results and discussion
Patients or the public will be involved in synthesising and interpreting the study results. The findings of this research will be helpful for lay people to identify the gaps of implemented interventions and address the issues related to the barriers. Policymakers can also use the research findings to promote scaling up of the implemented interventions globally and facilitate designing of future approaches based on technology and innovations.
The results of the study will be disseminated through peer-reviewed publications, academic conferences, and to all patients and the public involved in this study. The findings will be disseminated to the public through local media (Radio) and newspapers articles.
Conclusions
The findings of this paper will inform the design of future user-centred interventions tailored to patients and public needs.
Supporting information
S1 Checklist. PRISMA-P (Preferred Reporting Items for Systematic review and Meta-Analysis Protocols) 2015 checklist: Recommended items to address in a systematic review protocol*.
https://doi.org/10.1371/journal.pone.0284028.s001
(DOC)
S1 Table. Tools for assessing trustworthiness of online health information.
https://doi.org/10.1371/journal.pone.0284028.s002
(DOCX)
S2 Table. Criteria used for evaluation of online website information.
https://doi.org/10.1371/journal.pone.0284028.s003
(DOCX)
References
- 1. Chung SY, Nahm ES. Testing reliability and validity of the eHealth Literacy Scale (eHEALS) for older adults recruited online. Comput Inform Nurs [Internet]. 2015 Apr 1 [cited 2022 Mar 31];33(4):150–6. Available from: https://pubmed.ncbi.nlm.nih.gov/25783223/. pmid:25783223
- 2. Adjekum A, Blasimme A, Vayena E. Elements of Trust in Digital Health Systems: Scoping Review. J Med Internet Res 2018;20(12):e11254 https://www.jmir.org/2018/12/e11254 [Internet]. 2018 Dec 13 [cited 2022 Mar 31];20(12):e11254. Available from: https://www.jmir.org/2018/12/e11254. pmid:30545807
- 3. Wang Y, McKee M, Torbica A, Stuckler D. Systematic Literature Review on the Spread of Health-related Misinformation on Social Media. Social Science & Medicine. 2019 Nov 1;240:112552. pmid:31561111
- 4. Ripoll L, Matos JC. Information reliability: criteria to identify misinformation in the digital environment. [cited 2022 Mar 31];34:2020. Available from: http://dx.doi.org/10.22201/iibi.24488321xe.2020.84.58115.
- 5. Gallagher S, Doherty DT. Searching for health information online: characteristics of online health seekers. J Evid Based Med [Internet]. 2009 May [cited 2022 Mar 31];2(2):99–106. Available from: https://pubmed.ncbi.nlm.nih.gov/21348996/. pmid:21348996
- 6. Choi NG, Dinitto DM. Internet Use Among Older Adults: Association With Health Needs, Psychological Capital, and Social Capital. Journal of Medical Internet Research [Internet]. 2013 [cited 2022 Jul 25];15(5). Available from: /pmc/articles/PMC3668603/. pmid:23681083
- 7. Nangsangna RD, Da-Costa Vroom F. Factors influencing online health information seeking behaviour among patients in Kwahu West Municipal, Nkawkaw, Ghana. Online Journal of Public Health Informatics [Internet]. 2019 Sep 20 [cited 2022 Jul 25];11(2):13. Available from: /pmc/articles/PMC6788904/. pmid:31632607
- 8. Mattsson S, Olsson EM, Johansson B, Carlsson M. Health-Related Internet Use in People With Cancer: Results From a Cross-Sectional Study in Two Outpatient Clinics in Sweden. J Med Internet Res [Internet]. 2017 May 1 [cited 2021 Jul 22];19(5). Available from: https://pubmed.ncbi.nlm.nih.gov/28506959/. pmid:28506959
- 9. Fuzhi W, Dan L, Weiwei S, Tingting Y, Dehua H, Wei P, et al. Health Information Literacy and Barriers of Online Health Information Seeking Among Digital Immigrants in Rural China: A Preliminary Survey: https://doi.org/101177/2158244019856946 [Internet]. 2019 Jun 12 [cited 2022 Mar 31];9(2). Available from: https://journals.sagepub.com/doi/full/10.1177/2158244019856946.
- 10. Chu JT, Wang MP, Shen C, Viswanath K, Lam TH, Chan SSC. How, When and Why People Seek Health Information Online: Qualitative Study in Hong Kong. Interactive Journal of Medical Research [Internet]. 2017 Dec 12 [cited 2022 Jul 25];6(2):e24. Available from: /pmc/articles/PMC5743920/. pmid:29233802
- 11. Sun Y, Zhang Y, Gwizdka J, Trace CB. Consumer Evaluation of the Quality of Online Health Information: Systematic Literature Review of Relevant Criteria and Indicators. Journal of Medical Internet Research [Internet]. 2019 May 1 [cited 2022 Jul 25];21(5). Available from: /pmc/articles/PMC6521213/. pmid:31045507
- 12. Kaicker J, Dang W. Assessing The Quality And Reliability Of Health Information On ERCP Using The DISCERN Instrument. Health Care: Current Reviews. 2016;4(2):1–4.
- 13. Keselman A, Smith CA, Murcko AC, Kaufman DR. Evaluating the Quality of Health Information in a Changing Digital Ecosystem. J Med Internet Res [Internet]. 2019 Feb 1 [cited 2022 Mar 31];21(2). Available from: https://pubmed.ncbi.nlm.nih.gov/30735144/.
- 14. Hämäläinen EK, Kiili C, Räikkönen E, Marttunen M. Students’ abilities to evaluate the credibility of online texts: The role of internet-specific epistemic justifications. Journal of Computer Assisted Learning [Internet]. 2021 Oct 1 [cited 2022 Mar 31];37(5):1409–22. Available from: https://onlinelibrary.wiley.com/doi/full/10.1111/jcal.12580.
- 15. Paul J, Macedo-Rouet M, Rouet JF, Stadtler M. Why attend to source information when reading online? The perspective of ninth grade students from two different countries. Computers and Education. 2017 Oct 1;113:339–54.
- 16. Freeman JL, Caldwell PHY, Scott KM. The Role of Trust When Adolescents Search for and Appraise Online Health Information. J Pediatr [Internet]. 2020 Jun 1 [cited 2022 Apr 27];221:215–223.e5. Available from: https://pubmed.ncbi.nlm.nih.gov/32446485/. pmid:32446485
- 17. Hoeken H. Anecdotal, Statistical, and Causal Evidence: Their Perceived and Actual Persuasiveness. Argumentation 2001 15:4 [Internet]. 2001 [cited 2022 Apr 27];15(4):425–37. Available from: https://link.springer.com/article/10.1023/A:1012075630523.
- 18. Lenaerts G, Geertruida E Bekkering, Goossens M, de Coninck L, Delvaux N, Cordyn S, et al. Tools to Assess the Trustworthiness of Evidence-Based Point-of-Care Information for Health Care Professionals: Systematic Review. Journal of Medical Internet Research [Internet]. 2020 Jan 1 [cited 2021 Nov 29];22(1). Available from: /pmc/articles/PMC6996752/. pmid:31951213
- 19. Diviani N, van den Putte B, Giani S, van Weert JCM. Low health literacy and evaluation of online health information: a systematic review of the literature. J Med Internet Res [Internet]. 2015 May 1 [cited 2022 Mar 31];17(5). Available from: https://pubmed.ncbi.nlm.nih.gov/25953147/. pmid:25953147
- 20. Fogg BJ, Soohoo C, Danielson DR, Marable L, Stanford J, Tauber ER. How do users evaluate the credibility of Web sites?: A study with over 2,500 participants. Proceedings of the 2003 Conference on Designing for User Experiences, DUX ‘03. 2003.
- 21. Wathen CN, Burkell J. Believe it or not: Factors influencing credibility on the Web. Journal of the American Society for Information Science and Technology [Internet]. 2002 Jan 1 [cited 2022 Apr 27];53(2):134–44. Available from: https://onlinelibrary.wiley.com/doi/full/10.1002/asi.10016.
- 22. Metzger MJ, Flanagin AJ, Medders RB. Social and Heuristic Approaches to Credibility Evaluation Online. Journal of Communication [Internet]. 2010 Sep 1 [cited 2022 Mar 27];60(3):413–39. Available from: https://academic.oup.com/joc/article/60/3/413/4098547.
- 23. Metzger MJ. Making sense of credibility on the web: Models for evaluating online information and recommendations for future research. Journal of the American Society for Information Science and Technology. 2007 Nov;58(13):2078–91.
- 24. Lang A. The Limited Capacity Model of Mediated Message Processing. Journal of Communication [Internet]. 2000 Mar 1 [cited 2022 Apr 27];50(1):46–70. Available from: https://academic.oup.com/joc/article/50/1/46/4110103.
- 25. Ross L, Greene D, House P. The “False Consensus Effect”: An Egocentric Bias in Social Perception and Attribution Processes.
- 26. Pirolli P. Rational Analyses of Information Foraging on the Web. 2005.
- 27. Lee K, Hoti K, Hughes JD, Emmerton LM. Interventions to Assist Health Consumers to Find Reliable Online Health Information: A Comprehensive Review. Available from: www.plosone.org.
- 28. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ [Internet]. 2015 Jan 2 [cited 2022 Apr 12];350. Available from: https://pubmed.ncbi.nlm.nih.gov/25555855/. pmid:25555855
- 29. Darshana Wickramasinghe N, Samarutilake N, Wettasinghe MC, Feiler J, Morgan A, Kousoulis AA, et al. Public health programmes to promote mental health in young people: a systematic integrative review protocol. BMJ Open [Internet]. 2020;10:37241. Available from: http://bmjopen.bmj.com/.
- 30. Pace R, Pluye P, Bartlett G, Macaulay AC, Salsberg J, Jagosh J, et al. Testing the reliability and efficiency of the pilot Mixed Methods Appraisal Tool (MMAT) for systematic mixed studies review. International Journal of Nursing Studies. 2012 Jan 1;49(1):47–53. pmid:21835406
- 31. Whittemore R, Knafl K. The integrative review: updated methodology. Journal of Advanced Nursing [Internet]. 2005 Dec 1 [cited 2022 Apr 12];52(5):546–53. Available from: https://onlinelibrary.wiley.com/doi/full/10.1111/j.1365-2648.2005.03621.x. pmid:16268861