https://orcid.org/0000-0002-9287-8755
Figures
Abstract
Background
Healthcare providers are critical to the successful, sustainable and impactful implementation of digital health. Despite the growing interest, Digital Health Innovations (DHIs) are often implemented without sufficient evidence, leading to numerous short-lived projects and a limited understanding of their impact on health systems and outcomes. In 2023, the World Health Organization (WHO) introduced a new Classification of Digital Health Interventions, identifying four core DHI user groups. This review aims to synthesise evidence on the impact of DHIs designed for one of these user groups—healthcare providers—on delivering health services in low- and lower-middle-income countries (LLMICs).
Methods
We will conduct an umbrella review, analysing systematic reviews on DHIs for healthcare providers. Data will be extracted using deductive coding before being thematically analysed according to the 11 DHIs for service providers outlined in the WHO’s 2023 Classification.
Discussion
This umbrella review will explore, synthesise, and evaluate the quality of evidence on how healthcare providers in LLMICs utilise DHIs to address service delivery challenges and their effectiveness. To our knowledge, this is the first comprehensive synthesis of evidence focused on DHIs designed for use by healthcare providers in LLMICs. It is also the first review to align with WHO’s taxonomy for DHIs, as outlined in the WHO Classification of Digital Interventions, Services and Applications in Health.
Citation: Craig AT, Lawford H, Miller M, Chen-Cao L, Woods L, Liaw S-T, et al. (2025) Use and impact of digital technology in supporting health providers deliver care in low- and low-middle-income countries: A systematic review protocol. PLoS ONE 20(2): e0319190. https://doi.org/10.1371/journal.pone.0319190
Editor: Addis Eyeberu, Haramaya University, ETHIOPIA
Received: September 10, 2024; Accepted: January 28, 2025; Published: February 21, 2025
Copyright: © 2025 Craig et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: No datasets were generated or analysed during the current study. All relevant data from this study will be made available upon study completion.
Funding: This research is funded by the Asia Pacific Observatory for Health Systems and Policy, Grant No. 1002187-0.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The digital revolution has ushered in a new era of possibilities for healthcare systems worldwide, from electronic health records and telemedicine to artificial intelligence (AI) in medical diagnosis and remote patient monitoring. In the evolving healthcare ecosystem, healthcare providers have emerged at the forefront of this transformation, leveraging digital technologies to navigate and address the multifaceted challenges faced in delivering care when and where it is required.
The adoption of digital technology within healthcare has been fuelled by several factors: cost-effectiveness, demand for data-driven decision-making, technology advancements, and new analytical methods (such as machine learning). These factors have opened opportunities to gather, process, exchange, and analyse medical and health service data securely and in near real-time. This has aided the delivery of data-driven clinical care and service management in developed and developing countries, with governments, donors, and multilateral institutions recognising the opportunity digital technologies offer. In May 2018, governments unanimously adopted a World Health Assembly resolution that called for the World Health Organization (WHO) to develop a digital strategy to support universal health coverage (UHC) at a national level and ultimately achieve the health aims of the Sustainable Development Goals (SDGs) [1,2].
Amid the heightened interest, digital health innovations (DHI) have often been rolled out without examining the evidence, resulting in a proliferation of short-lived projects and diverse digital tools [3,4]. Consequently, there is limited understanding or definitive consensus on the impact of DHIs on health systems and health outcomes [5–7]. This concern was highlighted by the WHO Bellagio eHealth Evaluation Group, which stated, “To improve health and reduce health inequalities, rigorous evaluation of eHealth is necessary to generate evidence and promote the appropriate integration and use of technologies” [8].
While recognising the innovative role digital technologies can play in strengthening health systems, there is a need to understand their contribution better and generate evidence to support future investment decisions. This is most important in low- and lower-middle-income countries (LLMIC), where health service delivery is resource-sensitive and where digital health must add demonstrable benefits to justify the allocation of resources that could otherwise be spent on facilities, equipment, staff, medicines, and other essential commodities [6,7].
In October 2023, to articulate what DHIs can be used to overcome health system challenges, WHO produced the Classification of Digital Interventions, Services and Applications in Health (CDISAH) [9]. In the CDISAH, DHIs are organised into four categories based on their primary user group: (i) persons (i.e., consumers of healthcare services), (ii) healthcare providers, (iii) healthcare managers and support personnel, and (iv) data service managers.
This protocol has been written in line with the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) checklist (S1 File) [10].
Objectives
This review will explore the evidence for the effective use of DHIs by ‘healthcare providers’ (the second group in the CDISAH) in LLMICs to address health system challenges. Our objective is to provide a synthesis of evidence about how DHIs have been used to good effect to support health service delivery in LLMICs.
Method
Given the plethora of published literature on DHIs, an umbrella review will be conducted. An umbrella review is a “systematic collection and assessments of multiple systematic reviews and meta-analyses done on a specific research topic” [11].
We will follow the umbrella review process developed by Arksey and O’Malley, which involves five steps: (1) identification of research question/s; (2) identification of relevant studies; (3) study screening and selection; (4) charting the data; and (5) collating, summarising, synthesising, and reporting results [12].
The review will be undertaken in accordance with the PRISMA-P guidelines (S1 File) [13].
The review will involve five steps (Fig 1).
Step 1: Research questions
The review will seek answers to two questions:
- How are healthcare providers in LLMIC using DHIs to address health system challenges (as outlined in the CDISAH)?
- What is the quality of the systematic reviews being analysed?
We use the Problem, Intervention, Comparison, and Outcome (PICO) framework to develop the research questions. Each component is defined below.
- Problem: Inefficient, ineffective, or inequitable delivery of, or access to healthcare
- Intervention: DHIs used to support health systems needs
- Comparison: The status quo or non-digital approaches to health service delivery
- Outcome: Change to access, effectiveness, efficiency, or reach of health services or impact on health equity.
For this review, a healthcare provider will be defined as a member of the health workforce who delivers health services, including doctors, nurses, community health providers, allied health professionals, and health volunteers. DHIs will be defined as digital and mobile technologies used to support health system needs [14]. Low and low-middle-income countries will be those countries identified as such in the World Bank’s ‘Country Classifications by Income Level: 2024–2025’ [15].
Step 2: Identifying studies
A predefined search strategy encompassing a combination of keywords with Boolean operators in the following syntax will be used. The syntax will include ‘Digital Health’ (and related terms) AND ‘Universal Health Coverage’ (and related terms)] AND ‘LLMICs’ (and related terms).
Four electronic databases—PubMed, Embase, SCOPUS, and Web of Science—will be searched through the Ovid interface. Limiters applied will include ‘humans,’ ‘English,’ ‘published between January 2010 and August 2024 (approximately 15 years), and ‘review articles only.’
Fig 2 provides a preliminary search strategy developed for PubMed.
Step 3: Screen and selection studies
Inclusion and exclusion criteria.
An article must meet the following criteria to be included in the review. Articles must (i) be review articles; (ii) describe a systematic data identification and extraction method; (iii) relate to a DHI implemented in an LLMIC; (v) be available in English; and (vi) be published between January 2010 and August 2024.
Studies will be excluded if they are not presented as review articles, do not include a description of the article screening and selection method, if the method used was not systematic, or if the DHIs discussed were not applied in LLMIC contexts.
Study selection.
A two-stage study selection process will be applied.
First, two authors will independently screen the title and abstract of articles using the abovementioned inclusion and exclusion criteria. Articles with which the reviewers disagreed were automatically be included in the second (full-text screening) stage.
At the full-text screening stage, the two authors will independently assess potentially eligible references for inclusion. A PRISMA flow diagram will document the excluded references and reasons for full-text exclusions.
The ‘Covidence—Better systematic review management’ software (www.covidence.org/) will support all study selection phases.
Step 4: Extract data
A standardised data extraction form will be developed and piloted on five studies, with adjustments made as necessary. The extraction will cover study characteristics (title, author, year, journal, study type), details of the DHIs discussed (intervention, location, description), and their impact (both positive and negative) on health service delivery and function. Data related to context and exemplar case studies will also be extracted.
Given the anticipated substantial heterogeneity among the included studies, we will not perform a meta-analysis or meta-synthesis. Instead, we will conduct a narrative inductive thematic synthesis of the extracted data, following the methods outlined by Terry, Hayfield, Clarke, and Braun [16]. This approach involves iterative coding and recoding until dominant themes emerge. Some refer to it as a textual narrative synthesis, which is characterised by using a standardised data extraction process, unlike a simple narrative synthesis [17]. This method has been employed in systematic reviews [18,19].
Emergent themes will then be categorised as per the taxonomy for DHIs designed for healthcare providers outlined in the CDISAH. These are (i) identifying and registering persons, (ii) generating a person-centred health record, (iii) delivering healthcare provider decision support, (iv) telemedicine; (v) healthcare provider communication, (vi) referral coordination, (vii) scheduling and activity planning for healthcare providers, (viii) healthcare provider training, (ix) prescription and medication management, (x) laboratory and diagnostic imaging management and (xi) healthcare provider financial transactions [9].
Step 5: Collating, analysis and reporting of results
We anticipate publishing the results in a peer-reviewed publication and presenting results at relevant conferences and meetings. Should the opportunity arise, we will also engage in dialogue with policymakers to support knowledge interpretation and translation into policy and practice.
Systematic review quality assessment
To evaluate the methodological quality and risk of bias, authors will assess all studies using the ‘A MeaSurement Tool to Assess systematic Reviews’ (AMSTAR 2) tool [20,21]. AMSTAR 2 used a rating system based on the presence or absence of critical and non-critical weaknesses across 16 ‘quality’ domains. Each domain is evaluated individually to determine the overall quality of a systematic review. Reviews are not assigned an overall numerical score but instead are categorised into four levels of confidence based on the ratings: high confidence (i.e., no or one non-critical weakness); moderate confidence (more than one non-critical weakness); low confidence (one critical flaw with or without non-critical weaknesses; and critically low confidence (more than one crucial flaw with or without non-critical weaknesses) [20].
Discussion
This umbrella review will explore, synthesise, and evaluate the quality of evidence on how healthcare providers in LLMICs utilise DHIs to address service delivery challenges and their effectiveness. To our knowledge, this review will be the first to provide a comprehensive synthesis of evidence specifically focused on DHIs used by healthcare providers in LLMIC and the first to align with WHO’s taxonomy for DHIs, as outlined in WHO’s 2023 Classification of Digital Interventions, Services and Applications in Health.
Our review will be founded on an exhaustive search strategy across multiple reference databases, enabling us to identify strengths and gaps in the existing knowledge related to the use of DHIs. This approach will also help pinpoint avenues for future research and opportunities to enhance practices.
Limitations
We anticipate significant heterogeneity among the studies. Given the nature of the source literature (i.e., systematic reviews), we expect some important nuances to be overlooked and unavailable to us. Additionally, we appreciated that not all relevant research will be captured in the systematic reviews we draw on, and some studies may appear across multiple reviews. We expect that the findings may underemphasise the importance of local context, dynamics, and champions in implementing and succeeding digital health initiatives.
References
- 1.
World Health Organization. Agenda item 12.4. Digital health resolution, In: Seventy-first world health assembly. WHO; 2018.
- 2.
World Health Organization. WHO guidelines: recommendations on digital interventions for health system strengthening. Geneva, Switzerland; 2019. p. 150.
- 3. Sanyal C, Stolee P, Juzwishin D, Husereau D. Economic evaluations of eHealth technologies: a systematic review. PLoS ONE. 2018;13(6):e0198112. Epub 2018 Jun 14. pmid:29897921; PMCID: PMC5999277
- 4. Gentili A, Failla G, Melnyk A, Puleo V, Tanna GLD, Ricciardi W, et al. The cost-effectiveness of digital health interventions: a systematic review of the literature. Front Public Health. 2022;10:787135. Epub 2022 Aug 30. pmid:36033812; PMCID: PMC9403754
- 5. Tomlinson M, Rotheram-Borus MJ, Swartz L, Tsai AC. Scaling up mHealth: Where is the evidence? PLoS Med. 2013;10(2):e1001382. Epub 2013 Feb 21. pmid:23424286; PMCID: PMC3570540 from NIH K23 MH-096620
- 6.
Cullen R. The use of ICT in the health sector in pacific island countries. In: Cullen R, Hassall G, editors. Achieving sustainable e-government in Pacific Island States. Public Administration and Information Technology. 27. Cham, Switzerland: Springer Science and Business Media Springer; 2017. p. 305–35.
- 7.
Craig AT, Beek K, Godinho M, Ansari S, Jonnagaddala J, Linhart C, et al. Digital health and universal health coverage: opportunities and policy considerations for Pacific Island health authorities. New Delhi: WHO; 2022.
- 8. Labrique A, Vasudevan L, Weiss W, Wilson K. Establishing standards to evaluate the impact of integrating digital health into health systems. Glob Health Sci Pract. 2018;6(Suppl 1):S5–17. Epub 2018 Oct 12. pmid:30305335; PMCID: PMC6203412
- 9.
World Health Organization. Classification of digital interventions, services and applications in health: a shared language to describe the uses of digital technology for health. 2nd ed. Geneva, Switzerland: World Health Organization; 2023 Oct 24.
- 10. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al.; PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1. pmid:25554246
- 11. Belbasis L, Bellou V, Ioannidis JPA. Conducting umbrella reviews. BMJ Med. 2022;1(1):e000071. pmid:36936579
- 12. Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8(1):19–32.
- 13. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. PLoS Med. 2021;18(3):e1003583. pmid:33780438
- 14.
World Health Organization. Recommendations on digital interventions for health system strengthening. Geneva, Switzerland: WHO; 2019. p. 206.
- 15.
World Bank. World bank country and lending groups. Washington, DC: World Bank Group; 2024 [cited 2023 Dec 20. ]. Available from: https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups?_gl=1*1c523ut*_gcl_au*MzA1NTg0ODIyLjE3MjQ4ODYzMDc
- 16.
Terry T, Hayfield N, Clarke V, Braun V. The SAGE handbook of qualitative research in psychology. London: SAGE Publications Ltd; 2017.
- 17. Xiao Y, Watson M. Guidance on conducting a systematic literature review. J Plan Educ Res. 2019;39(1):93–112.
- 18. Aiyegbusi OL, Macpherson K, Elston L, Myles S, Washington J, Sungum N, et al. Patient and public perspectives on cell and gene therapies: a systematic review. Nat Commun. 2020;11(1):6265. pmid:33293538
- 19. Jessop Z, Dobbs T, Ali S, Combellack E, Clancy R, Ibrahim N, et al. Personal protective equipment for surgeons during COVID-19 pandemic: systematic review of availability, usage and rationing. J Br Surg. 2020;107(10):1262–80.
- 20. Bojcic R, Todoric M, Puljak L. Adopting AMSTAR 2 critical appraisal tool for systematic reviews: speed of the tool uptake and barriers for its adoption. BMC Med Res Methodol. 2022;22(1):104. pmid:35399051
- 21. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. pmid:28935701