https://orcid.org/0000-0002-1104-2359
Figures
Abstract
We sought to evaluate the impact of transitioning a multi-country HIV training program from in-person to online by comparing digital training approaches implemented during the pandemic with in-person approaches employed before COVID-19. We evaluated mean changes in pre-and post-course knowledge scores and self-reported confidence scores for learners who participated in (1) in-person workshops (between October 2019 and March 2020), (2) entirely asynchronous, Virtual Workshops [VW] (between May 2021 and January 2022), and (3) a blended Online Course [OC] (between May 2021 and January 2022) across 16 SSA countries. Learning objectives and evaluation tools were the same for all three groups. Across 16 SSA countries, 3023 participants enrolled in the in-person course, 2193 learners participated in the virtual workshop, and 527 in the online course. The proportions of women who participated in the VW and OC were greater than the proportion who participated in the in-person course (60.1% and 63.6%, p<0.001). Nursing and midwives constituted the largest learner group overall (1145 [37.9%] vs. 949 [43.3%] vs. 107 [20.5%]). Across all domains of HIV knowledge and self-perceived confidence, there was a mean increase between pre- and post-course assessments, regardless of how training was delivered. The greatest percent increase in knowledge scores was among those participating in the in-person course compared to VW or OC formats (13.6% increase vs. 6.0% and 7.6%, p<0.001). Gains in self-reported confidence were greater among learners who participated in the in-person course compared to VW or OC formats, regardless of training level (p<0.001) or professional cadre (p<0.001). In this multi-country capacity HIV training program, in-person, online synchronous, and blended synchronous/asynchronous strategies were effective means of training learners from diverse clinical settings. Online learning approaches facilitated participation from more women and more diverse cadres. However, gains in knowledge and clinical confidence were greater among those participating in in-person learning programs.
Citation: Kiguli-Malwadde E, Forster M, Eliaz A, Celentano J, Chilembe E, Couper ID, et al. (2023) Comparing in-person, blended and virtual training interventions; a real-world evaluation of HIV capacity building programs in 16 countries in sub-Saharan Africa. PLOS Glob Public Health 3(7): e0001654. https://doi.org/10.1371/journal.pgph.0001654
Editor: Hannah Hogan Leslie, University of California San Francisco, UNITED STATES
Received: February 6, 2023; Accepted: June 5, 2023; Published: July 24, 2023
Copyright: © 2023 Kiguli-Malwadde et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data are available in a public, open access repository. Extra data can be accessed via the Dryad data repository at http://datadryad.org/ with the doi:10.7272/Q6WQ021N.
Funding: EKM, MF, JC, EC, IDC, ETD, MRD, OG, CH, JK, SM, LM, AKM, PM, AR, DS, FS, DVZ, MJAR and FS were all supported by the US Health Resource Services Administration, through a PEPFAR grant. The funder had no role in the study and was not involved in the study design, collection or analysis of data, nor in the decision to publish or prepare this manuscript.
Competing interests: Judy Khanyola is an author and also a editor for PLOS GPH. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
Even before the COVID-19 pandemic, there was a pressing need to expand the health workforce and optimize team-based care across sub-Saharan Africa (SSA), especially in settings of scarce health resources.] While causing unprecedented disruptions to clinical care and health professions training opportunities, the COVID-19 pandemic presented a unique opportunity to rapidly scale new training modalities, including maximizing the use of online learning platforms [1–6]. Numerous African and international training programs developed and implemented remote educational programs to address safety concerns regarding COVID-19 transmission and to ensure continuity of training programs in spite of government interventions to reduce infection and transmission [3,7–10]. Moreover, many higher education institutions in Africa adopted emergency remote teaching and web-based learning using blended-learning strategies to reduce the physical presence of students on campus [4,11]. One such example was the Strengthening Inter-professional Education in HIV (STRIPE HIV) initiative, which rapidly transitioned its in-person training program to online [12]. STRIPE HIV is an HIV training program funded by the United States government through the President’s Emergency Plan for AIDS Relief (PEPFAR), that includes a diverse network of health professional training institutions across SSA, led by the African Forum for Research and Education in Health (AFREhealth) and the University of California, San Francisco [13]. At the start of the COVID-19 pandemic, the STRIPE team rapidly implemented remote educational programming, including developing a learning management system (LMS) accessible to health professions at affiliated training institutions across SSA [1]. The internet-based educational programs consisted of inter-professional HIV training content for pre-service and in-service learners from diverse professional cadres; while the format was altered to accommodate online learning, the curriculum was the same as that employed pre-COVID [12,13].
In advance of implementing this online approach to training, limited research comparing the effectiveness of in-person and remote educational formats across SSA had been undertaken [6,14]. Previous research has evaluated the impact of numerous capacity-building interventions to target individual healthcare provider behavior in low and middle-income countries (LMICs) [14–17], but very little research has evaluated the comparative benefits of online capacity-building interventions in such settings. The in-person and online remote educational programs developed by STRIPE HIV and AFREhealth provide a unique opportunity to evaluate whether transitioning to online learning significantly impacted inter-professional training in HIV care. As such, we sought to compare the impact of the in-person and online formats of inter-professional education on knowledge and confidence among learners of diverse cadres across fourteen countries in SSA.
Methods
Curriculum design and implementation: Year one in-person program
A case-based curriculum in inter-professional HIV training was developed and implemented during year one (Y1) of STRIPE HIV’s educational program, previously described.[12] The program was developed using Kern’s six-step approach by a collaborative team of local and international HIV practitioners and health professions education experts [18]. The inter-professional HIV training included 17 case-based modules with at least four required modules delivered in person over a two-day workshop. The program was initially implemented across 14 countries in SSA between October 2019 and April 2020.
Curriculum design and implementation: Year two online synchronous program
In year two (Y2), which coincided with the spread of COVID-19 across SSA, participating academic institutions opted for one of two learning modalities based on Internet access and existing expertise using educational technologies. Many schools opted to pursue employ a Virtual Workshop (VW) format, which consisted of synchronous online workshops. This approach utilized a revised version of the in-person training content from Y1, adapted for delivery over Zoom and taught by both local and international facilitators, but incorporating both small-group activities and large-group discussions like the in-person approach. Learners from fourteen countries, including learners from several countries that had not previously (Algeria, Madagascar, Mozambique, and Somalia), participated in the Virtual Workshop beginning in May 2021.
Curriculum design and implementation: Year two online blended program
Several other academic institutions opted to teach the same modular content using an online approach that included blended asynchronous and synchronous content (hereafter referred to as the Online Course [OC]). The asynchronous content in this blended program was completed independently online using a Moodle-based learning management system. After completion of the asynchronous content, learners took part in synchronous inter-professional sessions carried out on Zoom. Learners participated in a synchronous session associated with each of the modules using a flipped classroom approach [19]. The Online Course was implemented across six countries in SSA beginning in June 2021.
As outlined in our previous work [12,13], each STRIPE collaborating institution recruited learners from different cadres (e.g. nursing, medicine, pharmacy, etc.), stages of professional development (pre-service, early graduate, etc.) and clinical settings (clinic, hospital, classroom, etc.) according to local training priorities. Moreover, collaborating institutions employed different, non-standardized recruitment strategies to optimize the participation of learners, with some institutions providing airtime or access to the Internet to learners who participated in VW or OC courses.
Study design & subjects
We compared learner knowledge and confidence across the three educational programs: the In-Person Training (Y1), the Virtual Workshop (Y2), and the Online Course (Y2). Subjects were learners who completed the STRIPE registration survey, as well as pre-test and post-test knowledge and confidence assessments. We evaluated self-reported learner demographics including gender identity, training level, health profession, and country of residence. Pre-service learners were defined as learners enrolled in a health profession training institution and working towards their degree. In-service or post-graduate, clinical providers were learners who had already graduated from their health professions training program and engaged in clinical practice. Additionally, we defined early in-service learners as learners who had graduated from training fewer than 12 months prior to participating in the STRIPE HIV course.
To compare the three learning approaches, we evaluated data from four training modules that utilized the same pre-test and post-test assessments across the three educational formats. The four case-based modules were titled New HIV Diagnosis and Antiretroviral Therapy (ART) Initiation in a Woman of Childbearing Age, Management of HIV-TB Co-Infection, PMTCT & Care for the Pregnant Woman with HIV, and Care for the Paediatric Patient with HIV. The specific knowledge and confidence assessment questions from each module utilized in the analysis are shown in S1 Table. Since the learning objectives, key content, and learning activities were the same or very similar, regardless of which approach was employed, the learning time for these modules was similar for the In-Person, OC and VW formats.
Knowledge assessment
We evaluated learning among participants by assessing the mean difference in pre-test and post-test knowledge scores and percent change in knowledge scores. For Y1, data were collected during the period of training from October 2019 to March 2020. For Y2, data were collected from May 2021 to January 2022. We compared mean knowledge differences and percent increase across the educational programs and stratified knowledge differences by health profession and training level.
Confidence assessment
We evaluated confidence among participants by assessing the mean difference in pre-test and post-test confidence scores in three arenas: clinical confidence, confidence in working as part of an inter-professional team (IP Confidence), and confidence in implementing quality improvement strategies (QI Confidence). Across these three confidence domains, participants were invited to complete a series of questions asked on a Likert scale of 1 (“I feel uncomfortable with this topic/need supervision from my supervisor”) to 4 (“I feel very comfortable with this topic/without supervision as though in independent practice”). Fifteen questions assessed clinical confidence, two questions assessed IP confidence and one question addressed confidence to leverage QI tools in clinical practice. These confidence scores were summed for each participant, and the mean differences in confidence scores were compared across the three educational programs, stratified by health profession and training level, using Analysis of Variance (ANOVA) and Tukey’s Honest Significance Tests as outlined below.
Statistical analysis
We compared learner demographics using Pearson chi-squared tests. Wilcoxon signed-rank test was used to identify baseline pairwise differences between the pre-test and post-test means for each educational program overall, as well as by training level and health profession. We used Analysis of Variance (ANOVA) and Tukey’s Honest Significance Test (HSD) to compare mean differences in knowledge and confidence between the three educational programs. Analyses were carried out using RStudio (2022.07.1+554). Additionally, p<0.05 was considered statistically significant.
Ethics statement
The design of the training program, including the topics covered and the format of the training, was informed by input from focus-group discussions with patient groups, learners (both pre-service and early career professionals) and HIV educators from a variety of settings in SSA, and has been previously described [13,20]. Assessment tools, evaluating learners knowledge and confidence, were also piloted with a subset of multidisciplinary learners before the full program was launched. All learners were given access to their pre and post score test results, via the program’s website. In addition, aggregate, site-level evaluation data were also posted on the program’s website. The protocol for this project was reviewed and approved by the University of California, San Francisco’s Institutional Review Board (IRB) in San Francisco, California. Verbal consent was required at the time of participation in the study as approved by the IRB (protocol #: 19–28,447).
Findings
Learner demographics
During Y1, from October 2019 to March 2020, 5027 participants from 12 countries (Botswana, Ethiopia, Ghana, Kenya, Lesotho, Malawi, Nigeria, South Africa, Tanzania, Uganda, Zambia, and Zimbabwe) participated in the In-Person Course, and 3023 of those learners completed pre-test and post-test assessments. During Y2, from June 2021 to November 2022, 7107 learners from 15 countries in sub-Saharan Africa (Algeria, Botswana, Ethiopia, Ghana, Kenya, Lesotho, Madagascar, Malawi, Mozambique, Nigeria, South Africa, Somalia, Tanzania, Zambia, and Zimbabwe) enrolled in the VW and OC programs and 3477 of those learners completed the pre-test and post-test assessments. Among the 3477 remote learners included in the study, 2595 completed the VW training and 628 completed the OC training (Table 1). Complete demographic data was available for 3022, 2193 and 527 learners in the In-person, VW and OC respectively, while pre- and post-course evaluative data was available for 3027, 2595 and 629 participants in each of these groups.
Gender
Among Y1 participants, 1570 (51.9%) identified as female, 1281 (42.4%) identified as male, and 172 (5.7%) identified as other. The proportions of female learners in the Y2 Virtual Workshop and Y2 Online Course were significantly greater than the Y1 in-person program (60.1% [n = 1319] and 63.6% [n = 335] vs. 51.9% [n = 1570], p<0.001).
Training level
Among learners in the Y1, 1755 (58.1%) identified as pre-service, 727 (24.1%) identified as new providers, less than 12 months from graduation, and 540 (17.9%) identified as post-graduate providers, more than 12 months from graduating their training program (Table 1). The proportions of in-service new providers in the Y2 Virtual Workshop and Y2 Online Course were significantly lower than the Y1 in-person program (13.3% [n = 291] and 12.6% [n = 66] vs 24.1% [n = 727], p<0.001).
Health professions
In both the Y1 In-person program and remote Y2 educational programs, at least four different health professions participated in the training. Health professions included laboratory, medical, nursing/midwifery, pharmacy, and other, (We included physical therapists and nutritionists in the “other” category given the small number of learners from those professional cadres.) Nursing/midwifery learners constituted the largest proportion of learners in the In-person (n = 1145 [37.9%]) and VW (n = 949 [43.3%]) courses, while medical providers constituted the largest group in the OC (n = 140 [26.8%]).
Knowledge assessment
Across all subgroups of learners, HIV knowledge scores increased between the pre- and post-course tests; the mean difference in pre vs. post-test knowledge scores among participants in Y1 was 1.2 compared to 0.5 among VW participants and 0.7 among OC participants (S3 Table). Moreover, the percent increase in knowledge scores was significantly lower for those who participated in either the VW or OC formats than among learners who participated in the Y1 In-person training (+6.0% and +7.6% vs +13.6%, p<0.001) (Table 2).
Across the three training levels, the increase in knowledge scores was greatest for participants in the Y1 In-person course compared to either VW or OC across stages of professional development (Y1 in-person pre-service learners experienced a 14.1% [+1.3 points] mean increase in knowledge vs. 5.1% [+0.5 points] for Y2 VW learners vs. 7.4% [+0.6 points] for OC learners [p<0.001]; Y1 in-person new graduate learners experienced a mean 13.4% [+1.2 points] increase in knowledge vs. 6.5% [+0.6 points] for Y2 VW learners vs. 7.4% [+0.7 points] for OC learners[p<0.001]; and Y1 in-person post-graduate learners experienced a mean 12.5% [+1.1 points] increase in knowledge vs. 7.6% [0.7 points] for Y2 VW learners vs. 11.6% [+1.0 points] for OC learners [p<0.001]). Among pre-service and post-graduate new provider learners there were no significant differences in knowledge scores noted between VW and OC participants, but there were greater gains in knowledge among post-graduate provider learners who participated in the OC compared to the VW (Tables 2 and S3). Regardless of the health professions cadre, gains in knowledge were greater among Y1 In-person learners compared to learners who participated in the VW or OC formats. Among learners from nursing, pharmacy and other cadres, there was no difference in knowledge gains between participants in the VW and OC formats.
Confidence assessment
Baseline confidence levels were higher among post-graduate learners with more than 12 months of clinical experience compared to other types of learner. However, across all learner subgroups, self-perceived confidence increased between the pre- and post-course assessments (Table 3, Figs 1 and 2). Overall, the percent increases in confidence scores were significantly lower among learners who participated in the VW or OC than among learners who participated in the Y1 in-person training ((Y1 in-person pre-service learners experienced 18.9% [+13.6 points] mean increase in confidence vs. 15% [+10.8 points] for Y2 VW learners vs. 14.3% [+10.3 points] for OC learners [p<0.001])) (Tables 3 and S4). Gains in confidence were significantly greater among pre-service learners (p<0.001) and post-graduate new provider learners (p<0.001) who participated in Y1 in-person training compared to the VW or OC programs (Fig 1); differences in confidence between the three programs were not significantly different among post-graduate provider learners (p = 0.053). Increases in confidence were greater among medical, nursing/midwifery, and pharmacy learners participating in Y1 in-person training compared to the VW or OC formats (p = 0.016, p<0.001 and p<0.001 respectively); differences in confidence among laboratorian participants between the three course were not significantly different (p = 0.876) (Fig 2). Regardless of the domain of confidence assessed ((i) clinical confidence, (ii) confidence to engage in Inter-professional practice (IP), and (iii) confidence in quality improvement practice (QI)) there was no significant difference in gains in confidence between participants in the VW or OC formats (S1 Fig).
Sensitivity analysis
There was a small proportion of participants for whom we had complete demographic data but not evaluative data. We undertook a sensitivity analysis to determine if those participants for whom we did not have complete demographic records responded differently to the pre- and post-surveys from those for whom we did not have complete demographic data. In this additional analysis, S1 Table, we found no differences in mean scores for knowledge or confidence between those for whom we had demographic data and those we did not, regardless of whether they were in the IP, VW or OC group.
Discussion
In this multi-country capacity-building intervention, which included over 13,000 learners from diverse cadres and clinical settings, in-person and different online educational strategies all led to significant improvements in knowledge and clinical confidence. Moreover, the transition from in-person to online and remote training that occurred because of the COVID-19 pandemic, did not have a major deleterious impact on learner outcomes, as demonstrated by the gains in both knowledge and confidence across all three educational programs and for all types of learners. Nonetheless, gains in knowledge and clinical confidence were greater among in-service learners in Y1 compared to learners in Y2. These findings have several important programmatic and policy implications, as outlined below.
Firstly, it was notable that the magnitude of gains in technical knowledge and clinical confidence were smaller among learners who participated in the synchronous Virtual Workshop and blended Online Course compared to those in the in-person training. Across all domains of confidence assessed, mean improvements in subjective confidence were greatest among learners who participated in the in-person program, regardless of cadre and career stage. We are cautious drawing conclusions about the implications of these findings on clinical outcomes since this was not assessed in this study. Nonetheless, these findings validate other research demonstrating the importance of online tools as satisfactory for acquiring knowledge, but inferior modalities for acquisition of clinical skills or technical confidence [21,22]. These findings underscore the importance of in-person learning for inter-professional training. They also highlight the potential limitations of on-line training, which may be considerable, if deficiencies in knowledge and confidence lead to adverse clinical outcomes. These results should be weighed against the relative merits of online remote learning modalities. For example, online learning tools are likely to be more affordable in many parts of SSA, especially given that learners can participate even from remote or inaccessible geographic locations.10 Recent studies in SSA have highlighted substantial opportunity costs associated with in-person training programs for in-service learners that are potentially obviated by distance learning modalities [14,15,23]. These benefits notwithstanding [24], more research is warranted to ensure that online capacity-building interventions are used effectively, and in combination with traditional in-person strategies, to foster clinical confidence and inter-professional collaboration, and determine if they lead to improvements in provider performance and superior clinical outcomes. It is notable that confidence gains were greater for learners who participated in the synchronous Virtual Workshops compared to the blended Online Course format, highlighting that even when in-person workshops may not be feasible or affordable, strategies to facilitate real-time learning using online video-streaming platforms, can be leveraged, even in resource-variable settings in SSA.
Secondly, this analysis underscores how educational technologies can play a critical role in addressing inequities in access to training opportunities in SSA. Despite the profound, disruptive impacts of the COVID-19 pandemic, both the online synchronous and blended programs enabled more diverse and inclusive approaches to inter-professional HIV training for in-service learners. Learners in Y2 were more likely to be female and from more diverse cadres than learners in Y1. These findings are in contrast with other data highlighting how the COVID-19 pandemic revealed and increased pre-existing inequities in professional development opportunities for healthcare professionals in LMICs [2,25,26], Although many health professions training institutions in SSA lack access and capacity to use digital technologies to deliver HIV training [27] our results affirm the critical role that online training interventions can play in advancing professional development opportunities especially for those healthcare professions historically underserved by capacity building interventions [28].
Thirdly, this study highlights the importance of adaptive and resilient educational strategies in response to an emergent infectious disease threat. Although very few of the partner institutions involved in the program had adopted online learning tools before COVID-19, the pandemic accelerated a shift to digital technologies in health professions education even in countries with limited digital infrastructure. Moreover, use of digital platforms allowed learners in diverse settings to acquire new knowledge rapidly and effectively. While further research is critical to better understand how to optimally leverage digital tools in combination with more immersive in-person teaching formats, our study offers an African-specific blueprint for how to rapidly upskill the healthcare workforce, even in the setting of a major public health emergency.
The study has several limitations. Firstly, data is from an HIV-specific training program, and evaluative data limited to learners who completed the registration survey and assessment questions. Further research is warranted to determine if lessons learned from the analysis are applicable to other disease-specific capacity-building interventions. The three different approaches employed different pedagogical techniques (for example both in-person and virtual workshops included small group breakouts) that might have impacted learner outcomes as much as the medium of delivery (online vs. in-person). More health professions education research to understand how best to employ digital tools in Africa is a priority that has been highlighted by numerous stakeholders [16,29–31]. Additionally, our assessment was limited to knowledge and confidence, as data regarding clinical outcomes was not available. Since different health professional training institutions participated in the program in Y1 and Y2 (in part due to digital literacy and internet infrastructure), we were also unable to compare learner outcomes between Y1 and Y2 within individual institutions or country. Nonetheless, we assert that the aggregate findings have broad applicability to diverse Africa settings. Finally, institutions in Y2 were different from Y1 since participation was limited by digital literacy and Internet infrastructure. In Y2, we only included information from learners that were able to participate in the digital modalities highlighted here; the study does not address barriers faced by learners unable to access online training programs in SSA. These limitations highlight substantial digital inequities in health professions training institutions between and within countries in SSA [5,25], and underscore the ongoing need to support the development of local technological infrastructure for online education across SSA.
Supporting information
S1 Fig. Confidence score difference by education program for Clinical confidence (A, n = 3027), Interprofessional Confidence (B, n = 3027), and Confidence to incorporate Quality Improvement (Ql) into clinical practice (C, n = 3027).
https://doi.org/10.1371/journal.pgph.0001654.s001
(TIF)
S1 Table. Assessment questions for each module.
https://doi.org/10.1371/journal.pgph.0001654.s002
(DOCX)
S2 Table. The maximum score for each assessment category.
https://doi.org/10.1371/journal.pgph.0001654.s003
(DOCX)
S3 Table. Total learner baseline mean knowledge score differences for each educational program (Y1 In-person, Y2 Virtual Workshop, and Y2 Online Course) by Total Score, Training Level, and Health Profession compared using Wilcoxon Signed Rank Test.
https://doi.org/10.1371/journal.pgph.0001654.s004
(DOCX)
S4 Table. Total learner baseline mean confidence score differences for each educational program (Y1 In-person, Y2 Virtual Workshop, and Y2 Online Course) by Total Score and Confidence Type using Wilcoxon Signed Rank Test.
https://doi.org/10.1371/journal.pgph.0001654.s005
(DOCX)
References
- 1. Reid M, Suleman F, De Villiers M. The SARS-CoV-2 pandemic: An urgent need to relook at the training of the African health workforce. S Afr Med J. 2020;110(4):12875. Epub 2020/07/14. pmid:32657733.
- 2. Barteit S, Jahn A, Banda SS, Barnighausen T, Bowa A, Chileshe G, et al. E-Learning for Medical Education in Sub-Saharan Africa and Low-Resource Settings: Viewpoint. J Med Internet Res. 2019;21(1):e12449. Epub 20190109. pmid:30626565; PubMed Central PMCID: PMC6329426.
- 3. Marsh RH, Plyler C, Miller M, Klar R, Adeiza M, Wachekwa I, et al. Facing COVID-19 in Liberia: Adaptations of the Resilient and Responsive Health Systems Initiative. Ann Glob Health. 2021;87(1):101. Epub 20211008. pmid:34707981; PubMed Central PMCID: PMC8499715.
- 4. Kagawa MN, Chipamaunga S, Prozesky D, Kafumukache E, Gwini R, Kandawasvika G, et al. Assessment of Preparedness for Remote Teaching and Learning to Transform Health Professions Education in Sub-Saharan Africa in Response to the COVID-19 Pandemic: Protocol for a Mixed Methods Study With a Case Study Approach. JMIR Res Protoc. 2021;10(7):e28905. Epub 20210728. pmid:34254943; PubMed Central PMCID: PMC8320735.
- 5. Chitungo I, Mhango M, Mbunge E, Dzobo M, Musuka G, Dzinamarira T. Utility of telemedicine in sub-Saharan Africa during the COVID-19 pandemic. A rapid review. Hum Behav Emerg Technol. 2021;3(5):843–53. Epub 20211102. pmid:34901772; PubMed Central PMCID: PMC8653215.
- 6. Skinner NA, Job N, Krause J, Frankel A, Ward V, Johnston JS. The Use of Open-Source Online Course Content for Training in Public Health Emergencies: Mixed Methods Case Study of a COVID-19 Course Series for Health Professionals. JMIR Med Educ. 2023;9:e42412. Epub 20230223. pmid:36735834; PubMed Central PMCID: PMC9999253.
- 7. Enyama D, Balti EV, Simeni Njonnou SR, Ngongang Ouankou C, Kemta Lekpa F, Noukeu Njinkui D, et al. Use of WhatsApp(R), for distance teaching during COVID-19 pandemic: Experience and perception from a sub-Saharan African setting. BMC Med Educ. 2021;21(1):517. Epub 20211002. pmid:34598681; PubMed Central PMCID: PMC8486629.
- 8. Ogbenna A, Drane D, Crowe AN, Oyedele O, Hauser J, Soyannwo O, et al. Building the Nigerian Palliative Care Workforce: An Interdisciplinary Distance Learning Training Program. Ann Glob Health. 2022;88(1):96. Epub 20221027. pmid:36380743; PubMed Central PMCID: PMC9615602.
- 9. Downie A, Mashanya T, Chipwaza B, Griffiths F, Harris B, Kalolo A, et al. Remote Consulting in Primary Health Care in Low- and Middle-Income Countries: Feasibility Study of an Online Training Program to Support Care Delivery During the COVID-19 Pandemic. JMIR Form Res. 2022;6(6):e32964. Epub 20220614. pmid:35507772; PubMed Central PMCID: PMC9200055.
- 10. Khoza-Shangase K, Moroe N, Neille J. Speech-Language Pathology and Audiology in South Africa: Clinical Training and Service in the Era of COVID-19. Int J Telerehabil. 2021;13(1):e6376. Epub 20210622. pmid:34345349; PubMed Central PMCID: PMC8287713.
- 11. Cleland J, McKimm J, Fuller R, Taylor D, Janczukowicz J, Gibbs T. Adapting to the impact of COVID-19: Sharing stories, sharing practice. Med Teach. 2020;42(7):772–5. Epub 20200513. pmid:32401079.
- 12. Kiguli-Malwadde E, Budak JZ, Chilemba E, Semitala F, Von Zinkernagel D, Mosepele M, et al. Developing an interprofessional transition course to improve team-based HIV care for sub-Saharan Africa. BMC Medical Education. 2020;20(1):499. pmid:33298029
- 13. Kiguli-Malwadde E, Forster M, Martin S, Chilemba E, Couper I, Motlhatlhedi K, et al. Evaluating the impact of a multicountry interprofessional training programme to improve HIV knowledge and clinical confidence among healthcare workers in sub-Saharan Africa: a cohort study. BMJ Open. 2022;12(7):e060079. Epub 20220720. pmid:35858724; PubMed Central PMCID: PMC9305810.
- 14. Rowe SY, Peters DH, Holloway KA, Chalker J, Ross-Degnan D, Rowe AK. A systematic review of the effectiveness of strategies to improve health care provider performance in low- and middle-income countries: Methods and descriptive results. PLoS One. 2019;14(5):e0217617. Epub 20190531. pmid:31150458; PubMed Central PMCID: PMC6544255.
- 15. Leslie HH, Gage A, Nsona H, Hirschhorn LR, Kruk ME. Training And Supervision Did Not Meaningfully Improve Quality Of Care For Pregnant Women Or Sick Children In Sub-Saharan Africa. Health Aff (Millwood). 2016;35(9):1716–24. pmid:27605655.
- 16. Rowe AK, Labadie G, Jackson D, Vivas-Torrealba C, Simon J. Improving health worker performance: an ongoing challenge for meeting the sustainable development goals. BMJ. 2018;362:k2813. Epub 20180730. pmid:30061366; PubMed Central PMCID: PMC6283359.
- 17. Squires JE, Sullivan K, Eccles MP, Worswick J, Grimshaw JM. Are multifaceted interventions more effective than single-component interventions in changing health-care professionals’ behaviours? An overview of systematic reviews. Implement Sci. 2014;9:152. Epub 20141006. pmid:25287951; PubMed Central PMCID: PMC4194373.
- 18. Thomas PA, Kern DE, Hughes MT, Chen BY. Curriculum development for medical education: a six-step approach. Third edition. ed. Baltimore: Johns Hopkins University Press; 2016. xii, 300 pages p.
- 19. Prober CG, Khan S. Medical education reimagined: a call to action. Acad Med. 2013;88(10):1407–10. pmid:23969367.
- 20. Kiguli-Malwadde E, Budak JZ, Chilemba E, Semitala F, Von Zinkernagel D, Mosepele M, et al. Developing an interprofessional transition course to improve team-based HIV care for sub-Saharan Africa. BMC Med Educ. 2020;20(1):499. Epub 2020/12/11. pmid:33298029; PubMed Central PMCID: PMC7725217.
- 21. Abbasi MS, Ahmed N, Sajjad B, Alshahrani A, Saeed S, Sarfaraz S, et al. E-Learning perception and satisfaction among health sciences students amid the COVID-19 pandemic. Work. 2020;67(3):549–56. pmid:33185620.
- 22. Vaona A, Banzi R, Kwag KH, Rigon G, Cereda D, Pecoraro V, et al. E-learning for health professionals. Cochrane Database Syst Rev. 2018;1:CD011736. Epub 20180121. pmid:29355907; PubMed Central PMCID: PMC6491176.
- 23. Ridde V. Per diems undermine health interventions, systems and research in Africa: burying our heads in the sand. Trop Med Int Health. 2010;15(7):E1–E4. pmid:28639744.
- 24. McKenzie-White J, Mubuuke AG, Westergaard S, Munabi IG, Bollinger RC, Opoka R, et al. Evaluation of a competency based medical curriculum in a Sub-Saharan African medical school. BMC Med Educ. 2022;22(1):724. Epub 20221014. pmid:36242004; PubMed Central PMCID: PMC9569118.
- 25. Schmutz AMS, Jenkins LS, Coetzee F, Conradie H, Irlam J, Joubert EM, et al. Re-imagining health professions education in the coronavirus disease 2019 era: Perspectives from South Africa. Afr J Prim Health Care Fam Med. 2021;13(1):e1–e5. Epub 20210810. pmid:34476976; PubMed Central PMCID: PMC8424738.
- 26. Al-Balas M, Al-Balas HI, Jaber HM, Obeidat K, Al-Balas H, Aborajooh EA, et al. Correction to: Distance learning in clinical medical education amid COVID-19 pandemic in Jordan: current situation, challenges, and perspectives. BMC Med Educ. 2020;20(1):513. Epub 20201216. pmid:33327927; PubMed Central PMCID: PMC7741866.
- 27. Aboshady OA, Radwan AE, Eltaweel AR, Azzam A, Aboelnaga AA, Hashem HA, et al. Perception and use of massive open online courses among medical students in a developing country: multicentre cross-sectional study. BMJ Open. 2015;5(1):e006804. Epub 20150105. pmid:25564149; PubMed Central PMCID: PMC4289712.
- 28. Sharp A, Jain V, Alimi Y, Bausch DG. Policy and planning for large epidemics and pandemics—challenges and lessons learned from COVID-19. Curr Opin Infect Dis. 2021;34(5):393–400. pmid:34342301; PubMed Central PMCID: PMC8452318.
- 29. Agyepong IA, Sewankambo N, Binagwaho A, Coll-Seck AM, Corrah T, Ezeh A, et al. The path to longer and healthier lives for all Africans by 2030: the Lancet Commission on the future of health in sub-Saharan Africa. Lancet. 2017;390(10114):2803–59. Epub 2017/09/18. pmid:28917958.
- 30. Omaswa F, Kiguli-Malwadde E, Donkor P, Hakim J, Derbew M, Baird S, et al. Medical Education Partnership Initiative gives birth to AFREhealth. Lancet Glob Health. 2017;5(10):e965–e6. Epub 2017/09/16. pmid:28911758; PubMed Central PMCID: PMC5712669.
- 31. Rowe AK, Rowe SY, Peters DH, Holloway KA, Chalker J, Ross-Degnan D. Effectiveness of strategies to improve health-care provider practices in low-income and middle-income countries: a systematic review. Lancet Glob Health. 2018;6(11):e1163–e75. Epub 20181008. pmid:30309799; PubMed Central PMCID: PMC6185992.