https://orcid.org/0000-0003-1257-969X
Figures
Abstract
Objective
This review evaluates the effectiveness of smartphone applications in improving academic performance and clinical practice among healthcare professionals and students.
Methods
This study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Articles were retrieved from PubMed, Scopus, and Cochrane library through a comprehensive search strategy. Studies that included medical, dental, nursing, allied healthcare professional, undergraduates, postgraduates, and interns from the same disciplines who used mobile applications for their academic learning and/or daily clinical practice were considered.
Results
52 studies with a total of 4057 learner participants were included in this review. 33 studies (15 RCTs, 1 cluster RCT, 7 quasi-experimental studies, 9 interventional cohort studies and 1 cross-sectional study) reported that mobile applications were an effective tool that contributed to a significant improvement in the knowledge level of the participants. The pooled effect of 15 studies with 962 participants showed that the knowledge score improved significantly in the group using mobile applications when compared to the group who did not use mobile applications (SMD = 0.94, 95% CI = 0.57 to1.31, P<0.00001). 19 studies (11 RCTs, 3 quasi-experimental studies and 5 interventional cohort studies) reported that mobile applications were effective in significantly improving skills among the participants.
Conclusion
Mobile applications are effective tools in enhancing knowledge and skills. They can be considered as effective adjunct tools in medical education by considering their low expense, high versatility, reduced dependency on regional or site boundaries, online and offline, simulation, and flexible learning features of mobile apps.
Citation: Chandran VP, Balakrishnan A, Rashid M, Pai Kulyadi G, Khan S, Devi ES, et al. (2022) Mobile applications in medical education: A systematic review and meta-analysis. PLoS ONE 17(3): e0265927. https://doi.org/10.1371/journal.pone.0265927
Editor: M. Shafiqur Rahman, University of Dhaka, BANGLADESH
Received: February 19, 2021; Accepted: March 10, 2022; Published: March 24, 2022
Copyright: © 2022 Chandran et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Applications or apps are software programs designed to run on a computer/tablet/mobile phone to accomplish a particular purpose. Mobile applications play an integral role in medical education, as healthcare professionals (HCPs) and students use these emerging technologies during their training and practice. They have become inevitable in clinical educational settings, particularly as they are accessible for learning anywhere. Many applications are designed to support HCPs with significant tasks such as documentation and time management, health record management and access, consulting and networking, information and reference acquisition, clinical care and monitoring, medical education, training, and clinical decision making [1]. Apps are popular as they have a multimedia approach and include images, videos, texts, and podcasts [2].
The medical education framework encompasses a highly standardized curriculum in a range of preclinical and clinical settings. The designs and specifications of this framework are defined by the Medical Education Board of every country. Competent HCPs are not born; they are educated to combine the art of science with recent concepts of illness, diagnosis, treatment, and empathy. Education in medical institutions endured abrupt disturbances in the face of the 2019 coronavirus pandemic (COVID-19) [3]. There is ambiguity surrounding how long the condition will continue to exist [4]. Since the pandemic, educational experts are realizing that the implementation and administration of medical curricular modifications will be versatile, building on the current pedagogical framework [5].
In today’s context, as mobile apps offer new learning opportunities, mobile learning is emerging as the newer form of learning and implemented as a result of this pandemic. Most experts suggest that mobile learning initiatives will significantly enhance the learning processes in healthcare. Several randomized trials of mobile learning approaches that aimed at improving knowledge, skills, attitude, and satisfaction have been released. Nevertheless, these results were not carefully checked, and there was no quantification in the effectiveness of mobile learning interventions [6]. Studies suggest that more than 85% of HCPs and medical students use a smartphone, and 30–50% use medical apps for learning and collecting information [7]. The field of medical applications development is very diverse and many applications are being designed to fulfill the needs of medical students and HCPs. As time progressed, smartphones and mobile apps have started to replace conventional knowledge acquisition settings and provide medical students with unparalleled ease of access to medical information and expertise [8]. Despite this acceptance, there is limited research that substantiates the claim that the use of smartphones is effective in enhancing the academic performance of HCPs and/ or students [9].
Mobile apps have a relatively low expense, high versatility, and reduced dependency on regional or site boundaries which encourage stakeholder investments (countries, networks, and universities) and learner demands. This review aims to evaluate the effectiveness of smartphone-based applications in improving academic performance and clinical practice among healthcare professionals and/or students by assessing the impact of the use of smartphone-based interventions and applications in knowledge acquisition and skill levels.
Materials & methods
This study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Guidelines for reporting the findings. The study protocol was registered in PROSPERO: International Prospective Register of Systematic Reviews with the registration number: CRD42019133670 [10] before conducting the study.
Criteria for inclusion of studies
Type of studies.
Quantitative studies such as randomized controlled trials (RCTs), cluster RCTs (cRCTs), quasi-experimental studies, interventional cohort studies, and cross-sectional studies that assessed the change in knowledge or skill following the use of any mobile application in the study population were included. We excluded studies that assessed the perception of the use of mobile applications, non-English publications, editorials, reviews, conference proceedings, qualitative studies and those on the design and development process.
Type of participants.
Medical, dental, nursing, allied HCPs, undergraduates, postgraduates, and internship students from the same disciplines who used mobile applications for academic learning and/or daily clinical practice were included. Participants were not excluded based on their age, gender, experience, and any other socio-demographic characters.
Types of intervention.
We included studies that assess the effectiveness of online/offline mobile applications in the acquisition of knowledge and skill development. Studies that include mobile applications related to drug information, guidelines, health parameter calculators, diagnosis, disease, medical notes, case studies, and simulation-aided models to improve knowledge and skills in practice as an intervention were also included. Any intervention using stationary technology, such as desktop computers, e-learning courses attended through internet platforms on mobile phones/tablets/computers, and mobile applications on non-medical topics related to knowledge and skill development among HCPs/ students were excluded.
Search methods for identification of studies
Electronic & other resources searches.
We retrieved articles from PubMed, Scopus, and Cochrane library databases using a comprehensive search strategy from inception till December 2020. We identified all the possible keywords and medical subject heading (MeSH) terms for the terms “healthcare professionals”, “doctors”, “nurses”, “pharmacists”, “allied healthcare professionals”, “paramedical professionals” and “mobile apps” from the previously published studies and databases. We also used the truncation search method (*) to avoid the possibility of missing studies due to changes in terms in different studies. Further, we searched Google, bibliography of included studies, and relevant reviews published in the same area to look for any other additional studies. The detailed search strategy in different databases is provided in the S1 Appendix.
Selection of studies and data extraction.
Two review authors independently determined the eligibility of the studies by examining the title and abstract of the retrieved studies based on the pre-defined inclusion and exclusion criteria, followed by full-text screening. Two review authors independently extracted data from the included studies, using a data extraction form developed using Microsoft Excel. The following information was extracted: characteristics of study (authors name, year of publication, study location), characteristics of the population (type of healthcare professionals/students [year of professional education]), study design, number of participants in study groups at baseline and at the time of completion, characteristics of intervention and/ or control (name and content details of the mobile app, nature of control), outcome measures (scale used), result (knowledge/skill score in mean and standard deviation (SD)/ standardized mean difference (SMD) and SD/median/P value) and limitations of the included studies. All the disagreements during the study selection and data extraction were resolved by consensus or discussion with another researcher.
Assessment of risk of bias in the included studies.
The quality of RCTs was assessed using the Cochrane risk of bias assessment tool [11] and other study designs were (interventional and observational) assessed using the Modified New castle Ottawa scale [12]. Cochrane risk of bias assessment tool assesses the following criteria of studies: (i) random sequence generation, (ii) allocation concealment, (iii) blinding of participants and personnel, (iv) blinding of outcome assessment, (v) incomplete outcome data, (vi) selective reporting and (vii) any other bias not included in other types into high risk, low risk and unclear risk categories. Modified New castle Ottawa scale assesses the following criteria: (i) representativeness of intervention group (1 point) (ii) selection of comparison group (1point) (iii) comparability of the comparison group (2 points) (iv) study retention (1 point) (v) blinding of assessment (1 point), totaling a maximum of 6 points. Two independent reviewers were involved in the quality assessment and any disagreements were resolved by discussion and consensus.
Measures of mobile application effectiveness.
We separately analyzed the change in knowledge and skill acquisition by using mobile applications among the HCPs and/or students. Our analysis was based on the consideration of continuous outcome variables (change in participants’ knowledge and/or skill score). Studies assessing both skill and knowledge were reported separately in both domains with relevant outcome details.
Data synthesis.
The characteristics of the included studies and outcome measures collected were presented in a tabular form and a narrative synthesis was performed. The meta-analysis was performed using the Review Manager version 5.3. [13] Studies reported a single combined score for knowledge and/or skill with continuous data such as mean and SD or those that were calculated from the available data was considered for the meta-analysis. Comparisons of different types of mobile applications are excluded. Studies with multiple arms and multiple applications are considered as separate studies if their knowledge/skill scores were presented separately. The analyzed data is presented as a standardized mean difference along with SD. All the statistical analyses were performed by one author and cross-checked by another author and any disagreements were resolved by discussion.
Heterogeneity and publication bias.
We used a random-effect model for the data analysis. Further, the sources of heterogeneity were explored through a subgroup analysis according to the study design. Sensitivity analysis was done to ensure the robustness of the study findings by eliminating studies, which had lesser weight and the least number of participants. Funnel plot asymmetry by visual inspection was used for the detection of publication bias, which was further assessed for statistical significance by Egger’s and Begg’s tests. Stata trial version 15 software was used for the detection of publication bias.
Results
Description of studies
Results of the search.
The literature search identified a total of 5849 records out of which 4600 records were screened after the removal of duplication. Finally, 52 articles were included following the exclusion of 4116 records after the initial screening and 432 articles after the full-text screening. The process of identification, screening, eligibility, and synthesis of findings is illustrated in the PRISMA flow diagram (Fig 1).
Summary & characteristics of studies.
Out of 52 included studies, 29 assessed change in the knowledge level [8, 14–41], 10 assessed change in the skill level [42–51], and 13 assessed change in both the knowledge and skill level [9, 52–63]. These studies that provided data from a total of 4057 (sample ranges from 5 to 448) learner participants were included in this review. The included studies consist of 27 RCTs, 1 cluster RCT, 8 quasi-experimental studies, 15 interventional cohort studies, and 1 cross-sectional study, which was published between 2011 and 2020. Major portion of the studies (36/52) were performed in developed countries namely USA (n = 10) [18, 21, 23, 24, 32, 34, 38, 42, 50, 60], UK (n = 7) [37, 43, 44, 47–49, 52], South Korea (n = 5) [53, 58–61], Brazil (n = 4) [15, 25, 40, 45], Germany (n = 3) [17, 19, 63], Spain (n = 3) [29, 55, 56], Turkey (n = 2) [22, 54], Chile (n = 2) [9, 51], while the remaining 10 studies were from France [16], Australia [20], China [27], Iran (n = 4) [14, 28, 30, 41], Rwanda [57], India [33], Taiwan [35], Canada [36], Pakistan [8], Thailand [39], Indonesia [26], Palestine [31] and Finland [46]. The duration of the use of mobile applications ranged from onr time use to a year long usage. The characteristics of the included studies are given in the S2 Appendix. It contains study author, country, year, study design, population, sample size, intervention details, outcome measures, and reported limitations in individual studies.
Quality of included studies.
The quality RCTs and cluster RCT were assessed using the Cochrane risk of bias assessment tool and the results are summarized as the risk of bias graph (S3 Appendix) and summary graph (S4 Appendix). Each category of risk of bias was represented as percentages among the included studies. Interventional and observational studies other than RCTs were assessed using the Modified New castle Ottawa scale. The quality assessment score of included studies using the New castle Ottawa Scale is summarized in the S5 Appendix.
Effect of intervention on knowledge of HCPs
Qualitative analysis.
Out of 52 studies, 42 studies reported changes in knowledge level after the use of mobile applications among HCPs/students either in comparison with the control group or comparing the pre and post-test scores of the same group. 33 studies [15 RCTs [9, 14, 15, 17–22, 24–26, 52, 56, 57], 1 cluster RCT [27], 7 quasi-experimental studies [28, 29, 31, 40, 58, 60, 61], 9 interventional cohort studies [33–39, 41, 62] and 1 cross-sectional study [8]] reported that mobile applications were effective tools that led to a significant improvement of knowledge level among the participants. Among that, one RCT conducted by Noll et al., [17] compared the effectiveness of augmented reality mobile application vs. mobile application and concluded that augmented reality mobile application was more effective in increasing the knowledge level than mobile application without augmented reality. 8 studies [5 RCTs [16, 23, 53–55], 1 quasi-experimental study [30], and 2 interventional cohort studies [32, 63]] reported that mobile applications were not effective in improving knowledge among study participants. Among that, one RCT conducted by Kim et al., 2018 [53] compared the effectiveness of interactive mobile application vs non-interactive mobile application and concluded that there is no significant difference in the knowledge level improvement between the two groups. Another quasi-experimental study conducted by Young Yoo et al., [59] reported that mobile applications were effective in improving knowledge in one study condition only (out of the 2 conditions). The overall summary of the effects of mobile applications on the knowledge of HCPs is illustrated in Table 1.
Quantitative analysis.
Pooled effect of 15 [14, 15, 19, 20, 26–29, 31, 36, 58, 60, 61] studies with 962 participants showed that the knowledge score improved significantly in the group using the mobile application when compared to the group that does not use mobile application (SMD = 0.94, 95% CI = 0.57 to1.31, P<0.00001) (Study by Kang 2018 [58] is considered as two studies in the meta-analysis as it analyzes the effectiveness of two different applications and reported the outcomes separately. The study by Kang 2020 [61] is considered as two studies in the meta-analysis as it has three groups and analyzed outcomes separately). The random effect model was considered because of the significant statistical heterogeneity among studies (Fig 2). A subgroup analysis was performed because of the heterogeneity in the included study designs. The pooled effect of individual study designs (Cluster RCT, quasi experimental studies and interventional cohort study) showed that participants using mobile applications had better knowledge enhancement scores compared to the group that did not use mobile applications among the study participants (Cluster RCT: SMD = 2.19, 95% CI = 1.72 to2.66, P<0.00001; quasi-experimental studies: SMD = 0.97, 95% CI = 0.59 to 1.36, P<0.00001; interventional cohort study: SMD = 1.69, 95% CI = 1.06 to 2.33, P <0.00001) giving total weightage of 67.8%.
Effect of intervention on skills of HCPs
Qualitative analysis.
Out of 52 studies, 23 studies reported changes in skill level after the use of mobile applications among healthcare professionals/students either in comparison with the control group or comparing the pre and post-test scores of the same group. 19 studies [11 RCTs [9, 43–45, 47, 48, 52, 53, 55–57], 3 quasi-experimental studies [58, 60, 61] and 5 interventional cohort studies [49–51, 62, 63]] reported that mobile applications were effective tool and significantly improved skill levels among participants. Among that, one RCT conducted by Kim et al., in 2018 [53] compared the effectiveness of an interactive mobile application vs a non-interactive mobile application and concluded that interactive mobile application was more effective in improving skill than the non-interactive mobile application. Three studies [2 RCTs [46, 54] and 1 quasi-experimental study [59]] reported that mobile applications were not effective in improving skill among study participants. One RCT conducted by Nadir et al., in 2019 [42] reported that a mobile application was effective in improving skill in one study condition only (out of two conditions). The overall summary of the effects of mobile applications on the knowledge of HCPs is illustrated in Table 2.
Quantitative analysis.
A meta-analysis of six [46, 58, 60, 61] studies with 381 participants studies showed no significant change in skill level among the group using the mobile applications and the group that did not use mobile applications (SMD = 0.36, 95% CI = -0.23 to 0.96, P = 0.23). Included study designs for data synthesis were RCT (n = 1) [46] and quasi-experimental studies (n = 5) [58, 60] (Study by Kang 2018 [58] is considered as two studies in the meta-analysis as it analyzed the effectiveness of two different applications and reported outcomes separately. The study by Kang 2020 [61] is considered as two studies in the meta-analysis as it has three groups and analyzed outcomes separately). The random-effect model was considered because of the significant statistical heterogeneity among studies. A subgroup analysis was performed based on study designs. The pooled effect of quasi-experimental studies showed that the group using the mobile application was better at skill enhancement than the group that did not use mobile application among the study participants (SMD = 0.59, 95% CI = 0.18 to 0.99, P = 0.04) giving a total weightage of 82.4% (Fig 3).
Sensitivity analysis.
A sensitivity analysis was performed by removing four studies from the knowledge assessment and one study from the skill assessment that has less weightage, which also supported the previous findings. The result of sensitivity analysis is depicted in S6 Appendix and S7 Appendix.
Publication bias.
An obvious asymmetry was observed by the visual inspection of the funnel plot (S8 Appendix), which was not statistically confirmed by Egger’s (P = 0.5) and Begg’s test (P = 0.49). Hence there is no publication bias among the included studies in the meta-analysis.
Educational area of mobile applications.
The majority of researchers used mobile applications for education on anatomy, surgery, respiratory conditions, dermatology, basic life support (BLS), pathology, dose calculation, and radiology. The characteristic details of the intervention and educational area details are depicted in S2 Appendix and Fig 4.
Network connectivity and mobile apps
Most of the applications that cater to medical education were web-based mobile applications that require an internet connection for access. There was no significant change in the enhancement of knowledge and/or skill among participants using online and offline versions of mobile apps (S9 Appendix). Developing mobile applications which can be accessed without internet connectivity will increase the acceptability and use of mobile apps among healthcare professionals and/or students living in low to middle-income countries and rural areas.
Mobile application operating system
Android and iPhone operating systems (iOS) were used for mobile applications. Both operating system-based applications were effective in enhancing knowledge or skill. There was no significant change in the enhancement of knowledge or skill among participants who used android or iOS mobile apps (S10 Appendix). Developing mobile applications which can be operated through android will increase the acceptability and use of mobile apps among healthcare professionals and/or students living in low to middle-income countries.
Discussion
This study attempted to evaluate the effectiveness of mobile applications in enhancing knowledge and/or skill among HCPs and/or students. The findings from the study support the claim that mobile applications are an effective tool. Medical, paramedical, and allied healthcare professionals were the target population for evaluating the effectiveness of mobile apps in enhancing knowledge and skills. The educational topic, duration of intervention, nature of control/comparison group, and features of mobile apps were different in different studies. Even if the population and mobile apps have their differences, every study compared the change in knowledge and/or skill quantitatively with or without a comparison group. Variations were present in providing the type of educational material to the control/comparison group in different studies. Studies compared the effectiveness of mobile app learning vs traditional teaching or teaching using audio-video slides, simulator training, hardcopy learning materials and e-learning through computer platforms. All the studies compared the change in knowledge and/or skill level by using a mobile application with any of the regular teaching methods, which did not affect the quality of study results. The meta-analysis of the change in the skill of the included studies was not statistically significant. This is possibly due to the limited number of studies in the meta-analysis. Studies that reported skill as a single comparable score were only considered for the meta-analysis. The majority of studies reported multiple component scores in evaluating the skill of healthcare professionals. The systematic review findings supported the use of mobile applications in the enhancement of skill.
Different studies used different mobile applications with different educational topics. The “Touch surgery” application was used in five studies [21, 44, 47, 57, 62] but the educational topic was different. The choice of topics in a mobile app may differ due to the variety of study population, area of interest/expertise of study coordinators and/or study participants and required facilities (e. g.: simulation) in the institution.
Studies among various healthcare professionals reported mixed results regarding the usefulness of the e-learning, mobile learning and technology-enhanced learning. A Cochrane systematic review conducted by Vaona et al in 2018 compared traditional learning with e-learning and reported that e-learning may make little or no difference in health professionals’ behaviours, skills or knowledge [64]. A study conducted by Subhash et al, 2015 among medical students reported that smartphones can be effectively used for learning [65]. A study conducted by Snashall et al, 2016 among medical students reported that medical apps can be used as an adjunct in medical education, though the evidence remains limited [2]. A qualitative systematic review by the Digital Health Education Collaboration conducted by Lall et al, 2019 on the implementation of mobile learning for medical and nursing education reported that mobile learning can potentially play a substantial role in learning [66]. A study conducted by Dickinson et al, 2020 on the educational applications for surgery residents reported that technology-enhanced learning has become prevalent in surgical education and future studies are needed to assess the efficacy of educational apps for surgical education [67]. By considering all the other published studies to the best of the authors’ knowledge, this is the first systematic review that has attempted to quantitatively analyze the change in knowledge and/or skills by using any mobile application.
Additionally, this is the first systematic review assessing the effectiveness of online vs offline and android vs iOS based mobile applications in enhancing knowledge and/or skill quantitatively. Even if more studies focused on online applications, offline applications also were equally effective. Either android or iOS based mobile applications were used in different studies. Offline mobile applications, which can be accessed through android and iOS will increase the wide acceptability of mobile apps and reduce the economic burden. Moreover, they help tackle problems due to poor internet connectivity. It is the latest growing trend and developers of mobile apps should take tremendous interest in creating these applications [68].
No restrictions were imposed in the publication period during the literature search. But the included studies were published between 2011 and 2020. During the first decade of the 2000s, m-learning had grown in different forms and directions [69]. This may be the reason for the publication period ranging from 2011 to 2020. Among the 52 included studies around 36 studies are from developed nations. The remaining studies are only developing countries. This may be because the concept of m–learning originated in Europe and this encouraged researchers and educators to reconsider their view on mobile technology as a pedagogical tool [70]. Other possible reasons may include lack of resources, fear of adopting mobile technology, lack of skills of instructors, interruption in power supply and internet connectivity, affordability issues, low bandwidth, and trust deficit. This could impact the global representativeness of the review findings. Therefore, future research should address the effectiveness of mobile applications in learning in diverse populations from other countries.
Cochrane risk of bias assessment tool was used to assess the quality of included RCTs. The quality of the majority of included studies in all sub-domains was high to moderate except for the blinding of participants and personnel, which showed a high risk of bias based on the Cochrane tool. It is not possible to blind the participants in the study population as it provides a mobile application for education to the intervention group. Hence the overall risk of bias in the included studies can be considered as low to moderate. Modified New castle Ottawa scale was used to assess the risk of bias in the included studies other than RCTs. All studies have a comparison group scored 4 or more. Hence these can be considered as moderate to high-quality studies. Single group studies can be considered as low-quality studies based on lesser scores in Modified New castle Ottawa scale scoring. The findings from this review can be considered before developing any mobile application in medical education as the results are interpreted from high-quality studies.
However, this study has some limitations. In this study, the search was conducted in three databases only and this may lead to the missing of studies. We tried to avoid the biases due to the missing of studies by selecting large databases such as PubMed, Cochrane library and Scopus and also by developing a comprehensive search strategy by collecting maximum keywords from the published studies and MeSH term search. According to Cochrane guidelines, a minimum of two large electronic databases should be searched. We have chosen 3 major scientific databases for searching studies. Additionally, we have searched the bibliography of included studies, related reviews, and free search in Google Scholar. This resulted in the identification of five more studies. Hence the chance of missing studies is limited in our review. Secondly, we have excluded studies published in other languages except for English, conference proceedings and grey literature. This may lead to the missing of studies published in non-English languages. However, excluding studies from conference proceedings and grey literature sources may increase the strength of study findings by avoiding irrelevant and incomplete data. Heterogeneity in the assessment of knowledge and/or skill of the study population was another limitation. We have included studies that reported quantitative measurement of knowledge and/or skill score to avoid biases due to various measurement approaches. The meta-analysis was performed for studies provided mean and SD or could be calculated from the reported measures only.
Future directions
Smartphone learning encourages conversation, communication, and cooperation, as well as increased participation. Millennial learners prefer participant-centered, active, and self-directed. Mobile applications based learning can be integrated into medical education to evaluate its performance. Learning materials should be available in a digital format to be feasible to fuel smartphone learning and online education. The importance of education and training using mobile applications is enhanced in the current COVID-19 scenario. A faculty-guided strategy to select appropriate and cost-effective medical education applications can be used. Universities or healthcare organizations should adopt policies for faculty, staff and/or students regarding the use of mobile applications for educational purposes. Multicenter randomized controlled trials with a longer duration have to be conducted to assess the effectiveness and the retention of knowledge and/or skills of mobile applications in medical education among HCPs and/or students. Necessary measures should be employed to avoid dropout, cross-communication between participants and software compatibility issues among trial participants. Other validated practical methods should be employed to assess the knowledge and/or skill validated questionnaires.
Conclusion
Mobile applications are effective tools in enhancing knowledge and skills among healthcare professionals. Online/offline and android/iOS based applications were equally effective in enhancing knowledge. The prevailing pandemic situation demands medical education to increasingly utilize the opportunities of e-learning instead of conventional teaching. Mobile applications can be considered as an effective adjunct tool in medical education by considering the low expense, high versatility, reduced dependency on regional or site boundaries, online, offline, simulation and learning wherever features of mobile apps.
Supporting information
S1 Appendix. The detailed search strategy in different databases.
https://doi.org/10.1371/journal.pone.0265927.s001
(DOCX)
S2 Appendix. The characteristics of included studies.
https://doi.org/10.1371/journal.pone.0265927.s002
(DOCX)
S3 Appendix. Cochrane risk of bias graph of included RCTs.
https://doi.org/10.1371/journal.pone.0265927.s003
(PNG)
S4 Appendix. Cochrane risk of bias summary of included RCTs.
https://doi.org/10.1371/journal.pone.0265927.s004
(PNG)
S5 Appendix. New castle ottawa score of included studies.
https://doi.org/10.1371/journal.pone.0265927.s005
(DOCX)
S6 Appendix. Sensitivity analysis: Effect of intervention on knowledge of HCPs.
https://doi.org/10.1371/journal.pone.0265927.s006
(PNG)
S7 Appendix. Sensitivity analysis: Effect of intervention on the skill levels of HCPs.
https://doi.org/10.1371/journal.pone.0265927.s007
(PNG)
S9 Appendix. Network connectivity and mobile applications.
https://doi.org/10.1371/journal.pone.0265927.s009
(JPG)
S10 Appendix. Mobile application operating system.
https://doi.org/10.1371/journal.pone.0265927.s010
(DOCX)
Acknowledgments
The authors would like to acknowledge Manipal Academy of Higher Education, Manipal College of Pharmaceutical Sciences and Department of Pharmacy Practice for all the support and library facilities for the best possible completion of this study.
References
- 1. Ventola CL. Mobile Devices and Apps for Health Care Professionals: Uses and Benefits. P T. 2014 May;39(5):356–64. pmid:24883008
- 2. Snashall E, Hindocha S. The Use of Smartphone Applications in Medical Education. Open Medicine Journal [Internet]. 2016 Dec 27 [cited 2021 Jan 22];3(1). Available from: https://benthamopen.com/FULLTEXT/MEDJ-3-322
- 3. Wayne DB, Green M, Neilson EG. Medical education in the time of COVID-19. Sci Adv [Internet]. 2020 Jul 29 [cited 2021 Jan 22];6(31). Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7399480/ pmid:32789183
- 4.
Medical Student Education in the Time of COVID-19 | Medical Education and Training | JAMA | JAMA Network [Internet]. [cited 2021 Jan 22]. Available from: https://jamanetwork.com/journals/jama/fullarticle/2764138
- 5.
Medical education during COVID-19: Lessons from a pandemic | British Columbia Medical Journal [Internet]. [cited 2021 Feb 9]. Available from: https://bcmj.org/special-feature-covid-19/medical-education-during-covid-19-lessons-pandemic
- 6. Free C, Phillips G, Watson L, Galli L, Felix L, Edwards P, et al. The effectiveness of mobile-health technologies to improve health care service delivery processes: a systematic review and meta-analysis. PLoS Med 2013;10(1):e1001363. pmid:23458994
- 7. Ozdalga E, Ozdalga A, Ahuja N. The smartphone in medicine: a review of current and potential use among physicians and students. J Med Internet Res 2012;14(5):e128. pmid:23017375
- 8. A H, Mu S, Mn K, A H, Mb A, T S. Usage and types of mobile medical applications amongst medical students of Pakistan and its association with their academic performance. Pak J Med Sci 2019;35(2):432–6. pmid:31086528
- 9. Martínez F, Tobar C, Taramasco C. Implementation of a Smartphone application in medical education: a randomised trial (iSTART). BMC Med Educ [Internet]. 2017 Sep 18 [cited 2021 Jan 22];17. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5604333/ pmid:28923048
- 10. Chandran Viji, Thunga Girish, Pai Girish, Khan Sohil, Devi Elsa, Poojari Pooja, et al. Smart phone apps for health professional education. PROSPERO 2019 CRD42019133670 Available from: http://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42019133670.
- 11.
Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions. Repr. with corr. Chichester: Wiley-Blackwell; 2009. 649 p. (Cochrane book series).
- 12. Cook DA, Reed DA. Appraising the quality of medical education research methods: the Medical Education Research Study Quality Instrument and the Newcastle-Ottawa Scale-Education. Acad Med 2015;90(8):1067–76. pmid:26107881
- 13.
RevMan [Internet]. [cited 2021 Jan 22]. Available from: /online-learning/core-software-cochrane-reviews/revman
- 14. Bonabi M, Mohebbi SZ, Martinez-Mier EA, Thyvalikakath TP, Khami MR. Effectiveness of smart phone application use as continuing medical education method in pediatric oral health care: a randomized trial. BMC Medical Education 2019;19(1):431. pmid:31752833
- 15. Velasco HF, Cabral CZ, Pinheiro PP, Azambuja R de CS, Vitola LS, Costa MR da, et al. Use of digital media for the education of health professionals in the treatment of childhood asthma. J Pediatr (Rio J) 2015;91(2):183–8. pmid:25431855
- 16. Clavier T, Ramen J, Dureuil B, Veber B, Hanouz J-L, Dupont H, et al. Use of the Smartphone App WhatsApp as an E-Learning Method for Medical Residents: Multicenter Controlled Randomized Trial. JMIR Mhealth Uhealth 2019;7(4):e12825. pmid:30964435
- 17. Noll C, von Jan U, Raap U, Albrecht U-V. Mobile Augmented Reality as a Feature for Self-Oriented, Blended Learning in Medicine: Randomized Controlled Trial. JMIR Mhealth Uhealth 2017;5(9):e139. pmid:28912113
- 18. Samra S, Wu A, Redleaf M. Interactive iPhone/iPad App for Increased Tympanic Membrane Familiarity. Ann Otol Rhinol Laryngol 2016;125(12):997–1000. pmid:27670957
- 19. Albrecht U-V, Folta-Schoofs K, Behrends M, von Jan U. Effects of mobile augmented reality learning compared to textbook learning on medical students: randomized controlled pilot study. J Med Internet Res 2013;15(8):e182. pmid:23963306
- 20. Stirling A, Birt J. An enriched multimedia eBook application to facilitate learning of anatomy. Anat Sci Educ 2014;7(1):19–27. pmid:23650104
- 21. Amer KM, Mur T, Amer K, Ilyas AM. A Mobile-Based Surgical Simulation Application: A Comparative Analysis of Efficacy Using a Carpal Tunnel Release Module. J Hand Surg Am 2017;42(5):389.e1–389.e9. pmid:28318742
- 22. Küçük S, Kapakin S, Göktaş Y. Learning anatomy via mobile augmented reality: Effects on achievement and cognitive load. Anat Sci Educ 2016;9(5):411–21. pmid:26950521
- 23. Brown A, Ajufo E, Cone C, Quirk M. LectureKeepr: A novel approach to studying in the adaptive curriculum. Med Teach 2018;40(8):834–7. pmid:30033790
- 24. Lacy FA, Coman GC, Holliday AC, Kolodney MS. Assessment of Smartphone Application for Teaching Intuitive Visual Diagnosis of Melanoma. JAMA Dermatol 2018;154(6):730–1. pmid:29799954
- 25. Fernandes Pereira FG, Afio Caetano J, Marques Frota N, Gomes da Silva M. Use of digital applications in the medicament calculation education for nursing. Invest Educ Enferm 2016;34(2):297–304. pmid:28569933
- 26. Putri NO, Barlianto W, Setyoadi . The Influence of Learning with Mobile Application in Improving Knowledge about Basic Life Support. IJPHRD 2019;10(8):1010–4.
- 27. Wang F, Xiao LD, Wang K, Li M, Yang Y. Evaluation of a WeChat-based dementia-specific training program for nurses in primary care settings: A randomized controlled trial. Appl Nurs Res 2017;38:51–9. pmid:29241520
- 28. Zia Ziabari SM, Monsef Kasmaei V, Khoshgozaran L, Shakiba M. Continuous Education of Basic Life Support (BLS) through Social Media; a Quasi-Experimental Study. Arch Acad Emerg Med 2019;7(1):e4. pmid:30847439
- 29. Briz-Ponce L, Juanes-Méndez JA, García-Peñalvo FJ, Pereira A. Effects of Mobile Learning in Medical Education: A Counterfactual Evaluation. J Med Syst 2016;40(6):136. pmid:27098779
- 30. Golshah A, Dehdar F, Imani MM, Nikkerdar N. Efficacy of smartphone-based Mobile learning versus lecture-based learning for instruction of Cephalometric landmark identification. BMC Med Educ 2020;20(1):287. pmid:32867758
- 31. Salameh B, Ewais A, Salameh O. Integrating M-Learning in Teaching ECG Reading and Arrhythmia Management for Undergraduate Nursing Students. International Journal of Interactive Mobile Technologies (iJIM) 2020;14(01):82–95.
- 32.
Using an Audience Response System Smartphone App to Improve Resident Education in the Pediatric Intensive Care Unit.—Abstract—Europe PMC [Internet]. [cited 2021 Jan 22]. Available from: https://europepmc.org/article/pmc/pmc5912270
- 33. Deshpande S, Chahande J, Rathi A. Mobile learning app: A novel method to teach clinical decision making in prosthodontics. Educ Health (Abingdon) 2017;30(1):31–4. pmid:28707634
- 34. Man C, Nguyen C, Lin S. Effectiveness of a smartphone app for guiding antidepressant drug selection. Fam Med 2014;46(8):626–30. pmid:25163042
- 35. Liu R-F, Wang F-Y, Yen H, Sun P-L, Yang C-H. A new mobile learning module using smartphone wallpapers in identification of medical fungi for medical students and residents. Int J Dermatol 2018;57(4):458–62. pmid:29399785
- 36. Fralick M, Haj R, Hirpara D, Wong K, Muller M, Matukas L, et al. Can a smartphone app improve medical trainees’ knowledge of antibiotics? Int J Med Educ 2017;8:416–20. pmid:29200402
- 37.
Weldon M, Poyade M, Martin JL, Sharp L, Martin D. Using Interactive 3D Visualisations in Neuropsychiatric Education. In: Rea PM, editor. Biomedical Visualisation: Volume 2 [Internet].Cham: Springer International Publishing; 2019 [cited 2021 Jan 22]. p. 17–27. (Advances in Experimental Medicine and Biology). Available from: https://doi.org/10.1007/978-3-030-14227-8_2
- 38. Smeds MR, Thrush CR, Mizell JS, Berry KS, Bentley FR. Mobile spaced education for surgery rotation improves National Board of Medical Examiners scores. J Surg Res 2016;201(1):99–104. pmid:26850190
- 39. Hirunyanitiwattana T, Pirompanich P, Boonya-anuchit P, Maitree T, Poachanukoon O. Effectiveness of Asthma Knowledge in Nurses using Asthma Care Application Compared with Written Asthma Action Plan. JOURNAL OF THE MEDICAL ASSOCIATION OF THAILAND 2020;103(4):16.
- 40. Baccin CRA, Dal Sasso GTM, Paixão CA, de Sousa PAF. Mobile Application as a Learning Aid for Nurses and Nursing Students to Identify and Care for Stroke Patients: Pretest and Posttest Results. Comput Inform Nurs 2020;38(7):358–66. pmid:32427611
- 41. Ameri A, Khajouei R, Ameri A, Jahani Y. LabSafety, the Pharmaceutical Laboratory Android Application, for Improving the Knowledge of Pharmacy Students. Biochemistry and Molecular Biology Education 2020;48(1):44–53. pmid:31657885
- 42.
JMIR Medical Education—Impact of an Electronic App on Resident Responses to Simulated In-Flight Medical Emergencies: Randomized Controlled Trial [Internet]. [cited 2021 Jan 22]. Available from: https://mededu.jmir.org/2019/1/e10955/
- 43. Mamtora S, Sandinha MT, Ajith A, Song A, Steel DHW. Smart phone ophthalmoscopy: a potential replacement for the direct ophthalmoscope. Eye (Lond) 2018;32(11):1766–71. pmid:30042410
- 44. Haubruck P, Nickel F, Ober J, Walker T, Bergdolt C, Friedrich M, et al. Evaluation of App-Based Serious Gaming as a Training Method in Teaching Chest Tube Insertion to Medical Students: Randomized Controlled Trial. J Med Internet Res 2018;20(5):e195. pmid:29784634
- 45. Oliveira MLB de, Verner FS, Kamburoğlu K, Silva JNN, Junqueira RB. Effectiveness of Using a Mobile App to Improve Dental Students’ Ability to Identify Endodontic Complications from Periapical Radiographs. Journal of Dental Education 2019;83(9):1092–9. pmid:31010890
- 46. Strandell-Laine C, Saarikoski M, Löyttyniemi E, Meretoja R, Salminen L, Leino-Kilpi H. Effectiveness of mobile cooperation intervention on students’ clinical learning outcomes: A randomized controlled trial. J Adv Nurs 2018;74(6):1319–31. pmid:29444335
- 47. Bartlett RD, Radenkovic D, Mitrasinovic S, Cole A, Pavkovic I, Denn PCP, et al. A pilot study to assess the utility of a freely downloadable mobile application simulator for undergraduate clinical skills training: a single-blinded, randomised controlled trial. BMC Medical Education 2017;17(1):247. pmid:29228934
- 48. Low D, Clark N, Soar J, Padkin A, Stoneham A, Perkins GD, et al. A randomised control trial to determine if use of the iResus© application on a smart phone improves the performance of an advanced life support provider in a simulated medical emergency. Anaesthesia 2011;66(4):255–62. pmid:21401537
- 49. McMullan M. Evaluation of a medication calculation mobile app using a cognitive load instructional design. Int J Med Inform 2018;118:72–7. pmid:30153925
- 50. Meyer AND, Thompson PJ, Khanna A, Desai S, Mathews BK, Yousef E, et al. Evaluating a mobile application for improving clinical laboratory test ordering and diagnosis. J Am Med Inform Assoc 2018;25(7):841–7. pmid:29688391
- 51. Quezada J, Achurra P, Jarry C, Asbun D, Tejos R, Inzunza M, et al. Minimally invasive tele-mentoring opportunity-the mito project. Surg Endosc 2020;34(6):2585–92. pmid:31363891
- 52. Naveed H, Hudson R, Khatib M, Bello F. Basic skin surgery interactive simulation: system description and randomised educational trial. Advances in Simulation 2018;3(1):14. pmid:30038804
- 53. Kim H, Suh EE. The Effects of an Interactive Nursing Skills Mobile Application on Nursing Students’ Knowledge, Self-efficacy, and Skills Performance: A Randomized Controlled Trial. Asian Nurs Res (Korean Soc Nurs Sci) 2018;12(1):17–25.
- 54. Bayram SB, Caliskan N. Effect of a game-based virtual reality phone application on tracheostomy care education for nursing students: A randomized controlled trial. Nurse Educ Today 2019;79:25–31. pmid:31102793
- 55. Fernández-Lao C, Cantarero-Villanueva I, Galiano-Castillo N, Caro-Morán E, Díaz-Rodríguez L, Arroyo-Morales M. The effectiveness of a mobile application for the development of palpation and ultrasound imaging skills to supplement the traditional learning of physiotherapy students. BMC Med Educ 2016;16(1):274. pmid:27756288
- 56. Lozano-Lozano M, Galiano-Castillo N, Fernández-Lao C, Postigo-Martin P, Álvarez-Salvago F, Arroyo-Morales M, et al. The Ecofisio Mobile App for Assessment and Diagnosis Using Ultrasound Imaging for Undergraduate Health Science Students: Multicenter Randomized Controlled Trial. J Med Internet Res 2020;22(3):e16258. pmid:32154784
- 57. Bunogerane GJ, Taylor K, Lin Y, Costas-Chavarri A. Using Touch Surgery to Improve Surgical Education in Low- and Middle-Income Settings: A Randomized Control Trial. J Surg Educ 2018;75(1):231–7. pmid:28712686
- 58.
Development and Evaluation of “Chronic Illness Care Smartphone Apps” on Nursing Students’ Knowledge, Self-efficacy, and Learning Experience.—Abstract—Europe PMC [Internet]. [cited 2021 Jan 22]. Available from: https://europepmc.org/article/med/29901475
- 59. Yoo I-Y, Lee Y-M. The effects of mobile applications in cardiopulmonary assessment education. Nurse Educ Today 2015;35(2):e19–23. pmid:25537170
- 60. Kim S-J, Shin H, Lee J, Kang S, Bartlett R. A smartphone application to educate undergraduate nursing students about providing care for infant airway obstruction. Nurse Educ Today 2017;48:145–52. pmid:27810633
- 61. Kang SR, Shin H, Lee J, Kim S-J. Effects of smartphone application education combined with hands-on practice in breast self-examination on junior nursing students in South Korea. Jpn J Nurs Sci 2020;17(3):e12318. pmid:31957268
- 62. Shore BJ, Miller PE, Noonan KJ, Bae DS. Predictability of Clinical Knowledge Through Mobile App-based Simulation for the Treatment of Pediatric Septic Arthritis: A Pilot Study. J Pediatr Orthop 2018;38(9):e541–5. pmid:30036289
- 63. Ebner F, De Gregorio A, Schochter F, Bekes I, Janni W, Lato K. Effect of an Augmented Reality Ultrasound Trainer App on the Motor Skills Needed for a Kidney Ultrasound: Prospective Trial. JMIR Serious Games 2019;7(2):e12713. pmid:31042155
- 64. Vaona A, Banzi R, Kwag KH, Rigon G, Cereda D, Pecoraro V, et al. E-learning for health professionals. Cochrane Database Syst Rev 2018;1:CD011736. pmid:29355907
- 65. Subhash TS, Bapurao TS. Perception of medical students for utility of mobile technology use in medical education. International Journal of Medicine and Public Health 2015;5(4):7.
- 66. Lall P, Rees R, Law GCY, Dunleavy G, Cotič Ž, Car J. Influences on the Implementation of Mobile Learning for Medical and Nursing Education: Qualitative Systematic Review by the Digital Health Education Collaboration. J Med Internet Res 2019;21(2):e12895. pmid:30816847
- 67. Dickinson KJ, Bass BL. A Systematic Review of Educational Mobile-Applications (Apps) for Surgery Residents: Simulation and Beyond. J Surg Educ 2020;77(5):1244–56. pmid:32307244
- 68. Shah R. Commentary. Ann Intern Med 2018;169(2):JC9. pmid:30014098
- 69. Pedro LFMG, Barbosa CMM de O, Santos CM das N. A critical review of mobile learning integration in formal educational contexts. International Journal of Educational Technology in Higher Education 2018;15(1):10.
- 70. Rosman P. M-LEARNING—AS A PARADIGM OF NEW FORMS IN EDUCATION. 2008;7.