The coronavirus disease 2019 (COVID-19) pandemic led to far-reaching restrictions of social and professional life, affecting societies all over the world. To contain the virus, medical schools had to restructure their curriculum by switching to online learning. However, only few medical schools had implemented such novel learning concepts. We aimed to evaluate students’ attitudes to online learning to provide a broad scientific basis to guide future development of medical education.
Overall, 3286 medical students from 12 different countries participated in this cross-sectional, web-based study investigating various aspects of online learning in medical education. On a 7-point Likert scale, participants rated the online learning situation during the pandemic at their medical schools, technical and social aspects, and the current and future role of online learning in medical education.
The majority of medical schools managed the rapid switch to online learning (78%) and most students were satisfied with the quantity (67%) and quality (62%) of the courses. Online learning provided greater flexibility (84%) and led to unchanged or even higher attendance of courses (70%). Possible downsides included motivational problems (42%), insufficient possibilities for interaction with fellow students (67%) and thus the risk of social isolation (64%). The vast majority felt comfortable using the software solutions (80%). Most were convinced that medical education lags behind current capabilities regarding online learning (78%) and estimated the proportion of online learning before the pandemic at only 14%. In order to improve the current curriculum, they wish for a more balanced ratio with at least 40% of online teaching compared to on-site teaching.
This study demonstrates the positive attitude of medical students towards online learning. Furthermore, it reveals a considerable discrepancy between what students demand and what the curriculum offers. Thus, the COVID-19 pandemic might be the long-awaited catalyst for a new “online era” in medical education.
Citation: Stoehr F, Müller L, Brady A, Trilla A, Mähringer-Kunz A, Hahn F, et al. (2021) How COVID-19 kick-started online learning in medical education—The DigiMed study. PLoS ONE 16(9): e0257394. https://doi.org/10.1371/journal.pone.0257394
Editor: Mohammed Saqr, KTH Royal Institute of Technology, SWEDEN
Received: February 8, 2021; Accepted: September 1, 2021; Published: September 21, 2021
Copyright: © 2021 Stoehr et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: FS and LM are supported by the Clinician Scientist Fellowship “Else Kröner Research College: 2018_Kolleg.05”. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Since early 2020, the whole world has been in a state of suspension of normal activity due to the coronavirus disease 2019 (COVID-19) pandemic. Almost all countries have been heavily impacted by the outbreak . With the aim of stopping the spread of the virus, far-reaching restrictions of social and professional life have been adopted . Medical schools had to restructure their curriculum by switching from on-site teaching to online teaching [3–6]. Because of the rapid progress of the pandemic, these changes had to be implemented under considerable time pressure.
Today’s students are highly interested in innovative teaching methods including online learning, networked learning, simulation-based learning and others [7–9]. However, as medical teaching is mainly based on traditional, ex-cathedra concepts, only a minority of medical schools had implemented such innovative concepts prior to COVID-19 [10–12]. One frequent argument against online learning in the past was that its implementation is a time-intensive and expensive process, due to the lack of the infrastructure needed [8, 13].
However, in the wake of the pandemic, such teaching concepts have been successfully introduced to a much greater extent in recent months [3–6]. However, it can be assumed that quickly switching to fully online teaching led to “emergency remote teaching” instead of a structured, dedicated online teaching curriculum. Although, the current situation might present a unique opportunity for the modernization of medical education in order to fulfill students’ teaching needs, opinions still differ on how medical education should be delivered in the future.
Thus, we aimed to describe the current status of teaching in medical education by evaluating medical students’ attitudes towards online learning in general, the “real life” implementation of online learning in medical schools during the pandemic and possible teaching scenarios after the pandemic. We decided to conduct an international, multicenter study in order to provide a broad, scientific basis to guide future development of medical education.
2.1 Study set-up
This survey was conducted as a cross-sectional study including several medical schools in 12 different countries (Bulgaria, Canada, Greece, Germany, Hungary, Ireland, Spain, Switzerland, the Netherlands, Turkey, Ukraine, USA). As this survey was addressed to medical students, this study is based on a non-probability, voluntary sample. We followed the recommendations of the EQUATOR network (“https://www.equator-network.org”) and conducted this web-based, cross-sectional study in accordance with the “Checklist for Reporting Results of Internet E-Surveys (CHERRIES)”  and with the “Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies”  (S1 Checklist). Institutional review board approval was waived by the Ethics Committee of the Medical Association of Rhineland-Palatinate. The study design was conducted in accordance to the Declaration of Helsinki. The survey started on June 12, 2020 and was closed on August 7, 2020.
2.2 Questionnaire design
A dedicated questionnaire was designed together with the Center for Quality Assurance and Development (ZQ) of the Johannes Gutenberg University Mainz. The questionnaire consisted of various sections covering, in particular, the following aspects: Current online learning situation at medical schools, types of online learning currently offered and desired in future, technical and social aspects of online learning, attitudes towards the current and future role of online learning in medical education, suitability of teaching concepts for online learning, as well as current and desired ratio between online and on-site learning. In total, the questionnaire comprised 35 questions. Students were asked to answer the questions using a 7-point Likert scale (1 = “strongly disagree,” 2 = “disagree,” 3 = “somewhat disagree,” 4 = “neutral,” 5 = “somewhat agree,” 6 = “agree,” to 7 = “strongly agree”). We deliberately chose a 7-point scaling system for the following reasons: compared to a 5-point scale, a 7-point scale leads to a higher variance of answers and thus to higher reliability and more nuanced trend analysis ; scaling up higher to 9-point or even 11-point systems, however, would not yield added value regarding the information obtained and would have strained the abstraction capabilities of our respondents . The entire questionnaire is attached in the supplement (S1 Questionnaire).
2.3 Validation of the questionnaire
The questionnaire underwent a two-step external validation approach in order to further enhance the quality of the study. First, cognitive pretesting was performed on a small sample size of 10 students. First, cognitive pretesting was performed on a small sample size of 10 students ; second, pilot testing was performed on a larger cohort of 50 medical students from two university medical centers . The final questionnaire was transferred to an established online survey tool (SurveyMonkey®, www.surveymonkey.com), which was also used for data collection.
2.4 Distribution of the questionnaire
Invitations to take part in the study were sent via email from the Deans’ offices and/or via social media to all students of the participating medical schools. The invitations were further distributed by the German (BVMD) and the European Medical Students’ Association (EMSA). All invitations contained an identical short introductory text and the hyperlink to access the survey. The students were informed that the survey results would be anonymous and that they were collected for research purposes only.
2.5 Statistical analysis
Final survey results were extracted as CSV file and subsequently analyzed using R 4.0.2 (A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, https://www.R-project.org; accessed July 2020). Figures were plotted using the ggplot2 and likert packages [20, 21]. Mean and standard deviation were calculated to analyze results . Comparison between responses was performed using Welch’s t-test [23, 24]. A p-value <0.05 was considered statistically significant. For further subgroup analysis regarding sex and study year, Cohen’s d was calculated; effect size was considered large if >0.5.
3.1 Participants’ demographics
A total of 3286 participants completed the questionnaire. The demographic data recorded were sex (33% male, 66.2% female, 0.5% non-binary), age (mean 23.6 years), and current study year (mean 3rd study year) (Table 1). The detailed distribution of the participants according to study year and gender can be found in Fig 1.
A total of 29 participants are not included in this figure due to: non-binary gender (n = 16), missing gender (n = 9), and missing study year (n = 4).
3.2 Survey results
In the following, survey results for each particular category are presented in written form as well as graphically (Figs 2–4 and S1 Fig). To increase comprehensibility, the text contains only the key findings and summarizes “strongly disagree”, “disagree” and “somewhat disagree” as disagreement and “somewhat agree”, “agree” and “strongly agree” as agreement. For statistical analyses, the original categories were used. All detailed data including the particular results per item, particular response rates per item and means per item are provided in the supplementary material (S1–S4 Tables). The entire results are presented visually in Fig 4.
Ranking ranges from highly suitable (lecture) to unsuitable (practical training).
The lower violin plot depicts the ratio between online learning and on-site learning wished for the future (mean 42%).
Centered stacked bar plot showing the detailed responses of the participants regarding the “current online learning situation” (A), the “technical aspects of online learning” (B), the “social aspects of online learning” (C), and the “attitudes towards the current and the future role of online learning in medical education” (D). Orange tones represent “disagreement”, grey represents “neutral”, and blue tones represent “agreement”.
3.2.1 Current online learning situation at medical school.
The vast majority of medical schools successfully switched to online learning (78%), although only very few (13%) offered many online courses before the pandemic. Most students were satisfied with the quantity and quality of the online courses offered by their medical schools during the pandemic (67% and 62%, respectively). To capture the participants’ comfort with online activity, they were asked about the extent of their social media use, which are frequently used by a vast majority (82%) (Fig 4 and S1 Table).
3.2.2 Types of online learning currently offered and desired in future.
Pre-recorded lectures or seminars without the possibility of interaction (88%) and interactive live lectures or seminars (76%) were by far the most common types of online learning offered by medical schools. Online platforms and resources for self-learning/self-assessment modules were also common (70%). Chats (with/without video stream) (40%) were less frequently offered, and audio podcasts (13%) were uncommon. The ranking of the participants’ preferences regarding their desired types of online learning in future followed exactly the same order (S1 Fig).
3.2.3 Technical aspects of online learning.
The great majority of students already owned the electronic devices required for online learning (97%) and was further willing to invest in such devices (74%). Most participants preferred a computer (79%), fewer preferred a tablet (20%), and only very few used a smartphone (1%). Most participants felt comfortable using the software solutions for online learning (80%). All in all, three-quarters of the participants felt well prepared for online learning (76%) (Fig 4 and S2 Table).
3.2.4 Social aspects of online learning.
Online learning provided increased flexibility (84%). However, a considerable proportion had difficulties in motivating themselves to follow online learning modules (42%). For around two-thirds, online learning is perceived as creating the risk of social isolation (64%) and does not offer sufficient possibilities for interaction with fellow students (67%). Very few are concerned about their privacy when using online learning (14%) (Fig 4 and S3 Table).
3.2.5 Attitudes towards the current and future role of online learning in medical education.
The vast majority believes that medical education in general lags behind current capabilities regarding online learning (78%). Thus, few participants were satisfied with the current role of online learning in medical education (27%). Only 22% considered the previous experience of teachers in online learning to be sufficient but, in contrast, a vast majority expects them to be familiar with online teaching (83%). For around three-quarters of the participants, switching to online learning led to a higher or unchanged attendance of courses (70%) (Fig 4 and S4 Table).
3.2.6 Suitability of teaching concepts for online learning.
The participants´ rating regarding the suitability of online learning for different teaching concepts is as follows: 1. lecture: (91%); 2. seminar: (57%); 3. bedside teaching: (12%); 4. practical training: (9%) (Fig 2).
3.2.7 Current and desired ratio between online and on-site learning.
In the last two questions, students were asked to estimate the ratio between online learning and on-site teaching before the pandemic and to state the ratio desired for the future. Before the pandemic, students estimated the ratio between online learning and on-site teaching at 14% online vs. 86% on-site teaching. In future, participants wish for a more balanced ratio of 42% online teaching and 58% on-site teaching (p<0.05); however, the distribution shows considerable variability between 25% and 75% online teaching. The detailed distribution is provided in a violin plot (Fig 3).
3.2.8 Subgroup analyses regarding gender and study year as well as country of studies.
Further subgroup analyses according to gender and study year were performed. According to effect sizes, neither gender nor study year had a significant impact on the abovementioned results. The detailed statistical analyses can be found in the supplementary material (S5 and S6 Tables).
With more than 3,000 participants from countries with a wide geographical distribution, this study about online learning in medical education underlines the immense relevance of this topic for medical students and for medical education in general. In summary, our study suggests the positive attitude of an overwhelming majority of medical students towards online learning. Furthermore, students are convinced that medical schools have heretofore neglected digitization of learning and that medical education is lagging behind its capabilities. Consistent with that, medical students estimate the pre-pandemic proportion of online learning in medical education at only 14%. In future, students wish for a more balanced ratio with at least 40% of online learning compared to on-site teaching.
Before the pandemic, teaching in most medical schools has been mainly based on “traditional” concepts requiring physical presence of the students [11, 13, 25, 26]. The outbreak of the COVID-19 pandemic led to a sudden switch from on-site teaching to almost entirely online teaching [3–6]. However, this switch was mostly born out of necessity and might have caused problems in the teaching “logistics” of medical schools. Technical infrastructure had to be installed and teachers had to adapt their courses immediately. However, our survey results point out that students were nevertheless satisfied with both quality and quantity of the online courses provided. This means that most medical schools were able to master the change from on-site to online teaching at short notice. This refutes the widespread opinion that switching to online learning is both a time-intensive and expensive process due to the lack of the infrastructure needed [8, 13, 27]. To summarize, our results indicate that there might be more infrastructure available for online teaching in medical schools than initially expected.
Despite this successful switch to online learning, however, downsides of online learning should be considered. One issue is the possible danger of social isolation because of reduced “real” discussions and less interaction with fellow students or teachers [7, 28]. This also includes a lack of distinction between home and “workplace” due to increased study times at home . Our results suggest that the participants took this issue seriously. However, social isolation may be aggravated in view of the concurrent situation of a mandatory nationwide quarantine in many countries. Thus, it might be assumed that after the pandemic, social life will take place again.
Furthermore, our results indicate that a considerable proportion of participants had difficulties in motivating themselves to follow online learning modules. However, this seems to be a multifactorial issue and recent studies on students’ motivation suggest that the major “motivational” factor may be the teaching type (e.g. problem-based vs. lecture-based) and not the teaching format (e.g. online vs. on-site teaching) [30, 31]. Because of the rapid switch to online learning, most medical schools did not have a specific curriculum framework of online learning. Thus, possible solutions for the future could be to not only provide single online courses but to integrate these into a dedicated framework including a schedule for each subject and frequent performance reviews, to avoid leaving students behind.
However, even before the pandemic, declining attendance rates of students have become part of everyday medical school life [32, 33]. Various possible reasons have been discussed [32, 33]; and, as our results indicate, change of students’ learning mentalities and environment seem to be major factors. For example, access to broad online learning opportunities without the necessity for any physical presence is very easy nowadays. Our results show that medical students often use online content and appreciate various types of online learning. Furthermore, most students are willing to invest in the electronic devices required for online learning, which implies their high intrinsic motivation. With this in mind, online teaching could be an answer to the needs of an increasingly diverse student community . As online learning provides increased flexibility, it could be crucial in terms of accommodating student diversity and inclusivity.
Despite this obviously strong demand for online learning among medical students, there are parts of medical education, which seem unsuitable for online learning. For example, medical students as doctors-to-be require live patient contact via face-to-face interaction to develop clinical skills. The participants of our study recognize this as an important point, and judge digitization in medical education in a highly differentiated way. The students deem lectures and seminars but not bedside teaching and practical skills training appropriate for online teaching. Nonetheless, regarding elements such as practical skills training, many options exist for using innovative teaching concepts as a “dry run” before facing real patients [35, 36]. This is in line with several studies underpinning the positive effect of laparoscopic, arthroscopic and endovascular simulator training which increases not only hands-on skills but also can stimulate interest in the particular subspecialty [37–39]. Furthermore, virtual reality technologies including e.g. real-time holoportation and smartglasses are advancing rapidly, and allow for a virtual “real life” experience . There are already promising results showing the successful use of those innovative technologies in clinical education [40, 41].
Regarding the results of this study, the overall challenge is to reasonably implement innovative online learning concepts in the curriculum. Although the shift towards online learning started as “emergency remote teaching” instead of a structured transition, several innovative concepts had been implemented during the last few months. Nevertheless, the participants in our study are convinced that the current curriculum can be enriched by the use of online teaching methods. Thus, as a benchmark for the future, students deem a ratio of around 42% online and 58% on-site teaching ideal.
In order to improve current medical education sustainably, a primary goal should be the implementation of a curriculum framework of online learning. Having the desired ratio of online and on-site teaching in mind, With this in mind, “blended learning”, which combines on-site and online learning as a “hybrid”, could be a good model [42, 43]. In order to actively enhance the learning process, students would have to prepare the teaching content independently using online resources before a dedicated, problem-centered on-site course is delivered [42, 44, 45].
Our study is a questionnaire-based survey set up on a non-probability, voluntary sample and therefore entails typical pitfalls of those. We had to consider selection bias, meaning that interested students are more likely to complete the questionnaire . This might particularly apply to those students who were included via social media, which we did not record due to the anonymous study design. However, we had a very high number of participants from many different medical schools in many countries, suggesting a well-mixed pool of participants. Due to our study design, we could only calculate completion rate, since no information was available on the total number of potentially contacted participants. Furthermore, there is the potential for social desirability, meaning that participants choose the answer that they assume is favourable . To attenuate this kind of response bias, we created a completely anonymous and untraceable study design and instructed the students that the survey results were for research purposes only. Furthermore, we only investigated participants from industrialized countries across Europe and North America, and results could be entirely different if the survey would have been distributed in e.g. emerging or developing countries. Further studies could include those countries in order to gain deeper insights regarding socio-economic dimensions of teaching in medical education.
Ultimately, we decided on a cross-sectional study design, which allows only for a snapshot. Thus, as a limitation, our results may not necessarily translate into long-term opinions informing students’ perception of online learning. To assess these effects, further longitudinal studies are necessary. Such follow-up studies could e.g. focus on the purpose of teaching and which concepts or combinations are most suitable to impart knowledge to medical students. This could provide a more in-depth analysis of different teaching types. Furthermore, we primarily focused on students’ attitudes towards online teaching. Thus, further research focusing on effectiveness and markers of mental health may be needed to better understand the impact of distance learning on students. Further studies could also investigate teachers’ attitudes towards online teaching as well as potential differences in how both groups see “their” future of medical education.
In summary, there is an obvious discrepancy between what medical students desire and what current curricula offer them. Therefore, it is our duty to restructure medical education not only for the present but even more so for the post-pandemic future in a sustainable manner. As medical students clearly see potential for improvement regarding the online teaching skills of their lecturers, medical faculties should put further effort in imparting these skills to lecturers.
Positively speaking, the COVID-19 pandemic might be the long-awaited and much-needed catalyst for a new online teaching era in medical education. Furthermore, the current situation gives students as well as teachers a unique opportunity to create and further advance innovative learning and teaching concepts together. The medical community seems to understand the need for change and has already started the process.
S1 Checklist. Guidelines for reporting observational studies.
S1 Questionnaire. Questionnaire investigating various aspects of online learning in medical education.
S1 Fig. Ranking of the participants’ preferences regarding their desired types of online learning currently offered and in future.
Boxplots showing the proportion of types of online learning currently offered (A) and the ranking of types of online learning desired in future by the students from top (most desired) to bottom (least desired) (B).
S1 Table. Current online learning situation at medical school (n = 3286).
SD = standard deviation; N/A = not available.
S2 Table. Technical aspects of online learning school (n = 3286).
SD = standard deviation; N/A = not available.
S3 Table. Social aspects of online learning (n = 3286).
SD = standard deviation; N/A = not available.
S4 Table. Attitudes towards the current and the future role of online learning in medical education (n = 3286).
SD = standard deviation; N/A = not available.
S5 Table. Subgroup analysis regarding study year.
We thank all participating students all over the world. Furthermore, we want to thank Prof. P. Mildenberger, Prof. J. Bruix, the German (BVMD) and the European Medical Students’ Association (EMSA) for their extraordinary effort supporting this study. We thank Dr. C. Ruckes from the Interdisciplinary Center for Clinical Studies (IZKS) for his help with statistical analysis.
Ethics approval and consent to participate: Institutional review board approval was waived by the Ethics Committee of the Medical Association of Rhineland-Palatinate. Written informed consent was not necessary/applicable. Invitations to take part in the study were sent via email from the Deans’ offices and/or via social media to all students of the participating medical schools. All invitations contained an identical short introductory text and the hyperlink to access the survey. The participants were informed that participation is voluntary, that the survey results would be anonymous and that they were collected for research purposes only.
- 1. World Health Organization. WHO Coronavirus Disease (COVID-19) Dashboard. 2020. Available: https://covid19.who.int
- 2. Bedford J, Enria D, Giesecke J, Heymann DL, Ihekweazu C, Kobinger G, et al. COVID-19: towards controlling of a pandemic. Lancet. 2020;395: 1015–1018. pmid:32197103
- 3. Finn GM, Brown MEL, Laughey W, Dueñas A. #pandemicpedagogy: Using Twitter for knowledge exchange. Med Educ. 2020; medu.14242. pmid:32418270
- 4. Huddart D, Hirniak J, Sethi R, Hayer G, Dibblin C, Rao BM, et al. #Med StudentCovid–How social media is supporting students during COVID‐19. Med Educ. 2020; medu.14215. pmid:32392354
- 5. Moszkowicz D, Duboc H, Dubertret C, Roux D, Bretagnol F. Daily medical education for confined students during COVID‐19 pandemic: A simple videoconference solution. Clin Anat. 2020; ca.23601. pmid:32253771
- 6. Woolliscroft JO. Innovation in Response to the COVID-19 Pandemic Crisis. Acad Med. 2020. pmid:32282372
- 7. Ali NS, Hodson-Carlton K, Ryan M. Students’ Perceptions of Online Learning: Implications for Teaching. Nurse Educ. 2004;29: 111–115. pmid:15167578
- 8. Attardi SM, Rogers KA. Design and implementation of an online systemic human anatomy course with laboratory. Anat Sci Educ. 2015;8: 53–62. pmid:24920278
- 9. Mynbayeva A, Sadvakassova Z, Akshalova B. Pedagogy of the Twenty-First Century: Innovative Teaching Methods. New Pedagogical Challenges in the 21st Century—Contributions of Research in Education. InTech; 2018.
- 10. Irby DM, Cooke M, OʼBrien BC. Calls for Reform of Medical Education by the Carnegie Foundation for the Advancement of Teaching: 1910 and 2010. Acad Med. 2010;85: 220–227. pmid:20107346
- 11. McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ. Revisiting ‘A critical review of simulation-based medical education research: 2003–2009.’ Med Educ. 2016;50: 986–991. pmid:27628715
- 12. Skochelak SE, Stack SJ. Creating the Medical Schools of the Future. Acad Med. 2017;92: 16–19. pmid:27008357
- 13. O’Doherty D, Dromey M, Lougheed J, Hannigan A, Last J, McGrath D. Barriers and solutions to online learning in medical education–an integrative review. BMC Med Educ. 2018;18: 130. pmid:29880045
- 14. Eysenbach G. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res. 2004;6: e34. pmid:15471760
- 15. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet (London, England). 2007;370: 1453–7.
- 16. Saris W, Gallhofer I. Design, Evaluation, and Analysis of Questionnaires for Survey Research, 2nd edition. Wiley, Hoboken. 2007.
- 17. Krosnick J, Presser S. Question and Questionnaire Design. Handbook of Survey Research, 2nd Edition Elsevier, San Diego. 2009.
- 18. Lenzner T., Neuert C., & Otto W. Cognitive Pretesting. GESIS Surv Guidel. 2016; GESIS Surv.
- 19. Hassan ZA, Schattner P, Mazza D. Doing A Pilot Study: Why Is It Essential? Malaysian Fam physician Off J Acad Fam Physicians Malaysia. 2006;1: 70–73. Available: https://pubmed.ncbi.nlm.nih.gov/27570591
- 20. Bryer J, Speerschneider K. Package ‘likert.’ 2016.
- 21. Wickham H. ggplot2. New York, NY: Springer New York; 2009. https://doi.org/10.1007/978-0-387-98141-3
- 22. Knapp TR. Treating ordinal scales as interval scales: an attempt to resolve the controversy. Nurs Res. 1990;39: 121–3. Available: http://www.ncbi.nlm.nih.gov/pubmed/2315066 pmid:2315066
- 23. Rasch D, Kubinger KD, Moder K. The two-sample t test: pre-testing its assumptions does not pay off. Stat Pap. 2011;52: 219–231.
- 24. Ruxton GD. The unequal variance t-test is an underused alternative to Student’s t-test and the Mann–Whitney U test. Behav Ecol. 2006;17: 688–690.
- 25. Kassebaum DG. Change in medical education. Acad Med. 1989;64: 446–7. pmid:2751782
- 26. Mourad A, Jurjus A, Hajj Hussein I. The What or the How: a Review of Teaching Tools and Methods in Medical Education. Med Sci Educ. 2016;26: 723–728.
- 27. Maloney S, Haas R, Keating JL, Molloy E, Jolly B, Sims J, et al. Breakeven, Cost Benefit, Cost Effectiveness, and Willingness to Pay for Web-Based Versus Face-to-Face Education Delivery for Health Professionals. J Med Internet Res. 2012;14: e47. pmid:22469659
- 28. Ali A, T Smith D. Comparing Social Isolation Effects on Students Attrition in Online Versus Face-to-Face Courses in Computer Literacy. Issues Informing Sci Inf Technol. 2015;12: 011–020.
- 29. Rose S. Medical Student Education in the Time of COVID-19. JAMA. 2020;323: 2131. pmid:32232420
- 30. Hwang SY, Kim MJ. A comparison of problem-based learning and lecture-based learning in an adult health nursing course. Nurse Educ Today. 2006;26: 315–321. pmid:16364510
- 31. Oliveira PC, Oliveira CG. Using conceptual questions to promote motivation and learning in physics lectures. Eur J Eng Educ. 2013;38: 417–424.
- 32. Desalegn AA, Berhan A, Berhan Y. Absenteeism among medical and health science undergraduate students at Hawassa University, Ethiopia. BMC Med Educ. 2014;14: 81. pmid:24731511
- 33. Nevins EJ, Moori PL, Alexander L, Richards B, Bleasdale V, Sharma AK. Could Attendance at Medical School be improved? A Prospective Study of Medical Education at The University of Liverpool: Study of Attendance at a UK Medical School. MedEdPublish. 2016;5.
- 34. DeLisa JA, Lindenthal JJ. Commentary: reflections on diversity and inclusion in medical education. Acad Med. 2012;87: 1461–3. pmid:23111256
- 35. Al-Elq A. Simulation-based medical teaching and learning. J Fam Community Med. 2010;17: 35. pmid:22022669
- 36. So HY, Chen PP, Wong GKC, Chan TTN. Simulation in medical education. J R Coll Physicians Edinb. 2019;49: 52–57. pmid:30838994
- 37. Laubert T, Thomaschewski M, Auerswald P, Zimmermann M, Brüheim L, Keck T, et al. [Implementation of a Laparoscopic Simulation Training in Undergraduate Medical Education—The Lübeck Toolbox-Curriculum]. Zentralbl Chir. 2018;143: 412–418. pmid:28505682
- 38. Rahm S, Wieser K, Wicki I, Holenstein L, Fucentese SF, Gerber C. Performance of medical students on a virtual reality simulator for knee arthroscopy: an analysis of learning curves and predictors of performance. BMC Surg. 2016;16: 14. pmid:27015842
- 39. Stoehr F, Schotten S, Pitton MB, Dueber C, Schmidt F, Hansen NL, et al. Endovascular simulation training: a tool to increase enthusiasm for interventional radiology among medical students. Eur Radiol. 2020. pmid:32221683
- 40. Pottle J. Virtual reality and the transformation of medical education. Futur Healthc J. 2019;6: 181–185. pmid:31660522
- 41. Huber T, Wunderling T, Paschold M, Lang H, Kneist W, Hansen C. Highly immersive virtual reality laparoscopy simulation: development and future aspects. Int J Comput Assist Radiol Surg. 2018;13: 281–290. pmid:29151194
- 42. Liu Q, Peng W, Zhang F, Hu R, Li Y, Yan W. The Effectiveness of Blended Learning in Health Professions: Systematic Review and Meta-Analysis. J Med Internet Res. 2016;18: e2. pmid:26729058
- 43. Rowe M, Frantz J, Bozalek V. The role of blended learning in the clinical education of healthcare students: A systematic review. Med Teach. 2012;34: e216–e221. pmid:22455712
- 44. Ramnanan C, Pound L. Advances in medical education and practice: student perceptions of the flipped classroom. Adv Med Educ Pract. 2017; Volume 8: 63–73. pmid:28144171
- 45. Tolks D, Romeike BF, Ehlers J, Kuhn S, Kleinsorgen C, Huber J, et al. The online inverted classroom model (oICM). A blueprint to adapt the inverted classroom to an online learning setting in medical and health education. MedEdPublish. 2020;9.
- 46. Nour S, Plourde G. Pharmacoepidemiology in the Prevention of Adverse Drug Reactions. Pharmacoepidemiology and Pharmacovigilance. Elsevier; 2019. pp. 25–65. https://doi.org/10.1016/B978-0-12-816183-8.00003–2
- 47. Furnham A. Response bias, social desirability and dissimulation. Pers Individ Dif. 1986;7: 385–400.