https://orcid.org/0000-0002-8944-4948
Correction
17 Jun 2021: Amir Rad F, Otaki F, Baqain Z, Zary N, Al-Halabi M (2021) Correction: Rapid transition to distance learning due to COVID-19: Perceptions of postgraduate dental learners and instructors. PLOS ONE 16(6): e0253683. https://doi.org/10.1371/journal.pone.0253683 View correction
Figures
Abstract
The outbreak of Coronavirus disease 2019(COVID-19) necessitated an abrupt transition from on campus, face-to-face sessions to online, distance learning in higher education institutions. The purpose of this study was to investigate the perceptions of postgraduate dental learners and instructors about the transition to distance learning, including the changes to the learning and teaching and its efficaciousness. A convergent mixed methods approach to research was utilized. All the instructors and postgraduate learners in a dental college were invited to participate in an online survey. Quantitative data was analyzed using descriptive and inferential analyses on SPSS for Windows version 25.0, and for the responses to the open-ended questions, multi-staged thematic analysis was utilized. Both groups of stakeholders: learners and instructors, were satisfied with the rapid transition to distance learning due to COVID-19. Instructors were significantly more satisfied than the learners. The stakeholders adapted well to the change. The perception of the stakeholders regarding the case-based scenarios significantly influenced their level of satisfaction. As perceived by the stakeholders, the transition to distance learning entailed advantages and challenges. Going through the experience enabled the stakeholders to develop informed opinions of how best to sustain learning and teaching irrespective of how matters unfold in relation to the pandemic. In conclusion, the worldwide dental education community faced unprecedented challenges due to the onset of COVID-19. From a macro perspective, decision-makers must not miss out on the valuable opportunities, inherent in the experience, to reinforce curriculums, and maximize learning and teaching.
Citation: Rad FA, Otaki F, Baqain Z, Zary N, Al-Halabi M (2021) Rapid transition to distance learning due to COVID-19: Perceptions of postgraduate dental learners and instructors. PLoS ONE 16(2): e0246584. https://doi.org/10.1371/journal.pone.0246584
Editor: Timothy C. Guetterman, University of Michigan, UNITED STATES
Received: September 11, 2020; Accepted: January 21, 2021; Published: February 8, 2021
Copyright: © 2021 Rad et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: The authors received no specific funding for this work.
Competing interests: No authors have competing interests.
Introduction
It took the coronavirus disease 2019 (COVID-19) two months to traverse national borders, across multiple continents. On March 11th 2020, the World Health Organization (WHO) labeled the COVID-19 as a pandemic [1]. The outbreak of COVID-19 led to a rapidly evolving situation that impacted the education system worldwide [2]. Continuing education delivery through alternative learning and teaching channels abruptly became a top priority for institutions aiming to keep the impact of the crisis on education to a minimum. Following the endorsement of national social distancing directives, education institutions, across many countries, had no option but to resort to distance learning environments and other e-learning resources [3]. Implementation of distance learning, in the United Arab Emirates (UAE), for all higher education institutions’ became a requirement starting from March 22nd 2020 until the end of June 2020, where almost three quarters of the second half of 2019–2020 academic year was conducted online [4].
Rapid transition from on campus face-to-face classes to online distance learning sessions took place. Many educators had never delivered sessions in online environment, which required them to acquire an extensive set of skills over a short period of time. They also needed to adapt their offerings’ content and structure, and select the most suitable methods to engage their learners in the virtual environment. The impact of these restrictions was exacerbated in health professions education, due to the suspension of all elective treatments where the associated experiential education constitutes the core of the learning and teaching [5]. There are pedagogical approaches for planning distance learning courses, which require special techniques of course design, instructional design, and communication methods [6, 7]. Yet, given the urgency of the COVID-19 situation, institutions did not get the space to plan for and adapt the proper, systematic way of transitioning, which usually involves a substantial amount of capacity building (i.e., offering learning and development opportunities for involved stakeholders) and of change management (i.e., striving to minimize the inevitable resistance and its counterproductive consequences to any institutional changes). The learners who were accustomed to face-to-face interactions had to rapidly adapt to distance learning and the online environment. Moreover, the challenges around rapidly transitioning to distance learning were exacerbated by the multiple changes and restrictions accompanying COVID-19, and the resulting psychosocial stressors that the learners and the educators have been facing [8, 9]. Educators and learners needed to increase networking, foster humanity in their connections, and enhance their communication effectiveness before, during, and after their online engagements [10]. This experience influenced the way they create meaning and reflect upon learning and teaching [11].
The novelty of the situation and how the involved parties have been adapting to this situation constitute excellent opportunities for investigation given that most research studies, to date, have been conducted in “typical” (relatively more stable) educational environments. It is important to examine and reflect on this experience to better prepare for the potentiality of reoccurrence or the experiencing of other emergencies requiring immediate transition to online teaching and learning. Moreover, the lessons learned from this sudden transition of adults’ education hold the potential of positively transforming post-pandemic learning and teaching, especially in programs, that rely heavily on clinical training, since they were impacted the most since the onset of this pandemic. This is particularly relevant to postgraduate dental education since learning and teaching heavily rely on clinical specialty training.
Therefore, the purpose of this study was to investigate the perceptions of postgraduate dental learners and instructors in relation to the sudden transition to distance learning, including the changes to the learning and teaching and its efficaciousness, and how these stakeholders modified their learning or teaching to adapt to this abrupt change.
Accordingly, in this study, we strive to address the following research question: how was the rapid transition to distance learning due to COVID-19 perceived by postgraduate dental learners and instructors?
Materials and methods
Context of the study
This study was undertaken at Hamdan Bin Mohammed College of Dental Medicine (HBMCDM) at Mohammed Bin Rashid University of Medicine and Health Sciences (MBRU), Dubai, UAE. HBMCDM is a new postgraduate dental school, launched in Academic Year 2013–2014, that offers three-year full-time specialty dental postgraduate programs in endodontics, orthodontics, pediatric dentistry, periodontics, and prosthodontics.
Responding to COVID-19
To curtail the spread of the COVID-19 pandemic and in response to the directive of the Ministry of Education (Decree 229) [12], HBMCDM along with all other educational institutions in the UAE switched to complete distance learning as of March 22nd 2020 until the end of the academic year (July 9th, 2020 in the case of HBMCDM- the College under investigation in this study). HBMCDM continued all didactic educational activities as scheduled. At the time of transition, the first-year learners were preparing their research protocols, and the third-year learners were mainly engaged in the preparation of the graduate dissertation. Both of those groups of learners were able to pursue their research work. Some of the second-year learners, had to stop the empirical (i.e., data collection) part of their studies given the implemented COVID-19 directives. Thesis Advisory Committees held discussions and advising sessions related to the different stages of the scientific research method of the dissertations (ranging from research conceptualization to determining of research methodology, and data collection tools and analyses, all the way to preparing of the manuscript) with the learners via Microsoft Teams.
For the clinical training, the Curriculum Committee at the dental college decided that the learners will have to compensate for the missed clinical activities to ensure the attainment of the required clinical competences at a later stage. To make-up for the generated gap, the Committee introduced a compensation program in the following academic year. In the meantime, the College focused on virtual Case-Based Discussion (CBD) sessions and involved learners in teleconsultations under the supervision of faculty members. Also, some learners were deployed to off-training sites to assist authorities with COVID-19 contact tracing and sampling.
Transitioning to distance learning
Platforms.
Two digital platforms were utilized for the delivery of distance learning. HBMCDM had an existing Learning Management System (LMS) that has been in function since 2014. Course instructors use the LMS to post course information and content, and to conduct assessments and post grades. It is also used for interactive discussions within specific courses. The College maintained the use of the LMS throughout the distance learning. The heavy reliance on Microsoft Teams was novel. It constituted the main platform which provided a medium for real-time class presentations by both learners and instructors, and for research dissertation-related interactions and clinical CBD. In addition, some instructors pre-recorded their lectures with the support of the educational technologists at MBRU to ensure the learners can access the content at their convenience.
Instructors’ professional learning and development.
All instructors were required to undergo learning and development sessions concerning the transition to distance learning, delivered by MBRU Faculty Development and Information Technology (IT) support teams. These sessions were conducted via Microsoft Teams. The IT support team assigned technical support personnel to each instructor. In addition, the Faculty Development team provided one-to-one consulting to support the instructors in learning design, where instructions were adapted to the respective instructors’ learning and teaching specifications. The instructors were advised to shorten the teaching sessions’ length, not to exceed one hour each. Instructors were also advised to provide reading material before the teaching sessions to enable the suggested shortening of session.
The teaching sessions.
The length of time allocated to different classes remained unchanged. As for the lectures’ scheduled timing, the College maintained the original schedule for the classes assigned at the beginning of the second semester of Academic Year 2019–2020. Instructors registered class attendance on MBRU Self-Service portal.
As mentioned earlier, the College decided to add an additional two-hour CBD to the original two-hour CBD to have in total four-hours across two sessions per week. These sessions were meant to engage the graduate learners in specific clinical skills including diagnosis, decision-making, and treatment planning, by encouraging critical thinking and providing constructive multi-stream dialogue between the learners and instructors.
Changes in assessments.
The Student Admission and Progression Committee in the College implemented significant changes in assessment methods to accommodate for the absence of live proctoring. Instructors were encouraged to consider feasible alternative tracks that emphasize equity and hold the learners accountable to academic integrity. The instructors needed to collectively identify alternative means of conducting summative assessment to ensure the attainment of the learners’ intended course learning outcomes and readiness to progress.
The Curriculum Committee decided to uphold the weightage of the summative assessments for all courses, as determined at the beginning of the semester. Emphasis was placed on maximizing formative Multiple-Choice Question (MCQ) type quizzes to ensure that the individual lectures’ learning outcomes were met. Instructors were encouraged to conduct assessments using clinical scenarios to test the learners’ diagnosis, clinical judgment, problem-solving skills, and treatment planning competences, especially in complex multidisciplinary cases. The oral clinical exams, which are based on unseen case-scenarios, were conducted using Microsoft Teams.
The LMS system, through which the exams were conducted, deployed a lockdown browser requirement which prevented the learners from opening any other application on their devices while taking the exam. Also, activation of the webcam, in the learners’ devices, was required. The system was enabled to detect any abnormal activity by the exam-taker, and flag it to be checked by the course instructor. All learners and instructors received training on how to take the exam using the lockdown browser and the webcam. In addition, detailed written instructions were sent to the learners before each summative exam.
Research design
The study’s ethical approval was granted by the MBRU, Institutional Review Board (Reference # MBRU-IRB-2020-032). A multi-phased mixed methods convergent study design [13–15] was adopted, as demonstrated in Fig 1, to systematically understand the stakeholders’ perceptions regarding the rapid transition to distance learning. This study was characterized by four sequential phases: data collection, data analysis, information integration, and knowledge generation. In the first phase, the qualitative and quantitative data were concurrently collected (from both groups of stakeholders: instructors and learners). The second phase of this study it involved analyzing the quantitative datasets independently from the qualitative datasets. Following that, we integrated the generated information, comparing and relating findings from the independent analyses. The information integration, which is expected to raise the validity of the study’s findings, was in terms of sources (i.e., learners and instructors) and types (i.e., qualitative and quantitative), and relied on joint model analysis [16, 17]. The interpretation of the integrated information led to the generation of knowledge, which constituted the last phase of the research design.
Data collection
The data was collected using a survey that was designed specifically for this study. It aimed at assessing the perception of learners and instructors regarding the rapid transition to distance learning due to COVID-19, and its effect on the learning and teaching at the College.
The survey was developed by three researchers (FAR, FO, and MAH), who started by looking into how the various universities worldwide are planning on evaluating their distance learning performance since the onset of COVID-19. Among the universities which had their surveys readily available were the following: the University of Minnesota, University of Pittsburgh, University of Saskatchewan, and Rutgers University [18]. These resources were retrieved from an online community of Institutional Research professionals where an asynchronous forum discussion around “Impact of COVID-19 on evaluations in Higher Education Institutions” has been taking place. These surveys were thoroughly reflected upon, and in turn, extracted segments from all got contextualized and in turn adapted for this study. The survey was composed of three segments. The first segment is composed of 8 components, as per Table 1, measured against a Likert-type scale of five points (1: Strongly Disagree, 2: Disagree, 3: Neutral, 4: Agree, and 5: Strongly Agree). Out of those eight components, three were replicated for both learners and instructors. Three others were replaced for the instructors with an alternative which reflects the other side of the same coin, and two were unique to the learners.
The second segment entailed the following two dichotomous questions (Yes/ No), each followed by a separate open-ended question requiring the participant to elaborate:
- The transition to the online environment, in response to the COVID-19, significantly impacted my learning (or my teaching) in these courses.
- The transition to the online environment, in response to the COVID-19, significantly impacted the courses’ structure and delivery.
As for the last segment of the survey, it was meant to be exploratory to solicit for qualitative data using the following open-ended questions:
- What were some of the advantages of transitioning to distance learning?
- What were some of the challenges that you faced due to transitioning to distance learning?
- Please reflect upon aspects of the alternative modes of instruction deployed that were particularly supportive of your learning (for the learners’ distance learning) during the COVID-19 pandemic.
- What aspects of those alternative modes of instruction would you like to sustain on the long run (even after returning to regular face-to-face sessions)?
All full-time faculty of the College were invited to participate. Part time faculty who were involved in distance learning lectures were also invited to participate. All students enrolled in the College, at the time of the survey in the three different levels and the five different postgraduate programs, were invited to participate. Participation in this data collection initiative was completely voluntary. The privacy and the data confidentiality of the learners were protected, and no personal identifiers were recorded. The survey was open for participation from May 1st-31st 2020. In the respective academic year, the HBMCDM faculty was composed of 21 instructors and served 63 learners. The faculty group had five females and 16 males with an average age of 48 years. The faculty were of 12 different nationalities with 4 professors, 5 associate professors, 5 assistant professors, and 7 lecturers. Out of the 21 faculty members, 19 were full-time and 2 were part-time. The learners consisted of 44 females and 19 males. They came from 16 different nationalities with an average age of 30.9 years.
Due to the unexpected transition to online learning in this semester, the Strategy and Institutional Excellence department (SIE) of the University decided to substitute the individual courses’ assessment surveys usually conducted in a specific semester with this current survey to allow the faculty and learners to provide their feedback regarding this transition, while avoiding survey fatigue. An email, with the survey link, was sent by SIE team to all participants on May 1st and subsequently weekly email reminders went out until the survey closed on May 31st. Additionally, the Dean of the College sent separate emails to both the faculty and students’ groups to encourage participation.
Data analyses
Quantitative analysis.
Quantitative descriptive analysis. The quantitative data was descriptively analyzed using SPSS for Windows version 25.0. For each of the eight quantitative components, the mean and standard deviation were calculated. An overall score of satisfaction was calculated for both stakeholders together (i.e., across the six components that are common to both stakeholders), along with a score of satisfaction for the learners (i.e., across all eight components) and another one for the instructors (i.e., across the six components that constituted the instructors’ tool).
Since the scale used for capturing the perception of the learners and instructors was tailor-made for this study, the validity tests of Cronbach’s Alpha and the Principal Component Analysis (PCA) were performed to ensure internal consistency and check external variance, respectively, of the adapted tool.
For the inferential analyses, to select the appropriate tests, a test of normality was conducted for each of the eight components, and for all three scores of satisfaction (overall, and learners and instructors). The data of each of the eight components, independently, and the overall and learners’ scores of satisfaction all turned out to be not normally distributed. As for the instructors’ data, it turned out to be normally distributed.
Quantitative inferential analysis. Accordingly, Mann-Whitney tests were used to compare the overall score of satisfaction, and each component independently, between both groups of stakeholders (learners and instructors), and the overall score of satisfaction and learners’ score of satisfaction, between those who answered ‘Yes’ (versus those who answered ‘No’) to each of the two dichotomous questions of the second segment of the survey. As for the instructors’ satisfaction score, Independent T-test was used to uncover whether, or not, this satisfaction score significantly differs between the two options of each of the dichotomous questions.
In addition, Chi-squared was used to assess any potential associations between the two dichotomous variables of the second segment of the survey and the two groups of stakeholders.
Finally, the Kruskal- Wallis test was conducted to assess the extent to which the overall and learners’ scores of satisfaction can be explained by changes in the stakeholders’ perception of the components of the scores, respectively. In order to investigate the same associations for the instructors’ score of satisfaction, ANOVA was deployed.
Qualitative analysis.
The qualitative data analysis started after the conclusion of the data collection phase. The data was analyzed using thematic analysis by three researchers (MAH, FAR, and FO). The subjectivity of the researchers was recognized, right from the start of the analysis, to avoid affecting the integrity of the qualitative analysis trajectory. Prominent patterns were identified after thorough examination of datasets. The process was inductive, based on the constructivist epistemology. It is worth noting that the consistency, in relation to the underlying theoretical assumptions, was assured throughout the study. This iterative, interpretative approach enabled the researchers to gain a detailed understanding of the phenomenon under investigation (i.e., rapid transition to distance learning at HBMCDM at MBRU).
The process of analysis followed the six-step framework initially introduced by Braun and Clarke (2006) [19]. This multi-staged approach to thematic analysis has been encouraged in research concerning health professions education [20]. NVivo software version 12 plus (QSR International Pty Ltd, Vic, Australia) was used to code the data, and in turn expedite the categorization of the relevant text fragments.
The analysis process started with the researchers acquainting themselves with the data. The data was segmented into meaningful statements. The data collected from each of the two groups of stakeholders was handled separately. Then, as a second step, the text fragments that refer to the same aspect of the distance learning experience were compiled together, labeling each with an all-encapsulating title. Accordingly, the qualitative data was examined line-by-line, while assigning codes to text fragments, until data saturation was attained.
The researchers reflected upon areas of harmony and discord mentioned by the participants. The resulting categorization schemes of the two groups of stakeholders were mapped onto each other to compare perceptions. The same themes (more or less) surfaced in the separate analyses.
Following that, the discrete concepts, from both datasets, underwent several rounds of reflections, where the various ways by which the concepts could relate to one another were identified. This led to the generation of categories that comprehensively cover all that surfaced in relation to the two research questions, which set the stage for the researchers to work on step three. The researchers examined the categories, again, to find the best way to merge them into higher order themes.
The generated themes and categories were then reviewed as part of stage four to ensure that the data within each grouping are sufficiently similar, and data in between the clusters are distinct enough to deserve segregation. All the themes and categories were then labeled and defined to complete stage five. This constituted the basis of the study’s conceptual framework which guided the last step of reporting upon the findings.
Mixed methods integration. To optimize understanding of the mixed insights, the findings from both types of analyses (quantitative and qualitative) were merged through mapping them onto each other and carefully reflecting upon them. This mixed methods integration was done using an iterative joint display (i.e., meta matrix) analysis process to promote methodological rigor which ultimately led to meta inferences [15, 21]. The researchers explored how the output of the analyses relate to one another; they compared (and contrasted) the findings from the two analyses to weave together a meaningful narrative. They looked for areas where there was direct alignment between the quantitative and qualitative findings (i.e., where the findings of the two analyses confirmed each other). They also searched for areas that were unique to either of the analyses (quantitative or qualitative), with the intention of further expanding the generated insights, and investigated whether these unique areas complement the areas that were identified as common in both analyses. The researchers were open, throughout the integration process, to potential discordance/ divergence between the findings of the two analyses.
Results
Out of those 63 learners, 53 responded (i.e., response rate = 84%). As for the instructors, a total of 18 faculty members responded (i.e., response rate = 86%). Each of the 71 participants was given a unique identification number. The unique identification numbers were complimented with ‘R’ for the 53 learners, and ‘I’ for the 18 instructors (i.e., participants 1 through 53 are followed by ‘R’, and 54 through 71 by ‘I’).
Quantitative analyses
Descriptive.
The reliability score of Cronbach’s Alpha for the evaluation instrument, that captured the perception of the stakeholders was 93.3%. The percentage of the total average of the learners, instructors, and both groups of stakeholders were 82.55%, 91.13%, and 84.63%, respectively, as per Table 2.
According to the PCA, 90.7% of the variance can be explained by the instrument which means the instrument is not only reliable but also valid to measure what it is intended to measure.
Inferential.
As illustrated in Fig 2, the instructors, with a mean of satisfaction of 31.84(±2.85), rated the distance learning experience higher than the learners, with a mean of satisfaction of 28.75(±5.05) (P = 0.023).
The overall and learners’ scores of satisfaction were associated with all six components (P<0.05). As for the instructors’ score of satisfaction, it was only significantly associated with the perception of instructors regarding the following component: “the case-based scenarios, used throughout the distance learning, was helpful, in my opinion, to develop the learners’ clinical knowledge” (P = 0.001).
The learners who perceived the transition not to impact the courses’ structure and delivery were significantly more satisfied (P = 0.008). However, whether or not the stakeholders perceived the transition to impact the learning and teaching was not significantly associated with their level of satisfaction.
Qualitative data
The thematic analysis resulted in four interrelated themes: ‘Factors that worked to the benefit of the stakeholders’ (i.e., advantages) and ‘Factors that the stakeholders were not in favor of’ (i.e., challenges), ‘Modifications in learning or teaching’, and ‘Lessons learned and suggestions for the future’, as illustrated in this study’s conceptual framework (Fig 3). Within the Factors that worked to the benefit of the stakeholders theme, five categories surfaced: Efficiency, Convenience, Work-life Balance, Autonomy, and Cooperation. The Factors that the stakeholders were not in favor of theme encapsulated four other categories labeled as: Scope of interactions and learners’ engagement, Clinical teaching, IT limitations, and Diffusion of boundaries. The third theme: Modifications in learning or teaching, encapsulated four categories: Self-directed learning, Collaborative learning, Flipped Teaching, and Shortening of lectures. As for the Lessons learned and suggestions for the future theme, it included text fragments that refer to aspects that will be sustained, those that will be (further) leveraged, and those that require improvements.
Theme 1: Factors that worked to the benefit of the stakeholders (i.e., advantages).
This theme refers to the strengths of the rapid transition to distance learning, as perceived by the two groups of stakeholders.
Efficiency.
Both groups of stakeholders agreed that the changes accompanying the transition saved time and energy. The processes around learning and teaching became more efficient, and existing resources became more accessible. All of this was enabled via the adapted IT platform, where the multifunctional tools allowed for deploying diverse teaching modalities and conducting differing activities:
R9: “…we were getting plenty of learning materials and resources, and given enough time to ask and express our opinions…”
F9: “…it saved time and allowed for more focus on explaining the context of the respective topics. I find the changes…to be very efficient…”
Work-life balance.
The stakeholders also agreed that these changes led to better work-life balance, particularly among working mothers. It added value to have the flexibility to work and study from the comfort of one’s home. Moreover, it became easier for the stakeholders to fulfill their responsibilities, around the multiple roles that they play in their lives.
R15: “…it has been advantageous to mothers … I have kids- I needed to support in their distance learning… They also needed a lot of support and encouragement to adapt to the new situation…”
F4: “…the comfort of staying at one’s home- no commuting, remaining close to family, and the ease of connecting with anyone at almost any time…”
Convenience.
The changes led to arrangements and configurations that were favorably perceived by the faculty members. The IT platform facilitated the stakeholders’ workflow, where communication and scheduling recurrently surfaced as advantages. The enabled asynchrony was leveraged by the stakeholders, where they maximized reflection, and made sure all queries are attended to. The increased accessibility was much appreciated by the stakeholders, where they could attend to their duties anywhere at almost any time.
R2: “…especially for people living outside Dubai, in other Emirates… by not commuting, I was able to save a lot of time and energy, which I usually spend commuting to University, and while stuck in traffic. I also made use of the time in between lectures…”
F2: “…the platform that constituted the core of the distance learning was even handier and user friendly than the Learning Management System. The comments and discussions typed in the chat boxes got saved which in of itself had been a great advantage…”
F6: “…the ease of communication whenever needed… the ease of scheduling for and attending and participating in meetings on Microsoft Teams… there had been more flexibility and opportunities for networking, and better accessibility…”
Autonomy.
The learners also felt that the new normal gave them more autonomy where they had more control over their schedules and managed their times better. Somehow, the onset of the pandemic, along with the transition to distance learning, created an opportunity for the learners to rise to the occasion. It became clear that the transition created more space for the learners, who were proactive about leveraging it to their advantage.
R2: “…I got into the habit of reviewing the lectures immediately after the respective sessions, so basically the transition gave me more autonomy in managing my own schedule and making the best use of my time…”
Cooperation.
The transition to distance learning, and the virtual environment, facilitated work across physical barriers (interpersonal, and across disciplines and even nations), which was especially noticed by faculty members. This led to enhanced teamwork and better collaborations.
F2: “…the distance learning created a lot of networking and collaborating opportunities- interdisciplinary and international teamwork and teaching became much easier…”
F3: “…… this openness to change and to alternatives, and the associated flexibility, are enabling plenty of collaboration and co-creating opportunities across barriers…”
Theme 2: Factors that the stakeholders were not in favor of (i.e., challenges).
This theme refers to the weaknesses and difficulties of the rapid transition to distance learning, as perceived by the two groups of stakeholders, and the struggles they faced along this virtual journey.
Limited scope of interactions and levels of learners’ engagement.
The transition expectedly led to a lessening of the scope of interactions, where people became physically, and in some instances socially, distant. In terms of knowledge exchange, this distancing required the deployment of a new set of skills, which not everyone had at the time of transition.
R6: “…the absence of the lecturer’s physical presence sometimes makes receiving and digesting the information more difficult… it is not that easy to concentrate and remain focused without seeing and interacting with the instructor in person…”
The physical distancing, integral to the changes, affected people in differing ways. The people’s personalities played a role. Some learners, for example, became quieter than usual and less engaged in their own learning process.
F5: “…getting all the learner to contribute especially the shy or quiet ones had been a challenge…you do not see their facial expressions and nonverbals…”
Clinical teaching.
The hands-on, experiential education, which lies at the core of health professions’ education, had to be put on hold due to the COVID-19 pandemic. A lot of the practical content that is usually delivered by clinical exposure got replaced by alternatives that seem to have added value but did not offer an equivalent to what the learners had missed out on.
F3: “…there is no alternative to hands-on and practical activities. IT might offer solutions that effectively replace part of the experience, but that is it… health professions’ education requires clinical experience and experiential learning…”
F10: “…clinical courses, though, cannot be delivered through distance learning- this constitutes the bulk of any postgraduate dental training… technology gave exciting solutions to the pandemic situation, one will not become a doctor ‘remotely’…”
Information Technology glitches and limitations.
The stakeholders referred to technical glitches that occurred sporadically. They also referred to certain teaching techniques that they could not deploy online, via the Microsoft Teams platform.
F2: “…IT glitches, although rare, caused interruptions in the flow of teaching…”
F4: “…Inability to draw and offer pictorial demonstrations and sketches of certain visual concepts in dentistry…”
Diffusion of boundaries.
Although, as mentioned before, distance learning and working from afar enabled the crossing of physical barriers, in many instances, it led to diffusion of boundaries. During the pandemic, life felt more like a continuum, with no clear boundaries, for the learners and faculty members. Remaining in a single physical space created a lot of overlaps in the minds of the stakeholders. Not needing to commute to University made the differentiation between work and personal duties quite challenging.
R4: “… finding a quiet place at home was not easy… distraction when family members, especially the kids, pass-by and require my attention assuming I am not engaged or busy with something else…”
R9: “…distractions at home, especially that all other members of the family were also working and/ or studying from home…”
Theme 3: Modifications in learning or teaching.
This theme refers to how the two groups of stakeholders modified their approaches to learning or teaching to adapt to the rapid transition to distance learning. The novel learning and teaching environment became conducive to favorable behavioral and attitudinal changes. Both groups of stakeholders seem to have organically grown and developed, personally and professionally, from this experience. The perceptions of the two groups map onto each other, like two sides of the same coin.
Self-directed learning.
The learners exhibited more proactiveness throughout distance learning. Learners took the initiative and went out of their way to foster their learning throughout the virtual experience. They attributed the tendency of self-directed learning to having more space and time at hand.
R8: “…the space… enabled us to resort to and use more resources than usual to study… access to journals and articles helped me a lot…”
R14: “…I attended free dental webinars, from all over the world- that was amazing…”
Collaborative learning (vis-à-vis group-based learning).
Teaching became more group-based and learners were more collaborative. The instructors assigned more group-based exercises. The learners were more open to sharing resources and engaging with each other; efforts to come together and co-create were evident on both sides.
R17: “…transition to team-based learning had been conducive to our learning… Working in groups had been better because we divided the workload and assigned responsibilities. Each one worked independently, across differing roles, towards a common goal. We became more productive, as a team…”
F7: “…learners, from differing cohorts and programs, used to have difficulties in setting mutually-convenient meeting times to work together (due to their different timetables). This experience opened doors for new collaboration opportunities…”
Flipped teaching.
The faculty members resorted to flipped teaching, more often, where the direct instruction moves from the group to the individual learning space. This group space is transformed into an interactive learning environment where the instructor guides the learners as they apply concepts and engage creatively in the subject matter.
F1: “…we adapted flipped learning models, which I think is better than the traditional ‘spoon feeding’ learning, especially for postgraduate teaching which our learners were used to…”
Shortening of lectures.
Along the same line, most of the sessions got shortened.
R1: “…the longer the session, the less we are able to remain focused. We became way more attentive during the lectures that were reduced in duration…”
F7: “…almost all lectures were shortened, since two-hour distance learning lectures did not work for us…”
Theme 4: Lessons learned and suggestions for the future.
This theme encapsulated what the stakeholders acquired from and their recommendations due to this first-hand experience with distance learning. The institutional knowledge generated from this experience better prepared the stakeholders for distance learning. Due to this experience and what the two groups of stakeholders gained from it, the institutional capacity for distance learning is expected to improve.
Aspects that will be sustained.
Some of the aspects worked well and will be maintained as is.
R13: “…the additional case-presentations exposed us to new cases that we do not typically see in clinical training. Getting more frequently exposed to case-based discussions enabled us to examine certain cases more and elaborate upon them…”
R15: “…teleconsultations via Microsoft Teams (that we started using, with the patients, only after COVID-19) offered amazing learning experiences, and is going to stay with us in the Orthodontics department, irrespective of how the modes of delivery shape-up…”
Moreover, the capacity to create virtual learning experiences that are equivalent to that offered in the classroom setting will enable educational continuity for learners who for some valid reason cannot make it to the University.
R7: “…distance learning is beneficial for learners who cannot attend lectures, in person, on campus. Instead of missing classes, they can attend them online…”
Aspects that will be (further) leveraged.
Other aspects worked well and were discovered to be worth capitalizing upon in a systematic manner. These aspects exhibited potential that is worth exploiting to maximize the learning and teaching experiences.
R9: “…we recommend for us to have more joint sessions with postgraduate learners in other universities… to attend more webinars offered by other universities… and to conduct more interdisciplinary discussion sessions within the university…”
F2: “…engaging more with learners and faculty members in other institutions, across the world…”
Aspects that require improvements.
There were also aspects that were identified to require improvements. In other words, the experience unearthed matters that did not work well and/ or entailed noticeable opportunities for improvement. The value of the stakeholder suggestions is that they stem from actual firsthand experience.
R6: “…to use more of pre-existing educational videos such as procedural videos or clinical scenarios during the lectures… it makes the information more digestible…”
R12: “…having access to recordings of the live lectures, with the PowerPoint together (Picture-in-Picture), would be helpful… we can go back and listen to the instructor’s comments at any time…”
Mixed methods integration
Mapping the output of the quantitative analysis onto that of the thematic analysis unearthed a holistic perspective of the situation, illustrated in the study’s side-by-side joint display [16, 17] (Fig 4). The convergence of findings enabled the development of a thorough understanding of what had been happening since the transitioning to distance learning. On its own, the quantitative analysis showed the attitudinal reactions of the stakeholders, where they positively rated the overall experience, and the individual aspects of the experience that came together to create the whole. These findings were confirmed, and better understood and expanded upon by looking into the analysis of the narrative data. In addition, the qualitative data analysis, on its own, uncovered a few weaknesses of what had been taking place along with some concerns raised by the stakeholders around what might have gone wrong with the adapted IT platforms. The same analysis also independently showed the ways by which the stakeholders modified their behaviors. In some cases, these modifications appeared as instant reactions to the transition. Other behavioural changes were more proactive in nature. The stakeholders gradually changed to leverage the surfacing strengths and opportunities for improvement, and circumemvent the weaknesses and the arising concerns.
The first iteration shows how secondary color Green emerged by mixing the primary color Blue with the primary color Yellow (which constitutes an analogy of the lateral and critical thinking that took place, and the meta inferences that were generated). In the second iteration, each quadrilateral represents an area of findings. The Blue area was identified from the quantitative analysis. The Yellow areas were identified from the qualitative analysis. As for the Green areas, they were derived/ interpreted by mapping the findings of the quantitative onto the qualitative ones.
Discussion
This study showed that both groups of stakeholders: learners and instructors, were satisfied with the rapid transition to distance learning due to COVID-19. The transition was characterized by several advantages as perceived by the stakeholders. The continued learning and teaching via the online platforms saved time and energy, especially around commuting back and forth between one’s home and campus, and increased the efficiency of the associated processes, all of which were considered more convenient for the stakeholders relative to the face-to-face configuration. This led to enhancing the work-life balance for the learners and instructors. The learners favored having more control over their schedules, and both parties were happy with the increased cooperation across the board. This is in concordance with previous literature that highlighted enabling self-paced learning, and allowing for more time and space flexibility as prominent advantages of distance learning [22, 23]. Whereas another study, demonstrated an undesirable result of increased autonomy among learners who are not self-regulated and/ or not equipped with time management skills [24]. Accordingly, supporting learners and instructors in developing those competencies would help them to maximize their learning and teaching experiences, while striking a better work-life balance.
Although both parties scored highly in terms of satisfaction with distance learning, the group of instructors turned out to be significantly more satisfied than the group of learners. It was previously suggested in the literature that among the factors that influence the level of satisfaction of online teaching for instructors are self-gratification, intellectual challenge, interest in using technology, and the associated professional development opportunities [25]. The instructors’ overall level of satisfaction was associated with their satisfaction with the additional CBD sessions. The same component was also an antecedent to the overall score of satisfaction of learners. The qualitative data analysis uncovered the same findings, where both groups of stakeholders would like to maintain additional online CBD sessions in the future. The overall learners’ score of satisfaction was associated with their perception of the case-based learning and all the rest of the components of the adapted tool which proved to be valid and transferrable to other similar contexts.
Moreover, learners who perceived the transition to be seamless, without impacting the courses’ structure and delivery, were significantly more satisfied. Hence, it is essential to work towards instilling a culture of change when planning for systematic transitioning to distance learning. It is also vital to ensure providing sufficient technical support along the way. This will decrease the level of resistance previously proven to enhance the overall distance learning experience [26–28].
The stakeholders also pinpointed challenges that they faced in their distance learning experiences. The perceived added value of CBD did not substitute for hands-on experience. The distance learning was satisfying for non-clinical teachings only. As for the clinical training, it lagged. Resources, which offer equivalents to real-life experiences, clinical trainings, and/ or interactions with patients, were non-existent.
Another prominent challenge was the noticeable decrease of in-class contributions; the learners highlighted how their concentration span seemed to become shorter after the transition, and the instructors reflected upon the difficulties that they faced in keeping the learners focused and engaged. Along the same lines, it is established in the literature that maintaining the level of interaction and keeping learners engaged are among the most prominent challenges instructors face in online teaching [28, 29]. This challenge was further exacerbated whenever the stakeholders faced technical glitches. Fortunately, those incidences were rare, as reported by several stakeholders, but it is important to keep this potentiality in mind because at the end of the day the IT platform constitutes the medium through which the whole experience occurs. Any hurdles on that front would result in interruptions to the online learning and teaching experience.
The learners and instructors noticed that (and in turn reported upon how) they organically modified their learning or teaching styles, respectively, to adapt to the circumstances and to maximize the experiences. The external consequences led to behavioral changes, for both parties. The literature establishes that choices of pedagogical approaches by instructors are expected to affect learners’ learning approaches [30]. Some instructors reported shortening of their lectures. Others mentioned resorting to flipped learning approaches which entailed making use of Questions and Answers, discussions, quizzes (i.e., formative assessments to ensure timely attainment of the intended learning outcomes of the sessions), and providing feedback throughout the online sessions. The learners, in turn, deployed more self-directed learning, which is desired in fostering life-long learning. Collaborative learning increased. The instructors assigned more group activities, and the learners became more likely to support each other, share resources, and work together towards common goals. All of which are in alignment with the principles of connectivism [31]. This can be further emphasized by consideration of virtual-based situated learning, where the content in the online medium needs to be engaging, the environment needs to be contextualized, cultivation of participation and fostering active learning become the priorities, and forming and nurturing community-of-practice is recommended [32].
A lot of insights and lessons learned were acquired from the rapid transition to distance learning, which can be effectively assimilated and reinforced with pre-existing evidence to better integrate and sustain distance learning. Blended learning has been implemented, with favorable outcomes in other postgraduate dental schools [33–35]. Considering the high acceptance of distance education among this study’s learners and instructors, effective integration of blended education, complemented with a flipped classroom in certain subjects, holds the potential for substantial benefits for both groups. The concept of blended education can incorporate asynchronous and synchronous learning elements [36, 37]. As a result of the pandemic, teleconsultation was employed in some of the programs and appreciated for offering the learners opportunities to acquire new consultation skills. In relation to the clinical exposure, the massive appreciation around the CBD, and the debriefing sessions after the presentations of clinical scenarios, can be leveraged to enhance critical thinking, decision making, and clinical reasoning skills [38, 39]. Besides, this andragogy, which proved to be of substantial added value to the online learning and teaching in the College under investigation, can be systematically integrated into the curriculum for it to evolve into full-fledged case-based learning.
Despite the challenges, the COVID-19 pandemic created plenty of novel opportunities in the worldwide dental community. To make up for the distance and to make good use of the space that this period has created, the dental educators became more engaged via virtual webinars and online conferences. Those opportunities were open and easily accessible to all members of the community-at-large, where knowledge and resource sharing among the stakeholders expanded in scale and scope. A network of dental education institutions was formed, which nurtured international collaborations. Bridging between educational institutions and clinical practitioners was also evident. This interconnected learning community is expected to support postgraduate learners in their future job search and academic collaborations. It also encourages the learners and instructors to develop novel, innovative pedagogical techniques by assimilating materials that are readily available online (e.g., case-scenario banks or educational procedural videos), based upon the constructivist theoretical underpinnings (via strategies such as: inculcation and integration). Along the same lines, among the vast array of resources that learners get access to, are videos of procedures that enable clinicians to practice the associated set of skills prior to implementing it for a patient, all of which are desired in any curriculum but seldom occurs [40, 41].
This study has a few limitations. In alignment with the principles of the Institutional Research function, complete anonymity of the participants was maintained. Therefore, the gender, age, and current level in the participants’ respective programs were not recorded. It would have been interesting to know if the satisfaction and the perceived impact of the transition are associated with those demographic variables. The qualitative data also offered a lot of insights that could have been further explored with alternative data collection tools (e.g., focus group sessions).
Moreover, although the focused study design enabled the development of thorough insights, the findings’ generalizability is limited to institutions that are contextually and characteristically like MBRU. This limitation is further pronounced given the exceptionality of the times of the COVID-19 pandemic (on all fronts). Both groups of stakeholders faced significant changes in their personal and social lives. Their day-to-day life was majorly disrupted, and they were all under a lot of pressure due to the halting of the clinical training which, in principle, entails 60% of the postgraduates’ time. This left the stakeholders with more time to study and to engage in other virtual educational activities which might not remain the case after resuming the clinical learning. Therefore, any decision on the changes in the curriculum towards blended learning needs to be tailored in accordance with the reality of post-pandemic circumstances. In addition, this study evaluated the official platforms of learning. It was evident, though, that unofficial learning tools such as Social Media Applications (SMA) were of added value to the learning and teaching. Correctly understanding how such learning tools can be maximized is essential when planning for distance learning.
It would be great to build upon the findings of this study in the following directions: To start with, it is worth exploring the long-term effect of this unprecedented abrupt change in the educational method, due to the COVID-19 pandemic, on the learners as they progress. Moreover, it would be useful to develop a contextualized competency-based model that would constitute the foundation for instructors’ professional development. It would be interesting for the College to adapt action research to develop, with the thorough engagement of the stakeholders, contingency learning, and teaching plans for crises. Additionally, one of the most significant changes identified with this rapid change centered on assessment. This is an area for which further exploration would help understand the broader impact of the switch to online instruction. Another component demanding further investigation is the loss of hands-on experience for clinical students. Increased attention and analysis related to outcomes is merited given the discipline’s core requirement of developed hand and chairside communications skills.
Finally, the distance learning started after the spring break, which means the first-year learners had about six months of face-to-face learning, which enabled them to build rapport with their instructors and colleagues. It would be interesting to investigate if first-year learners, who sign-up for blended learning from the beginning of the academic year, would perceive matters differently.
Conclusion
The abrupt transition to distance learning, due to COVID-19, was perceived favorably by the involved stakeholders at the respective College. This unexpected change entailed overcoming plenty of challenges, but also uncovered substantial opportunities that are worth capitalizing upon in health professionals learning and teaching. The lessons learned and the first-hand knowledge that stakeholders acquired from the reaction to the onset of the pandemic can be leveraged to innovatively develop and reinforce postgraduate dental curriculums. This reality-check has put stakeholders of higher education in a position to work together and share knowledge. This in turn will enable them to be better prepared for any such transitions and will equip the learners and educators with the skills to maximize the learning experiences and ensure educational continuity.
Supporting information
S1 Appendix. Distance learning survey for learners and instructors.
https://doi.org/10.1371/journal.pone.0246584.s001
(DOCX)
References
- 1. World Health Organization. Coronavirus disease (COVID-19) pandemic. 2020 [Available from: At: https://www.who.int/emergencies/diseases/novel-coronavirus-2019.
- 2. UBNESCO. COVID-19 Educational Disruption and Response 2020 [Available from: https://en.unesco.org/covid19/educationresponse.
- 3. Gewin V. Five tips for moving teaching online as COVID-19 takes hold. Nature. 2020;580(7802):295–6. pmid:32210377
- 4. The United Arab Emirates’ Government portal. Distance learning in times of COVID-19 2020 [Available from: https://u.ae/en/information-and-services/education/distance-learning-in-times-of-covid-19.
- 5. Huntley RE, Ludwig DC, Dillon JK. Early Effects of COVID-19 on Oral and Maxillofacial Surgery Residency Training-Results From a National Survey. J Oral Maxillofac Surg. 2020;78(8):1257–67. pmid:32536436
- 6. Ellaway R, Masters K. AMEE Guide 32: e-Learning in medical education Part 1: Learning, teaching and assessment. Med Teach. 2008;30(5):455–73. pmid:18576185
- 7. Masters K, Ellaway R. e-Learning in medical education Guide 32 Part 2: Technology, management and design. Med Teach. 2008;30(5):474–89. pmid:18576186
- 8. Marshall AL, Wolanskyj-Spinner A. COVID-19: Challenges and Opportunities for Educators and Generation Z Learners. Mayo Clin Proc. 2020;95(6):1135–7. pmid:32376100
- 9. Saddik B, Hussein A, Sharif-Askari FS, Kheder W, Temsah M-H, Koutaich R, et al. Increased levels of anxiety among medical and non-medical university students during the COVID-19 pandemic in the United Arab Emirates. medRxiv. 2020. pmid:33177898
- 10. Carlson ER. COVID-19 and Educational Engagement. J Oral Maxillofac Surg. 2020;78(7):1049–51. pmid:32371146
- 11. Li HO, Bailey AMJ. Medical Education Amid the COVID-19 Pandemic: New Perspectives for the Future. Acad Med. 2020. pmid:32657786
- 12.
Ministry of Education. Ministrial Decree no. 229 for the Year of 2020 Regarding Distance Education. United Arab Emirates2020.
- 13. Moseholm E, Fetters MD. Conceptual models to guide integration during analysis in convergent mixed methods studies. Methodological Innovations. 2017;10(2):2059799117703118.
- 14. Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs-principles and practices. Health Serv Res. 2013;48(6 Pt 2):2134–56. pmid:24279835
- 15. Guetterman TC, Fetters MD, Creswell JW. Integrating Quantitative and Qualitative Results in Health Science Mixed Methods Research Through Joint Displays. Ann Fam Med. 2015;13(6):554–61. pmid:26553895
- 16.
Fetters MD. Performing fundamental steps of mixed methods data analysis. The Mixed Methods Research Workbook: Activities for Designing, Implementing, and Publishing Projects: SAGE Publications, Inc; 2019.
- 17.
Fetters MD. Developing a Joint Display. The Mixed Methods Research Workbook: Activities for Designing, Implementing, and Publishing Projects: SAGE Publications, Inc.; 2019.
- 18. Explorance. BAC to Blue Community within the Explorance Help Center 2029 [Available from: https://support.explorance.com/hc/en-us/community/posts/360058413472-Impact-of-Coronavirus-on-evals.
- 19. Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. 2006;3(2):77–101.
- 20. Kiger ME, Varpio L. Thematic analysis of qualitative data: AMEE Guide No. 131. Med Teach. 2020:1–9.
- 21. Johnson RE, Grove AL, Clarke A. Pillar Integration Process: A Joint Display Technique to Integrate Data in Mixed Methods Research. Journal of Mixed Methods Research. 2019;13(3):301–20.
- 22. Fidalgo P, Thormann J, Kulyk O, Lencastre JA. Students’ perceptions on distance education: A multinational study. International Journal of Educational Technology in Higher Education. 2020;17(1):18.
- 23. De Paepe L, Zhu C, Depryck K. Online Dutch L2 learning in adult education: educators’ and providers’ viewpoints on needs, advantages and disadvantages. Open Learning: The Journal of Open, Distance and e-Learning. 2018;33(1):18–33.
- 24. Romero M, editor Distance learners’ Work Life Learning Balance2011.
- 25. Bolliger DU, Wasilik O. Factors influencing faculty satisfaction with online teaching and learning in higher education. Distance Education. 2009;30(1):103–16.
- 26. Maguire LL. Literature Review–Faculty Participation in Online Distance Education: Barriers and Motivators. Online Journal of Distance Learning Administration. 2005;8(1):1–16.
- 27. Flores SA. Distance learning: Preparation and practice for the adult learner. The Journal of Human Resource and Adult Learning. 2017;13(1):43–50.
- 28.
Distance learning: The shift to interactivity. CAUSE Professional Paper Series [Internet]. Boulder, CO: Educause. 1997. Available from: http://www.acousticslab.org/dots_sample/module4/SherronAndBoettcher1997_InteractivityShift.pdf.
- 29. O’Doherty D, Dromey M, Lougheed J, Hannigan A, Last J, McGrath D. Barriers and solutions to online learning in medical education—an integrative review. BMC Med Educ. 2018;18(1):130. pmid:29880045
- 30. Gibbs G, Coffey M. The Impact Of Training Of University Teachers on their Teaching Skills, their Approach to Teaching and the Approach to Learning of their Students. Active Learning in Higher Education. 2004;5(1):87–100.
- 31. Goldie JGS. Connectivism: A knowledge learning theory for the digital age? Med Teach. 2016;38(10):1064–9. pmid:27128290
- 32.
Annetta LA, Folta E, Klesath M. Situated Learning and Engagement in Distance Education. In: V-Learning. Dordrecht: Springer; 2010.
- 33. Bridges S, Chang JW, Chu CH, Gardner K. Blended learning in situated contexts: 3-year evaluation of an online peer review project. Eur J Dent Educ. 2014;18(3):170–9. pmid:24460682
- 34. Rosenbaum PE, Mikalsen O, Lygre H, Solheim E, Schjøtt J. A Blended Learning Course Design in Clinical Pharmacology for Post-graduate Dental Students. Open Dent J. 2012;6:182–7. pmid:23248716
- 35. Hussain F, Leinonen E, Millar BJ. Blended learning and an exploration of student expectations on a Master’s prosthodontics programme with reassessment at five years. Br Dent J. 2018;225(5):441–7. pmid:30191901
- 36. Shank P. (The right) digital modalities to deliver digital learning: Part 1 2020 [Available from: https://elearningindustry.com/asynchronous-and-synchronous-modalities-deliver-digital-learning.
- 37. Sandars J, Correia R, Dankbaar M, de Jong P, Goh PS, Hege I, et al. Twelve tips for rapidly migrating to online learning during the COVID-19 pandemic. MedEdPublish. 2020;9(1):82.
- 38. Khatami S, Macentee MI. Evolution of clinical reasoning in dental education. J Dent Educ. 2011;75(3):321–8. pmid:21368256
- 39. Anbarasi K, Vijayaraghavan P, Sathasivasubramanian S, Kandaswamy D. Integrated case scripts to enhance diagnostic competency. J Clin Exp Dent. 2015;7(3):e348–55. pmid:26330929
- 40. Dennick R. Constructivism: reflections on twenty five years teaching the constructivist approach in medical education. Int J Med Educ. 2016;25(7):200–5. pmid:27344115
- 41. Pascoe M, Monroe F, Macfarlane H. Taking Constructivism One Step Further: Post Hoc Analysis of a Student-Created Wiki. JMIR Med Educ. 2018;4(1):e16. pmid:29903697