https://orcid.org/0000-0002-4247-3952
Figures
Abstract
Purpose
This study aimed to measure stakeholder satisfaction with our usual delivery format, which previously relied on a blend of didactic lectures and clinical skills sessions compared to a revised format, which had more emphasis on online learning. We hypothesised that the online flipped classroom (OFC) would facilitate delivery of content in the wake of the pandemic, and result in improved levels of student satisfaction and knowledge gain.
Design
Non randomised intervention study. Group 1 = Traditional delivery (TD) and Group 2 = OFC group.
Methods
A validated course evaluation questionnaire (CEQ) compared perspectives of teaching faculty (n = 5) and students with the traditional delivery (TD) of the 4th year ophthalmology clinical attachment and an OFC approach (TD n = 129 v OFC n = 114).
Results
The OFC group (n = 114; response rate = 24.6%) reported significantly reduced satisfaction with staff motivation of students and provision of feedback, compared to TD (n = 129; response rate = 17.8%). OFC students also felt it was harder to determine what standard of work was expected and found the course less beneficial at helping develop problem-solving skills. Students were dissatisfied with the level of choice afforded by the OFC, specifically how they would learn and assessment options. No significant difference in exam score was observed between the TD and OFC groups. For faculty (n = 5), there was no evidence of a difference between OFC and TD.
Citation: Ní Gabhann-Dromgoole J, Murphy CC, Boland F, Doyle AJ, Pawlikowska T (2023) How has medical student learning changed with the pivot to online delivery of ophthalmology in the pandemic? PLoS ONE 18(3): e0282829. https://doi.org/10.1371/journal.pone.0282829
Editor: Priti Chaudhary, AIIMS: All India Institute of Medical Sciences, INDIA
Received: February 25, 2022; Accepted: February 23, 2023; Published: March 30, 2023
Copyright: © 2023 Ní Gabhann-Dromgoole et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This study was supported by the Department of Ophthalmology, Royal Victoria Eye and Ear Hospital (RVEEH) and the Health Professions Education Centre (HPEC), RCSI, University of Medicine and Health Sciences. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The global lockdown and government mandated social distancing measures enforced following the COVID-19 pandemic required that the clinical attachment Ophthalmology module be delivered online. This pivot to the online environment during the COVID-19 pandemic incorporated the flipped classrooms (FC) approach and offered a student-centred learning environment where the traditional face-to-face teaching pedagogy was transitioned into an online distance module using the Blackboard Collaborate (BBc) platform [1]. A pre pandemic review into the effectiveness of a FC concluded that the approach is a promising teaching strategy that can be utilised to promote learner engagement and motivation [6], however, there is little evidence to correlate the implementation of the FC and learner outcomes. In this study, the FC approach was adopted not only as a means to deliver content but also because it afforded the opportunity to engage students in this online environment, enhance problem solving skills and to promote deep learning [2]. Overall, in line with Blooms taxonomy and the FC pedagogy our revised module aimed to move from lower order to higher order cognition [3].
Introduction of technology is not an entirely new concept in medical education or ophthalmology, where studies have previously examined the effectiveness of video webcasts [4], and how digital learning compared to traditional didactic teaching interventions [5]. While there has been increasing interest in the use of a FC approach in medical education [6] and ophthalmology [7–11] there is controversy in the literature regarding this form of teaching particularly with regard to demonstrating positive changes in knowledge and skills. Several systematic reviews have investigated the efficacy of flipped classrooms [6, 12, 13], and while many studies reported the effectiveness of the flipped classroom in terms of learner outcomes categorised according to Kirkpatrick’s framework [14] and Blooms taxonomy, the varying methods for implementation have produced varying results [13]. Additionally, there is a paucity of research detailing changes in these aspects with respect to the use of FC to provide a means of providing continued education for ophthalmology students during the pandemic.
A review of developments in medical education in response to the COVID-19 pandemic [15, 16] found that seven studies described utilising the FC educational intervention [17–23] BEME Guide No 63 identified two studies describing the use of a combination of videoconferencing, FC (with question-and-answer time), video review of surgical procedures, and surgical simulators [21, 22] and BEME Guide No 64 identified further studies which described the incorporation of active learning strategies in response to the pandemic including FC [17, 19, 23]. One study by Sud et al described their pivot of undergraduate ophthalmology [20] and positive student feedback from this intervention in terms of ease of accessing material, asynchronous nature and ability to view multiple times. The student feedback also detailed some negative aspects of the intervention related to the diminished interactivity of this approach and the increased time for any doubts to be clarified. However, they do not detail any changes in knowledge gain or skill acquisitions.
There is significant interest in, and positives attributed to the FC, however, Diel et al suggest that further work is required to assess the efficacy of the FC approach for ophthalmology in a completely virtual setting, as necessitated by COVID-19 physical distancing constraints [24].
In response to the consequences of the COVID-19 pandemic, we pivoted a didactic and clinical ophthalmology module to an online flipped classroom approach encompassing the same learning objectives for 4th year medical students. We hypothesised that the flipped classroom delivery would result in more engagement with the material, better problem solving and so impact positively on their learning. Thus, as the literature regarding an entirely online flipped classroom approach for ophthalmology is limited our study is timely as it examines the perceptions of both students and faculty with the online FC pedagogy, representing Kirkpatrick–level 1- reaction [25]. Additionally, the evaluation of the impact of introducing this pedagogical model had on knowledge gain was determined, representing Kirkpatrick Level 2- learning [26].
Methods
Ethics
This study was reviewed and approved by the Research and Ethics Committee (REC) of the RCSI, University of Medicine and Health Sciences and was conducted according to the principles expressed in the Declaration of Helsinki. Written informed consent was be obtained from all participants (REC 202006015).
Module description
Our objective was to consistently address the learning outcomes (S1 Appendix) despite the change in delivery supporting constructive alignment whilst responding to the changes precipitated the by COVID-19 pandemic regarding social distancing etc (REF Irish Government Guidelines on closing in March 2020).
Subjects and study design
Study populations.
All students and faculty involved in the delivery of this module were invited to participate. Participants (243 students) in this study were 4th year senior cycle medical students enrolled in RCSI on an Ophthalmology clinical attachment that takes place 20 times during the academic year. Traditional delivery (TD) of the 4th year ophthalmology clinical attachment began on the 23rd September 2019 in the Royal Victoria Eye and Ear Hospital (RVEEH) and finished 09th March 2020. As a result of the global pandemic, an online distance module was devised. Students engaged in an online flipped classroom (OFC) which commenced on March 16th 2020 and finished on the 22nd May 2020. Faculty members (n = 5) were involved in both TD and OFC between the 23rd September 2019 and the 22nd May 2020 (Fig 1). As this is a natural experiment, two groups were spontaneously created; Group 1 = TD and Group 2 = OFC group.
Traditional delivery (TD) group (pre COVID-19 usual ophthalmology module)
Students (129 students) attended didactic lectures (Red eye, Sudden loss of vision, Gradual loss of vision, Ocular trauma) prior to commencing their clinical attachment. Students were assigned into groups by the SARA (Student, Academic & Regulatory Affairs) office RCSI (10–12 in each) before commencing their clinical attachment week. The traditional ophthalmology module consisted of a review of pre-recorded videos of ophthalmic examinations performed by ophthalmologists in the Royal Victoria Eye and Ear Hospital (RVEEH) on patients and simulated patients. Students attended clinical tutorials (Cataract, Glaucoma, Diabetic Retinopathy, Age-related Macular Degeneration (AMD)), which consisted of 60-min small group teaching sessions (face-to-face lectures with 15-minute question and answer session) led by an Ophthalmologist. Students then engaged in 60-min of patient led teaching which consisted of taking patient history, reading patient charts, examining patients etc. Students also engaged in practical skills sessions (Patient Based Teaching, Clinical Skills, Outpatients Department) and peer led teaching sessions, where students collaboratively presented in groups of 2 or 3 a joint presentation on a topic assigned at the beginning of the week. Knowledge was tested upon completion of the module via a multiple- choice question (MCQ) exam. Clinical Competency (skills) were assessed by practical examination of fundoscopy skills.
Online Flipped Classroom (OFC) group (during COVID-19 online ophthalmology module)
Students (114 students) were assigned into groups by the SARA office RCSI (12–18 in each) before attending a week long online ophthalmology course facilitated by ophthalmology department from the RVEEH. This week consisted of an online flipped classroom pedagogical approach where students reviewed pre-recorded lecture content and slide sets of the lecture material, which were made available in advance of online interactive classroom sessions. The small group didactic lectures were replaced by PowerPoint presentations with recorded audio. Clinical tutorials which were small group teaching sessions followed by patient encounters in the pre-COVID19 traditional delivery course were converted to interactive BBc tutorials on-line. Specifically students were asked to watch the pre-recorded video lectures (Cataract, Glaucoma, Diabetic Retinopathy, AMD) online in advance of the one hour interactive session led by an Ophthalmologist. This was supplemented by slide sets of the didactic lecture material without audio. After the pre-class lecture, students attended a synchronous, online, live interactive session on the same topic. These BBc sessions included problem solving, clinical vignettes and MCQs relating to the recorded lecture. Facilitators prepared clinical cases and related MCQs that addressed learning outcomes and promoted engagement for use during the interactive online session. The facilitator encouraged problem solving using the poll feature of BBc which encouraged both discussion and active learning. To facilitate a student-centred Peer Assisted Learning (PAL) pedagogical approach [27] students were provided with details on how to prepare PowerPoint presentations with audio for online peer-led teaching sessions. Knowledge was tested upon completion of the module via a multiple- choice question (MCQ) exam. Table 1 details the Red eye session format for TD versus OFC groups. A similar format was used for all topics delivered as part of the OFC approach.
Instrument and data collection
The majority of studies investigating perceptions of and satisfaction with FC utilise a questionnaire (either standardised or in house) to garner feedback from the students [13]. The CEQ is utilised annually by universities in Australia and the UK, among undergraduate students, to determine student satisfaction and to identify areas for improvement [28, 29]. There is a significant body of evidence in the literature supporting the reliability and validity of this questionnaire in higher education [29–35]. Observable differences in respondent scores in diverse fields of study or between institutions offering similar programmes in the same field demonstrates the discriminant validity of the CEQ [34]. In relation to investigations specifically focussing on FC and ophthalmology students Tang, Lin and Zhu et al have utilised the CEQ [7, 8, 11], which Broomfield et al determined was an appropriate instrument for course evaluation in medical education [32]. To date Paul Ramsden’s CEQ exists in three different formats: the CEQ23; CEQ30; and CEQ36. Each item of the questionnaire is answered using a standard 5-point Likert scale where the levels of agreement ranged from “strongly agree” (scoring a “1”) to “strongly disagree” (scoring a “5”). The CEQ36 measures six constructs established as important learning environment features within the context of higher education [29–31]:
- Good teaching (GT)
- Clear goals and standards (CG)
- Appropriate assessment (AA)
- Appropriate workload (AW)
- Generic skills (GS)
- Emphasis on Independence (IN)
To investigate key stakeholders’ perceptions of and satisfaction with the online flipped classroom, all students (n = 243) and faculty (n = 5) were invited to complete the CEQ36 online via Survey Monkey.
Additionally, final anonymised MCQ exam scores were obtained for each student in the study.
Statistical analysis
Descriptive statistics were used to describe the characteristics of the two groups and Chi-square test/Fisher exact test, or independent samples t test used to explore differences between the groups. The scores of the MCQ final exam were compared using independent samples t test. The questionnaire data given to students were analysed using Mann-Whitney-U tests, to explore potential differences between the groups. Data was collected from each staff member in relation to both teaching methods and hence the staff questionnaire data were analysed using Wilcoxon signed rank tests. No adjustment was made for multiple comparisons. During analysis responses for the ‘agree’ and ‘strongly agree’ categories were combined, similarly responses for ‘disagree’ and ‘strongly disagree’ category were combined. All statistical analyses were performed in GraphPad Prism V5 or Stata v13.
Digital training
Blackboard Collaborate (BBc) has previously been shown to have utility as a platform to support nursing students placement learning. Several studies have highlighted the importance training to develop student’s digital literacy to facilitate student engagement with this form of technology [36, 37]. To support this guides to the use of BBc were prepared and provided to the students ahead of the online module. Digital training was provided to ophthalmology faculty along with support guides for the use of the BBc platform.
Results
243 undergraduate medical students who received TD (n = 129) or OFC (n = 114) delivery of the ophthalmology clinical attachment were invited to participate. All students in the TD and OFC attended in person or online, completed the module and MCQ examination. Twenty-three students in the TD group and 28 students in the OFC group agreed to take part in the study and completed an online CEQ. This represents an overall participation rate of 17.8% for the TD group and 24.6% for the OFC group. All faculty (n = 5) delivered both the TD and the OFC module and completed the questionnaire giving a response rate of 100%. The demographic distribution of the participants is presented in Table 2. No gender difference was observed between TD and FC groups when compared to the class as a whole (column 1–3). Overall analysis of the demographic data indicated that a greater proportion of younger females participated in the online survey overall and specifically in the TD cohort, while the OFC survey participants were more presentative of the class as a whole.
Student perceptions
Table 3 summarizes the responses from the students regarding the six constructs established as important learning environment features within the context of higher education: Good Teaching (GT), Generic Skills (GS), Appropriate Assessment (AA), Appropriate Workload (AW), Clear Goals and Standards (CG), Emphasis on Independence (IN) [29–31]. We observed significant differences between the responses of the TD and OFC groups regarding the learning experience, perceived value of the flipped classroom, teaching process, and the evaluation system.
Good teaching scale
There was no evidence of a difference in student perceptions about the amount of time staff spent commenting on their work (Q9) or the effort staff made to understand student difficulties (Q20). Both the TD and OFC group felt that lecturers were good at explaining (Q23) and worked hard to make their topics interesting (Q25). Furthermore, the TD and OFC students felt that faculty had an interest in what students had to say (Q31) and that they tried to get the best out of all the students (Q33). However, compared to the TD group who attended the on-site traditional clinical attachment, students felt that the OFC approach did not motivate students to do their best (Q4, P = 0.01) or provide adequate feedback on how they were doing (Q22, P = 0.041) on the GT scale.
Clear goals and standards scale
There was no evidence of a difference in student perceptions on the goals and standards (CG) scale specifically about what was expected from them (Q8 & 18), about the objectives of the course (Q24) and faculty expectations of students being made clear (Q35). The OFC group were significantly less satisfied with the CG. Specifically, students felt it was harder to determine what standard of work was expected (Q1, P = 0.046).
Generic skills scale.
There was no evidence of a difference in student perceptions about the capacity for the TD or OFC course to sharpen their analytic skills (Q6), develop their ability to work as a team member (Q11), improve confidence about tackling unfamiliar problems (Q12), improve communication skills (Q13) or develop their ability to plan work (Q28). On the GS scale students felt that the OFC approach was significantly less beneficial in terms of helping develop their problem skills when compared to the TD group (Q2, P = 0.048).
Appropriate assessment scale.
There was no evidence of a difference between the TD and OFC group in student perceptions of the impression faculty gave about learning from students (Q7), that a good memory is all that is required to do well on the course (Q10), and that staff are more interested in testing what students have memorised (Q17). There was also no evidence of a difference between student perceptions from the two groups in relation to asking too many questions about facts (Q26). However, there was difference between the TD and OFC group in their perceptions of the form that feedback was given (Q29) or that just by working hard around exam times they could get through the course (Q32).
Appropriate workload scale.
There was no evidence of a difference in student perceptions in relation to the amount of topics covered in the syllabus (Q14), the amount of time given to learn (Q19), the pressure felt by students (Q27) or how the volume of work affects comprehension of topics (Q36). Interestingly, on the AW scale there was a significant difference in feeling about the workload being too heavy in TD group compared to the OFC group (Q5, P = 0.002).
Emphasis on independence scale.
There was no evidence of a difference in student perceptions between the TD and OFC group on the IN scale regarding opportunities to choose the particular areas you want to study (Q3), that the course encouraged them to pursue their academic interests (Q15) or the opportunities to discuss faculty how they were going to learn in this course (Q30). Analysis of the IN scale indicates that students were dissatisfied with the level of choice afforded by the OFC. Students within the TD group found that they had little choice over how they would learn (Q16, P = 0.018) or over the work they had to do (Q21, P = 0.011). Additionally, students in the OFC group felt that they had little choice in the way they were assessed compared to the TD group (Q34, P > 0.001).
Questions regarding the value of the flipped classroom.
Previous studies have highlighted questions within the CEQ survey, which provide insights into the perceived value of the flipped classroom [7], indicated by FC in Table 3. The FC scale questions overlap with the GT and GS scale; specifically questions 2, 4, 5, 11, 12, 13 and 28. Student survey responses indicated a lack of satisfaction with the online flipped classroom approach. As mentioned above students in the TD group felt that staff did less to motivate them (Q2, P = 0.048) and that there were less opportunities to improve their problem-solving skills (Q4, P = 0.01). While not reaching statistical significance within the FC scale a reduced percentage of students within the OFC group agreed that the online flipped classroom helped develop their ability to work as a team member compared to the TD group (Q11, 52.17% v 25% agree TD v OFC), tackle unfamiliar problems (Q12, 52.18% v 39. 28% agree TD v OFC) or develop their ability to plan their own work (Q28, 21.74% v 39.28% disagree TD v OFC). Overall, study participants showed no preference for traditional delivery of the ophthalmology over the online flipped classroom approach (Q 37). When asked to rate the statement “Overall, I am satisfied with the quality of this course” there was no evidence of a difference in the rating between the TD and the OFC group.
Comparison of faculty perspectives between the traditional delivery and the online flipped classroom.
Table 4 summarises the feedback from the faculty who taught the didactic and clinical–skills ophthalmology module on site and delivered the OFC ophthalmology module. There was no evidence of differences across the five scales. We compared trends in agreement from Face-to-Face teaching to online flipped classroom teaching for faculty, we defined changes as; agreement increased, stayed the same or decreased from face-to-face to the online approach (Table 5). On the GS scale, this analysis revealed that more faculty agreed that they put a lot of time into commenting on students work (Q9) and providing feedback (Q22) for the online versus the face-to-face approach. CS scale showed faculty agreed that it was easy to determine the standard of work expected (Q1) while the aims and objectives of this course were not made very clear (Q 24) for the OFC versus the TD approach. Comparison of the GS scale indicated that faculty perceived that the OFC approach supported improvements in student problem solving skills (Q2), analytic skills (Q6) and ability to tackle unfamiliar problems (Q12). An increased number of faculty disagreed that lecturers asked questions specifically about facts (Q26) on the AA scale for OFC approach versus traditional face-to-face teaching. Comparison of trends for the IN scale indicated that faculty felt the online flipped classroom approach had increased opportunities for students to choose particular areas of study (Q3), that the course encouraged students to develop their own academic interests (Q15) and provided more options regarding how students were going to learn on the course (Q16).
Comparison of overall student performance on final multiple-choice exam
Next we compared students’ exam scores before and after the educational intervention for all students in the TD (n = 129) and OFC groups (n = 114) and students in the TD (n = 23) and OFC groups (n = 28) who responded to the survey. Students answered 20 ophthalmology multiple-choice questions (MCQ) as part of completing the course. Each question had the same weight, and the total score was converted into a 0–100 scale. Independent samples t test was used to compare the differences between the two groups. This analysis of the final exam MCQ score showed that there were no statistical differences between the TD and OFC group (P = 0.0651). Comparison of the final exam MCQ score for survey responders between the TD and OFC found no evidence of a statistical difference in the score achieved. Overall, this indicates that the OFC did not negatively influence knowledge gain.
Discussion
The primary aim of this study was to measure stakeholder satisfaction with our usual delivery format, which previously relied on a blend of didactic lectures and clinical skills sessions compared to a revised format, which relied on a FC format to facilitate continued delivery of the ophthalmology module. It was hypothesised that as an educational intervention in response to the COVID-19 pandemic, a FC approach would not only facilitate delivery of content but result in improved or equivalent levels of student satisfaction and knowledge gain as determined by a CEQ and MCQ examination.
In pivoting from didactic face-to-face teaching to a solely online delivery of the ophthalmology module, we found that students were satisfied with the quality of this course in both the TD and the OFC groups. However, students in the OFC group were less satisfied with staff motivation of students and provision of feedback, compared to the TD group. OFC students also felt it was harder to determine what standard of work was expected and found the course less beneficial in terms of helping develop their problem-solving skills. Furthermore, students were dissatisfied with the level of choice afforded by the online FC, compared to TD group. Of note, while student responses indicated a lack of satisfaction with the OFC approach compared to traditional face-to-face teaching exam scores were consistent between both groups.
Recent reviews have examined developments in medical education interventions in the wake of the pandemic [15, 16]. These reviews found that a significant proportion of interventions focused on pivoting traditional teaching online (53%) with a significant proportion of these studies using the same teaching approaches for face-to-face teaching and their online approach. These reviews noted there was a lack of interventional outcome data in the studies reviewed. More recently, Ferrara et al conducted an online survey to assess the changes ophthalmology residents and fellows have experienced in ophthalmology training related to the current COVID-19 pandemic and collected responses from 32 different countries. Based on the survey responses of ophthalmology trainees’ perspectives the authors suggest that incorporation of technological educational tools, can increase the capacity to cope successfully with the current situation and continued use may improve the effectiveness of training programs in the long term [38]. However, the perspective of Irish ophthalmology students, trainees or fellows was not included in this study.
Herein the current study describes an educational intervention and uniquely provides data on the perspectives of those involved: both student and faculty satisfaction and perceptions of an online flipped classroom approach for ophthalmology teaching. We devised a remote synchronous and asynchronous module that was instigated in response to the global COVID-19 pandemic. By adopting an online flipped classroom approach and incorporating elements of Blooms taxonomy we aimed to move away from recall and enhance critical thinking and application of knowledge [3]. Overall, our results are in direct contrast to existing literature regarding the utility of a flipped classroom approach for delivering ophthalmology content to medical students. Previously it has been shown that students preferred the flipped classroom approach to the traditional lecture method as it helped them to develop problem solving, creative thinking and team working skills [7, 10]. However, we found significant dissatisfaction with the online flipped classroom approach among the OFC group. Significant levels of dissatisfaction were observed in problem solving, communication, staff motivation and provision of feedback. Previous groups examining a flipped classroom approach for ophthalmology students have reported higher levels of knowledge gain (increased exam scores) compared to the traditional teaching group [8–10]. The current study found that exam outcome was consistent with both approaches: the remote flipped classroom approach did not improve knowledge gain for the OFC group compared to the prior didactic and clinical skills-based module.
A recent review detailed the types of uncertainty faced by students, including those relating to the educational process and the global coronavirus pandemic, and explored the potential negative impact this can have on learning [39]. Unger et al reported that students experienced a significant level of anxiety in relation to online learning and that this anxiety persisted beyond an initial 3-week practice period [40].
Shahrvini et al similarly reported that students experienced anxiety with the transition of their curriculum online and perceived this mode of delivery had negatively impacted the quality of instruction [41]. Students reported satisfaction with the flexibility of remote learning, however Shahrvini et al suggests that removal of face-to-face teaching and adoption of a solely online program, which leads to digital fatigue, coupled with loss of practical elements of the curriculum contributed to the students negative perceptions towards online learning [41]. They suggest that video casted lectures uploaded in advance, electronic health record and telehealth training for students, in addition to training for teaching faculty to increase technological fluency should be considered to optimise remote learning curricula [41]. A survey conducted by Mishra et al highlights that the COVID-19 lockdown has brought with it, uncertainty, anxiety and higher stress levels among ophthalmology trainees across India due to the disruption of training program schedules [42]. Therefore, anxiety, uncertainty and digital fatigue may be factors contributing to student dissatisfaction with the online flipped classroom approach in our study.
Faculty preparedness is another aspect that may have influenced student satisfaction with our educational intervention. Overall, despite literature describing a range of innovative ways to deliver teaching in response to the pandemic, there is relatively little existing in current literature focusing on faculty development or support. Of note the BEME Guide No 63 [16] highlights three studies that describe measures used to support medical educators move to online including adapting existing programs for online delivery, curating resources, providing a webinar where best practices were shared and establishing a twitter community of practice [43–45]. Furthermore, BEME Guide No 64 [15]: described the faculty development intervention undertaken by Buckley who established a virtual faculty development session to bring together regionally dispersed teaching faculty groups together to learn with and from each other using social networking theory [46]. This virtual faculty development initiative had a range of benefits including supporting faculty to embrace the new challenges of teaching online and importantly providing skills to allow faculty to attend to student well-being, however some issues with its implementation were highlighted. Specifically, that the online webinar did not permit informal side bar conversations, that organisers should consider inclusion of smaller break out rooms to promote discussion along with additional forms of follow up commination eg email [46].
Faculty in RCSI, including those involved in ophthalmology teaching, attended an in-person training course before the closure of the campus, which aimed to increase faculty digital competence with the online delivery platform BBc as we pivoted delivery of our module online. Results from our study indicate that students felt on the ‘Good Teaching Scale’ that the faculty approach to delivery of the OFC did not motivate students to do their best or provide adequate feedback. This resulted in students feeling that the module was overall less beneficial in terms of helping them to develop their problem skills when compared to the TD group. Thus, in line with the main findings of the recent BEME reviews our results confirm that significant additional levels of support are required to support faculty to develop and implement online learning.
Additional factors potentially contributing to the overall dissatisfaction with the course include the lack of clinical experiences in the redesigned ophthalmology curriculum, reduced digital efficacy and technical issues. Kostaki et al found that student engagement negatively correlated with technical difficulties and home distractions [47]. Furthermore, they reported that computer self-efficacy positively contributed towards engagement as students could remedy technical difficulties [47]. Brockman et al compared student perceptions of an online or in-person microbiology laboratory and found that while students have positive perceptions of digital online activities there was still a significant preference for a blend of online and in-person practical activities [48].
There has been significant interest in the FC pedagogy a means to continue to deliver education during these extraordinary times [49–51]. While many researchers have indicated that this approach should be maintained long term in an effort to modernise medical education [42, 52], Fisher et al advised that the strategy for the flipped classroom is important, as the environment needs to facilitate learning rather than influence learning to be satisfactory for students [53].
Change to curriculum
Our investigations and comparison with contemporary literature indicate that students prefer face-to-face teaching with practical elements. While inclusion of online technology within the curriculum is essential to enhance student digital proficiency and future proof against future disruptions, elements of the traditional curriculum are indispensable.
Therefore, future developments should include a blend of traditional classroom-based and remote learning approaches combined with in person practical elements including direct patient contact with mitigated risk. Additionally training of both faculty and students will help to increase digital proficiency and engagement as online elements are predicted to be a central feature on medical education developments.
Study limitations
The results from our investigations should be viewed in light of the limitations of this study. One such limitation is the lack of student participation in the study resulting in poor respondent rates. The ongoing global pandemic at the time of participant recruitment is a potential factor leading to a lack of study respondents. Additionally we have identified the delay from module completion to distribution of the post-clinical attachment survey as another factor influencing study participation. However, by using the widely validated CEQ we believe that this study offers a unique and important reflection of the impact that the pandemic has had on student learning during the pandemic. Another limitation is that that the OFC did not include clinical skills training which a staple in the TD approach is. It was not possible for our department to teach clinical skills online and consequently this aspect of the clinical attachment was not assessed for the OFC groups. Future studies seeking to determine changes in student and learning and student satisfaction with the OFC for delivery of Ophthalmology content should address these limitations by inclusion of our suggested changes to the curriculum.
Supporting information
S1 Appendix. Ophthalmology module learning outcomes.
https://doi.org/10.1371/journal.pone.0282829.s001
(DOCX)
Acknowledgments
Helpful discussions and advice given by Dr Jenny Moffett. Support from the Student, Academic & Regulatory Affairs office in RCSI.
References
- 1. Sohrabi B, Iraj H. Implementing flipped classroom using digital media: A comparison of two demographically different groups perceptions. Computers in Human Behavior. 2016;60:514–24.
- 2. Prober CG, Heath C. Lecture Halls without Lectures—A Proposal for Medical Education. New England Journal of Medicine. 2012;366(18):1657–9. pmid:22551125
- 3. Conklin J. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. Educational Horizons. 2005;83(3):154–9.
- 4. Grover S, Kelmenson AT, Chalam KV, Edward DP. Webcasts for Resident Education. Ophthalmology. 2012;119(1):199–200.e1. pmid:22214944
- 5. Mayorga E, Golnik K, Palis G. One-Year Progress in Ophthalmic Education: Annual Review. The Asia-Pacific Journal of Ophthalmology. 2015;4(6). pmid:26716435
- 6. Chen F, Lui AM, Martinelli SM. A systematic review of the effectiveness of flipped classrooms in medical education. Medical Education. 2017;51(6):585–97. pmid:28488303
- 7. Lin Y, Zhu Y, Chen C, Wang W, Chen T, Li T, et al. Facing the challenges in ophthalmology clerkship teaching: Is flipped classroom the answer? PloS one. 2017;12(4):e0174829–e. pmid:28384167
- 8. Tang F, Chen C, Zhu Y, Zuo C, Zhong Y, Wang N, et al. Comparison between flipped classroom and lecture-based classroom in ophthalmology clerkship. Medical Education Online. 2017;22(1):1395679. pmid:29096591
- 9. Yang Y, Xu C-C, Jia Y, Zou Y-X, Ao Y, Huang Z-Q, et al. Flipped classroom approach to ophthalmology clerkship courses for Chinese students of eight-year program. Annals of Eye Science. 2017;2(7):40.
- 10. Ding C, Li S, Chen B. Effectiveness of flipped classroom combined with team-, case-, lecture- and evidence-based learning on ophthalmology teaching for eight-year program students. BMC Medical Education. 2019;19(1):419. pmid:31727043
- 11. Zhu L, Lian Z, Engström M. Use of a flipped classroom in ophthalmology courses for nursing, dental and medical students: A quasi-experimental study using a mixed-methods approach. Nurse education today. 2020;85:104262. pmid:31759243
- 12. Chen K-S, Monrouxe L, Lu Y-H, Jenq C-C, Chang Y-J, Chang Y-C, et al. Academic outcomes of flipped classroom learning: a meta-analysis. Medical Education. 2018;52(9):910–24. pmid:29943399
- 13. King AM, Gottlieb M, Mitzman J, Dulani T, Schulte SJ, Way DP. Flipping the Classroom in Graduate Medical Education: A Systematic Review. Journal of graduate medical education. 2019;11(1):18–29. pmid:30805092
- 14. Yardley S, Dornan T. Kirkpatrick’s levels and education ‘evidence’. Medical Education. 2012;46(1):97–106.
- 15. Daniel M, Gordon M, Patricio M, Hider A, Pawlik C, Bhagdev R, et al. An update on developments in medical education in response to the COVID-19 pandemic: A BEME scoping review: BEME Guide No. 64. Medical Teacher. 2021;43(3):253–71. pmid:33496628
- 16. Gordon M, Patricio M, Horne L, Muston A, Alston SR, Pammi M, et al. Developments in medical education in response to the COVID-19 pandemic: A rapid BEME systematic review: BEME Guide No. 63. Medical Teacher. 2020;42(11):1202–15. pmid:32847456
- 17. Beer L, Gray M, Carbajal MM, French H, Vasquez M, Bauserman M, et al. "Megaflip," a Novel Approach to National Collaboration for Flipped Classroom Education. Acad Pediatr. 2020;20(6):758–9. pmid:32502533
- 18. Hironmoy Roy KR, Satyajit Saha, Ghosal Asis Kumar. Study on Students’ Perceptions for Online Zoom-app based Flipped Class Sessions on Anatomy Organised during the Lockdown Period of COVID-19 Journal of Clinical and Diagnostic Research. 2020;14(6):AC01–AC4.
- 19. Naidoo N, Akhras A, Banerjee Y. Confronting the Challenges of Anatomy Education in a Competency-Based Medical Curriculum During Normal and Unprecedented Times (COVID-19 Pandemic): Pedagogical Framework Development and Implementation. JMIR Med Educ. 2020;6(2):e21701. pmid:32873536
- 20. Sud R, Sharma P, Budhwar V, Khanduja S. Undergraduate ophthalmology teaching in COVID-19 times: Students’ perspective and feedback. Indian journal of ophthalmology. 2020;68(7):1490–1. pmid:32587212
- 21. Chick RC, Clifton GT, Peace KM, Propper BW, Hale DF, Alseidi AA, et al. Using Technology to Maintain the Education of Residents During the COVID-19 Pandemic. Journal of surgical education. 2020;77(4):729–32. pmid:32253133
- 22. Kanneganti A, Lim KMX, Chan GMF, Choo SN, Choolani M, Ismail-Pratt I, et al. Pedagogy in a pandemic—COVID-19 and virtual continuing medical education (vCME) in obstetrics and gynecology. Acta Obstet Gynecol Scand. 2020;99(6):692–5. pmid:32418212
- 23. Roy SF, Cecchini MJ. Implementing a structured digital-based online pathology curriculum for trainees at the time of COVID-19. Journal of Clinical Pathology. 2020;73(8):444. pmid:32366598
- 24. Diel RJ, Yom KH, Ramirez D, Alawa K, Cheng J, Dawoud S, et al. Flipped ophthalmology classroom augmented with case-based learning. Digit J Ophthalmol. 2021;27(1):1–5. pmid:33907530
- 25.
Kirkpatrick DL. Evaluating training programs: the four levels. San Francisco: Berrett-Koehler. 1994.
- 26. Kirkpatrick DL. Evaluating training programs: Tata McGraw-hill education; 1975.
- 27. Friel O, Kell D, Higgins M. The Evidence Base for Peer Assisted Learning in Undergraduate Medical Education: a Scoping Study. MedEdPublish. 2018;7.
- 28. Griffin P, Coates H, McInnis C, James R. The Development of an Extended Course Experience Questionnaire. Quality in Higher Education. 2003;9(3):259–66.
- 29.
Hanbury A. Comparative review of British, American and Australian national surveys of undergraduate students. York: Higher Education Academy; 2007.
- 30. Wilson KL, Lizzio A, Ramsden P. The development, validation and application of the Course Experience Questionnaire. Studies in Higher Education. 1997;22(1):33–53.
- 31. Ramsden P. A performance indicator of teaching quality in higher education: The Course Experience Questionnaire. Studies in higher education. 1991;16(2):129–50.
- 32. Broomfield D, Bligh J. An evaluation of the ’short form’ course experience questionnaire with medical students. Medical Education. 1998;32(4):367–9. pmid:9743797
- 33. Richardson JTE. A British evaluation of the Course Experience Questionnaire. Studies in Higher Education. 1994;19(1):59–68.
- 34. Richardson JT. Instruments for obtaining student feedback: A review of the literature. Assessment & evaluation in higher education. 2005;30(4):387–415.
- 35. Byrne M, Flood B. Assessing the Teaching Quality of Accounting Programmes: An evaluation of the Course Experience Questionnaire. Assessment & Evaluation in Higher Education. 2003;28(2):135–45.
- 36. Al-Drees A, Khalil MS, Meo SA, Abdulghani HM. Utilization of blackboard among undergraduate medical students: Where we are from the reality? Journal of Taibah University Medical Sciences. 2015;10(1):16–20.
- 37. Hart T, Bird D, Farmer R. Using blackboard collaborate, a digital web conference tool, to support nursing students placement learning: A pilot study exploring its impact. Nurse Education in Practice. 2019;38:72–8. pmid:31195366
- 38. Ferrara M, Romano V, Steel DH, Gupta R, Iovino C, van Dijk EHC, et al. Reshaping ophthalmology training after COVID-19 pandemic. Eye. 2020;34(11):2089–97. pmid:32612174
- 39. Moffett J, Hammond J, Murphy P, Pawlikowska T. The ubiquity of uncertainty: a scoping review on how undergraduate health professions’ students engage with uncertainty. Advances in Health Sciences Education. 2021:1–46.
- 40. Unger S, Meiran W. Student Attitudes Towards Online Education during the COVID-19 Viral Outbreak of 2020: Distance Learning in a Time of Social Distance. International Journal of Technology in Education and Science. 2020;4:256–66.
- 41. Shahrvini B, Baxter SL, Coffey CS, MacDonald BV, Lander L. Pre-Clinical Remote Undergraduate Medical Education During the COVID-19 Pandemic: A Survey Study . Res Sq. 2020:rs.3.rs-33870.
- 42. Mishra D, Nair AG, Gandhi RA, Gogate PJ, Mathur S, Bhushan P, et al. The impact of COVID-19 related lockdown on ophthalmology training programs in India—Outcomes of a survey. Indian journal of ophthalmology. 2020;68(6):999–1004. pmid:32461413
- 43. Cleland J, McKimm J, Fuller R, Taylor D, Janczukowicz J, Gibbs T. Adapting to the impact of COVID-19: Sharing stories, sharing practice. Medical Teacher. 2020;42(7):772–5. pmid:32401079
- 44. Finn GM, Brown MEL, Laughey W, Dueñas A. #pandemicpedagogy: Using Twitter for knowledge exchange. Medical Education. 2020;54(12):1190–1. pmid:32418270
- 45. Keegan DA, Chan MK, Chan TM. Helping medical educators worldwide to pivot curricula online: pivotmeded.com. Med Educ. 2020;54(8):766–7. pmid:32392352
- 46. Buckley H. Faculty development in the COVID-19 pandemic: So close—yet so far. Med Educ. 2020;54(12):1189–90. pmid:32436272
- 47. Kostaki D, Karayianni I. Houston, we Have a Pandemic: Technical Difficulties, Distractions and Online Student Engagement 2021.
- 48. Brockman RM, Taylor JM, Segars LW, Selke V, Taylor TAH. Student perceptions of online and in-person microbiology laboratory experiences in undergraduate medical education. Medical Education Online. 2020;25(1):1710324. pmid:31928152
- 49. Reyna J. Twelve Tips for COVID-19 friendly learning design in medical education. MedEdPublish. 2020;9(103).
- 50. Kaup S, Jain R, Shivalli S, Pandey S, Kaup S. Sustaining academics during COVID-19 pandemic: The role of online teaching-learning. Indian journal of ophthalmology. 2020;68(6):1220–1. pmid:32461490
- 51. Chen CH, Mullen AJ. COVID-19 Can Catalyze the Modernization of Medical Education. JMIR Med Educ. 2020;6(1):e19725. pmid:32501809
- 52. Wong TY, Bandello F. Academic Ophthalmology during and after the COVID-19 Pandemic. Ophthalmology. 2020;127(8):e51–e2. pmid:32359842
- 53. Fisher RL, LaFerriere R, Rixon A. Flipped learning: An effective pedagogy with an Achilles’ heel. Innovations in Education and Teaching International. 2020;57(5):543–54.