The effects of the flipped classroom have been demonstrated in various fields of education in recent years. Training in emergency medicine is also beginning to gradually implement the flipped classroom; however, its practical effect in emergency medicine contexts is not yet clear.
The present study investigates the effects of the flipped classroom on advanced cardiopulmonary life support (ACLS) training implemented among practicum students in emergency medicine.
The study randomly assigned into control and experimental conditions 108 fourth year students in the College of Medicine at Yonsei University, in Seoul, who were scheduled to take clinical practice in emergency medicine between March and July 2017. Students were taught about ACLS in either a traditional lecture-based classroom (control condition) or a flipped classroom (experimental condition); then, simulation training with ACLS scenarios was carried out. Finally, each student was rated on performance using a rating form developed in advance.
ACLS simulation scores of the students in the flipped classroom were 70.9±10.9, which was higher than those of the students in the traditional classroom (67.1±11.3); however, this difference was not statistically significant (p = 0.339). In addition, the difference in student satisfaction as measured on a survey was statistically insignificant (p = 0.655).
Citation: Beom JH, Kim JH, Chung HS, Kim SM, Ko DR, Cho J (2018) Flipped-classroom training in advanced cardiopulmonary life support. PLoS ONE 13(9): e0203114. https://doi.org/10.1371/journal.pone.0203114
Editor: Didier Lepelletier, Nantes University Hospital, FRANCE
Received: October 27, 2017; Accepted: August 14, 2018; Published: September 5, 2018
Copyright: © 2018 Beom et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
The concept of the “flipped classroom” was first implemented in 2012 ; since then, its effect has been demonstrated in various fields of education. Unlike in the traditional classroom, students in the flipped classroom review material and obtain knowledge ahead of class, and engage in group activities or projects relying on that knowledge in class [1–6]. Effects of flipped learning stemming from this element of active participation of students have been shown in various contexts. Most students have shown higher posttest scores compared with pretest scores and higher student satisfaction with the class when taught in a flipped classroom than in a traditional classroom [3,7–9]. It has also been reported, however, that national medical licensing examination scores showed no difference between types of instruction [6,7]. In one study, students scored higher on a skills test for reading an electrocardiogram in the flipped classroom, and spent more time studying alone before class than those who learned in a lecture-based classroom . Belfi et al. compared pre- and posttest scores among students who underwent training in radiology in the flipped classroom, in the lecture-based classroom, and in self-study, respectively; although students in all three groups showed increase in scores after training, the greatest improvement was in the flipped-classroom students, while the other two groups improved at similar, lower levels .
The training paradigm of medical schools is shifting to one of performance-based training, because medical students need to develop competencies in medical care for successful performance of their future responsibilities, in addition to the medical knowledge that doctors must have [2,12]. Developing competencies requires accumulating hands-on experience treating actual patients and cannot be based on simple knowledge transfer, memorization, or written tests. In order to assess the competencies of graduating medical students, the Republic of Korea began implementing the Objective Structured Clinical Examination (OSCE) and Clinical Performance Examination (CPX) its National Medical Licensing Examination in 2010. The basis for incorporating these tests was the argument proposed in “Miller’s Pyramid” that the assessment of “knows” (i.e., assessing knowledge level using a written test) is the most primitive stage of assessment, and that assessment needs to move toward assessing “shows” (i.e., assessing whether the student can demonstrate and implement what he or she learned) .
The importance of competency in actual medical treatment is especially high in emergency response, such as response to cardiac arrest, because the seriousness of failure to perform properly under psychological and time pressures in emergency is not mitigated whatever the practitioner’s level of abstract knowledge. For this reason, training in emergency medicine is also beginning to gradually implement the flipped classroom; however, its practical effect in emergency medicine contexts is not yet clear. Lew reported that both students and teachers in emergency medicine rated the flipped classroom high in terms of satisfaction with the class, in a session teaching an approach to chest pain . Tan et al. reported similar results for an emergency medicine practicum . Rose et al. found that residents in emergency medicine who had learned pulmonary resuscitation of children through simulation and skills training scored higher on posttest than pretest using multiple-choice questions and on satisfaction .A study on teaching advanced cardiopulmonary life support (ACLS) found that students’ scores on various written tests improved when the flipped classroom was used . These studies, however, did not assess students’ actual competency.
It is important for ACLS to be taught using a performance-based approach and evaluated with competency-based methods as opposed to written tests. However, teaching students the required knowledge and skills by actually allowing them to treat patients with cardiac arrest has many practical and ethical complications. Therefore, the present study investigates the effects of the flipped classroom as a means of teaching ACLS to clinical clerkship students in emergency medicine using a simulation approach.
Study design and participants
This study was randomized and single blinded. We randomly assigned to experimental and control groups 108 fourth-year students in the College of Medicine at Yonsei University, in South Korea, who were scheduled to be in clinical practice in emergency medicine between March and July 2017. The control group learned ACLS in a traditional lecture-based classroom, and the experimental group in a flipped classroom. We assigned students randomly to groups using computer software in February; of 108 students, 53 were assigned to the traditional lecture-based classroom and 55 to the flipped classroom.
During the student orientation on the first day of clinical practice for each group, one of the researchers briefed students on the study, handed out the study participation consent forms, asked them to read the form, had a Q&A session with the students regarding the study, and asked for their consent to participate. This researcher was in a position uninvolved in student grades and evaluations. The researcher informed students that their participation would have no bearing on their course grades and that their participation status would not be processed until their course grades had been submitted. Students had at least eight hours to make their decision on participation, and submitted the consent form in a sealed envelope at the end of the practicum period. We received the submitted consent forms after students’ final course grades in practical training had been submitted to the College of Medicine, that is, upon completion of practical training for both groups. The faculty members who evaluated the educational effects of the two teaching methods using the simulation were unaware of the status of individual students regarding study participation or of the assigned teaching method. Students who insisted on a particular teaching method or who did not attend the ACLS training classes or the simulation assessment session were excluded from the study.
The primary outcome was simulation rating scores and the secondary outcome was the students’ satisfaction survey.
This study was approved by Institutional Review Board of Severance Hospital [Approval No. 4-2016-0415] and the Clinical Research Information Service (CRIS); Korea Centers for Disease Control and Prevention, Ministry of Health and Welfare (Republic of Korea) [KCT0002416].
Practical training in emergency medicine lasted for two weeks, including a one-hour class on ACLS and two hours of simulation. In the traditional classroom, the instructor gave a one-hour lecture using PowerPoint slides and held a Q&A session. In the flipped classroom, the instructor emailed students a PowerPoint file with a recorded explanation three days before the class, to allow them to study in advance; then, in class, students watched a video of an example of ACLS and engaged in group discussion. The video contained a “bad case” scenario developed by the Korean Association of Cardiopulmonary Resuscitation for educational purposes. After watching the video, students were asked to find and correct the errors in the treatment procedure, in teams (3 or 4 students). After the ACLS class, students received two hours’ simulation training to reduce potential difference in understanding of the simulation and underwent assessment of the educational effects in a familiar environment. In addition, to maintain objectivity, different faculty members separately conducted classroom instruction, simulation training, and student assessment, and faculty members ensured that all student groups received the same instructional content throughout the study period. On the last day of practical training, students in both groups performed an identical ACLS scenario on a simulation mannequin (Laerdal Medical, Stavanger, Norway), in teams of three or four; the researchers scored their performance as a team using a rating form developed in advance. Students were trained in the simulation in separate rooms with different access routes, to prevent exchange of information on the simulation. In addition, to determine whether any differences in assessment results of the two groups were due to features of that particular simulation, the OSCE and CPX scores that were scored during the emergency medicine clerkship were compared and they also went advanced trauma life support (ATLS) simulation training and assessment, conducted in the same manner as for ACLS.
The simulation rating form consisted of items on individual skills and a global rating. A total of 13 items on skills concerned whether each skill was performed properly; each item scored one point if the skill was demonstrated. The global rating item scored up to five points based on the assessment of overall performance on the simulation. (Table 1)
The highest possible score depended on the scenario: 18 points if fibrillation was necessary and 17 if not. We converted scores to a 100-point scale. Due to the nature of simulation training, scores applied to the whole team instead of each individual. After the assessment, the participants completed a survey to determine their satisfaction with the respective methods of instruction they had experienced by grading between 1 to 10 points by themselves (Table 2).
We calculated sample sizes for a two-sample means test using previous study data of 2 teams (181 students) with a traditional classroom group means of 8.03, SD±1.01, and a flipped classroom group means of 8.72, SD±1.01, in the ECG interpretation score . Based on this data, the sample size was obtained using PASS (version 12, NCSS, LLC, Kaysville, Utah, USA). We estimated that a sample size of 53 pairs (106 participants) would be sufficient to evaluate the primary outcome at a significance level of 0.05 (two-sided) with 80% power, considering a 10% dropout rate.
We used R version 3.2.2 (The R Foundation for Statistical Computing, Vienna, Austria) for the statistical analyses. We performed score comparison using t-test and Wilcoxon rank-sum test, and frequency comparison using chi-squared test and Fisher’s exact test. For all tests, we defined p-value <0.05 as statistically significant.
Out of 122 practicum students during the study period, 108 participated in the study. The 14 students who did not participate included 13 students who had already received training before preparation for the study was complete, and thus could not consent to participation, and one student who could not participate in clinical practice due to illness. (Fig 1)
ACLS: Advanced Cardiopulmonary Life Support.
We present demographic information on study participants in Table 3.
ACLS simulation scores of the students in the flipped classroom were 70.9±10.9, which was higher than those of the students in the traditional classroom (67.1±11.3); however, the difference was not statistically significant (p = 0.339). The difference in student satisfaction as measured on the survey was also statistically insignificant (p = 0.655).
The students in the flipped classroom showed higher scores on the following individual items: checking if the patient is conscious, requesting help or speaker announcement, and properly administering epinephrine. The students in the lecture-based classroom showed higher scores only on checking at 2-minute intervals to see if self-circulation was restored (Table 4)
A variety of efforts are being made to shift the curriculum of medical training from a teacher-centered to a learner-centered model. This is because modern medical training is inefficient and inflexible, and focuses on getting good test scores instead of developing professional competencies [2,19]. The flipped classroom is an effective teaching method for changing students from passive to active learners and creating integrative educational strategies, as the teacher provides students with class materials ahead of time for students to study alone, while classroom activities such as discussion that require students’ active participation are the focus during the class .
The flipped classroom demonstrates better outcomes than the traditional lecture-based approach for students’ comprehension and application of course content . A previous study compared the written test results of practicum students in emergency medicine who had learned about ACLS in a flipped classroom versus those who had learned in a regular classroom using lecture and simulation, and showed that the students in the flipped classroom scored higher than the other students . The present study randomly assigned senior medical students on their practicum in emergency medicine to learn ACLS in two groups, one in a flipped classroom and one in a traditional lecture-based classroom. Competency assessment based on simulation performance results showed no statistically significant difference between groups. However, the observed difference was, although not statistically significant, tending toward 100 points which is important because our checklists were made through the essential process of ACLS. The success of ACLS does not merely depend on any one particular action or process. It depends on the results of the appropriation of the whole action or process.
There are other studies that do not demonstrate any effect of the flipped classroom. Chen et al. conducted a systematic review of nine studies that evaluated the flipped classroom in medical training, and reported that its effect on knowledge and skills change was inconclusive, as the statistical data required to estimate effect size could be obtained from only four studies . A study with students in an emergency medicine practicum similarly found that those who had learned in a flipped classroom did not show more positive outcomes than those in a traditional classroom on a multiple-choice test; however, due to the lack of data on how well students had prepared for the class in the flipped classroom, this study’s findings may have been limited in value . In another study, on the comparative effects of the flipped classroom for students on their ophthalmology practicum, final test scores showed no significant difference between treatment and control groups . It has even been noted that, although the flipped classroom is an attractive teaching method for residents who do not have enough time for class, actual results show that only satisfaction with the method, not its substantive educational effect, is high . Finally, a study of a neuroanatomy course showed that the flipped classroom evidenced no differences in test scores or in satisfaction from the traditional teaching method, which the investigator of the study interpreted as being because there was too much content to teach .
Overall, research that has demonstrated the efficacy of the flipped classroom has been mostly based on the comparison of written test results. Written tests are certainly an easy way to assess students; however, since medical training needs to develop and assess actual practical competencies for medical treatment, it remains important to conduct more studies assessing competencies, whether using simulations or in the actual job setting, instead of simply comparing written test results to evaluate the educational effects of the flipped-classroom method.
Our study has some limitations. First, where most previous studies in this area have focused on students’ satisfaction and changes in knowledge and skills, and have failed to compare student outcomes in higher domains, for example as presented in Kirkpatrick’s classification (e.g., change in behavior, professional competencies, patient outcomes), the present study focused on competencies, teamwork, and problem-solving skills using simulation as an assessment method; however, this study could not compare patient outcomes or long-term behavior change because of the limitations of the simulation and time limitations. Second, we could not find an objective way to measure students’ compliance. In a different context, Heitz et al. reported that only 31% of students failed to properly prepare for class . In the present study, it is likely that the discussion format used in the flipped classroom placed pressure on students to prepare for the class; however, there are no relevant data. Third, we could not eliminate the possibility of the cross-contamination issue because students were not bound to their group assignment, and therefore could study with students from the other group. However, within the group, students were trained on simulation in separate rooms with different access routes, to ensure they did not share information on the simulation. Also, we did not reveal the checklists to students to ensure they did not find out what was going to happen before the simulation. Fourth, we have the limitation of being unable to be free of multiple testing issues while verifying each hypothesis for multiple questions.
Finally, due to the structure of the medical school practicum, we implemented training for only two weeks, even though there were many required items to teach; consequently, only one hour of the flipped classroom was applied to ACLS training. Thus, the minimum number of hours needed for the flipped classroom to be effective has not yet been established, and further study will be needed on these issues.
Nevertheless, it is worth appreciating the fact that almost all students and teachers preferred the flipped classroom, although the studies found only limited or no effects, because motivation is the first step for adult learners to gain educational benefits.
An ACLS competency assessment conducted using a simulation after instruction in ACLS undergone by senior medical students randomly assigned to either a flipped classroom or a traditional lecture-based classroom showed no statistically significant difference between the two teaching methods.
ACLS Scenario: Pulmonary Embolism→Non-Shockable Arrest
- ■. A 56-year-old male patient complained of difficulty breathing at dawn and visited the emergency room. The patient was diagnosed with lung cancer and malignant pleural effusion.
- ■. When he arrived in the emergency room, his mental state was drowsy (moaning).
- ■. Vital signs:
- Blood pressure 80/60 mmHg; pulse rate 68 times/min; respiration rate 24 times/min
- Body temperature 36.8 degrees Celsius; oxygen saturation 92%
- ECG normal sinus rhythm; lung sound clear
- ■. Adjust the oxygen saturation pattern according to the oxygen therapy
- ■. Patient’s electrocardiogram rhythm suddenly turns into pulseless electrical activity (PEA) and consciousness disappears.
- ■. Ensure that the treatment is conducted according to the non-shockable rhythm algorithm.
- ■. Debriefing points: non-shockable rhythm + PEA + reversible causes (5H & 5T)
ACLS Scenario: Hyperkalemia→Shockable Arrest
- ■. A 56-year-old male patient complained of swollen face and decreased urination in recent months. Today he complained of dizziness and fainted and visited the emergency room.
- ■. When he arrived in the emergency room, his mental state was drowsy (moaning).
- ■. Vital signs:
- Blood pressure 80/60 mmHg; pulse rate 110times/min; respiration rate 24 times/min;
- Body temperature 36.8 degrees Celsius; oxygen saturation 92%;
- ECG Tall T-wave, wide QRS
- ■. Adjust the oxygen saturation pattern according to the oxygen therapy
- ■. Patient's electrocardiogram rhythm suddenly turns into wide QRS (ventricular tachycardia) and ventricular fibrillation, and consciousness disappears.
- ■. Ensure that the treatment is conducted according to the shockable rhythm algorithm
- ■. the participant suspects hyperkalemia and confirms the lab results and administers Ca++ agent and NaHCO3, let spontaneous circulation return (ROSC)
- ■. If the participant continues cardiopulmonary resuscitation (CPR) without suspicion of hyperkalemia, K+ value is confirmed at appropriate time (11mEq/L)
- ■. Debriefing points: shockable rhythm + ventricular tachycardia/ventricular fibrillation + reversible causes (5H & 5T)
- 1. McDonald K, Smith CM. The flipped classroom for professional development: part I. Benefits and strategies. Journal of continuing education in nursing. 2013;44(10):437–8. pmid:24098988
- 2. Mehta NB, Hull AL, Young JB, Stoller JK. Just imagine: new paradigms for medical education. Academic medicine: journal of the Association of American Medical Colleges. 2013;88(10):1418–23.
- 3. Ojennus DD. Assessment of learning gains in a flipped biochemistry classroom. Biochemistry and molecular biology education: a bimonthly publication of the International Union of Biochemistry and Molecular Biology. 2016;44(1):20–7.
- 4. Corrias A, Cho Hong JG. Design and implementation of a flipped classroom learning environment in the biomedical engineering context. Conference proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society IEEE Engineering in Medicine and Biology Society Annual Conference. 2015;2015:3985–8.
- 5. Khanova J, McLaughlin JE, Rhoney DH, Roth MT, Harris S. Student Perceptions of a Flipped Pharmacotherapy Course. American journal of pharmaceutical education. 2015;79(9):140. pmid:26839429
- 6. Morgan H, McLean K, Chapman C, Fitzgerald J, Yousuf A, Hammoud M. The flipped classroom for medical students.The clinical teacher. 2015;12(3):155–60. pmid:26009948
- 7. Liebert CA, Lin DT, Mazer LM, Bereknyei S, Lau JN. Effectiveness of the Surgery Core Clerkship Flipped Classroom: a prospective cohort trial. American journal of surgery.2016;211(2):451–7.e1. pmid:26687962
- 8. Fox J, Faber D, Pikarsky S, Zhang C, Riley R, Mechaber A, et al. Development of a Flipped Medical School Dermatology Module. Southern medical journal. 2017;110(5):319–24. pmid:28464171
- 9. Park SE, Howell TH. Implementation of a flipped classroom educational model in a predoctoral dental course.Journal of dental education. 2015;79(5):563–70. pmid:25941150
- 10. Rui Z, Lian-Rui X, Rong-Zheng Y, Jing Z, Xue-Hong W, Chuan Z. Friend or Foe? Flipped Classroom for Undergraduate Electrocardiogram Learning: a Randomized Controlled Study.BMC medical education. 2017;17(1):53. pmid:28270204
- 11. Belfi LM, Bartolotta RJ, Giambrone AE, Davi C, Min RJ. "Flipping" the introductory clerkship in radiology: impact on medical student performance and perceptions.Academic radiology. 2015;22(6):794–801. pmid:25592027
- 12. Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR. The role of assessment in competency-based medical education.Medical teacher. 2010;32(8):676–82. pmid:20662580
- 13. Miller GE. The assessment of clinical skills/competence/performance. Academic medicine: journal of the Association of American Medical Colleges. 1990;65(9 Suppl):S63–7.
- 14. Lew EK. Creating a contemporary clerkship curriculum: the flipped classroom model in emergency medicine.International journal of emergency medicine. 2016;9(1):25. pmid:27623956
- 15. Tan E, Brainard A, Larkin GL. Acceptability of the flipped classroom approach for in-house teaching in emergency medicine. Emergency medicine Australasia: EMA. 2015;27(5):453–9. pmid:26235786
- 16. Rose E, Claudius I, Tabatabai R, Kearl L, Behar S, Jhun P. The Flipped Classroom in Emergency Medicine Using Online Videos with Interpolated Questions.The Journal of emergency medicine. 2016;51(3):284–91.e1. pmid:27381954
- 17. Boysen-Osborn M, Anderson CL, Navarro R, Yanuck J, Strom S, McCoy CE, et al. Flipping the Advanced Cardiac Life Support Classroom with Team-based Learning: Comparison of Cognitive Testing Performance for Medical Students at the University of California, Irvine, United States. Journal of educational evaluation for health professions. 2016;13:11. pmid:26893399
- 18. Rui Zeng, Lian-rui Xiang, Rong-zheng Yue, Jing Zeng, Xue-hong Wan, Chuan Zuo. Friend or Foe?Flipped Classroom for Undergraduate Electrocardiogram Learning: a Randomized Controlled Study. BMC Medical Education 2017;17:53 pmid:28270204
- 19. Makhdoom N, Khoshhal KI, Algaidi S, Heissam K, Zolaly MA. ‘Blended learning’ as an effective teaching and learning strategy in clinical medicine: a comparative cross-sectional university-based study. Journal of Taibah University Medical Sciences. 2013;8(1):12–7.
- 20. Bliuc A-M, Goodyear P, Ellis RA. Research focus and methodological choices in studies into students' experiences of blended learning in higher education.The Internet and Higher Education. 2007;10(4):231–44.
- 21. DeRuisseau LR. The flipped classroom allows for more class time devoted to critical thinking. Advances in physiology education. 2016;40(4):522–8. pmid:28145270
- 22. Chen F, Lui AM, Martinelli SM. A systematic review of the effectiveness of flipped classrooms in medical education.Medical education. 2017;51(6):585–97. pmid:28488303
- 23. Heitz C, Prusakowski M, Willis G, Franck C. Does the Concept of the "Flipped Classroom" Extend to the Emergency Medicine Clinical Clerkship? The western journal of emergency medicine. 2015;16(6):851–5. pmid:26594277
- 24. Lin Y, Zhu Y, Chen C, Wang W, Chen T, Li T, et al. Facing the challenges in ophthalmology clerkship teaching: Is flipped classroom the answer? PloS one. 2017;12(4):e0174829. pmid:28384167
- 25. Cooper AZ, Hsieh G, Kiss JE, Huang GC. Flipping Out: Does the Flipped Classroom Learning Model Work for GME? Journal of graduate medical education. 2017;9(3):392–3. pmid:28638535
- 26. Whillier S, Lystad RP. No differences in grades or level of satisfaction in a flipped classroom for neuroanatomy. The Journal of chiropractic education. 2015;29(2):127–33. pmid:25902472