http://orcid.org/0000-0003-1639-1483
Figures
Abstract
Objectives
We sought to establish the feasibility of chunking crew resource management (CRM) training into micro-size interventions and to compare different training approaches in the context of micro-learning.
Design
We evaluated whether participants in micro-learning CRM activities achieved learning objectives following training. In a between-subjects design, groups were observed for behaviour during a simulation that was part of a 15-minute modular intervention and tested for recollection afterwards.
Participants
The 129 participants recruited for this study were medical students, who already had relevant experience treating patients.
Interventions
The experimental setting involved three 5-minute components: video, simulation, and debriefing. Different groups viewed videos involving different didactic concepts: one group observed a videotaped concrete example of a medical care team applying a CRM tool (example group), and one group observed a videotaped lecture on the same topic (lecture group).
Main outcome measures
All simulations were videotaped and coded in detail for the occurrence of and time spent engaging in team behaviour and medical care. Questionnaires were administered before, immediately after, and two weeks after the intervention. We compared the groups’ behaviour during the simulation (team cooperation and medical care), retention of knowledge from the training content, and results of the evaluation.
Results
Both groups exhibited most of the behaviours included in the content of the instructional videos during the simulations and recollected information 2 weeks later. The example group exhibited significantly more of the training content during the simulation and demonstrated better retention 2 weeks later. Although the example group spent more time on team coordination, there was no difference in the number of executed medical measures.
Conclusion
Delivering CRM training in chunks of relatively short and highly standardised interventions appears feasible. In this study, the form of didactical presentation caused a difference in learning success between groups: a traditional lecture was outperformed by an instructional video demonstrating a practical example.
Citation: Gross B, Rusin L, Kiesewetter J, Zottmann JM, Fischer MR, Prückner S, et al. (2019) Microlearning for patient safety: Crew resource management training in 15-minutes. PLoS ONE 14(3): e0213178. https://doi.org/10.1371/journal.pone.0213178
Editor: Vitomir Kovanovic, University of South Australia, AUSTRALIA
Received: August 23, 2018; Accepted: February 15, 2019; Published: March 7, 2019
Copyright: © 2019 Gross et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The full dataset of video recordings cannot be shared publicly because participants would be identifiable. It was a requirement of the Ethics Committee to not store or share those recordings beyond the immediate study purposes. All relevant data to reproduce the intervention are within the manuscript and its Supporting Information files.
Funding: AZ has received research grants from Klinikum der Universität München, Grant number FöFoLe 576. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Team training is considered a crucial factor for patient safety in healthcare [1]. Crew resource management (CRM) has become a leading example of such training. CRM holds its roots in aviation where it was developed over the last four decades to address failures of interpersonal communication, decision-making, and leadership. Modern CRM involves an array of tools for error management in high-risk environments [2].
Structured briefing models
Representative tools in CRM include structured briefing models, such as SBAR (Situation; Background; Assessment; Recommendation) [3–5] and FOR-DEC (Facts; Options; Risks and Benefits; Decision; Execution; Check) [6]. Such acronyms support communication and decision making, particularly during critical situations. However, they may be perceived as time-consuming in stressful situations and users may tire of using these formal communication processes where an unambiguous procedure exists or a single solution is clearly evident [6]. These potential barriers to acceptance should be considered for healthcare teams, where–unlike aviation crews–staff configurations are more heterogeneous, with a number of subspecialties involved and frequently changing while critical situations may arise multiple times during a typical shift [7]. Tools that are perceived as unwieldy will not gain acceptance under such conditions. Weller et al. developed a structured call-out tool based on SBAR that is individualised for the daily routine of medical teams: Stop; Notify; Assessment; Plan; Priorities; Invite ideas (SNAPPI) [8]. This acronym was designed to improve information-sharing within medical teams during crisis situations and to create a shared understanding of the clinical situation. This call-out tool might be relevant within a wider variety of situations because it is less formal than pre-existing acronyms that stem from an aviation or military context. Weller et al. were able to teach anaesthetists using SNAPPI through a 15 minute video-based educational intervention.
The science of training
More than a decade ago, Salas and colleagues raised several critical concerns regarding the qualification of CRM training faculty, the design of training, the didactical concepts underlying the system, and the embedding of the tools in an organisational context. They questioned the “science of training”, proposing that the way training is designed, delivered, and implemented can greatly influence its effectiveness [9–11]. This critique remains relevant today, highlighting potential shortcomings of CRM in healthcare. However, multiple recent reviews have concluded that the content and characteristics of team training [12,13], including CRM training [14,15], were seldom reported in a reproducible way. These findings indicate that more is known about the “if” than the “how”, in terms of the way training works.
Rather than theoretical knowledge transfer, CRM training typically focuses on equipping professionals with practical skills for working in teams, communicating effectively, and making decisions in real-world critical situations. Learners usually first develop procedural knowledge to understand information about complex system components, states, and their functions. According to the human factors research that underlies aviation CRM, such “schemata” contain sequences of appropriate actions for different types of tasks [16]. For tasks including social interactions, educational psychologists speak of “collaboration scripts” serving as a guide for the distribution of roles and activities among team members [17]. Learners can develop such collaboration scripts and schemata through training and experiences in a simulated environment involving authentic situations that are characteristic of their later professional practice [16,18].
Experiential Learning Theory (ELT) provides a theoretical framework to design experience-based training interventions. This framework conceptualises learning as a recursive process of experiencing, reflecting, thinking, and acting [19]. ELT describes a “cycle of experiential learning” consisting of four modes. The first two modes, Concrete Experience and Abstract Conceptualisation, allow learners to grasp experience, whereas the latter modes, Reflective Observation and Active Experimentation, support learners to convert experience into knowledge. Fig 1 shows a visualisation of the ELT cycle. Accordingly, CRM training can be designed as a spiral of experiences, observations, and reflections which nourish a learner’s pool of scripts and schemata for critical situations.
The figure demonstrates how the three parts of the intervention (1. Video, 2. Simulation, 3. Debriefing) touched all four bases of the ELT cycle.
Challenges to traditional training formats
Applying CRM training in practice involves overcoming the barriers involved in traditional educational formats, which can arise from economic and demographic factors. In a systematic review of the literature regarding CRM training in healthcare, training formats were typically found to be interventions delivered in a traditional attendance-based seminar format extending over 1 day with approximately five to 15 participants [20]. Such training durations of several hours can place a burden on health care institutions that already have to cope with limited staffing levels [21]. Scheduling a whole team for training requires not only a direct investment but also additional opportunity costs related to the necessity to keep facilities operational with a second team [22]. This can result in a conflict of interest between allocating staff to training for patient safety and upholding an appropriate level of service.
However, demographic trends additionally increase the need to adapt academic and vocational education to the changing ways in which medical students and junior staff acquire new skills. A growing cohort of “digital learners” [23] or “new millennium learners” [24] have new habits and expectations regarding education. These learners are reported to prefer receiving information quickly, acquiring knowledge independently from senior staff, utilising multiple sources and digital content, and spending less time in books and more time dealing with medical cases [24,25]. Moreover, students interacting with learning content is now the norm, not the exception. The impact of such new developments in education, in terms of both technology and audience, presents a particular challenge for traditional higher education systems, where lectures may no longer be the only or best way of transferring knowledge [26,27].
Getting the most out of a short intervention
One potentially helpful response to new learning requirements and economic restraints is to chunk the learning process into smaller episodes, skill elements, or “knowledge nuggets”, which involve minimal time consumption and operating expense, and can be part of a modular curricular setting. Such educational concepts have been labelled “microlearning” [28], although no conclusive definition or uniform body of research is currently available. We have adopted the term here, to describe a very concise intervention of teaching a small group of students within a relatively short period of 5 to 15 minutes and then giving and taking feedback on the performances [29]. In management and executive education, learning in “rapid bite-sized chunks” has become a successful product in the corporate training market [30]. In addition, for schools and universities, implications of modularisation in small units of knowledge and chunking of contents are currently receiving attention [27,31].
One example of a standardised and modularised CRM curriculum is the TeamSTEPPS 2.0 program of the US-American Agency for Healthcare Research and Quality [32]. The TeamSTEPPS teaching material includes short videos in which actors demonstrate the application of good teamwork in real-world situations. The advantages of such formats have been advocated by Clay-Williams et al. who designed CRM modules of 2-hour workshops as alternatives to full-day seminars that can be problematic and impractical for the “time-poor clinician”.[33]
Rationale for the current study
The current study was conducted to extend the approach of teaching a CRM tool during a self-contained intervention taking less than 15 minutes, as it has already been established by Weller et al.’s SNAPPI tool and the modules of the TeamSTEPPS program. Thus, the research project sought to further adopt the concept of microlearning to create a modular and lightweight CRM training program substantiated on the Experiential Learning Theory. The intervention concept is designed to be delivered in a standardised module that is largely independent of trainer personality and qualifications, and short enough to be implemented in the daily routine of healthcare organisations.
Specifically, this study focused on “how” to train. When intervention time is restricted to only a short interval to acquire CRM skills, we wanted to examine the importance of the instructional approach for theory input. For this purpose, the performance of an instructional video showing a practical example was compared with a traditional academic lecture.
Research question
To generally demonstrate the feasibility of microlearning for CRM training, we examined if it would be possible to develop content for a 15-minute intervention that would enable learners to achieve high scores with respect to demonstrating learning objectives.
The specific research question for our experiment was: To what extent does the instructional approach during a module (instructional video vs. lecture) affect immediate learning outcomes and retention of knowledge?
To control for possible negative effects of the CRM intervention, we examined if an increased emphasis on social team processes would concur with technical measures of medical care.
Methods
Comparing example-based video content to lecture-based video content involved a between-subjects design. Two experimental groups completed a 15-minute intervention composed of either an instructional video or videotaped lecture, followed by simulation and debriefing. The key difference between groups was the didactical concept of the videos: one group viewed a concrete example of a medical care team applying a CRM tool (example group) whereas another group viewed a videotaped lecture on the same topic (lecture group). See evaluation design section for an overview of the three parts of intervention and the evaluation methods.
Ethics
The human subjects research activities were reviewed and approved by the ethics committee of the medical faculty at LMU Munich (review number 227–16), written consent was obtained from all participants. The individuals depicted in the transcripts of the videos (S1 File) are professional actors (example video) or member of faculty (lecture video) and gave written informed consent (as outlined in PLOS consent form) to publish screenshots of the videos.
CRM Tool
In this study, we developed a CRM tool to deliver distinctive educational content. The tool needed to be unique in concept and wording, meaning that participants could not be familiar with it and biased by prior theoretical input or practical experience. The authors, consisting of a group of experts from medicine (LR), psychology (JK, AZ), educational (JZ) and management sciences (BG), developed a cohesive three-step protocol for team briefings. It was inspired by structured briefing models, such as SBAR and FOR-DEC, as well as Weller et al.’s SNAPPI method [8]. We intended to create an even more lightweight tool compared to the established, but rather formal, acronyms to accommodate the fast-paced and often recurring critical situations healthcare teams face on a daily basis.
We named the proposed CRM tool “Team Check” to provide a distinctive key phrase that would be observable during a simulation. The purpose of the protocol is to first gather and align distributed information in a team, then mutually agree on a plan for the next steps, and finally, to distribute roles and activities among team members. See Table 1 for a description of the Team Check procedure.
A similar communication tool is propagated as “10-for-10 principle” in the medical simulation community [34]. It describes a form of call-out team briefing in order to assess the situation thoroughly and select the correct actions as a process of expert decision-making. See Table 2 for a description of the 10-for-10 principle. In comparison, the Team Check protocol is designed to be more focussed, but it still covers all the same aspects. Both tools aim at improving communication for better teamwork and crisis management [35]. The 10-for-10 principle was therefore selected as intervention for the comparison group in this study.
Intervention
The experimental setting contained three 5-minute components: video, simulation, and debriefing. The intervention was designed to give participants the opportunity to touch all bases of the ELT cycle: watching the video initiates Abstract Conceptualisation and Reflective Observation. Participants were then prompted to learn through Active Experimentation and Concrete Experience during the simulation. Finally, participants experienced Reflective Observation again during the debriefing session (see Fig 1). The whole setting and procedures were pre-tested in nine complete runs to train the faculty and optimise the simulation scenario and timings. The three components of the intervention are described below.
1. Video.
Participants in the example group watched an instructional video showing a healthcare team applying the Team Check procedure. The video was created in the professional style of a documentary television series in the first half, then changed to a more educational style by recapitulating each step of the Team Check protocol to elaborate on its purpose and application. The video ended with short interviews with the actors describing the benefits of the Team Check protocol and why it is useful in their particular role. The actors illustrated different role models: An experienced female physician as the team leader, a senior male nurse, and a female medical student.
The lecture group watched a video of the same length showing a recorded lecture about the danger of uncoordinated teamwork during critical situations in healthcare. To avoid these deficits, the “10-for-10 principle” was presented to this group as team briefing tool. The codeword “Stop” was introduced to initiate a step-back method for all team members to reassess the situation and bundle information. See supplementary material (S1 File) for a transcript of the videos. To avoid bias in the lecture group, the lecturer was selected from the faculty of our university hospital, who regularly gives human factors trainings and is well-rated in evaluations. The lecturer was allowed 5 weeks to prepare before the videotaping, and gave a typical presentation that is part of the CRM education curriculum for medical students at the university hospital.
Although the videos for both groups contained similar theoretical content, the video viewed by the example group involved dramaturgy designed to address the ELT areas of Abstract Conceptualisation and Reflective Observation, whereas the lecture group received a traditional lecture-based form of abstract transfer of knowledge.
2. Simulation.
After watching the video, participants of both groups underwent an identical simulated scenario of a standard situation in an emergency ward. In this simulation, a patient was admitted by ambulance after syncope of unclear origin. Participants were assigned to different roles, as described below. All participants were instructed as medical students assisting in a particular department as part of their education. One was from the surgical program, one was from the internal department of the hospital, and the third participant was briefed to assist in the ambulance service. All participants were given typical working clothes and provided with a folder of documents. The simulation was arranged so that each participant found documentation in their folder, indicating that their department had previously seen the patient. The ambulance role had the paramedic protocol for the day, and the internal and surgical roles each had medical records from their department at which the patient was previously seen, but on different days and with different diagnostic results.
Before the simulation began, all participants were briefly familiarised with the setting, equipment, and simulation mannequin (Sim Man 3G, Leardal, Stavanger, Norway). Participants had access to an array of medical devices that are typical for an examination room, including a stethoscope, blood pressure cuff, diagnostic penlight, pulse oximeter, NaCl infusion, and oxygen masks. An electrocardiogram (ECG) monitor was always attached to the patient and running.
The simulation followed a predefined sequence, as described in Fig 2. To create a situation of distributed information involving the need for team processes, participants were brought into the scenario with staggered timing. The medical conditions were arranged so that a critical emergency never occurred (e.g., resuscitation). The patient was awake at all times and responded to the participants. Nonetheless, the situation demanded a senior physician’s attention, and amongst all the available measures of medical care, the application of supplemental oxygen and a call for assistance (“call for help”) represent the essential and urgent actions in the situation.
The Flowchart demonstrates the scenarios’ standardised course of events. Patient conditions and external events followed a predefined schedule, participants entered the scenario with staggered timings. The effective timings as surveyed during the simulations (n = 41) are reported as mean (M) and standard deviation (SD).
To avoid a bias, the scenario had substantial differences to the example situation presented in the Team Check video. It was designed to have no similarities either in team roles or in clinical details: during the simulation the patient was awake and able to respond at all times, showing a critical change in vital parameters, being admitted in this moment (with ambulance staff still present), anamnesis and documented information available but shared amongst departments, and a hierarchy-free team of juniors form internal and chirurgical department and ambulance service. The Team Check video in contrast featured a team constellation of a resident, a senior nurse and a student, while the patient was admitted several hours ago, with unknown anamnesis, being somnolent and not responsive, with advanced diagnostic processes already started.
3. Debriefing.
After the simulation, participants left the simulator and were invited to a debriefing in the next room. Following debriefing guidelines, participants were asked about their experience during the simulation and what strategies helped them cope with challenges, using open questions. If no keywords from the video were mentioned (like “Team Check” for the example group and respectively “10 for 10” or “Stop” for the lecture group), it was explicitly asked if the video was helpful. All participants were complimented for their performance in the simulation and were given the chance to ask questions, so that all participants were released with positive feedback.
Recruiting strategy
Participants were recruited via newsletter and social media from medical students at an advanced stage of their degree at two local universities. Inclusion criteria were that the students had already completed their preliminary medical examination or gathered relevant experience treating patients during an earlier occupation (e.g., working as nurse or paramedic). This group of participants was able to be immersed in the clinical scenario but unlikely to have already been exposed to CRM or leadership training, rather than residents or practicing physicians. Participants were randomly invited in groups of three which were alternately assigned to A or B intervention.
Evaluation design
All simulations were videotaped and coded afterwards in terms of team cooperation and medical care. Details of video recordings processing are reported in the data analysis section below. Additional data were collected with questionnaires at three points in time: before participants saw the video (t0), after the debriefing (t1), and 2 weeks later (t2). The t0 questionnaire contained demographic items and asked for prior experiences. In the t1 questionnaire, participants evaluated the training on a 6-point Likert scale (items depicted in S2 File) and were asked to note the video elements they remembered in a free text field. With an offset of two weeks after the intervention, participants were sent a link to an online questionnaire via e-mail to observe the sustainability of knowledge acquired during the intervention. This t2 questionnaire was again asking to note elements from the intervention in free text fields and querying if the participant was able to apply some of the training content in practice. See Table 3 for the data collected at different points in time.
Data analysis
To investigate the effects of interventions on group interactions during the simulations, all video recordings were analysed using behavioural observation coding. The videos were coded using MAXQDA [36] following the behaviour coding scheme (see Table 4). The coders were blinded to the video that the participants watched before the simulation. Coders saw recordings of the simulations, including a display of the patient’s current vital parameters. Team processes (e.g., calling a team briefing), measures of medical care (e.g., checking pupil reflexes), and events during the scenario itself (e.g., changes in vital parameters or phone calls) were marked and coded by noting a time interval or time stamp in the videos with a resolution of seconds. If behaviours occurred simultaneously (e.g., one team member applying oxygen mask or blood pressure cuff while others exchanging information in the team) each of the behaviours was marked separately in an overlapping way. See Table 4 for the full coding scheme.
Inter-rater reliability was validated for 10% of the sample (n = 5 videos) with a result of a Cohen’s κ = .82 for attributed behaviours and an intra-class correlation (two-way mixed, absolute agreement, single measures) of .99 [CI95: .99–1] for starting times and .82 [CI95: .75-.87] for durations, indicating high levels of agreement. Codes were clustered into the categories “team behaviour”, “medical care” and “simulation” for events that were descriptive for the simulated scenario. The coding scheme is provided in table 4.
The post-test questionnaires contained free-text fields to assess which of the training contents the participants remembered. Answers were mostly formulated as keywords; one researcher counted them when matching to the team behavioural codes. Video coding data and the questionnaires (see Table 3) were consolidated using an Oracle database and analysed using R [37].
To test our main research question, the study groups were compared regarding the occurrence of team behaviour using χ2 tests, i.e., if a group engaged in planning and prioritising. Time spent engaging in team behaviours was compared using t-tests for non-paired group differences. Group differences concerning training evaluations were tested using Mann-Whitney-U tests. Recall of training content was tested using a mixed-design analysis of variance. To control for interference between team processes and medical care, group differences were compared respectively. The alpha level was set to p = .05 for all statistical tests. To correct for multiple testing, Bonferroni-Holm correction was conducted within the tested subsets.
Results
Participants
A total of 129 participants were recruited for this study. Six participants were excluded from the analysis because they were students who filled in for participants who did not show up for their session. These students were from the same cohort as the other participants but were affiliated with the simulation centre. They were instructed about standard behaviour in the ambulance role, because this team member could go to a passive role in the simulated scenario as a realistic behaviour, and evenly distributed between example group (n = 3) and lecture group (n = 3). Another complete group of three participants had to be excluded because one team member revealed during the simulation that she was not a medical student, potentially biasing group behaviour. This resulted in a final study sample of 120 participants. The participants’ demographics are shown in Table 5.
Group comparison
The groups were compared on the following measures: behaviour during the simulation (team cooperation and medical care), retention of knowledge from training contents, and results of the evaluation. Means, standard deviations, and the results of the group comparisons are shown in Table 6.
The groups did not differ regarding previous professional experience (see Table 5). We also surveyed participants regarding previous team trainings with focus on team procedures they had participated in. There were also no group differences (chi2(1) = 0.08, p = .774): For the example group, 49% reported partaking in such a training prior to our intervention and for the lecture group, 55% reported experiencing a similar training. Previous team training experience and previous professional experience were not associated with any of the independent variables we measured.
1. Behaviour during the simulation.
Both groups engaged in team processes (i.e., exchanging information, planning and prioritising tasks) at least once during the simulations. On average, the teams applied 3.14 out of 4 (SD = 0.75) aspects of team cooperation that were taught in both video instruction conditions. However, as shown in Table 7, more teams in the example group tended to show team behaviours characterising the content of the CRM tool. The difference was statistically significant for the explicit call-out of the team briefing (in the example group “Team Check”, in the lecture group the “Stop” word). The example Group A showed an average of 3.5 of the 4 aspects of team coordination that were taught to both groups, while the lecture group showed 2.75 aspects; this difference was statistically significant (T [39.9] = 3.71, p = .001, d = 1.5).
The groups did not significantly differ in terms of the number of medical measures that were conducted, time spent talking to the patient, time of the first supply of supplemental oxygen, or time of the first call for help (see Table 6). Comparing the application of medical measures separately also revealed no difference between groups (see Table 7).
Comparison of how the groups allocated their time in the simulator revealed that the example group spent significantly more time coordinating the team overall. With 135 seconds they devoted almost twice as much time exchanging information, planning and prioritising compared with the 77 seconds the lecture group spent for team coordination. The cumulative amounts of time coded during the simulations are shown in Fig 3, grouped for medical care and team coordination. For the example group, the sum of the coded time intervals was overall higher than for the lecture group (t[32] = 2.30, p = .03, d = 0.70). This can be observed as all aspects of the groups’ behaviour were coded separately and overlapping if activities occurred at the same time (see data analysis section).
The example group spent almost twice as much time coordinating the team compared with the lecture group. The example group also cumulated more time intervals of actions and team behaviours.
2. Retention of knowledge from training content.
Table 6 demonstrates how participants remembered the training content at t1 and t2. Directly after training, the example group remembered significantly more elements of the CRM tool that were presented in the videos than the lecture group. The effect was even larger 2 weeks after the training. Both groups tended to recall less after this time (F[1] = 10.91, p = .001, ηpartial = .12 for the main effect; F[1] = 1.04, p = .312, ηpartial = .01 for the interaction).
3. Evaluation of the training.
Both groups rated the scenario as relevant for practice (85% of participants rated as 5 or 6 points on the 6-point Likert scale) and realistic (67%), and the training content was considered useful for daily routines (53%). Table 6 shows a comparison of participants’ ratings of the videos. Members of the example group perceived the video as more helpful.
During the survey 2 weeks after training, participants were asked if they were able to apply techniques they were taught during the training. In the example group, 10 of 47 (21%) members answered this question with “yes”, compared with five of 51 (10%) members of the lecture group. This difference was not statistically significant (χ2 [1] = 2.48, p = .115).
Discussion
Both groups exhibited most of the behaviours included in the instructional videos during the simulations and were able to recollect them 2 weeks later. This could be descriptive evidence of the feasibility of microlearning in this context. Participants generally considered the intervention realistic and helpful.
Observer coding of behaviour in the simulator provided a detailed analysis method, enabling us to closely examine how groups allocated their collective time during a critical situation. In addition, the between-groups design effectively demonstrated the differences caused by changing the training method. The current findings highlight the importance of “how” to train in the context of CRM, identifying potentially useful areas for optimisation of educational programs.
The example group, who watched the Team Check video, demonstrated significantly more of the trained content during the simulation. This group had better retention two weeks after the intervention. They also reported that the tool was more helpful. Although the example group spent more time on team coordination, the teams did not significantly differ in the number of executed medical measures. Thus, in the current study, the didactic concept of the instruction caused a difference between the learning success of the groups.
Also, the example group tended to initiate the first team briefing earlier (defined as the first exchange of information after the handover) and to spend more time delegating tasks. Both of these mean differences showed small to medium effect sizes but did not reach a level of statistical significance. The lecture group spent significantly more time on medical care. They also tended to spend more time talking to the patient. Even if these differences did not reach statistical significance, they showed medium to large effect sizes (see Table 6) and should be validated in future studies with larger samples.
The results indicate that a discrete CRM tool similar to a structured team briefing can be learned in a 15-minute intervention. Thus, it appears feasible to design a training intervention encompassing all components of ELT in this short time span. Participants demonstrated newly-acquired competencies in the simulation and recalled them several weeks later. In addition, participants reported the format to be helpful. These findings are comparable with those of Weller et al., who evaluated the SNAPPI call-out method using simulations 4–6 weeks after participants saw a short video instruction [8].
To what extent does the instructional approach during a module affect immediate learning outcomes and retention of knowledge?
The example group had better retention of training contents immediately after and 2 weeks after the intervention. This group also considered the tool to be more helpful. Although both groups had the same exposure time, the results revealed that the instructional video showing an example of good CRM practice was significantly more effective regarding knowledge retention, but also with regards to its immediate impact on participants’ behaviour during the simulation scenario than a traditional lecture on the same topic. The potential of applying psychological and educational expertise to the design of academic lessons, even for very theoretical subjects, was demonstrated in a previous study of students in a physics class, who performed significantly better when taught in small groups and interactive courses, compared with a traditional lecture [26].
Medical care in its application and especially communication and decision making in critical situations are practical tasks that require procedural knowledge as opposed to academic considerations and conceptual (theoretical) knowledge [38]. The interventions for our two groups contained the same topics but differed in how a lecture versus role players framed content into a concrete medical setting. In a similar vein, Semler et al. demonstrated that showing a 12-minute video of a simulated emergency, exemplary managed by faculty staff, was similarly effective to participating in a simulation and more effective than being presented teamwork principles by traditional didactic slide presentation [39]. Our example group received both a concrete example in a medical context and a simulation experience. The results can be interpreted that specificity of content and training opportunity might be a key factor to acquiring procedural knowledge for teamwork situations.
Does an increased emphasis on team processes concur with measures of medical care?
We found a clear difference in how the groups allocated their time during the simulation. The example group spent more time on team coordination processes, such as exchanging information and planning actions, and less time on actual measures of medical care with the patient. Among all possible measures in the simulator, the application of supplemental oxygen and call for assistance by a senior physician (“call for help”) were the essential and urgent actions in the given situation. We observed no difference between the groups in those crucial actions.
Importantly, an increased focus on team processes must not compete with practical measures of care. When cumulating the times spent on team coordination and patient treatment, the example group spent more time engaged in both of these activities. As the example group coordinated their team while simultaneously preparing the patient treatment, they were able to undertake more activities in the same time, compared with the lecture group. Similar observations have previously been reported by studies of the clinical performance of teams during critical events [40,41] and research on CRM training from a learning perspective [42].
Limitations
One weakness of the study is related to the recruitment of medical students as a study group. Because the cohort of students dispersed after the intervention, a longitudinal study of the personal and organisational impact of training was not possible. However, the current study sample was chosen deliberately to avoid spill-over effects from previous team trainings or leadership experience.
Furthermore, the study was designed as an A/B comparison without a direct control group. This design avoided ethical concerns that may have arisen from exposing participants to a critical situation without any preparation, or with irrelevant instructions. Also, the two videos differed in more than one way. Team Check and 10-for-10 are similar in content but put in different tools. Additionally, one was presented as example video, the other as lecture. As the training participants received during this study was focused on only one aspect of teamwork, we are limited in our ability to generalize whether the simulation behavior differences would extend to more diverse clinical scenarios as well.
Third, this study design is unable to answer the question to what extent CRM-like behaviour might be natural behaviour in a social group that participants might had exhibited during the simulation even without a prior intervention. The team behaviour “Asking for consent” could be considered as control variable in this regard. This aspect was not part of the lecture group’s video but was included in the Team Check protocol. Only 5% of the lecture group teams demonstrated this behaviour during the simulation by themselves, while 32% of the Team Check teams exhibited this behaviour after they saw it in the instructional video (see Table 7). This group difference reached a medium effect size but did not reach statistical significance due to the small sample size. Even though groups did not differ regarding experiences with team trainings or relevant professional experience, we cannot say with certainty to what extent the tested behaviour would have occurred regardless of the training.
Meaning of the study: Possible implications for clinicians and policymakers
We adopted the concept of microlearning to describe a relatively short and standardised intervention. The current results support an idea worth further investigation: by modularising CRM training content into smaller chunks and using professional instructional videos, a curriculum can be decomposed into increments of a manageable size. Lightweight incremental training modules could ameliorate the effects of understaffing in hospitals when used to blend learning into daily staff routines, as microlearning portions.
Finally, the current study contributes to the “science of training” in CRM and patient safety. The differences we observed between the effectiveness of an academic lecture compared with an action-oriented instructional video indicate further potential for enhancing the impact of training.
Unanswered questions and future research
The current study did not utilise the full potential of microlearning. Smartphone apps, as well as interactive and self-directed content in modern e-learning should be embraced to foster the educational landscape in patient safety. Further experiments and developments in this field are necessary.
It should be noted that the ELT cycle operates as a spiral, in the sense of a recurring process. The current study presented a single run through the ELT cycle at best. However, a cascading training design involving multiple modules can be expected to have further benefits. It may be useful for future research to evaluate a curriculum of multiple incremental training interventions.
Given that all participants in the current study were medical students, it is important to evaluate the degree to which our findings can be translated to the education of experienced healthcare staff. In addition, the organisational context of the hospital in which such CRM interventions are to be embedded should be investigated to examine the potential influence on training success.
Supporting information
S1 File. Video interventions.
Transcript of the instructional video (example group) and videotaped lecture (lecture group).
https://doi.org/10.1371/journal.pone.0213178.s001
(DOCX)
S2 File. Survey questions.
Questions and answering options administered at t0, t1,and t2.
https://doi.org/10.1371/journal.pone.0213178.s002
(PDF)
Acknowledgments
We are grateful to Carolin Braun, M.A., who edited a draft of the manuscript and was an essential source of support during all phases our research. We also thank Julia Reichling, MSc., and Stefanie Menzel, BSc., who assisted in organising the experiment and collecting data, as well as Benjamin Knight, MSc., for editing a draft of this manuscript. The authors would like to thank the reviewers of the manuscript who have provided us with concise suggestions on how to improve our work.
References
- 1.
Patient safety curriculum guide: multi-professional edition. Geneva: World Health Organization; 2011.
- 2. Helmreich RL, Merritt AC, Wilhelm JA. The Evolution of Crew Resource Management Training in Commercial Aviation. Int J Aviat Psychol. 1999;9: 19–32. pmid:11541445
- 3. Leonard M, Graham S, Bonacum D. The human factor: the critical importance of effective teamwork and communication in providing safe care. Qual Saf Health Care. 2004;13 Suppl 1: i85–90. pmid:15465961
- 4. De Meester K, Verspuy M, Monsieurs KG, Van Bogaert P. SBAR improves nurse–physician communication and reduces unexpected death: A pre and post intervention study. Resuscitation. 2013;84: 1192–1196. pmid:23537699
- 5. Haig KM, Sutton S, Whittington J. SBAR: A Shared Mental Model for Improving Communication Between Clinicians. Jt Comm J Qual Patient Saf. 2006;32: 167–175. pmid:16617948
- 6. Soll H, Proske S, Hofinger G, Steinhardt G. Decision-Making Tools for Aeronautical Teams: FOR-DEC and Beyond. Aviat Psychol Appl Hum Factors. 2016;6: 101–112.
- 7. Kapur N, Parand A, Soukup T, Reader T, Sevdalis N. Aviation and healthcare: a comparative review with implications for patient safety. JRSM Open. 2016;7: 205427041561654. pmid:26770817
- 8. Weller JM, Torrie J, Boyd M, Frengley R, Garden A, Ng WL, et al. Improving team information sharing with a structured call-out in anaesthetic emergencies: a randomized controlled trial. Br J Anaesth. 2014;112: 1042–1049. pmid:24561645
- 9. Salas E, Wilson KA, Burke CS, Bowers CA. Myths about crew resource management training. Ergon Des Q Hum Factors Appl. 2002;10: 20–24.
- 10. Salas E, Wilson KA, Burke CS, Wightman DC. Does crew resource management training work? An update, an extension, and some critical needs. Hum Factors J Hum Factors Ergon Soc. 2006;48: 392–412.
- 11. Salas E, Tannenbaum SI, Kraiger K, Smith-Jentsch KA. The Science of Training and Development in Organizations: What Matters in Practice. Psychol Sci Public Interest. 2012;13: 74–101. pmid:26173283
- 12. Weaver SJ, Lyons R, DiazGranados D, Rosen MA, Salas E, Oglesby J, et al. The Anatomy of Health Care Team Training and the State of Practice: A Critical Review: Acad Med. 2010;85: 1746–1760. pmid:20841989
- 13. Hughes AM, Gregory ME, Joseph DL, Sonesh SC, Marlow SL, Lacerenza CN, et al. Saving Lives: A Meta-Analysis of Team Training in Healthcare. J Appl Psychol. 2016; pmid:27599089
- 14. O’Dea A, O’Connor P, Keogh I. A meta-analysis of the effectiveness of crew resource management training in acute care domains. Postgrad Med J. 2014;90: 699–708. pmid:25370080
- 15. Zeltser MV, Nash DB. Approaching the Evidence Basis for Aviation-Derived Teamwork Training in Medicine. Am J Med Qual. 2010;25: 13–23. pmid:19801422
- 16. Endsley MR. Toward a Theory of Situation Awareness in Dynamic Systems. Hum Factors J Hum Factors Ergon Soc. 1995;37: 32–64.
- 17. Fischer F, Kollar I, Stegmann K, Wecker C. Toward a Script Theory of Guidance in Computer-Supported Collaborative Learning. Educ Psychol. 2013;48: 56–66. pmid:23378679
- 18. Kiesewetter J, Kollar I, Fernandez N, Lubarsky S, Kiessling C, Fischer MR, et al. Crossing boundaries in interprofessional education: A call for instructional integration of two script concepts. J Interprof Care. 2016;30: 689–692. pmid:27309966
- 19.
Kolb AY, Kolb DA. Experiential Learning Theory: A Dynamic, Holistic Approach to Management Learning, Education and Development. The SAGE Handbook of Management Learning, Education and Development. 1 Oliver’s Yard, 55 City Road, London EC1Y 1SP United Kingdom: SAGE Publications Ltd; 2009. pp. 42–68. https://doi.org/10.4135/9780857021038.n3
- 20. Gross B, Rusin L, Kiesewetter J, Zottmann J, Fischer MR, Prückner S, et al. Crew resource management training in healthcare: a systematic review of intervention design, training conditions and evaluation. BMJ Open. 2019;
- 21. Aiken LH, Sloane DM, Bruyneel L, Van den Heede K, Griffiths P, Busse R, et al. Nurse staffing and education and hospital mortality in nine European countries: a retrospective observational study. The Lancet. 2014;383: 1824–1830. pmid:24581683
- 22. Rampel T, Gross B, Zech A, Prückner S. Simulation centres in German hospitals and their organisational aspects: Expert survey on drivers and obstacles. GMS J Med Educ. 2018;35. pmid:30186950
- 23. Gallardo-Echenique EE, Marqués-Molías L, Bullen M, Strijbos J-W. Let’s talk about digital learners in the digital era. Int Rev Res Open Distrib Learn. 2015;16.
- 24.
Pedro F. New Millennium learners in higher education: Evidence and policy implications. International Conference on 21st Century Competencies, Brussels: OECD/CERI. 2009.
- 25. Boomers Oblinger D., Gen-Xers and Millennials. Understanding the New Students. Educ Rev. 2003;38: 36–47.
- 26. Deslauriers Louis, Schelew Ellen, Wieman Carl. Improved Learning in a Large-Enrollment Physics Class. Sci New Ser. 2011;332: 862–864.
- 27. Bridges D. Back to the Future: The higher education curriculum in the 21st century. Camb J Educ. 2000;30: 37–55.
- 28.
Hug T. Didactics of microlearning: concepts, discourses and examples. Münster, Germany: Waxmann; 2007.
- 29.
Hug T. Microlearning: A New Pedagogical Challenge. In: Hug T, Lindner M, Bruck PA, editors. Microlearning: emerging concepts, practices and technologies; proceedings of microlearning 2005; learning & working in new media environments. 1. ed. Innsbruck: Univ. Press; 2006. pp. 7–11.
- 30. Armstrong Steven J., Eugene Sadler-Smith. Learning on Demand, at Your Own Pace, in Rapid Bite-Sized Chunks: The Future Shape of Management Development? Acad Manag Learn Educ. 2008;7: 571–.
- 31. Gobet F. Chunking models of expertise: implications for education. Appl Cogn Psychol. 2005;19: 183–204.
- 32.
TeamSTEPPS 2.0 Course Management Guide. In: AHRQ Website [Internet]. Sep 2012 [cited 21 Mar 2018]. Available: http://www.ahrq.gov/teamstepps/instructor/essentials/coursemgmt.html
- 33. Clay-Williams R, Greenfield D, Stone J, Braithwaite J. On a Wing and a Prayer: An Assessment of Modularized Crew Resource Management Training for Health Care Professionals: J Contin Educ Health Prof. 2014;34: 56–67. pmid:24648364
- 34. Rall M, Glavin R, Flin R. The ‘10-seconds-for-10-minutes principle.’ Bull R Coll Anaesth. 51: 2614–2616.
- 35. Brindley PG, Reynolds SF. Improving verbal communication in critical care medicine. J Crit Care. 2011;26: 155–159. pmid:21482347
- 36.
VERBI. MAXQDA [Internet]. Berlin, Germany: VERBI Software. Consult. Sozialforschung GmbH; Available: www.maxqda.de
- 37.
R Core Team. R: A language and environment for statistical computing. [Internet]. Vienna, Austria: R Foundation for Statistical Computing; 2017. Available: www.R-project.org
- 38. Krathwohl DR. A Revision of Bloom’s Taxonomy: An Overview. Theory Pract. 2002;41: 212–218.
- 39. Semler MW, Keriwala RD, Clune JK, Rice TW, Pugh ME, Wheeler AP, et al. A Randomized Trial Comparing Didactics, Demonstration, and Simulation for Teaching Teamwork to Medical Residents. Ann Am Thorac Soc. 2015;12: 512–519. pmid:25730661
- 40. Manser T, Harrison TK, Gaba DM, Howard SK. Coordination Patterns Related to High Clinical Performance in a Simulated Anesthetic Crisis: Anesth Analg. 2009;108: 1606–1615. pmid:19372344
- 41. Burtscher MJ, Manser T, Kolbe M, Grote G, Grande B, Spahn DR, et al. Adaptation in anaesthesia team coordination in response to a simulated critical event and its relationship to clinical performance. Br J Anaesth. 2011;106: 801–806. pmid:21429954
- 42. Zottmann JM, Dieckmann P, Taraszow T, Rall M, Fischer F. Just watching is not enough: Fostering simulation-based learning with collaboration scripts. GMS J Med Educ. 2018;35.