Describe brief (less than half a day) interventions aimed at improving healthcare team functioning.
A systematic review on brief team interventions aimed at role clarification and team functioning (PROSPERO Registration Number: CRD42018088922). Experimental or quasi-experimental studies were included. Database searches included CINAHL, Medline, EMBASE, PUBMED, Cochrane, RCT Registry-1990 to April 2020 and grey literature. Articles were screened independently by teams of two reviewers. Risk of bias was assessed. Data from the retained articles were extracted by one reviewer and checked by a second reviewer independently. A narrative synthesis was undertaken.
Searches yielded 1928 unique records. Final sample contained twenty papers describing 19 studies, published between 2009 and 2020. Studies described brief training interventions conducted in acute care in-patient settings and included a total of 6338 participants. Participants’ socio-demographic information was not routinely reported. Studies met between two to six of the eight risk of bias criteria. Interventions included simulations for technical skills, structured communications and speaking up for non-technical skills and debriefing. Debriefing sessions generally lasted between five to 10 minutes. Debriefing sessions reflected key content areas but it was not always possible to determine the influence of the debriefing session on participants’ learning because of the limited information reported.
Interest in short team interventions is recent. Single two-hour sessions appear to improve technical skills. Three to four 30- to 60-minute training sessions spread out over several weeks with structured facilitation and debriefing appear to improve non-technical skills. Monthly meetings appear to sustain change over time.
Short team interventions show promise to improve team functioning. Effectiveness of interventions in primary care and the inclusion of patients and families needs to be examined. Primary care teams are structured differently than teams in acute care and they may have different priorities.
Citation: Kilpatrick K, Paquette L, Jabbour M, Tchouaket E, Fernandez N, Al Hakim G, et al. (2020) Systematic review of the characteristics of brief team interventions to clarify roles and improve functioning in healthcare teams. PLoS ONE 15(6): e0234416. https://doi.org/10.1371/journal.pone.0234416
Editor: César Leal-Costa, Murcia University, SPAIN
Received: March 18, 2020; Accepted: May 25, 2020; Published: June 10, 2020
Copyright: © 2020 Kilpatrick et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: KK, LP, MJ, ET, NF, VL, NG, MDB, CAD received the award from Canadian Institutes of Health Research Program: Project Grant Competition: 201603PJT Application Number: 363528 https://cihr-irsc.gc.ca/e/193.html The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
There is a growing interest worldwide to understand how to improve team functioning and team performance in healthcare settings . Poor team functioning has been identified as a critical factor of adverse events of patient safety . Globally, four out of 10 patients in primary and ambulatory care are harmed  134 million adverse events occur in hospitals contributing to 2.6 million deaths; and medication errors cost an estimated 42 billion USD annually . In Canada, it is estimated that preventable patient safety incidents occur every minute and 18 seconds . Several national and international reports [6–9] highlight that improved team functioning lead to better outcomes for patients, providers and healthcare systems.
In their seminal review, Cohen and Bailey  defined a team as a group of two or more people, who are interdependent in their respective tasks and share common goals and responsibilities for results. Team functioning is influenced by processes that included decision-making, communication, cohesion, care coordination, problem-solving and focus on patients and families . Mathieu et al. (2019)  updated their review of the team effectiveness literature in organizational research conducted in the last 10 years. They identified 29 meta-analyses including 30 structural and process factors that predicted team effectiveness . They argued that team effectiveness is a multi-dimensional and complex construct where effective teams navigate between different structures, mediating mechanisms (e.g., processes), and external influences to efficiently produce tangible outputs that are high quality . In healthcare, teams rely on the contribution of many professionals with different expertise to meet the increasingly complex needs of the population [13–15]. Team training is seen as essential to improve team performance [16, 17].
Role clarity between providers has been identified as an important factor to improve team functioning [16, 18, 19]. The lack of role clarity, lack of understanding of the boundaries between roles, and poorly defined scope of practice can jeopardize teamwork [20, 21]. Such problems are particularly salient given the context of healthcare reforms and system restructuring . According to Hudson et al. , role understanding is an integral part of teamwork because it generates trust and mutual respect. Greater understanding of others’ roles in the team promotes role clarity to foster optimal utilization of all professional roles and improve patient outcomes and health system cost-effectiveness . Hence, role clarity is key to effective team training interventions.
Teams are active learning systems where individuals develop relationships and apply knowledge to solve problems . McEwan et al. (2017)  completed a systematic review and meta-analysis of teamwork training and interventions (n = 51). These authors identified four types of interventions for teamwork including didactic lectures/presentations, workshops, simulations, and on-site review activities. McEwan et al. (2017)  determined that teamwork interventions exerted a moderate effect on teamwork and team performance. However, approximately two-thirds of the teams identified by McEwan et al. (2017)  were outside of healthcare and included academia and experimental laboratory research.
Marlow et al. (2017)  completed a systematic review to examine team training interventions in healthcare (n = 197) and found that team training included a variety of training methods to address the needs of a wide range of care providers. The most frequent interventions identified in the review centered on improving team processes such as teamwork, awareness of the environment, leadership, shared understanding, decision-making, communication, coordination and team role knowledge. These researchers did not identify interventions lasting less than one day.
Team-based interventions where members are engaged are more effective . In addition, interventions are more effective if they target several dimensions of teamwork simultaneously and are specific to the setting . Sidani and Braden (2011)  defined interventions as rational actions and interrelated behaviours directed toward addressing a specific aspect of a problem to achieve a common goal . These authors highlighted that interventions vary in their level of complexity from simple to complex. Complex interventions are made of several components and interrelated parts [27, 28]. When examining interventions, researchers [1, 27] have noted key characteristics to consider included the dose (e.g., duration, frequency), mode of delivery (e.g., written, verbal), and type of intervention.
As indicated above, there is consensus in the literature on the dynamic nature of healthcare teams, their contributions to quality of care and how longer team interventions can improve team functioning. However, as clinical loads continue to increase, due to greater complexity of health problems, ageing population and severe limitations imposed on resources, longer team training is less and less attractive. There is thus a growing need to envision short term interventions that can provide needed support and have an impact on team performance. Our team aims to address this gap in our understanding and describe the characteristics of brief (less than half a day) team interventions that contribute to improving team functioning.
Materials and methods
We conducted a systematic review to describe the characteristics of brief team interventions to clarify roles and improve functioning in healthcare teams.
The research targeted experimental or quasi-experimental studies published or pre-published between January 1990 and April 2020. The databases explored included CINAHL, Medline, EMBASE, PUBMED, Cochrane, RCT Registry-1990 to April 2020. Records were retrieved on April 21st 2020. A search for existing systematic reviews in the Cochrane Database and Prospero Registry was conducted. The gray literature was explored using the strategies proposed in Grey Matters (2014) , notably via the ProQuest, GraySource Index and Google Scholar databases. Searches were also conducted to find abstracts or conference proceedings and pre-publications. In addition, the reference lists of selected papers were examined to identify additional studies. We worked with an academic librarian to develop and validate the search strategy and identify keywords for each database. Search strategies are provided in the Appendix. No language restriction was applied.
Inclusion and exclusion criteria
We included randomized controlled trials (RCTs), experimental and quasi-experimental designs because we were looking to identify the characteristics of team interventions that were known to be effective. We retained systematic reviews (with or without meta-analysis) to conduct a hand search of the reference lists. An expanded search to include other research designs (e.g., observational study) was not necessary given the number of studies that were identified.
We included all studies where the intervention lasted less than a half day or 4.5 hours using experimental and quasi-experimental designs. We included teams in different contexts, within and outside of healthcare. Interventions developed for healthcare teams could be in primary and acute care, and include providers such as physicians, medical specialists, nurses, nurse practitioners, nurse clinicians, nursing assistants, licensed practical nurses, social workers, physiotherapists, occupational therapists, pharmacists, support personnel (e.g., secretaries, clerks), and patients and families. Primary care was defined as comprehensive healthcare services for common health concerns at the point of entry to the healthcare system . Acute care included in-hospital or specialized ambulatory care .
We excluded studies where the intervention lasted more than a half day or 4.5 hours. The primary aim of the review was to identify effective short team interventions. As proposed by Higgins et al. (2019), we excluded observational and longitudinal studies and as well as qualitative methodologies as these studies are at increased risk of bias .
We retained interventions that influenced team functioning or team processes. Interventions could be geared to different members of the healthcare team, patients, families, managers or support staff. Data were extracted to determine key characteristics of the interventions including setting, duration, type of intervention, frequency and sequence of activities. Comparators and control conditions included no intervention or the usual functioning of the team.
The Cochrane handbook for systematic reviews of interventions served as a guide for this systematic review . A review protocol was developed and published with PROSPERO (Registration Number: CRD42018088922) . Training sessions were conducted with all assessors (n = 8) to review inclusion and exclusion criteria, the screening instrument and answer questions. All the publications identified following the application of the search strategy were uploaded into the Endnote reference management software and duplicates were removed. Subsequently, titles and abstracts, if available, were reviewed independently by two reviewers using the RAYYAN web application to exclude articles that were not relevant considering the inclusion criteria . Full texts were reviewed if abstracts were not available.
Full-text review was undertaken for articles that met the inclusion criteria. Reviewers independently assessed if they met the inclusion criteria and a final decision was made about their inclusion in the systematic review. A third researcher (KK) acted as a tie-breaker in case of disagreement between reviewers. A PRISMA flow chart was generated to demonstrate the steps for selecting studies and document the reasons for exclusion . Assessor agreement on all inclusions and exclusions was 90.5%. Using Cohen’s kappa, we obtained substantial inter-rater agreement at 61% across the eight assessors paired two by two .
The data extraction form was developed using the recommendations of Kennedy et al. (2019) , and pilot-tested with extractors. Data from the retained articles were extracted by one reviewer and checked by a second reviewer independently. The extractions were integrated into a table to identify first author and year, country, characteristics of participants, characteristics of the intervention (e.g., duration, type), data collection instruments, use of a theoretical/conceptual framework, design, risk of bias, results, limits, strengths and funding sources. When more than one paper was published for the same study, data were extracted using one form and color coded to link the extraction back to the relevant article.
Risk of bias assessment
We used the risk of bias assessment tool proposed by Kennedy et al. (2019) . This tool was selected because it allowed us to assess study rigour in randomized and non-randomized intervention studies. The instrument includes eight items (i.e., cohort, pre/post comparison group, pre/post intervention data, random assignment of participants to intervention, random selection of participants for assessment, follow-up rate of 80% or more, comparison groups equivalent on socio-demographics, comparison groups equivalent at baseline on outcome measures). If a criterion was met, a score of one was indicated. If the criterion was not met, a score of zero was indicated. If the information provided did not allow the reviewer to assess fulfillment of the criterion, NR was indicated for not reported. If a criterion was not applicable because of the study design, we indicated NA. As proposed by Kennedy et al. (2019) , the NAs and NRs were assigned a zero to indicate that the criterion was not met. The instrument’s inter-rater reliability using Cohen’s kappa was moderate to substantial (0.41 to 0.80) for all items . To gain a better understanding of the strength and gaps of the knowledge base, a total score was calculated by item and overall for each study. The highest possible score was eight. If no psychometric properties were reported for the instruments used in the studies, we searched the literature to determine the psychometric properties of the instruments. These papers are listed in the table.
The searches yielded 1712 unique records of which 1505 were excluded during title and abstract review. Following full text review of the remaining 207 papers, 187 were excluded based on reasons listed in Fig 1. Ultimately, the search yielded 20 papers [39–53] reporting on 19 studies. One study was reported in two papers [49, 50]. Fernandez et al. (2020)  reported on the development of the intervention used in their study (Rosenman et al., 2019). All the manuscripts were published in English. The retained studies were published between 2009 and 2020 (see Table 1). Studies were conducted in Australia , Belgium , France , Germany [40, 57], New Zealand , United States [42–52, 54], Singapore , and Taiwan . Key study characteristics are presented in Table 1 and outlined below.
Studies were conducted in acute care in-patient settings including the operating room/post-anesthesia care unit [40–42, 44, 51], emergency department [52, 54], intensive care unit transport team , medical/surgical units [39, 58], labour and delivery , and crisis/cardiac arrest [43, 45, 49, 50, 55–57]
A total of 6338 participants were included in the studies. Of these, 2955 were in the control group and 3049 in the intervention group. One study did not provide the number of participants in the intervention and control groups . Three studies [41, 44, 45] included a pre-post design and only reported the total number of participants in the study. Researchers reported on studies with physicians only [40, 41, 52], medical residents/students/interns only [39, 42, 49, 50, 54, 56], nursing and medical students , nurses only [44, 48], nursing students only , and a mix of care providers [45, 47, 51, 53, 57, 58]. Only one study reported on a mix of care providers and patients .
Gender of providers was reported in 13 studies [40, 42–45, 47, 48, 52, 54–58] with more women in eight studies [42, 44, 45, 47, 48, 55, 57, 58], more men in two study [52, 54], and approximately equal representation of men and women in three studies [40, 43, 56]. Gender of patients reported as approximately half for women  in one study and mostly men in one study . Gender was not reported for providers in 6 studies [39, 41, 46, 49–51, 53].
The age of providers was reported in ten studies [40, 42, 47, 48, 51, 52, 54–57] and ranged from 16 years to over 60 years in the identified studies. The age of providers was not reported in nine studies [39, 41, 43–46, 49, 50, 53, 58]. Patients with medical conditions were aged on average 59.5 years (standard deviation: 18.9)  and patients treated for trauma were aged 43–45 years .
A range of validated and non-validated instruments were reported. No instrument clearly stood out but the Anesthetists Non-Technical Skills (ANTS) was used in three studies [40, 43, 53]. The ANTS was used to measure task management, teamwork, situational awareness and decision-making [40, 43, 53]. Jankouskas  reported Cronbach α values ranging from 0.66 to 0.83 for the ANTS. Eight studies included instruments that were developed or adapted for the study including the assessment of technical skills , simulation scenario checklists [52, 56], team leadership and patient care measure , computer experience questionnaire , team performance , program evaluation , trainee reactions to training session, and the Medical Performance Assessment tool .
Nine distinct instruments were used to assess teamwork. Beck et al.  used the German version of the Team Assessment Scale where raters assessed team performance on three subscales (Cronbach α 0.67–0.81). Coppens et al.  included a mix of self-reported and assessor evaluated instruments with Cronbach α values ranging from 0.76–0.90 for the Team Efficacy and the General Self-Efficacy Scales, and the Clinical Teamwork Scale (CTS) (Kappa .78; interclass correlation .98) . Kalisch  included the Nursing Teamwork Survey to measure trust, team orientation, backup, shared mental model and team leadership . Teamwork knowledge test examined using an eight-item test consistent with the study’s conceptual framework . Internal consistency (Cronbach α = 0.94) and test-retest reliability (Cronbach α = 0.92) were excellent . No additional evidence of validity provided by Kalisch et al. (2015) . Liaw et al.  included the Attitudes Towards Interprofessional Health Care Teams (Cronbach α: 0.82.) and Interprofessional Socialization and Valuing Scale (Cronbach α: 0.95). Mahramus  incorporated the TEAM Tool that includes 11 items to examine teamwork skills during resuscitation. Internal consistency ranged from .94 to .97. Weller  integrated the TeamSTEPPS and the Hospital Survey on Patient Safety Culture (HSOPS) questionnaires. No psychometric assessment provided in the paper.
Additional validated instruments were identified. Barzallo Salazar  used two validated instruments to measure how individuals make decisions (i.e., General Decision Making Scale and Self-Construal Scale). No psychometric properties provided by the authors. O’Leary  included the Safety Attitudes Questionnaire (SAQ). The SAQ has demonstrated internal consistency, test-retest reliability, and convergent validity. No specific values provided in the text. Acceptable to excellent values for Cronbach α and inter-rater reliability reported by the authors. Patient and provider questionnaires were adapted from the literature to measure satisfaction with bedside rounds but no psychometric assessment was provided . Oner et al.  used the modified Pian-Smith grading scale is a 5-point instrument to measure facial expression and body language to represent saying and doing nothing to advocating and inquiring repeatedly. Inter-rater agreement after training was 100%.
Five studies were supported by a theoretical or a conceptual framework [43, 44, 51, 52, 57]. Authors identified 1) the Salas framework to highlight team leadership, orientation, performance behaviours and backup behaviours[44, 57]; 2) team training and social learning theory to provide both declarative knowledge and implementation examples and teach the knowledge, skills, and attitudes ; 3) team effectiveness conceptual framework to represent the behavioural, cognitive, and affective domains ; 4) a multi-level training evaluation framework to examine trainee reactions and learning, on the job behaviours, and results .
Characteristics of the Interventions
Duration. Thirteen studies included interventions that lasted two hours or less [39, 41, 42, 44–47, 49, 50, 52, 55–58]. Six studies included interventions that lasted up to four hours [40, 43, 48, 51, 53, 54].
Frequency. Interventions were delivered in a single session [39–41, 43–45, 48–52, 54–58] or as part of daily rounds [46, 47]. Interventions could be spread over two days to one week  or over three months . Three studies included an email reminder and a follow-up simulation to determine if changes in behaviour were sustained over time [41, 48, 51]. Two studies incorporated monthly refresher sessions [46, 53].
Mode of delivery. Interventions were delivered on-site [42, 46, 47, 53, 61] or outside of the usual place of work [40, 41, 43–45, 48–52, 54–58] when specific equipment or additional space was needed. Different formats were used including scenarios with actors [40, 42, 48, 53, 54], mannequins [39, 43, 45, 49, 50, 52, 54–58], video-recorded scenarios [41, 43, 45, 52], didactic material [39, 43, 52, 54, 58], podcast and a virtual environment [44, 58], focus group discussions ; structured communication tools [46, 47, 58]; and role playing [39, 48, 51].
Type of Interventions.
Simulations. Simulation was the most frequently proposed intervention. Sixteen studies were identified [39–45, 48–50, 52–58]. High- and low-fidelity simulations were conducted for technical and non-technical skills to review basic surgical techniques and surgical errors , transport for critical care patients , resuscitation [40, 43, 45, 49, 50, 52, 55–57], leadership training , a virtual environment to resolve day-to-day conflicts in nursing teams , assertiveness training , and structured communication [41, 58]. Scenarios were delivered either all at once or broken down into several sessions (up to 3). The high-fidelity sessions required more extensive preparation ahead of the simulation.
Communication. Communication included structured communication and speaking-up.
Structured communication was included in seven studies [39, 41, 46, 47, 51, 54, 58]. Interventions included Situation, Background, Assessment, Recommendation (SBAR) [39, 49, 50, 54, 58] and the Stop; Notify; Assessment; Plan; Priorities; Invite ideas (SNAPPI) structured communication , standardized interprofessional bedside rounds to present and discuss patients’ care plans [46, 58], interdisciplinary rounds co-led by the nurse manager and physician incorporating a structured communication tool to address the needs of newly admitted patients , interactive role playing and didactic training to improve interprofessional teamwork in the operating theater .
Speaking up was included in two studies [42, 48]. In the Barzallo Salazar  study, the senior surgeon created an environment conducive to speaking up by encouraging trainees to speak up using a scripted scenario. In the Oner  study, nurses received assertiveness and advocacy training to determine if it influenced their speaking up behaviours.
Leadership training. One study  focussed on leadership training for physician residents in trauma care. The single, four-hour session included facilitated discussion of trauma leadership skills (30–45 min), a 30-minute didactic session describing leadership behaviors in trauma care, simulations, and debriefing. Simulations adapted to each participant’s learning needs while meeting curriculum requirements . During the simulation, each participant functioned as the team leader, while the second participant observed using the leadership checklist. Debriefing occurred immediately after each simulation. Self-identified areas for improvement and instructor observations informed subsequent simulations. An individualized learning plan was developed for each participant.
Debriefing. Debriefing was identified in thirteen studies in the current review [40, 41, 43–45, 48–52, 54–56, 58]. Most debriefing sessions lasted between five to 10 minutes for technical skills with the longest session lasting 30 minutes for non-technical skills. Debriefing sessions were completed immediately after the simulation scenarios in most cases and reflected key content areas (e.g., crisis management, conduct of resuscitation, teamwork behaviours, medical management). Coppens et al.  examined the contribution of debriefing following training on crisis resource management training, and found higher scores in the intervention group on teamwork (p = .011), team efficacy (p < .001) and technical skills (p = .014). No significant difference was noted for self-efficacy (p = 0.157) . Trained facilitators were used in two studies [43, 45]. To facilitate learning, participants were provided with positive examples of teamwork behaviours [44, 45]. They were asked to reflect on what they had learned, their performance [48, 54, 55], what they would do differently in the future [44, 55]. Non-blaming techniques were specified in the Jankouskas study . Debriefing was pre-recorded in two studies [40, 41]. Additionally, Zauzig  developed a distinct debriefing strategy for each simulation scenario. Only one session was conducted with no improvement in non-technical skill performance in the Zauzig  study. The level of detail of the debriefing sessions and their content was not always clearly described. It was not always possible to determine the influence of the debriefing session on participants’ learning because of the limited information provided.
Risk of bias of included studies.
Overall, the included studies met between two to six of the eight criteria. The studies by Chang et al., Jankouskas et al. and Zauzig were rated highest [36, 39, 49] [40, 43, 53]. No cohort or longitudinal study was identified in the retained studies. Eight studies reported pre/post intervention data [40, 41, 43–45, 48, 53, 54]. Eleven studies reported a random assignment of participants to the intervention [40–43, 48–50, 52–55, 57]. The other studies included a convenience sample [39, 44, 45, 51, 56, 57], random assignment at the unit level [46, 47, 57] or the process of randomization was not described. Random selection of participants for assessment was not used in any of the retained studies. A follow-up and reporting rate for at least 80% of participants was achieved in twelve studies [39–43, 45, 47, 52, 53, 55, 56, 58]. Jankouskas et al. (2011)  reported one outcome with less than an 80% follow up (response time: oxygen placement). However, this was due to non performance of the task by the control and intervention groups rather than a risk of bias in the conduct of the study. We assessed that the researchers had met the criterion.
Further, baseline socio-demographic characteristics were provided in 14 studies [40, 42, 43, 45, 47, 48, 51–58]. No differences in baseline characteristics were reported in nine studies [40, 42, 43, 48, 52–54, 57, 58]. Two studies [45, 47] reported baseline differences between the groups. Three studies [51, 55, 56] provided baseline characteristics but no comparison between the groups. Four studies did not report any information [39, 44, 46, 49, 50]. The comparison of baseline socio-demographic information was not applicable in one study  because it was a pretest/posttest design. Finally, comparison groups were compared at baseline on disclosure in seven studies [40, 41, 43, 51–54].
Researchers outlined the steps taken to limit the risk of bias including participants blinded to the study purpose , data collectors blinded to the assignment of participants [39, 41, 43, 46, 48–50, 52, 54, 56–58], assessor training to ensure inter-rater reliability [41, 43, 49, 50, 54, 58]. Three study [43, 54, 58] reported reliability indices ranging from 0.90 to 1.0). In other studies, actions during the conduct of the study increase the risk of bias. Examples included the senior surgeon participating in the scenario destined for the control and the intervention groups , research team members involved in the simulation or data collection [39, 47, 56], and no clear indication of assessor training [49, 50, 56].
Funding sources were reported in ten studies [40, 41, 44, 46, 47, 49, 50, 52–54, 58]. The role of the funder in the design, conduct or reporting of the research project was reported in two studies [46, 52]. Authors generally reported no conflict of interest except for  who reported that some co-authors had potential conflicts of interests.
The purpose of the systematic review was to identify the characteristics of brief interventions that were known to be effective to clarify roles of healthcare team members and improve team functioning. Our review highlights that research into brief team interventions is emerging as an important topic internationally. In our study sample, brief team interventions were developed to address issues in hospital settings with a range of providers including physicians, physician residents, nurses, nursing assistants, respiratory therapists, nursing and medical students, and patients in a medical ward and trauma care. No studies were conducted in primary care. High-fidelity simulations were conducted for technical skills in the operating theater and code teams to simulate a cardiac arrest or other types of crisis situations for patients. These studies required extensive preparation, highly specialized environments, and extensive resources. Structured communication and speaking-up were used for non-technical skills and required less preparation before study initiation but more sustained follow-up over the course of the study. Leadership training for non-technical skills as a short team intervention appears promising. Studies examining non-technical skills can be conducted in the teams’ usual work environment. Single training sessions can be used to improve technical skills. However, single debriefing sessions may be insufficient to improve non-technical skills. Our findings extend the review findings of Marlow et al. (2017)  who examined effective team training interventions but did not identify short interventions.
Only two studies included patients and providers to examine the effectiveness of the intervention. Guler et al. (2017)  highlighted that patient experience is a key indicator to team performance. Our study highlights that there is a clear need for studies focusing on brief team interventions to clarify roles and improve team functioning that include patients and families as part of the healthcare team. White et al. (2018)  argued that most healthcare teams face important challenges because team membership changes across rotations and shiftwork. It is thus imperative for teams to focus on communication and clarifying roles of team members, including the roles of patients and families.
Several studies included in the review were at high risk of bias. This is concerning as it represents a threat to internal validity. Only three studies were at low risk of bias. Keeping this in mind, some characteristics appear to show promise. Two- to four-hour sessions appear reasonable to engage provider participation. Training providers for technical skills using two-hour sessions followed by feedback appears to improve skill level, task management and performance in situations such as cardiac arrests or crisis situations in the operating theater. Training for non-technical skills including communication, care coordination, understanding one’s role and the role of others in the team (role clarity) appears to require more time with 4-hour training. Three to four training sessions lasting 30 minutes to one hour spread out over several weeks with structured facilitation and debriefing appear to improve the use of non-technical skills. Monthly meetings appear to sustain change over time. A recently published feasibility study by Fontenot and White (2019)  examining moral distress of nurses in the intensive care unit included an intervention with four 30-minute debriefing sessions every two weeks. The authors assessed that the intervention was feasible and acceptable in a busy work environment, and the debriefing improved non-technical skills related to self-awareness and management of moral distress. The cost and availability of replacement personnel for trainees are additional factors to consider when planning training sessions.
Simulation-based training followed by debriefing sessions provides a safe setting for healthcare professionals to develop non-technical skills. Debriefing is a key element when using simulation-based studies to enhance learning and self-awareness . However, one debriefing session does not improve performance of non-technical skills. Previously, didactic methods of training and video-based learning were mostly used to hone technical and non-technical skills of healthcare providers away from clinical environment . Gradually, as the need to mimic the clinical setting increases, simulation settings must evolve rapidly to provide a more realistic experience for learners and include patients in simulations and debriefings. It is particularly important to plan debriefing sessions using a debriefing framework  and consider including patient actors in the debriefing sessions. Low-fidelity simulation may be more beneficial when limited resources are available. In addition, in-situ training is necessary to investigate feasibility of implementing team skills in a clinical environment where the challenges of the healthcare system reside . Although different simulation training methods have been utilized to demonstrate the significance of acquiring teamwork competencies among healthcare members, there remains a gap in translating the outcomes of simulation training in the clinical setting.
It is imperative to transfer the outcomes of team interventions from simulation settings to clinical environments . Despite efforts to demonstrate the effect of simulation on improving non-technical skills, it continues to be a challenge . In the studies mentioned in this systematic review, various brief team interventions were implemented in different settings and measured using diverse validated and non-validated instruments. Thus, it is crucial to develop brief team interventions based on theoretical constructs of team functioning measured using conceptually coherent validated instruments that appropriately evaluate different aspects of brief team interventions in a simulation-based and in-situ settings.
Some studies were excluded from the systematic review even though the intervention lasted less than four hours (e.g., [70–73]). As highlighted by Fiscella et al. (2016)  teams in sports and in primary care share several challenges (e.g., role clarity, communication) to improve team performance, yet the most prominent among them is to align teamwork competencies and clinical practice requirements of providers. An important consideration in the decision to retain an article in our systematic review was the ability to translate the interventions to the healthcare context. Seidl (2017)  attempted to develop team skills using LEGO serious play in an academic setting . Prichard et al. (2007)  proposed to work on team skills by building an AM radio. Dalenberg et al. (2009)  examined the contribution of military cadets discussions of a team strategy to identify and disable an adversary. Volpe et al. (1996)  examined how training and workload while flying a fighter jet in a simulator influenced team processes. Cannon-Bowers et al. (1998)  built on the Volpe  study to understand how cross-training for young Navy recruits who needed to monitor a radar screen improved their ability to distinguish quickly between hostile and non-hostile contacts. These findings were difficult to apply to healthcare teams but they may provide different strategies to consider to improve team functioning and team performance.
Some limitations need to be kept in mind with the current review. We searched extensively for published and unpublished RCTs with no restrictions on language or geography. However, we may have missed studies because of the lack of standardized terminology in this emerging area. The quality of reporting was an important consideration in our review. In many cases, researchers did not adequately describe the study participants (e.g., age, profession, gender) or the intervention. Using reporting guidelines (e.g., CONSORT 2010) will promote the completeness and accuracy of study reporting . More complete reporting of participants’ gender would allow for the determination of intervention effects according to gender.
As indicated above, several studies were at an increased risk of bias. Although we reviewed additional literature to assess the instruments used in the included studies, incomplete reporting made it difficult to accurately assess some studies for risk of bias. Similarly, we may have scored debriefing sessions at an increased risk of bias due to incomplete descriptions of the sessions and the use of training debriefing instructors. More rigorous studies are needed using validated tools to measure outcomes as well as the inclusion of a theoretical or conceptual framework to guide study conduct. Careful consideration needs to be given to when to use of high-fidelity simulations given the prohibitive costs of the material and resource intensive preparation to conduct high quality simulations. Our results indicate that low-fidelity simulations may be an appropriate intervention for the acquisition on non-technical skills.
Our review identified three key knowledge gaps where additional research is needed. Subsequent research needs to examine the effectiveness of interventions in teams in primary care, the inclusion of patients and families and evaluating short team interventions in different settings. We identified one study using simulation training for nurses working in a correctional facility . The study was excluded from our review because it did not meet all of our eligibility criteria. Subsequent research needs to focus on areas outside the hospital setting. Interventions in primary care teams are needed because these teams are structured differently than teams in acute care and they may have different priorities. Fleury et al. (2019)  completed a cross-sectional survey of mental health teams (n = 315) in primary and specialized care, and found that team attributes (e.g., type of professional, recovery promotion) had a greater impact on team functioning in primary care teams while team processes were more important in specialized care teams. As argued by Marriage et al. (2016)  current team assessment tools are based on judgments of observable behaviours because they provide a quantifiable account of team performance. Future research also needs to focus on measuring the processes of teamwork rather than solely the outcomes of teamwork [2, 79, 80]. The inclusion of patients and families at all stages of the intervention’s development and the evaluation of the intervention’s impact is essential in the context of patient centered care. Finally, the inclusion of arts or serious play methodology in the development of brief interventions may support the emergence of creative solutions to enhance team functioning.
We conducted a systematic review to determine the characteristics of brief interventions to clarify roles and improve functioning in healthcare teams. We identified 19 experimental and quasi-experimental studies that tested interventions lasting less than half a day or five hours. High-fidelity simulations were used to develop technical skills to manage cardiac arrests and crisis situations. These sessions were shorter but required more extensive preparation. Structured communications required longer sessions with participants but may be more effective to develop non-technical skills. Debriefing can be used to support the acquisition of technical and non-technical skills. Incomplete reporting of study information was found in several studies and risk of bias was assessed as high for several studies in our sample. Intervention characteristics that appear to influence successful outcomes include using three to four 30 to 60 minutes sessions spread over two to four weeks and debriefing with a trained facilitator. Monthly follow-ups appear to sustain change over time for non-technical skills. Additional research is needed in primary care and with patients and families. We anticipate that these brief interventions can be implemented on a large scale in healthcare teams to support role clarification for patients, families and providers.
- 1. Baik D, Abu-Rish Blakeney E, Willgerodt M, Woodard N, Vogel M, Zierler B. Examining interprofessional team interventions designed to improve nursing and team outcomes in practice: A descriptive and methodological review. J Interprof Care. 2018;32: 719–727. pmid:30084723
- 2. Marriage B, Kinnear J. Assessing team performance—Markers and methods. Trends Anaesth Crit Care. 2016;7–8: 11–16.
- 3. Auraaen A, Slawomirski L, Klazinga N. The economics of patient safety in primary and ambulatory care: Flying blind. 2018. OECD Health Working Papers, No. 106, OECD Publishing, Paris. Available from: https://doi.org/10.1787/baf425ad-en.
- 4. World Health Organization. Patient Safety 2019. Available from: https://www.who.int/patientsafety/en/.
- 5. RiskAnalytica. The Case for Investing in Patient Safety in Canada. August 2017. Available from: https://www.patientsafetyinstitute.ca/en/About/Documents/The%20Case%20for%20Investing%20in%20Patient%20Safety.pdf.
- 6. Institute of Medicine. To err is human: Building a safer health system. 2000. Washington, DC: The National Academies Press. Available from: https://doi.org/10.17226/9728.
- 7. 2015 National Healthcare Quality and Disparities Report and 5th anniversary update on the National Quality Strategy. Rockville, MD: Agency for Healthcare Research and Quality; April2016. AHRQ Pub. No. 16–0015. Available from: https://www.ahrq.gov/sites/default/files/wysiwyg/research/findings/nhqrdr/nhqdr15/2015nhqdr.pdf.
- 8. Etchells E, Mittmann N, Koo M, Baker M, Krahn M, Shojania K, et al. The Economics of Patient Safety in Acute Care: Technical Report. Canadian Patient Safety Institute. 2015. Available from: https://www.patientsafetyinstitute.ca/en/toolsResources/Research/commissionedResearch/EconomicsofPatientSafety/Documents/Economics%20of%20Patient%20Safety%20-%20Acute%20Care%20-%20Final%20Report.pdf.
- 9. Fiscella K, Mauksch L, Bodenheimer T, Salas E. Improving care teams' functioning: Recommendations from team science. Jt Comm J Qual Patient Saf. 2017;43: 361–368. pmid:28648222
- 10. Cohen SG, Bailey DE. What makes teams work: Group effectiveness research from the shop floor to the executive suite. J Manage. 1997;23: 239–290.
- 11. Kilpatrick K. How do nurse practitioners in acute care affect perceptions of team effectiveness? J Clin Nurs. 2013;22: 2632–2647. pmid:23534828
- 12. Mathieu J, Gallagher P, Domingo M, Klock E. Annual review of organizational psychology and organizational behavior. Embracing complexity: Reviewing the past decade of team effectiveness research. Annu Rev Organ Psychol Organ Behav. 2019;6:17–46.
- 13. Brown J, Lewis L, Ellis K, Stewart M, Freeman TR, Janet Kasperski MJ. Conflict on interprofessional primary health care teams–Can it be resolved? J Interprof Care. 2011;25: 4–10. pmid:20795830
- 14. Landsperger JS, Semler MW, Wang L, Byrne DW, Wheeler AP. Outcomes of nurse practitioner-delivered critical care: A prospective cohort study. Chest. 2016;149: 1146–1154. pmid:26836900
- 15. Martin-Misener R, Harbman P, Donald F, Reid K, Kilpatrick K, Carter N, et al. Cost-effectiveness of nurse practitioners in primary and specialised ambulatory care: Systematic review. BMJ Open. 2015;5: e007167. pmid:26056121
- 16. Ulrich B, Crider NM. Using teams to improve outcomes and performance. Nephrol Nurs J. 2017;44: 141–151. pmid:29165965
- 17. Buljac-Samardzic M, Doekhie KD, van Wijngaarden JDH. Interventions to improve team effectiveness within health care: A systematic review of the past decade. Human resources for health. 2020;18(1):2. pmid:31915007
- 18. Feistritzer NR, Jones PO. A proof-of-concept implementation of a unit-based advanced practice registered nurse (APRN) role: Structural empowerment, role clarity and team effectiveness. Nurs Clin North Am. 2014;49: 1–13. pmid:24485182
- 19. Ly O, Sibbald SL, Verma JY, Rocker GM. Exploring role clarity in interorganizational spread and scale-up initiatives: The 'INSPIRED' COPD collaborative. BMC Health Serv Res. 2018;18: 680. pmid:30176870
- 20. Cantril C, Christensen D, Moore E. Standardizing roles: Evaluating oncology nurse navigator clarity, educational preparation, and scope of work within two healthcare systems. Clin J Oncol Nurs. 2019;23: 52–59. pmid:30681989
- 21. Kilpatrick K, Lavoie-Tremblay M, Ritchie JA, Lamothe L, Doran D. Boundary work and the introduction of acute care nurse practitioners in healthcare teams. J Adv Nurs. 2012;68: 1504–1515. pmid:22117596
- 22. Hudson CC, Gauvin S, Tabanfar R, Poffenroth AM, Lee JS, O’Riordan AL. Promotion of role clarification in the Health Care Team Challenge. J Interprof Care. 2017;31: 401–403. pmid:28140704
- 23. Ganann R, Weeres A, Lam A, Chung H, Valaitis R. Optimization of home care nurses in Canada: A scoping review. Health Soc Care Community. 2019;27: e604–e621. pmid:31231890
- 24. Sookhoo D, Thurston C. Effectiveness and experiences of team-based learning in nurse education programs: A mixed methods systematic review protocol. JBI Database System Rev Implement Rep. 2018;16: 1912–1921. pmid:30335037
- 25. McEwan D, Ruissen GR, Eys MA, Zumbo BD, Beauchamp MR. The effectiveness of teamwork training on teamwork behaviors and team performance: A systematic review and meta-analysis of controlled interventions. PLoS One. 2017;12: e0169604. pmid:28085922
- 26. Marlow SL, Hughes AM, Sonesh SC, Gregory ME, Lacerenza CN, Benishek LE, et al. A systematic review of team training in health care: Ten questions. Jt Comm J Qual Patient Saf. 2017;43: 197–204. pmid:28325208
- 27. Sidani S, Braden CJ. Design, evaluation, and translation of nursing interventions. 1st ed. West Sussex: Wiley-Blackwell; 2011.
- 28. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: New guidance. Medical Research Council. 2008. Available from: https://mrc.ukri.org/documents/pdf/complex-interventions-guidance/.
- 29. CADTH Information Services. Grey Matters: A Practical Deep-Web Search Tool for Evidence-based Medicine. 2014.
- 30. Valaitis R, Martin-Misener R, Wong ST, MacDonald M, Meagher-Stewart D, Austin P, et al. Methods, strategies and technologies used to conduct a scoping literature review of collaboration between primary care and public health. Prim Health Care Res Dev. 2012;13: 219–236. pmid:22336106
- 31. Kilpatrick K, Harbman P, Carter N, Martin-Misener R, Bryant-Lukosius D, Donald F, et al. The acute care nurse practitioner role in Canada. Nurs Leadersh. 2010;23: 114–139. pmid:21478690
- 32. Higgins JPT, Lopez-Lopez JA, Becker BJ, Davies SR, Dawson S, Grimshaw JM, et al. Synthesising quantitative evidence in systematic reviews of complex health interventions. BMJ Glob Health. 2019;4(Suppl 1): e000858. pmid:30775014
- 33. Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.0 (updated July 2019). Cochrane, 2019. Available from: www.training.cochrane.org/handbook.
- 34. Paquette L, Kilpatrick K, Jabbour M, Landry V, Tchouaket E, Beaulieu M-D, Fernandez N. Systematic review of the characteristics of a brief team intervention to optimize roles in healthcare teams. PROSPERO 2018 CRD42018088922 Available from: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42018088922
- 35. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan–a web and mobile app for systematic reviews. Syst Rev. 2016;5: 210. pmid:27919275
- 36. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 2009;6: e1000097. pmid:19621072
- 37. Sim J, Wright CC. The kappa statistic in reliability studies: Use, interpretation, and sample size requirements. Phys Ther. 2005;85: 257–268. pmid:15733050
- 38. Kennedy CE, Fonner VA, Armstrong KA, Denison JA, Yeh PT, O'Reilly KR, et al. The evidence project risk of bias tool: Assessing study rigor for both randomized and non-randomized intervention studies. Syst Rev. 2019;8: 3. pmid:30606262
- 39. Marshall S, Harrison J, Flanagan B. The teaching of a structured tool improves the clarity and content of interprofessional clinical communication. Qual Saf Health Care. 2009;18: 137–140. pmid:19342529
- 40. Zausig YA, Grube C, Boeker-Blum T, Busch CJ, Bayer Y, Sinner B, et al. Inefficacy of simulator-based training on anaesthesiologists' non-technical skills. Acta Anaesthesiol Scand. 2009;53: 611–619. pmid:19419355
- 41. Weller JM, Torrie J, Boyd M, Frengley R, Garden A, Ng WL, et al. Improving team information sharing with a structured call-out in anaesthetic emergencies: A randomized controlled trial. Br J Anaesth. 2014;112: 1042–1049. pmid:24561645
- 42. Barzallo Salazar MJ, Minkoff H, Bayya J, Gillett B, Onoriode H, Weedon J, et al. Influence of surgeon behavior on trainee willingness to speak up: A randomized controlled trial. J Am Coll Surg. 2014;219: 1001–1007. pmid:25256368
- 43. Jankouskas TS, Haidet KK, Hupcey JE, Kolanowski A, Murray WB. Targeted crisis resource management training improves performance among randomized nursing and medical students. Simul Healthc. 2011;6: 316–326. pmid:21937959
- 44. Kalisch BJ, Aebersold M, McLaughlin M, Tschannen D, Lane S. An intervention to improve nursing teamwork using virtual simulation. West J Nurs Res. 2015;37: 164–179. pmid:24763705
- 45. Mahramus TL, Penoyer DA, Waterval EM, Sole ML, Bowe EM. Two hours of teamwork training improves teamwork in simulated cardiopulmonary arrest events. Clin Nurse Spec. 2016;30: 284–291. pmid:27509565
- 46. Monash B, Najafi N, Mourad M, Rajkomar A, Ranji SR, Fang MC, et al. Standardized attending rounds to improve the patient experience: A pragmatic cluster randomized controlled trial. J Hosp Med. 2017;12: 143–149. pmid:28272589
- 47. O'Leary KJ, Wayne DB, Haviley C, Slade ME, Lee J, Williams MV. Improving teamwork: Impact of structured interdisciplinary rounds on a medical teaching unit. J Gen Intern Med. 2010;25: 826–832. pmid:20386996
- 48. Oner C, Fisher N, Atallah F, Son MA, Homel P, Mykhalchenko K, et al. Simulation-based education to train learners to "speak up" in the clinical environment: Results of a randomized trial. Simul Healthc. 2018;13: 404–412. pmid:30407961
- 49. Thomas EJ, Williams AL, Reichman EF, Lasky RE, Crandell S, Taggart WR. Team training in the neonatal resuscitation program for interns: Teamwork and quality of resuscitations. Pediatrics. 2010;125: 539–546. pmid:20156896
- 50. Katakam LI, Trickey AW, Thomas EJ. Speaking up and sharing information improves trainee neonatal resuscitations. J Patient Saf. 2012;8: 202–209. pmid:23007245
- 51. Weaver SJ, Rosen MA, DiazGranados D, Lazzara EH, Lyons R, Salas E, et al. Does teamwork improve performance in the operating room? A multilevel evaluation. Jt Comm J Qual Patient Saf. 2010;36: 133–142. pmid:20235415
- 52. Fernandez R, Pearce M, Grand JA, Rench TA, Jones KA, Chao GT, et al. Evaluation of a computer-based educational intervention to improve medical teamwork and performance during simulated patient resuscitations. Crit Care Med. 2013;41: 2551–2562. pmid:23949473
- 53. Chang YC, Chou LT, Lin HL, Huang SF, Shih MC, Wu MC, et al. An interprofessional training program for intrahospital transport of critically ill patients: Model build-up and assessment. J Interprof Care. 2019 Jan 23. pmid:30669900
- 54. Fernandez R, Rosenman ED, Olenick J, Misisco A, Brolliar SM, Chipman AK, et al. Simulation-based team leadership training improves team leadership during actual trauma resuscitations: A randomized controlled trial. Crit Care Med. 2020;48(1):73–82. pmid:31725441
- 55. Coppens I, Verhaeghe S, Van Hecke A, Beeckman D. The effectiveness of crisis resource management and team debriefing in resuscitation education of nursing students: A randomised controlled trial. J Clin Nurs. 2018;27(1–2):77–85. pmid:28401617
- 56. Evain JN, Perrot A, Vincent A, Cejka JC, Bauer C, Duclos A, et al. Team planning discussion and clinical performance: a prospective, randomised, controlled simulation trial. Anaesth. 2019;74(4):488–96. pmid:30768684
- 57. Beck S, Doehn C, Funk H, Kosan J, Issleib M, Daubmann A, et al. Basic life support training using shared mental models improves team performance of first responders on normal wards: A randomised controlled simulation trial. Resuscitation. 2019;144:33–9. pmid:31505232
- 58. Liaw SY, Wu LT, Wong LF, Soh SLH, Chow YL, Ringsted C, et al. "Getting everyone on the same page": Interprofessional team training to develop shared mental models on interprofessional rounds. J Gen Intern Med. 2019;34(12):2912–7. pmid:31515736
- 59. Kiesewetter J, Fischer MR. The teamwork assessment scale: A novel instrument to assess quality of undergraduate medical students' teamwork using the example of simulation-based ward-rounds. GMS Z Med Ausbild. 2015;32(2):Doc19. pmid:26038684
- 60. Kalisch BJ, Lee H, Salas E. The development and testing of the nursing teamwork survey. Nurs Res. 2010;59: 42–50. pmid:20010044
- 61. Marshall JC, Bosco L, Adhikari NK, Connolly B, Diaz JV, Dorman T, et al. What is an intensive care unit? A report of the task force of the World Federation of Societies of Intensive and Critical Care Medicine. J Crit Care. 2017;37: 270–276. pmid:27612678
- 62. Rosenman ED, Vrablik MC, Brolliar SM, Chipman AK, Fernandez R. Targeted simulation-based leadership training for trauma team leaders. West J Emerg Med. 2019;20(3):520–6. pmid:31123555
- 63. Guler PH. Patient experience: A critical indicator of healthcare performance. Front Health Serv Manage. 2017;33: 17–29. pmid:28319979
- 64. White BAA, Eklund A, McNeal T, Hochhalter A, Arroliga AC. Facilitators and barriers to ad hoc team performance. Proc (Bayl Univ Med Cent). 2018;31: 380–384. pmid:29904320
- 65. Fontenot NM, White KA. Using evidence-based debriefing to combat moral distress in critical care nurses: A pilot project. J Nurs Educ Pract. 2019;9: 1–12.
- 66. Gardner AK, Kosemund M, Hogg D, Heymann A, Martinez J. Setting goals, not just roles: Improving teamwork through goal-focused debriefing. Am J Surg. 2017;213: 249–252. pmid:27765182
- 67. Brown LL, Overly FL. Simulation-based interprofessional team training. Clin Pediatr Emerg Med. 2016;17: 179–184.
- 68. Endacott R, Gale T, O’Connor A, Dix S. Frameworks and quality measures used for debriefing in team-based simulation: A systematic review. BMJ Stel. 2019;5: 61–72.
- 69. Bryant K, Aebersold ML, Jeffries PR, Kardong-Edgren S. Innovations in simulation: Nursing leaders' exchange of best practices. Clin Simul Nurs. 2019 Oct 26.
- 70. Dalenberg S, Vogelaar ALW, Beersma B. The effect of a team strategy discussion on military team performance. Mil Psychol. 2009;21 Suppl 2: S31–S46.
- 71. Cannon-Bowers JA, Salas E, Blickensderfer E, Bowers CA. The impact of cross-training and workload on team functioning: A replication and extension of initial findings. Hum Factors. 1998;40: 92–101.
- 72. Seidl T. Case Study 1: Playful team reflection using LEGO® Serious Play®. Int J Game Base Learn. 2017;7: 83–86.
- 73. Volpe CE, Cannon-Bowers JA, Salas E, Spector PE. The impact of cross-training on team functioning: An empirical investigation. Hum Factors. 1996;38: 87–100. pmid:8682521
- 74. Fiscella K, Fogarty C, Salas E. What can primary care learn from sports teams? J Ambul Care Manage. 2016;39: 279–285. pmid:27232689
- 75. Prichard JS, Ashleigh MJ. The effects of team-skills training on transactive memory and performance. Small Group Res. 2007;38: 696–726.
- 76. Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, et al. CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340: c869. pmid:20332511
- 77. Díaz DA, Shelton D, Anderson M, Gibert GE. The effect of simulation-based education on correctional health teamwork and communication. Clin Simul Nurs. 2019.
- 78. Fleury MJ, Grenier G, Bamvita JM, Chiocchio F. Variables associated with perceived work role performance among professionals in multidisciplinary mental health teams overall and in primary care and specialized service teams, respectively. Eval Health Prof. 2019;42: 169–195. pmid:28974105
- 79. Kilpatrick K, Paquette L, Bird M, Jabbour M, Carter N, Tchouaket E. Team functioning and beliefs about team effectiveness in inter-professional teams: Questionnaire development and validation. J Multidiscip Healthc. 2019;12: 827–839. pmid:31632051
- 80. Kilpatrick K, Tchouaket E, Paquette L, Guillemette C, Jabbour M, Desmeules F, et al. Measuring patient and family perceptions of team processes and outcomes in healthcare teams: Questionnaire development and psychometric evaluation. BMC Health Serv Res. 2019;19: 9. pmid:30612571
- 81. Fletcher G, Flin R, McGeorge P, Glavin R, Maran N, Patey R. Anaesthetists' Non-Technical Skills (ANTS): Evaluation of a behavioural marker system. Br J Anaesth. 2003;90: 580–588. pmid:12697584
- 82. Guise JM, Deering SH, Kanki BG, Osterweil P, Li H, Mori M, et al. Validation of a tool to measure and promote clinical teamwork. Simul Healthc. 2008;3(4):217–23. pmid:19088666
- 83. Weller J, Shulruf B, Torrie J, Frengley R, Boyd M, Paul A, et al. Validation of a measurement tool for self-assessment of teamwork in intensive care. Br J Anaesth. 2013;111(3):460–7. pmid:23558846
- 84. Luszczynska A, Scholz U, Schwarzer R. The general self-efficacy scale: Multicultural validation studies. J Psychol. 2005;139(5):439–57. pmid:16285214
- 85. Kim J, Neilipovitz D, Cardinal P, Chiu M, Clinch J. A pilot study using high-fidelity simulation to formally evaluate performance in the resuscitation of critically ill patients: The University of Ottawa Critical Care Medicine, High-Fidelity Simulation, and Crisis Resource Management I Study. Crit Care Med. 2006;34(8):2167–74. pmid:16775567
- 86. American Institutes for Research. TeamSTEPPS® Teamwork Perceptions Questionnaire Manual. 2010. Available from: https://www.ahrq.gov/sites/default/files/wysiwyg/teamstepps/instructor/reference/teamperceptionsmanual.pdf.
- 87. Sexton JB, Helmreich RL, Neilands TB, Rowan K, Vella K, Boyden J, et al. The Safety Attitudes Questionnaire: Psychometric properties, benchmarking data, and emerging research. BMC Health Serv Res. 2006;6: 44. pmid:16584553
- 88. Blegen MA, Gearhart S, O'Brien R, Sehgal NL, Alldredge BK. AHRQ's hospital survey on patient safety culture: Psychometric analyses. J Patient Saf. 2009;5: 139–144. pmid:19920453
- 89. Ballangrud R, Husebo SE, Hall-Lord ML. Cross-cultural validation and psychometric testing of the Norwegian version of the TeamSTEPPS® teamwork perceptions questionnaire. BMC Health Serv Res. 2017;17: 799. pmid:29197381