https://orcid.org/0000-0002-2809-4227
Figures
Abstract
Introduction/Background
Mass-casualty incidents (MCIs) and disasters require an organised and effective response from medical first responders (MFRs). As such, novel training methods have emerged to prepare and adequately train MFRs for these challenging situations. Particular focus should be placed on extended reality (XR), which encompasses virtual, augmented and mixed reality (VR, AR, and MR, respectively), and allows participants to develop high-quality skills in realistic and immersive environments. Given the rapid evolution of high-fidelity simulation technology and its advantages, XR simulation has become a promising tool for emergency medicine. Accordingly, this systematic review aims to: 1) evaluate the effectiveness of XR training methods and 2) explore the experience of MFRs undergoing such training.
Methods
A comprehensive search strategy will encompass four distinct themes: MFRs, disasters/MCIs, education and simulation, and XR. Four databases (MEDLINE, EMBASE, CINAHL and LILACs) will be searched along with an in-depth examination of the grey literature and reference lists of relevant articles. MetaQAT will be used as a study quality assessment tool and integrated into Covidence as part of the data extraction form. Given the predicted high heterogeneity between studies, it may not be possible to standardise data for quantitative comparison and meta-analysis. Thus, data will be synthesised in a narrative, semi-quantitative manner.
Citation: Otero-Varela L, Cintora AM, Espinosa S, Redondo M, Uzuriaga M, González M, et al. (2023) Extended reality as a training method for medical first responders in mass casualty incidents: A protocol for a systematic review. PLoS ONE 18(3): e0282698. https://doi.org/10.1371/journal.pone.0282698
Editor: Muhammad Shahzad Aslam, Xiamen University - Malaysia Campus: Xiamen University - Malaysia, MALAYSIA
Received: July 22, 2022; Accepted: February 14, 2023; Published: March 23, 2023
Copyright: © 2023 Otero-Varela et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: No datasets were generated or analysed during the current study. All relevant data from this study will be made available upon study completion.
Funding: Funding support for this research was received from the European Union’s Horizon 2020 research and innovation program (grant number 101021775) awarded to MCCA. The funders provided their approval to publish this manuscript, but had no role in study design, data collection and analysis or manuscript preparation.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Mass casualty incidents (MCIs) and disasters are unforeseeable events that require an organised and effective first-line response from emergency medical service providers, also referred to as medical first responders (MFRs). These highly demanding situations typically entail providing urgent medical care to a large number of victims at the same time, with the scarcity of technical resources [1, 2]. Under these circumstances, the classification of victims or triage is a dynamic and always-evolving process that depends on the demands of the environment and the severity of the patient’s injuries. Accordingly, MFRs are faced with difficult decisions to determine the preferential care and priority of evacuation for each patient, considering their chance of survival [3, 4]. As MCIs and disasters are globally increasing [5], it is paramount that MFRs are prepared and adequately trained for these challenging situations/environments/settings.
Traditional training techniques for disaster preparedness are broad and varied: from lectures and seminars, or practical tabletop exercises, to live drills including cards, actors or even sophisticated mannequins [6–8]. In recent years, simulation-based education has grown in popularity, as it offers multiple advantages, such as the possibility to train in safe environments without direct risk to participants (psychologically and physically) and the opportunity for repetitive practice to acquire a certain level of competence [9, 10]. Furthermore, the rapid evolution of high-fidelity simulation technology has led to the widespread use of novel training tools in medicine, with a particular focus on extended reality (XR), i.e., virtual, augmented and mixed reality (VR, AR, and MR, respectively). Such systems integrate immersive environments and realistic scenarios that allow participants to increase the speed of gaining knowledge and develop high-quality skills [11–13].
Enhancing the training MFRs receive is fundamental to strengthening the preparedness and response to MCIs and disasters. Previous systematic reviews have assessed the use of XR simulation training. However, these did not focus on the out-of-hospital emergency setting [14, 15]. Conversely, other reviews have identified relevant articles highlighting what could be a promising and novel tool for disaster medicine, although the effectiveness of the training was not assessed [9, 11]. Accordingly, this systematic review will examine the existing literature on relevant primary research studies to 1) evaluate the effectiveness of MCI training methods using XR simulation and 2) explore the experience of MFRs. Therefore, the research questions that this study will address are as follows:
- Primary question: What is the effectiveness of extended reality simulation as a tool to train medical first responders in mass casualty incidents?
- Secondary questions:
- ○ What tools and metrics were used to measure the effectiveness of the XR simulation training?
- ○ What is the experience of medical first responders training for mass-casualty incidents using XR simulation? What’s their perceived impact of such training?
Materials and methods
This protocol is registered at the International Prospective Register of Systematic Reviews (PROSPERO #CRD42021275692) and adheres to the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) [16].
Eligibility criteria
Detailed inclusion and exclusion criteria are available in Table 1 and described below.
Study types.
To conduct a thorough systematic review, this review will not be restricted to interventional studies but will incorporate all original research reporting on XR training for medical first responders. Accordingly, experimental, quasi-experimental and observational studies will be included. Among these, health technology assessments and health economic evaluations are also expected to be captured.
Participants.
This review will focus on medical first responders, defined as out-of-hospital healthcare professionals from the emergency medical service. Derivatives and variations of this term will also be accepted.
Intervention.
Educational activities or training using extended reality (XR) simulation, including virtual reality (VR), mixed reality (MR) and augmented reality (AR) [17].
Virtual reality. VR is the best known of these technologies. It is fully immersive, making people think they are in a different environment or world from the real one. Using a head-mounted display (HMD) or headset, users can experience a computer-generated world with images and sounds. They also can manipulate objects and move around using touch controllers while connected to a console or PC.
Augmented reality. AR superimposes digital information on real-world elements. AR keeps the real world at the centre, but enhances it with other digital details, overlaying new layers of perception and complementing the reality or environment.
Mixed reality. MR brings together the natural world and digital elements. Using state-of-the-art sensing and imaging technologies, users interact and manipulate physical and virtual components and environments in mixed reality. Mixed reality allows users to see and immerse themselves in the world, even when interacting with a virtual environment using their own hands, without the need to remove their headsets. In addition, it provides the ability to have one foot (or type) in the real world and the other in an imaginary place, thus breaking down the basics between the real and the imaginary.
Comparator.
Studies comparing the intervention group to controls [18, 19] (i.e., medical first responders trained using another simulation or training) will be included. Also, studies with no control group (e.g., before and after) will be selected.
Setting.
Simulated scenarios of mass casualty incidents, including those caused by natural disasters or human-made.
Main outcomes.
The outcomes of interest are the effectiveness of the XR simulation training for MFRs in mass casualty incidents, and the perception of MFRs. There is no standard definition for these outcomes; but instead, these will be determined based on the information reported in the included articles. For instance, participants’ experience could be interpreted as satisfaction, opinion, acceptability, or usability; while effectiveness can also have many definitions, including but not limited to triage accuracy, time to triage, or intervention correctness.
Said outcomes will be classified within Kirkpatrick levels (Table 2), given that it is the most effective and most used approach for evaluating training [20]. To better address the secondary questions, the metrics and tools used for effectiveness will also be captured.
Information sources and search strategy
The following bibliographic databases will be searched with no date, country, or language restrictions: MEDLINE (Medical Literature Analysis and Retrieval System Online), EMBASE (Excerpta Medica Database), CINAHL (Cumulative Index to Nursing and Allied Health Literature), and LILACS (transl. Latin American and Caribbean Literature in Health Sciences).
To capture additional studies, clinical trials registries and health technology assessment databases will be searched, namely: ClinicalTrials.gov, the WHO International Clinical Trials Registry Platform and Cochrane Central Register of Controlled Trials (CENTRAL), INAHTA (International Network of Agencies for Health Technology Assessment), and BRISA (trans. Regional Base for Health Technology Assessment Reports of the Americas). Furthermore, grey literature will be reviewed, including Google searches, the websites of relevant emergency medicine and disaster preparedness organisations, and the reference lists and bibliographies of the included studies. Field experts will also be contacted for further information about ongoing or unpublished studies.
The search strategy for online databases was developed by the research team and revised by a research librarian with expertise in systematic reviews. As a result, the search strategy includes four broad themes:
- Medical first responders (participants)
- Mass casualty incidents (setting)
- Education and simulation (intervention)
- Extended reality (intervention)
MeSH terms (when applicable), titles and abstracts, and keywords will be searched within each theme using the Boolean operator “OR”. These four searches will be combined using the Boolean operator “AND”. The proposed keywords and MeSH terms used in this systematic review are available in S1 Table and an example search strategy is provided in S2 Table.
Study selection and data extraction
Both abstract and full-text screening phases will be done independently and in duplicate with the support of Covidence. This web-based software platform streamlines the screening and data extraction processes. Identified records will be compiled in the reference management software Endnote™, and then uploaded into Covidence. Titles and abstracts will be first scanned following the above eligibility criteria, and articles will be selected for full-text review if: 1) both reviewers agree or 2) the abstract does not provide sufficient information to decide. Conflicts between reviewers will be identified and discussed until an agreement is reached. Study authors will be contacted when necessary for further clarification if crucial information is missing from the included articles. Inter-rater agreement will be assessed using the Kappa statistic for both screening stages.
Two authors will independently extract relevant information about each included study: first author, publication year, country, study design, setting, number and type of participants, as well as details about the intervention and comparator. Results regarding the effectiveness of the training and how effectiveness was measured (i.e., tools and metrics) will be collected and classified within Kirkpatrick levels [20]. Participants’ experiences from the disaster training will also be captured. The accuracy of the data extraction process will be guaranteed by a third reviewer, who will ensure consensus is achieved. Additionally, the data extraction form will include the different items comprising the risk of the bias assessment tool.
Risk of bias assessment
MetaQAT will be used as a risk of bias and study quality assessment tool [21], given its versatility. As stated in the eligibility criteria, all original studies (experimental, quasi-experimental and observational) can be included; therefore, it is preferable to use one tool that allows for evaluating individual studies regardless of their research design, like MetaQAT [22]. Assessing all studies with the same tool could help prevent possible biases and heterogeneity, as opposed to simultaneously using different tools that are specific to determined study designs (e.g., Cochrane risk-of-bias tool for RCTs).
MetaQAT is a validated tool that allows for rigorous appraisal, consisting of 8 items to assess each study’s relevancy, reliability, validity, and applicability [22]. As a result, the risk of bias will be assessed at the study level, and will be reported as high, low, or unclear.
Furthermore, data on funding will be extracted, in order to asses whether included studies are subject to direct financial conflicts of interest, such as studies funded by their developers.
Strategy for data synthesis
If possible, data from included studies will be pooled for meta-analysis, and random-effect meta-analysis will be performed using the statistical software Stata®. However, there is expected heterogeneity in the following variables: type of MFR, participants’ demographic characteristics, previous experience with XR, intervention type (i.e., VR, AR, MR), length of training, setting, effectiveness outcomes and measurement, as well as participants’ perception. Consequently, if substantial heterogeneity is found, results will be presented in a narrative form with semi-quantitative analysis using descriptive statistics.
Identified studies (both included and excluded) will be summarised in a PRISMA flow diagram, and data extracted from those selected will be tabulated into study characteristics and summary findings. Results from included studies will be elaborated in detail: relevant elements will be reported and summarised for each type of XR simulation training. Further, to better address the research questions, study findings will be stratified by the effectiveness of the training and its classification within Kirkpatrick’s evaluation model [20]. The tools and metrics used to measure effectiveness will be described. Participants’ experience will also be outlined.
Discussion
To our knowledge, this will be the first systematic review to evaluate the use of extended reality simulation to train medical first responders for mass casualty incidents.
Among the strengths of this review are the in-depth search strategy, the specific inclusion/exclusion criteria, and the validated quality appraisal tool (i.e., MetaQAT). Therefore, this protocol provides a rigorous template for future scoping or systematic reviews for effective training methods for medical first responders.
Outcomes of this study will interest medical first responders, policy-makers, emergency medical service agencies, and disaster management authorities. Our findings will also provide a direction for future researchers seeking to improve the training of medical first responders.
A survey of several European countries on how they train and prepare for disasters underlines the need for training programmes that emphasise the creation and use of safe training scenarios, set training objectives in medical aspects of disaster management, and provide a clearer understanding of the medical aspects of disaster management. Following this choice, they want the exercise to concentrate mainly on the prehospital medical care and management aspects. They prioritise medical coordination procedures, medical management at the site, medical alert procedures, assessment of immediate needs, medical resources management, medical transport protection and safety procedures [23]. In another European survey, the EU Commission gave the following recommendations: organise focused expert meetings on the abovementioned subjects. Promote joint exercises, collect and promote best practices by supporting research for evidence-based results, and promote cross-border cooperation and possibly preplanned reinforcements [24]. All in all, there is a perceived lack of adequate training with current methods and the need to advance in new technologies and types of simulation to achieve the proposed objectives for disaster training [25].
Supporting information
S1 Table. Systematic review search parameters: Keywords and mesh terms for search strategy (classified by theme).
https://doi.org/10.1371/journal.pone.0282698.s001
(DOCX)
S2 Table. Search strategy for EMBASE database.
Accessed on October 1st, 2021.
https://doi.org/10.1371/journal.pone.0282698.s002
(DOCX)
References
- 1. Basogain X, Olabe M, Espinosa K, Rouèche C, Olabe J. Realidad Aumentada en la Educación: una tecnología emergente. Escuela Superior de Ingeniería de Bilbao, EHU Recuperado de http://bitly/2hpZokY. 2007;
- 2. Barsom EZ, Graafland M, Schijven MP. Systematic review on the effectiveness of augmented reality applications in medical training. Surg Endosc. 2016 Oct;30(10):4174–83. Epub 2016 Feb 23. pmid:26905573; PMCID: PMC5009168.
- 3. Vincent DS, Sherstyuk A, Burgess L, Connolly KK. Teaching mass casualty triage skills using immersive three-dimensional virtual reality. Acad Emerg Med. 2008 Nov;15(11):1160–5. Epub 2008 Aug 10. pmid:18699829.
- 4. Iserson KV, Moskop JC (2007). Triage in medicine, Part I: Concept, History, Types. Annals Emergency Medicine. 49(3):275–281.
- 5. Brohi K, Tallach R. Mass casualty medicine: time for a 21st-century refresh. British Journal of anaesthesia. 2021 Dec 20. pmid:34949438
- 6. Kim J, Lee O. Effects of a simulation-based education program for nursing students responding to mass casualty incidents: A pre-post intervention study. Nurse Educ Today. 2020 Feb;85:104297. Epub 2019 Nov 21. pmid:31778863.
- 7. Martin-Ibañez L, Roman P, Diaz-Córtes MDM, Fernández-Sola C, Granero-Molina J, Cardona D. Intentional mass-casualty incident simulation-based training: A qualitative study into nursing students’ perceptions and experiences. Nurse Educ Today. 2021 Oct;105:105051. Epub 2021 Jul 3. pmid:34256215.
- 8. Saber DA, Strout K, Caruso LS, Ingwell-Spolan C, Koplovsky A. An Interprofessional Approach to Continuing Education With Mass Casualty Simulation: Planning and Execution. J Contin Educ Nurs. 2017 Oct 1;48(10):447–453. pmid:28954180.
- 9. Gunshin M, Doi K, Morimura N. Use of high‐fidelity simulation technology in disasters: an integrative literature review. Acute Medicine & Surgery. 2020 Jan;7(1):e596. pmid:33364034
- 10. Fritz PZ, Gray T, Flanagan B. Review of mannequin‐based high‐fidelity simulation in emergency medicine. Emergency Medicine Australasia. 2008 Feb;20(1):1–9. pmid:17999685
- 11. Duan YY, Zhang JY, Xie M, Feng XB, Xu S, Ye ZW. Application of virtual reality technology in disaster medicine. Current medical science. 2019 Oct;39(5):690–4 Erratum in: Curr Med Sci. 2020 Dec;40(6):1205 pmid:31612384
- 12. Mills B, Dykstra P, Hansen S, Miles A, Rankin T, Hopper L, et al. Virtual reality triage training can provide comparable simulation efficacy for paramedicine students compared to live simulation-based scenarios. Prehospital Emergency Care. 2020 Jul 3;24(4):525–36. pmid:31580178
- 13.
Goldstein I. L., & Ford J. K. (2002). Training in organisations: Needs assessment, development, and evaluation (4th ed.). Belmont, CA: Wadsworth.
- 14. Quick JA. Simulation training in trauma. Missouri Medicine. 2018 Sep;115(5):447. pmid:30385994
- 15. McGrath JL, Taekman JM, Dev P, Danforth DR, Mohan D, Kman N, et al. Using virtual reality simulation environments to assess competence for emergency medicine learners. Academic Emergency Medicine. 2018 Feb;25(2):186–95. pmid:28888070
- 16. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1. pmid:25554246
- 17.
VR vs. AR vs. MR: What You Need to Know. Intel Corporation [cited Jul 4 2022]. Available from: https://www.intel.com/content/www/us/en/tech-tips-and-tricks/virtual-reality-vs-augmented-reality.html
- 18. Ingrassia PL, Ragazzoni L, Carenzo L, Colombo D, Ripoll Gallardo A, Della Corte F. Virtual reality and live simulation: a comparison between two simulation tools for assessing mass casualty triage skills. Eur J Emerg Med. 2015 Apr;22(2):121–7. pmid:24841770.
- 19. Kirkpatrick D. (1996). Great ideas revisited. Training & Development, 50(1), 54–60.
- 20. La Duke P. How to Evaluate Training: Using the Kirkpatrick Model. Prof Saf 2017 08;62(8):20–21.
- 21.
Ontario Agency for Health Protection and Promotion (Public Health Ontario), Rosella LC, Pach B, Morgan S, Bowman C. Meta-tool for quality appraisal of public health evidence: PHO MetaQAT. Toronto, ON: Queen’s Printer for Ontario; 2015
- 22. Rosella L, Bowman C, Pach B, Morgan S, Fitzpatrick T, Goel V. The development and validation of a meta-tool for quality appraisal of public health evidence: Meta Quality Appraisal Tool (MetaQAT). Public Health. 2016;136:57–65. pmid:26993202
- 23. Delooz H, Debacker M, Moens G, Johannik K; I SEE Partnership. European survey on training objectives in disaster medicine. Eur J Emerg Med. 2007 Feb;14(1):25–31. pmid:17198322.
- 24. Domres BD, Rashid A, Grundgeiger J, Gromer S, Kees T, Hecker N, et al. European survey on decontamination in mass casualty incidents. Am J Disaster Med. 2009 May-Jun;4(3):147–52? pmid:19739457.
- 25. Cynthia L. Foronda CL, Fernandez-Burgos M, Nadeau C, Kelley CN, Henry MN. Virtual Simulation in Nursing Education: A Systematic Review Spanning 1996 to 2018. Simul Healthc. 2020 Feb;15(1):46–54. pmid:32028447