https://orcid.org/0000-0003-3817-2783
Figures
Abstract
Introduction
Triage is a crucial tool for managing a Multiple Victim Incident (MVI). One particularly problematic issue is the communication of results to the chain of command and control. Favourable data exists to suggest that digital triage can improve some features of analogue triage. Within this context we have witnessed the emergence of the Valkyries Project, which is working to develop strategies to respond to MVIs, and especially cross-border incidents. To that end, an IT platform called “SIGRUN” has been created which distributes, in real time, all the information to optimise MVI management. A full-scale simulation, held on the Spain-Portugal border and featuring contributions from different institutions on both sides of the border, put to the test the role of information digitalisation in this type of incidents.
Objective
To evaluate the impact of the synchronous digitalisation of information on the optimal management of Multiple Victim Incidents.
Method
Clinical evaluation study carried out on a cross-border simulation between Spain and Portugal. A Minimum Data Set (MDS) was established by means of a modified Delphi by a group of experts. The digital platform “SIGRUN” integrated all the information, relaying it in real time to the chain of command and control. Each country assigned two teams that would carry out digital and analogue triage synchronously. Analogue triage variables were gathered by observers accompanying the first responders. Digital triage times were recorded automatically. Each case was evaluated and classified simultaneously by the two participating teams, to carry out a reliability study in a real time scenario.
Results
The total duration of the managing of the incident in the A group of countries involved compared to the B group was 72.5 minutes as opposed to 73 minutes. The total digital assistance triage (AT) time was 37.5 seconds in the digital group, as opposed to 32 minutes in the analogue group. Total evacuation (ET) time was 28 minutes in the digital group compared with 65 minutes in the analogue group. The average differences in total times between the analogue and the digital system, both for primary and secondary evaluation, were statistically significant: p = 0.048 and p = 0.000 respectively. For the “red” category, AT obtained a sensitivity of 100%, also for ET, while with regard to AT safety it obtained a PPV of 61.54% and an NPV of 100%, and for ET it obtained a PPV of 83.33% and an NPV of 100%. For the analogue group, for AT it obtained a sensitivity of 62.50%, for ET, 70%, for AT safety it obtained a PPV of 45.45% and an NPV of 92.31%, while for ET it obtained a PPV of 70% and an NPV of 92.50%. The gap analysis obtained a Kappa index of 0.7674.
Citation: Behzadi Koochnai N, Muñoz Romo R, Riera López N, Caballero Cubedo R, Gómez de la Oliva S, Martin de Rosales Cabrera T, et al. (2024) Digitalisation of information and management optimisation in Multiple Victim Incidents. Analytical study. PLoS ONE 19(5): e0303247. https://doi.org/10.1371/journal.pone.0303247
Editor: Inge Roggen, Universitair Kinderziekenhuis Koningin Fabiola: Hopital Universitaire des Enfants Reine Fabiola, BELGIUM
Received: October 13, 2023; Accepted: April 22, 2024; Published: May 14, 2024
Copyright: © 2024 Behzadi Koochnai et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files
Funding: This paper is supported by the European Union's Horizon 2020 research and innovation program under grant agreement No. 101020676 for the VALKYRIES project (Harmonization and Pre-Standardization of Equipment, Training and Tactical Coordinated procedures for First Aid Vehicles deployed on European multi-victim Disasters). the funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
In recent decades, the increase in Multiple Victim Incidents (MVI) and disasters where the health and emergency services have been overwhelmed, at least initially, by the number of victims they had to treat [1] highlights the need for transversal strategies such as those stated in the Sendai Framework for Disaster Risk Reduction 2015–2030 [2].
The optimisation of management of this type of incidents translates into an increase in survival. One of the most commonly-used tools for improving MVI management and saving a greater number of lives is that of triaging [3]. This optimisation of management involves classification, prioritisation and distribution of resources bearing in mind not only the severity of the patients’ situation but also the prognosis for their injuries, survival, quality of life and the resources available to achieve the best results for all of the victims [4–6].
There are many types of triage, but they all share the same principle of victim prioritisation and management. One of the most-commonly used triage models is START (Simple Triage and Rapid Treatment), which is based on a system that divides patients up into four categories: patients classified as “red” present life-threatening injuries requiring immediate treatment, patients classified as “yellow” present serious injuries but their treatment can wait, “green” patients present minor injuries, and “black” patients are those that have either passed away or are expected to do so [7].
However, several studies have questioned the correct execution of a triage during an MVI [8,9]. Errors in the triage process such as under-triage (which may result in patients not receiving the care they need within the proper time) and over-triage (which means the patient is classified with a higher severity level) represent an improper use of resources and in both cases results in an increase in mortality [4].
Another very common problem in the triage process is the communication of results to the entire care chain and the chain of command and control of the MVI in order to be able to properly manage the necessary resources [10,11]. Analogue recording of data and the need to communicate same in a non-synchronous manner, whether it is by telephone or TETRA (Terrestrial Trunked Radio), produces a cyclical and inopportune arrival of information [12,13].
Some studies have offered encouraging data regarding digital triage which can, at least partly, improve certain crucial aspects of analogue triage, such as: reducing the triage execution time with greater sensitivity and specificity [14], reducing treatment time for more severe patients thanks to less managing time and more effective medical decisions [15], better accessibility and universality of healthcare in communities with limited resources, such as rural areas [16] and facilitating medical research thanks to its practicality in terms of data gathering, thus helping to create algorithms and protocols more efficiently, among other points [17].
However, we need to continue to make progress in the research until we can integrate these systems into our standard practice [18]. In a recent meta-analysis, Wallace WH, et al. also state that we need to carry on researching this area given the variability of results and low accuracy of studies [19].
Within this context, the strategic lines of transnational research of the Horizon Europe-H2020 projects have resulted in the emergence of the Valkyries project (grant agreement no. 101020676), which aims to develop, integrate and demonstrate capabilities that can enable an immediate, co-ordinated response to emergencies, including search and rescue, health and safety, in natural/man-made disaster scenarios with multiple victims, and with particular application to cross-border incidents.
To achieve the aforementioned objectives, an IT platform by the name of “SIGRUN” has been created that integrates and relays to the care chain and the chain of command and control, in real time, all the basic essential information, known as the Minimum Data Set (MDS), in order to optimise managing of the MVI. The MDS was developed by an expert committee using the modified Delphi method.
A full-scale simulation, held on the Spain-Portugal border in May 2023, with more than 50 victims and featuring the participation of over 400 servicepeople and members of different organisations responsible for managing emergencies put to the test the role of information digitalisation in this type of incident. The simulation involved the participation of the emergency departments of each of the two countries.
Method
A clinical evaluation study carried out in a large scale, cross-border simulation between Spain and Portugal and with the direct involvement of both countries’ emergency departments and with more than 400 servicepeople and members of different organisations on the ground.
The following points were considered during the preparatory stage:
- Creating the Chain of Command and Control: a group of experts established the chain of command and control with the heads of the Triage Post—Advanced Health Post (AHP)–Evacuation Post–Health Command Post (HCP)–Incident Management Team (IMT) of the coordinating centre.
Establishing the Minimum Data Set (MDS): using the modified Delphi method, an expert committee defined the MDS to be used for the optimal management of the MVI. The items we selected to that end were: the feasibility of collecting the data, real-time onward communication of the data to the whole chain of command and control, and being considered essential to good management of an MCI. Subsequently, the process was completed using the Utstein method to improve data quality, which was modified appropriately for formal systematic collection of essential information in a public health emergency. The process had three phases. In the first one, the essential principles for governing the whole process were selected: being considered essential for the proper management of MCIs in cross-border areas, adequate capacity to collect and report the data without any need to change the protocols and procedures of first responders, the proficiency and experience of experts (at least 20 years’ experience in the field of knowledge), ensuring the implementation of the operational procedures of first responders and taking into account the maximum degree of agreement between experts. In the second phase, the items were voted on in line with the literature. In the third phase, the Delphi method was established through three rounds of voting by the group of experts, eliminating the items that did not obtain any votes. To ensure that a consensus was obtained, the degree of agreement between the experts was determined by calculating Cohen’s Kappa.
- Defining the incident and victims: an expert committee agreed on such items as type of incident, number of victims, pathologies of same, colour code for care triage and evacuation and their clinical evolution, among other points.
Each country assigned two teams to carry out, in parallel and without interfering with each other, digital and analogue triage for 25 assigned victims. The victims (identical on both sides of the border) were represented on the day of the simulation by 10 actors, while 15 clinical records provided clinical information and two triage files, one analogue (from the country that treated the patient) and the other with a QR code to digitalise the information gathered, with analogue notes of same being made in the event of a digital malfunction (which did not occur). See Figs 1 and 2.
- Digital platform for the synchronous gathering and distribution of information: a modular, multilayer, high-security digital platform called SIGRUN, developed specifically for this project, integrated all the information from the MDS and relayed it in real time to the entire chain of command and control.
- System for gathering variables: more than 10 observers of the project accompanied the first responders and the command heads in order to monitor the analogue communications and triage times. The digital triage time was recorded automatically.
With regard to analysis of the data, this study was organised into two clearly differentiated stages:
Validating the digital triage system as compared with the analogue triage system
Usually, when designing a study aimed at calculating or sometimes estimating the basic indicators to evaluate the effectiveness of a diagnostic test, an N-sized sample is taken of a specific population to which the screening test has been applied, and the criteria of truth for making the estimates. In the present study, this sample was determined by the volunteers who took part in the drill where the management of the cross-border incident was staged. To that end, a sample was selected of N1 sick patients and another of N2 non-sick individuals, diagnosed as such by the benchmark test, applied to N individuals (N1 + N2), thus creating a 2x2 table, and in this way evaluating the diagnostic test.
Analysis and validation of the strategy was carried out by adhering to the minimum requirements for the publication of diagnostic tests (STARD initiative) [20]. Thus, basic indexes were calculated: sensitivity (S), specificity (SP) and positive and negative predictive values (PPV and NPV). In the same way, other indicators were calculated such as the validity index (VI) or correct hit ratio, and the Youden index (YI) [21], or the ratio for the rectified probability of detecting illness.
To calculate these values, we needed to select a “Gold Standard” that would define patients’ real severity levels, given that without this definition, according to Windle et al [22], it is not possible to test and validate a triage system. In our study, we establish as a benchmark the traditional analogue system executed by experts, as we stated previously (Table 1).
S and SP are the traditional basic measurements of the diagnostic value of a test. They measure the diagnostic discrimination of a test in relation to a benchmark criterion which is considered the truth. These indicators allow us in principle to directly compare the effectiveness of a test with that of others, and to expect similar results when they are applied in different countries, regions, or settings. S indicates the ability of the test to detect a sick subject; that is, it expresses how "sensitive" the test is to the presence of the illness or condition. To quantify its expression, probabilistic terms are used. SP indicates the ability of the test to identify as healthy (not sick) those who are. The PPV would be the probability that a truly urgent subject had been classified as urgent (red colour), and the NPV would be the probability that a non-urgent subject had been classified as non-urgent (yellow and green colours).
With regard to managing the exercise, it was established that each case should be evaluated and classified simultaneously by the two participating teams, so as to be able to determine the reliability of the tests. This enabled us to carry out a reliability study in real time and scenario, and not in hypothetical situations. Using this premise, we also proceeded to apply the evaluation of a test for the two triage tools via the parallel method [23], whereby all these tests are applied simultaneously to the same sample of individuals, so that all individuals that obtain negative results in all tests are considered to be negative, and all others positive.
Finally, as a complement to validity, we examined the consistency between the benchmark and the valuations of the assigned teams. In this way, we obtained the corresponding Cohen’s Kappa indexes [24] for the evaluation of the consistency level.
To solve the problem due to a part of the observed agreement (in principle unknown) that may be attributable exclusively to chance, we use the kappa index developed by Cohen in 1960. Its purpose is to quantify the degree of agreement once the part that can be attributed is eliminated, exclusively at random. This index relates the agreement that observers exhibit, beyond that which is due to chance, with the potential agreement also beyond chance. To do this, we calculate the difference between the proportion of observed agreement and the proportion of agreement expected by chance. If this is equal to zero, then the degree of agreement that has been observed can be attributed entirely to chance. If the difference is positive, this indicates that the degree of agreement is greater than what would be expected if only chance were operating and vice versa: in the (admittedly unlikely) case in which the difference was negative then the data would be exhibiting less agreement than that which is expected only by chance. Kappa is the quotient between that quantity and the maximum agreement that can be expected without the intervention of chance. This index, when the observers are independent, takes the value 0, and reaches the maximum value of 1 only if there is perfect agreement between the observers and finally, it is never less than –1. To identify what kappa value can be considered as an indicator of good agreement, in 1972 Landis and Koch proposed a scale for interpreting the kappa value that considers a value greater than or equal to 0.40 as acceptable and values greater than 0 as excellent 0.75 [25].
Evaluation of the triage tools was carried out at several different levels: first of all, the ability to classify a patient correctly as “red”, both in the primary and the secondary evaluation, and secondly the ability to classify a patient correctly as “yellow”, both in the primary and the secondary evaluation. It was suggested that for the urgent levels (red and yellow), there should be no discrepancies, given that these were the most severe patients. Thus, it was decided that an analysis should be carried out by separating both levels and leaving non-urgent levels (green and black) grouped together. Likewise, to assess the safety of the triage system, we focused on predictive values, both PPV and NPV. These severity levels were validated given that they were considered critical for responding to the triaging objectives.
Evaluating the impact of the synchronous digitalisation of information on reducing analogue communication time for the chain of command and control of the Multiple Victim Incident
To do this, we set up a clinical evaluation study with a total sample size of 100 clinical records. Once the simulation had been carried out, we evaluated the total time spent managing the incident and the congestion of telecommunications systems bearing in mind two independent groups with quantitative variables applying the Mann-Whitney U test (Wilcoxon signed-rank test of related samples) [26]. This test enabled us to determine whether there were any differences in the averages between the groups we were comparing, owing to the different ways the data was distributed, which as they involved time variables, their behaviour was different to normal. Furthermore, due to the nature of the study and that the sample and its size were determined by the logistical conditions of the exercise, the statistical power of the study was calculated. The power of a hypothesis test is the probability that the test correctly rejects the null hypothesis. Therefore, in some studies with negative results it will be concluded that there are no differences when there really are. This error is known as a type II error. The probability of making an error of this type is usually denoted by β and its complement, 1-β, which is what is known as statistical power or statistical power [27].
Subsequently, we attempted to monitor the possible confusion effect produced by those factors which, as they are related to the exposure factor in the study, determine the appearance of the result through stratification via the participant groups (countries A and B), an effect that had already been observed previously (consistency between observers).
Ethical and legal aspects of the project
The participating volunteers who simulated patients were students of the Degree in Health Emergency Technician in Extremadura, Spain. In the simulation exercise, the participating actors were recruited by the General Directorate of Emergencies of Extremadura (Spain), who informed and obtained their verbal consent to participate in the exercise, not by the Valkyries project. The head of the Valkyries project is aware of this consent from the General Directorate of Emergencies of Extremadura, in compliance with ethical and legal requirements. None of them received any financial reward for their activities. We individually collect their consent to participate to the Use-Case providing the information on the project and on the dissemination activities. The consent was freely given by all participants and nobody revoked it. All the activities followed the internal procedures of the simulation activities, which are fully compliant with the applicable legal framework. As only simulated data were used and no personal or patient data were required, the study did not need to be assessed by an Ethics Committee beforehand. The VALKYRIES Project did not deal with personal data. The Consortium of VALKYRIES was made up of ethical-legal experts, who were involved in both compliance, regulatory and standardization tasks, and a declaration was signed by those responsible for the Consortium, showing the commitment to comply with all ethical-legal issues. Within this declaration, the decision of the Consortium to develop the work without hiring volunteers, not processing personal data, is stated. The case study scenarios involved first responders to receive feedback on the effectiveness of the interoperable platform through the application of simulated and unreal data. The simulated data were generated directly by the Consortium.
Results
Of the evaluation and validation of the triage system
The corresponding data for S and SP were obtained, which are shown in Table 1, with the digital Care Triage system VALKYRIA (VALK CT) obtaining an S for the “red” category of 100%, and for the “yellow” category of 41.67%. For the Digital Evacuation Triage VALKYRIA (VALK ET) obtained an S for the “red” category of 100%, and for the “yellow” category of 58.33%. With regard to the safety of the care Triage System, for the groups of cases identified as “red” and “yellow”, we estimated a PPV of 61.54% and an NPV of 100%, compared with an estimated PPV of 71.43% and an NPV of 72%, respectively, which means that Non-Urgent cases were identified, a priori, by the Triage System as Non-Emerging, with an 18% margin of error, and 0% for the Emerging cases. With regard to ET, for the groups of cases identified as “red” and “yellow”, a PPV of 83.33% was estimated, compared to an estimated PPV of 87.50% and an NPV of 77.27%, respectively, which means that the Non-Urgent cases were identified, a priori, by the triage system as Non-Emerging, with a 13% margin of error and 0% for the Emerging cases for the digital system (VALK ET).
With regard to the Analogue Care Triage tool (ANALOG CT), we obtained an S for the “red” category of 62.50%, and 83.33% for the “yellow” category, in the CT. For the Analogue Evacuation Triage (ANALOG ET) we obtained an S for the "red" category of 70%, and 58.33% for the “yellow” category. As for safety in the ANALOG CT system, for the groups of cases identified as “red” and “yellow”, a PPV of 45.45% and an NPV of 92.31% were estimated for the former, and a PPV of 47.62% and an NPV of 81.82% for the latter, respectively, which means that Non-Emerging cases were identified as such, a priori, by the triage system, with an 18.18% margin of error, and 52.38% for Emerging cases. For the ANALOG ET, for the groups of cases identified as “red” and “yellow”, a PPV of 70% and an NPV of 92.50% were estimated for the former, and a PPV of 87.50% and an NPV of 77.27% for the latter, respectively, which means that the Non-Emerging cases were identified as such, a priori, by the triage system, with a 30% margin of error, with 7.50% for Emerging cases and 22.73% for cases that can wait and 12.50% for delayed Urgencies.
The results of the parallel testing showed, for VALK ET and ANALOG ET, an S for “red” levels of 100% in Evacuation Triage, with an estimated PPV of 66.67% and an NPV of 100%, respectively, which means that Non-Emerging cases were identified as such, a priori, by the triage system, with a margin of error of 0%, and of 33.33% for the Emerging cases if we assume that both tests confirm the benchmark result being applied at the same time by the two teams. For the “yellow” levels, an S of 75% was obtained, with an estimated PPV of 90% and an NPV of 85% respectively, which means that the Non-Urgent cases were identified as such, a priori, by the triage system, with a 10% margin of error and 15% for the Emerging cases under the same assumption. The rest of the results are shown in Table 1. As we can see, for the CT, these values show results that are quite significantly poorer when both tools are applied at the same time by the two teams.
To complement the criterion validity, an examination was made between the benchmark and the valuations of the different triage systems. Table 2 shows the results of the gap analysis. For these data, a Kappa Index was obtained of 0.7674 and 0.6591, respectively, for the detection of “red” and “yellow” cases, a good consistency according to the criteria of Landis-Koch [25], which means that the triage systems facilitated a good triaging level assignation for the classified patients.
Of the evaluation of the results of the exercise
The total duration of managing the incident for the group of participating countries A compared to group B was 193 minutes as opposed to 210 minutes.
The total time for Care Triage was 38 minutes for country A group compared to 31 minutes for country B. A difference was found between the primary triage times separated by colour-coded severity levels, tending towards a reduction of the average time spent. In the VALK group (“red”: 32 seconds, “yellow”: 49 seconds, “green”: 41 seconds) compared to the ANALOG group (“red”: 45.5 seconds, “yellow”: 34.5 seconds, “green”: 30 seconds).
The total time for secondary triage was 28 minutes in the VALK group compared to 65 minutes in the ANALOG group. The secondary triage times separated by colour-coding were significantly different between the VALK group (“red”: 41.5 seconds, “yellow”: 33 seconds, “green”: 28 seconds) and the ANALOG group (“red”: 115 seconds, “yellow”: 170 seconds “green”: 60 seconds).
Congestion times in communications for the benchmark system were analysed separately in the Triage Post and in the Advanced Health Post for groups A and B.
The total congestion time in communications in the Triage Post was 252 seconds for the primary evaluation and with a line-engaged period of 2310 seconds in the Advanced Command Post for the secondary evaluation for group A. Meanwhile, for group B, a period of 1108 seconds was observed for the primary evaluation, and with a line-engaged period of 1625 for the secondary evaluation.
The total line-engaged time for communications in the Advanced Health Post with the Coordinating Centre was 42.7 minutes for group A and 45.5 minutes for group B. All of the stratified results for the participating groups are shown in Table 3.
In the evaluation of the statistical significance of the differences between the averages of total times found by applying the Wilcoxon signed-rank test for related samples, both for the primary and the secondary evaluation, a difference in average times was obtained between the ANALOG and the VALK systems with a statistical significance of p = 0.048 for the primary evaluation and p = 0.000 for the secondary evaluation. Therefore, we can conclude that the differences found are statistically significant.
Regarding this point, it must be taken into account that non-parametric tests usually offer less statistical power than their corresponding parametric significance tests, so with non-parametric tests the probability of rejecting the null hypothesis is lower while the alternative hypothesis is true. For all that, we thought it would be interesting to calculate the power of the study. Generally, one tends to work with a power of around 80% or 90%. Thus, we finally obtain with our data: If we want to detect a minimum difference of 7 seconds: Bilateral approach: 81.29% as static power. If we want to detect a minimum difference of 5 seconds: Bilateral approach: 52.98% as static power. Therefore, we observe that to detect a difference of about 7 seconds in the management of the incident, the power of the test is quite adequate, as we have commented, but for minor differences it decreases considerably. Due to the nature of the study, we believe that these differences of less than 7 seconds are not very relevant. Therefore, we can affirm that the power of the test can be considered quite adequate to respond to the purposes of the study [27].
Finally, we evaluated once again the differences found by stratifying for the participant groups, given that this factor, as we noted previously, can act as a possible confusion factor. Thus, we obtained an average difference of times between the ANALOG and the VALK systems with a statistical significance, for group A, of p = 0.548 for the primary evaluation and p = 0.000 for the secondary evaluation; for group B we obtained an average difference of times between the ANALOG and the VALK systems with a statistical significance of p = 0.038 for the primary evaluation and p = 0.000 for the secondary evaluation. Therefore, we can conclude that for group A, during the primary evaluation, no differences in statistical significance were found. The difference between the raw value of the Wilcoxon signed-rank test for related samples and the value obtained for each group confirmed the confusion effect that the variable exerts, as a result of which it was stratified. Table 4 shows the percentage of correct assignment of suit colour to the most critical Red and Yellow patients.
Discussion
In various studies, the validity of triage has been assessed based on the predictive capacity for hospitalisation, patient death and length of stay, in relation to the level assigned. In our study, owing to the nature of the type of triage and scope, we opted to assess criterion validity through the concept of S and SP, understanding that triaging functions as a screening tool for classifying patients. Moll stresses the importance of a triage system having a high sensitivity to identify those patients whose condition might worsen if they do not receive correct treatment [28–30].
As a first step, we observe that the digital triage tool differentiates between Emerging patients (who cannot wait), and Non-Emerging patients (who can wait), which is why it is important to know the values of Sensitivity (100%), Specificity (88.10%) and Negative Predictive Value (100%); in this case all of these are above 80%, which are considered to be good values, both in Care Triage and in Evacuation. This was not the case for Urgency as such, which showed lower S values in both evaluations.
The fact that the S of the test and the NPV are very high corroborates the idea that the model designed is safe, given that it minimises the phenomenon of infra-triage, a situation that is necessary in a triage model in order that the most severe patients should be seen as soon as possible, thus minimising the probability that their treatment is delayed.
The PPV shown for both evaluations signifies that a significant proportion of patients classified as emerging (red) and urgent (yellow), were not in fact so. In principle this does not affect these patients negatively, but strategies must be sought to improve these values, given that a greater number of patients mistakenly classified as emerging and urgent can affect the amount of treatment time given to less urgent cases (green).
In the same way, the NPV was analysed for each of the levels, with slightly low values detected for cases classified as “yellow” level in both evaluations, with hardly any differences observed with the analogue tool. For the “red” level, the NPV was always 100% for the digital tool, but not for the analogue one which, even though it delivered good figures, never reached 100% in either of the evaluations.
Furthermore, the parallel evaluation tests showed that in the event that both tools agreed, they did not especially enhance the effectiveness of the triage, which would support the observation of the non-equivalence in value of the two tools.
The gap analysis carried out was used for the comparison between observers, without actually assessing at that moment which of the two were carrying out under-triage or over-triage, with the aim of evaluating the reliability of the system. In this way we obtained “very good” data according to Landis-Koch criteria for both groups to compare in the detection of Emerging and acceptable cases in the case of Urgent cases for Evacuation Triage. We should point out that the different composition of both teams in terms of training levels could be one of the determining factors for the poor results obtained for the Evacuation Triage.
Another fundamental point to bear in mind is the impact of the synchronous digitalisation of information on the managing of MVI by the chain of command and control. In the analogue case, the heads of the chain of command and control of the MVI had to speak on the telephone or via TETRA for an average of 44 minutes to obtain the crucial information, which represents a significant delay in management given that the information is not being transmitted in real time. However, in the digital case, the information was synchronous and shared between the entire chain of command and control, including the receiving hospitals, who were able to prepare themselves well in advance for the type and number of patients that would be arriving within one or two hours. Another important point is the maximum traceability of patients and their respective data from minute 0, which helps to improve the management of each case.
Conclusions
After having evaluated the Triage System through the developed digital tool, its validity has been demonstrated compared to the analogue tool, as a result of which its use is recommended. The Triage System using said tool shows good Sensitivity and Specificity for Emerging patients (“red” category) and Urgent patients (“yellow” category), though Sensitivity is significantly lower for the latter. The high Negative Predictive Value makes it highly effective as a screening tool for emerging cases. Patients classified as Emerging in care triage and evacuation triage showed the best results, as a result of which we can state that the Triage ensures special attention for these patients.
The synchronous digitalisation of information reduces analogue communication to the minimum, thereby maximising situational awareness of the incident among the chain of command and control, which results in the optimal management of the incident as a whole and possible improvement in terms of victim survival.
Acknowledgments
For the execution of this study, we would like to thank the research team of the VALKYRIES project, the Fundación para la Investigación e Innovación Biosanitaria de Atención Primaria (FIIBAP) and the European Programs Technical Support Unit (UTAPE) of the Department of Health of the Community of Madrid.
References
- 1.
Lomaglio L, Ansaloni L, Catena F, Sartelli M, Coccolini F. Mass Casualty Incident: Definitions and Current Reality. In: Kluger Y, Coccolini F, Catena F, Ansaloni L, editors. WSES Handbook of Mass Casualties Incidents Management Hot Topics in Acute Care Surgery and Trauma. Springer; 2020.
- 2.
Oficina de las Naciones Unidas para la Reducción del Riesgo de Desastres. Marco de Sendai para la Reducción del Riesgo de Desastres 2015–2030. Vol. 1a edición, Tercera Conferencia Mundial de las Naciones Unidas. Sendai (Japón); 2015.
- 3. Brohi K, Tallach R. Mass casualty medicine: time for a 21st century refresh. Br J Anaesth. 2022 Feb;128(2):e65–7. pmid:34949438
- 4. Aitken P, FitzGerald G. Disaster triage: evidence, consistency and standard practice. Emerg Med Australas. 2012 Jun;24(3):222–4. pmid:22672161
- 5. Jenkins JL, McCarthy ML, Sauer LM, Green GB, Stuart S, Thomas TL, et al. Mass-casualty triage: time for an evidence-based approach. Prehosp Disaster Med. 2008;23(1):3–8. pmid:18491654
- 6. Christian MD. Triage. Crit Care Clin. 2019 Oct;35(4):575–89. pmid:31445606
- 7. Franc JM, Kirkland SW, Wisnesky UD, Campbell S, Rowe BH. METASTART: A Systematic Review and Meta-Analysis of the Diagnostic Accuracy of the Simple Triage and Rapid Treatment (START) Algorithm for Disaster Triage. Prehosp Disaster Med. 2022 Feb;37(1):106–16. pmid:34915954
- 8. Killeen JP, Chan TC, Buono C, Griswold WG, Lenert LA. A wireless first responder handheld device for rapid triage, patient assessment and documentation during mass casualty incidents. AMIA. Annu Symp proceedings AMIA Symp. 2006;2006:429–33. pmid:17238377
- 9. Kamler JJ, Taube S, Koch EJ, Lauria MJ, Kue RC, Rush SC. Effectiveness of and Adherence to Triage Algorithms During Prehospital Response to Mass Casualty Incidents. J Spec Oper Med. 2023;23(1):59. pmid:36853853
- 10. McSwain NE. Disaster response. Natural disaster: Katrina. Surg Today. 2010 Jul 26;40(7):587–91.
- 11. Carter H, Drury J, Amlôt R, Rubin GJ, Williams R. Effective responder communication improves efficiency and psychological outcomes in a mass decontamination field experiment: implications for public behaviour in the event of a chemical incident. PLoS One. 2014;9(3):e89846. pmid:24595097
- 12. Gabbe BJ, Veitch W, Curtis K, Martin K, Gomez D, Civil I, et al. Survey of major trauma centre preparedness for mass casualty incidents in Australia, Canada, England and New Zealand. EClinicalMedicine. 2020 Apr;21:100322. pmid:32382716
- 13. Lenert LA, Kirsh D, Griswold WG, Buono C, Lyon J, Rao R, et al. Design and evaluation of a wireless electronic health records system for field care in mass casualty settings. J Am Med Informatics Assoc. 2011 Nov 1;18(6):842–52. pmid:21709162
- 14. Henning E, Bakir MS, Haralambiev L, Kim S, Schulz-Drost S, Hinz P, et al. Digital versus analogue record systems for mass casualty incidents at sea-Results from an exploratory study. PLoS One. 2020;15(6):e0234156. pmid:32502206
- 15. Lai L, Wittbold KA, Dadabhoy FZ, Sato R, Landman AB, Schwamm LH, et al. Digital triage: Novel strategies for population health management in response to the COVID-19 pandemic. Healthcare. 2020;8(4). pmid:33129176
- 16. Ziebart C, Kfrerer ML, Stanley M, Austin LC. A Digital-First Health Care Approach to Managing Pandemics: Scoping Review of Pandemic Self-triage Tools. J Med Internet Res. 2023 May 17;25:e40983. pmid:37018543
- 17. Roque Mazoni S, Andrade J, da Silva Antonio P, Baraldi S, Frates Cauduro FL, Fernandes dos Santos PH, et al. Triage Strategies for COVID-19 Cases: A Scope Review. Inq J Heal Care Organ Provision, Financ. 2022 Jan 12;59:004695802210958.
- 18. Churruca K, Ellis LA, Pope C, MacLellan J, Zurynski Y, Braithwaite J. The place of digital triage in a complex healthcare system: An interview study with key stakeholders in Australia’s national provider. Digit Heal. 2023 Jan 23;9.
- 19. Wallace W, Chan C, Chidambaram S, Hanna L, Iqbal FM, Acharya A, et al. The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review. npj Digit Med. 2022 Aug 17;5(1):118. pmid:35977992
- 20. Bossuyt PM. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. BMJ. 2003 Jan 4;326(7379):41–4. pmid:12511463
- 21. Hughes G. Youden’s Index and the Weight of Evidence. Methods Inf Med. 2015 Jan 22;54(02):198–9.
- 22. Windle J. Don’t throw triage out with the bathwater. Emerg Med J. 2003 Mar 1;20(2):119–20. pmid:12642520
- 23. García-Pérez MA. Statistical criteria for parallel tests: A comparison of accuracy and power. Behav Res Methods. 2013 Dec 15;45(4):999–1010. pmid:23413034
- 24. Cohen J. A Coefficient of Agreement for Nominal Scales. Educ Psychol Meas. 1960 Apr 2;20(1):37–46.
- 25. Landis JR, Koch GG. The Measurement of Observer Agreement for Categorical Data. Biometrics. 1977;33(1). pmid:843571
- 26. Sundjaja JH, Shrestha R, Krishan K. McNemar And Mann-Whitney U Tests. 2023.
- 27. A Review of Statistical Power Analysis Software. Bull Ecol Soc Am. 1997;78(2).
- 28. Baumann MR. Evaluation of the Emergency Severity Index (version 3) Triage Algorithm in Pediatric Patients. Acad Emerg Med. 2005 Mar 1;12(3):219–24. pmid:15741584
- 29. Naeger DM, Kohi MP, Webb EM, Phelps A, Ordovas KG, Newman TB. Correctly using sensitivity, specificity, and predictive values in clinical practice: How to avoid three common pitfalls. Vol. 200, American Journal of Roentgenology. 2013. pmid:23701086
- 30. Moll HA. Challenges in the validation of triage systems at emergency departments. J Clin Epidemiol. 2010 Apr;63(4):384–8. pmid:19875271