https://orcid.org/0000-0002-3264-6792
Figures
Abstract
Background
Exercise in different health sectors is an important step in preparing programs for disaster risk management. The present study aimed to develop and validate a tool for evaluating disaster exercises during the design stage in the health sector.
Methods
This methodological study was conducted in two phases using an explanatory sequential mixed method approach. Semi-structured interviews with 25 disaster-related health professionals were conducted during the qualitative phase (item generation), and a systematic review was done to evaluate items pool of disaster exercises design stage tool. The quantitative phase (item reduction) assessed both face and content validity, as well as reliability using Cronbach’s alpha and intra-class correlation coefficient.
Results
At the first stage four main categories and eleven subcategories were extracted from the data, the main categories including "coordination, command and guidance of exercise", "hardware and software requirements of exercise ", "organizational exercise resources", and "communication and exercise public information". The initial items pool included 164 items that were reduced to 50 after the assessment of validity (face and content). Cronbach’s alpha (0.89) and intra-class correlation coefficient (ICC = 0.72) tests indicated that the tool is also reliable.
Conclusion
The research findings provide a new categorization perspective to understand the disaster exercises evaluation in the health sector. The existing 50-item tool can evaluate disaster exercises design stage in the health sector through collecting data with appropriate validity and reliability.
Citation: Sheikhbardsiri H, Nekoei-Moghadam M, Yarmohammadian MH, Khankeh H, Aminizadeh M, Ebadi A (2022) Developing and psychometric testing of the evaluation tool for disaster exercises design stage: A mixed method study. PLoS ONE 17(3): e0260581. https://doi.org/10.1371/journal.pone.0260581
Editor: Md Nazirul Islam Sarker, Neijiang Normal University, CHINA
Received: October 18, 2020; Accepted: November 12, 2021; Published: March 22, 2022
Copyright: © 2022 Sheikhbardsiri et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The datasets generated and/or analyzed during the current study are not publicly available due to restrictions by the Ethics Committee of Kerman University of Medical Sciences. Interested and qualified researchers who meet the criteria for access to confidential data may submit data access requests to (kmu_Research@yahoo.com).
Funding: The author(s) received no financial support for the research, authorship, and/or publication of this article.
Competing interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Introduction
Disaster is a serious problem occurring over a short or long period of time that causes widespread human, material, economic or environmental loss which exceeds the ability of the affected community or society to cope using its own resources [1, 2].
Human societies have long been plagued by natural disasters, causing significant physical harm and death to a large number of people [3]. Today, disasters are on the rise around the world and have had a massive impact on human life and health, disturbing society’s capability to meet its fundamental needs and causing damage, inability and death of a large portion of the population [4].
Every year, an average of 354 natural hazards are recorded in the Emergency Events Database (EED), killing 68273 people and affecting 210 million people, with damages totaling more than $ 141 billion [2, 5]. Disasters cause small and large catastrophes that are sometimes difficult to recover from, depending on the type, scope, frequency, and density of the population exposed to the hazards [6]. Unpredictability is one of the most significant characteristics of natural hazards [7]. As a result, the only way to deal with this phenomenon is to be prepared, which can then prevent or reduce the amount of damage caused by it [7, 8]. The health systems, as one of the main pillars of the communities, play the most important role in disaster risk reduction [4, 9]. Disaster preparedness provides a platform for designing effective, realistic, and coordinated planning, reducing duplication of efforts and increasing its overall effectiveness and response efforts by National Societies, households, and community members [2, 10].
Krzysztof Goniewicz et al. concluded that disaster management qualifications should be connected to the hospital quality improvement process, with a focus on health worker in order to improve hospital preparedness [11]. Disaster exercises are the most important way to maintain and improve preparedness plans, and they can be used to test and validate policies, programs, and procedures, teach personnel, and improve inter-organizational communication and coordination [10]. Establishing and promoting specific functions codified in the response operational plan at different staff and operational levels of healthcare require the execution of operation-based and discussion-based exercises to identify and improve the strengths of the program and promote the relevant capabilities [12]. Exercise evaluation is an important step in designing and implementing exercise. It systematically investigates and indicates how well the exercise met its objectives. It also determines the strengths and weaknesses of the disaster exercise program [10, 13].
Jen Keynes et al. from Johns Hopkins University conducted a comprehensive study in 2009 that included a review of hospital preparedness in accordance with the National Incident Management System (NIMS). In this study, eight available instruments were used to assess the hospital’s preparedness: the Association for Professionals In Infection Control and Epidemiology (AIC), the Centers for Disease Control and Prevention (CDC), the Centers for Medicare and Medical Services (CMS), The Joint Commission (TJC), the Veterans Health Administration (VJHA), the South Carolina Department of Health (SCDH), the Texas Department of Health (TDH), and Agency for Healthcare Research and Quality(AHRQ). These instruments were examined with the principles of the NIMS and finally it was concluded that each of the existing instruments had defects and did not cover all the principles of NMIS [14]. Tang et al. conducted a study titled "developing an evaluation tool for hospital emergency preparedness in China." The researchers presented a methodology for evaluating hospital preparation drills in this study. The instrument is made up of 68 elements that assess structural and non-structural vulnerability, with 21 of them being structural. During a review of the tool, it was discovered that there was no item to evaluate the dimensions of hospital performance, and even the non-structural section only addresses a few dimensions, whereas modern preparedness assessment tools are expected to assess hospital capacity, service continuity, and the level of safety required to provide services by hospital staff. In addition, accurate information regarding the validation of the tool was not presented [15]. Mollie w et al. presented six evaluation modules to capture strengths and weaknesses of different aspects of hospital disaster response, including Predrill Module, Incident Command Center Module, Triage Zone Module, Treatment Zone Module, decontamination Zone Module and Group Debriefing Module. They also reported that these tools can evaluate specifics for general observation and documentation, victim tracking, biological incidents, and radiological incidents [16]. Nekoie-Moghadam et al. conducted a systematic review of the tools and checklists used to assess hospital disaster readiness and found that none of the reviewed checklists and tools addressed all of the elements required for a comprehensive hospital preparedness assessment [17].
One of the main obstacles to the accurate and scientific evaluation of exercises in the health sector is the lack of a standard and holistic tool for the evaluation of disaster exercises [18]. Many financial resources are spent in the health sector on disaster preparedness exercises. As there is no plan to control such exercises and they are not evaluated accurately to identify the strengths of organizations executing exercises, macro targeting and perspectives on health system preparedness in unexpected events have not been achieved, and they have not been measured scientifically [13, 19]. The above emphasizes the design and development of a comprehensive tool for evaluation of disaster exercises in the healthcare system in the exercise design phase to provide necessary protocols and instructions for evaluating disaster preparedness exercises. The present tool was developed to overcome the aforementioned obstacles and challenges, to evaluate accurately health exercises, and to increase the level of preparedness and the operational capacity of the health sector in disasters. The findings of this study also can be used as a guideline for developing standard preparedness exercises of the health system for emergencies and disasters.
Methods
Study design
This mixed-method study was carried out in two phases to design a comprehensive tool for the evaluation of disaster exercises design stage in the health sector. In the qualitative phase, interviews with Iranian disaster experts and systematic review were used for (Item Generation) and the quantitative phase was used for (Item Reduction) and validation of the measurement tool.
Qualitative phase (Item generation)
To create the items, three major steps were taken: a) conducting interview with experts using directed content analysis to identify and develop the concept of disaster exercise evaluation, b) conducting systematic review and c) incorporating steps 1 and 2 for item pool.
Step 1: Qualitative study.
This qualitative study was conducted in Iran, one of the most disaster-prone countries in the world. The study population included 25 experts who had practical experience or theoretical knowledge about "designing of preparedness exercises in disasters" and had participated in at least one operation-based or discussion-based exercise. A purposive sampling method with maximum diversity was used to select participants. Sampling was carried out until data saturation occurred, i.e. when the researcher concluded that additional interviews would not provide new information. Participants included seven pre-hospital managers, four hospital managers, two nursing experts in hospital emergencies committee, four experts in Emergency Operation Centers (EOC) in the Universities of Medical Sciences, six health experts in disaster risk reduction, and two vice chancellors of the logistic of the Universities y of Medical Sciences. The interviewees answered to a similar set of questions, which began with "have you ever experienced the disaster preparedness exercises of the health system?", "Describe the worst and best disaster exercises that you have experienced in the health system, "Based on your experience, what components and features should be considered in a standard discussion-based exercise? What components and features should be considered in a standard operation-based exercise? What requirements and functions should be considered when designing operation- and discussion-based exercises in disasters? Based on the above guide, additional questions were raised during the interview and when authors found new concepts. Moreover, for concept saturation, we use who, when, why and how as well as "Could you please give an example" or "Please explain more" for data and concept saturation. The interviews were recorded and lasted between 25 and74 minutes. The interviewer and interviewee agreed on the location and time of the interview; also, memos and notes were taken during the interviews to accurately describe and interpret the responses.
Step 2: A systematic review.
The following step involved reviewing scientific studies and literature relevant to the research topic in order to add several items and complete the item pool. the papers searched using specific and relevant keywords in search engines and research databases, including ISI Web of science, PubMed, Scopus, Science Direct, Ovid, Pro-Quest, Wiley, Google Scholar and Persian database such as ISC, SID from January 1, 2000, to June 24, 2018. The following keywords and search strategies were used in conjunction with the Boolean operators OR and AND: (simulation, practice, drill, exercise, design, instrument, tool, questionnaire, measurement, checklist, scale, test, inventory, battery, evaluation, assessment, appraisal, emergency, disaster, crisis, hazard, catastrophe, hospital, prehospital, health centers, treatment centers). This step was executed according to the PRISMA guideline [20].
Inclusion criteria. Persian and English articles evaluating disaster exercises in the health sector (both manmade and natural disasters), the articles addressing discussion- or operation-based exercises, articles describing the design of disaster exercises in the health system, and the articles addressing various techniques used to evaluate disaster exercises in the healthcare system.
Exclusion criteria. studies evaluating the performances of other organizations during disasters, including partner and support organizations such as firefighting, the Red Crescent, police and welfare studies evaluating the process of disaster risk management, including prevention and mitigation, response and recovery, studies presenting at conferences and proceedings, studies publishing in languages other than English and Persian before the year 2000.
Step 3: Incorporation of steps 1 and 2.
The extracted items of disaster exercise were merged in the systematic and qualitative steps. The tables extracted from each study step were separated, and in this step, all components and characteristics get combined; redundant items get removed and similar ones get merged. Independent of the systematic review and qualitative study, new categories and subcategories get created. Since the new categories and subcategories served as the foundation for the pool of items, they get evaluated with greater sensitivity. The final table, containing the theme, category, subcategory, and codes expands the main exercise evaluation as converted into items. The research team examined questions and eliminated or modified several. Finally, an initial format for the disaster exercises design stage evaluation tool (DEDSET) prepared, consisting of 163 questions. Subsequently, the primary questionnaire’s validity determined. The psychometric properties of the DEDSET examined for face and content validity, as well as reliability.
Quantitative phase (Item reduction)
Face validity.
Ten experts experienced in disasters examined the tool’s dimensions and their relationships in order to determine the qualitative face validity of the items and analyzed the tool in terms of ease of completion, legibility, grammar, and the writing style of items in terms of ambiguity, level of difficulty and fitness. The research team re-read the phrases and incorporated experts’ suggestions. The item effect was used to quantitatively calculate the instrument’s face validity. For this purpose, participants assigned a value to each item using the five-point Likert scale ranging from five (quite important) to one (not important at all). Impact score = Frequency (%) ×Importance. The impact score was considered to be greater than 1.5 [21]. At this stage, no question was removed.
Content validity.
For qualitative content validity to be ensured, ten health professionals experienced in disasters were asked to express their corrective views in terms of grammar, the use of appropriate words, proper placement of items, proper scoring and appropriateness of the selected dimensions. The questionnaire items revised in response to experts’ suggestions. Thus, most of the items transcribed by adding, substituting more common and understandable words, which led to the clarification of the vague items.
Content validity ratio (CVR) and content validity index (CVI) were used to evaluate the quantitative content validity. To begin, the CVR was calculated and a panel of experts was asked to rate each item on a three-point scale: necessary, useful, and not necessary. In this phase, the content validity ratio was calculated using the Lawshe formula (1975), which is acceptable with a score of 0.64 or greater [22]. CVR will be calculated using the following formula:
The criterion of "relevance" was used for each item on the one-point Likert scale to determine the content validity index. For this purpose, 10 experts were asked to determine the correlation between the questionnaire items according to the subscales of the questionnaire on a Likert scale ranging from one (not relevant) to four (completely relevant). Finally, K* will be calculated as follows, using the agreement ratio for the relevance of each item (I-CVI) and the probability of the chance agreement. According to Polit, the minimum number of evaluators required to calculate kappa using this method is three; the number of evaluators will be 10 in the present study. Kappa values of 0.59–0.40, 0.74–0.60, and > 0.74 will be considered poor, good, and excellent, respectively. In this study, only items with kappa of at least 0.74 will be accepted [23].
Construct validity.
In general, tools are classified as reflective and formative. In reflective tools, the items that make up the dimensions of the tool are conceptually related and structural validity is required to examine the dimensions of the tool. In formative tools, the items that make up the dimensions of the tool are not conceptually related [24]. Modern instrumentation for formative checklists precludes exploratory and confirmatory factor analysis [24]. In this study, different areas of health were involved in evaluating health system preparation exercises, including health, treatment, support, food and drug, so due to the different nature and functional areas of these sections in exercise, there was a lack of conceptual and functional relationship between items of components and eventually this tool had a formative nature.
Tool Reliability.
Six operational exercises at Iran University of Medical Sciences were chosen for this study to test instrument reliability. Two evaluators evaluated each exercise independently. After the evaluators completed their evaluations of all exercises, the collected data was entered into SPSS 22 and the reliability of the instrument was examined using internal consistency and Intraclass Correlation Coefficient (ICC).
Ethical considerations.
This study was approved by the Ethics Committee of Kerman University of Medical Sciences (IR.KMU.REC.1397.351). All participants voluntarily participated in this study. With the permission of the participants, their voices were recorded during the qualitative phase of the study. Names of the participants were not mentioned in the study, and instead of their names, codes were used in the interview texts. Other ethical considerations include explaining the research purpose and assuring the confidentiality of information to research units, voluntary participation in research, presenting research results verbally and in writing to research units and requesting a written informed consent.
Results
Qualitative phase (Item generation)
Step 1: Qualitative study.
Table 1 shows how an original theme of the exercise design, four main categories, and eleven subcategories were formed. In the qualitative study, 382 codes were extracted from the concept evaluation of disaster exercises in the health sector.
Step 2: Systematic review.
The initial search of the literature in electronic databases yielded 5578 documents. The number of documents decreased to 2789 articles after duplicates, books, dissertations, and presentations were filtered. First, the titles and abstracts were reviewed to find those related to the evaluation of health exercises for emergencies and disasters and 123 eligible articles were extracted based on systematic screening. Then, all 123 selected full‑text papers were reviewed, and finally, 10 papers reported evaluation of health preparedness exercises for emergencies and disasters. In a systematic review, 203 codes of the components of evaluation of health preparedness exercises were extracted from 10 related articles.
Results of the second phase of the study
Face and content validities.
To begin, four items were corrected for spelling and no items were removed during the qualitative face validity stage. Following the calculation of the impact score, 63 items with impact scores ≥ 1.5 were considered favorable and kept for further analysis in the quantitative face validity stage.
In order to assess the qualitative content validity, most of the items were transcribed by adding, reducing or replacing some words with more common and understandable words, which led to the clarification of ambiguous items. Items with the same implications were removed and some of them were merged because of overlapping. At these stages, 8 items were merged into other items, and 55 items were kept for further analysis. Based on the opinions of 10 experts, the CVR was considered significant during the quantitative content validity stage (> 0.59). Thus, three items were removed. At the end of this stage, the number of items in the questionnaire was reduced to 52 ones. According to Polit, two items with Kappa coefficients > 0.74 were omitted [23], bringing the total number of items to 50. Then, based on the mean CVI scores of all items, the mean CVI of the whole tool was calculated, with 0.9 being the acceptable standard [25]. Table 2 shows that 0.97 obtained in the present study is acceptable.
Tool reliability results.
Internal consistency of the dimensions of the tool for evaluation of disaster exercise design was more than 0.7 and the internal consistency of the whole tool was 0.89, indicating high reliability. ICC was also used to calculate the stability of the tool, which was higher than 0.75 for all questions. ICC of the whole tool was 0.90, indicating that the DEDSET was reliable (Tables 3 and 4).
The final tool.
The checklist for evaluation of exercise design in the healthcare system includes 50 items divided into four dimensions of "Coordination, Command and Conduction of Exercise", "Exercise Hardware and Software Requirements", Organizational Resources of Exercise "," Communication and General Informing of Exercise ". The checklist items were weighed and scored based on the opinions of the experts experienced in disaster response in Iran. The checklists are scored based on the functions that are expected to be performed in the stage of exercise design.
- Mark No. 2 if the expected function was performed correctly and on time.
- Mark No.1 if the expected function was performed, but its quality or timing was improper.
- Mark No.0 f the expected function was not performed.
The cutoff point of the tool.
According to the total scores of 50 items on the three-point Likert scale, the maximum and minimum scores were 100 and 0, respectively. In this tool, scores 0–33.5 indicate poor preparedness, 33.6–66.5 indicate moderate preparedness, and 66.6–100 indicate good preparedness. In addition, the level of preparedness of each health sector is calculated and interpreted in terms of percentage by using the linear transformation formula, converting the tool’s score into a percentage and comparing it with the maximum and minimum scores of the tool. Poor preparedness is represented by 0–33.5 percent, medium preparedness is represented by 33.5–66.5 percent, and good preparedness is represented by 66.5–100 percent.
Discussion
Coordination, command and conduction of exercise, exercise hardware and software requirements, organizational exercise resources, communication and general exercise information are the main components affecting the design of preparedness exercises in health system. The findings of this study indicated that coordination, command and conduction of exercise were some of the foundations for designing discussion-based and operation-based exercises of the health system in disasters. This finding is consistent with findings of the studies [26, 27]. Other studies have shown that in order for an organization to be fully prepared to respond effectively to an incident by conducting exercises, all constituent organizational units must synchronize and coordinate functions [28, 29].
According to the findings, the main foundation of the exercise program in the design phase is the provision of hardware and software requirements before executing the main exercise. Issues such as risk assessment and prioritization of hazards, provision of a risk map, operational incident plan, early warning program, exercise scenario and execution of a discussion-based exercise before an operation-based exercise should be considered during the design phase of the exercise. Various studies suggest that risk assessment determines the nature and extent of the risk and is based on the analysis of potential hazards as well as the vulnerability of the property, livelihoods and environment involved with risk or potential harm [30, 31].
One of the factors explaining the concept of the exercise design was the provision of organizational resources for exercise. This finding is consistent with findings of the studies [32, 33]. Regarding the preparedness of health organizations through design and implementation of exercise, enhancement of the organizational efficiency is dependent on increasing the human resource efficiency, which in turn is dependent on developing knowledge, skills and creating appropriate behaviors for successful performance in disasters.
Communications and public information on exercise was one of the main categories of interviewing with health experts. According to some research, one of the requirements of organizational preparedness in disasters is the ability to predict appropriate communication between organizations and important locations [34, 35].
There are various tools in the world for the evaluation of disaster exercises in hospitals as part of the healthcare sector. These evaluation tools have been presented as activity-based checklists, but there was no fundamental macro-level tool for all health departments, including treatment, health, support, food and drug to design different types of exercises. For example, Heidaranlu et al. [36] designed and validated a tool for evaluation of the functional preparedness of the hospital in natural hazards. When this tool was compared to the one used in our study, it was discovered that while they were similar in the validation process, they were different in construct validity. Our tool requires no exploratory factor analysis (EFA) or confirmatory factor analysis (CFA) because of the study’s formative nature. The tool mentioned above is used to evaluate the functional preparedness of hospitals, but our tool specifically evaluates the functional preparedness of the entire healthcare sector to respond to disasters through evaluation of the exercise design stage.
Cheung et al. [37] wrote a book called Hospital Preparedness Exercise: Atlas of Resources and Tools. The book covered topics such as exercise design, execution and evaluation, as well as a review of resources available for hospital functions. The book only provided sources for available evaluation tools of hospitals, with no information on the tools’ items, the validity and reliability.
Tang et al. conducted a systematic review of available preparedness tools in their study titled "Development of a tool for evaluation of hospital emergency preparedness in China". In this study, they presented a tool for the evaluation of hospital exercises. The tool did not include any items to examine the functional dimensions of the hospital, and it only dealt with a few dimensions. Preparedness evaluation tools are expected to assess the capacity of hospitals, services and the level of safety required for hospital staff. In addition, they did not provide in-depth information on the validity and reliability of the tool developed in their study. The main limitation of the studies selected in the final analysis for tool development, according to the researcher, was a lack of validity and reliability of the tools [15]. The current tool was developed in the Persian language and then translated into English. Thus, non-Persian speaking researchers validated the translation.
Limitation
This study had several limitations. As our study, samples were disaster exercises in the healthcare system, coordination for the design and execution of disaster exercises required significant costs and the overcoming of the maze of administrative hierarchies in the research quantitative phase for performing tool reliability. Some operational exercises were done with the help of the research team and communication with the Emergency Operations Center (EOC) in Iran’s health ministry in order to solve this problem. The second limitation was the lack of factor analysis with varimax rotation of the items to improve definition of each measurement in the categories and subcategories. Therefore, further research and investigation are needed for advanced validation.
Conclusion
The study’s tool was developed as a result of interviews with twenty-five disaster professionals in health field, as well as a literature review and validation procedure. Our study tool includes the most important indicators and dimensions required for disaster exercises design in the health sector, including exercise coordination, command and guidance, exercise hardware and software requirements, organizational resources of the exercise, and communications and general informing of the exercise. Therefore, the present tool can play an important role in increasing the level of preparedness and improving the operational capacity of health and treatment against disasters by evaluating accurately the disaster exercise design stage in the healthcare sector.
In this study, we only did the preliminary validation for the developed tool, it is recommended that a study on the implementation of construct validity should be conducted to complete the validation process.
Acknowledgments
The authors would like to express their gratitude to all the disaster specialists who participated in this study.
References
- 1. Sheikhbardsiri H, Yarmohammadian MH, Khankeh H, Khademipour G, Moradian MJ, Rastegarfar B, et al. An operational exercise for disaster assessment and emergency preparedness in south of Iran. Journal of Public Health Management and Practice. 2020;26(5):451–6. pmid:32732718
- 2. Verheul ML, Dückers ML. Defining and operationalizing disaster preparedness in hospitals: a systematic literature review. Prehospital and disaster medicine. 2020;35(1):61–8. pmid:31826788
- 3. Yousefian S, Sohrabizadeh S, Jahangiri K. Identifying the Components Affecting Intra-Organizational collaboration of Health sector in Disasters: Providing a Conceptual Framework Using a Systematic Review. International Journal of Disaster Risk Reduction. 2021:102146.
- 4. Bazyar J, Pourvakhshoori N, Safarpour H, Farrokhi M, Khankeh HR, Daliri S, et al. Hospital disaster preparedness in Iran: a systematic review and meta-analysis. Iranian journal of public health. 2020;49(5):837–50. pmid:32953672
- 5. Guha-Sapir D, Vos F, Below R, Ponserre S. Annual disaster statistical review 2011: the numbers and trends. 2012.
- 6. Pourvakhshoori N, Norouzi K, Ahmadi F, Hosseini M, Khankeh H. Nursing in disasters: A review of existing models. International emergency nursing. 2017;31:58–63. pmid:27423385
- 7. Nejadshafiee M, Bahaadinbeigy K, Kazemi M, Nekoei-Moghadam M. Telenursing in incidents and disasters: a systematic review of the literature. Journal of emergency nursing. 2020;46(5):611–22. pmid:32360295
- 8. Roshani D, Karimian A. Earthquake Preparedness in Iranian Hospitals: A Systematic Review and Meta-Analysis. Bulletin of Emergency & Trauma. 2021;9(1):1. pmid:33937419
- 9. Gin JL, Levine CA, Canavan D, Dobalian A. Including homeless populations in disaster preparedness, planning, and response: a toolkit for practitioners. Journal of public health management and practice. 2020.
- 10. Aminizadeh M, Farrokhi M, Ebadi A, Masoumi GR, Kolivand P, Khankeh HR. Hospital preparedness challenges in biological disasters: A qualitative study. Disaster Medicine and Public Health Preparedness. 2020:1–13. pmid:33148363
- 11. Goniewicz K, Goniewicz M, Włoszczak-Szubzda A, Burkle FM, Hertelendy AJ, Al-Wathinani A, et al. The importance of pre-training gap analyses and the identification of competencies and skill requirements of medical personnel for mass casualty incidents and disaster training. BMC public health. 2021;21(1):1–11. pmid:33388037
- 12. Ghanizadeh G, Bahadori M, Hosseini-Shokouh S-M. Comparison of Earthquake Disaster Preparedness in Tehran Military and Non-military Hospitals. Journal of Military Health Promotion. 2020;1(3):135–46.
- 13. Husna C, Kamil H, Yahya M, Tahlil T, Darmawati D. Does Tabletop Exercise Enhance Knowledge and Attitude in Preparing Disaster Drills? 2020. 2020;10(2):9.
- 14. Jenkins JL, Kelen GD, Sauer LM, Fredericksen KA, McCarthy ML. Review of hospital preparedness instruments for National Incident Management System compliance. Disaster medicine and public health preparedness. 2009;3(S1):S83–S9. pmid:19491593
- 15. Tang R, Fitzgerald G, Xiang-Yu H, Yang-Ping W. Building an evaluation instrument for China’s hospital emergency preparedness: A systematic review of preparedness instruments. Disaster medicine and public health preparedness. 2014;8(1):101–9. pmid:24622400
- 16. Jenckes MW, Catlett CL, Hsu EB, Kohri K, Green GB, Robinson KA, et al. Development of evaluation modules for use in hospital disaster drills. American journal of disaster medicine. 2007;2(2):87–95. pmid:18271157
- 17. Nekoie-Moghadam M, Kurland L, Moosazadeh M, Ingrassia PL, Della Corte F, Djalali A. Tools and checklists used for the evaluation of hospital disaster preparedness: A systematic review. Disaster medicine and public health preparedness. 2016;10(5):781–8. pmid:27231031
- 18. Sheikhbardsiri H, Doustmohammadi MM, Mousavi SH, Khankeh H. Qualitative Study of Health System Preparedness for Successful Implementation of Disaster Exercises in the Iranian Context. Disaster Med Public Health Prep. 2020:1–10. pmid:33023696
- 19. Hoominfar E, Radel C. Contested Dam Development in Iran: A Case Study of the Exercise of State Power over Local People. Sustainability. 2020;12(13):5476.
- 20. Page MJ, McKenzie J, Bossuyt P, Boutron I, Hoffmann T, Mulrow C, et al. Updating guidance for reporting systematic reviews: development of the PRISMA 2020 statement. 2020.
- 21. Lang TA, Hodge M, Olson V, Romano PS, Kravitz RL. Nurse–patient ratios: a systematic review on the effects of nurse staffing on patient, nurse employee, and hospital outcomes. JONA: The Journal of Nursing Administration. 2004;34(7):326–37.
- 22. Lawshe CH. A quantitative approach to content validity. Personnel psychology. 1975;28(4):563–75.
- 23. Polit DF, Beck CT. The content validity index: are you sure you know what’s being reported? Critique and recommendations. Research in nursing & health. 2006;29(5):489–97. pmid:16977646
- 24.
Polit DF, Yang F. Measurement and the measurement of change: a primer for the health professions: Wolters Kluwer Philadelphia, PA; 2016.
- 25. Polit D, Beck T, Owen S. Focus on research methods is the CVI an acceptable indicator of content validity. Res Nurs Health. 2007;30:459–67. pmid:17654487
- 26. Kaji AH, Lewis RJ. Assessment of the reliability of the Johns Hopkins/Agency for Healthcare Research and Quality hospital disaster drill evaluation tool. Annals of emergency medicine. 2008;52(3):204–10. e8. pmid:17933427
- 27. Norman I, Aikins M, Binka F, Nyarko K. Hospital all-risk emergency preparedness in Ghana. Ghana medical journal. 2012;46(1). pmid:22605887
- 28. Perry M. Natural disaster management planning: A study of logistics managers responding to the tsunami. International Journal of Physical Distribution & Logistics Management. 2007.
- 29. Rüter A, Kurland L, Gryth D, Murphy J, Rådestad M, Djalali A. Evaluation of disaster preparedness based on simulation exercises: a comparison of two models. Disaster medicine and public health preparedness. 2016;10(4):544–8. pmid:26841703
- 30.
Organization WH. Hospital and health facility emergency exercises: guidance materials. Manila: WHO Regional Office for the Western Pacific; 2010.
- 31. Laberge M, Spence N. The evacuation of two rural hospitals following a tsunami warning: What happened next. Journal of business continuity & emergency planning. 2019;12(4):316–30. pmid:31200795
- 32. Maher A, Yeganeh SM, Lari MA, Sayedin SH. The study of the quality and capacity of equipments’ functionality and non-structural vulnerability in selected Tehran general hospitals during an earthquake. International Journal of Health System and Disaster Management. 2014;2(2):93.
- 33. Zaboli R, Sh T, Amerion A, Moghaddasi H. Survey of Tehran city hospitals disaster preparedness for disaster. Journal Mil Med. 2006;8(2):103–11.
- 34. Seba A, Nouali-Taboudjemat N, Badache N, Seba H. A review on security challenges of wireless communications in disaster emergency response and crisis management situations. Journal of Network and Computer Applications. 2019;126:150–61.
- 35.
Sutton JN, Spiro ES, Johnson B, Fitzhugh SM, Greczek M, Butts CT, editors. Connected communications: Network structures of official communications in a technological disaster. ISCRAM; 2012.
- 36. Esmail Heidaranlu B, Ebadi A, Ardalan A, Khankeh H. A scrutiny of tools used for assessment of hospital disaster preparedness in Iran. American journal of disaster medicine. 2015;10(4):325–38. pmid:27149314
- 37. Cheung M VA-T VD, Xiong W, Hupert N. Hospital Preparedness Exercises: Atlas of Resources and Tools 2010.