https://orcid.org/0000-0002-0091-3323
Figures
Abstract
Antidote stocking represents a major challenge to hospitals all over the world, including Kuwait. In order to assist hospitals to reduce costs and improve patient care, an essential antidote list can be used as an initial foundation for securing sufficient antidote availability at healthcare institutions. The aim of our study is to generate a nationally relevant essential antidote list for emergency care hospitals in Kuwait using the e-Delphi method by establishing consensus through a multidisciplinary expert group of healthcare providers. An electronic survey with 47 essential antidotes was developed. The e-Delphi method was used, with three rounds of voting, to determine expert consensus on an essential antidote list for hospitals in Kuwait. A purposive sample of healthcare professionals from governmental and private hospitals were selected for this study (n = 30). Consensus was gained if ≥75% of the expert panel agreed on the inclusion of the antidote, without any strong disagreements. Round 1 of the e-Delphi resulted in 41 antidotes reaching consensus and seven new antidotes suggested by the expert panel. Round 2 had two antidotes (out of seven newly suggested ones) reaching consensus. Round 3 was a confirmatory round, where the expert group agreed on their previous rounds’ opinions. This resulted in the development of an essential antidote list with 43 antidotes. The optimal approach for ensuring adequate availability of antidotes is continuous monitoring of local poisoning incidence and antidote requirements through collaborations between academic researchers and emergency care clinicians. The development of an essential antidote list, with expert consensus, is one of the initial steps in securing a foundation for appropriate provision of antidotes at all healthcare institutions. This is the first study that the authors are aware of that demonstrates that the e-Delphi technique can consolidate recommendations of experts in emergency medicine to provide a list of essential antidotes.
Citation: Al-Taweel D, Koshy S, Al-Ansari S, Al-Haqan A, Qabazard B (2022) Expert consensus for a national essential antidote list: E-Delphi method. PLoS ONE 17(6): e0269456. https://doi.org/10.1371/journal.pone.0269456
Editor: Menno I. Gaakeer, ADRZ: Admiraal De Ruyter Ziekenhuis, NETHERLANDS
Received: May 22, 2021; Accepted: May 20, 2022; Published: June 16, 2022
Copyright: © 2022 Al-Taweel et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Poisoning is a paramount public health issue that is responsible for frequent visits to the emergency department and hospital admissions. Effective management of poisoning requires adequate availability of antidotes, with supportive measures, in hospitals that provide effective emergency care [1, 2]. Antidotes have a critical role in the emergency care of a patient admitted for poisoning or drug overdose, as it prevents death, and decreases the duration of therapy and hospital stay. Studies have reported inadequate availability of antidotes in emergency care facilities, which experts warn will lead to increased morbidity and mortality [1, 3–5]. Unfortunately, the stocking of antidotes has been a “persistent concern” for over 35 years [1]. Countries have conducted regular audits on antidotes stocking, and many researchers have recommended the development of a national list and stocking guidelines to standardize care [4, 6, 7]. A globally recognized consensus guideline recommended a “list” of antidotes to be stocked for timely delivery of essential antidotes to the emergency department [1]. It is advocated that this list must be modified and adopted by each healthcare institute based on a hazard vulnerability assessment (HVA) to meet local antidote needs [1, 8]. It is also highly recommended that the list should be determined by the concerned stakeholders consisting of physicians, pharmacists, and other healthcare professionals associated with emergency care [1, 9]. Organizing a meticulous HVA is challenging as it is resource-intensive and time-consuming, requiring comprehensive face-to-face interactions. It requires collaboration between hospitals and other institutions as each hospital is challenged with its own specific geographic, political and social features. In the absence of a rigorous HVA, the list of essential antidotes can be created by reaching consensus within expert groups, by employing methods such as the Delphi technique, the nominal group technique or through a consensus development conference [10].
The Delphi method was developed in the 1950s and became extensively accepted for gathering and merging opinions on a particular topic to set goals and investigate policies. It is well adapted for building consensus, with a succession of questionnaires, to procure data from a group of selected members [11]. The Delphi method has been used to establish consensus in emergency medicine, including antidote stocking [12–14]. The e-Delphi method is a computerized version of the Delphi method that depends on an internet-based platform offering tremendous savings in time and cost, and advantages in data management [15].
In Kuwait, a country with a population of 4.3 million [16], approximately 1150 cases of poisonings due to medications and chemicals are reported annually [17]. A prospective study in Kuwait indicated an increasing prevalence of poisoning cases, with 57.8% of patients admitted for intentional poisoning, and 42.2% for unintentional poisoning [17]. Drug overdose toxicity is the third most common cause of accidental deaths in the country [18]. A 2010 study in Kuwait reported that approximately 35% to 50% of poisoning cases between 2000 and 2005 were pediatric poisonings due to drugs and biological substances [19]. A 2020 national audit of antidote stocking in Kuwait concluded that there is inadequate stocking of some antidotes in both the public and private hospital sector. The same study recommended the development of national guidelines for antidote stocking in Kuwait and the customization of guidelines to suit the local poisoning cases [8]. To assist hospitals to reduce costs and improve patient care, there is an urgent need to develop expert consensus guidelines, including a locally relevant list of essential antidotes, through a systematic, explicit, and transparent decision-making process [8].
The aim of our study is to generate a nationally relevant list of essential antidotes to secure adequate availability of antidotes in hospitals that provide emergency care in Kuwait using the e-Delphi technique. To our knowledge, this is the first study that uses an e-Delphi technique to consolidate recommendations of experts in emergency medicine to provide a list of essential antidotes for the management of adult patients with poisoning.
Methods
Study design and population
The e-Delphi method was used in this study, with three rounds of voting, to determine expert consensus on the development of an essential antidote list for hospitals in Kuwait that provide emergency care services. Ethics approval for this study was provided by the Health Science Centre Ethics Committee at Kuwait University as well as the Ministry of Health Ethics committee (Ethics number: 2019/1225).
A purposive sample of healthcare professionals from different geographical regions across Kuwait and with diverse specialties (including medical toxicology, emergency medicine, internal medicine, critical care medicine, pediatrics, pediatric critical care, hospital pharmacy and clinical pharmacy) from both governmental and private hospitals were selected for this study to form the expert panel. The additional inclusion criteria were that healthcare professionals were qualified as experts if they had i) at least 2 years’ post-qualification experience (unless they were an emergency medicine resident), ii) practiced in Kuwait for at least 2 years. Thirty experts were initially contacted and asked to participate in consensus development.
Survey content and administration
A questionnaire was developed using the commercial survey software Qualtrics® (Provo, USA) with a preliminary list of 47 antidotes. This initial list was composed of antidotes retrieved from a national audit conducted in Kuwait in 2020 [8], that was based on evidence-based international guidelines [1, 20]. The research group (which included an academic toxicologist) reviewed in-depth the initial list and made necessary additions and deletions, as needed. Additionally, before commencing the Delphi process, the Delphi questionnaire was piloted with three emergency care physicians for their expert opinions. The purpose of this step of the process was to identify antidotes applicable for Kuwait and ensure that the Delphi process was evidence-based as well as consultative.
The survey instrument was composed of three sections. The first section collected the experts’ demographic data (e.g. site of practice, specialties, and position) in addition to their opinion on whether Kuwait needs to have its own list of essential antidotes and their knowledge on the existence of an essential antidote list. The second section included a list of 47 antidotes, with their relevant indication for use. Experts were asked to rate their level of agreement on whether to include each antidote in the essential antidote list for Kuwait on a four-point Likert scale (strongly agree/agree/disagree/strongly disagree). It has been demonstrated that four-point scales produce stable findings in Delphi studies [21]. In the third section, experts were asked to add any new antidotes that they considered to be needed in the essential list of antidotes in Kuwait that were not on our initial list. A comments box was added to allow experts to write their opinions.
Data were collected via an online questionnaire through three rounds between March-November 2020. The study survey was pre-tested on three experts; a specialist in emergency medicine, an internist and family medicine physician. They were asked to comment on the content, clarity and phrasing of the questions as well as the layout of the questionnaire. Their feedback informed the decision to include the indications for each antidote (as some antidotes are likely to have other indications not associated with emergency care).
A formal iterative process was conducted. The survey clearly displayed the objectives of the study and provided clear instructions for the experts on how to complete the survey. Experts were informed that they had the freedom to withdraw from the study at any time. As no consistent threshold for consensus exists [22], the following threshold was defined for our study: “Consensus was gained if ≥75% of the experts agreed on the inclusion of the antidote, provided there was no strong disagreement in the results”. Although the literature has reported many cut-offs for Delphi studies [23, 24], systematic reviews by Diamond et al. and Foth et al. have concluded that consensus is most commonly defined based on the percentage of agreement with a specific criterion, followed by the percentage of participants who rate items at the upper extremes of the Likert scales used (i.e. items scored agree or strongly agree on a Likert scale) [25, 26]. However, the range reported for consensus acceptance is very wide (50–97%) [27]. Experts were given two weeks to respond, with two reminders sent two weeks apart. The second and third questionnaire rounds were sent 4 weeks after the previous round was finished. To complete the e-Delphi process, experts were required to respond across all three rounds. Therefore, those who did not respond to Round 1 were not invited to participate in Round 2.
e-Delphi rounds
Round 1 involved circulating the electronic survey to 30 experts. Experts indicated their level of agreement on the inclusion of each antidote in the essential list of antidotes for Kuwait and were also provided with the opportunity to add any new antidotes they believe should be included in the essential antidotes list. Experts were given two weeks to respond, with a reminder provided after one week. The results of Round 1 of the Delphi method were collated, and a summary of the responses was documented to provide a preliminary level of group consensus for each item.
A second questionnaire, showing the summary response of the whole group and the consensus level, was sent back to the experts in Round 2. The experts had the chance to reflect on their previous answers and re-rank their initial judgment in light of the group’s response. Any antidote that had not reached consensus or had gained a “strongly disagree” vote in Round 1, was re-rated. Additionally, any newly suggested antidotes from Round 1 were also voted on.
In Round 3, experts were provided with a summary response of the whole group for Round 2 and the consensus level. Experts were, again, given the opportunity to reflect on their answers for Round 2 and re-rank their initial judgment in light of the group’s response. Any antidote that had not reached consensus or had gained a “strongly disagree” vote in Round 2, was re-rated. Additionally, any newly suggested antidotes from Round 1 were also voted on.
Patient and public involvement
Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.
Data analysis
Data was exported from Qualtrics® to Microsoft Excel for analysis. Descriptive analysis was used to present the experts demographic characteristics as well as their levels of agreement for the inclusion of each individual antidote in numbers and percentages. Antidotes that were rated “strongly agree” or “agree” by at least 75% of the experts were included in the final list.
Results
Of the 30 experts invited to participate in the study, 22 experts agreed to participate in our study. All 22 experts completed Round 1 and Round 2 (100% response rate), and 20 out of 22 experts completed Round 3 (90.9% response rate). Table 1 shows the demographics of the expert group in all three rounds. Gender distribution was consistent across all three rounds, with a higher percentage of males. More than half of the experts worked in the government sector, and specialized in emergency medicine. The majority of experts participating in the study were medical doctors, with only three pharmacists taking part.
When asked whether Kuwait needs to have its own list of essential antidotes, all experts, apart from one, agreed on Kuwait having its own essential antidote list. The participant who did not agree stated that in his opinion, “given the landscape, a more regional list may be more adequate”. Moreover, when experts were asked about whether they were aware of any essential antidote list currently used in Kuwait, the experts unanimously reported that they were not aware of any list or guidelines for antidote availability in Kuwait.
e-Delphi rounds
Fig 1 shows the step-by-step process of developing the expert consensus list of essential antidotes in Kuwait.
Details on each round are provided below:
Round one of the e-Delphi study asked the experts to rate their level of agreement to the inclusion of 47 antidotes in the essential antidotes list for Kuwait. In addition, experts were asked to suggest any essential antidote they believed should be added to the list. Overall, out of the 47 antidotes, the panel recommended 41 antidotes to be included in the essential antidote list; only 6 antidotes did not reach consensus (% consensus was <75% or had at least one strongly disagree answer), these were: acetylcysteine PO, dicobalt edetate, black widow spider antivenin, ethanol IV, mesna and Prussian blue. The six antidotes were re-assessed during Round 2. In addition, the experts suggested seven new antidotes to be added to the essential antidote list, these were: sugammadex, unithiol, glucarpidase, hyperbaric oxygen therapy, prothrombin complex concentrate, uridine triacetate and calcium trisodium. These were added to the questionnaire in round two.
Round two of the e-Delphi study consisted of two parts. The first part asked the experts to reflect on their ratings for those antidotes that did reach consensus in Round 1 (based on the group’s consensus level), and part two asked them to re-rate the 6 antidotes that did not reach consensus in Round 1 as well as the 7 newly suggested antidotes. All experts were happy with their Round 1 results for those antidotes that reached consensus (n = 41), and those were therefore added to the final list of essential antidotes in Kuwait (and were not included in Round 3). With regards to the 6 antidotes that did not reach consensus in Round 1, experts confirmed their responses and did not agree on having them added to the essential antidote list. The 6 antidotes were therefore removed and were not included in Round 3. With regards to the 7 newly suggested antidotes, 2 antidotes reached consensus (sugammadex and prothrombin complex concentrate) while 5 did not reach consensus. These antidotes were reviewed in Round 3.
Round three of the e-Delphi study consisted of a reflection on the 2 antidotes that did reach consensus from Round 2, and a re-rating of the 5 antidotes that did not reach consensus (unithiol, glucarpidase, hyperbaric oxygen therapy, uridine triacetate and calcium trisodium). The expert panel confirmed their consensus on the inclusion of both sugammadex and prothrombin complex concentrate in the essential antidote list for Kuwait. However, they did not reach consensus on the inclusion of the 5 antidotes mentioned above in the essential list of antidotes in Kuwait–those 5 antidotes were therefore removed. Table 2 provides the consensus level for each antidote in each round.
In total, 43 antidotes were included in the final list of essential antidotes for Kuwait. The final list of essential antidotes is provided in Table 3.
Discussion
Effective management of poisoning requires adequate stocking of antidotes in hospitals that provide emergency care. Antidote stocking represents a major challenge to hospitals all over the world, including Kuwait. It has been reported that public and private hospitals in Kuwait have suboptimal stocks of essential antidotes [8]. In order to assist hospitals to reduce costs and improve patient care, this study aimed to develop an essential antidote list for Kuwait through expert consensus using the e-Delphi method. To our knowledge, this study is the first study to use the e-Delphi technique in the Europe, Middle East and Africa (EMEA) region to generate a nationally relevant list of antidotes by multidisciplinary expert consensus, to secure adequate availability of antidotes in emergency care hospitals.
One of the characteristics of the Delphi method for reaching consensus is that it does not generate right or wrong answers or any definitive answers but rather, a valid expert opinion on each standard [28]. It has the advantage of including large, diverse groups, that are geographically dispersed, to obtain a reliable data through a large number of viewpoints, while preserving anonymity [10]. One unique advantage of the Delphi technique is the provision of controlled feedback to the expert panel, where obtained data is analyzed and presented in an easily interpretable format to all the experts after every round. This controlled feedback ensures that the experts view their previous results against the group results and have the opportunity to change their responses if needed. This ensures stability of results when final closure is done [29]. Specifically, the e-Delphi method is beneficial whenever the judgment of experts is needed but face-to-face interaction is not feasible.
Our expert panel consisted of 22 experts in Round 1 and 2, and 20 experts in Round 3 (9% dropout rate). Acceptable dropout rates in Delphi studies have been reported to be 20% across three rounds of consensus development [21, 30, 31]. In addition, literature on Delphi studies shows that a minimum of twelve experts is generally considered acceptable to enable consensus achievement, as Delphi sample sizes depend more on group diversity in reaching consensus than statistical power [32]. This highlights the reason why our purposive sample of 22 experts was needed, as they were geographically dispersed around the healthcare regions in Kuwait, and from both the public and private sector with diverse specialties.
Our study’s preliminary antidote list (that contained 47 antidotes) was based on the results of a national audit of antidotes in Kuwait that looked at the availability of antidotes in public and private hospitals that provide emergency care in 2020 [8]. In this study, Round 1 resulted in 41 antidotes out of 47 antidotes to be included in the final list of essential antidotes for Kuwait, while Round 2 and 3 provided consensus on two of the newly suggested antidotes by the expert panel, giving a final list of 43 antidotes. A total of 11 antidotes did not reach consensus in this study, despite going through at least 2 rounds. Reasons behind some antidotes not reaching consensus are: i) less costly alternatives available (e.g IV acetylcysteine more affordable than PO) [33], ii) a more acceptable (more tolerable) alternative available (e.g. IV acetylcysteine more tolerable than PO, with shorter course of therapy) [33], iii) risky side effects (e.g. dicobalt edetate and black widow spider antivenin) [34–37] iv) sub-specialized need (e.g. mesna needed for cyclophosphamide toxicity, which is used mostly in cancer care) [34] and v) world-wide unavailability (e.g. Prussian blue) [38].
Two new antidotes reached the consensus to be added to the essential antidotes list in Kuwait, these were: sugammadex and prothrombin complex concentrate. Sugammadex is a novel antidote used for the immediate reversal of neuromuscular blockade induced by aminosteroid muscle relaxants such as rocuronium, vecuronium and pancuronium [39]. The expert panel also agreed on the inclusion of prothrombin complex concentrate (PCC) in the essential antidote list for Kuwait. PCC, indicated for rapid anticoagulation reversal of vitamin K antagonists particularly in the emergent setting (e.g., intracranial hemorrhage or a need for urgent invasive surgery), has been recommended over the classical treatment with fresh frozen plasma (FFP) and vitamin K owing to the smaller volume of PCC needed compared to FFP [40]. Another indication for PCC, although not FDA approved yet, is the reversal of direct oral anticoagulants (DOAC)-induced anticoagulation when a more specific antidote is unavailable [41].
The list of essential antidotes is a tool that would support healthcare facilities to be better prepared in the management of poisoned patients in emergency care departments as the availability of an antidotes essential list will help emergency physicians and other healthcare providers to provide optimum and timely care for patients with risk of poisoning. Emergency care and hospital managers may aim to ensure appropriate stocks are available of these essential antidotes. Future work has to be conducted to generate a minimum stock level to treat an adult patient for 24 hours, as well as cost-effectiveness studies. Pilot testing of the final list of essential antidotes before widespread implementation would ensure optimal feasibility and applicability. In addition, this essential list of antidotes would need to be updated regularly to take into consideration any changes in antidote needs by the poisoned population and availability or supply.
This is the first study in Kuwait that aims to provide a useful and clinically relevant list of essential antidotes based on an expert consensus for hospitals that provide emergency in Kuwait. The main strength of this study is the use of the e-Delphi method, as it proved to be ideal in establishing expert consensus on our study subject, and is a valid method to establish consensus for development of a national essential antidote list and can be used in different countries. Development of a national essential antidote list required a process that is evidence-based as well as consultative. We believe our methodology has displayed a model that integrated best evidence [1, 20] with a consultative process for consensus development–the e-Delphi method. The e-Delphi survey, conducted over three rounds with experts from diverse care settings, ensured a reliable collection of professional inputs on this important subject matter. One advantage of the Delphi method is that it offers anonymity which helps to eliminate the fear or influence of a dominating expert [42] which can lead to erroneous data collection and defective consensus [43]. In particular, the use of the e-Delphi method has been proven to remove certain biases that are observed with other face-to-face consensus processes such as the nominal group approach [44]. Performing a local/regional Delphi study will ensure that all countries have the required stock and type of antidote to meet their local needs. Ideally, hospitals should conduct their own HVA to determine their antidote requirements, however, as mentioned earlier, this can be time intensive and costly.
One of the limitations of this study is that there are not enough updated epidemiological studies for the incidence of poisoning in Kuwait. This can justify the reason for some antidotes not reaching consensus or having at least a strongly disagree answer such as mesna, unithiol and CA-DTPA. The need to conduct a hazard vulnerability assessment in hospitals of Kuwait is paramount to understand the scale of poisoning locally.
Conclusions
This study has highlighted the importance of development of an essential antidote list for the management of poisoned patients by physicians at emergency care departments in Kuwait through a reliable, iterative process using the e-Delphi method. It reemphasizes the need for continuous planning and evaluation of the local needs of antidotes for poisoning cases in countries, by the successful collaboration of academic expertise with clinical practitioners’ experience to develop the most appropriate evidence-based framework. The optimal approach for ensuring appropriate stocking of antidotes and, hence, successful management of poisoning would be the utilization of the recommendations from the expert panel consensus together with an antidote hazard vulnerability assessment specific for each hospital. Future work should determine the applicability of the list across hospitals in Kuwait and the impact of list adoption on the management of actual poisoning cases.
Acknowledgments
The authors would like to acknowledge all members of the expert group for their participation in this study.
References
- 1. Dart RC, Goldfrank LR, Erstad BL, Huang DT, Todd KH, Weitz J, et al. Expert Consensus Guidelines for Stocking of Antidotes in Hospitals That Provide Emergency Care. Annals of emergency medicine. 2018;71(3):314–25 e1. pmid:28669553
- 2. Thanacoody RH, Aldridge G, Laing W, Dargan PI, Nash S, Thompson JP, et al. National audit of antidote stocking in acute hospitals in the UK. Emerg Med J. 2013;30(5):393–6. pmid:22875840
- 3. Al-Sohaim SI, Awang R, Zyoud SH, Rashid SM, Hashim S. Evaluate the impact of hospital types on the availability of antidotes for the management of acute toxic exposures and poisonings in Malaysia. Hum Exp Toxicol. 2012;31(3):274–81. pmid:21478291
- 4. Mansour A, Al-Bizri L, El-Maamary J, Al-Helou A, Hamade R, Saliba E, et al. National study on the adequacy of antidotes stocking in Lebanese hospitals providing emergency care. BMC Pharmacol Toxicol. 2016;17(1):51. pmid:27817746
- 5. Ong HC, Yang CC, Deng JF. Inadequate stocking of antidotes in Taiwan: is it a serious problem? J Toxicol Clin Toxicol. 2000;38(1):21–8. pmid:10696920
- 6. Salem WA SR, Qureshi I, Al-Bukari AA, Shaat EBJ, Moinudheen J et al. Appropriate Utilization and Stocking of Antidotes in Qatar Public Hospitals. Asia Pacific Journal of Medical Toxicology. 2017;6(3):72–8.
- 7. AlTamimi A, Malhis NK, Khojah NM, Manea SA, AlTamimi A, AlShammary SA. Antidote Availability in Saudi Arabia Hospitals in the Riyadh Province. Basic Clin Pharmacol Toxicol. 2018;122(2):288–92. pmid:29068170
- 8. Al-Taweel D, Al-Haqan A, Qabazard B, Abuzainah D, Koshy S, Alcienae F, et al. Availability of Antidotes in Kuwait: A National Audit. J Emerg Med. 2020;58(2):305–12. pmid:31818607
- 9. Marraffa JM, Cohen V, Howland MA. Antidotes for toxicological emergencies: a practical review. Am J Health Syst Pharm. 2012;69(3):199–212. pmid:22261941
- 10.
WHO. WHO Handbook for Guideline Development 2014 [Available from: https://apps.who.int/iris/handle/10665/145714.
- 11. Hsu C-C SB. The Delphi Technique: Making Sense Of Consensus. Practical Assessment, Research and Evaluation. 2007;12(10).
- 12. Sohn CH, Ryoo SM, Lim KS, Kim W, Lim H, Oh BJ. Kind and estimated stocking amount of antidotes for initial treatment for acute poisoning at emergency medical centers in Korea. J Korean Med Sci. 2014;29(11):1562–71. pmid:25408590
- 13. Dart RC, Goldfrank LR, Chyka PA, Lotzer D, Woolf AD, McNally J, et al. Combined evidence-based literature analysis and consensus guidelines for stocking of emergency antidotes in the United States. Annals of emergency medicine. 2000;36(2):126–32. pmid:10918103
- 14. O’Donnell SM CA, Adams J, Long E, Babl FE. Delphi assessment of audit and research priorities in an emergency department. Emerg Med Australas. 2020;32(4):556–61. pmid:32020705
- 15. Donohoe H SM, Tennant B. Advantages and Limitations of the e-Delphi Technique. American Journal of Health Education. 2012;43:38–46.
- 16.
Central Statistical Bureau. Population Estimates of Kuwait 2020 [Available from: https://www.csb.gov.kw/Pages/Statistics_en?ID=67&ParentCatID=1.
- 17. Al-Mutawa A, Abahussain E, Hedaya M, S. K. Intentional and unintentional drug poisonings in Mubarak Al-Kabeer hospital, Kuwait. J Pharma Care Health Sys. 2015;2(3).
- 18. Al-Kandary N, Al-Waheeb S. Patterns of accidental deaths in Kuwait: a retrospective descriptive study from 2003–2009. BMC public health. 2015;15:302. pmid:25884428
- 19. Abahussain EA, Ball DE. Pharmaceutical and chemical pediatric poisoning in Kuwait: a retrospective survey. Pharmacy practice. 2010;8(1):43–9. pmid:25152792
- 20.
Royal College of Emergency Medicine and National Poisons Information Service Guideline on Antidote Availability for Emergency Departments (2017).
- 21. Akins RB TH, Cole BR. Stability of response characteristics of a Delphi panel: application of bootstrap data expansion. BMC Med Res Methodol. 2005;5(37).
- 22. James D, Warren-Forward H. Research methods for formal consensus development. Nurse researcher. 2015;22(3):35–40. pmid:25587865
- 23. Birko S, Dove ES, Ozdemir V. Evaluation of Nine Consensus Indices in Delphi Foresight Research and Their Dependency on Delphi Survey Characteristics: A Simulation Study and Debate on Delphi Design and Interpretation. PLoS One. 2015;10(8):e0135162. pmid:26270647
- 24. Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One. 2011;6(6):e20476. pmid:21694759
- 25. Diamond IR, Grant RC, Feldman BM, Pencharz PB, Ling SC, Moore AM, et al. Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014;67(4):401–9. pmid:24581294
- 26. Foth T, Efstathiou N, Vanderspank-Wright B, Ufholz LA, Dutthorn N, Zimansky M, et al. The use of Delphi and Nominal Group Technique in nursing education: A review. Int J Nurs Stud. 2016;60:112–20. pmid:27297373
- 27. Barrios M GG, Nuno L, Gomez-Benito J. Consensus in the delphi method: What makes a decision change? Technological Forecasting and Social Change. 2020;163.
- 28. Keeney S, Hasson F, McKenna HP. A critical review of the Delphi technique as a research methodology for nursing. Int J Nurs Stud. 2001;38(2):195–200. pmid:11223060
- 29. Nasa P JR, Juneja D. Delphi methodology in healthcare research: How to decide its appropriateness. World J Methodol. 2021;11(4):116–29. pmid:34322364
- 30. Henderson EJ RG. Development of a community-based model for respiratory care services. BMC Health Serv Res 2012;12(193). pmid:22776670
- 31. Vogel C, Zwolinsky S., Griffiths C. et al. A Delphi study to build consensus on the definition and use of big data in obesity research. Int J Obes 2019;43:2573–86. pmid:30655580
- 32. Okoli C PS. The Delphi method as a research tool: an example, design considerations and applications. Information and Management. 2004;42:15–29.
- 33. Marchetti A, Rossiter R. Managing acute acetaminophen poisoning with oral versus intravenous N-acetylcysteine: a provider-perspective cost analysis. J Med Econ. 2009;12(4):384–91. pmid:19916738
- 34. Graham J, Traylor J. Cyanide Toxicity. StatPearls. Treasure Island (FL)2021.
- 35. Marrs TC, Thompson JP. The efficacy and adverse effects of dicobalt edetate in cyanide poisoning. Clin Toxicol (Phila). 2016;54(8):609–14. pmid:27352227
- 36. Offerman SR, Daubert GP, Clark RF. The treatment of black widow spider envenomation with antivenin latrodectus mactans: a case series. Perm J. 2011;15(3):76–81. pmid:22058673
- 37. Murphy CM, Hong JJ, Beuhler MC. Anaphylaxis with Latrodectus antivenin resulting in cardiac arrest. J Med Toxicol. 2011;7(4):317–21. pmid:22052335
- 38. Altagracia-Martínez M K-JJ, Martínez-Núñez JM, Ríos-Castañeda C, López-Naranjo F. Prussian blue as an antidote against radioactives Thallium and Cesium poisoning. Orphan Drugs: Research and Reviews. 2012;2:13–21.
- 39. Akha AS, Rosa J 3rd, Jahr JS, Li A, Kiai K. Sugammadex: cyclodextrins, development of selective binding agents, pharmacology, clinical development, and future directions. Anesthesiol Clin. 2010;28(4):691–708. pmid:21074746
- 40. Tomaselli GF, Mahaffey KW, Cuker A, Dobesh PP, Doherty JU, Eikelboom JW, et al. 2017 ACC Expert Consensus Decision Pathway on Management of Bleeding in Patients on Oral Anticoagulants: A Report of the American College of Cardiology Task Force on Expert Consensus Decision Pathways. J Am Coll Cardiol. 2017;70(24):3042–67. pmid:29203195
- 41. Allison TA, Lin PJ, Gass JA, Chong K, Prater SJ, Escobar MA, et al. Evaluation of the Use of Low-Dose 4-Factor Prothrombin Complex Concentrate in the Reversal of Direct Oral Anticoagulants in Bleeding Patients. J Intensive Care Med. 2020;35(9):903–8. pmid:30244638
- 42. P Iskandar YH, Mokhtar N. Developing consensus in identifying challenges of undergraduate orthodontic education in Malaysian public universities using e-Delphi. Eur J Dent Educ. 2020;24:590–600. pmid:32374909
- 43. Msibi P.N., Mogale R., De Waal M. & Ngcobo N. Using e-Delphi to formulate and appraise the guidelines for women’s health concerns at a coal mine: A case study. Curationis. 2018;41(1), a1934. https://doi.org/10.4102/curationis.v41i1.1934
- 44. Eubank B.H., Mohtadi N.G., Lafave M.R., Wiley J.P., Bois A.J., Boorman R.S. et al. Using the modified Delphi method to establish clinical consensus for the diagnosis and treatment of patients with rotator cuff pathology. BMC Medical Research Methodology. 2016;16.