Figures
Abstract
Background
Currently, antivenoms are the only specific treatment available for snakebite envenoming. In Brazil, over 30% of patients cannot access antivenom within its critical care window. Researchers have therefore proposed decentralizing to community health centers to decrease time-to-care and improve morbidity and mortality. Currently, there is no evidence-based method to evaluate the capacity of health units for antivenom treatment, nor what the absolute minimum supplies and staff are necessary for safe and effective antivenom administration and clinical management.
Methods
This study utilized a modified-Delphi approach to develop and validate a checklist to evaluate the minimum requirements for health units to adequately treat snakebite envenoming in the Amazon region of Brazil. The modified-Delphi approach consisted of four rounds: 1) iterative development of preliminary checklist by expert steering committee; 2) controlled feedback on preliminary checklist via expert judge survey; 3) two-phase nominal group technique with new expert judges to resolve pending items; and 4) checklist finalization and closing criteria by expert steering committee. The measure of agreement selected for this study was percent agreement defined a priori as ≥75%.
Results
A valid, reliable, and feasible checklist was developed. The development process highlighted three key findings: (1) the definition of community health centers and its list of essential items by expert judges is consistent with the Brazilian Ministry of Health, WHO snakebite strategic plan, and a general snakebite capacity guideline in India (internal validity), (2) the list of essential items for antivenom administration and clinical management is feasible and aligns with the literature regarding clinical care (reliability), and (3) engagement of local experts is critical to developing and implementing an antivenom decentralization strategy (feasibility).
Conclusion
This study joins an international set of evidence advocating for decentralization, adding value in its definition of essential care items; identification of training needs across the care continuum; and demonstration of the validity, reliability, and feasibility provided by engaging local experts. Specific to Brazil, further added value comes in the potential use of the checklist for health unit accreditation as well as its applications to logistics and resource distribution. Future research priorities should apply this checklist to health units in the Amazon region of Brazil to determine which community health centers are or could be capable of receiving antivenom and translate this expert-driven checklist and approach to snakebite care in other settings or other diseases in low-resource settings.
Author summary
Checklists have been developed and validated to improve patient safety and effectiveness of care in several fields, including emergency medicine, intensive care, and surgery. The Brazilian Ministry of Health (MoH) supplies antivenoms (AVs) to the health system at no cost to patients. AV access is thus limited to hospitals, most of which are in urban areas and difficult for rural, remote, and indigenous populations to reach. Currently, there is no evidence-based method to evaluate the capacity of health units for AV treatment, nor what the absolute minimum supplies and staff are necessary for safe and effective AV administration and clinical management. In this study, we aim to develop and validate a checklist to evaluate the minimum requirements for community health centers to adequately treat snakebite envenoming in the Amazon region of Brazil. This study joins an international set of evidence advocating for decentralization, adding value in its definition of essential care items, represented by Human Resources, and Equipment, Supplies and Medicines, to provide safe and effective treatment for SBE patients in remote endemic areas.
Citation: Serrão-Pinto T, Strand E, Rocha G, Sachett A, Saturnino J, Seabra de Farias A, et al. (2024) Development and validation of a minimum requirements checklist for snakebite envenoming treatment in the Brazilian Amazonia. PLoS Negl Trop Dis 18(1): e0011921. https://doi.org/10.1371/journal.pntd.0011921
Editor: Kalana Prasad Maduwage, University of New England, AUSTRALIA
Received: October 10, 2023; Accepted: January 15, 2024; Published: January 19, 2024
Copyright: © 2024 Serrão-Pinto et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All data underlying our findings are fully available in the manuscript files.
Funding: J.S. and W.M. were funded by Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq productivity scholarships). W.M. was funded by Fundação de Amparo à Pesquisa do Estado do Amazonas (PRÓ-ESTADO, call 011/2021 - PCGP/FAPEAM, call 010/2021 - CT&I ÁREAS PRIORITÁRIAS, call 003/2022 - PRODOC/FAPEAM, and POSGRAD RESOLUÇÃO N. 002/2023) and by the Ministry of Health, Brazil (proposal No. 733781/19-035). Research reported in this publication was partly supported by the Fogarty International Center of the National Institutes of Health under Award Number R21TW011944. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
In the Brazilian Amazon, the incidence of snakebite envenoming (SBE), though underestimated, is roughly 30,000 cases per annum [1]. Currently, antivenoms are the only specific treatment available for SBE. The clinical care window for antivenom (AV) effectiveness is six hours [1]. Antivenom administered after this window is less effective in reversing systemic damage and has little or no effect on local tissue damage [2]. Timely access to AV is thus crucial to avoid preventable complications, disabilities, and death [3]. In Amazonia, Brazil, however, over 30% of snakebite patients cannot access AV within six hours [4].
This is due in part to the AV distribution structure. Currently, the Brazilian Ministry of Health (MoH) supplies AVs to the health system at no cost to patients [5]. The MoH first distributes AVs to each state. State health secretaries are responsible for distributing AVs to municipalities, and municipalities then distribute AVs to their hospitals. AV access is thus limited to hospitals, most of which are in urban areas and difficult for rural, remote, and indigenous populations to reach. The national health care system in Brazil, Sistema Único de Saúde (SUS), provides access to care to these populations through community health centers (CHC).
CHCs provide primary care to designated populations [6]. In the remote and rural regions of Amazonia, where snakebites tend to occur, CHCs are often the only health facility [4]. To reduce the delay in antivenom treatment and improve patient outcomes, clinical and research experts in the State of Amazonas have called for a decentralization strategy in which antivenom is supplied to CHCs in addition to hospitals [4,7,8].
Our research team has been utilizing data and implementation science to establish a comprehensive understanding of SBE in the Brazilian Amazonia and develop such a strategy for decentralizing AV treatment to CHCs [1,4,5,7,9–17] (Fig 1). We have proposed: 1) developing and validating a culturally relevant clinical practice guideline, 2) training health professionals according to the guideline, 3) implementing the standardized protocol, 4) assessing perceptions and acceptability of the changes, and 5) estimating the impact on timely AV access. These steps, however, rest on the assumption that CHCs are well-equipped and staffed with personnel trained in administering AV treatment. Currently, there is no evidence-based method to evaluate the capacity of health units for AV treatment [18], nor what the absolute minimum supplies and staff are necessary for safe and effective AV administration and clinical management [8,19].
A) Care pathways for snakebite patients under the current antivenom distribution structure. B) Care pathways under the proposed antivenom decentralization to community health centers (CHCs). Figure built using https://openclipart.org/ as source of the images or icons.
Checklists, specifically criteria of merit checklists [20], have been developed and validated to improve patient safety and effectiveness of care in several fields [21,22], including emergency medicine, intensive care, and surgical disciplines [23–26]. In Brazil, health authorities have established several checklists to ensure adequate capacity and quality of health services: hospitals [27], community health centers [28], surgical services [29], vaccination services [30], intensive care units [31], clinical laboratories [32], and pharmacies [33]. In terms of antivenom decentralization to CHCs, a criteria of merit checklist for the staff and supplies involved in antivenom treatment and subsequent clinical management is needed to determine which CHCs are or could be capable of receiving antivenom, and to guide authorities in accrediting health units in Brazil to perform these procedures.
Our objective was thus to develop and validate a checklist to evaluate the minimum requirements for community health centers to adequately treat snakebite envenoming in the Amazon region of Brazil.
Methods
Ethics statement
The study was approved by the Research Ethics Committee, Fundação de Medicina Tropical Dr. Heitor Vieira Dourado (FMT-HVD; CAAE: 52735721.7.0000.0005, approved on 5 November 2021). Written informed consent was obtained from all participants of the study.
Study design
A modified-Delphi approach was utilized as we contextualized the study to a specific neglected health issue in a specific setting, and the results were time-sensitive in their direct application to the development and implementation of an antivenom decentralization strategy [34]. The checklist was developed in accordance with the best practices outlined by Bichelmeyer, Scriven, and Stufflebeam [20,35,36] to define the minimum requirements for community health centers to adequately treat snakebite envenoming in the Amazon region of Brazil. The modified-Delphi approach consisted of four rounds: 1) iterative development of preliminary checklist by expert steering committee; 2) controlled feedback on preliminary checklist via expert judge survey; 3) two-phase nominal group technique [37,38] with new expert judges to resolve pending items; and 4) checklist finalization and closing criteria by expert steering committee (Fig 2).
Measure of agreement
The measure of agreement selected for this study was percent agreement, specifically joint-probability of agreement (JPA) [39]. For each type of community health center (1, 2, or 3), JPA was calculated by the sum of agreement on an item marked into a specific priority classification (Essential, Unessential, Desirable, or Undesirable), divided by the total responses, multiplied by 100. Consensus on an item was defined a priori as a JPA greater than or equal to 75%. Items with a JPA ≥75% in Essential or Desirable were considered validated. Items marked into Unessential or Undesirable with a JPA of ≥75% were dropped. Items with a JPA <75% in all four classifications were discussed in a nominal group meeting and either validated or dropped. Additional items suggested by judges in the survey round were discussed in the nominal group meeting, classified by priority, and either added or dropped. The nominal group meeting, and thus the modified-Delphi process, ended when consensus was reached on all items.
Across the Delphi process, three different types of community health centers as well as item priority (i.e., unessential versus essential to adequate treatment) were considered. The primary outcome was measure of agreement, specifically percent agreement.
Expert selection and recruitment
Expert steering committee.
A steering committee of five experts were recruited via email to develop the initial checklist, coordinate consensus rounds, and verify the finalized version. Two experts with postdoctoral degrees and extensive experience in snakebite clinical care and research, a male pharmacist (W.M.) and a female nurse (J.S.), led this effort from the FMT-HVD, a tertiary care hospital providing SBE care to patients across the Brazilian Amazon. One expert with a medical and postdoctoral degree (F.H.W), also highly experienced with SBE clinical care and research, was recruited from the Butantan Institute, the largest antivenom producer and distributor in the country. Two experts with medical degrees and global experience treating and researching snakebite envenoming (C.S., C.G.) were recruited from the Department of Emergency Medicine of the Duke School of Medicine in the United States. These team members have previously collaborated to develop a culturally relevant clinical practice guideline for snakebite treatment in Brazil as well as a multimodal health system intervention to decentralize antivenom from hospitals to community health centers in the Amazon region.
Expert judges.
To obtain a culturally and geographically diverse panel of judges, potential experts were identified via four lines of inquiry: 1) professors of graduate programs in Tropical Medicine and Clinical Toxicology, provided their lines of research were related to snakebite envenoming in Brazil; 2) a MEDLINE search on snakebite envenoming and related topics over the past five years; 3) health professionals and clinical care directors working at tertiary hospitals; and 4) public health coordinators from the Ministry of Health in Brazil. These potential experts were invited to participate via email and asked to recommend additional experts. Potential experts recommended through this snowball method were also invited to participate. Email invitations to all potential experts included a study description, an informed consent form, and a survey composed of a demographic questionnaire and the first version of the checklist. If there was no response within 20 days of the email invitation or the survey was not fully completed within 20 days of signing the consent form, the potential expert was not included as an expert judge in the survey round.
Potential experts from the survey round who did not respond, refused survey participation, or did not complete the survey fully were emailed a second time and invited to participate in the nominal group meeting. New potential experts identified via snowball sampling were also invited via email to the nominal group meeting. The email invitation for the nominal group meeting included a study description, informed consent form, and a demographic questionnaire. If there was no response within 20 days of the invitation, the potential expert was not included in the nominal group meeting. All potential experts were scored based on the criteria outlined in Table 1. Potential experts were excluded if their score was below five points.
Research team and reflexivity
The research team included the expert steering committee (W.M., J.S., F.H.W, C.S., C.G.); four validation specialists (E.S., G.S., E.T., F.R.); a qualitative specialists (J.S.) with nursing experience in clinical SBE care and research; a specialist in indigenous health (A.S.F.), a clinical laboratory scientist (J.D.B.-S.); a qualitative researcher (E.S.); a licensed physical therapist and qualitative data specialist with experience in SBE research (A.T.); a psychologist with extensive qualitative and quantitative data analytics training and experience (J.V.); a clinical pharmacist (T.S.P.); a clinical researcher (A.A.); and a data management specialist (A.S.).
Modified-Delphi method
Round 1: Iterative development of preliminary checklist by expert steering committee.
The steering committee developed the first version of the checklist in three, iterative online meetings spaced one week apart. In the first meeting, items were listed in a Word file in a brainstorming process along with a basic description. This process was structured according to the SBE clinical management and antivenom administration procedures in the clinical practice guideline from the Ministry of Health [40] as well as the recently developed clinical practice guideline specifically for community health centers [10]. Items were thus listed for the categories outlined in these guidelines: first aid on admission; diagnosis (Bothrops, Lachesis, Crotalus, and Micrurus) and clinical classification (mild, moderate, or severe) of SBEs; Lee–White clotting test procedure; preparation of antivenom before administration; antivenom administration; wound care; patient follow-up during the stay in the healthcare unit; referral of the patient to the higher-level health units, if necessary; and receiving and storing antivenoms. At the end of the first meeting, the list of items was shared with committee members to review over the week. In the second meeting, the committee discussed the list. Any additional items thought of during the week were suggested and included in the checklist. The committee was provided another week to review the list. In the third meeting, the committee finalized the preliminary version of the checklist and added more detailed descriptions of each item and its role in snakebite envenoming care (S1 File).
Round 2: Controlled feedback on preliminary checklist via expert judge survey.
Expert judges received a survey to provide controlled feedback on the preliminary checklist. The survey was divided into three sections: 1) judge demographics, 2) context for the checklist, and 3) the preliminary version of the checklist. Demographics included gender, age, profession, experience, and education. The context section situated expert judges within the three types of health units (Table 2) and four levels of priority (Table 3).
For each type of health unit, expert judges classified items by priority. Expert judges were also provided with a free text field to suggest additional items and/or provide comments to improve the checklist. Items with a JPA ≥75% in Essential or Desirable were considered validated. Items marked into Unessential or Undesirable with a JPA of ≥75% were dropped. Items with a JPA <75% in all four classifications as well as items suggested by expert judges in the free text portion of the survey were discussed in the next round.
Round 3: Two-phase nominal group technique with new expert judges.
New expert judges, independent from Round 2, attended a two-phase nominal group meeting [37,38]. The aim of the meeting was to reach consensus on items with a JPA <75% and items suggested by the expert judges in the previous round. A lead member of the expert steering group (W.M.) and a graduate student observer (T.S.P) conducted this meeting. In the first phase of the meeting, expert judges classified items by the four priority levels for each type of health unit. Responses were collected anonymously by the facilitators (W.M, T.S.P) and organized on a spreadsheet. JPA was calculated for each item, then fed back to expert judges. The second phase of the meeting was discussion until consensus regarding the items with a JPA <75%. Up to two additional nominal group meetings were planned to reach consensus and close Round 3.
Round 4: Checklist finalization and closing criteria by expert steering committee.
The expert steering committee produced a final version of the minimum requirements checklist based on the previous three rounds of development and controlled feedback. Items considered Unessential with a JPA ≥75% were not included in the checklist. Items considered Undesirable with a JPA ≥75% were also not included in the checklist, but noted at the end in a warnings section. The checklist was finalized according to best practices for checklist development, validation, and practical use outlined by Bichelmeyer, Scriven, and Stufflebeam [20,35,36].
Results
Results are presented according to each modified-Delphi round. A summary of expert judge responses from Round 2 and Round 3 is provided in Fig 3.
Round 1: Iterative development of preliminary checklist
The expert steering committee structured the first version of the checklist in five sections (Table 4). The first section, Health Unit Information, includes items regarding identification information of the community health center as well as its basic infrastructure and capacity. The second section, Human Resources, outlines personnel capacity and availability across different professional categories: nursing technicians and nurse assistants, nurses, physicians, laboratory personnel, and pharmacists. The remaining three sections list the Equipment, Supplies, and Medicines utilized in snakebite envenoming care.
Prior to completing the first version of the checklist, the expert steering committee discussed the priority of some items. The presence of electricity and the ability to transfer patients were defined as Essential for all three types of health units. Electricity is required to store antivenom, as it is refrigerated, and the capacity for patient transfer to hospitals is necessary in severe cases. Items required for the 20-minute whole blood clotting test (necessary for determining antivenom indication) were defined as Essential for all units. The ability of a health unit to operate 24/7 was considered desirable in Type 1 community health centers (most basic care), and essential in Type 2 (more advanced CHC) and Type 3 (hospitals) units.
Round 2: Expert judge survey of preliminary checklist
Participant characteristics.
A total of 35 potential experts were invited to participate in the survey round. Nine invitations were not returned and two were refused. A total of 24 invitations were thus accepted. Two of these potential experts did not fully complete the survey and were excluded. An additional two were excluded based on the minimum criteria for participation as experts.
A total of 20 potential experts were included as expert judges (Table 5). The majority were nurses (75%), men (60%), and from Amazonas State (50%). The average age was 44.75 years. Professional experience averaged almost 12 years, with most expert judges holding advanced degrees (70%), previous or current professor positions in universities (85%), published articles (75%), and clinical SBE care experience (95%). The average expert criteria score was 10.90. See S2 File for additional characteristics.
Survey of preliminary checklist.
All items in the Health Unit Information section were validated for all types of health units. No item suggestions were made by the expert judges. For Type 1 community health centers, less than half the items in the remaining sections (42.0%) were validated. Most items, however, were validated for Type 2 (76.8%) and Type 3 (91.3%). All items validated in this round were classified as Essential. See S3 File for each item classifications and its JPA for the three types of health units. Overall, the items in Supplies (39.3%) and Medicines (41.7%) sections had the lowest JPA in Type 1. The Equipment section had the lowest JPA in Type 2 (75.0%) and Type 3 (87.5%) centers.
In terms of free text recommendations, the expert judges suggested the addition of 11 items. A reclining stretcher, multiparameter patient monitor, and hospital screen were added in the Equipment section. A central venous catheter, indwelling urinary catheter, laryngeal mask airway, and urine collection bag were added in Supplies. Anticonvulsants, antiemetics, atropine, and bicarbonate were added in Medicines. No items were dropped.
Round 3: Two-phase nominal group technique
Participant characteristics.
A total of 12 potential experts were invited to participate in the nominal group meeting. Two invitations were refused. 10 potential experts accepted and met the expert criteria (Table 6). Half of the expert judges were men (50%). The average age was 46.1 years. Professions included physicians (40%), nurses (40%), and pharmacists (20%). All expert judges were from Amazonas State. Professional experience averaged 19.4 years, with most expert judges holding advanced degrees (100%), previous or current professor positions in universities (80%), published articles (70%), and clinical SBE care experience (90%). Average expert criteria score was 12.3. See S4 File for additional characteristics.
Nominal group meeting.
Phase 1. A full list of unvalidated and added items from the survey round is provided in S4 File. Most of the unvalidated items reached consensus in Type 3 (76.5%) units. However, expert judges did not reach consensus on the majority of items in Type 1 (74.5%) and Type 2 (63.0%) centers. Overall, in Type 1, the sections with the lowest JPA items were Human Resources (0.0%), Medicines (9.1%), and Equipment (17.6%). In Type 2, items in Supplies (20.0%) had the lowest JPA. In Type 3, items in Equipment (66.7%) had the lowest JPA. Of the items validated in Phase 1, seven were considered Essential and six were considered Desirable for Type 1 community health centers. All items validated for Type 2 and Type 3 were considered Essential. All items suggested by survey round expert judges were added for a total of 80 items.
Phase 2. Anonymous Phase 1 results were fed back to the expert judges. The open discussion focused on Essential versus Desirable item classification in Type 1 community health centers, with some conversation on item priority in Type 2 centers as well. Judges raised concerns that without certain items, Type 1 centers would not be able to adequately treat potential adverse reactions from antivenom. Most judges, however, advocated that the function of Type 1 centers is early treatment, including antivenom, and patients can be transferred to higher level care (Type 2 or Type 3) in the rare case of an adverse reaction. After discussion, all expert judges agreed with this opinion, and classified 36 items (45.0%) as Desirable and 44 items (55.0%) as Essential in Type 1. However, in Type 2 centers, expert judges classified most items as Essential (97.5%). Expert judges considered all 80 items (100%) as Essential in Type 3 units. No items were considered Unessential or Undesirable. Only one nominal group meeting was required to reach JPA ≥75% on all items.
Round 4: Checklist finalization and closing criteria
The final lists of essential and desirable items, respectively, for all three types of health units are outlined in Table 7.
With this final list, the expert steering committee outlined which procedures each type of health unit should be equipped to perform (Table 8). The Essential items considered for Type 1 centers are sufficient for antivenom premedication, storage, and administration, whereas a significant number of items required for treatment of early adverse reactions, management of complications, patient follow-up, and patient accommodation were considered Desirable. Type 2 centers had two or less items considered Desirable for each procedure. Type 3 units had all items for these procedures classified as Essential.
The final minimum requirements checklist is available in English and Portuguese in S5 File.
Discussion
To our knowledge, this study is the first to determine the minimum supplies and staff required for safe and effective antivenom administration in community health centers, and, in doing so, develop a standardized checklist to evaluate health unit capacity for antivenom. Our results highlighted three key findings: (1) the definition of Type 1 health centers and its list of Essential items by expert judges is consistent with the Brazilian Ministry of Health, WHO snakebite strategic plan, and a general snakebite capacity guideline in India (internal validity), (2) the list of Essential items for antivenom administration and clinical management is feasible and aligns with the literature regarding snakebite clinical care (reliability), and (3) engagement of local experts is critical to developing and implementing an antivenom decentralization strategy (feasibility).
Internal validity: Snakebite envenoming care in community health centers
Our results defined community health centers as primary care clinics capable of providing emergency care to snakebite envenoming patients, including antivenom, with the capacity to refer all or severe patients to a higher-level unit after antivenom administration. Health unit capacity was further defined by the minimum Human Resources, Equipment, Supplies, and Medicines outlined in the final checklist for Type 1.
These results align with the Brazilian MoH definition of CHCs and their scope of practice. The MoH states CHCs are primary health facilities with a small team, usually one doctor, nurse, nursing assistant (matches Essential items in Type 1 centers), and at least four community health workers (CHW) [41]. CHWs were not included in the checklist—likely due to its explicit focus on clinical care. In terms of practice, the MoH states services provided by CHCs include preventative care, public health interventions, maternal and child health care, management of chronic non-communicable diseases, referrals to higher-level care as well as social, sanitation, and other services [42,43]. The Equipment, Supplies, and Medicines items listed in the checklist, excluding antivenom, fall under those required to perform these services [44]. Further, in 2020, the SUS launched the Requalifica Programme to construct, expand, and refurbish CHCs with the goal of ensuring adequate infrastructure [45]. This aligns with the Basic Health Information section of the checklist requiring a regular power supply.
Our results are also in accordance with the WHO definition of CHCs and their capacity. The WHO strategy for prevention and control of SBE specifically aims to improve “access to essential medicines, including antivenoms, and all other medical drugs, equipment, and consumable items” and ensure “appropriate staffing” in primary health care services [46]. Our checklist categories—Human Resources, Medicines, Equipment, and Supplies—match these target areas. The WHO strategy, however, does not specify the essential items recommended for snakebite care in CHCs. This study adds to the literature in offering a detailed perspective on the minimum requirements for safe, effective antivenom administration and clinical management of snakebites in a low-resource, high-burden region.
One other study in India, although general, also provides perspective on this tenet of the WHO strategy. A team of researchers utilized data from a population-linked facility survey conducted by the MoH and Family Welfare, Government of India, to conceptualize structural capacity for snakebite care [18]. This assessment is largely like our checklist, but differed in that it was more conceptual and had a focus beyond clinical care in its inclusion of two additional categories: Governance and Finance and Health Management Information Systems. After explaining its assessment model, the study explicitly called for a specific health facility survey (like the checklist developed) to assess SBE care capacity [18].
Reliability: Essential items for antivenom administration and clinical management
The debate regarding essential items for Type 1 centers parallels the data and health professional opinions present in the literature, specifically the critical need for antivenom in CHCs, longstanding fear of adverse reactions to antivenom, and additional training for health professionals. This concordance suggests the data informing the checklist is reliable.
There was no debate among expert judges regarding whether antivenom was essential in all three types of health units. Several other studies among professionals in the Brazilian Amazon ubiquitously argue antivenom as an essential medicine in CHCs [2,4,5,8,11,13–15,17,19,47,48]. In addition, research on snakebite envenoming care across India [49,50], French Guiana [51], Burkina Faso [52], and Myanmar [53] highlights the need for antivenom availability and accompanying equipment in their primary health care centers. A multi-country study regarding access to antivenom in Malaysia, Thailand, Indonesia, Philippines, Vietnam, Lao PDR, and Myanmar also explicitly argues “strengthening the supply chain of antivenoms to ensure that antivenoms are readily accessible at the point of service,” primary health units [54].
While antivenom is internationally recognized as an essential medicine in primary care, some expert judges in our study raised serious concerns regarding the capacity of CHCs to adequately treat potential early adverse reactions to antivenom. This fear has been documented in previous Brazilian Amazon [19] and rural India [55] studies and likely stems from the more frequent and severe reactions observed with older, pre-2000s antivenoms that required hospital treatment [56,57]. With newer, improved antivenoms, the frequency of early adverse reactions is low ‐ around 15% in the Brazilian Amazon [58] ‐ and almost always a mild skin reaction as reported by studies in Brazil [58] and Costa Rica [59]. It is worth noting that whether items required for treating reactions were essential to Type 1 CHCs was the largest obstacle to reaching consensus amongst the expert judges. The ultimate decision to not include these items was only accepted given the existing referral pathway from CHCs (Type 1, Type 2) to hospitals (Type 3).
The reluctance of some judges to accept leveraging lower-level units for time-sensitive treatment, even with referral networks in place, is likely due to the hospital-centric narrative and training surrounding snakebite envenoming care. This is supported by how easily the judges reached consensus on essential items in Type 3 health units, or hospitals. But, when asked to determine Essential versus Desirable items for the lower-level Type 2 and Type 1 centers, there was significantly more disagreement and debate surrounding whether patient care can be extended from hospitals to earlier along the care continuum despite its safety and proven effectiveness in other settings, namely antivenom decentralization in Costa Rica [59] and a pilot trial of a nurse-led antivenom clinic Tanzania [60].
This is reflected in the expert judges’ call for specifically trained physicians, nurses, and nursing assistants in CHCs, also in accordance with the literature in Southeast Asia [54], Nigeria [61], India [55], and Burkina Faso [52]. In contrast with this view, several studies also highlight lack of knowledge and call for additional, reformed training amongst hospital-based health providers, in particular the United States [62] and Bhutan [63]. Most decentralization initiatives recommend training for the primary health care providers, but, considering the lengthy debate between expert judges in this study and emerging literature on potential gaps in SBE training for hospital-based providers, we suggest decentralizing training programs across the continuum of care. Hospital-centric curricula, nor a sole CHC focus, are adequate to address a severe, time-sensitive disease predominantly occurring in remote and rural areas.
Feasibility: Engagement of local experts for successful decentralization planning
A modified-Delphi method engaging local experts was necessary to develop a valid and reliable minimum requirements checklist that is feasible within the current health unit structure in the Brazilian Amazon. Without engaging local experts, 11 items (notably only one of which considered essential in Type 1 centers) would not have been added to the checklist, nor would training across the care continuum—specifically how to address early adverse reactions—been identified as a priority to antivenom decentralization strategies.
Several studies have argued the success of health care decentralization programs is highly dependent on the context [64]. Influencing contextual factors include cultural norms, values, practices, and beliefs; geographic environment; and a well-functioning health system, including logistics support, supplies, and equipment [65]. A study on decentralization and health system performance in India identified three determinants of performance: health workers, health facilities, and agents of decision making, patients, and the community [66].
Regarding health workers and agents of decision-making, care coordination, training, and support for CHC health professionals is critical to decentralization success. Our team has been conducting trainings with health professionals as funding allows, and established both an online group and an SMS-based care coordination messaging system to connect professionals with each other as well as experts at our institute/hospital, the Foundation for Tropical Medicine in Manaus. The online group includes doctors, pharmacists, nurses, and even biologists to support these health professionals real-time in the identification and management of snakebites, from determining the snake species to medication administration. The SMS-based messaging system connects professionals with an expert at our institute/hospital to answer any questions, also in real-time. In addition, health professionals in Type 1 and Type 2 units often communicate with each other via radio or telephone, and engage senior staff to support and advise in snakebite envenoming cases.
Narrowing in on health facilities, the study detailed the “availability of infrastructure, equipment, and supplies” as well as “accreditation status” as critical components [66]. The checklist developed addresses these two components and represents an initial milestone for determining the absolute minimum supplies and staff necessary for safe and effective antivenom administration and clinical management. Given the structure of the Brazilian health system is already decentralized to municipalities, and the essential items align with the current capacity of CHCs, antivenom decentralization is feasible.
Limitations
As with all Delphi studies, the quality of data collected is tied to the qualifications and experience of the participating experts. To obtain the most robust data possible, we engaged three groups of expert judges across four rounds of iteration, and expert qualifications, experience, and demographics were disclosed in detail to promote transparency. A potential bias exists in the lack of health professionals actively working in community health centers. The objective of the study, however, was to determine the essential items to safe and effective clinical care of snakebites. Clinical and clinical research experts were thus engaged.
Conclusion
Decentralization of antivenom access is a set of strategic actions to reduce mortality and morbidity from snakebite envenoming, primarily affecting historically neglected and invisible populations. This study joins an international set of evidence advocating for decentralization, adding value in its definition of essential care items; identification of training needs across the care continuum; and demonstration of the validity, reliability, and feasibility provided by engaging local experts. Specific to Brazil, further added value comes in the potential use of the checklist for health unit accreditation as well as its applications to logistics and resource distribution. Future research priorities should apply this checklist to CHCs in the Amazon region of Brazil to determine which CHCs are or could be capable of receiving antivenom and translate this expert- driven checklist and approach to snakebite care in other settings or other diseases in low-resource settings.
Supporting information
S1 File. Version of the minimal requirements checklist created by the steering committee for experts’ validation.
https://doi.org/10.1371/journal.pntd.0011921.s001
(DOCX)
S2 File. Characterization of the 20 expert judges responsible by the first phase of the validation process of the minimal requirements checklist.
https://doi.org/10.1371/journal.pntd.0011921.s002
(XLSX)
S3 File. Joint-probability of agreement for each item by type of unit.
https://doi.org/10.1371/journal.pntd.0011921.s003
(XLSX)
S4 File. Characterization of the 10 expert judges responsible by the second phase of the validation process (by nominal group technique) of the minimal requirements checklist.
https://doi.org/10.1371/journal.pntd.0011921.s004
(XLSX)
S5 File. Final version of the minimal requirements checklist.
https://doi.org/10.1371/journal.pntd.0011921.s005
(DOCX)
Acknowledgments
We would like to extend a special thanks to the experts who kindly agreed to participate in this study. We know the time of these professionals is very valuable, and that makes us especially grateful.
References
- 1. Feitosa EL, Sampaio VS, Salinas JL, Queiroz AM, da Silva IM, Gomes AA, et al. Older Age and Time to Medical Assistance Are Associated with Severity and Mortality of Snakebites in the Brazilian Amazon: A Case-Control Study. PLoS ONE. 2015 Jul 13;10(7):e0132237. pmid:26168155
- 2.
de Oliveira SS, de Souza Sampaio V, de Almeida Gonçalves Sachett J, Alves EC, da Silva VC, de Lima JAA, et al. Snakebites in the brazilian amazon: current knowledge and perspectives. In: Gopalakrishnakone P, Faiz SMA, Gnanathasan CA, Habib AG, Fernando R, Yang C-C, et al., editors. Clinical Toxinology. Dordrecht: Springer Netherlands; 2016. p. 1–22.
- 3. Magalhães SFV, Peixoto HM, de Almeida Gonçalves Sachett J, Oliveira SS, Alves EC, Dos Santos Ibiapina HN, et al. Snakebite envenomation in the Brazilian Amazon: a cost-of-illness study. Trans R Soc Trop Med Hyg. 2020 Sep 1;114(9):635–42. pmid:32239168
- 4. Cristino JS, Salazar GM, Machado VA, Honorato E, Farias AS, Vissoci JRN, et al. A painful journey to antivenom: The therapeutic itinerary of snakebite patients in the Brazilian Amazon (The QUALISnake Study). PLoS Negl Trop Dis. 2021 Mar 4;15(3):e0009245. pmid:33661895
- 5. Hui Wen F, Monteiro WM, Moura da Silva AM, Tambourgi DV, Mendonça da Silva I, Sampaio VS, et al. Snakebites and scorpion stings in the Brazilian Amazon: identifying research priorities for a largely neglected problem. PLoS Negl Trop Dis. 2015 May 21;9(5):e0003701. pmid:25996940
- 6. Macinko J, Guanais FC, de Fátima M, de Souza M. Evaluation of the impact of the Family Health Program on infant mortality in Brazil, 1990–2002. J Epidemiol Community Health. 2006 Jan;60(1):13–9. pmid:16361449
- 7. Beck TP, Tupetz A, Farias AS, Silva-Neto A, Rocha T, Smith ER, et al. Mapping of clinical management resources for snakebites and other animal envenomings in the Brazilian Amazon. Toxicon: X. 2022 Dec;16:100137. pmid:36160931
- 8. Fan HW, Monteiro WM. History and perspectives on how to ensure antivenom accessibility in the most remote areas in Brazil. Toxicon. 2018 Sep 1;151:15–23. pmid:29908262
- 9. Ye JJ, Joao Felipe Hermann Costa Scheidt, De Andrade L, Thiago Augusto Hernandes Rocha, Fan HW, Monteiro W, et al. Antivenom accessibility impacts mortality and severity of Brazilian snake 2 envenomation: a geospatial information systems analysis. medRxiv. 2020 Jan 1;
- 10. Rocha GDS, Farias AS, Alcântara JA, Machado VA, Murta F, Val F, et al. Validation of a culturally relevant snakebite envenomation clinical practice guideline in brazil. Toxins (Basel). 2022 May 28;14(6). pmid:35737037
- 11. Murta F, Strand E, de Farias AS, Rocha F, Santos AC, Rondon EAT, et al. “two cultures in favor of a dying patient”: experiences of health care professionals providing snakebite care to indigenous peoples in the brazilian amazon. Toxins (Basel). 2023 Mar 3;15(3). pmid:36977085
- 12. Val F, Alcântara JA, Maciel Salazar GK, Farias AS, Monteiro WM, Sachett JG de A. Disability secondary to snakebites in rural Amazon: What are the impacts? Toxicon. 2020 Apr 20;177 Suppl 1:S19.
- 13. Maciel Salazar GK, Saturnino Cristino J, Vilhena Silva-Neto A, Seabra Farias A, Alcântara JA, Azevedo Machado V, et al. Snakebites in “Invisible Populations”: A cross-sectional survey in riverine populations in the remote western Brazilian Amazon. PLoS Negl Trop Dis. 2021 Sep 9;15(9):e0009758. pmid:34499643
- 14. da Silva Souza A, de Almeida Gonçalves Sachett J, Alcântara JA, Freire M, Alecrim M das GC, Lacerda M, et al. Snakebites as cause of deaths in the Western Brazilian Amazon: Why and who dies? Deaths from snakebites in the Amazon. Toxicon. 2018 Apr;145:15–24. pmid:29490236
- 15. Feitosa ES, Sampaio V, Sachett J, Castro DB de, Noronha M das DN, Lozano JLL, et al. Snakebites as a largely neglected problem in the Brazilian Amazon: highlights of the epidemiological trends in the State of Amazonas. Rev Soc Bras Med Trop. 2015;48 Suppl 1:34–41. pmid:26061369
- 16. Schneider MC, Vuckovic M, Montebello L, Sarpy C, Huang Q, Galan DI, et al. Snakebites in rural areas of brazil by race: indigenous the most exposed group. Int J Environ Res Public Health. 2021 Sep 5;18(17). pmid:34501955
- 17. Strand E, Murta F, Tupetz A, Barcenas L, Phillips AJ, Farias AS, et al. Perspectives on snakebite envenoming care needs across different sociocultural contexts and health systems: A comparative qualitative analysis among US and Brazilian health providers. Toxicon: X. 2023 Mar;17:100143. pmid:36578905
- 18. Bhaumik S, Norton R, Jagnoor J. Structural capacity and continuum of snakebite care in the primary health care system in India: a cross-sectional assessment. BMC Prim Care. 2023 Aug 11;24(1):160. pmid:37563556
- 19. Monteiro WM, Farias AS de, Val F, Neto AVS, Sachett A, Lacerda M, et al. Providing antivenom treatment access to all brazilian amazon indigenous areas: “every life has equal value”. Toxins (Basel). 2020 Dec 5;12(12).
- 20. Scriven M. The Logic and Metholodology of Checklists [Internet]. Western Michigan University; 2005 [cited 2023 Aug 22]. Available from: https://wmich.edu/sites/default/files/attachments/u350/2014/logic%26methodology_dec07.pdf
- 21. Thomassen Ø, Espeland A, Søfteland E, Lossius HM, Heltne JK, Brattebø G. Implementation of checklists in health care; learning from high-reliability organisations. Scand J Trauma Resusc Emerg Med. 2011 Oct 3;19:53. pmid:21967747
- 22. Shillito J, Arfanis K, Smith A. Checking in healthcare safety: theoretical basis and practical application. Int J Health Care Qual Assur. 2010;23(8):699–707. pmid:21125965
- 23. Haugen AS, Sevdalis N, Søfteland E. Impact of the world health organization surgical safety checklist on patient safety. Anesthesiology. 2019 Aug;131(2):420–5. pmid:31090552
- 24. Gillespie BM, Marshall A. Implementation of safety checklists in surgery: a realist synthesis of evidence. Implement Sci. 2015 Sep 28;10:137. pmid:26415946
- 25. Haynes AB, Weiser TG, Berry WR, Lipsitz SR, Breizat A-HS, Dellinger EP, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009 Jan 29;360(5):491–9. pmid:19144931
- 26. Pronovost P, Needham D, Berenholtz S, Sinopoli D, Chu H, Cosgrove S, et al. An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med. 2006 Dec 28;355(26):2725–32. pmid:17192537
- 27.
Ministério da Saúde. RESOLUÇÃO-RDC No 307, DE 14 DE NOVEMBRO DE 2002 [Internet]. 2002 [cited 2023 Mar 11]. Available from: https://bvsms.saude.gov.br/bvs/saudelegis/gm/2017/prt2436_22_09_2017.html
- 28.
Ministério da Saúde. PORTARIA No 340, DE 4 DE MARÇO DE 2013 [Internet]. 2013 [cited 2023 Sep 17]. Available from: https://bvsms.saude.gov.br/bvs/saudelegis/gm/2013/prt0340_04_03_2013.html
- 29.
Ministério da Saúde. RESOLUÇÃO-RDC No 50, DE 21 DE FEVEREIRO DE 2002 [Internet]. 2002 [cited 2023 Sep 17]. Available from: https://bvsms.saude.gov.br/bvs/saudelegis/anvisa/2002/rdc0050_21_02_2002.html
- 30.
Ministério da Saúde. RESOLUÇÃO ‐ RDC N o 197, DE 26 DE DEZEMBRO DE 2017 [Internet]. 2017 [cited 2023 Sep 17]. Available from: https://www.in.gov.br/materia/-/asset_publisher/Kujrw0TZC2Mb/content/id/1432311/do1-2017-12-28-resolucao-rdc-n-197-de-26-de-dezembro-de-2017-1432307
- 31.
Ministério da Saúde. RESOLUÇÃO No 7, DE 24 DE FEVEREIRO DE 2010 [Internet]. 2010 [cited 2023 Sep 17]. Available from: https://bvsms.saude.gov.br/bvs/saudelegis/anvisa/2010/res0007_24_02_2010.html
- 32.
Ministério da Saúde. BRASIL M da S. RESOLUÇÃO-RDC No 302, DE 13 DE OUTUBRO DE 2005. [Internet]. 2005 [cited 2023 Sep 17]. Available from: https://bvsms.saude.gov.br/bvs/saudelegis/anvisa/2005/res0302_13_10_2005.html
- 33.
Ministério da Saúde. Diretrizes para estruturação de farmácias no âmbito do SUS [Internet]. 2009 [cited 2023 Sep 17]. Available from: https://www.gov.br/saude/pt-br/composicao/sectics/daf/cbaf/qualifar-sus/eixo-cuidado-antigo/arquivos/diretrizes-para-estruturacao-farmacias-ambito-sus.pdf/view
- 34. Nasa P, Jain R, Juneja D. Delphi methodology in healthcare research: How to decide its appropriateness. World J Methodol. 2021 Jul 20;11(4):116–29. pmid:34322364
- 35. Bichelmeyer B. Checklist for Formatting Checklists. 2023 Apr 10 [cited 2023 Aug 22]; Available from: https://wmich.edu/sites/default/files/attachments/u350/2014/formattingchecklist.pdf
- 36. Stufflebeam DL. Evaluation checklists: practical tools for guiding and judging evaluations. American Journal of Evaluation. 2001 Mar;22(1):71–9.
- 37. Harvey N, Holmes CA. Nominal group technique: an effective method for obtaining group consensus. Int J Nurs Pract. 2012 Apr;18(2):188–94. pmid:22435983
- 38. McMillan SS, King M, Tully MP. How to use the nominal group and Delphi techniques. Int J Clin Pharm. 2016 Jun;38(3):655–62. pmid:26846316
- 39. Diamond IR, Grant RC, Feldman BM, Pencharz PB, Ling SC, Moore AM, et al. Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol. 2014 Apr;67(4):401–9. pmid:24581294
- 40.
Ministério da Saúde. Guia de Vigilância Epidemiológica - 7a edição [Internet]. Ministério da Saúde; 2022 Oct [cited 2023 Aug 22]. Available from: https://www.gov.br/saude/pt-br/assuntos/saude-de-a-a-z/f/febre-tifoide/publicacoes/guia-de-vigilancia-epidemiologica-7a-edicao/view
- 41. Andrade MV, Coelho AQ, Xavier Neto M, Carvalho LR de, Atun R, Castro MC. Brazil’s Family Health Strategy: factors associated with programme uptake and coverage expansion over 15 years (1998–2012). Health Policy Plan. 2018 Apr 1;33(3):368–80. pmid:29346551
- 42. Özçelik EA, Massuda A, Castro MC, Barış E. A comparative case study: does the organization of primary health care in brazil and turkey contribute to reducing disparities in access to care? Health Syst Reform. 2021 Jul 1;7(2):e1939931. pmid:34402403
- 43. Macinko J, Harris MJ. Brazil’s family health strategy—delivering community-based primary care in a universal health system. N Engl J Med. 2015 Jun 4;372(23):2177–81. pmid:26039598
- 44.
OECD. Improving public health services. OECD economic surveys: brazil 2015. OECD; 2015. p. 91–119.
- 45.
Ministério da Saúde. Plano Nacional de Saúde (PNS)—Ministério da Saúde [Internet]. 2021 [cited 2023 Sep 25]. Available from: https://www.gov.br/saude/pt-br/acesso-a-informacao/gestao-do-sus/instrumentos-de-planejamento-do-sus/pns
- 46.
World Health Organization. WHO | Snakebite envenoming—A strategy for prevention and control [Internet]. 2019 [cited 2020 Sep 10]. Available from: https://www.who.int/snakebites/resources/9789241515641/en/
- 47. de Farias AS, Cristino JS, da Costa Arévalo M, Carneiro Junior A, Gomes Filho MR, Ambrosio SA, et al. Children Growing Up with Severe Disabilities as a Result of Snakebite Envenomations in Indigenous Villages of the Brazilian Amazon: Three Cases and Narratives. Toxins (Basel). 2023 May 23;15(6). pmid:37368653
- 48. Magalhães SFV, Peixoto HM, Moura N, Monteiro WM, de Oliveira MRF. Snakebite envenomation in the Brazilian Amazon: a descriptive study. Trans R Soc Trop Med Hyg. 2019 Mar 1;113(3):143–51. pmid:30476298
- 49. Bawaskar HS, Bawaskar PH, Bawaskar PH. Primary health care for snakebite in India is inadequate. Lancet. 2020 Jan 11;395(10218):112. pmid:31929008
- 50. Gajbhiye RK, Chaaithanya IK, Munshi H, Prusty RK, Mahapatra A, Palo SK, et al. National snakebite project on capacity building of health system on prevention and management of snakebite envenoming including its complications in selected districts of Maharashtra and Odisha in India: A study protocol. PLoS ONE. 2023 Feb 17;18(2):e0281809. pmid:36800356
- 51. Houcke S, Pujo JM, Vauquelin S, Lontsi Ngoula GR, Matheus S, NkontCho F, et al. Effect of the time to antivenom administration on recovery from snakebite envenoming-related coagulopathy in French Guiana. PLoS Negl Trop Dis. 2023 Apr 24;17(4):e0011242. pmid:37093856
- 52. Bamogo R, Thiam M, Nikièma AS, Somé FA, Mané Y, Sawadogo SP, et al. Snakebite frequencies and envenomation case management in primary health centers of the Bobo-Dioulasso health district (Burkina Faso) from 2014 to 2018. Trans R Soc Trop Med Hyg. 2021 Nov 1;115(11):1265–72. pmid:34581814
- 53. Schioldann E, Mahmood MA, Kyaw MM, Halliday D, Thwin KT, Chit NN, et al. Why snakebite patients in Myanmar seek traditional healers despite availability of biomedical care at hospitals? Community perspectives on reasons. PLoS Negl Trop Dis. 2018 Feb 28;12(2):e0006299. pmid:29489824
- 54. Patikorn C, Ismail AK, Abidin SAZ, Blanco FB, Blessmann J, Choumlivong K, et al. Situation of snakebite, antivenom market and access to antivenoms in ASEAN countries. BMJ Glob Health. 2022 Mar;7(3). pmid:35296460
- 55. Chaaithanya IK, Abnave D, Bawaskar H, Pachalkar U, Tarukar S, Salvi N, et al. Perceptions, awareness on snakebite envenoming among the tribal community and health care providers of Dahanu block, Palghar District in Maharashtra, India. PLoS ONE. 2021 Aug 5;16(8):e0255657. pmid:34351997
- 56. Fan HW, Marcopito LF, Cardoso JL, França FO, Malaque CM, Ferrari RA, et al. Sequential randomised and double blind trial of promethazine prophylaxis against early anaphylactic reactions to antivenom for bothrops snake bites. BMJ. 1999 May 29;318(7196):1451–2. pmid:10346769
- 57. Cardoso JL, Fan HW, França FO, Jorge MT, Leite RP, Nishioka SA, et al. Randomized comparative trial of three antivenoms in the treatment of envenoming by lance-headed vipers (Bothrops jararaca) in São Paulo, Brazil. Q J Med. 1993 May;86(5):315–25.
- 58. Mendonça-da-Silva I, Magela Tavares A, Sachett J, Sardinha JF, Zaparolli L, Gomes Santos MF, et al. Safety and efficacy of a freeze-dried trivalent antivenom for snakebites in the Brazilian Amazon: An open randomized controlled phase IIb clinical trial. PLoS Negl Trop Dis. 2017 Nov 27;11(11):e0006068. pmid:29176824
- 59. Sasa M, Segura Cano SE. New insights into snakebite epidemiology in Costa Rica: A retrospective evaluation of medical records. Toxicon: X. 2020 Sep;7:100055. pmid:32776004
- 60. Yates VM, Lebas E, Orpiay R, Bale BJ. Management of snakebites by the staff of a rural clinic: the impact of providing free antivenom in a nurse-led clinic in Meserani, Tanzania. Ann Trop Med Parasitol. 2010 Jul;104(5):439–48. pmid:20819312
- 61. Habib AG. Public health aspects of snakebite care in West Africa: perspectives from Nigeria. J Venom Anim Toxins Incl Trop Dis. 2013 Oct 17;19(1):27. pmid:24134780
- 62. Tupetz A, Barcenas LK, Phillips AJ, Vissoci JRN, Gerardo CJ. BITES study: A qualitative analysis among emergency medicine physicians on snake envenomation management practices. PLoS ONE. 2022 Jan 7;17(1):e0262215. pmid:34995326
- 63. Sapkota S, Pandey DP, Dhakal GP, Gurung DB. Knowledge of health workers on snakes and snakebite management and treatment seeking behavior of snakebite victims in Bhutan. PLoS Negl Trop Dis. 2020 Nov 30;14(11):e0008793. pmid:33253152
- 64. Abimbola S, Baatiema L, Bigdeli M. The impacts of decentralization on health system equity, efficiency and resilience: a realistic synthesis of the evidence. Health Policy Plan. 2019 Oct 1;34(8):605–17.
- 65. Kok MC, Kane SS, Tulloch O, Ormel H, Theobald S, Dieleman M, et al. How does context influence performance of community health workers in low- and middle-income countries? Evidence from the literature. Health Res Policy Syst. 2015 Mar 7;13:13. pmid:25890229
- 66. Panda B, Thakur HP. Decentralization and health system performance ‐ a focused review of dimensions, difficulties, and derivatives in India. BMC Health Serv Res. 2016 Oct 31;16(Suppl 6):561. pmid:28185593