The World Health Organization (WHO) commissioned a systematic review of literature to facilitate evidence syntheses for the development of emergency risk communication (ERC) guidelines for its member states. The goal of this review was to integrate ERC best practices into governmental and non-governmental health systems for all emergencies of public health concern, by addressing three questions: (1) to identify best practices for the integration of ERC into national and international public health preparedness; (2) to identify mechanisms to establish effective intra-agency, inter-agency, and/or cross-jurisdictional information sharing; and (3) to identify methods to coordinate risk communication activities between responding agencies across organizations and levels of response. The review covered scientific and grey literature publications between January 2003 and February 2016, and searches were conducted in 17 English language electronic libraries besides Chinese, Portuguese and Spanish language databases. A mixed deductive-inductive process was used to synthesize findings across studies through identifying thematic areas. While 8,215 articles were initially retrieved, after a sequential screening process, the final evidence syntheses comprised of 21 articles for question (1) and 24 for questions (2) and (3) combined (due to overlap of themes). The confidence in findings was assessed by the Qualitative Evidence Syntheses (GRADE-CERQual) tool. PRISMA guidelines were followed to the extent possible given the limitations inherent to a review largely based on qualitative studies. The identified literature was very context-specific and referred to mechanisms, practices from the field, and recommendations that were derived from planning or response efforts implemented at the national or local levels in specific countries. Integration of ERC functions into public health emergency preparedness, planning and response activities was influenced by reforming components of the leadership structure when needed, modifying organizational factors, and nullifying restrictions (including amending laws/ regulations) that might have been an obstacle to the timely release of information. Exercises and trainings were recognized as effective strategies to identify the barriers and successes in this process of integration. Key elements to enhance information sharing and coordination across organizations included the creation of networks, task-forces and committees across disciplines, organizations and geographic areas. Engagement of local stakeholders was also important to guarantee the flow of information up and down the incident command system. On the whole, few empirical studies, especially from low- and middle-income countries, related to the WHO research questions, demonstrating the need for research in these areas. To facilitate an accurate identification of the gaps, the authors suggest integrating current findings with case studies across the WHO regions to better understand the specific evidence that is needed in practice across the multitude of ERC functions.
Citation: Jha A, Lin L, Short SM, Argentini G, Gamhewage G, Savoia E (2018) Integrating emergency risk communication (ERC) into the public health system response: Systematic review of literature to aid formulation of the 2017 WHO Guideline for ERC policy and practice. PLoS ONE 13(10): e0205555. https://doi.org/10.1371/journal.pone.0205555
Editor: Aram Dobalian, University of Memphis, UNITED STATES
Received: July 23, 2017; Accepted: September 28, 2018; Published: October 31, 2018
Copyright: © 2018 Jha et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files. The data is also available with WHO and has been cited in text.
Funding: This manuscript is based on the report developed by the Division of Policy Translation and Leadership Development under Contract Number 201400531: Evidence Syntheses to support the World Health Organization (WHO) Guidelines on Emergency Risk Communication, Department of Communications, Office of the Director-General, World Health Organization. ES was the project PI and received funding from the WHO. The WHO (funder) was involved in expert advisory capacity in the methodology, quality assessment and overall conduct of this systematic review.
Competing interests: The authors have declared that no competing interests exist.
Emergency risk communications (ERC) consist of "the real-time exchange of information, advice and opinions" between decision-makers, experts and the general public.  Over the last decade, nations have increasingly faced challenges in acquiring, processing and communicating information to protect the physical, social and economic wellbeing of their citizens during emergency situations. This challenge is in part due to the lack of evidence on how to best communicate among responding agencies and with the public. 
ERC is one of the eight core functions that World Health Organization (WHO) member states are required to fulfill as part of the International Health Regulations (IHR) . All types of public health emergencies, ranging from infectious diseases outbreaks and pandemics to weather related events and manmade disasters, present national leaders with the challenge of communicating risk to the affected populations while maintaining trust, transparency and consistency of messages. While there are existing best practices and training in the field of ERC, there are few comprehensive, evidence-based, systems-focused principles and guidelines to support its practice. In response to this need, and recognizing that access to information is a fundamental right of an affected population, the WHO established a Guideline Development Group (GDG) in 2015 to steer the development of guidance on ERC for its member states.  This guideline has been recently published by the WHO, and provides advice and direction on how member states can integrate the best practices of risk communication into critical governmental and non-governmental health systems for all emergencies of public health concern (natural/man-made disasters, infectious disease epidemics/pandemics, and terrorism). 
The WHO commissioned the Emergency Preparedness, Research, Evaluation and Practice (EPREP) program at the Harvard T.H. Chan School of Public Health to conduct a systematic review of literature to facilitate evidence syntheses for the development of ERC guidelines for its member states (the full report has been made available by the WHO).  The goal of this review was to integrate ERC best practices into governmental and non-governmental health systems for all emergencies of public health concern, by addressing three questions: (1) to identify best practices for the integration of ERC into national and international public health preparedness; (2) to identify mechanisms to establish effective intra-agency, inter-agency, and/or cross-jurisdictional (such as cross-border; national with sub-national jurisdictions, etc.) information sharing; and (3) to identify methods to coordinate risk communication activities between responding agencies across organizations and levels of response. It was envisaged that answering these questions would facilitate the integration of knowledge regarding the effectiveness of ERC practices and structures into the WHO guideline development process.
Recognizing that the evidence answering these questions was more likely to be qualitative or mixed-methods research, the WHO developed the "Setting, Perspective, phenomenon of Interest, Comparison, Evaluation of impact" (SPICE) format to facilitate the interpretation of these questions and guide development of search terminology. The detailed breakdown of the SPICE format for each question is provided in the published WHO Guideline. 
The review was primarily conducted for all scientific and grey literature in English, Chinese, Portuguese and Spanish language databases, between January 1, 2003 and February 7–9, 2016 (specific dates mentioned with the list of databases in Tables 1–3). Fourteen scientific and three grey literature English language electronic libraries were searched, and specific searches were carried out for publications in Chinese ("China Academic Journals Full-text" and "China National Knowledge Infrastructure" databases), Portuguese ("LILACS/SCielo" and "Mirage—Fiocruz" databases) and Spanish ("REDYLAC/SCielo" database). Manual cross-referencing of articles judged to be relevant to the questions of interest was also conducted for additional publications.
Since a key objective was to look for evidence in the low and middle-income countries, the search strategies in all global databases (like PubMed, Cochrane, WHO Library etc.) were expanded to also include publications in Arabic, French, Russian, Italian and Japanese, besides the above-mentioned languages (these additional languages were selected on the basis of linguistic expertise available to the researchers). The web pages of a number of governmental and non-governmental public health agencies from French, Spanish, and Portuguese speaking countries, as well as Italy (as the major Italian-speaking nation) were screened. The research team also communicated with several risk communication experts and academics across USA, Europe (England, Germany, Switzerland, members of European CDC), Asia (India, China, Japan, Bangladesh, Vietnam), Latin America (Portugal, Argentina) and the Middle East (Qatar) to substantiate the understanding of systemic gaps and political-cultural sensibilities in ERC, as well as to request guidance to find region-specific publications related to the WHO questions.
The articles retrieved from the initial search (n = 8,215) were independently screened by AJ and SS for English language articles (including English language abstracts of articles in Japanese and German), LL for Chinese articles, ES for Italian language articles, and GA and ES for Spanish and Portuguese language articles. No articles published in Arabic, French or Russian languages were identified, though it was possible that the English language abstract of such an article was included among those retrieved from the global databases. The detailed search strategies for English, Chinese, Portuguese and Spanish language databases are provided in Tables 1–3.
In the title review phase, the authors specifically screened for articles including the terms "communication" or related words (risk/ crisis communication) along with an event of interest like any natural or man-made disaster, any infectious disease outbreak or epidemic or pandemic, or any event of terrorism (or simulation studies on these scenarios). Articles were also included if the terms "risk management,” "preparedness," "preparedness exercise," or “knowledge” were mentioned in the title. In the next phase, the researchers evaluated 1899 abstracts to include only: (i) research based on an empirical study [results/recommendations based on some observation or experiment]; and (ii) research directly or indirectly providing information to address at least one of the three WHO questions. This screening ensured that only articles which clearly violated the above criteria were excluded. Subsequently, 880 full-text articles were reviewed to determine eligibility for inclusion based on the assessment that they addressed components of the WHO SPICE breakdown, the details of which can be found in the published WHO Guidelines (Annex 3: SPICE questions #1, 2 and 6). 
To determine if the research was addressing the WHO questions, two reviewers independently read the article (with the exception of articles in Chinese language), and based on the SPICE breakdown identified sentences (in the article) supported by empirical data that would provide information which directly answered a WHO question or was related to the question (indirectly answered the question). Subsequently the two reviewers met to discuss their judgment, and in case of disagreement a third reviewer was consulted. Furthermore all articles deemed to be indirectly related were discussed with the WHO team to make sure the information was related to the questions developed by WHO.
Categorization of articles.
Study title, first author, year of publication, type of publication (i.e. journal article, organizational report etc.), country/ region of the study, study design (i.e. qualitative, quantitative, mixed-methods or case study, with subtypes as applicable), population/sample studied or addressed (i.e. general population, affected/ vulnerable groups, stakeholders etc …), the type of disaster/crisis situation described and the phase of disaster (i.e. prevention, preparedness, response, recovery/rehabilitation) addressed were listed. The objectives of each study were noted, and the key topic area and principal findings and/or recommendations were carefully described. The relevance of an article to one or more of the WHO questions under consideration was critically judged and differences in opinion were resolved through consultations (as described earlier).
An article was defined to be "directly relevant" to a specific WHO question if: (1) it had an empirical study design, and (2) the findings and/or recommendations directly answered that particular WHO question. If an article did not satisfy both these criteria, it was coded as "indirectly relevant" and not included in the final evidence synthesis.
Methods of evidence synthesis
Four methodological streams were identified to classify and characterize the selected articles: 1) quantitative (randomized group comparison, non-randomized group comparison and descriptive surveys), 2) qualitative (interviews, focus groups and textual/content analysis), 3) mixed-methods (use of both quantitative and qualitative techniques) and 4) case studies (description of the response to a particular emergency/crisis situation). The literature was then synthesized using the Best Fit Framework Synthesis described by Booth et al.  and Barnett-Page , which involves searching for existing theories/conceptual frameworks and creating a composite framework describing findings within each theme/concept. A mixed deductive-inductive process was used to synthesize findings across studies, starting with the SPICE components for each WHO question to develop the themes, and subsequently adding more thematic areas based on the literature.
For question 1, the syntheses of evidence is presented separately for English (n = 6) and Chinese language (n = 15) publications because of the different types of studies included in the two languages and difficulties in merging concepts and themes between the two. However, integration of the themes in a unified synthesis among articles in the two aforementioned languages was possible for questions 2 and 3, and therefore the evidence syntheses for these two questions (all language publications) have been presented together.
Quality assessment of the evidence
A number of tools were applied to assess the quality of each study based on the methodology and study design: the Critical Appraisal Skills Program (CASP) tools were used to assess cohort studies and qualitative studies (including content analysis) , the Mixed Methods Appraisal Tool (MMAT) for studies using mixed methods , the Cochrane risk of bias tool for randomized interventional studies , the AMSTAR evaluation for systematic reviews , and the critical appraisal checklist for studies including surveys (The BMJ, Table E) . Subsequently, confidence in findings was assessed by the use of the Qualitative Evidence Syntheses (GRADE-CERQual) tool, and a level of confidence was assigned to each study (high/ moderate/ low/ very low) based on methodological limitations, relevance, coherence and adequacy of data . The GRADE-CERQual assessment was completed both for individual articles (S1–S3 Tables) and for the syntheses of findings derived from multiple articles within each WHO question (Tables 4–6). GRADE principles were adapted for application to descriptive quantitative studies and GRADE CERQual principles were applied to mixed-method studies. Neither adaptation has been approved by the tool originators. Please refer to the WHO Guideline for ERC policy and practice 2017  for details.
For the overall conduct of this systematic review, the PRISMA guidelines were followed to the extent possible given the limitations inherent to a review largely based on qualitative studies .
Characterization of the literature
Among the 8,215 articles retrieved, 5,946 were in English (may have included other language publications that had English abstracts), 1,415 in Chinese, 481 in Portuguese and 373 in Spanish. Through title screening, 6,316 (77%) articles were excluded, while 1,899 (23%) abstracts were reviewed, leading to 880 (11% of all retrieved) full text articles for further scrutiny. After the final stage appraisal, 21 full text articles were selected for Question 1 (6 in English and 15 in Chinese), and 24 more for Questions 2 and 3 combined (21 in English, 2 in Chinese and 1 in Portuguese). Fig 1 depicts the step-wise flow of literature.
E = English, C = Chinese, P = Portuguese, S = Spanish.
Among the 21 articles identified as related to question (1), 6 (28.5%) were qualitative, 6 mixed-methods, 5 (24%) quantitative and 4 (19%) case studies. Most articles focused on ERC in China, Hong Kong and Taiwan (18, 86%), while the others covered ERC in Cambodia, Indonesia, Japan, Laos, Vietnam, Thailand and UK (1 each, with overlap). Twelve (57%) of these 21 articles addressed all hazard situations (general disasters), while 9 were based on pandemic influenza and other infectious disease outbreaks. Tables 4 and 5 summarize the evidence. S1 and S2 Tables list the individual study findings within methodological streams and evaluation of confidence.
Among the 24 articles included for the final evidence syntheses for questions (2) and (3) combined, the majority (12, 50%) were qualitative in design, with 5 (21%) case studies, 4 (17%) quantitative and 3 (12%) mixed method approaches. Seven (29%) articles focused on the US, and 3 each on Netherlands and China (including Hong Kong and Taiwan). Other regions represented included Asia (Cambodia, Vietnam, Thailand, Indonesia, Laos, and Myanmar), the Middle East (Turkey, Iran, Israel, Palestine, and Jordan), Australia, New Zealand, Canada and Brazil. Among the disaster-types, 7 (29%) articles addressed all-hazard situations, 4 addressed pandemic influenza/infectious diseases, 3 described floods, 2 each addressed hurricanes/ tornados, volcanos, terrorism and anthrax scares, and 1 each described a wildfire, earthquake or other major accident. Table 6 summarizes the evidence. S3 Table lists the individual study findings within methodological streams and evaluation of confidence.
Evidence synthesis: (1) How can ERC best be integrated into national and international public health emergency preparedness planning and response activities?
English language literature
The articles identified as relevant to this question presented examples of mechanisms that might lead to the integration of ERC functions into the leadership structure. Such mechanisms were summarized under four themes: (1) placing ERC functions into the national leadership structure [16, 17]; (2) creating organizational proximity of ERC practitioners to national response leadership [18, 19]; (3) developing laws, regulations, policies, and frameworks in support of ERC [16, 18, 20, 21]; and (4) the use of trainings and exercises as a mechanism for testing the effectiveness of the system [17, 20] (cross cutting theme).
Restructuring of existing organizational components of agencies engaged in preparedness and response efforts, as well as developing novel intra- and inter-agency coordination frameworks were often needed to achieve integration of ERC functions into the national preparedness/disaster response leadership. Yen et al.  described how a new public health policy by the Taipei City Government formed the basis for implementing an integrated infection control system to respond to emerging infectious diseases during the aftermath of the SARS outbreak in 2003. The Taipei Division of Disease Control and Prevention was reformed to function as the nodal unit for implementing crisis management programs and policies, including integration of early outbreak detection through hospital-based and school-based surveillance, prompt epidemiological investigations and preventive responses. These structural reforms allowed the Division to launch a swift response during the 2007 acute hemorrhagic conjunctivitis outbreak through its multi-channel, mass risk communication program. In the same context, it was Taiwan's Communicable Disease Act of 2006 which facilitated Taipei's ability to launch large-scale SMS campaigns in 2007, as this Act provided the necessary legal platform to allow government agencies to override people’s right to privacy (government could use cellular service providers to send out six free public service messages per year) when responding to epidemics or disasters.
Assessing pandemic influenza preparedness in six Asian countries (Cambodia, Indonesia, Laos, Taiwan, Thailand and Vietnam), Hanvoravongchai and colleagues  observed that in nations with well-functioning health systems, pandemic preparedness was integrated within existing mechanisms such as national disaster preparedness frameworks; while those with weaker systems relied heavily on vertical programs for coordination and response. The team further described the importance of simulation exercises in these countries that demonstrated the gaps in coordination between the various stakeholders (health sector and beyond). Thailand ensured at least one table-top exercise at the central level and in each province; Vietnam covered airports and borders, in addition to administrative levels; Indonesia conducted a first-of-its-kind full scale preparedness exercise in Bali in 2008. Furthermore, the WHO and the Mekong Basin Disease Surveillance (MBDS) Network coordinated several regional cross-country exercises.
Cope  noted that the lack of authority to release information to the public was a critical barrier to ERC throughout the the chain of command in the Chinese public health system. On the other hand, shared public health intelligence between the Hong Kong Special Administrative Regional Government, Mainland (China) Ministry of Health and Macao Health Bureau facilitated functioning of joint emergency responses in the event of cross-boundary public health emergencies.  DRIP, a societal platform for disaster risk information which facilitated the acquisition and dissemination of scientific expertise on risk information from a large number of governmental/non-governmental agencies and research institutions, was utilized as a major tool in Japan's disaster risk governance. 
For each of these thematic areas, the overall synthesized evidence was considered to be of moderate confidence level (GRADE-CERQual). Most of the contributory publications (to each area) were individually judged to have only minor concerns regarding methodology, and/or adequacy of data, and/or coherence; hence these articles individually provided evidence of a "moderate" confidence level. The confidence in the pooled evidence is a reflection of such a majority of articles.
Chinese language literature
The articles identified to respond to question (1) described practices across three functional areas: (1) how various government agencies used the micro-blogging platform "Weibo" for ERC [22–28]; (2) integration of a national health hotline into the emergency preparedness and management system [27, 29, 30]; and (3) collaboration between the US and Chinese CDCs to build ERC capacity across the country [31–36].
Government agencies in China utilized the social media platform "Weibo" (biggest Chinese micro-blogging platform, often referred to as “Chinese Twitter”) for public communication during normal times as well as in public health emergencies (like H1N1, H7N9, terrorist attacks, and the Ya’An earthquake) with the goals of achieving timely, open, two-way communications, monitoring public opinion, controlling rumors, addressing public concerns and improving government services. There had been a consistent national effort to expand the use of Weibo across China, and to establish a standardized operation, training and evaluation mechanism for its service performance. [22–28] In order to effectively control online rumors during health or humanitarian crises, a swift response was mounted by releasing official counter-messages, working with law enforcement to identify and penalize sources of rumors, and assessing the effectiveness of these counter-measures through continuous monitoring of Weibo.
A national hotline service (12320, China's only such call-in facility) was developed by the Ministry of Health (MoH) and integrated into the emergency response system. This was successfully tested as an important channel of communication during the 2008 formula contamination crisis, 2008 Beijing Olympics, 2009 A(H1N1) pandemic, and the recent measles campaign. [27, 29, 30] This hotline has helped shape national ERC strategies such as developing protocols to monitor public reactions and opinions through providing a direct, two-way communication between health agencies and the public. and has facilitated public health consultations across China’s provinces, municipalities and autonomous regions.
Within China's governance framework, these novel approaches had helped integrating ERC as a system response involving multiple agencies and the target (affected) population. Further, the Chinese CDC had been working in close collaboration with its US counterpart to conduct assessment of ERC needs at local public health agencies, through conducting tabletop and functional exercises. [31–36]
Similar to synthesized evidence from English language publications, the overall synthesized evidence was considered to be of moderate confidence level (CERQual) for each of these three functional areas.
Evidence synthesis: (2) What are the best mechanism(s) to establish effective intra-agency, inter-agency, and/or cross-jurisdictional (such as cross-border; national with sub-national jurisdictions, etc.) information sharing for emergency risk communication? and (3) What are the best practices and protocols to ensure coordination of risk communication activities between responding agencies across organizations and levels of response?
The articles (in English, Chinese, and Portuguese) identified to respond to these two questions (combined) presented examples of mechanisms to enhance information sharing and coordination. Such mechanisms were summarized under three themes: (1) creation of task forces/committees [37–43] and networks [44, 45] to enhance ERC, and their elements of functionality); (2) use of information systems to enhance ERC (tools and platforms) [46–54]; and (3) mechanisms to facilitate local stakeholders’ engagement in ERC [55–60].
The formation and functioning of collaborative platforms like task forces, networks and committees had been attributed to facilitate efficient information sharing between national and sub-national authorities, as well as between agencies. Chess et al.  cited the role of a bioterrorism task force in New Jersey, USA which served as a platform for ERC sharing between partner agencies like public health and law enforcement during to the 2001 Anthrax incidents. The diverse agencies had developed mutual trust through this pre-existing task force, and this lay the foundation for improved intersectoral coordination and intelligence exchange. It had been pointed out that emergency responders were far more likely to trust and interact with people/agencies with whom they had an existing professional relationship, and such networking improved agility in carrying out emergency response measures. [39, 40] Specific roles for Information Managers and/or Public Information Officers at local agencies had been proposed to be improve intra-agency coordination. [41, 42]
Gresham  described the collaborative health information sharing network, Middle East Consortium on Infectious Disease Surveillance, between Israel, Jordan and the Palestinian Authority. This forum brought together politically divergent states and served as a platform to boost regional health intelligence exchange, capacity development through laboratory and risk communications training, and implementation of innovative communication technology. This partnership greatly assisted cross-border preparedness (including airport and border screening, laboratory testing) and ERC strategies during the H1N1 pandemic, building on pre-existing trust and thoroughly exercised national/regional emergency plans and protocols. Similarly, the MBDS Network between Cambodia, China (originally just Yunnan province and, since 2008, including Guangxi Province), Lao People’s Democratic Republic, Myanmar, Thailand and Vietnam served as a platform to coordinate sub-regional infectious disease surveillance and control. Regional-level coordinated preparedness and prevention efforts led to better control of the pandemic within the participating countries. 
The literature provided several examples where planned investment in communication infrastructure including better operationalization of wireless communication channels, increased coordination between responding agencies, and better understanding of communication needs improved the overall disaster management. This was specifically demonstrated during the response to two earthquakes, three months apart, in Turkey.  Thiago et al  described the successful collaboration between the government and Civil Defense in Brazil which led to the development and rapid testing of novel channels of ERC utilizing social networks (Facebook, Twitter) and mobile phones. Funded by the European Union, the Flood Information and Warning System along the Dutch-German border had optimized communication between water-management and crisis-management agencies.  On the other hand, Kapucu  described how the absence of an integrated information system greatly hindered coordinated response of agencies (police, fire department and port authority) following the World Trade Center terrorist attack in New York City. Chang et al.  studied the process of information sharing and coordination within Taiwan's emergency management information system during the catastrophic 2009 typhoon. They recommended identification and designation of an agency that had the best geographic reach (in this case, the police) to lead the process of ERC sharing. Militello  had observed that, given the diverse levels of knowledge and experience of different teams at an emergency operations center (EOC), low-cost substitutes like notebooks, whiteboards, and telephone books, in addition to electronic tools, could greatly improve functional efficiency by removing any asymmetric skills barriers.
Different mechanisms were proposed to engage local stakeholders in formulating and implementing ERC strategies. Ardalan  suggested formation of Village Disaster Taskforces through community participation in Iran, to function as operational units in the early warning mechanism by facilitating spread of ERC to the lowest levels of the chain. Cole et al.  proposed that community emergency management coordinators should look to utilize existing social networks in small rural municipalities for public education and disaster risk reduction activities. Citing the example of 2011 Brisbane floods (Australia), Shepherd  emphasized the need to address culturally and linguistically diverse populations through the incorporation of appropriate ERC materials into centralized resources. Gultom  described a community-based risk information sharing network in Indonesia, the Merapi Circle Information Networks, which developed local radio stations and recruited trusted community representatives to harness ground resources in order to be better equipped in emergency preparedness. A study from Badung, Indonesia found that the Women's Welfare Association leaders were in a unique position to act as key facilitators in the early warning system at sub-district, city or ward levels of governance. 
The functioning of ERC is intricately linked to the varying political and cultural landscapes present across nations. Therefore, in some circumstances, centralized ERC systems may work better than localized ones, or vice versa. Researchers had noted that decentralized health systems (e.g. as in Indonesia) faced greater challenges in implementing preventive and outbreak response measures, and the level of efficiency depended heavily on local political commitment.  In contrast, there was a need for increased decision-making power at the level of provincial and local public health agencies in China to enable them to release critical ERC to the public, circumventing barriers in organizational hierarchy.  Hence, issues like political goodwill and leadership, as well as the structure of the national health system (degree of centralization) are to be considered as key factors in planning and policy-making for ERC.
The development of ERC policies and capacities through regional partnerships and guidelines seemed to be well received. For example, the European Union (EU) had enacted legislation on a cross-border integrated emergency response system, including coordination and information exchange between constituent nations. The Health Security Committee (HSC) Communicators’ Network under the EU provides crisis communication expertise and guidance as part of a comprehensive strategy for the successful management of public health threats. The information-sharing protocol is implemented through the establishment and activation of a list of contact points within the EU, the European CDC and the WHO. 
Geographic variations in capacity and practice of ERC strongly necessitated the formulation of evidence-based universal guidelines by the WHO to help member states develop frameworks to integrate ERC as a system response during emergencies of public health concern. Such communication needs to be transparent, timely and based on the best available scientific evidence, in order to ensure the maximal physical, social and economic well-being of citizens.
The identified literature referred to mechanisms, practices from the field, and recommendations that were derived from planning or response efforts implemented at the national or local levels in specific countries, but did not provide direct evidence of transferability to other contexts. Factors that seemed to be related to the integration of ERC functions in national and international public health emergency preparedness, planning and response activities included renovation of components of the leadership structure when needed, modification of organizational factors, nullifying restrictions that might hinder the timely release of information, and amendments to laws and regulations where feasible. Exercises and trainings were recognized as strategies to identify barriers and successes in the integration of ERC functions into preparedness, planning and response efforts. Key elements to enhance information sharing and coordination across organizations included the creation of networks, task-forces and committees across disciplines, organizations and geographic areas. The functionality of information systems was a key element for the sharing of information by tailoring such systems to the needs of the users. Engagement of local stakeholders was equally important to guarantee the flow of information up and down the incident command system.
Despite conducting a very thorough literature review across multiple language databases, the authors felt that more sensitive approaches may be needed to gather useful evidence in a non-Western non-English context, for example, when conducting region-specific case studies. It was observed that the Chinese language articles tended to be succinct in the methodology section and elaborative about policy or program implications, with an emphasis on status report and actions taken or recommendations for future steps. This utility-driven approach might be useful in offering actionable information to practitioners on the ground in the context of China, but presented a challenge in quality assessment. With the Spanish and Portuguese language publications, there was a general paucity of empirical literature on ERC. Risk communication in emergency preparedness was mainly addressed by the social health, communication and technology, and human sciences fields; very little of this work was produced by researchers or practitioners working in public health. In addition, differences in organizational response structure, especially in Latin America, contributed to this overall finding.
On the whole, few empirical studies, especially from low- and middle-income countries were related to the WHO research questions. The authors attempted to circumvent this shortcoming by searching databases in Chinese, Portuguese and Spanish, as well as relaxing the strict definition for empirical literature to include more case studies and to reflect a broader distribution of country experiences and knowledge. However, this observed bias against empirical studies from low- and middle-income countries may be partly due to the limitations of the authors in their ability to assess a broader range of languages. Furthermore, the fact that ERC is still not precisely-defined as a field of research meant that there were challenges in identifying sensitive search terms and keywords that would incorporate the varied disciplines that cover this field. It is probable that in addition to differences in terms across disciplines, terms may also differ across nations, and even among professionals trained in similar disciplines.
The authors believe that the lack of empirical studies across the questions solicited by the WHO demonstrates an overall need for research in these areas. However, an accurate identification of research gaps should be achieved by integrating the results of this review with case studies across the WHO regions to better understand what type of evidence is needed in practice across the multitude of ERC functions. Such an approach may ensure that research is produced in the topic areas of greatest need for practice.
S1 Table. Individual study findings within methodological streams and evaluation of confidence–Question 1, English language literature.
S2 Table. Individual study findings within methodological streams and evaluation of confidence–Question 1, Chinese/ Mandarin literature.
S3 Table. Individual study findings within methodological streams and evaluation of confidence–Questions 2 & 3 (combined).
The Harvard EPREP team gratefully acknowledges the support provided during formulating literature search strategies and in retrieving articles by Mr. Paul Bain, librarian at the Francis A. Countway Library, Harvard University and Mr. Tomas Allen, librarian at the WHO. We are also grateful to the experts in ERC who shared with us their experience and knowledge during the interviews, and helped us identify additional sources of grey literature; and to Mr. Noah Klein for editorial support.
- 1. Barnes MD, Hanson CL, Novilla LM, Meacham AT, McIntyre E, Erickson BC. Analysis of media agenda setting during and after Hurricane Katrina: implications for emergency preparedness, disaster response, and disaster policy. Am J Public Health. 2008;98(4):604–10. Epub 2008/03/01. pmid:18309133; PubMed Central PMCID: PMC2376984.
- 2. Centers for Disease Control and Prevention. Crisis & Emergency Risk Communication (CERC) USA: Office of Public Health Preparedness and Response, CDC; 2016 [cited 2017 January 5]. Available from: https://emergency.cdc.gov/cerc/.
- 3. World Health Organization. International Health Regulations (IHR): WHO; 2017 [cited 2017 June 4]. Available from: http://www.who.int/topics/international_health_regulations/en/.
- 4. World Health Organization. Risk Communication Geneva: WHO; 2016 [cited 2017 January 5]. Available from: http://www.who.int/risk-communication/background/en/.
- 5. World Health Organization. Communicating Risk in Public Health Emergencies: A WHO Guideline for Emergency Risk Communication (ERC) policy and practice2017 January 12, 2018 [cited 2018 January 12]:[78 p.]. Available from: http://www.who.int/risk-communication/guidance/download/en/.
- 6. Savoia E, Lin L, Short S, Jha A, Argentini G, Klein N, et al. Evidence Syntheses to Support the WHO Guidelines on Emergency Risk Communication2016 January 12, 2018 [cited 2018 January 12]. Available from: http://www.who.int/risk-communication/guidance/process/Harvard-WHO-report-q1.pdf?ua=1.
- 7. Booth A, Noyes J, Flemming K, Gerhardus A, Wahlster P, van der Wilt GJ, et al. Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions 2016 [cited 2016 August 25]. Available from: http://www.integrate-hta.eu/wp-content/uploads/2016/02/Guidance-on-choosing-qualitative-evidence-synthesis-methods-for-use-in-HTA-of-complex-interventions.pdf.
- 8. Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009;9:59. Epub 2009/08/13. pmid:19671152; PubMed Central PMCID: PMC3224695.
- 9. Critical Appraisal Skills Programme. CASP Checklists UK: CASP; 2016 [cited 2016 August 25]. Available from: http://www.casp-uk.net/.
- 10. Pluye P, Robert E, Cargo M, Bartlett G, O’Cathain A, Griffiths F, et al. Proposal: A mixed methods appraisal tool for systematic mixed studies reviews2011 [cited 2017 January 5]. Available from: http://mixedmethodsappraisaltoolpublic.pbworks.com/.
- 11. The Cochrane Collaboration. Cochrane Handbook for Systematic Reviews of Interventions 2011 [cited 2017 January 5]. Available from: http://handbook.cochrane.org/chapter_8/8_assessing_risk_of_bias_in_included_studies.htm.
- 12. Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Medical Research Methodology. 2007;7(1):10. pmid:17302989
- 13. BMJ. Table E: Critical appraisal checklist for a questionnaire study 2004 [cited 2016 August 25]. Available from: http://www.bmj.com/content/suppl/2004/05/27/328.7451.1312.DC1.
- 14. Lewin S, Glenton C, Munthe-Kaas H, Carlsen B, Colvin CJ, Gülmezoglu M, et al. Using Qualitative Evidence in Decision Making for Health and Social Interventions: An Approach to Assess Confidence in Findings from Qualitative Evidence Syntheses (GRADE-CERQual). PLOS Medicine. 2015;12(10):e1001895. pmid:26506244
- 15. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700. Epub 2009/07/23. pmid:19622552; PubMed Central PMCID: PMC2714672.
- 16. Yen MY, Wu TS, Chiu AW, Wong WW, Wang PE, Chan TC, et al. Taipei's use of a multi-channel mass risk communication program to rapidly reverse an epidemic of highly communicable disease. PLoS One. 2009;4(11):e7962. Epub 2009/12/04. pmid:19956722; PubMed Central PMCID: PMC2776508.
- 17. Hanvoravongchai P, Adisasmito W, Chau PN, Conseil A, de Sa J, Krumkamp R, et al. Pandemic influenza preparedness and health systems challenges in Asia: results from rapid analyses in 6 Asian countries. BMC Public Health. 2010;10:322. Epub 2010/06/10. pmid:20529345; PubMed Central PMCID: PMC2896940.
- 18. Cope JR, Frost M, Richun L, Xie R. Assessing knowledge and application of emergency risk communication principles among public health workers in China. Disaster Med Public Health Prep. 2014;8(3):199–205. Epub 2014/05/03. pmid:24785302; PubMed Central PMCID: PMC4675457.
- 19. Chambers J, Barker K, Rouse A. Reflections on the UK's approach to the 2009 swine flu pandemic: conflicts between national government and the local management of the public health response. Health Place. 2012;18(4):737–45. Epub 2012/06/12. pmid:22682089.
- 20. Lam PY. Avian influenza and pandemic influenza preparedness in Hong Kong. Ann Acad Med Singapore. 2008;37(6):489–96. Epub 2008/07/12. pmid:18618061.
- 21. Ikeda S, Nagasaka T. Emergent Framework of Disaster Risk Governance towards Innovating Coping Capability for Reducing Disaster Risks in Local Communities. International Journal of Disaster Risk Science. 2011;2(2):1–9. WOS:000209368600001.
- 22. Chen. 陈靓, 汪青云. 突发事件下的政务微博角色探析. (The role of Government MicroBlogs during emergencies). 东南传播杂志 (Mass Media). 2013;8.
- 23. Chen. 陈然, 谢薇. 突发事件中政务微博舆情应对的问题及对策 (Strategies of governmental microblogs' response to emergencies). 今传媒 (Contemporary Mass Media). 2014;10:16–7.
- 24. Liu. 刘国恒, 李灵辉, 郭汝宁, 代吉亚, 吴发好, 宋铁. 人感染H7N9禽流感网络谣言监测与防控策略探讨 (Monitor, prevention and control online rumors about people becoming infected with H7N9 virus.). 华南预防医学 [South China Journal of Preventive Medicine]. 2013;5.
- 25. Index) 政政GWRRGWI. 人民网舆情监测室与微博数据中心 (The People's Public Opinion Monitoring Office and Weibo Data Center), 2012–2016. 2016.
- 26. Zeng. 曾繁旭, 钱琪瑶. 传播链条、社会网络与公众回应 社会化媒体时代的风险沟通效果研究. 新闻与写作 [News and Journalism]. 2015;6.
- 27. Zhang XE, Yang Y, Shang R, Simons J, Quek B, Yin X, et al. 利用社交網站開展疾病監測與公眾感知 (Leveraging social networking sites for disease surveillance and public sensing: the case of the 2013 avian influenza A(H7N9) outbreak in China). WPSAR. 2015;6(2). https://doi.org/10.5365/wpsar.2015.6.1.013
- 28. Zhou. 周莉, 李晓, 黄娟. 政务微博在突发事件中的信息发布及其影响 (Government Weibo Information dissemination and its impact during emergencies). 新闻大学 (Journalism Bimonthly). 2015;02.
- 29. Jiang. 麻疹疫苗强化免疫相关舆情监测分析 (Use of 12320 to monitor the public response on measle vaccination campaign). 中国公共卫生 (Chinese Journal of Public Health). 2012;3.
- 30. Wang. 王超, 谭枫. 12320 在甲型H1N1 流感中风险沟通与舆情监测作用 (Use of 12320, an official public health hotline, for ERC and public opinions monitoring during the A(H1N1) pandemic). 中国公共卫生 (Chinese Journal of Public Health). 2010;11.
- 31. Hao. 郝爱华, 李海麟, 袁俊, 蔡文峰, 沈纪川, 景钦隆, 王鸣. 广州市疾病预防控制人员应急风险沟通需求调查 (ERC capacity and needs assessment for Guangzhou City CDC staff). 预防医学情报杂志 (Journal of Preventive Medicine). 2009;5:420–2.
- 32. Ma Y, Qian L, Tong L, Mao Q. 风险沟通在我国应对甲型H1N1流感中的运用 (Application of risk communication in the response to the Influenza A/H1N1 outbreaks in China). 中国健康教育 (Chinese Journal of Health Education). 2010;1.
- 33. Shao. 邵安娜, 徼晓菲, 赵雯, 解瑞谦, 常慧, 杨小宝, 欧剑鸣, 肖[王乐], 张莹. 风险沟通在线培训工具包本土化开发应用效果的评价 (Localized application of a risk communication online training toolkits). 中国健康教育 (Chinese Journal of Health Education). 2014;30(4):340–3.
- 34. Song. 宋铁, 刘国恒, 金连梅, 涂文校, 方艳, 易建荣, 代吉亚. 中国疾控机构媒体监测能力建设现状 (Capacity building for media monitoring for China CDC). 中国公共卫生 (Chinese Journal of Public Health). 2016;6.
- 35. Xie. 解瑞谦, 唐雪峰, 欧剑鸣, 阚坚力. 突发公共卫生事件风险沟通中准备工作内容的研究 (Integrating risk communication preparedness framework into public health emergency preparedness). 中国健康教育 (Chinese Journal of Health Education). 2011;27(8):607–9.
- 36. Zhang. 张巍, 张继彬, 陈国永, 张刚, 李杰, 钱玲, 毛群安. 县域突发公共卫生事件大众风险沟通现状研究 (Assessment of county-level public health emergencies risk communication capacity). 医学与社会 (Medicine and Society). 2011;24(11):30–3.
- 37. Chess C, Clarke L. Facilitation of risk communication during the anthrax attacks of 2001: the organizational backstory. Am J Public Health. 2007;97(9):1578–83. Epub 2007/08/02. pmid:17666692; PubMed Central PMCID: PMC1963291.
- 38. Clarke CE, Chess C. False alarms, real challenges—one university's communication response to the 2001 anthrax crisis. Biosecur Bioterror. 2006;4(1):74–83. Epub 2006/03/21. pmid:16545026.
- 39. Nowell B, Steelman T. Communication under Fire: The Role of Embeddedness in the Emergence and Efficacy of Disaster Response Communication Networks. Journal of Public Administration Research and Theory. 2015;25(3):929–52. WOS:000357884800011.
- 40. Schraagen JM, Veld MHit, De Koning L. Information Sharing During Crisis Management in Hierarchical vs. Network Teams. Journal of Contingencies and Crisis Management. 2010;18(2):117–27.
- 41. Bharosa N, Lee J, Janssen M. Challenges and obstacles in sharing and coordinating information during multi-agency disaster response: Propositions from field exercises. Information Systems Frontiers. 2010;12(1):49–65.
- 42. Howard AF, Bush HM, Shapiro RM 2nd, Dearinger A. Characteristics of Kentucky local health departments that influence public health communication during times of crisis: information dissemination associated with H1N1 novel influenza. J Public Health Manag Pract. 2012;18(2):169–74. Epub 2012/01/31. pmid:22286286.
- 43. Holmes BJ, Henrich N, Hancock S, Lestou V. Communicating with the public during health crises: Experts' experiences and opinions. Journal of Risk Research. 2009;12(6):793–807. 2010-12176-004.
- 44. Gresham L, Ramlawi A, Briski J, Richardson M, Taylor T. Trust across borders: responding to 2009 H1N1 influenza in the Middle East. Biosecur Bioterror. 2009;7(4):399–404. Epub 2009/12/24. pmid:20028248.
- 45. Moore M, Dausey DJ. Response to the 2009-H1N1 influenza pandemic in the Mekong Basin: surveys of country health leaders. BMC Res Notes. 2011;4:361. Epub 2011/09/20. pmid:21923914; PubMed Central PMCID: PMC3184283.
- 46. Celik S, Corbacioglu S. Role of information in collective action in dynamic disaster environments. Disasters. 2010;34(1):137–54. Epub 2009/08/18. pmid:19682005.
- 47. Seyedin SH, Jamali HR. Health Information and Communication System for Emergency Management in a Developing Country, Iran. Journal of Medical Systems. 2011;35(4):591–7. WOS:000293300600017. pmid:20703530
- 48. Ipe M, Raghu TS, Vinze A. Information intermediaries for emergency preparedness and response: A case study from public health. Information Systems Frontiers. 2010;12(1):67–79.
- 49. Bharosa N, Lee J, Janssen M, Rao HR. A case study of information flows in multi-agency emergency response exercises. Proceedings of the 10th Annual International Conference on Digital Government Research: Social Networks: Making Connections between Citizens, Data and Government; Puebla, Mexico. 1556226: Digital Government Society of North America; 2009. p. 277–82.
- 50. Leonard GS, Stewart C, Wilson TM, Procter JN, Scott BJ, Keys HJ, et al. Integrating multidisciplinary science, modelling and impact data into evolving, syn-event volcanic hazard mapping and communication: A case study from the 2012 Tongariro eruption crisis, New Zealand. Journal of Volcanology and Geothermal Research. 2014;286:208–32. WOS:000346551400016.
- 51. Thiago RT, Pacheco DC, Baldessar MJ, Benciveni FAM. Redução de riscos de desastres na prática: a participação da Defesa Civil e do município em plataformas digitais de redes sociais Democracia digital e governo eletronico (Disaster Risk Reduction in Practice: Civil Defense and Municipality Participation in Digital Social Networking Platforms Digital Democracy and Electronic Government). Capa. 2013;2(9).
- 52. Chang. 张四明 王瑞夆. 台湾重大灾害灾情查通报系统运作之经验分析 (Operations of the Emergency Information Management System in Taiwan). 复旦公共行政评论 (Public Management Review). 2013;2.
- 53. Kapucu N. Interagency Communication Networks During Emergencies Boundary Spanners in Multiagency Coordination. The American Review of Public Administration. 2006;36(2):207–25.
- 54. Militello LG, Patterson ES, Bowman L, Wears R. Information flow during crisis management: challenges to coordination in the emergency operations center. Cognition, Technology & Work. 2007;9(1):25–31.
- 55. Ardalan A, Holakouie Naieni K, Kabir MJ, Zanganeh AM, Keshtkar AA, Honarvar MR, et al. Evaluation of Golestan Province's Early Warning System for flash floods, Iran, 2006–7. Int J Biometeorol. 2009;53(3):247–54. Epub 2009/02/17. pmid:19219465.
- 56. Cole JM, Murphy BL. Rural hazard risk communication and public education: Strategic and tactical best practices. International Journal of Disaster Risk Reduction. 2014;10:292–304. WOS:000357733900022.
- 57. Shepherd J, van Vuuren K. The Brisbane flood: CALD gatekeepers' risk communication role. Disaster Prevention and Management. 2014;23(4):469–83. WOS:000341723200011.
- 58. Gultom DI, Joyce Z. Crisis communication capacity for disaster resilience: community participation of information providing and verifying in Indonesian volcanic eruption. Australian and New Zealand Third Sector Research, 2014.
- 59. Mulyasari F, Shaw R. Role of women as risk communicators to enhance disaster resilience of Bandung, Indonesia. Natural Hazards. 2013;69(3):2137–60. http://dx.doi.org/10.1007/s11069-013-0798-4. pmid:1504417386; 2011–562094.
- 60. Lei L. 高速公路突发事件组织间应急信息沟通实证研究 (Empirical study on emergency response information sharing during a highway crisis) 管理评论 (Management Review). 2015;27(9):213–20.
- 61. THE EUROPEAN PARLIAMENT AND THE COUNCIL OF THE EUROPEAN UNION. DECISION No 1082/2013/EU OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 22 October 2013 on serious cross-border threats to health and repealing Decision No 2119/98/EC 2013 [cited 2016 August 20]. Available from: http://ec.europa.eu/health//sites/health/files/preparedness_response/docs/decision_serious_crossborder_threats_22102013_en.pdf.