Figures
Abstract
This article contributes an empirical analysis of information sharing practices on Twitter relating to the use of face masks in the context of COVID-19. Behavioural changes, such as the use of face masks, are often influenced by people’s knowledge and perceptions, which in turn can be affected by the information available to them. Face masks were not recommended for use by the UK public at the beginning of the COVID-19 pandemic. Due to developments in scientific understanding, the guidance changed and by the end of 2020 they were mandatory on public transport and in shops. This research examines tweets in this longitudinal context and, therefore, provides novel insights into the dynamics of crisis communication in an ongoing crisis event with emerging scientific evidence. Specifically, analysis of the content of tweets, external resources most frequently shared, and users sharing information are considered. The conclusions contribute to developing understanding of the digital information ecology and provide practical insights for crisis communicators. Firstly, the analysis shows changes in the frequency of tweets about the topic correspond with key guidance and policy changes. These are, therefore, points in time official channels of information need to utilise the public’s information seeking and sharing practices. Secondly, due to changes in face mask guidance and policy, the current literature on digital information ecology is insufficient for capturing the dynamic nature of a long-term ongoing crisis event. Challenges can arise due to the prolonged circulation of out-of-date information, i.e. not strategic misinformation, nor “mis”-information at all, which can have serious ramifications for crisis communication practitioners. Thirdly, the role of traditional media and other journalism/broadcasting platforms in shaping conversations is evident, as is the potential for scientific organisations’ and individual people’s Twitter user accounts. This plurality of contributors needs to be acknowledged and understood to inform crisis communication strategies.
Citation: Baker H, Concannon S, So E (2022) Information sharing practices during the COVID-19 pandemic: A case study about face masks. PLoS ONE 17(5): e0268043. https://doi.org/10.1371/journal.pone.0268043
Editor: Chi Ho Yeung, Education University of Hong Kong, CHINA
Received: September 28, 2021; Accepted: April 15, 2022; Published: May 5, 2022
Copyright: © 2022 Baker et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files. Please note, full Tweets have not been provided due to Twitter copyright permissions. However, we have provided Tweet IDs in the supporting information and the External URLs analysed.
Funding: Dr Hannah Baker’s and Dr Emily So’s research project is called ‘Expertise Under Pressure’, and Dr Shauna Concannon’s is entitled ‘Giving Voice to Digital Democracies’. Both projects are part of the Centre for Research in the Arts, Social Sciences and Humanities (CRASSH) at the University of Cambridge, and funded by the Humanities & Social Change International Foundation (https://hscif.org/cambridge/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
On 31 December 2019, an unknown cause of pneumonia was reported to the World Health Organisation (WHO) county office in China [1]. Subsequently the outbreak of this new disease, now known as COVID-19, led to the WHO declaring it a global pandemic on the 11 March 2020 [2]. Mitigation measures to reduce the spread were implemented throughout the world, with ‘hands, face, space’ being one of the slogans introduced in September 2020 by the UK Government to promote the non-pharmaceutical mitigation measures; meaning: wash your hands, wear a face mask (also referred to as face covering) and keep your space from people outside your own household [3]. For the ease of reading, the term ‘face masks’ is used throughout this paper to refer to both face masks and face coverings for public use e.g., a non-medical mask such as cloth rather than a respirator.
The effectiveness of mitigation measures, such as face mask use, is reliant on strong public support and the masses complying [4–6]. People’s level of knowledge, attitudes and risk perceptions can determine their readiness to accept these types of behavioural changes [4, 7]. As individual perceptions of threats are often influenced by the information received [8], research has demonstrated that clear, consistent and effective public messaging is likely to increase the public’s adherence to guidelines [9, 10]. This study focuses on one of the recommended mitigation strategies: face masks. Using a Twitter dataset (study period: 22 January—1 August 2020), the dissemination of information about face masks is analysed, with the aim of developing an understanding on the digital information ecology and how these information sharing practices could have affected the public’s understanding of the scientific evidence and expertise related to their use. These insights can be used to guide future public information campaigns [11]. This aim is underpinned by the following research questions (RQ):
- RQ1: What type of information about face masks was deemed as newsworthy and shareable?
- RQ2: When did users choose to share information about face masks during the COVID-19 pandemic?
- RQ3: What were the main external resources of information about face masks?
- RQ4: Which user’s tweets were most widely circulated?
The aim of the first question is to set the scene and identify what topics were discussed and whether scientific evidence and expertise formed part of this. The other questions explore the complexity of information circulation practices. As we are interested in the circulation of scientific evidence, we also investigate if these information sharing behaviours differ for scientific content.
We begin by outlining the changes in the guidance and use of face masks in the UK and its devolved nations. The academic foundation for this work is then discussed and we set the theoretical context by drawing upon work and theories associated with the digital information ecology and crisis communication. The methodology section provides detail on the Twitter datasets we have used for the analysis, including how the tweets were filtered and then analysed. Our findings are presented under the headings: topic modelling and URL subject content; frequency of tweets and circulation of scientific resources; popular domains; and user profiles with the most interaction. These findings are then discussed under headings corresponding with the research questions. Our conclusions and suggested further work are presented at the end of the paper.
Analysing tweets in the longitudinal context of an ongoing crisis such as COVID-19 provides novel insight and understanding into the dynamics and influences of crisis communication [8]. The research, therefore, contributes to developing understanding of the digital information ecology, in particular the circulation of scientific information and expertise in an ongoing crisis context which is subject to uncertainty and emerging scientific evidence. This includes insight into when information is shared, including concerns about the recirculation of out-of-date articles, and the plurality of information resources and communicators online in information circulation, including the dominant role of traditional media but also some scientific organisations and individuals.
Changes in the guidance and use of face masks
Hand washing and physical distancing were encouraged since the introduction of the UK’s COVID-19 restrictions in March 2020, however, the use of face masks was not mandatory, or even recommended by the Government and its devolved nations until a later date. The first devolved nation to recommend use on public transport was Scotland on 28 April 2020, whilst the first nation to mandate use on public transport was England from 15 June 2020. These requirements were followed by additional restrictions including the use of masks in shops [12]. As shown in Table 1, the changes in guidelines and mandating masks varied between the devolved nations during 2020.
The purpose of using face masks for the public is to reduce source virus transmission, particularly from asymptomatic or pre-symptomatic individuals when physical distancing is not possible or predictable [9, 25]. The Scientific Advisory Group for Emergencies (SAGE) and its specialist sub-committees provide advice for the UK Government [26]. Near the start of the then epidemic (4 February 2020, before COVID-19 was classified as a global pandemic by the WHO), SAGE was advised by the New and Emerging Respiratory Virus Threats Advisory Group (NERVTAG) that there was ‘limited to no evidence of the benefits of the general public wearing facemasks as a preventative measure’ and face masks were only advised for health and social care workers visiting people who may be infectious, whilst surgical face masks were recommended for symptomatic individuals [27]. As more studies were conducted on COVID-19, the scientific evidence developed [10]. Minutes from a meeting of SAGE on 21 April 2020 (released 29 May 2020), states the evidence of masks for source control and for protecting the wearer from infection is weak, but marginally positive for the use of masks. As a precautionary principle, SAGE, therefore, recommended the use of cloth masks in some higher risk settings where social distancing was not always possible, giving public transport and shops as examples but also advising that any policy decision should not jeopardise the supply of masks to those in settings where there was stronger evidence on their effect e.g. health and care settings [28]. Furthermore, on 4 June 2020, the Environmental Working Group, one of SAGE’s sub-committees released a report on COVID-19 transmission, advising that if 2m face-to-face distancing could not be achieved, additional mitigation measures including (but not limited to) face masks and minimising the duration of exposure should be undertaken [29]. At a similar time (5 June 2020), the WHO updated their guidance, which now included advising governments to encourage the general public to wear masks in areas with widespread transmission and where physical distancing was difficult, giving public transport, shops and confined crowded environments as examples. In his opening remarks at a media briefing, the WHO’s Director-General stated these changes were based on ‘evolving evidence’ [30].
These changes in face mask recommendations for the public required behavioural changes. During (and beyond) our study period, Imperial College London and YouGov’s COVID-19 behavioural tracker was a global survey collecting fortnightly data exploring the public’s attitudes and health behaviours [31]. When asked ‘Thinking about the last 7 days…how often have you worn a face mask outside your home to protect yourself or others from coronavirus (COVID-19)?’, in the 15 May 2020 UK survey, 15% answered ‘always’ and 63% ‘not at all’. There was a notable increase in ‘always’ responses from the 3 July 2020 when 17% answered always, to 55% on the 29 July 2020, with a further 20% answering ‘frequently’. These dates correspond with some of the policy changes shown in Table 1.
The Royal Society and British academy [9] reported the uptake of mask use was slow in the UK due to a lack of clear recommendations for the lay public attributed to i) an over reliance on an evidence-based medicine approach and therefore an assertation the evidence was weak; ii) inconsistent and changing advice from supranational organisations, such as the WHO; iii) concern over the applicability of scientific findings in different contexts e.g. countries and types of pandemic; and iv) mix of supply concerns for Personal Protection Equipment (PPE) e.g. difference between surgical face masks for medical professionals and masks recommended to the public. A separate study (conducted 17–20 July 2020) by King’s College London and Ipsos MORI [32] showed that 13% of interviewees felt the government only wanted the lay public to wear face masks as a way of control and 10% felt wearing them was bad for their health. Due to these objections and/or hesitancies, mandating the use of face masks did cause controversy for some members of the public [33].
Elsewhere globally, reasons provided for higher levels of adherence of face mask use in China and Hong Kong compared to other countries were government advertising campaigns, health education programs, imposing punishment for those not wearing a mask and previous experiences in outbreak management during the 2003 Severe Acute Respiratory Syndrome (SARS) epidemic [34–37]. In contrast, face masks not being a cultural norm, alongside mask shortages and mixed messaging about their use, were provided as reasons for lower accordance of mask-wearing in comparison to other mitigation strategies in Malaysia [4]. These attitudes and practices towards face mask use are often informed by people’s knowledge and risk perceptions [7]. This in turn relates to the public’s understanding of science and which can be affected by the information available to them, and therefore the digital information ecology (for those that use the internet).
Digital information ecology and crisis communication
Following Perreault & Perreault’s work on the Communication Ecology of Pandemic reporting [38], we acknowledge that there are several microenvironments within the crisis communication ecology, and this study examines how journalistic outputs, citizen commentary, governmental and public health messaging, as the associated communication practices coincide. Each warrants consideration, as the motivations and agendas of different actors in the network contribute to the wider communication ecology. Despite increased efforts to understand the digitally mediated communication practices in both scientific and crisis communication contexts, there is no systematised theorisation that attends to these overlapping and related concerns. Developing this understanding is therefore key for effective crisis communication [39].
For this investigation we focus on one platform in this digital information ecology, Twitter. Twitter is a type of microblogging social media platform which is and has been used as a platform for rapid and widespread sharing of information in crisis situations, including previous pandemics such as H1N1/Swine Flu and the Zika outbreak, as well as for controversial health issues such as vaccines [8, 11, 40–42]. Crisis events, such as the COVID-19 pandemic, are often subject to uncertainty, where there is a lack of information and a need for sensemaking [43]. Infodemiology methods can provide ‘direct and honest data on health information seeking behaviour’ [44] and be useful for identifying how the public use and share information sources [41]. Using previous Twitter studies as precedents [11, 45–50], we focus on the following in our study: tweet content, frequency, external resources for information, and users circulating information. Each of these aspects of the digital information ecology, alongside relevant theoretical perspectives, are now discussed in turn.
Tweet content
Separate studies have shown that a high proportion of tweets are informative, meaning they seek to share information or updates [48, 51]. Furthermore, Twitter can be used to address emotional needs e.g. an individual expressing concerns with their online network [43]. Existing theoretical research, on Agenda-Setting, for example, has considered how media reporting, as well as how policy and public agendas, direct public attention towards particular issues. Concluding, topics with high relevance and uncertainty often lead to a higher need for orientation [52]. Identifying the content of Tweets can help to identify topics of interest at particular points in time [53]. For example, one study (conducted 2 February– 15 March 2020) identified twelve topics of concern during the pandemic, one being ‘face-mask wearing’. These ‘face-mask wearing’ tweets commonly focused on the importance of mask use in reducing the spread of the virus and shortages in PPE supplies [45].
Tweet frequency
The ongoing and long-term nature of the COVID-19 pandemic contrasts the majority of existing studies on risk communication which tend to focus on ‘short-term’ natural hazard events e.g. earthquakes (the event rather than impacts being short-term) [8]. The frequency of tweets about a topic over time can be used as an indication of public attention about that topic. Previous studies have shown that public attention to a topic can be short, referred to as ‘attention saturation’. This ‘attention’ includes the public’s awareness, risk perceptions and tendencies towards behavioural changes [54–56]. An example being the heightened attention levels caused by local COVID-19 cases leading to people seeking information about symptoms. As mixed messaging, including inconsistent and unclear messages, from governments has been identified in papers as a hindrance to the public’s understanding and adoption of behavioural change [11, 57], including face masks [9], arguments have been put forward encouraging policy makers to take advantage of the period of active information seeking to ensure clear advice is given to the public and/or to reinforce messaging [54, 55].
External resources of information on Twitter
The rise of the internet and social media over the past few decades has substantially changed the media environment, including the role and input of traditional media (i.e., newspapers which were traditionally in print). Their role has been challenged by providing other channels to alternative information [58]. This includes the use of external resources and knowledge on Twitter [8]. In crisis situations, this volume of resources is often exacerbated as the public are exposed to more competitive and conflicting information [39]. Concurrently, as outlined in Media Dependence Theory studies, in the absence of credible and complete information from official sources the public seek alternative channels of information [59]. The high volume of information during the COVID-19 pandemic has led to high levels of medical misinformation propagation, with challenges of reducing its spread being coined as an ‘infodemic’ [47, 51, 60, 61].
One way of capturing the dissemination of external resources and the credibility of information is through the analysis of the links to web addresses, commonly known as URLs. One study of COVID-19 related tweets identified that high quality journals and information were rarely circulated, and identified the top domains mentioned in COVID-19 related tweets were social media websites, including YouTube and Instagram, in addition to broadcasting and news media websites such as the BBC and the Guardian [46]. In another study, links were given scores based on their credibility [47]. They found that links to what they defined as low-credibility sources accounted for 0.89% of the total tweet volume, which was comparable to the volume share (0.98%) of the New York Times, a traditional media outlet which they describe as a reliable source. They also found low-credibility sources to have a higher tweet volume than the Centre for Disease Control (CDC), the USA’s health protection agency, which accounted for 0.65%. Another study expresses a preference towards sharing tweets containing official government-based resources and evidence-based public health information after their analysis of tweets by G7 world leaders, in which only 28.6% of tweets classified as informative contained links to official sources [48].
Active participants in information circulation
The Social-Mediated Crisis Communication (SMCC) model conceptualises ‘information source’ as where the crisis information originates. This can be either an organisation experiencing the crisis or a third party, including a journalist or social media creator [62]. Alongside the diversity of digital platforms, changing models of scientific crisis communication and intensifying digitalisation have increased the diversity of communicators and these ‘information sources’. It is now possible for anyone to enter the ‘communications fray’ and ‘audiences are turning into active participants’ in public discourses on science–this has led to a pluralistic and participatory digital media ecology for science [63], but also increasing polarising views and controversy [38].
A critique of crisis communication theory is the focus on organisations, rather than how publics communicate with each other, leading to calls to focus on multiple actors engaged in crisis response [62, 64]. Furthermore, within this changing digital information system, Agenda-Setting theory recognises that these different actors (i.e., news media and individuals) are unlikely to have the same agendas when disseminating information [43]. Within this theoretical framework, there have been suggestions that traditional news media have lost some of their power and visibility in information circulation due to the plurality of voices on Twitter. However, some studies, foci including COVID-19 and climate change, have found that traditional news media still have a high profile–thus the increase of ‘information sources’ and actors has modified the traditional media’s role rather than eliminated it. [43, 63, 65]. Within this information landscape, concerns have also been expressed that science communicators are not achieving a high profile due to the crowded digital ecology, yet there are arguments that there is scope for them to do so [63].
There is commonly a perception that official sources are more credible than unofficial sources and that government health agency and news media are more successful than their social peers for ‘debunking misinformation’ in a public health crisis [39]. In the context of COVID-19, one study (conducted 27 February 2020 using a sample of 673 tweets) [51] found a high prevalence of informative tweets were from ‘informal individuals or groups’ (66.6%) and ‘news outlets or journalists’ (16.5%). In contrast to this study, an analysis of tweets collected from 21 January—31 March 2020 found the most active users were from verified accounts belonging to news sources and political figures, commenting that these influential figures and platforms often use Twitter to disseminate breaking news and amplify their messaging [49]. However, even if verified, studies have also shown that tweets from political figures can be prone to misinformation, which can easily be amplified [50]. Due to this potential influence, recommendations have been made to closely monitor branded organisations and celebrity user accounts to prevent the spread of misinformation at an early stage [61].
Our own research uses the concept of the digital information ecology and these infodemiology methods to frame, and inform, our analysis of Twitter data focusing on face mask specific tweets. We evaluate what got tweeted and by whom, aiming to develop understanding on how these information sharing practices could have affected the public’s understanding of the scientific evidence and expertise in relation to face masks. The following section outlines the methods used to answer the research questions outlined in the introduction.
Method
Data collection and ‘Twitter datasets’
Our analysis utilises Banda et al.’s [66] COVID-19 Twitter chatter dataset which includes IDs of tweets relating to COVID-19. We used a subset of these tweet IDs to create a dataset of tweets covering the period between 22 January and 1 August 2020. Data was collected via the Twitter Application Programming Interface (API) using the python library Twarc, which allows you to search for tweets by ID and retrieve the tweet text and associated metadata, which included the date the tweet was created at, the number of retweets it received, and the user profile description.
To focus on discussion relating to the use of face masks, the dataset was filtered based on a selection of keywords. Specifically, only tweets that mention `face masks’ or `face coverings’, and possible spelling variations on these keywords (e.g. facemask, face-masks, face-covering, face-coverings, etc…) were included. To focus the dataset more closely on the UK-specific discussion of face masks, the data was filtered in several ways. Where available, geotag data or the optional location field in user profiles was used. While location data from profiles is self-reported by the user and may not always be accurate or up to date, it provides some indicator of location or affinity with the UK. Additionally, the dataset was filtered to only include English language tweets. Retweets and duplicates were also removed. This resulted in a total dataset of 95,876 unique tweets, which we call the ‘face mask tweets’ dataset.
In total, there are four ‘Twitter datasets’ referred to in the analysis section: ‘face mask tweets’, ‘most retweeted tweets’, ‘most tweeted URLs’ and ‘scientific evidence and expertise URL titles’. As described above, the ‘face mask tweets’ dataset includes all twitter data collected and filtered to face mask specific tweets from the UK. As the main interest of this study is the circulation of information about face masks the remaining three datasets focus exclusively on tweets that contain an external URL. External URLs included in tweets provide links to information outside of Twitter itself and are now a common part of Twitter practice, particularly when a user finds a topic newsworthy [67]. In total, 45,466 tweets containing an external URL were identified and 41,629 unique URLs were embedded within these.
The ‘face mask tweets’ dataset was first filtered to remove any tweets which did not include an external URL (i.e. removing those with no URL or a link to another tweet rather than source outside of Twitter), before the two separate sample datasets were created. Samples were then taken to facilitate a manual analysis. The ‘most retweeted tweets’ dataset sample contains tweets with an external URL which have more than ten retweets, accounting for 2,333 tweets, and representing 5.13% of the total ‘face mask tweets’ containing a URL. The ‘most tweeted URLs’ are URLS contained within more than 10 unique tweets in the ‘face mask tweets’ dataset. This dataset contains 333 URLs, accounting for 0.73% of the unique URLs. Previous studies have shown, the use of the retweet count (used in the ‘most retweeted tweets’ dataset) can be an indication of tweets that the most users express: the significance of information to their followers, feelings and/or get feedback/alert others [68, 69]. Retweets are used rather than likes in this paper as the focus is on the circulation of information, hence they are an important aspect of the social media information ecology [11, 42]. The ‘most tweeted URLs’ originate from users selecting links for inclusion and deeming them newsworthy enough to share with their network. The URL count is an indication of the number of tweets where different Twitter users chose to include a specific URL in their own tweet.
In both the ‘most retweeted tweets’ and ‘most tweeted URLs’ datasets the URL subject content was characterised (process described in the next section). To focus on ‘scientific evidence and expertise’, both datasets were filtered further to those where the URL subject matter was categorised as this i.e., reports on/discusses scientific evidence and/or expertise. As some URLs led to the same website, one reason being shortened URLs, duplicate titles were removed and for each title, total retweets and number of unique tweets containing the URL were calculated. This process of removing duplicates and identifying the shortened URLs was not undertaken for the other datasets due to feasibility of manually checking each title was unique and ensuring different links were not accidentally merged. In total 230 URL titles were identified across the ‘most retweeted tweets’ and ‘most tweeted URLs’ datasets categorised as scientific evidence and/or expertise. The datasets outlined were analysed in a variety of ways, which are now described.
Data analysis
An overview of the content analysed in each of the datasets described is provided in Table 2. This includes: topics and keywords within tweets, URL subject content, views towards face mask use for those categorised as ‘scientific evidence and expertise’, overall frequency of tweets, URL domains, and user profiles sharing URLs. Excluding the full ‘face mask tweets’ and user profile names (see sections about ethical and Twitter data considerations), all of these datasets are provided in S1 Appendix.
In this study, three timeframes (TFs) are used to separate the data (Table 3), corresponding with the first announcements of changes to the face mask recommendations and policy by one of the UK’s devolved nations.
Pre-processing and topic modelling.
The ‘face mask tweets’ dataset was pre-processed to remove URLs and stop words (i.e. words that feature commonly but do not include semantically rich information), all text was converted to lower case and the data were tokenised and lemmatised, grouping words that have the same stem (i.e. ‘thanking’, ‘thanks’ become ‘thank’). Due to the size of the ‘face mask tweets’ dataset, computational approaches for text analysis were employed. To identify common topics discussed within the data and help contextualise our study, we applied a commonly used probabilistic topic modelling for text analysis [53]. Latent Dirichlet Allocation (LDA)-based topic modelling is an unsupervised machine learning technique for the automatic description of documents [70]. Applying LDA to the dataset of tweets returns a set of topics, with each tweet described as a distribution of these topics. A label is generated by the researchers after manually inspecting 50 examples with the highest topic contribution for each topic. Each topic is also assigned a weighting to reflect how prominent it is in the observed data instance.
The LDA model was implemented using the python package Gensim [71] (with parameters: n-grams 1–3, passes = 20, iterations = 100, random_state = 100). The number of topics was manually selected based on interpretability and performance, with 10 topics providing the most coherent and interpretable output.
URL subject content.
The URL subject indicates the content of the information to outside resources being shared and/or circulated by Twitter users. For both the ‘most retweeted tweets’ and ‘most tweeted URLs’ datasets, the URLs titles were manually collected. A manual inductive coding process was then used to categorise the content of URLs based on their heading and sub-headings. Rather than using a pre-defined list, ‘researcher 1’ analysed all the URLs and identified emerging themes, whilst ‘researcher 2’ coded a random sub-sample (approximately 10%) of the data. After this first step, the categories were then refined following discussion between the researchers. Using the refined list of codes, the process of categorising the URLs was repeated. A full list of these codes and descriptions are provided in S2 Appendix.
To validate the coding between the two researchers, their categorisations were compared (see S3 Appendix). For the URL subject content’s main themes, our agreement was 75.91%, whilst sub-categories (of which there were more choices) was 64.55%. These variations were then discussed by the two researchers to identify if there was significant disagreement or if different categorisations were due to multiple categories being applicable and the inherent subjective nature of inductive coding. The latter was always the case.
Scientific evidence and expertise.
To facilitate a greater understanding of the scientific information being circulated, the top 25 URL titles in the ‘most retweeted tweets’ and ‘most tweeted URLs’ datasets (ranked by number of retweets or tweets) focusing on ‘scientific evidence and expertise’ were examined. By reading the entirety of their content we identified if the article supported/recommended the use of masks for the lay public, did not recommend/support use, or stated the evidence was inconclusive. The top 25 were chosen as they provided what we deemed a sufficient overview of the most circulated URLs, yet was feasible to read through the entirety of the content.
Frequency of tweets.
The overall frequency of tweets indicates the interest in face masks (determined by the presence of face mask/covering keywords in the text). Generating a frequency count of the total number of tweets per day in the ‘face mask tweets’ dataset provides an overview of changes in volume of tweet activity relating to face masks in the period studied (22 January—1 August 2020), and whether these corresponded with key guidance/policy changes. We also include Google Trends [72] data as an indicator of people’s search interest in ‘face masks’ or ‘face coverings’ on Google’s search engine.
Furthermore, due to the evolving nature of the scientific evidence and change in guidance, an analysis was undertaken examining the point in time the links referring to scientific evidence and/or expertise were shared. Using the date when the associated tweets were created at, it was possible to analyse when URLs appeared over the study time period.
Domains.
Categorising URL domains explores what type of content and information resources are being retweeted or shared [46–48]. We obtained descriptions of the domains from either the website itself or alternative sources (also available in S1 Appendix). Using the same manual coding process as the URL subject content, the domains in the ‘most retweeted tweets’ and ‘most tweeted URLs’ were then classified based on the researchers’ interpretation of descriptions of the domain host available on the domain website (see S2 Appendix for full list of categories). Our inter-annotator agreement was 76.79%. As with the subject categories, discrepancies were due to the applicability of several categories to one domain rather than a significant disagreement.
User profiles.
The analysis of user profiles allows for a better understanding of who is circulating information [61] and who the ‘major players’ are [42]. These were also manually examined using an inductive coding technique based on the user profile description at the time the dataset was created (August 2020). Each user was identified as either an individual or an organisation, and then the professional background assigned (see S2 Appendix). Unlike, URL subject content and domains, this analysis was limited to the ‘most retweeted tweets’ dataset. The inter-annotator agreement was 77.11% for the profession categories, which again were corroborated to ensure there were no significant disagreements.
Ethical considerations
Ethics approval was obtained by the lead researchers’ department, the Centre for Research in the Arts, Social Sciences and Humanities (CRASSH) at the University of Cambridge. As part of these considerations, informed consent was not deemed necessary as full tweet content has not been shared, nor the usernames associated with Twitter accounts of individual people (rather than organisations) in the paper or within the Supporting Information. Those individual accounts that may be identifiable from the discussion are those with a large following or interaction rate. The data has been stored on a password protected computer and cloud server. The ethics statement was approved on these principles.
Twitter data considerations
The study uses publicly available data in accordance with the Content Redistribution clause under Twitter’s Developer Agreement and Policy [73]. In compliance with Twitter’s Terms and Conditions, we are unable to publicly release the text of the collected tweets. However our publicly available dataset (S1 Appendix–Excel Tab ‘Most Retweeted Tweet dataset’, accessible via PLOS ONE’s server) includes the Tweet IDs, which are unique identifiers tied to specific tweets, of publicly available posts, in compliance with the Twitter Terms of Service [73]. These are openly available for others to use in compliance with Twitter’s Terms and Conditions. As previously outlined, our analysis used Banda et al.’s [66] COVID-19 Twitter chatter dataset as a starting point.
Limitations
While studies of social media can provide insights into public opinion and discussion on a given topic there are limitations. Firstly, the keyword sampling and data filtering techniques, such as excluding any tweet that did not provide some indication that the author was associated with the UK, will result in a bias in the type of data collected. Consequently, it is inevitable that such an approach will only capture a proportion of the discussion. Secondly, while the study can provide insight into information sharing practices on Twitter, the information ecology is clearly much more complex and comprised of multiple different social media and messaging platforms through which information is exchanged. Thirdly, our study is limited to the study period (22 January 2020–1 August 2020). As social distancing restrictions eased over the summer of 2021, including the change back from mandatory use to recommending use of face masks in indoor spaces in England, there was a re-emergence of tweets about face masks, as well as accusations in the media about mixed messaging [74]. Lastly, the manual analysis focuses on relatively small samples of data. This enabled a more qualitative exploration of the content than could otherwise have been achieved through automated techniques. As this takes an inductive approach for the interpretation and classification of content, there is always an element of subjectivity in the process. For instance, other researchers may have categorised the content using different terminology [48]. The subjective nature of classifying content can also be seen via differentiations in our own inter-annotator agreement scores (S3 Appendix). Despite these limitations, the universal character of our study contributes to developing understanding of the digital information ecology and provides practical insights for crisis communicators.
Analysis of Twitter content
Topic modelling and URL subject content
Topic modelling was used to identify common topics discussed within the data which helps to provide an overview on different aspects associated with face masks being discussed. The results of the ten topic LDA topic model, with the salient keywords for each of the topics are displayed in Table 4 (further data provided in S4 Appendix).
Overtime, we found there were downward trends in topics 6, 9 and 10 which relate to advocating for the use of face masks, PPE supplies and the global situation including the number of COVID-19 cases around the world. Upward trends are observed in topics 2, 5 and 7, categorised as: the adoption of mask wearing and a mix of discussion of pros and cons; shopping and resuming everyday activities with face masks; and announcements about the mandatory use of masks.
A variety of themes were also identified from the inductive analysis of URL subject content, with some themes comparable to the topic modelling e.g. downward trend for PPE supplies, and upward trend for guidance and policy related content (see Fig 1, with further data provided in S4 Appendix).
After the first announcement of masks being mandated (the third timeframe), the topics became dominated by information related to guidance and policy, scientific evidence and expertise, and COVID-19 cases and guidelines in other countries. Art is also dominant here due to the most retweeted tweet (Table 5) containing a URL about a well-known artist, Banksy: ‘Coronavirus: New Banksy piece on London Tube encourages people to wear masks’ [75]. The tweet received 10,786 retweets, with the next most retweeted tweet receiving 3,194 retweets. Meanwhile, other topics including ‘exemptions and disability concerns’ have not been discussed as much in comparison.
There is a notable difference in the proportions of retweets/tweets relating to ‘guidance and policy’ when comparing the overarching themes for the ‘most retweeted tweets’ and ‘most tweeted URL’ datasets (see S4 Appendix), with a higher proportion of ‘most tweeted URLs’ being categorised as ‘guidance and policy’ respective to the other categories in the dataset. As with the ‘most retweeted tweets’, the subject content categorised as ‘scientific evidence and expertise’ was popular in comparison to some of the other categories in the ‘most tweeted URLs’ dataset. Overall, 18.36% of the ‘most retweeted tweets’ and 12.87% of the ‘most tweeted URLs’ were categorised as ‘scientific evidence and expertise’.
Clearly, several topics related to face masks were discussed and circulated on Twitter with the communication of science becoming particularly evident as the guidelines changed. From those ranked in the top 25 according to retweet count categorised as ‘scientific evidence and expertise’, 15 supported use for the general public, 8 did not, 1 said the evidence was inconclusive, and another focused solely on the use of masks for medical staff. For the ‘most tweeted URLs’, 19 supported use, 4 did not support and two said the evidence was inconclusive. This variation of recommending/not recommending (or supporting/not supporting) use is also reflecting in Tables 6 and 7 which show the top five URL titles based on the sum of retweets and number of unique tweets containing the URL (all 25 URL titles are provided in S1 Appendix).
By focusing on the external resources categorised as ‘scientific evidence and expertise’, it is clear there was mixed messaging about their use. As the guidance changed overtime, we now focus on when information was being circulated and the public were seeking/sharing information.
Frequency of tweets and circulation of scientific resources
Face mask specific tweet frequency indicates the interest in face masks overtime. The frequency of tweets with and without a URL within the ‘face mask tweets’ dataset is presented in Fig 2A, alongside annotations of the key guidance and policy changes in England (raw data is provided in S4 Appendix). Fig 2B. includes Google Trend [72] data which indicates the public’s search engine behaviour/interest in the topics ‘face masks’ and ‘face coverings’.
a) Frequency of ‘face mask tweets’ with and without URLs. Annotated with key guidance and policy changes in England b) Google Trends data for search terms ‘face masks’ and ‘face coverings’ in the UK (Data source: Google Trends [72]).
For both tweet frequency and Google Trends [72] data, the results show clear peaks and troughs in either frequency or search interest corresponding with England’s guidance/policy changes, with the biggest peak for tweet frequency being 14 July 2020 when it was announced that face masks would become mandatory in England’s shops. Fig 2A also shows the proportion of tweets containing a URL. Overall, 46.58% of the tweets contained a URL but this decreased across the three timeframes posed (pre and post recommendations and mandating mask use).
When focusing on the external resources categorised as ‘scientific evidence and expertise’, we identified that some of the URLs, particularly those published earlier in the pandemic contained an editor’s note recognising that the scientific evidence had developed since the publication of the article. For example, in the URL entitled ‘Coronavirus: ’We do not recommend face masks for general wearing’, it now states ‘Update: This video contains advice that has since been updated—people across the UK now have to wear face coverings in certain circumstances when out of the house’ [80]. For this reason, we analysed the point in time that the URLs were shared. Fig 3 shows the results for two of the articles not recommending use [80, 81].
Annotated with corresponding policy/guidance changes.
In the case of the URLs shown in Fig 3, they resurfaced on Twitter after their initial publication, this was when the policy changed (or policy changes were announced) and face masks became mandatory in shops. The external URLs were contained in more tweets at this later point in time than their original publication date. A review of these tweets indicate that people were mainly sharing the articles alongside statements claiming that face masks were not beneficial and using them as evidence in arguments with other users, often referencing the expert status of the source (e.g. ‘England’s most senior doctor’). A smaller proportion referenced the articles in tweets calling attention to and questioning the changing guidance and recommendations.
These results provide useful insight into the ways public interest intensifies at certain points in a crisis, in addition to the inclusion of external URLs as a key information sharing practice. We have also identified that challenges can arise due to the prolonged circulation of out-of-date information. We now move on to identify the sources of these external resources feeding into Twitter discussions.
Popular domains
Beyond the content of external URLs, it is also important to understand the creators of the information resources being retweeted. We use domains as an indication of this. Fig 4 (with further detail provided in S4 Appendix) provides a comparison between the percentage of retweets/tweets in the domain categories in the ‘most retweeted tweets’ and ‘most tweeted URLs’ datasets.
Many domains identified in the analysis belong to journalism or broadcasting websites. In the ‘most tweeted URLs’ dataset, there is more of a spread of domain categories, although journalism and broadcasting remain dominant. Excluding URL shortening, official government websites were the next most retweeted/tweeted domain category but noticeably lower. For the filtered scientific evidence and expertise dataset, the proportion of domains for academic, scientific journals and organisations is greater, but journalism and broadcasting both remain dominant.
To provide more information on what the most popular domains were, the top five domains based on the total number of retweets/tweets for the overall datasets and those classified as ‘scientific evidence and expertise’ within these are shown in Table 8.
Within the highest ranked domains, those categorised as journalism or broadcasting include: The Independent, The Guardian, The Daily Mirror and the BBC. In a survey (conducted April 2019 –March 2020) assessing the monthly reach of leading national newspapers in the United Kingdom [87], The Independent, Guardian and The Daily Mirror are ranked within the top 5 (taking into account phones, tablets, desktops and prints). However, the two highest ranking newspapers in the survey, The Daily Mail and The Sun, only account for 1.17% and 0.96% of total retweets in our ‘most retweeted tweets’ dataset (ranks 20th and 24th). Therefore, these results suggest that the reach of these newspapers may vary depending on the type of platform being used or topics being discussed.
Turning to official resources for information, the WHO’s (https://www.who.int) and UK Government’s official domains (https://www.gov.uk) rank 34th and 14th in the ‘most retweeted tweets’ and 6th and 7th in the ‘most tweeted URLs’. When filtered to scientific evidence and expertise (and shortened URLs had been identified), the Lancet, an academic journal, ranks top of the ‘most retweeted tweets’. Oxford University was 5th in the ‘most tweeted URLs’. The WHO ranks at 17th in the ‘most retweeted tweets’ but does not feature in the ‘most tweeted URL’ sample, and the UK government domain does not feature in either filtered ‘scientific evidence and expertise’ dataset samples. We now discuss the users that are sharing this information.
User profiles with the most interaction
The analysis of user profiles indicates the type of people and accounts that chose to tweet about face masks and who were subsequently retweeted. The distribution of retweets attached to organisations compared to individuals was 58.16% and 41.84% respectively. When this was filtered to those categorised as ‘scientific evidence and expertise’ the distribution was similar: 54.24% and 45.76%. There was also variation in the user profession categories (see Fig 5 with further detail provided in S4 Appendix).
User accounts classified as n/a are not included.
The majority (73.87%) of user profiles classified as organisations in the ‘most retweeted tweets’ dataset represents broadcasting/journalism platforms, whilst 11.49% represents academic institutions. At an individual level, the types of users were more evenly spread between the categories. Notable differences include a higher proportion of those in the creative industries (meaning those describing themselves as an artist, author or similar profession); political/government positions; and medical and health care professionals. The proportion representing academia/research is similar to the organisational level. Due to these differences, the proportion representing the broadcasting/journalism profession is lower for those accounts belonging to individuals compared to organisations.
When filtered to focus on those URLs classified as ‘scientific evidence and expertise’ there is a large proportion (63.11%) of retweets, particularly at the organisational level, for those representing academia, research or universities in comparison to 34.91% from broadcasting and journalism organisations. At the individual level this breakdown is 24.91% and 17.33%. Although individuals connected to government/local authorities represent 6.30% of total retweets of URLs about ‘scientific evidence and expertise’, government organisations were at 0.0% in the sample taken.
As with the domains, it is useful to see examples of users in these categories. The top five user profiles ranked by total number of retweets (Fig 6A) are all organisations representing broadcasting or journalism platforms: SkyNews, BBCWorld, Independent, Guardian and BBCWorld. When extended to the Top 10 ranking user profiles, this list includes The Lancet (an academic journal) and individual user accounts. Based on their user profile descriptions, these individuals include an author & public speaker, whose Twitter account has since been suspended; a university professor; a former medic; and a journalist. If focusing on those categorised as ‘scientific evidence and expertise’ (Fig 6B) there are more accounts representing academia/research/universities and an additional account for an individual person within the top ranking 5. In the top 10, there is another individual account, which has now been suspended from Twitter. The dominance of a small group of user accounts over the total retweet volume can also be seen in Fig 6A and 6B.
a) ‘Most retweeted tweets’ dataset b) ‘Most retweeted tweets’ dataset filtered to URLs classified as ‘scientific evidence and expertise’.
This analysis section of the paper has presented our results for a range of aspects associated with the circulation of tweets and external URLs focusing on face masks during the COVID-19 pandemic. We now use this analysis to answer the research questions posed at the beginning of this paper.
Discussion
An overview of the research questions, datasets, methods used and key findings from our analysis is provided in Table 9. We discuss each of these in turn, including the theoretical and practical implications, in the following sub-sections. As a result of our findings, we also pose new questions that need to be considered in future research and tweeting practices in pressurised environments, such as an ongoing public health emergency.
What type of information about face masks was deemed as newsworthy and shareable?
The pandemic, with its global reach, was highly relevant to a wide audience, because the events had such a direct and dramatic impact on individual’s daily lives, everyday activities, and behaviours. Consequently, coupled with the high levels of uncertainty that were exacerbated by the contested value of face masks during this period, the need for orientation in this context was acute. Changes in priorities and focus of Twitter users are revealed by the shift in topics and URL subject content (Table 4 and Fig 1) over the timeframes studied (before recommending masks, after recommended and after mandatory). The shift and increasing focus on guidelines/policy in the ‘most retweeted tweets’ dataset, particularly after they became mandatory in England’s and Scotland’s shops, suggests that interest in a topic is linked to the latest news and major events–as covered in Agenda-Setting theory, a policy agenda/change has therefore directed the public’s attention [52]. As the guideline/policy changes were linked to developments in the scientific evidence, this provides a possible account for the rise in the number of URLs referring to scientific evidence and/or expertise. Additionally, as ‘scientific evidence and expertise’ was a popular subject category for external URL content, this type of information is clearly circulated on Twitter and is likely to have influenced Twitter users’ understanding of science, therefore supports the notion that the science communication landscape is changing through the digitalisation of the information ecology [63].
When did users choose to share information about face masks during the COVID-19 pandemic?
The overall frequency of tweets relating to face masks reflect how Twitter users’ interest on the topic waxed and waned at various points during the pandemic (Fig 2). In particular, peaks in the number of tweets related to face masks further supports our argument that there was an increase in the public’s situational awareness as the guidance and policy changed. Similar observations were made for people seeking information about ‘face masks’ or ‘face coverings’ on Google Trends [72] i.e. upward trends at the points of guidance/policy change. As these peaks (both tweets and Google searches) only lasted a few days it supports the notion that public attention is short, and people move onto the next topic of interest. It is at this point in time that those implementing policy and supporting the changes need to take advantage of people searching for or sharing information [54–56]. Additionally, the circulation of external resources of information and evidence, as expressed through URL sharing, was more prominent in the early stages of the pandemic, when sensemaking processes were most active [88, 89]. One explanation for this trend is that as face masks were not a cultural norm in the UK, and as the science and guidance was still emergent, this created an information gap; as a result, individuals were more actively engaged in information seeking and sharing practices. In line with studies using Media Dependence Theory as a framework, these observations show that people were demanding information because of the situational ambiguity [59].
While studies have developed theoretical standpoints on how to mitigate the effects of misinformation in public health crises [90], what our findings show is that specific challenges can arise due to the prolonged circulation of out-of-date information, i.e. not strategic misinformation, nor “mis”-information at all. The change in the advice and messaging about face masks is evident in the analysis of URLs circulated referring to scientific evidence and expertise (Tables 6 and 7). Part of the understanding of science is that it is constantly evolving, and this was provided as the reason for the change in guidance (for instance in Boris Johnson’s briefing on the 11 May 2020 [91]). However, the recirculation of articles not recommending use (Fig 3) indicates a lack of public consensus, with some individuals questioning the changes and, in some cases, using out-of-date, and since contested, articles to support their own views [10]. Although some URLs contain an editor’s note, the content of a URL may not be read before it is retweeted, with only the headlines captured [92]. As widespread endorsement of myths, conspiracy theories and misinformation that counter the scientific evidence can inhibit preventative behaviours [5, 51, 93–95], this recirculation of out-of-date articles is likely to undermine the changing recommendations and public health messaging and should be countered by official information sources and narratives. How then to contend with this, and how can we ‘undo the science’ if the messaging that came beforehand e.g. not recommending mask use, is out of date? Should scientific experts and public figures hold back from making statements in fear that, as is often the nature of scientific research, these may become subsequently invalidated? Timing is clearly vital and either the removal or updating out-of-date URLs is paramount, as well as utilising those moments that policy changes and the public are seeking information to reinforce messaging.
What are the main external resources of information about face masks?
Another layer of complexity to the digital information ecology identified in our analysis is the volume of external resources that Twitter users are exposed to. In line with other studies about Twitter posts and popular domains [46, 47], broadcasting/journalism platforms represented the majority of the domains in both the ‘most retweeted tweets’ and ‘most tweeted URLs’ datasets. The results suggest that the exposure/circulation of these newspapers may vary depending on the type of platform used (i.e. Twitter being a specific digital platform) or context (i.e. face masks), as two of the highest ranking newspapers in a separate survey on newspaper reach [87], did not rank as highly in our sample. Additionally, global information sharing practices are evident in our sample as international media outlets, such as the New York Times, are represented. A higher proportion of URLs linked to academic journals/scientific organisations and/or universities was identified when the dataset was filtered to tweets categorised as scientific evidence and expertise, but the twitter volume to these domains was still lower than broadcasting and journalism outlets (Fig 4).
As has been evidenced elsewhere in the literature [52, 96], it is the lack of good quality information that drives uncertainty, and as a consequence spawns uninformed, speculative and erroneous content and people will seek information from elsewhere [97]. One notable example from our dataset of an external resource countering the official government guidance was a domain we categorised as online journalism (which is a blog) ranked 10th in the ‘most retweeted tweets’ domains categorised as ‘scientific evidence and expertise’. The content of the identified URLs discouraged the use of face masks. In this case, the domain also had a higher twitter volume (for retweets classified as ‘scientific evidence and expertise’) than official channels of information such as the UK government and WHO within our sample–therefore corroborating with previous findings that unofficial unregulated resources can be circulated as much as, or more than, official resources [47].
One external official resource that is noticeable missing from our sample and these conversations about scientific evidence and expertise is SAGE’s official website which contains (since 29 May 2020 and with a delay [98]) summaries of their meetings and studies feeding into their discussions [99]. Although in some cases, scientific resources such as The Lancet were drawn upon, official government channels of information about the scientific advice were not to the same extent—it is highly likely this is due to accessibility (i.e. readability) of those documents compared to the overviews/interpretations provided by the media. The dominance of particular domains is also highly likely to be influenced by Twitter users, which we now discuss.
Which user’s tweets were most widely circulated?
As outlined in the existing literature, alongside the diversity of digital platforms, there is also a diversity of communicators within the ‘communications fray’ [62, 63]. As our study found that the distribution of retweets attached to organisations compared to individuals was 58.16% and 41.84% respectively, we agree with previous critiques that is it vital for Social-Mediated Crisis Communication theory to focus on how multiple actors engage with crisis response [62, 64].
At an organisational level, user profiles associated with broadcasting/journalism organisations had some of the highest interaction rates (Fig 5). In many cases it is likely that they are posting, and therefore amplifying [49], their own news stories, which is why these distributions are also high in the domain categorisation. This dominance corroborates with previous studies which found traditional news media still have a major role in information circulation despite the plurality of voices that are present [43, 63, 65]. Existing research in Social-Mediated Crisis Communication suggests people prefer ‘official sources’ and encourages the vocalisation from scientists and experts [64], and there are concerns expressed that the plurality of voices can impact the ability of scientific organisations and individuals to gain a high profile [63]. However, there are examples within our sample of this not being the case–for instance, when filtered to ‘scientific evidence and expertise’ URL content, the Lancet, an academic journal was the user with the highest number of retweets. As with traditional media, it is likely this is why their domain also featured highly.
We also found that individual users, including professional scientists, can trigger a similar level of retweets to organisations (Fig 6). Clearly, if they choose to actively engage with the platform, scientists can gain (but this is unlikely to be in every instance) high levels of exposure. However, at the same time, the dominant involvement of non-professional users (in this context referring to those outside the scientific or medical profession) in the dissemination of scientific information, suggests there is potential for these individuals to also have the same level of exposure as media organisations and those considered to be scientific or academic experts [97]. Previous studies have advised that individuals with potential influence are closely monitored to prevent the spread of misinformation [61]. This leads us to question, as the scientific evidence is not always in consensus i.e., some advocating for face mask use before the guidance changed and others countering the official narrative of the Government when they called for mandating mask use(bearing in mind policy decisions consider scientific evidence alongside other factors [98, 100, 101]), where is the line drawn between it being genuine scientific and/or political debate against harmful content and users being suspended–as was the case for two of the user profiles ranking within the top ten in our analysis?
In sum, the power of Twitter users on the digital information ecology and consequently circulation of scientific information should be acknowledged, and addressed, in the formulation of future public health communication strategies, a recommendation also made in the analysis of Twitter content in previous pandemics [102].
Conclusions and further work
This article contributes an empirical analysis of information sharing practices on Twitter relating to the use of face masks in the context of COVID-19. Due to the developments in scientific understanding and changes in face mask guidance and policy, we conclude the current literature on digital information ecology is insufficient for capturing the dynamic nature of a long-term ongoing crisis event. Although existing theoretical frameworks have provided useful approaches for conceptualising user’s motivations for accessing information [43, 96], their assessments of source credibility [47, 59] and organisation risk communication strategies [62], our work highlights that there are specific considerations to be factored in when crisis communication is dependent on emergent scientific evidence. We identified that challenges can arise due to the prolonged circulation of out-of-date information, i.e. not strategic misinformation, nor “mis”-information at all, which can have serious ramifications for crisis communication practitioners working to deliver good quality information. Additionally, our analysis shows changes in the frequency of tweets about the topic correspond with key guidance and policy changes. As recommended by previous studies [54, 55], news media exists within an attention economy, these are points in time official channels of information need to utilise the public’s information seeking and sharing practices. This is particularly important if out-of-date information is being circulated at the same time.
Our study also has implications for advancing communication research in light of how traditional news media still plays a prominent role within information sharing practices on Twitter despite the plurality of voices present and high volume of external resources being circulated. Concurrently, we found there is scope for academic/scientific/research organisations to have a voice within this landscape. Furthermore, as also identified in previous studies [49, 50], individual people, with strong agendas and/or contrasting viewpoints to one another, can have similar levels of Twitter volume (i.e. number of retweets) and therefore exposure as large organisations. This supports the critique that Crisis-Mediated Communication theory needs to focus on multiple actors in the crisis response [62, 64] and special consideration needs to be given to countering inaccurate information from information sources that are not regulated [61]. Overall, the findings of our study provide implications for advancing crisis communication theory and provide practical implications for crisis communicators and misinformation management.
There are several avenues for further work following this research, including a more granular analysis of the datasets produced, extension of the time analysed, and comparison between devolved nations (our study appears to be more reflective of England’s changing guidelines). As we have shown that traditional media and other journalism platforms are so prominent in the circulation of information, further work could inspect whether the scientific evidence and expertise is accurately reflected on these platforms. Furthermore, we have not established if the public discourse impacted media coverage or vice versa [103], further inspection may be able to shed some light on this. This may include identifying echo chambers and/or information cascades [102]. Lastly, we (as have previous studies [52]) discussed that Twitter users are likely to have different motivations for posting and resharing information–however we have not explored these motivations. To truly understand the digital information ecology, including why some posts are shared more than others, and there are differences between people’s original tweets and retweeting behaviours, further work on these agendas is welcomed.
Supporting information
S1 Appendix. Datasets.
Sheets/tabs show the datasets used and researcher’s categorisation of URL subject content, domains and user profiles.
https://doi.org/10.1371/journal.pone.0268043.s001
(XLSX)
S2 Appendix. Codebook and definitions.
Tables display the codes used for manual inductive analysis of text and provides definitions.
https://doi.org/10.1371/journal.pone.0268043.s002
(PDF)
S3 Appendix. Inter-annotator agreement.
Sheets/tabs show the method to calculate the inter-annotator agreement discussed in the paper’s methodology and limitation sections.
https://doi.org/10.1371/journal.pone.0268043.s003
(XLSX)
S4 Appendix. Analysis overview.
Sheets/tabs show our analysis/provide an overview of the Tweet frequency, topics by timeframe, URL subject categories, domain categories and user profiles.
https://doi.org/10.1371/journal.pone.0268043.s004
(XLSX)
References
- 1.
World Health Organisation. WHO | Pneumonia of unknown cause–China. WHO [Internet]. 2020 January 5 [Cited 2021 February 9]. Available from: http://www.who.int/csr/don/05-january-2020-pneumonia-of-unkown-cause-china/en/.
- 2.
World Health Organisation. WHO Director-General’s opening remarks at the media briefing on COVID-19. WHO [Internet] 2020 March 11 [Cited 2021 February 9]. Available from: https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19—11-march-2020.
- 3.
Gov.uk. New campaign to prevent spread of coronavirus indoors this winter. GOVUK [Internet] 2020 September 9 [Cited 2021 February 9]. Available from: https://www.gov.uk/government/news/new-campaign-to-prevent-spread-of-coronavirus-indoors-this-winter.
- 4. Azlan AA, Hamzah MR, Sern TJ, Ayub SH, Mohamad E. Public knowledge, attitudes and practices towards COVID-19: A cross-sectional study in Malaysia. PLOS ONE 2020;15. pmid:32437434
- 5. Bhagavathula AS, Aldhaleei WA, Rahmani J, Khubchandani J. Knowledge, Attitude, Perceptions and Practice towards COVID-19: A systematic review and Meta-analysis. MedRxiv 2020:2020.06.24.20138891 [Preprint] 2020 [cited 2020 June 24]. Available from: https://doi.org/10.1101/2020.06.24.20138891.
- 6. Czeisler ME, Tynan MA, Howard ME, Honeycutt S, Fulmer EB, Kidder DP, et al. Public Attitudes, Behaviors, and Beliefs Related to COVID-19, Stay-at-Home Orders, Nonessential Business Closures, and Public Health Guidance—United States, New York City, and Los Angeles, May 5–12, 2020. Morb Mortal Wkly Rep 2020;69:751–8. pmid:32555138
- 7. Dryhurst S, Schneider CR, Kerr J, Freeman ALJ, Recchia G, Bles AM van der, et al. Risk perceptions of COVID-19 around the world. J Risk Res 2020;23:994–1006.
- 8. Wang Y, Hao H, Platt LS. Examining risk and crisis communications of government agencies and stakeholders during early-stages of COVID-19 on Twitter. Comput Hum Behav 2021;114:106568. pmid:32982038
- 9. The Royal Society, The British Academy. Face masks and coverings for the general public: Behavioural knowledge, effectiveness of cloth coverings and public messaging. R Soc 2020:37. Available from: https://royalsociety.org/-/media/policy/projects/set-c/set-c-facemasks.pdf.
- 10. Hobbs A, Bunn S. COVID-19: July update on face masks and face coverings for the general public. UK Parliam–POST [Internet] 2020 July 19 [Cited 2021 February 9] Available from: https://post.parliament.uk/covid-19-july-update-on-face-masks-and-face-coverings-for-the-general-public/.
- 11. Thelwall M, Thelwall S. A thematic analysis of highly retweeted early COVID-19 tweets: consensus, information, dissent and lockdown life. Aslib J Inf Manag 2020;72:945–62.
- 12.
Public Health England. The Health Protection (Coronavirus, Wearing of Face Coverings in a Relevant Place) (England) Regulations 2020. London, UK: Public Health England; 2020.
- 13.
Department of Health and Social Care. Public advised to cover faces in enclosed spaces. GOVUK [Internet] 2020 May 11 [Cited 2021 February 9]. Available from: https://www.gov.uk/government/news/public-advised-to-cover-faces-in-enclosed-spaces.
- 14.
Department for Transport, The Rt Hon Grant Shapps. Face coverings to become mandatory on public transport. GOVUK [Internet] 2020 June 4 [Cited 2021 February 9] Available from: https://www.gov.uk/government/news/face-coverings-to-become-mandatory-on-public-transport.
- 15.
Department of Health and Social Care, The Rt Hon Matt Hancock MP. Oral statement to Parliament. Face coverings to be mandatory in shops and supermarkets from 24 July. GOVUK [Internet] 2020 July 14 [Cited 2021 February 9] Available from: https://www.gov.uk/government/speeches/face-coverings-to-be-mandatory-in-shops-and-supermarkets-from-24-july.
- 16.
Llywodraeth Cymru (Welsh Government). Press Release: 3 layer face coverings recommended, but not mandatory, in certain situations in Wales. GOVWALES [Internet] 2020 June 9 [Cited 2021 February 9] Available from: https://gov.wales/3-layer-face-coverings-recommended-not-mandatory-certain-situations-wales.
- 17.
Llywodraeth Cymru (Welsh Government). Press Release: All passengers to wear face coverings on public transport in Wales from Monday July 27. GOVWALES [Internet] 2020 July 13 [Cited 2021 February 9] Available from: https://gov.wales/all-passengers-wear-face-coverings-public-transport-wales-monday-july-27.
- 18.
Llywodraeth Cymru (Welsh Government). Press Release: Further constraints on meeting socially and compulsory face coverings to help prevent new coronavirus crisis. GOVWALES [Internet] 2020 September 11 [Cited 2021 February 9] Available from: https://gov.wales/further-constraints-meeting-socially-and-compulsory-face-coverings-help-prevent-new-coronavirus.
- 19.
Scottish Government (Riaghaltas na h-Alba). Coronavirus (COVID-19) update: First Minister’s speech 28 April 2020. gov.scot publications [Internet] 2020 April 28 [Cited 2021 February 9] Available from: https://www.gov.scot/publications/coronavirus-covid-19-update-first-ministers-speech-27-april-2020-1/.
- 20.
Scottish Government (Riaghaltas na h-Alba). Face coverings mandatory on public transport from 22 June. gov.scot. News [Internet] 2020 June 18 [Cited 2021 February 9] Available from: https://www.gov.scot/news/face-coverings-mandatory-on-public-transport-from-22-june/.
- 21. Scottish Government (Riaghaltas na h-Alba). Final Phase 2 measures confirmed. gov.scot. News [Internet] 2020 July 2 [Cited 2021 February 9] Available from: https://www.gov.scot/news/final-phase-2-measures-confirmed/.
- 22. Department of Health, Nidirect. Statement on the use of face coverings. nidirect [Internet] 2020 May 7 [Cited 2021 February 9] Available from: https://www.health-ni.gov.uk/news/statement-use-face-coverings.
- 23. Department for Infrastructure (An Roinn Bonneagair). Mallon announces face coverings to be mandatory on public transport from July 10. nidirect [Internet] 2020 July 2 [Cited 2021 February 9] Available from: https://www.infrastructure-ni.gov.uk/news/mallon-announces-face-coverings-be-mandatory-public-transport-july-10.
- 24. The Executive Office, NI direct. Executive statement on face coverings. nidirect [Internet] 2020 August 6 [Cited 2021 February 9] Available from: https://www.executiveoffice-ni.gov.uk/news/executive-statement-face-coverings.
- 25. Edelstein P, Ramakrishnan L. Report on Face Masks for the General Public—An Update. R Soc DELVE Initative [Internet] 2020 July 7 [Cited 2021 February 9] Available from: https://rs-delve.github.io/addenda/2020/07/07/masks-update.html.
- 26.
Gov.uk. Scientific Advisory Group for Emergencies. 2021 [Cited 2021 February 10] Available from: https://www.gov.uk/government/organisations/scientific-advisory-group-for-emergencies.
- 27.
Gov.uk. Addendum to the fourth SAGE meeting on Covid-19, 4 February 2020. 10 Victoria Street [Internet] 2020 February 4 [Cited 2021 February 10] Available from: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/888771/S0372_Fourth_SAGE_meeting_on_Wuhan_Coronavirus__WN-CoV__.pdf
- 28.
Gov.uk. Addendum to twenty-seventh SAGE meeting on Covid-19, 21st April 2020. Zoom [Internet] 2020 April 21 [Cited 2021 February 10] Available from: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/888799/S0396_Twenty-seventh_SAGE_meeting_on_Covid-19.pdf
- 29.
EMG-SAGE. Transmission of SARS-CoV-2 and Mitigating Measures. 2020 [Cited 2021 February 10] Available from: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/892043/S0484_Transmission_of_SARS-CoV-2_and_Mitigating_Measures.pdf
- 30.
World Health Organisation. WHO Director-General’s opening remarks at the media briefing on COVID-19–5 June 2020, 5 June 2020. World Health Organisation [Internet] 2020 June 5 [Cited 2021 February 10] Available from: https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19—5-june-2020.
- 31.
Imperial College London, YouGov. Workbook: YouGov & ICL COVID-19 Behaviour Tracker. 2021 [Cited 2021 July 13] Available from: https://ichpanalytics.imperialcollegehealthpartners.com/t/BDAU/views/YouGovICLCOVID-19BehaviourTracker/4Allbehaviorsovertime?:iid=1&:isGuestRedirectFromVizportal=y&:embed=y
- 32. King’s College London, Ipsos MORI. Becoming “Covid-secure”: how the UK is getting used to wearing face masks and other precautionary behaviours. 2020. [Cited 2021 February 9] Available from: https://www.ipsos.com/sites/default/files/ct/news/documents/2020-07/becoming_covid-secure.pdf
- 33. Czypionka T, Greenhalgh T, Bassler D, B. Bryant M. Masks and Face Coverings for the Lay Public. Ann Intern Med 2020. pmid:33370173
- 34. Kwok KO, Li KK, Chan HHH, Yi YY, Tang A, Wei WI, et al. Community Responses during Early Phase of COVID-19 Epidemic, Hong Kong. Emerg Infect Dis CDC 2020;26. pmid:32298227
- 35. Wen F, Meng Y, Cao H, Xia J, Li H, Qi H, et al. Knowledge, attitudes, practices of primary and middle school students at the outbreak of COVID-19 in Beijing: A cross-sectional online study. MedRxiv 2020:2020.06.29.20138628. [Preprint] 2020 [Cited 2021 January 30] Available at: https://doi.org/10.1101/2020.06.29.20138628.
- 36. Qian M, Wu Q, Wu P, Hou Z, Liang Y, Cowling BJ, et al. Anxiety levels, precautionary behaviours and public perceptions during the early phase of the COVID-19 outbreak in China: a population-based cross-sectional survey. BMJ Open 2020;10:e040910. pmid:33033099
- 37. Zhong B-L, Luo W, Li H-M, Zhang Q-Q, Liu X-G, Li W-T, et al. Knowledge, attitudes, and practices towards COVID-19 among Chinese residents during the rapid rise period of the COVID-19 outbreak: a quick online cross-sectional survey. Int J Biol Sci 2020;16:1745–52. pmid:32226294
- 38. Perreault MF, Perreault GP. Journalists on COVID-19 Journalism: Communication Ecology of Pandemic Reporting. Am Behav Sci 2021;65:976–91.
- 39. Lu X, Jin Y. Information vetting as a key component in social-mediated crisis communication: An exploratory study to examine the initial conceptualization. Public Relat Rev 2020;46:101891.
- 40. Chew C, Eysenbach G. Pandemics in the Age of Twitter: Content Analysis of Tweets during the 2009 H1N1 Outbreak. PLOS ONE 2010;5:e14118. pmid:21124761
- 41.
Addawood A. Understanding Misinformation on Twitter in the Context of Controversial Issues. PhD Thesis, University of Illinois, 2019. Available from: https://www.ideals.illinois.edu/handle/2142/104800
- 42. Ruan T, Kong Q, Zhang Y, McBride S, Lv Q. An Analysis of Twitter Responses to the 2019 Ridgecrest Earthquake Sequence, 2020.
- 43. Han C, Yang M, Piterou A. Do news media and citizens have the same agenda on COVID-19? an empirical comparison of twitter posts. Technol Forecast Soc Change 2021;169:120849.
- 44. Eysenbach G. Infodemiology and infoveillance: framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the Internet. J Med Internet Res 2009;11:e11. pmid:19329408
- 45. Abd-Alrazaq A, Alhuwail D, Househ M, Hamdi M, Shah Z. Top Concerns of Tweeters During the COVID-19 Pandemic: Infoveillance Study. J Med Internet Res 2020;22:e19016. pmid:32287039
- 46. Singh L, Bansal S, Bode L, Budak C, Chi G, Kawintiranon K, et al. A first look at COVID-19 information and misinformation sharing on Twitter. ArXiv [Preprint] 2020 [Cited 2021 January 30]. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7280879/.
- 47. Yang K-C, Torres-Lugo C, Menczer F. Prevalence of Low-Credibility Information on Twitter During the COVID-19 Outbreak. ArXiv200414484 Cs [Preprint] 2020. [Cited 2021 February 5]. Available from: https://doi.org/10.36190/2020.16.
- 48. Rufai SR, Bunce C. World leaders’ usage of Twitter in response to the COVID-19 pandemic: a content analysis. J Public Health 2020;42:510–6. pmid:32309854
- 49. Chen E, Lerman K, Ferrara E. Tracking Social Media Discourse About the COVID-19 Pandemic: Development of a Public Coronavirus Twitter Data Set. JMIR Public Health Surveill 2020;6:e19273. pmid:32427106
- 50. Gruzd A, Mai P. Going viral: How a single tweet spawned a COVID-19 conspiracy theory on Twitter. Big Data Soc 2020;7.
- 51. Kouzy R, Abi Jaoude J, Kraitem A, El Alam MB, Karam B, Adib E, et al. Coronavirus Goes Viral: Quantifying the COVID-19 Misinformation Epidemic on Twitter. Cureus 2020.;12. pmid:32292669
- 52. McCombs M, Weaver D. Voters’ Need for Orientation and Use of Mass Communication 1973.
- 53. Lopez CE, Vasu M, Gallemore C. Understanding the perception of COVID-19 policies by mining a multilanguage Twitter dataset. ArXiv200310359 Cs [Preprint] 2020 [Cited 2021 February 5]. Available from: http://arxiv.org/abs/2003.10359
- 54. Gozzi N, Tizzani M, Starnini M, Ciulla F, Paolotti D, Panisson A, et al. Collective Response to Media Coverage of the COVID-19 Pandemic on Reddit and Wikipedia: Mixed-Methods Analysis. J Med Internet Res 2020;22:e21597. pmid:32960775
- 55. Bento AI, Nguyen T, Wing C, Lozano-Rojas F, Ahn Y-Y, Simon K. Evidence from internet search data shows information-seeking responses to news of local COVID-19 cases. Proc Natl Acad Sci 2020;117:11220–2. pmid:32366658
- 56. Hu D, Lou X, Xu Z, Meng N, Xie Q, Zhang M, et al. More effective strategies are required to strengthen public awareness of COVID-19: Evidence from Google Trends. J Glob Health 2020;10:011003. pmid:32373339
- 57. Wolf MS, Serper M, Opsasnick L, O’Conor RM, Curtis LM, Benavente JY, et al. Awareness, Attitudes, and Actions Related to COVID-19 Among Adults With Chronic Conditions at the Onset of the U.S. Outbreak. Ann Intern Med 2020. pmid:32271861
- 58. Ferreira GB, Borges S. Media and Misinformation in Times of COVID-19: How People Informed Themselves in the Days Following the Portuguese Declaration of the State of Emergency. Journal Media 2020;1:108–21.
- 59. Jang K, Baek YM. When Information from Public Health Officials is Untrustworthy: The Use of Online News, Interpersonal Networks, and Social Media during the MERS Outbreak in South Korea. Health Commun 2019;34:991–8. pmid:29558170
- 60. Alam F, Dalvi F, Shaar S, Durrani N, Mubarak H, Nikolov A, et al. Fighting the COVID-19 Infodemic in Social Media: A Holistic Perspective and a Call to Arms. ArXiv200707996 Cs [Preprint] 2020 [Cited 2021 January 1]. Available from: http://arxiv.org/abs/2007.07996
- 61. Shahi GK, Dirkson A, Majchrzak TA. An Exploratory Study of COVID-19 Misinformation on Twitter. ArXiv200505710 Cs [Prepint] 2020 [Cited 2021 February 5]. Available from: http://arxiv.org/abs/2005.05710.
- 62. Austin L, Liu BF, Jin Y. How Audiences Seek Out Crisis Information: Exploring the Social-Mediated Crisis Communication Model. J Appl Commun Res 2012.
- 63. Weitkamp E, Milani E, Ridgway A, Wilkinson C. Exploring the digital media ecology: insights from a study of healthy diets and climate change communication on digital and social media. J Sci Commun 2021;20:A02.
- 64. Liu BF, Fraustino JD, Jin Y. Social Media Use During Disasters: How Information Form and Source Influence Intended Behavioral Responses. Commun Res 2016;43:626–46.
- 65. Veltri GA, Atanasova D. Climate change on Twitter: Content, media ecology and information sharing behaviour. Public Underst Sci 2017;26:721–37. pmid:26612869
- 66. Banda JM, Tekumalla R, Wang G, Yu J, Liu T, Ding Y, et al. A large-scale COVID-19 Twitter chatter dataset for open scientific research—an international collaboration. ArXiv200403688 Cs [Preprint] 2020. [Cited 2021 February 3] Available from: http://arxiv.org/abs/2004.03688 pmid:32550247
- 67. Boyd D, Golder S, Lotan G. Tweet, Tweet, Retweet: Conversational Aspects of Retweeting on Twitter. 2010 43rd Hawaii Int. Conf. Syst. Sci., 2010, p. 1–10.
- 68. Abdullah NA, Nishioka D, Murayama Y. Questionnaire Testing: Identifying Twitter User’s Information Sharing Behavior during Disasters. J Inf Process 2016;24:20–8.
- 69. Majmundar A, Allem J-P, Cruz TB, Unger JB. The Why We Retweet scale. PLOS ONE 2018;13:e0206076. pmid:30335827
- 70. Blei DM, Ng AY, Jordan MI. Latent dirichlet allocation. J Mach Learn Res 2003;3:993–1022.
- 71. Řehůřek R, Sojka P. Software Framework for Topic Modelling with Large Corpora, 2010, p. 45–50.
- 72.
Google. Google Trends [Internet] 2022 [Cited 2022 February 15]. Available from: https://trends.google.com/trends/?geo=GB
- 73.
Twitter. Developer Agreement and Policy–Twitter Developers [Internet] 2022 [Cited 2022 February 15]. Available from: https://developer.twitter.com/en/developer-terms/agreement-and-policy
- 74. Allegretti A. Mixed messages: the changing government narrative on face masks. The Guardian [Internet] 2021 July 12 [Cited 2021 September 2021]. Available from: https://www.theguardian.com/world/2021/jul/12/mixed-messages-the-changing-government-narrative-on-face-masks.
- 75. News Sky. Coronavirus: New Banksy piece on London Tube encourages people to wear masks. Sky News [Internet] 2020 July 15 [Cited 2021 September 2021]. Available from: https://news.sky.com/story/coronavirus-new-banksy-piece-on-london-tube-encourages-people-to-wear-masks-12028499
- 76. Chu DK, Akl EA, Duda S, Solo K, Yaacoub S, Schünemann HJ, et al. Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS-CoV-2 and COVID-19: a systematic review and meta-analysis. The Lancet 2020;395:1973–87. pmid:32497510
- 77. Lithwick D. America’s Heroism Trap. Yes, we are grateful for every person working on every front line. But the language of martyrdom distracts from what we could do about it all. Slate Mag [Internet] 2020 April 20 [Cited 2021 September 21]. Available from: https://slate.com/news-and-politics/2020/04/coronavirus-humans-vs-heroes.html
- 78. Prather K, Wang C, Schooley R. Reducing transmission of SARS-CoV-2. Science 2021;368:1422–4. pmid:32461212
- 79.
BBC News “The masks you throw away could kill a whale.” BBC News [Internet]. 2020 July 8 [Cited 2021 July 7]. Available from: https://www.bbc.co.uk/news/av/science-environment-53287940
- 80.
BBC News. “We do not recommend face masks for general wearing.” BBC News [Internet]. 2020 April 3 [Cited 2021 July 21]. Available from: https://www.bbc.co.uk/news/av/uk-52153145
- 81. Baynes C. Coronavirus: Face masks could increase risk of infection, medical chief warns, The Independent [Internet] 2020 March 12 [Cited 2021 July 7]. Available from: https://www.independent.co.uk/news/health/coronavirus-news-face-masks-increase-risk-infection-doctor-jenny-harries-a9396811.html
- 82. Apuzzo M, Gebrekidan S, Kirkpatrick D. How the World Missed Covid-19’s Silent Spread, The New York Times [Internet] 2020 June 28 [Cited 2021 July 7]. Available from: https://www.nytimes.com/2020/06/27/world/europe/coronavirus-spread-asymptomatic.html.
- 83.
BBC News. Coronavirus: Wear masks in crowded public spaces, says science body. BBC News [Internet] 2020 July 7 [Cited 2021 July 7]. Available from: https://www.bbc.co.uk/news/uk-53316491
- 84. Culbertson A. Coronavirus: Wearing surgical masks can reduce COVID-19 spread by 75%, study claims. Sky News [Internet] 2020 May 19 [Cited 2021 July 7]. Available from: https://news.sky.com/story/coronavirus-wearing-surgical-masks-can-reduce-covid-19-spread-by-75-study-claims-11990381
- 85. University of Oxford. Oxford COVID-19 study: face masks and coverings work–act now. [Internet] 2020 July 8 [Cited 2021 July 2021]. Available from: https://www.ox.ac.uk/news/2020-07-08-oxford-covid-19-study-face-masks-and-coverings-work-act-now
- 86. Howard J. To help stop coronavirus, everyone should be wearing face masks. The science is clear. The Guardian [Internet] 2020 April 4 [Cited 2021 July 7]. Available from: https://www.theguardian.com/commentisfree/2020/apr/04/why-wear-a-mask-may-be-our-best-weapon-to-stop-coronavirus
- 87.
Statista (using data from PAMCo and Comscore). Monthly reach of leading national newspapers in the United Kingdom from April 2019 to March 2020, by platform 2021. https://www.statista.com/statistics/246082/distribution-of-the-reach-of-selected-national-newspapers-in-the-uk-by-type/#statisticContainer (accessed February 14, 2022).
- 88. Majchrzak A, Jarvenpaa SL, Hollingshead AB. Coordinating Expertise Among Emergent Groups Responding to Disasters. Organ Sci 2007;18:147–61.
- 89. Stieglitz S, Bunker D, Mirbabaie M, Ehnis C. Sense-making in social media during extreme events. J Contingencies Crisis Manag 2018;26:4–15.
- 90. van der Meer TGLA, Jin Y. Seeking Formula for Misinformation Treatment in Public Health Crises: The Effects of Corrective Information Type and Source. Health Commun 2020;35:560–75. pmid:30761917
- 91.
Prime Minister’s Office, 10 Downing Street, The Rt Hon Boris Johnson MP. Prime Minister’s statement on coronavirus (COVID-19): 11 May 2020, GOVUK [Internet] 2020 May 2021 [Cited 2021 July 7] Available from: https://www.gov.uk/government/speeches/pm-statement-on-coronavirus-11-may-2020.
- 92. Tanaka Y, Sakamoto Y, Honda H. The Impact of Posting URLs in Disaster-Related Tweets on Rumor Spreading Behavior. 2014 47th Hawaii Int. Conf. Syst. Sci., 2014, p. 520–9.
- 93. Anju KJ, Arulsamy S. The Knowledge Towards Coronavirus among the people of Kerela and Tamilnadu. J Compos Theory 2020;XIII:241–8. ISSN: 0731-6755
- 94. Roozenbeek J, Schneider CR, Dryhurst S, Kerr J, Freeman ALJ, Recchia G, et al. Susceptibility to misinformation about COVID-19 around the world. R Soc Open Sci n.d.;7:201199. pmid:33204475
- 95. Rosenberg H, Syed S, Rezaie S. The Twitter pandemic: The critical role of Twitter in the dissemination of medical information and misinformation during the COVID-19 pandemic. Can J Emerg Med 2020;22:418–21. pmid:32248871
- 96. McCombs ME, Shaw DL, Weaver DH. New Directions in Agenda-Setting Theory and Research. Mass Commun Soc 2014;17:781–802.
- 97. Kim I, Valente T. COVID-19 Health Communication Networks on Twitter: Identifying Sources, Disseminators, and Brokers. Connections 2021;40.
- 98.
Sasse T, Haddon C, Nice A. Science advice in a crisis. UK: Institute for Government; 2020.
- 99.
Government Office for Science, Scientific Advisory Group for Emergencies. Government publishes SAGE minutes [Internet] 2020 May 29 [Cited 2021 September 21]. Available from: https://www.gov.uk/government/news/government-publishes-sage-minutes
- 100.
Gieryn TF. Boundaries of Science. Handb. Sci. Technol. Stud., 2455 Teller Road, Thousand Oaks California 91320 United States of America: SAGE Publications, Inc.; 1995, p. 392–443. https://doi.org/10.4135/9781412990127.n18
- 101. Jasanoff SS. Contested Boundaries in Policy-Relevant Science. Soc Stud Sci 1987;17:195–230.
- 102. Liang H, Fung IC-H, Tse ZTH, Yin J, Chan C-H, Pechta LE, et al. How did Ebola information spread on twitter: broadcasting or viral spreading? BMC Public Health 2019;19:438. pmid:31023299
- 103. Russell Neuman W, Guggenheim L, Mo Jang S, Bae SY. The Dynamics of Public Attention: Agenda-Setting Theory Meets Big Data. J Commun 2014;64:193–214.