PONE-D-22-05617Assessing the influence of French vaccine critics during the two first years of the COVID-19 pandemicPLOS ONE

Dear Dr. Faccin,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please note that all three reviews recognize the merit and significance of your work -- but they also identify a number of minor weaknesses. In my opinion, these weaknesses can be sufficiently addressed as long as you thoroughly follow the suggestions, or respond to the concerns, of the reviewers.

Please submit your revised manuscript by Jun 27 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Constantine Dovrolis

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf  and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. In your Methods section, please include additional information about your dataset and ensure that you have included a statement specifying whether the collection and analysis method complied with the terms and conditions for the source of the data.

3. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability.

Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized.

Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access.

We will update your Data Availability statement to reflect the information you provide in your cover letter.

4. Thank you for stating the following in the Acknowledgments Section of your manuscript:

“This research has benefited from the financial support of the Agence Nationale de la Recherche (projects TRACTRUST - ANR-20-COVI-0102 and SLAVACO - ANR 20-COV8-0009-01) and the ANRS-MIE (project MEDIACAM - ANRSCOV24)”

We note that you have provided additional information within the Acknowledgements Section that is not currently declared in your Funding Statement. Please note that funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

“The author(s) received no specific funding for this work.”

Please include your amended statements within your cover letter; we will change the online submission form on your behalf.

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

Reviewer #3: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: N/A

Reviewer #3: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Summary:

In this paper, the authors analyze the debates over the COVID-19 vaccine on the French-speaking segment of Twitter. First, they study the evolution of vaccine-related debates. They show that specific events increase the reach of the vaccine-critical activists on social media, but this information flow is relatively limited compared to mainstream media. Second, they analyze the community structure of discussions and examine how information flow between communities. Authors conclude that the largest far-right community is the echo chamber of conspiracy theorists. In contrast, a smaller community that consists of far-left actors is more capable of communicating vaccine-critical content to a broader public.

They assess the evolution of user engagement using a model similar to the SIS network epidemic model. Here, users that share vaccine-critical information are analogous to infectious individuals in the SIS model. Furthermore, their community analysis relies on hypergraphs derived from retweet data. Hypergraph helps differentiate a user who is retweeted N times for a single tweet and a user whose N tweets are individually retweeted one single time, which is crucial for capturing the dynamics of the information flow.

In conclusion, I think this paper presents a rigorous analysis of vaccine hesitancy on Twitter during the COVID-19 pandemic. I think the methodology is accurate, and the results are significant.

Minor Issues:

Some points regarding data collection are unclear to me:

(1) To the best of my understanding, "vaccine-critical URLs" refer to websites other than mainstream media. For example, websites of prominent actors in vaccine controversies. It may be helpful for the reader if this is reminded in the data subsection.

(2) "we searched our database for those URLs" is the database refer to dataset DataCovVac?

(3) I could not fully follow how the co-occurrence network is used to label URLs automatically? Did you also propagate the labels of media URLs to closest neighbors? It appears it is performed only for 285 vaccine-critical URLs. If this is the case, it is not clear how 382 media URLs were obtained from the initial 50 URLs.

I think that a figure (e.g., a flowchart) that explains the data collection process might be helpful. However, this is not a necessity.

I think the authors could mention the motivation behind using hypergraph instead of a standard retweet network earlier in the method section. I think it is a crucial choice, and the reason is explained in the middle of the results section.

Sometimes the COVID-19 outbreak is mentioned as an epidemic, while it is sometimes referred to as a pandemic. Is there a nuance based on the context, or are the words "epidemic" and "pandemic" used interchangeably? For example, the title says "COVID-19 pandemic" and the abstract says "COVID-19 epidemic" this might confuse the reader.

Reviewer #2: In this study, the authors investigated the influences and spreading of vaccine-critical content on social media. They focused on Twitter data and applied network analysis tools to answer two questions: (1) Did vaccine-critical contents exhibit a "rise" during the COVID breakout? (2) What are the roles that different communities (groups of

closely-connected Twitter users) play in the flow of vaccine-critical information.

Generally, the draft is clear about the questions and the general approaches through which these questions can be answered. Nevertheless, it could be improved in its technical soundness and presentation details.

Major comments:

1. In the abstract, one of the questions is formulated as "Who were the central actors in the diffusion of these (vaccine-critical) contents?". It doesn't seem that this question can be fully answered by the corresponding conclusion "the largest right-wing community has typical echo-chamber behavior... The smaller left-wing community is less permeable ..., but has a large capacity to spread it once adopted." For example, are these two communities the central actors? What about the rest of the communities identified? Are they less central? Why? To ensure that the question matches with conclusions, I suggest that this research question be re-formulated.

2. In "Results: The mesoscale structure of the information flow", it is not clear how the two metrics -- the escape probability and the average visit probability, are calculated. This problem is partially due to limited details in the description of behaviors of the random-walker in the Method section. Different definitions could lead to very different interpretations of the results.

a. Especially, the definition of average visit probability -- "the probability (for a random walker) to visit a node of a given community" could have various explanations. For example, do we assume here the random-walker has an equal probability to start from any nodes in the network and take only one step? Or, we let the random-walker randomly walk for a large number of steps (so that the position of the random walker follows a stationary distribution) and aggregate the result from this simulation?

b. The escape probability also suffers (but potentially less) from the same issue. We probably know that the random walker is located in a community. However, do we assume the random walker has an equal chance to start from any nodes in the community? Or node with a higher out-degree in a community has a higher chance to be a starting point?

Minor comments:

1. In "Methods: Measuring the engagement dynamics", Nt is defined as the total number of active users. What's the definition of an active user?

2. In the same section, are there supportive arguments for the specific selection of the engagement window to be 3 days? Would the result significantly change if we slightly vary this parameter?

3. In Fig. 3, the y-axis label on the left doesn't make much sense. I suppose that both of blue and orange curves correspond to the number of users engaging with vaccine-critical content and news media, correspondingly. Given that the orange curve has the label "Media", the blue curve should have the label "Vaccine-critical" instead of "Engaged".

4. In Fig. 6 & 7, what does the size of each circle refers to? size of each community? Please put this information in the caption of the figure to ensure clarity.

Reviewer #3: The manuscript is partly technically sound. The selected data collection mechanism and analysis methods are suitable for addressing the research questions. Some parts in the Data and methods and Results sections need more detailed explanation:

- The APIs used in data collection are not mentioned

- There are not details on what the dataset contains (e.g. tweet ids, tweet text, number of retweets, user ids, etc.)

- The words used for the queries, how you decided to use those and which are the words?

- The qualitative analysis of the users profiles to distinguish to left or right-partite.

- There is no information within the manuscript about wether the dataset, scripts to collect the data, scripts for analysis are available for the replication of the results or future works.

The findings of this work are well supported by the data both from within the text and the figures.

The paper structure, writing style, and language is appropriate for a research manuscript.

Comments to the authors:

1. explain the selection of 3 days time window for τ, in Measuring the engagement dynamics section.

2. add the references for "compartmental models" and SIS model in epidemiology

3. define how they can say that an engaged user looses interest in β calculation

4. explain more the community detection method in the section "Community detection".

5. There is a typo in Results section, first sentence, there is the word "weather" instead of "wether"

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Assessing the influence of French vaccine critics during the two first years of the COVID-19 pandemic

PONE-D-22-05617R1

Dear Dr. Faccin,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Constantine Dovrolis

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments: