PONE-D-21-40016Polarization and trust in the evolution of vaccine discourse on Twitter during COVID-19PLOS ONE

Dear Dr. Ojea Quintana,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jul 07 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Tingshao Zhu

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf  and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please ensure that you refer to Figure 6 in your text as, if accepted, production will need this reference to link the reader to the figure.

3. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments (if provided):

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The article has been written meticulously and addresses an important public health issue of developed and developing nations alike using an innovative approach. In my opinion all the findings and interpretations are scientifically sound as presented in the paper.

Reviewer #2: The manuscript is written in very prominent contemporary debate. Scientifically written in logical flows. "We have argued that this realignment shows that the dynamics of online discourse are driven by social connection and trust, rather than by underlying static values." Agreed.

Reviewer #3: The authors describe an analysis of the debate about COVID-19 on Twitter over the first five months of the pandemic. The final aim is to study English-language discourse around vaccines, vaccination-related engagement and discourse, and the causes of the changes in engagement. They describe the data collected, their approach, and the observations derived from the analyses. The authors focus their research on the top five communities they extracted (as they represent most of the English-language data they collected), summarising their statistics, and focusing all of their analyses on these groups. The authors provide detailed descriptions of the outcomes of their study by answering their research questions and properly discussing the limitations of their approach.

The article is well organised, and the outcomes are discussed thoroughly. I advise providing additional context and explanations for the following points.

- In the data collection part, when mentioning “...a series of vaccination-related keywords, hashtags, and short expressions…”, it would be good to have a few examples of such words/expressions (even if they are included in the Appendix) with a small discussion on how and why such words were chosen (maybe you could pick the most relevant and/or complex ones).

- The authors discarded “Users that only retweeted but never authored an original tweet” while keeping “...a very small number of bots”. Both users with retweets only and bots may play a signal-boosting function. Discussing the reasons behind the decision of keeping one category while discarding the other may improve the soundness of the article. Moreover, how different would the network be without bots’ data? Do bots really change the shape of the network or the interactions between the communities?

- The authors considered only the top five communities, mainly because they share content using the English language while representing 80% of the nodes and 90% of the tweets. How and why did the authors choose only the top five? What is the choice criterion? Was it only because of the language used within the identified communities?

- The authors state “Communities beyond the top five also tended to cluster around non-English-language accounts, which limit the utility of our dictionary-based tools”. Did the authors discard all the non-English content the top five communities shared? I advise discussing such a point and carrying out a quick analysis of the percentages of English content VS non-English for the top five (or even 10) groups to cover such an aspect, also providing further support for the discussion about the choice of the top five communities.

Furthermore, the authors mention the Appendix many different times within the text, although no Appendix is provided within the article.

Reviewer #4: 1. The article is very clear and well defined. But still some corrections are presents.

2. Authors need to focus the novelty in abstract section and expected output with actual output.

3. introduction section is too short to intro the proposed work motto.

4. Result section is very organized but images are unclear. Need to upgrade with high resolution images.

5. Try to use some recent papers as related work (2020-2022).

6. There are some grammatical correction, please recheck and correct.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Dr. Aftab Ahmad, MD (Community Medicine)

Reviewer #2: No

Reviewer #3: Yes: Andrea Tocchetti

Reviewer #4: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PONE-D-21-40016R1Polarization and trust in the evolution of vaccine discourse on Twitter during COVID-19PLOS ONE

Dear Dr. Ojea Quintana,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please see reviewer comments below.

Please submit your revised manuscript by Sep 11 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Hanna Landenmark

Staff Editor, PLOS ONE

on behalf of 

Tingshao Zhu

Academic Editor, PLOS ONE

Journal Requirements:

Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

Additional Editor Comments (if provided):

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #3: All comments have been addressed

Reviewer #4: All comments have been addressed

Reviewer #5: All comments have been addressed

Reviewer #6: (No Response)

Reviewer #7: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: No

Reviewer #6: Yes

Reviewer #7: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: No

Reviewer #6: Yes

Reviewer #7: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: No

Reviewer #6: Yes

Reviewer #7: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #3: Yes

Reviewer #4: Yes

Reviewer #5: No

Reviewer #6: Yes

Reviewer #7: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #3: The authors presented a study analysing the impact of COVID-19 on the vaccine discourse over the first 5 months of pandemic on Twitter. They clearly stated the research questions they address and provide detailed analyses covering such questions. The authors properly describe the methods, data, and procedures employed in the process.

Reviewer #4: Authors are successfully addressed all of the comments from reviewer. No further corrections are identified.

Reviewer #5: The topic presented by the authors is very interesting and timely. However, there are several major issues that need to be addressed.

1. The abstract is very vage and not very clear. Please revise in a way the reader knows exactly what to expect from the paper.

2. I like highlighting RQ1 - RQ3 on page 2. However, I suggest to be more specific.

RQ1. Which groups are most important in the English-language discourse around 40 vaccines on Twitter? I do not think that is what the authors do. Instead, the focus is on specific countries using the English language. Which countries are these? Add to RQ1!

RQ2. How did vaccination-related engagement and discourse change over the first 42 five months of the pandemic? Add years, ie. 02/2000 - 07/2000 or similar.

RQ3. What social forces might help explain observed changes in engagement? should be

RQ3. What social forces might help in explaining observed changes in engagement?

In general, the language needs to be improved of the paper. In its current form it is very poor language at many places and I cannot list all of them.

As explained in the sections coming (just another example of poor language).

3.To identify different communities, we used Gephi’s implementation of the Louvain 110 modularity algorithm developed by [22] This is unclear. Please discuss community/module detection algorithms in details, e.g.,

https://link.springer.com/article/10.1186/s12859-016-0979-8

4. Discussion: From the beginning it is unclear to me how to define 'Antivaxxers'? I searched the paper but could not find a proper definition (it should pop out immediately).

Question: Is the population of a country with a low vaccination rate (e.g. some African countries) in the category Antivaxxers? I is unclear to me if this is a negative term or neutral and if one has a choice to be in this group or is this possible determined by a third party.

In this context, it would be good to add some English-speaking African countries with a low vaccination rate and low death rate which I think exist. If the focus is only on the US a discussion of this would be sufficient.

Considering this the distinction Health-Antivax seem incorrect. I would suggest a revision of wording to make the presentation more factual.

4. From line 407 to end: I find the connection between the numerical results and the provided discussion unsatisfactory. The main problem is a lack of connection. The authors do not use the numerical results to provide a discussion but the discussion seems not well grounded at all. At least I could not see such a connection.

It is very important that the discussion focuses exclusively on the interpretation of the numerical results. At the end of the discussion, a wider perspective, which could be even speculative, could be provided.

I suggest to rewrite the entire discussion section.

5. Our corpus is also mostly in the English language, 481 which we estimate represents 68% of Twitter chatter worldwide pertaining to COVID. How can this information be relevant when Democrat-Republican are limited to the US only?

Similar to the point about Health-Antivax (see 4) also

Democrat-Republican is not well defined. Definitions of all 4 terms should be added to the methods section.

6. The conclusion section is similar unclear and lacks a connection to the numerical results.

Reviewer #6: This paper presents an original and as of yet unpublished study. The authors have clearly and effectively described their analyses and drawn appropriate (and quite interesting) conclusions from the analyses presented. The article is also well written and understandable, with one exception which stood out to me while reading the manuscript.

The dataset obtained from the Twitter Streaming API appears to be global, with no mention of any geographic restrictions from users, and several non-US users are listed, with the particular grouping of the "unorthodox" being non-American in nature. However, two of the communities are defined in terms of American political divisions, the public health measure is US-centered, and geolocation data are mentioned in the discussion section when addressing future work. I think it may be helpful to clarify briefly but explicitly either in the data collection subsection or in the community characterization subsection that, even though the labels "Republican" and "Democrats" are used for two communities, the data extend globally. As a secondary, optional, matter I would be interested to know why American political cleavages may define a global discussion space.

Reviewer #7: This paper reports an interesting analysis of the discourse on vaccines around the onset of the COVID-19 pandemic. In particular, it shows how the discourse changed after the WHO officially declared the pandemic in March 2021. Overall, I think that it is well done and makes a significant contribution.

Although I review the paper for the first time, I see that it is a resubmission. Therefore, I limit my critical comments to points that can be reasonably addressed at this stage of the review process.

First, I encourage the authors to clarify the geographic scope of the analysis. The dataset includes only tweets written in English, but is not restricted to particular countries. At the same time, the analysis relies on the categories of US politics, particularly the distinction between Republicans and Democrats. This tension should be clarified as early as possible in the paper.

Second, the link between vaccines and COVID was a bit confusing to me, given the time frame of the analysis (December 2019-June 2020). In that period, the emphasis was not on COVID vaccines, but more on the nature of the disease and mitigation strategies such as masks and containment. The authors do frame the study as an analysis of changes of the vaccination discourse following the emergence of COVID, but they could be more explicit that the discourse is on vaccines _in general_. Currently, readers may have the wrong expectation that the paper is about COVID vaccines.

Third, the pre-post comparisons are the most interesting parts of the analysis, but they could be discussed more in depth. For example, we see in Figure 1 that the structure of the retweet network changes significantly, but the point is barely elaborated in the text.

Finally, Figures 5 and 7 are not legible, partly because of the low resolution but also due to the lack of a legend and the small size of the points. These figures should be improved.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #3: Yes: Andrea Tocchetti

Reviewer #4: Yes: F.M. Javed Mehedi Shamrat

Reviewer #5: No

Reviewer #6: No

Reviewer #7: No

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

Polarization and trust in the evolution of vaccine discourse on Twitter during COVID-19

PONE-D-21-40016R2

Dear Dr. Ojea Quintana,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Hossein Kermani

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Congratulations!

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #4: All comments have been addressed

Reviewer #7: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #4: Yes

Reviewer #7: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #4: Yes

Reviewer #7: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #4: Yes

Reviewer #7: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #4: Yes

Reviewer #7: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #4: All comments have been addressed by the authors.

Still some grammatical errors exits, check and resolve the mistakes.

Reviewer #7: The authors have addressed my comments satisfactorily. I have no other comments.

Some additional words to meet word count.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #4: No

Reviewer #7: No

**********