PONE-D-21-23221Combining big and small data: An integrated data framework for policy guidance in times of dynamic economic shocksPLOS ONE

Dear Dr. Dörr,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Several aspects of the paper should be clarified, as also mentioned by the reviewers. The performance of the deep learning classifier should be evaluated using the recognized metrics (Precision, Recall, etc.). The usefulness of the approach should be better demonstrated, as also highlighted by one of the reviewers.

Please submit your revised manuscript by Dec 23 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Liviu-Adrian Cotfas

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Partly

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: No

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This work introduces an integrated data framework for guiding policymakers during a crisis in a timely and cost-effective manner.

The core of this work focuses on finding different strategies to analyze various data sources from companies that can provide the bigger picture during the Covid-19 pandemic, whether it's in an early stage near-real-time, in a follow-up, or a retrospective stage (in the aftermath of the shock).

The authors conclude that their framework provides policymakers with guidance for their economic support measures in times of sudden shocks.

Strengths

---------------------------------------------------------------------------

*) This work provides an interesting study of the economic sector struggles during Covid-19.

*) The idea to combine data from different data sources (e.g., online, surveys, official data) to aid in forecasting is very good, although not new.

*) Most of the references are new and relevant.

Weaknesses

-----------------

*) The title is more generic than what the article presents. The authors should deliver a "data framework for policy guidance in times of dynamic economic shocks" but the solution is very specific to the Covid-19 scenario. It will be very hard to generalize (to other types of crisis) as most of the steps are done manually and not automatically, e.g., they somehow chose some specific keywords /terms to search in the text (section 3.1) and some classes for the classifier. Something like automated detection of the keywords would make the solution more general. Even the evaluation of the model (XLM-RoBERTa) was done only manually, instead of also using specific evaluation metrics. They did not discuss the precision, accuracy, or recall of the model.

*) The theoretical and experimental analyses of the paper are not rigorous.

*) By only analyzing the Covid-19 scenario, they do not demonstrate that their solution is efficient and optimal in a general crisis situation. Also, at the start of the lockdown was very clear what sectors are going to suffer the most. I am not sure what in-depth insights this study offers.

To sum up, the primary issue with this paper is that the contribution as a piece of research is not clear.

Observations

--------------

- the figures and some of the tables were not added in the article, they can be found at the end of the document (all the figures) or can be downloaded separately (tables Si_Table).

Reviewer #2: The paper proposes a framework for guiding policy makers in the context of a major economic event by analyzing small and big data.

The topic is interesting and worth investigating, given the recent COVID-19 economic shock. The paper is well structured, includes a comprehensive enough literature review and uses recent transformer-based language models. However, many aspects of the paper are described in rather vague terms.

Additional details should be provided regarding the classifier used for categorizing the announcements on the companies’ websites. The current description can be considered quite vague. For example, the paper should state how many of the 4,347 manually classified text passages belonged to each considered category. Has the dataset been a balanced or an unbalanced one? How have the authors handled the training process if the training dataset has been unbalanced? Additionally, what happened when a disagreement between the two annotators has been encountered? The paper should evaluate the performance of the classifier used for using the standard metrics such as Precision, Recall, Accuracy and F1-Score.

Numerical values should also be provided in the text discussing Figure 3. The authors could also choose to mention the values in a table.

Precise information regarding the survey should also be included. The current version of the paper does not for example mention the exact number of companies that have completed the survey in the text of the paper (only included in Table 1).

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

An integrated data framework for policy guidance during the coronavirus pandemic: Towards real-time decision support for economic policymakers

PONE-D-21-23221R1

Dear Dr. Dörr,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Liviu-Adrian Cotfas

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments: