PCOMPBIOL-D-26-00338

Evaluation of short-term multi-target respiratory forecasts over winter 2024-25 in England using sub-ensemble contribution analyses

PLOS Computational Biology

Dear Dr. Kennedy,

Thank you for submitting your manuscript to PLOS Computational Biology. After careful consideration, we feel that it has merit but does not fully meet PLOS Computational Biology's publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jun 05 2026 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at ploscompbiol@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pcompbiol/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

* A letter that responds to each point raised by the editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. This file does not need to include responses to formatting updates and technical items listed in the 'Journal Requirements' section below.

* A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

* An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, competing interests statement, or data availability statement, please make these updates within the submission form at the time of resubmission. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter

We look forward to receiving your revised manuscript.

Kind regards,

Kimberly M. Pepin

Academic Editor

PLOS Computational Biology

Jennifer Flegg

Section Editor

PLOS Computational Biology

Additional Editor Comments:

Both reviewers found the manuscript interesting and provided helpful guidance for improving it. However, for further consideration of publication, substantially more details are needed for understanding methods and results. There are also some methodological issues raised by reviewers that will likely require re-analysis and additional visuals.

Currently the manuscript is framed as methods development but fails to describe what application of this method has revealed in terms of the biology of the system. Adding some text describing the novel biological insights will be important in revision. i.e., What is learned from knowing which models were useful at different epidemic stages? What additional information is gained from using this method over previous forecasting models for this system? The authors mention the approach allows for faster decision-making. What are the impacts?

Journal Requirements:

If the reviewer comments include a recommendation to cite specific previously published works, please review and evaluate these publications to determine whether they are relevant and should be cited. There is no requirement to cite these works unless the editor has indicated otherwise.

1) Please insert an Ethics Statement at the beginning of your Methods section, under a subheading 'Ethics Statement'. It must include:

i) The full name(s) of the Institutional Review Board(s) or Ethics Committee(s)

ii) The approval number(s), or a statement that approval was granted by the named board(s)

iii) A statement that formal consent was obtained (must state whether verbal/written) OR the reason consent was not obtained (e.g. anonymity). NOTE: If child participants, the statement must declare that formal consent was obtained from the parent/guardian.].

2) Please upload all main figures as separate Figure files in .tif or .eps format. For more information about how to convert and format your figure files please see our guidelines:

https://journals.plos.org/ploscompbiol/s/figures

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Authors:

Please note here if the review is uploaded as an attachment.

Reviewer #1: Attached.

Reviewer #2: See attached

**********

Have the authors made all data and (if applicable) computational code underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data and code underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data and code should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data or code —e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No: Code is provided; data is not.

Reviewer #2: Yes

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review?  For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

Figure resubmission:

While revising your submission, we strongly recommend that you use PLOS’s NAAS tool (https://ngplosjournals.pagemajik.ai/artanalysis) to test your figure files. NAAS can convert your figure files to the TIFF file type and meet basic requirements (such as print size, resolution), or provide you with a report on issues that do not meet our requirements and that NAAS cannot fix.

After uploading your figures to PLOS’s NAAS tool - https://ngplosjournals.pagemajik.ai/artanalysis, NAAS will process the files provided and display the results in the "Uploaded Files" section of the page as the processing is complete. If the uploaded figures meet our requirements (or NAAS is able to fix the files to meet our requirements), the figure will be marked as "fixed" above. If NAAS is unable to fix the files, a red "failed" label will appear above. When NAAS has confirmed that the figure files meet our requirements, please download the file via the download option, and include these NAAS processed figure files when submitting your revised manuscript.

Reproducibility:

To enhance the reproducibility of your results, we recommend that authors of applicable studies deposit laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

PCOMPBIOL-D-26-00338

Evaluation of short-term multi-target respiratory forecasts over winter 2024-25 in England using sub-ensemble contribution analyses

PLOS Computational Biology

Dear Dr. Kennedy,

Thank you for submitting your manuscript to PLOS Computational Biology. After careful consideration, we feel that it has merit but does not fully meet PLOS Computational Biology's publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jun 05 2026 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at ploscompbiol@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pcompbiol/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

* A letter that responds to each point raised by the editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. This file does not need to include responses to formatting updates and technical items listed in the 'Journal Requirements' section below.

* A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

* An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, competing interests statement, or data availability statement, please make these updates within the submission form at the time of resubmission. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter

We look forward to receiving your revised manuscript.

Kind regards,

Kimberly M. Pepin

Academic Editor

PLOS Computational Biology

Jennifer Flegg

Section Editor

PLOS Computational Biology

Additional Editor Comments:

Both reviewers found the manuscript interesting and provided helpful guidance for improving it. However, for further consideration of publication, substantially more details are needed for understanding methods and results. There are also some methodological issues raised by reviewers that will likely require re-analysis and additional visuals.

Currently the manuscript is framed as methods development but fails to describe what application of this method has revealed in terms of the biology of the system. Adding some text describing the novel biological insights will be important in revision. i.e., What is learned from knowing which models were useful at different epidemic stages? What additional information is gained from using this method over previous forecasting models for this system? The authors mention the approach allows for faster decision-making. What are the impacts?

Journal Requirements:

If the reviewer comments include a recommendation to cite specific previously published works, please review and evaluate these publications to determine whether they are relevant and should be cited. There is no requirement to cite these works unless the editor has indicated otherwise.

1) Please insert an Ethics Statement at the beginning of your Methods section, under a subheading 'Ethics Statement'. It must include:

i) The full name(s) of the Institutional Review Board(s) or Ethics Committee(s)

ii) The approval number(s), or a statement that approval was granted by the named board(s)

iii) A statement that formal consent was obtained (must state whether verbal/written) OR the reason consent was not obtained (e.g. anonymity). NOTE: If child participants, the statement must declare that formal consent was obtained from the parent/guardian.].

2) Please upload all main figures as separate Figure files in .tif or .eps format. For more information about how to convert and format your figure files please see our guidelines:

https://journals.plos.org/ploscompbiol/s/figures

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Authors:

Please note here if the review is uploaded as an attachment.

Reviewer #1: Attached.

Reviewer #2: See attached

**********

Have the authors made all data and (if applicable) computational code underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data and code underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data and code should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data or code —e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No: Code is provided; data is not.

Reviewer #2: Yes

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review?  For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

Figure resubmission:

While revising your submission, we strongly recommend that you use PLOS’s NAAS tool (https://ngplosjournals.pagemajik.ai/artanalysis) to test your figure files. NAAS can convert your figure files to the TIFF file type and meet basic requirements (such as print size, resolution), or provide you with a report on issues that do not meet our requirements and that NAAS cannot fix.

After uploading your figures to PLOS’s NAAS tool - https://ngplosjournals.pagemajik.ai/artanalysis, NAAS will process the files provided and display the results in the "Uploaded Files" section of the page as the processing is complete. If the uploaded figures meet our requirements (or NAAS is able to fix the files to meet our requirements), the figure will be marked as "fixed" above. If NAAS is unable to fix the files, a red "failed" label will appear above. When NAAS has confirmed that the figure files meet our requirements, please download the file via the download option, and include these NAAS processed figure files when submitting your revised manuscript.

Reproducibility:

To enhance the reproducibility of your results, we recommend that authors of applicable studies deposit laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Attachments

Attachment

Submitted filename: authorComments.docx

Attachment

Submitted filename: review.pdf

PCOMPBIOL-D-26-00338R1

Evaluation of short-term multi-target respiratory forecasts over winter 2024-25 in England using sub-ensemble contribution analyses

PLOS Computational Biology

Dear Dr. Kennedy,

Thank you for submitting your manuscript to PLOS Computational Biology. Both reviewers appreciate the work and agree that the manuscript has been improved substantially. However, they had some outstanding suggestions that should be addressed before the manuscript can be accepted. Therefore, we invite you to submit a revised version of the manuscript that addresses the new points raised - all which are fairly minor.

Please submit your revised manuscript by Sep 19 2026 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at ploscompbiol@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pcompbiol/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

* A letter that responds to each point raised by the editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. This file does not need to include responses to formatting updates and technical items listed in the 'Journal Requirements' section below.

* A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

* An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, competing interests statement, or data availability statement, please make these updates within the submission form at the time of resubmission. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

As the corresponding author, your ORCID iD is verified in the submission system and will appear in the published article. PLOS supports the use of ORCID, and we encourage all coauthors to register for an ORCID iD and use it as well. Please encourage your coauthors to verify their ORCID iD within the submission system before final acceptance, as unverified ORCID iDs will not appear in the published article. Only the individual author can complete the verification step; PLOS staff cannot verify ORCID iDs on behalf of authors.

We look forward to receiving your revised manuscript.

Kind regards,

Kimberly M. Pepin

Academic Editor

PLOS Computational Biology

Jennifer Flegg

Section Editor

PLOS Computational Biology

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Authors:

Please note here if the review is uploaded as an attachment.

Reviewer #1: The major concerns I had were mostly clarifying methods and expanding on or including discussion points that I thought were missing. The authors have done a good job responding to these comments. After reading through the responses to my comments (and the other reviewer comments) it is clear the authors made a good faith effort to revise the manuscript. The only additional request to complete Supplementary Table 2. It is supposed to have results but it is only a list of models used in the full Pareto analysis.

Reviewer #2: The updated manuscript addresses my concerns and those of the other reviewer. Key changes include:

- A more comprehensive results section (e.g. predicted/actual plots and forecast scores at sub-national levels)

- More detail about forecast models and evaluation methods

- An added analysis comparing the operational ensemble to individual operational models and to a matched retrospective ensemble

I believe the manuscript is now publishable. I have a few remaining minor scientific concerns, but I believe these can be addressed without re-review. I have also flagged some typos.

All page numbers refer to the compiled review PDF (including frontmatter).

## Remaining minor scientific concerns

Supp Table 1: Supplementary Table 1 is cited to justify the claim that "the ensemble was on average, provided both better numerical forecasts and trend direction forecasts of COVID-19 and Influenza at all spatial resolutions" (p. 39). To fully justify this claim (and fully address my prior comments), the table should include the seasonal average scores for each individual (retrospective) model (i.e. seasonal average scores for each of `ets,` `gam_cr`, etc.) in addition to the average score across individual (retrospective) models.

p. 43: "The pcWIS values for the operational ensemble are usually around the average for retrospective sub-ensembles; over the season the average pcWIS values for COVID-19 and Influenza operational ensemble forecasts were 4.69 × 10−7 and 5.62 × 10−7 respectively." The logic in this sentence is a bit unclear. The first clause makes a comparison between the operational ensemble and the average of retrospective sub-ensembles with some kind of time stratification ("usually"). The second clause reports an overall value for the whole season, only for the operational ensemble. These are separate points, and should probably be separate sentences. If the first point ("The pcWIS values for the operational ensemble are usually around the average for retrospective sub-ensembles") is to be retained, the authors must reference a figure or table that demonstrates it.

Supp Fig Q: Since 50% coverage was also assessed in the manuscript alongside 90% coverage, should include a version of this plot for 50% coverage as well.

Supp Fig Q: Is the line the overall average coverage over locations _and_ target dates? It should be. Should revise to make it so if not, and should define the line in the caption.

Supp Fig X: Is the outlier ICB with an unusually bad pcPCS but fairly typical RPS for COVID-19 possibly due to a scoring bug? Should investigate. If not due to a bug, the outlier should be noted and its cause explained to the extent possible. Optional related suggestion: perhaps worth making a version of the plot with that point excluded so that the trend for COVID excluding it is clearer. That would also enable plotting flu and COVID with shared x and y axis scales, which would also help the reader determine whether the strength of the association is stronger for one versus the other.

Supp Table 4: There is a problem with this table (possibly due to a code bug?). The mean retrospective ensemble scores should lie between the min and max scores, but do not always (e.g. COVID-19 pcWIS mean is greater than max, COVID-19 RPS mean is less than min).

## Typos and minor wording points

p. 25: "We compared operational retrospective forecasts". Missing word(s). Should be "We compared operational forecasts to retrospective forecasts" or "We compared operational and retrospective forecasts".

p. 26: "The sub-ensemble simulation showed individual models influenced the ensembles during different epidemic phases." I am not 100% sure what the intended point is here. Individual models always have some impact on an ensemble's forecast and its performance. Is the point here is that individual models often had a strong effect (good or bad) on ensemble performance in one epidemic phase but a weak effect in another? If so, that should be made more explicit.

p. 26: "single, more accurate" should be "single, ideally more accurate" or something similar. Need to acknowledge that ensembling does not necessarily succeed in its goal of improving predictions.

Fig 1: typo in legend: "operation" should be "operational".

p. 39: Extra word(s) in this clause: "this shows that the ensemble was on average, provided both better numerical forecasts and trend direction forecasts of COVID-19 and Influenza at all spatial resolutions." I think this should be: "the ensemble on average provided both better numerical forecasts and better trend direction forecasts of COVID-19 and Influenza at all spatial resolutions".

p. 43 extra words: "That being said, if we treat the operational ensemble's mostly likely trend direction at the national level matched the observed outcome on the majority of occasions". I think this should be simply: "That being said, the operational ensemble's mostly likely trend direction at the national level matched the observed trend direction on the majority of occasions".

p. 46: "As with any scoring-based approaches". "Approaches" should be "approach".

Supp Fig F, G caption: Use "improved performance" instead of "increasing performance" in these captions (avoids confusion due to negatively-oriented scores, etc.).

Supp Fig G: Figure title is incorrect. Says "pcWIS for regional RSV forecasts" but is actually pcWIS by age group for national forecasts.

Supp Fig Q: Typo in figure title. Title should be "Observed versus theoretical coverage" or "Observed versus nominal coverage".

Supp Fig U: Typo in spelling of "estimate" in the caption.

**********

Have the authors made all data and (if applicable) computational code underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data and code underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data and code should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data or code —e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: None

Reviewer #2: Yes

**********

PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review?  For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

Figure resubmission:

While revising your submission, we strongly recommend that you use PLOS’s NAAS tool (https://ngplosjournals.pagemajik.ai/artanalysis) to test your figure files. NAAS can convert your figure files to the TIFF file type and meet basic requirements (such as print size, resolution), or provide you with a report on issues that do not meet our requirements and that NAAS cannot fix.

After uploading your figures to PLOS’s NAAS tool - https://ngplosjournals.pagemajik.ai/artanalysis, NAAS will process the files provided and display the results in the "Uploaded Files" section of the page as the processing is complete. If the uploaded figures meet our requirements (or NAAS is able to fix the files to meet our requirements), the figure will be marked as "fixed" above. If NAAS is unable to fix the files, a red "failed" label will appear above. When NAAS has confirmed that the figure files meet our requirements, please download the file via the download option, and include these NAAS processed figure files when submitting your revised manuscript.

Reproducibility:

To enhance the reproducibility of your results, we recommend that authors of applicable studies deposit laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. Additionally, PLOS ONE offers an option to publish peer-reviewed clinical study protocols. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols

Dear Dr Kennedy,

We are pleased to inform you that your manuscript 'Evaluation of short-term multi-target respiratory forecasts over winter 2024-25 in England using sub-ensemble contribution analyses' has been provisionally accepted for publication in PLOS Computational Biology.

Before your manuscript can be formally accepted you will need to complete some formatting changes, which you will receive in a follow up email. A member of our team will be in touch with a set of requests.

Please note that your manuscript will not be scheduled for publication until you have made the required changes, so a swift response is appreciated.

IMPORTANT: The editorial review process is now complete. PLOS will only permit corrections to spelling, formatting or significant scientific errors from this point onwards. Requests for major changes, or any which affect the scientific understanding of your work, will cause delays to the publication date of your manuscript.

Should you, your institution's press office or the journal office choose to press release your paper, you will automatically be opted out of early publication. We ask that you notify us now if you or your institution is planning to press release the article. All press must be co-ordinated with PLOS.

Thank you again for supporting Open Access publishing; we are looking forward to publishing your work in PLOS Computational Biology.

Best regards,

Kimberly M. Pepin

Academic Editor

PLOS Computational Biology

Jennifer Flegg

Section Editor

PLOS Computational Biology

***********************************************************