PONE-D-24-38787A wearable-based AI algorithm for the remote early detection of SARS-CoV-2 infections: results from the COVID-RED study, a prospective randomised single-blinded crossover trialPLOS ONE

Dear Dr. Zwiers,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by Jan 19 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Cecilia Acuti Martellucci, M.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at 

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and 

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. We note that the current version ethics document has masked details. Before we proceed with your submission, please upload a clean version of the ethics document. 

3. During our internal review of your submission, we noted that your clinical trial registration number (NL9320) cannot be found on the the Netherlands Trial Register. Please provide the correct registration number and a link to the online registration page if possible. Thank you for your attention this request. 

4. Please note that funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. Please remove any funding-related text from the manuscript. 

5. Thank you for stating the following in the Competing Interests section: 

“I have read the journal's policy and the authors of this manuscript have the following competing interests: 

BMG, VK, AM, and MC are current or previous employees of Ava AG.

LCZ, TBB, BMG, DV, MW, BF, JB, DEG are current or previous employees of Julius Clinical.

BF is currently an employee of Haleon.

KG and OCW are current or previous employees of Dr Risch.

MR and LR are current employees and key shareholders at Dr Risch.”

We note that one or more of the authors are employed by a commercial company: Ava AG, Julius Clinical, Haleon, Dr Risch

1) Please provide an amended Funding Statement declaring this commercial affiliation, as well as a statement regarding the Role of Funders in your study. If the funding organization did not play a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and only provided financial support in the form of authors' salaries and/or research materials, please review your statements relating to the author contributions, and ensure you have specifically and accurately indicated the role(s) that these authors had in your study. You can update author roles in the Author Contributions section of the online submission form.

Please also include the following statement within your amended Funding Statement. 

“The funder provided support in the form of salaries for authors [insert relevant initials], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.”

If your commercial affiliation did play a role in your study, please state and explain this role within your updated Funding Statement. 

2) Please also provide an updated Competing Interests Statement declaring this commercial affiliation along with any other relevant declarations relating to employment, consultancy, patents, products in development, or marketed products, etc.  

Within your Competing Interests Statement, please confirm that this commercial affiliation does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: ""This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests). 

If this adherence statement is not accurate and there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include both an updated Funding Statement and Competing Interests Statement in your cover letter. We will change the online submission form on your behalf.

6. One of the noted authors is a group or consortium: COVID-RED consortium

In addition to naming the author group, please list the individual authors and affiliations within this group in the acknowledgments section of your manuscript. Please also indicate clearly a lead author for this group along with a contact email address.

7. Please include a caption for figure 1. 

8. Please upload a new copy of Figure 2 as the detail is not clear. Please follow the link for more information:

https://blogs.plos.org/plos/2019/06/looking-good-tips-for-creating-your-plos-figures-graphics/

https://blogs.plos.org/plos/2019/06/looking-good-tips-for-creating-your-plos-figures-graphics/

9. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information. 

10. We note that the original protocol file you uploaded contains a confidentiality notice indicating that the protocol may not be shared publicly or be published. Please note, however, that the PLOS Editorial Policy requires that the original protocol be published alongside your manuscript in the event of acceptance. Please note that should your paper be accepted, all content including the protocol will be published under the Creative Commons Attribution (CC BY) 4.0 license, which means that it will be freely available online, and any third party is permitted to access, download, copy, distribute, and use these materials in any way, even commercially, with proper attribution.

Therefore, we ask that you please seek permission from the study sponsor or body imposing the restriction on sharing this document to publish this protocol under CC BY 4.0 if your work is accepted. We kindly ask that you upload a formal statement signed by an institutional representative clarifying whether you will be able to comply with this policy. Additionally, please upload a clean copy of the protocol with the confidentiality notice (and any copyrighted institutional logos or signatures) removed.

Additional Editor Comments:

The submitted manuscript reports on a rigorous and extensive work. My major concern is the inconsistency between the results and the conclusions. The findings will be much more useful for future research if they are commented faithfully, without claims that are not supported by the data. Please ensure that this is revised throughout the whole manuscript.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: No

Reviewer #4: No

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The trial design is quite interesting, as it involves a crossover with two groups over two periods. The statistical analysis is well-defined and rigorously applied, and the results are well-reported. However, the specificity of the experimental protocol was notably low and disappointing. This has important implications, as a protocol that very frequently returns positive results tends to overestimate infections. While it is preferable to report an infection rather than miss it, the extremely low specificity raises concerns. I am resistant to say that a device would necessarily be effective or (cost effective) in these circumstances. The manuscript does point to the low specificity and says that future finetuning mechanisms are needed. I would like to see more discussion on how to do finetuning and to evaluate the effectiveness.

An initial suggestion would be to use the Youden index to establish an appropriate cutoff, ensuring the protocol is more effective in practice, although finetuning most likely involves more effort than simply using this index.

Other comments and suggestions are provided below.

Abstract

- The phrase "Performance evaluated using measures of diagnostic accuracy" is vague. It would be helpful to provide more specific details on the evaluation metrics used.

- The reported specificity of "0.8–4.4%" seems likely to be a typo. I believe it should be "0.8%–4.2%." Please verify this.

- The term "algorithm ingesting data" is somewhat technical (my impression) for an abstract. To reach a broader audience, I suggest rephrasing it to "data was used as input to the algorithm" or something along these lines.

Introduction

- "Relatively young sample of individuals" should be revised to "sample of relatively young individuals."

- Line 194: Correct the phrase to "daily diary."

Methodology

- Was a sample size analysis performed? While the number of people enrolled seems sufficiently large, there is a question about whether the study period was sufficient to observe a number of outcomes. I recommend including information on sample size analysis conducted.

- The description of "per-day analysis" is unclear. I understand it as calculating the number of infected days across all participants, which serves as a measure of person-time. Please verify if this interpretation is correct and consider clarifying the description in the text and in Table 5.

- Another suggestion is to report the Youden index (sensitivity + specificity - 1) and indicate under which circumstances this index was positive (or exceeded a defined cutoff).

Results

- Figure 2 requires higher image resolution or better text contrast. When I attempted to download it to try to see in better resolution, only the Figure 3 was downloaded. I double checked it but I am not sure why this happens.

- In S1 Appendix, I suggest adding to the captions that "survival probability" refers to the probability that the event has not occurred.

Discussion

As indicated above, I would like to see a clearer outline of how to fine-tune the protocol, as well as metrics for a cost-effectiveness analysis.

Reviewer #2: Main strength's of the paper:

1) Innovative concept and relevance

The COVID-RED study embodies an innovative approach by leveraging wearable technology and artificial intelligence (AI) to address a critical public health issue: the early detection of SARS-CoV-2 infections. This research is particularly relevant in light of the COVID-19 pandemic, where the ability to identify infections during the pre-symptomatic or asymptomatic phase could significantly curb transmission rates. The study bridges advanced AI algorithms with wearable medical devices, showcasing a futuristic and interdisciplinary method for monitoring public health. This integration of technology with healthcare represents a paradigm shift from traditional diagnostics to more proactive and predictive health monitoring.

2) Robust study design

The study design is meticulously structured to ensure reliability and scientific rigor:

- Randomized Controlled Trial (RCT): A prospective, single-blinded, two-period, two-sequence, randomized crossover trial was employed. This robust design minimizes biases and allows a direct comparison between the experimental (wearable-based AI algorithm) and control (symptom-based) conditions.

- Large Sample Size: The inclusion of 17,825 participants is a remarkable achievement, providing high statistical power and the ability to generalize findings across populations.

- Crossover Nature: The two-period crossover design ensures that each participant serves as their control, reducing variability and enhancing the internal validity of the results.

These features collectively underscore the study's scientific robustness and provide a solid foundation for its conclusions.

3) Use of cutting-edge wearable technology

The article highlights the use of the Ava bracelet, an FDA-cleared and CE-certified wearable device originally designed for fertility tracking, repurposed for infection detection. The bracelet's ability to measure multiple physiological parameters, including respiratory rate, heart rate, heart rate variability, wrist-skin temperature, and skin perfusion, is a testament to its advanced capabilities.The study emphasizes the bracelet’s potential for continuous health monitoring, enabling real-time alerts for possible infections.Repurposing existing technology for public health surveillance is a forward-thinking approach that maximizes the utility of available resources.The wearable is user-friendly, requiring participants to wear it during sleep and synchronize data via a mobile app, ensuring convenience and accessibility.

4) Significant findings supporting early detection

The article demonstrates that the experimental wearable-based AI algorithm outperforms symptom-only models in detecting SARS-CoV-2 infections. The algorithm achieved high sensitivity (93.8-99.2%) in detecting infections during the study period, ensuring that most infected individuals were identified. Alerts based on the wearable device were issued significantly earlier (median of 7 days before a positive test) compared to symptom-based alerts, which had no prior warning.These findings underscore the potential of wearable devices to serve as an early warning system, particularly valuable during infectious disease outbreaks.

5) Comprehensive Performance Evaluation

The article provides a detailed evaluation of the experimental algorithm's performance through various metrics:

- Sensitivity vs. Specificity: While acknowledging the trade-offs (high sensitivity but low specificity), the study emphasizes the algorithm’s strengths in detecting infections, particularly in pre-symptomatic phases.

- Multi-Faceted Analysis: Different analytical approaches—time-to-infection, time-to-indication, and per-day analyses—offer a comprehensive assessment of the algorithm’s utility.

6) Real-World Applicability

The study's context—conducted during the COVID-19 pandemic—adds real-world relevance to its findings. Unlike retrospective studies, this trial evaluated the algorithm’s effectiveness in real-time, simulating actual scenarios where timely detection is critical.By combining physiological data with laboratory-confirmed SARS-CoV-2 infections, the study achieves a holistic approach to infection detection.

Areas of improvement

1) Specific challenges

One of the study's key findings is the low specificity of the experimental algorithm (0.8–4.4%) compared to the symptom-only control algorithm (65–66.4%). While high sensitivity ensures most infections are detected, the high rate of false positives has significant implications. Frequent false alerts can overwhelm testing resources and create undue anxiety among users, reducing trust in the system.Excessive testing for false positives may strain public health systems, particularly during pandemics.

Suggestions:

- Incorporate additional parameters, such as behavioral data or exposure history, to refine the algorithm.

- Use advanced machine learning techniques, such as ensemble methods or Bayesian inference, to balance sensitivity and specificity better.

- Develop dynamic thresholds that adapt to local infection prevalence, reducing false positives in low-risk settings.

2) Algorithm adaptability

The study design required freezing the algorithm at the start of each study period, which limited its ability to adapt to changing epidemiological conditions or incorporate newly available data. The fixed algorithm could have reduced the model's effectiveness, especially as new variants of SARS-CoV-2 emerged or vaccination rates increased.

Suggestions:

- Implement continuous learning mechanisms to allow the algorithm to evolve in response to new data, such as emerging variants or shifting symptoms.

- Explore federated learning, where the algorithm learns from decentralized data sources without compromising user privacy, to increase adaptability.

3) Limited focus on specific infections

The algorithm was designed to detect SARS-CoV-2 infections but lacked the ability to differentiate between COVID-19 and other respiratory illnesses. Many false positives could result from physiological changes due to non-COVID-19 infections, such as the flu or common cold. The study did not explore the potential of wearable devices for broader infectious disease surveillance.

Suggestions:

- Expand the dataset to include physiological data from individuals with other respiratory infections.

- Train the algorithm to distinguish between SARS-CoV-2 and other illnesses, enhancing its diagnostic utility.

- Investigate the algorithm’s potential for detecting comorbid conditions or other diseases that impact similar physiological parameters.

4) Generalizability of findings

Although the study's sample size was large, its findings may have limited applicability to diverse populations:

- Demographic Limitations: Most participants were from the Netherlands, potentially limiting generalizability to populations with different demographics, healthcare systems, or COVID-19 prevalence rates.

- Age Range and Health Status: The study included primarily younger and healthier individuals, with fewer older or high-risk participants.

Suggestions:

- Conduct follow-up studies in diverse geographic locations and healthcare contexts to ensure broader applicability.

- Stratify results by demographic factors (e.g., age, gender, pre-existing conditions) to understand how the algorithm performs across subgroups.

Reviewer #3: This manuscript presents the COVID-RED study, which investigates the use of the Ava bracelet for the early detection of SARS-CoV-2 infections in real-time. The study is noteworthy for its scale, involving approximately 20,000 participants spanning diverse age groups and genders. The results demonstrate that integrating wearable data significantly enhances the sensitivity of COVID-19 detection, although it comes at the cost of reduced specificity compared to the control group without wearable data. The study is well-designed, and the dataset is particularly valuable due to its large sample size. The analyses are robust, and the manuscript is clearly written and well-structured.

However, my primary concern lies in the manuscript's alignment with its title, "A wearable-based AI algorithm for the remote early detection...". While the study focuses on the advantage of including wearable data, there is insufficient information about the AI algorithm itself. Details on the algorithm—both with and without wearable data—are largely absent. Since algorithmic performance can significantly impact predictive outcomes, it is essential to provide more information about the models used, their training processes, and any comparisons made to alternative algorithms. Without this critical component, it is challenging to assess whether the best-performing algorithms were employed for both experimental and control groups. I recommend including these details to enable a more comprehensive evaluation of the study.

Reviewer #4: Dear authors,

Thank you for your collective efforts to advance the use of wearable devices in infectious disease surveillance. Here are some points to consider regarding your paper:

There is no caption in the text for Figure 1. Other figures also seem to be incorrectly referenced in the body of text (1 for 2, 2 for 3, 3 for 1).

Line 220: A brief description of the ML algorithm used would be helpful.

Line 226: How were the participants who did not seek testing classified?

Line 229: How were the participants who did not return the kits classified? (completed follow-up/drop-out/…)

Line 232: repeated “the end of”

Line 309: How was the beginning of infection determined in asymptomatic cases?

Line 336: Table legends should be placed above the body of the table (not below).

Line 381: Please provide the value of AUC in addition to the stated metrics (line 316).

Line 384: The minimum and median time-to-indication for the experimental condition in both groups are seven days (Line 354). I see that it is stated in line 276, “The clinical endpoint of interest was the first red alert indicator in the week prior to the date at which the SARS-CoV-2 infection was confirmed through testing”. Here, the question is raised regarding the basis upon which this timeframe (one week or seven days) is determined. If a different duration was chosen, supposedly 3 days or 14 days, would we have seen the median to be 3 days and 14 days, respectively? Considering the high false positive rate of the model, more explanation could be helpful to differentiate it from a random model, stochastically raising alarms. Such a random model could have still achieved similar results of seven days, rendering the significance of this finding obsolete (Line 384).

Line 390: It is acknowledged that both sensitivity and specificity improved using the second definition. Although the experimental condition achieved a sensitivity superior to that of control, specificity was yet considerably lower. So, it would be incorrect to deduce that the experimental algorithm performed better “… in comparison to the control algorithm”. The accuracy is still lower than that of the control (50.2% Vs. 90.1%). A comparison cannot be based merely on sensitivity. It would be much more helpful if the area under the curve is presented for each algorithm as it was previously planned in the methods section (line 316) to help make a comparison.

Line 408: “This might be explained by the algorithm’s inability to differentiate between SARS-CoV-2 and other (respiratory) infections”. A much better explanation is provided on Line 422. Other infections, given that 67.7% of subjects received a red alert at least once, is a farfetched explanation.

Line 460: “… with the experimental algorithm achieving high sensitivity”. Partial representation of the findings-should be revised. Nonetheless, the study can “confirm a potential future role of wearable devices in infectious disease surveillance” even without an outstanding performance of the algorithm.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #1: No

Reviewer #2: Yes:  Axel Moyal

Reviewer #3: No

Reviewer #4: Yes:  Shahrokh Mousavi

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org . Please note that Supporting Information files do not need this step.

PONE-D-24-38787R1A wearable-based AI algorithm for the remote early detection of SARS-CoV-2 infections: results from the COVID-RED study, a prospective randomised single-blinded crossover trialPLOS ONE

Dear Dr. Zwiers,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

Please submit your revised manuscript by May 03 2025 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org . When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

• A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.
• A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.
• An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols . Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols .

We look forward to receiving your revised manuscript.

Kind regards,

Cecilia Acuti Martellucci, M.D.

Academic Editor

PLOS ONE

Additional Editor Comments:

I apologise for taking a long time to make a decision. I believe the manuscript was greatly improved upon revision, however I also agree with Reviewers 1 and 4. If this work is to contribute to the developement of the relative research domain, it should provide the details necessary for reproducibility.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: (No Response)

Reviewer #2: All comments have been addressed

Reviewer #3: All comments have been addressed

Reviewer #4: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: No

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: No

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: No

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: Several questions and concerns were raised in my review of the first submission regarding the methodology and the evaluation of the results in assessing the algorithm. I had hoped to see these questions addressed to better understand its performance and the reasons for the observed low specificity. This understanding is really important for a proper evaluation of the results. In fact, another review noted that the title of the manuscript specifically refers to the algorithm. Unfortunately, many of these issues cannot be fully addressed due to the proprietary nature of the algorithm, as stated in the authors’ response. Furthermore, additional evaluations are not possible for the same reason. I see this lack of description regarding the algorithm as a major weakness.

Reviewer #2: During the first review, I have already accepted the original submission with specific comments and do not have more remarks to add.

Reviewer #3: All my comments have been addressed by the authors and I recommend this paper to be accepted by PLOS One journal.

Reviewer #4: Dear authors,

Thank you for your responses. I appreciate your effort in addressing the raised concerns and making the necessary revisions. However, there still remains the issue regarding the reporting of the ROC AUC.

Research is built on the ability of others to understand and evaluate findings. Allowing for the reproducibility of your methodology to be disregarded due to the proprietary nature of your product, still either intentionally or unintentionally omitting standard performance metrics, especially one as widely used as AUC, compromises transparency. Even if the internal workings of an algorithm or the probability outputs are proprietary, reporting aggregate performance metrics does not reveal trade secrets.

Moreover, as you have elaborated in the introduction by reviewing other studies and comparing their reported AUCs, it is standard practice to report this aggregate metric. Future studies, as recommended in your discussion, would not have a sound basis to compare their model with the one developed here if this metric is omitted. This aggregate metric allows readers and researchers to fairly compare the proposed method to others. Not reporting this metric leaves a gap in the evaluation of the method's effectiveness. This issue directly affects the evaluation of your method's performance and the validity of the comparisons drawn thereafter. Addressing it is essential for upholding the manuscript's scientific rigor.

Finally, regarding the technical aspect of this issue, whatever environment is employed to develop an RNN can also be used to calculate AUC, as it provides “probability outputs” for the model’s predictions.

I should also remind you that the last line of methods (line 320) still lists AUC as a metric you were supposed to calculate and report.

Best regards

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #1: No

Reviewer #2: Yes:  AXEL MOYAL

Reviewer #3: No

Reviewer #4: Yes:  Shahrokh Mousavi

**********

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/ . PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org . Please note that Supporting Information files do not need this step.

Remote early detection of SARS-CoV-2 infections using a wearable-based algorithm: results from the COVID-RED study, a prospective randomised single-blinded crossover trial

PONE-D-24-38787R2

Dear Dr. Zwiers,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager®  and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Cecilia Acuti Martellucci, M.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #4: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #4: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #4: No

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #4: No

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #4: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: In my opinion, the title in the revised manuscript is better.

Reviewer #4: Dear Authors,

Thank you for your detailed responses and for the recent revisions made to your manuscript. While I remain somewhat concerned about the omission of the ROC AUC metric, I understand the constraints imposed by the proprietary nature of the model outputs and the associated practical challenges.

Despite these concerns, I find that the revisions you have implemented sufficiently address the issues raised.

Best regards

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean? ). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy .

Reviewer #1: No

Reviewer #4: Yes:  Shahrokh Mousavi

**********