Analyzing the performance of deep learning splice prediction algorithms

Nathan Fortier; Gabe Rudy; Andreas Scherer

doi:10.1371/journal.pone.0348885

Peer Review History

Original SubmissionDecember 23, 2025
18 Jan 2026 Decision Letter - Yang Zhang, Editor PONE-D-25-68125Analyzing the performance of deep learning splice prediction algorithmsPLOS One Dear Dr. Fortier, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Mar 04 2026 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Yang Zhang Academic Editor PLOS One Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Please note that PLOS One has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, we expect all author-generated code to be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse. 3. Thank you for stating the following in the Competing Interests section: “The authors are employees of Golden Helix, Inc., which develops and distributes VarSeq software containing the Legacy Ensemble splice prediction algorithms evaluated in this study. Golden Helix has no commercial relationship with the developers of SpliceAI, OpenSpliceAI, or CI-SpliceAI. The design, conduct, and reporting of this research were not influenced by commercial considerations.” Please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials, by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests). If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared. Please include your updated Competing Interests statement in your cover letter; we will change the online submission form on your behalf. 4. We noted in your submission details that a portion of your manuscript may have been presented or published elsewhere. “A preprint version of this manuscript has been posted on bioRxiv (doi: 10.1101/2025.11.20.689501). The current manuscript contains substantial revisions and improvements based on feedback received on the preprint. The manuscript is not under consideration at any other peer-reviewed journal.” Please clarify whether this [conference proceeding or publication] was peer-reviewed and formally published. If this work was previously peer-reviewed and published, in the cover letter please provide the reason that this work does not constitute dual publication and should be included in the current manuscript. 5. Thank you for stating the following in the Competing Interests section: “The authors are employees of Golden Helix, Inc., which develops and distributes VarSeq software containing the Legacy Ensemble splice prediction algorithms evaluated in this study. Golden Helix has no commercial relationship with the developers of SpliceAI, OpenSpliceAI, or CI-SpliceAI. The design, conduct, and reporting of this research were not influenced by commercial considerations.” We note that one or more of the authors are employed by a commercial company: Golden Helix, Inc 1. Please provide an amended Funding Statement declaring this commercial affiliation, as well as a statement regarding the Role of Funders in your study. If the funding organization did not play a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and only provided financial support in the form of authors' salaries and/or research materials, please review your statements relating to the author contributions, and ensure you have specifically and accurately indicated the role(s) that these authors had in your study. You can update author roles in the Author Contributions section of the online submission form. Please also include the following statement within your amended Funding Statement. “The funder provided support in the form of salaries for authors [insert relevant initials], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.” If your commercial affiliation did play a role in your study, please state and explain this role within your updated Funding Statement. 2. Please also provide an updated Competing Interests Statement declaring this commercial affiliation along with any other relevant declarations relating to employment, consultancy, patents, products in development, or marketed products, etc. Within your Competing Interests Statement, please confirm that this commercial affiliation does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests) . If this adherence statement is not accurate and there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared. Please include both an updated Funding Statement and Competing Interests Statement in your cover letter. We will change the online submission form on your behalf. 6. If the reviewer comments include a recommendation to cite specific previously published works, please review and evaluate these publications to determine whether they are relevant and should be cited. There is no requirement to cite these works unless the editor has indicated otherwise. Additional Editor Comments: Specifically, please (i) improve benchmark independence by adding more recent, well-curated and/or high-throughput functional splicing datasets and clearly reporting canonical vs non-canonical variant composition, and (ii) revise or justify any pre-filtering steps that may introduce bias. The explanation for OpenSpliceAI’s reduced performance on the Riepe dataset should be supported by direct empirical analysis (e.g., training data/isoform coverage comparison) and, where possible, validated on an expanded functional assay dataset. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ******** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The authors aim to compare CI-SpliceAI, OpenSpliceAI and SpliceAI in an independent manner. This manuscript addresses an important question, as a large proportion of medical genomics is conducted in commercial settings. The authors bring novel findings in this regard. I have minor revisions to suggest: The authors base one of their benchmark on the same dataset used by CI-SpliceAI. I have nothing against the dataset from CI-SpliceAI, but independent comparison should try to depart from the original advantageous datasets. Here are 2 examples of manually curated splice variants lists: https://pubmed.ncbi.nlm.nih.gov/36273432/, https://pubmed.ncbi.nlm.nih.gov/37878682/ I think these would be more relevant to use instead of the CI-SpliceAI dataset. I think these have a fair good overlap with clinvar variants, but they should be much more clean than clinvar. Same remark for the multi parallel assays used, published in 2021. I would have expected the authors to use a more recent and more thorough publication like https://pubmed.ncbi.nlm.nih.gov/38129864/ L76 “SpliceAI web interface”, I think the authors are referring to the broad institute web site? if so, please, reformulate. ll 153-157, I fail to see the logic of this pre-filtering, as it potentially mask some true prediction unseen by the old-school predictors. Indeed, more generally, this pre-filtering introduces some serious bias toward the legacy ensemble. If the authors wish to enrich their dataset in true splicing variants, they could have filtered for the pathogenic variants which are either synonymous or intronic. Looking manually through some of the pathogenic variants used in ClinVar Benchmark Dataset, I have mostly seen some canonical splice sites. I may be wrong, but I suspect these splice site variants constitute a good deal of the dataset. This might be a slight issue given the goal of the authors to explore the non canonical splice site variants (lines 3-6 of the manuscript). The authors should give this detail to the reader. More generally, this is also interesting to provide for the two other dataset (I mean, the respective proportions of canonical splice variants). Reviewer #2: This manuscript presents a timely and valuable benchmark of open-source splice prediction tools (OpenSpliceAI and CI-SpliceAI) against the industry-standard SpliceAI. By evaluating these models across multiple datasets, the study addresses a critical need in the field: validating permissively licensed alternatives that can be integrated into clinical pipelines. The confirmation that these open-source models generally reproduce the predictive power of the original SpliceAI is an important resource for the community. However, to fully establish these tools as reliable alternatives for clinical diagnostics, further rigorous investigation into specific performance discrepancies, particularly regarding functional validation, is necessary. Major Comments 1. The authors currently attribute OpenSpliceAI's significantly lower performance on the Riepe dataset to "gene-specific splicing patterns in ABCA4 and MYBPC3 that are not fully captured by OpenSpliceAI's training data". This explanation is currently speculative and, as a defense for a tool intended for genome-wide application, inadequate without empirical backing. It remains unclear whether this poor performance stems from the architecture/methodology or simply the training data source (RefSeq MANE vs. GENCODE). Since the splice sites used for the original SpliceAI training are accessible in its repository, it would be highly beneficial to directly compare the splice site overlap between the SpliceAI and OpenSpliceAI training sets. Specifically, the authors should verify if the isoforms or intronic regions relevant to these ABCA4 and MYBPC3 variants were excluded from the RefSeq MANE training set. Confirming whether these sites were "seen" by one model and not the other would definitively clarify if the failure is a data coverage issue. 2. Expanding Functional Assay Benchmarking While the inclusion of the Riepe dataset is a valuable starting point, its small sample size (N=213) and restriction to only two genes limit the ability to draw broad conclusions about the functional reliability of OpenSpliceAI compared to the original SpliceAI. The observed performance gap on this dataset is concerning, and N=213 is insufficient to determine if this is a systematic issue with "cryptic" splicing or an artifact of those specific genes. To robustly validate the tools, I strongly recommend expanding the benchmark to include larger, high-throughput functional datasets. Incorporating even a subset of the following resources would significantly strengthen the manuscript’s conclusions: * MFASS (Multiplexed Functional Assay of Splicing using Sort-seq): (Cheung et al., 2019) * MaPSy (Massively Parallel Splicing Assay): (Soemedi et al., 2017) * Vex-seq (Variant Exon Sequencing): (Adamson et al., 2018) Expanding the evaluation to include one or more of these high-throughput datasets would move the study beyond a simple reproduction effort and establish it as a definitive benchmark for the clinical community. It would also help clarify whether the current performance discrepancies are limited gene-specific artifacts or systematic differences in how the open-source models handle cryptic splicing. ******** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: Yes: Jean-Madeleine de Sainte Agathe Reviewer #2: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] To ensure your figures meet our technical requirements, please review our figure guidelines: https://journals.plos.org/plosone/s/figures You may also use PLOS’s free figure tool, NAAS, to help you prepare publication quality figures: https://journals.plos.org/plosone/s/figures#loc-tools-for-figure-preparation. NAAS will assess whether your figures meet our technical requirements by comparing each figure against our figure specifications. https://doi.org/10.1371/journal.pone.0348885.r001
Revision 1
9 Mar 2026 Author Response We thank the editor and reviewers for their thoughtful and constructive comments. We have carefully considered each point raised and have revised the manuscript accordingly. Below we provide a point-by-point response to each comment. Editor Comments Comment 1: Improve benchmark independence by adding more recent, well-curated and/or high-throughput functional splicing datasets and clearly reporting canonical vs non-canonical variant composition. Response: We have incorporated three additional benchmark datasets into the study: (1) the SPiP dataset (Leman et al., 2022), comprising 99,601 variants with experimentally validated splicing outcomes across 227 genes; (2) the Barbosa deep intronic benchmark (Barbosa et al., 2023), comprising 242 manually curated pathogenic deep intronic variants; and (3) the Barbosa ClinVar benchmark (Barbosa et al., 2023), comprising 53,600 intronic variants from ClinVar. We have also added explicit reporting of the proportion of variants altering the canonical splice site dinucleotides for each dataset. For the Barbosa deep intronic benchmark, canonical splice site variants are excluded by design, as all variants are required to be located more than 10 bp from the nearest annotated splice site. Comment 2: Revise or justify any pre-filtering steps that may introduce bias. Response: We have revised the Methods section to more explicitly acknowledge the potential bias introduced by the Legacy Ensemble pre-filtering step in the Extended ClinVar benchmark. Specifically, we now note that this filtering strategy may systematically exclude variants detectable only by deep learning models but not by any Legacy Ensemble algorithm, potentially conservatively underestimating the specificity advantage of deep learning approaches. We further note that this limitation is partially mitigated by the inclusion of the Barbosa deep intronic benchmark, which evaluates algorithm performance on deeply intronic variants without any pre-filtering based on legacy tool predictions. Comment 3: The explanation for OpenSpliceAI's reduced performance on the Riepe dataset should be supported by direct empirical analysis and, where possible, validated on an expanded functional assay dataset. Response: We have replaced the speculative explanation with a direct empirical comparison of the GENCODE v24 annotation used to train SpliceAI and the RefSeq MANE Select annotation used to train OpenSpliceAI at both the ABCA4 and MYBPC3 loci. This analysis revealed two key findings. First, ABCA4 resides on a held-out chromosome under the data split applied by both models, meaning neither was trained on ABCA4 splice sites, and any performance difference at this locus reflects generalization capacity rather than training data coverage. Second, while GENCODE v24 contributed three protein-coding isoforms for MYBPC3 encompassing 36 unique introns compared to the single MANE Select transcript with 34 introns, none of the Riepe intronic and splice-region variants in the MYBPC3 dataset fell within the GENCODE-exclusive introns. These findings rule out annotation coverage as an explanation for the observed performance gap. We further note that the convergence of this finding with the results on the Barbosa deep intronic benchmark, where OpenSpliceAI also underperformed relative to SpliceAI, suggests a more general difficulty with deeply intronic variant detection rather than a gene-specific effect. Reviewer 1 Comment 1: The authors base one of their benchmarks on the same dataset used by CI-SpliceAI. Independent comparison should try to depart from the original advantageous datasets. Two examples of manually curated splice variant lists were suggested: https://pubmed.ncbi.nlm.nih.gov/36273432/ and https://pubmed.ncbi.nlm.nih.gov/37878682/. Response: We thank the reviewer for this suggestion. We note that the two publications referenced correspond to the SPiP dataset (Leman et al., 2022) and the Barbosa benchmark (Barbosa et al., 2023), both of which have been incorporated into the revised manuscript as described above. We have retained the CI-SpliceAI benchmark dataset as it provides a useful point of comparison with the performance reported in the original CI-SpliceAI publication, but the addition of these independent datasets substantially strengthens the benchmark independence of our study. Comment 2: The SpliceAI web interface reference should be reformulated. Response: We have revised the text to explicitly identify this as the Broad Institute SpliceAI Lookup web interface and have included the URL directly in the manuscript text. Comment 3: The pre-filtering step potentially masks true predictions unseen by legacy predictors and introduces bias toward the legacy ensemble. Response: We have revised the Methods section to explicitly acknowledge this limitation, as described in our response to Editor Comment 2 above. Comment 4: The proportion of canonical splice site variants should be reported for all datasets. Response: We have added explicit reporting of canonical splice site dinucleotide-altering variant proportions for all datasets, as described in our response to Editor Comment 1 above. Reviewer 2 Comment 1: The explanation for OpenSpliceAI's reduced performance on the Riepe dataset is speculative and inadequate without empirical backing. Response: We have addressed this with direct empirical analysis as described in our response to Editor Comment 3 above. Comment 2: The functional assay benchmarking should be expanded to include larger high-throughput datasets such as MFASS, MaPSy, and Vex-seq. Response: We thank the reviewer for these suggestions. We have considered each of these resources carefully. However, these datasets present some limitations for our specific benchmarking goals. MaPSy (Soemedi et al., 2017) and MFASS (Chong et al., 2019) focus primarily on exonic variants, and MFASS and Vex-seq (Adamson et al., 2018) are based on minigene reporter systems that lack the native genomic context that SpliceAI and its reimplementations rely on for prediction, potentially limiting the interpretability of performance comparisons. Most importantly, all three datasets focus on exonic and near-exonic variants rather than the deeply intronic variants that represent the primary performance differentiator among the algorithms evaluated here. We have acknowledged these resources and their limitations in the Study Limitations section of the revised manuscript. The addition of the SPiP and Barbosa datasets, which include experimentally validated splicing outcomes across a broad range of variant types and genomic contexts, substantially expands the functional benchmarking scope of the study beyond the original Riepe dataset. We believe these revisions comprehensively address all comments raised by the editor and reviewers and have substantially strengthened the manuscript. We thank the reviewers again for their constructive engagement with our work. Attachments Attachment Submitted filename: ResponseToReviewers.pdf https://doi.org/10.1371/journal.pone.0348885.r002
22 Apr 2026 Decision Letter - Yang Zhang, Editor Analyzing the performance of deep learning splice prediction algorithms PONE-D-25-68125R1 Dear Dr. Fortier, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. For questions related to billing, please contact billing support. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Yang Zhang Academic Editor PLOS One Additional Editor Comments (optional): Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: I thank the authors for their revisions, which improved the quality of this manusript. They have better explained and clarify several points. Reviewer #2: I would like to commend the authors for their thorough and thoughtful revisions to the manuscript. The revised version successfully addresses the primary concerns raised during the initial review. Most notably, the incorporation of the three independent, external benchmark datasets—the SPiP dataset, the Barbosa deep intronic benchmark, and the Barbosa ClinVar benchmark—substantially strengthens the study's findings. This approach to external benchmarking provides a much more robust, reliable, and unbiased evaluation of the splice prediction algorithms. The authors have fully satisfied my previous concerns regarding benchmark independence. The manuscript is much improved, and the conclusions are now well-supported by independent data. ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: Yes: Jean-Madeleine de Sainte Agathe Reviewer #2: No ******** https://doi.org/10.1371/journal.pone.0348885.r003
Formally Accepted
Acceptance Letter - Yang Zhang, Editor PONE-D-25-68125R1 PLOS One Dear Dr. Fortier, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS One. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset You will receive further instructions from the production team, including instructions on how to review your proof when it is ready. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few days to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. You will receive an invoice from PLOS for your publication fee after your manuscript has reached the completed accept phase. If you receive an email requesting payment before acceptance or for any other service, this may be a phishing scheme. Learn how to identify phishing emails and protect your accounts at https://explore.plos.org/phishing. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Yang Zhang Academic Editor PLOS One https://doi.org/10.1371/journal.pone.0348885.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .