Enhancing the scalability of distance-based link prediction algorithms in recommender systems through similarity selection

Zhan Su; Zhong Huang; Jun Ai; Xuanxiong Zhang; Lihui Shang; Fengyu Zhao

doi:10.1371/journal.pone.0271891

Peer Review History

Original SubmissionAugust 23, 2021
4 Nov 2021 Decision Letter - Hua Wang, Editor PONE-D-21-27341Link prediction in recommender systems based on user-rating distance and similarity selectionPLOS ONE Dear Dr. Ai, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Dec 19 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Hua Wang Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Thank you for stating the following in the Acknowledgments Section of your manuscript: "The research project is supported by the Young Scientists Fund of the National Natural Science Foundation of China (Grant No. 61803264). Zhan Su and Jun Ai would want to convey their love and appreciation to Lingyi Ai, Zhan Su’s daughter, for inspiring us to keep fighting" We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: "Zhan Su received the funding by the Young Scientists Fund of the National Natural Science Foundation of China (Grant No. 61803264). The funder' website is at http://www.nsfc.gov.cn/. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript" Please include your amended statements within your cover letter; we will change the online submission form on your behalf 3. We note that you have stated that you will provide repository information for your data at acceptance. Should your manuscript be accepted for publication, we will hold it until you provide the relevant accession numbers or DOIs necessary to access your data. If you wish to make changes to your Data Availability statement, please describe these changes in your cover letter and we will update your Data Availability statement to reflect the information you provide Additional Editor Comments (if provided): One paragraph of the motivation of the paper with research questions is needed. That will attract readers in future. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Partly ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: No ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The authors present a user-item link prediction approach that combines user distance measurement with similarity-based user selection, and the experimental results show the model achieves satisfied performance. However, some questions should be question or solved: 1.Page 2, line 46, what does it means by “reducing the sparsity of the Slope One method.” 2.The authors mentioned some technical problems, but feel a little mixed, if authors can highlight two or three technical difficulties, and in the performance evaluation targeted to show the advantages of the proposed method, it would be better. 3.The main contribution of this article is mentioned on page 3, but the contribution should be linked to the requirements, as well as the problems that these contributions can answer. 4.Have the authors tried to change the similarity-selection equation (3) to other forms? For example, use \|s(u,v)\|>=T_h instead of s(u,v). Negative correlation is likely to be helpful in prediction. 5.Authors need to add more discussion on why the similarity selection improves the overall performance. Reviewer #2: The paper provides a link prediction method based on user distance and similarity selection. Some issues should be further considered in this work; - Refer to Algorithm 1, please provide more discussion on top-k prediction in recommendation list, in your work. Please give more explanation on it in the experimental results. - Could you please add AUC of the results in the experiments. -Please discuss in more detail the information you have employed in your work. As you know, there are some information in MovieLens. Have you only used rating rate? -Does the method help to solve the sparsity problem? Furthermore, there is a long tailed distribution for number of ratings. How does the proposed method deal with ling tailed ratings? It is more interesting to see the pitfalls of the method and future line of this study in Conclusion section. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review?** For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0271891.r001
Revision 1
5 Feb 2022 Author Response Reviewer#1, Concern # 1: Page 2, line 46, what does it means by “reducing the sparsity of the Slope One method.” Author response: Thank you for the comment. The main problem addressed in this paper is the scalability (the number of used neighbors in prediction) and prediction accuracy of model-based collaborative filtering. We have further elaborated on this goal by modifying the abstract and contribution sections. Author action: We have deleted the inaccurate expressions in the revised paper. ________________________________________ Reviewer#1, Concern # 2: The authors mentioned some technical problems, but feel a little mixed, if authors can highlight two or three technical difficulties, and in the performance evaluation targeted to show the advantages of the proposed method, it would be better. Author response: Thank you for the comment. The main technical problem addressed in this paper is the scalability (the number of used neighbors in prediction) and prediction accuracy of model-based collaborative filtering. We have further elaborated on this goal by modifying the abstract and contribution sections. Author action: In accordance with the reviewers' comments, we have revised the relevant expressions and marked them in the revised manuscript. On Page 3, marked yellow. ________________________________________ Reviewer#1, Concern # 3: The main contribution of this article is mentioned on page 3, but the contribution should be linked to the requirements, as well as the problems that these contributions can answer. Author response: Thank you for the comment. Author action: As the reviewer suggested, we have revised the contribution and marked them in the revised manuscript. On Page 3, marked yellow. ________________________________________ Reviewer#1, Concern # 4: Have the authors tried to change the similarity-selection equation (3) to other forms? For example, use \|s(u,v)\|>=T_h instead of s(u,v). Negative correlation is likely to be helpful in prediction. Author response: Thank you for the comment. We considered several similar techniques of filtering during the experimental exploratory phase, but the results were not promising. The findings of a typical comparative experiment are shown in the following figures (see attached file includes MAE and NDCG, the method reviewer suggested labeled as DSS-abs, the one we use in the paper is labeled as DSS), which illustrates that filtering neighbors with large absolute values retains both positively and negatively correlated neighbors (\|s(u,v)\|>=T_h), but will cause the prediction error to become larger and NDCG ranking is little worse than the original Slope One. Thus, we did not include other forms of experimental results in the revised manuscript. Author action: None ________________________________________ Reviewer#1, Concern # 5: Authors need to add more discussion on why the similarity selection improves the overall performance. Author response: Thank you for the comment. Author action: As suggested by the reviewer, we've added a corresponding discussion and analysis in the conclusion section. On Page 15 and 16, marked yellow. ________________________________________ Reviewer#2, Concern # 1: Refer to Algorithm 1, please provide more discussion on top-k prediction in recommendation list, in your work. Please give more explanation on it in the experimental results. Author response: Thank you for the comment. Author action: As suggested by the reviewer, we have added discussion of the Top-k prediction in the revised paper. On Page 9, marked yellow. ________________________________________ Reviewer#2, Concern # 2: Could you please add AUC of the results in the experiments. Author response: Thank you for the advice. Author action: As suggested by the reviewer, we've added Recall, Precision, Accuracy and ROC/AUC in the revised paper. On Page 12,13 and 14, add Fig 8, 9, 10 and 11, marked yellow. ________________________________________ Reviewer#2, Concern # 3: Please discuss in more detail the information you have employed in your work. As you know, there are some information in MovieLens. Have you only used rating rate? Author response: Thank you for the comment. We use only the ratings data from the MovieLens data set. Author action: We have added discussion in the revised paper to address all the detail of experiments. We also increased the number of users used in the experiments, and redrew completely new figures of the experiments. On Page 7, marked yellow. ________________________________________ Reviewer#2, Concern # 4: Does the method help to solve the sparsity problem? Furthermore, there is a long tailed distribution for number of ratings. How does the proposed method deal with ling tailed ratings? Author response: Thank you for the advice. Our main concern addressed in this paper is the scalability of model-based collaborative filtering, the proposed method aims to reduce the number of required neighbors of prediction, as well as improving the prediction accuracy. Thus, the method doesn’t help with the sparsity problem. Additionally, we don't currently consider the long-tail distribution of scores, but we'd would like to look into it in the future. Author action: We've included the suggestion in the conclusion section, along with a forecast for future study, and we're grateful to the reviewers for his or her insightful advice. On Page 16, marked yellow. ________________________________________ Reviewer#2, Concern # 5: It is more interesting to see the pitfalls of the method and future line of this study in Conclusion section. Author response: Thank you for the advice. The drawback of the algorithm is that the computational steps are added, but the reduction of the prediction error is not significant, and the similarity in the distance-based prediction process is not utilized in the previous calculation. Author action: Based on the reviewers' suggestions, we have added a relevant discussion in the conclusion section. On Page 16, marked yellow. Attachments Attachment Submitted filename: response-to-reviwers_v2.pdf https://doi.org/10.1371/journal.pone.0271891.r002
31 May 2022 Decision Letter - Hua Wang, Editor PONE-D-21-27341R1Enhancing the scalability of distance-based link prediction algorithms in recommender systems through similarity selectionPLOS ONE Dear Dr. Ai, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Jul 15 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Hua Wang Academic Editor PLOS ONE Journal Requirements: Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed Reviewer #2: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes Reviewer #2: Yes ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The authors have revised this manuscript carefully and stated how they revised in detail as the reviewers suggested, and I am satisfied with this revision in principle. The manuscript also includes a few writing mistakes, and I suggest the authors carefully to proofreading this manuscript. + Line 67 on Page 10: The statement of "Not only is the Slope One method simple to implement, but it is also highly effective” is suggested to be revised to "The Slope One method is not only simple to implement, but also highly effective". +The citations of figures in this manuscript are inconsistent, such as "Fig 2", "Figure 3", "figure 3", etc. +The metrics, such as NDCG (Line 290 on Page 18), DCG (Line 295 on Page 18), should be italic. In summary, this paper could be accepted after minor revise. Reviewer #2: (No Response) ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review?** For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0271891.r003
Revision 2
5 Jun 2022 Author Response Reviewer#1, Comment# 1: The authors have revised this manuscript carefully and stated how they revised in detail as the reviewers suggested, and I am satisfied with this revision in principle. The manuscript also includes a few writing mistakes, and I suggest the authors carefully to proofreading this manuscript. Author response: Thank you for the comment. We have carefully proofread the article. Author action: Writing errors and typos have been corrected in the revised paper, which were marked yellow in the marked manuscript. ________________________________________ Reviewer#1, Comment # 2: Line 67 on Page 10: The statement of "Not only is the Slope One method simple to implement, but it is also highly effective” is suggested to be revised to "The Slope One method is not only simple to implement, but also highly effective". Author response: Thank you for the comment. Author action: The issues pointed out by the reviewers have been corrected in the revised manuscript. ________________________________________ Reviewer#1, Concern # 3: The citations of figures in this manuscript are inconsistent, such as "Fig 2", "Figure 3", "figure 3", etc. Author response: Thank you for the comment. Author action: The issues pointed out by the reviewers have been corrected in the revised manuscript. ________________________________________ Reviewer#1, Concern # 4: The metrics, such as NDCG (Line 290 on Page 18), DCG (Line 295 on Page 18), should be italic. Author response: Thank you for the comment. Author action: The issues pointed out by the reviewers have been corrected in the revised manuscript. ________________________________________ Reviewer#2, No further comments Attachments Attachment Submitted filename: response-to-reviwers_v2.pdf https://doi.org/10.1371/journal.pone.0271891.r004
11 Jul 2022 Decision Letter - Hua Wang, Editor Enhancing the scalability of distance-based link prediction algorithms in recommender systems through similarity selection PONE-D-21-27341R2 Dear Dr. Ai, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Hua Wang Academic Editor PLOS ONE Additional Editor Comments (optional): The new version has been improved the paper quality. Great efforts! Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed ******** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes ****** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes ****** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes ****** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes ****** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: The authors have revised all the mistakes and answered all the questions, and I have no further comments. ****** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No ******** https://doi.org/10.1371/journal.pone.0271891.r005
Formally Accepted
19 Jul 2022 Acceptance Letter - Hua Wang, Editor PONE-D-21-27341R2 Enhancing the scalability of distance-based link prediction algorithms in recommender systems through similarity selection Dear Dr. Ai: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Hua Wang Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0271891.r006

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .