Postoperative Karnofsky performance status prediction in patients with IDH wild-type glioblastoma: A multimodal approach integrating clinical and deep imaging features

Tomoki Sasagasako; Akihiko Ueda; Yohei Mineharu; Yusuke Mochizuki; Souichiro Doi; Silsu Park; Yukinori Terada; Noritaka Sano; Masahiro Tanji; Yoshiki Arakawa; Yasushi Okuno

doi:10.1371/journal.pone.0303002

Peer Review History

Original SubmissionApril 17, 2024
13 Sep 2024 Decision Letter - Kevin Camphausen, Editor PONE-D-24-13237Postoperative Karnofsky performance status prediction in patients with IDH wild-type glioblastoma: a multimodal approach integrating clinical and deep imaging featuresPLOS ONE Dear Dr. Mineharu, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. ============================== Please submit your revised manuscript by Oct 28 2024 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript: A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Kevin Camphausen Academic Editor PLOS ONE Journal requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. Please note that PLOS ONE has specific guidelines on code sharing for submissions in which author-generated code underpins the findings in the manuscript. In these cases, all author-generated code must be made available without restrictions upon publication of the work. Please review our guidelines at https://journals.plos.org/plosone/s/materials-and-software-sharing#loc-sharing-code and ensure that your code is shared in a way that follows best practice and facilitates reproducibility and reuse. 3. Thank you for stating the following financial disclosure: [This work was supported by the Ministry of Education, Culture, Sports, Science, and Technology (MEXT) under the RIKEN joint research and collaboration fund for “Translational Research in Basic and Clinical Sciences for the Construction of an AI Pharmaceutical Platform”.]. Please state what role the funders took in the study. If the funders had no role, please state: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript." If this statement is not correct you must amend it as needed. Please include this amended Role of Funder statement in your cover letter; we will change the online submission form on your behalf. 4. We note that you have indicated that there are restrictions to data sharing for this study. For studies involving human research participant data or other sensitive data, we encourage authors to share de-identified or anonymized data. However, when data cannot be publicly shared for ethical reasons, we allow authors to make their data sets available upon request. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Before we proceed with your manuscript, please address the following prompts: a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information, data are owned by a third-party organization, etc.) and who has imposed them (e.g., a Research Ethics Committee or Institutional Review Board, etc.). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent. b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of recommended repositories, please see https://journals.plos.org/plosone/s/recommended-repositories. You also have the option of uploading the data as Supporting Information files, but we would recommend depositing data directly to a data repository if possible. Please update your Data Availability statement in the submission form accordingly. 5. In the online submission form, you indicated that [The clinical data in this study cannot be shared publicly because it contains potentially identifying or sensitive patient information. However, the clinical data are available, on reasonable request, from the corresponding author.]. All PLOS journals now require all data underlying the findings described in their manuscript to be freely available to other researchers, either 1. In a public repository, 2. Within the manuscript itself, or 3. Uploaded as supplementary information. This policy applies to all data except where public deposition would breach compliance with the protocol approved by your research ethics board. If your data cannot be made publicly available for ethical or legal reasons (e.g., public availability would compromise patient privacy), please explain your reasons on resubmission and your exemption request will be escalated for approval. Additional Editor Comments (if provided): [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes Reviewer #2: Yes ******** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: I Don't Know Reviewer #2: Yes ****** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: No Reviewer #2: Yes ****** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes Reviewer #2: Yes ****** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: This is very well written paper and the rationale is well thought out. I enjoyed reading this paper. I have only very minor comments. The paper would be enhanced by 1) discussing the limitations of the methodology, 2) the prospects for validation and 3) by comparing this analysis and its results with other similar studies. Reviewer #2: A manuscript entitled “Postoperative Karnofsky performance status prediction in patients with IDH wild-type glioblastoma: a multimodal 3 approach integrating clinical and deep imaging features” is submitted. Authors intended to develop a 6-month postoperative KPS prediction model by combining clinical data with deep learning-based image features from pre- and postoperative MRI scans. Overall, the study presents the data of original research. Conclusions are largely supported by the data. A few comments as below: 1.Please consider revising the statement in line 67-69, “The 6-month postoperative KPS was chosen for its clinical practicality, as it aligns with the standard timeframe for evaluating disease progression after a typical course of adjuvant treatments.” Most glioblastoma patients are in the middle of adjuvant treatment. 2. It would be important to address how the pre-surgical, intra-surgical variables and other clinical factors were selected. 3. It is unclear why patients were assigned into training and test sets by time range instead of randomly. 4. Despite the robust MRI-based model building utilizing the VAE, the multimodal model only improves the predictability slightly compared to clinical-only model. 5. It is unclear how could the KPS prediction model contribute to the surgical planning, as the modeling requires pre- and post-operative MRIs. ****** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No Reviewer #2: No ******** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. https://doi.org/10.1371/journal.pone.0303002.r001
Revision 1
3 Oct 2024 Author Response Detailed point-by-point responses to the Reviewers October 3, 2024 Thank you very much for your invaluable advice. We want to thank the editor and reviewers for their effort and time in reviewing our manuscript. We have carefully considered each comment. We fundamentally agree with all the comments and have incorporated the suggestions into the revised manuscript. The responses to the reviewers and the changes made in the revised manuscript are shown below. Response to Reviewer #1 Comment 1: " This is very well written paper and the rationale is well thought out. I enjoyed reading this paper. I have only very minor comments. The paper would be enhanced by 1) discussing the limitations of the methodology, 2) the prospects for validation and 3) by comparing this analysis and its results with other similar studies." Author's response: We really appreciate your careful review and kind comments. We have revised the Discussion and Limitations sections according to your suggestions. 1) Discussing the limitations of the methodology One of the main limitations in the development of our prediction model relates to the extraction of MRI features. Given the rapid advancements in deep learning architectures for medical image processing, this study employed only convolutional neural networks (CNNs) and variational autoencoders (VAEs). Incorporating more recent architectures like vision transformers can potentially enhance the performance of the KPS prediction model. 2) The prospects for validation The nature of the retrospective-single center analysis leads to potential selection bias. Multicenter external test sets, based on a prospective design, are necessary to verify the generalizability of the model. 3) Comparing this analysis and its results with other similar studies Research predicting the course of performance status in patients with malignant tumors is limited, in part due to the scarce availability of publicly accessible, precise clinical databases. However, in recent years, several studies have successfully employed machine learning approaches using clinical features to predict changes in performance status. These studies utilized only clinical features and developed unimodal models. As indicated in our study, a multimodal model that incorporates both clinical and imaging features can predict the performance status with significant accuracy in patients with malignant tumors, including glioblastoma. Changes to the text: “Multicenter external test sets, based on a prospective design, are necessary to verify the generalizability of the multimodal model.” (Limitations: page 18–19, lines 387–388) “Given the rapid advancements in deep learning architectures for medical image processing, this study employed only convolutional neural networks and VAEs. Incorporating more recent architectures like vision transformers may enhance the performance of our KPS prediction model.” (Limitations: page 19, lines 390–393) “However, in recent years, several studies have reported that machine learning approaches utilizing clinical features have successfully predicted changes in performance status. Using data from patients with glioblastoma, Della Pepa et al. developed a prediction model for KPS deterioration at 6 months postoperatively, achieving an AUC of 0.81 [10]. In another report, the prediction of poor performance status in patients with systemic cancer 6 months after diagnosis achieved an AUC of 0.807 [29]. These studies utilized only clinical features and developed unimodal models. As indicated in our study, a multimodal model that incorporates both clinical and imaging features can predict performance status with significantly greater accuracy in patients with malignant tumors, including glioblastoma.” (Discussion: page 18, lines 363–371) Response to Reviewer #2 Comment 1: " Please consider revising the statement in line 67-69, “The 6-month postoperative KPS was chosen for its clinical practicality, as it aligns with the standard timeframe for evaluating disease progression after a typical course of adjuvant treatments.” Most glioblastoma patients are in the middle of adjuvant treatment. " Author’s response: We are grateful for your thoughtful review. We fully agree that many patients are in the middle of adjuvant therapy at the 6-month postoperative mark in the standard protocol for glioblastoma. Our original statement was indeed misleading. We revised this sentence as follows. Changes to the text: “Commonly, the 6-month postoperative KPS score is used for evaluating disease progression in glioblastoma clinical trials [5].” (Introduction: page 4, lines 63–64) Comment 2: " It would be important to address how the pre-surgical, intra-surgical variables and other clinical factors were selected." Author’s response: We appreciate your crucial comment. The pre-surgical, intra-surgical, and other clinical parameters used in our study are based on similar variables used by previous machine learning research on prediction models of glioblastoma prognosis [1,2]. These parameters are routinely assessed and available in clinical practice. Changes to the text: “According to previous research on machine-learning-based models predicting glioblastoma prognosis, the following 28 variables were incorporated as clinical parameters [10,11].” (Materials and methods: page 6, lines 115–116) Comment 3: " It is unclear why patients were assigned into training and test sets by time range instead of randomly. " Author’s response: Thank you for your insightful comment. In studies with relatively small populations, the performance of a model on a test set can vary significantly with each random split, largely due to substantial changes in the distribution of the test set data [3]. Our study, with a cohort of 150 cases, may experience fluctuations in model performance that could either overestimate or underestimate the true performance, depending on the distribution of the test set in each random split. To mitigate this randomness and prevent the bias of arbitrarily selecting the test set through repeated random splits, we chose to split the cases based on the time of surgery into training and test sets in a 2:1 ratio. This method of splitting has been frequently employed in previous studies with similar cohort sizes to ensure reliability in the evaluation process [4,5]. Changes to the text: “In studies with relatively small populations, the performance of a model on a test set can significantly vary with each random split due to substantial changes in the distribution of the test set data [7]. To mitigate the bias associated with arbitrarily selecting the test set through repeated random splits, we chose to divide the cases based on their time range into training and test sets in a 2:1 ratio, a method frequently employed in prior studies with similar cohort sizes [8,9].” (Materials and methods: page 6, lines 106–111) Comment 4: " Despite the robust MRI-based model building utilizing the VAE, the multimodal model only improves the predictability slightly compared to clinical-only model." Author’s response: We appreciate your insightful comment. Indeed, the MRI-only model exhibited inferior performance compared to the clinical-only model in the training set (Figs 4A, B). Consequently, incorporating MRI features into the clinical variables slightly improved prediction performance relative to the clinical-only model (Fig 4E). Additionally, the grouped permutation feature importance analysis revealed that the top three variables contributing to the performance of the multimodal model were clinical features (Fig 4D). Although our findings suggest that imaging features have a lesser impact than clinical features on the performance of predictive models for glioblastoma patients, the integration of clinical parameters with MRI features consistently improved prediction accuracy. These results highlight the potential benefits of enhancing the clinical-based model with imaging features to improve prediction performance. Changes to the text: “Although MRI features had less impact on prediction performance than clinical features, we consistently observed an improvement in prediction performance when incorporating both clinical parameters and MRI features.” (Discussion: page 17, lines 339–342) Comment 5: " It is unclear how could the KPS prediction model contribute to the surgical planning, as the modeling requires pre- and post-operative MRIs." Author's response We are grateful for your invaluable comment. We apologize for any ambiguity in our initial discussion. We anticipate a scenario where AI-generated postoperative images, rather than actual images, are integrated into a KPS prediction model for surgical planning. Recent research has successfully developed surgical resectability prediction models and generative models that produce high-quality images of both pre- and postoperative MRIs [6–8]. Integrating AI-generated postoperative images into a KPS prediction model may clarify optimal resection strategies and significantly enhance surgical simulations, thereby improving patient outcomes. Changes to the text: “The prediction of prognosis using pre- and postoperative MRI may also contribute to surgical planning when combined with generative artificial intelligence (AI) that synthesizes postoperative images.” (Discussion: page 18, lines 375–377) “Furthermore, a recent study has demonstrated that generative AI models are capable of producing fine-quality images of pre- and postoperative brain tumors and normal parenchyma [32]. Integrating AI-generated postoperative images with a KPS prediction model might clarify optimal resection strategies and considerably enhance surgical simulations.” (Discussion: page 18, lines 380–383) References 1. Della Pepa GM, Caccavella VM, Menna G, Ius T, Auricchio AM, Chiesa S, et al. Machine learning–based prediction of 6-month postoperative Karnofsky performance status in patients with glioblastoma: capturing the real-life interaction of multiple clinical and oncologic factors. World Neurosurg. 2021;149: e866–e876. doi:10.1016/j.wneu.2021.01.082 2. Della Pepa GM, Caccavella VM, Menna G, Ius T, Auricchio AM, Sabatino G, et al. Machine learning-based prediction of early recurrence in glioblastoma patients: a glance towards precision medicine. Neurosurgery. 2021;89: 873–883. doi:10.1093/neuros/nyab320 3. Xu Y, Goodacre R. On splitting training and validation set: a comparative study of cross-validation, bootstrap and systematic sampling for estimating the generalization performance of supervised learning. J Anal Test. 2018;2: 249–262. doi:10.1007/s41664-018-0068-2 4. Park JE, Kim HS, Lee J, Cheong EN, Shin I, Ahn SS, et al. Deep-learned time-signal intensity pattern analysis using an autoencoder captures magnetic resonance perfusion heterogeneity for brain tumor differentiation. Sci Rep. 2020;10: 1–11. doi:10.1038/s41598-020-78485-x 5. Yun J, Yun S, Park JE, Cheong EN, Park SY, Kim N, et al. Deep learning of time-signal intensity curves from dynamic susceptibility contrast imaging enables tissue labeling and prediction of survival in glioblastoma. AJNR Am J Neuroradiol. 2023;44: 543–552. doi:10.3174/ajnr.A7853 6. Miao X, Chen H, Tang M, Huang D, Gao T, Chen Y. Post-operative MRI synthesis from pre-operative MRI and post-operative CT using conditional GAN for the assessment of degree of resection. Displays. 2024;83: 102742. doi:10.1016/j.displa.2024.102742 7. Marcus AP, Marcus HJ, Camp SJ, Nandi D, Kitchen N, Thorne L. Improved prediction of surgical resectability in patients with glioblastoma using an artificial neural network. Sci Rep. 2020;10: 1–9. doi:10.1038/s41598-020-62160-2 8. Mukherkjee D, Saha P, Kaplun D, Sinitca A, Sarkar R. Brain tumor image generation using an aggregation of GAN models with style transfer. Sci Rep. 2022;12: 1–17. doi:10.1038/s41598-022-12646-y ######################### Response to the Specific Journal Requirements Dear Editor, Thank you very much for the opportunity to revise and resubmit our manuscript. We sincerely appreciate the journal's support and efforts throughout the review process. We have addressed the specific requirements requested by the journal as follows: 1. We reviewed our manuscript against the PLOS ONE style templates and ensured full adherence to the formatting guidelines, including proper file naming conventions. 2. In compliance with PLOS ONE's code-sharing policy, we have made all author-generated code available via GitHub at the following URL: https://github.com/TomokiSasagasako/GBM_KPS_prediction.git. This link has been included in the manuscript to ensure accessibility and facilitate reproducibility. 3. A financial disclosure statement has been added to the manuscript, clarifying: "The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript." (page 20, lines 428–430) 4&5. Regarding the request for data publication, we consulted with our Research Ethics Committee. Unfortunately, due to the sensitive nature of the clinical data, which includes potentially identifiable information (such as the name of the hospital, diagnostic details, age, gender, and clinical outcomes), we are unable to provide unrestricted access. Sharing such information could lead to the identification of individual patients, and therefore it is not permissible under our ethical guidelines. This decision is consistent with the ethical guidance provided by the previously referenced article: http://www.bmj.com/content/340/bmj.c181.long. For documentation of data availability, we have updated our manuscript to include the following statement: " Data availability The code used to develop the model described herein is publicly available on GitHub: https://github.com/TomokiSasagasako/GBM_KPS_prediction.git The raw data in this study cannot be shared publicly as it includes identifiable information such as the name of the hospital (Kyoto University Hospital), diagnostic details, age, gender, and clinical outcomes. The raw data are available upon reasonable request from the corresponding author or Kyoto University Graduate School and Faculty of Medicine, Ethics Committee via email (ethcom@kuhp.kyoto-u.ac.jp) or telephone (+81-75-753-4680)." (page 20, lines 416–423) Thank you for your consideration. I look forward to your response. Sicerely, Yohei Mineharu Attachments Attachment Submitted filename: Response_to_Reviewers.docx https://doi.org/10.1371/journal.pone.0303002.r002
7 Oct 2024 Decision Letter - Kevin Camphausen, Editor Postoperative Karnofsky performance status prediction in patients with IDH wild-type glioblastoma: a multimodal approach integrating clinical and deep imaging features PONE-D-24-13237R1 Dear Dr. Mineharu, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice will be generated when your article is formally accepted. Please note, if your institution has a publishing partnership with PLOS and your article meets the relevant criteria, all or part of your publication costs will be covered. Please make sure your user information is up-to-date by logging into Editorial Manager at Editorial Manager® and clicking the ‘Update My Information' link at the top of the page. If you have any questions relating to publication charges, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Kevin Camphausen Academic Editor PLOS ONE Additional Editor Comments (optional): Thank you Reviewers' comments: https://doi.org/10.1371/journal.pone.0303002.r003
Formally Accepted
10 Oct 2024 Acceptance Letter - Kevin Camphausen, Editor PONE-D-24-13237R1 PLOS ONE Dear Dr. Mineharu, I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now being handed over to our production team. At this stage, our production department will prepare your paper for publication. This includes ensuring the following: * All references, tables, and figures are properly cited * All relevant supporting information is included in the manuscript submission, * There are no issues that prevent the paper from being properly typeset If revisions are needed, the production department will contact you directly to resolve them. If no revisions are needed, you will receive an email when the publication date has been set. At this time, we do not offer pre-publication proofs to authors during production of the accepted work. Please keep in mind that we are working through a large volume of accepted articles, so please give us a few weeks to review your paper and let you know the next and final steps. Lastly, if your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. If we can help with anything else, please email us at customercare@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Dr. Kevin Camphausen Academic Editor PLOS ONE https://doi.org/10.1371/journal.pone.0303002.r004

Open letter on the publication of peer review reports

PLOS recognizes the benefits of transparency in the peer review process. Therefore, we enable the publication of all of the content of peer review and author responses alongside final, published articles. Reviewers remain anonymous, unless they choose to reveal their names.

We encourage other journals to join us in this initiative. We hope that our action inspires the community, including researchers, research funders, and research institutions, to recognize the benefits of published peer review reports for all parts of the research system.

Learn more at ASAPbio .